AI - Episode 11 - From Demo to Production: Oh Shit, People Are Actually Using It

Real users have arrived, and so have latency, rate limits, privacy questions, and bills. We explore model choice, caching, fallbacks, logging, and operational ownership before comparing hosted APIs with local and self-hosted models. Sabrina brings the open-weight question; GPUs and quantization get plain-language explanations. The model turns out to be one dependency inside a much larger software system.