Search for a command to run...
Articles tagged with #mlops
A recursive agent spent the weekend asking itself the same question, failing the same check, and billing you for both. No error fired, because no metric you watched had the word stop in it. This article is about putting it there.
The dashboards are healthy, the error rate is zero, nothing shipped this week, and the model just told a customer something expensive and wrong. There is no commit to revert. Welcome to the incident that classical on call was never designed for.
It cleared every eval, aced the canary, and shipped on a Wednesday. By Friday it was mailing blank invoices to customers. The model did not regress. It just quietly stopped honouring a contract nobody had bothered to write down.
Nothing shipped. The git log is clean. The model is behaving differently anyway. In an LLM system, that is not a mystery. It is a versioning failure with a known architecture, and this article is that architecture.
The new model swept the golden set, the regression suite, and the LLM as judge evals. Then CSAT dropped. The eval did not lie. The eval measured exactly what it was built to measure. That was the problem.
When the interviewer says "design an ML serving platform at scale," your infra instincts will betray you on every dimension: the bottleneck is GPU memory, output length determines latency, and the batching strategy you learned for image classifiers is worse than useless here.
