Search for a command to run...
Series
Classic system-design problems, redrawn for the probabilistic era. Search, coding assistants, RAG, support agents, inference platforms, workflows
An LLM synthesis layer did not replace the search iceberg. It made every layer below it works harder: fast enough to feed the model, fresh enough to stay true, and structured enough to cite.
*When the model sits in the loop of a typing user, the job stops being "make the model smarter" and becomes "make a model that is slower than the user feels faster than the user." Good luck.*
The moment an LLM starts narrating retrieval results, your retriever becomes your access control layer. Filter permissions anywhere above that and you have built a very sophisticated data leak with excellent citations.
A support agent is the only employee you will ever deploy that can issue ten thousand refunds a second, has read every page of your help center, and will cheerfully invent a return policy if the retrieved documents come up short. Scale that carefully.
When the interviewer says "design an ML serving platform at scale," your infra instincts will betray you on every dimension: the bottleneck is GPU memory, output length determines latency, and the batching strategy you learned for image classifiers is worse than useless here.
Every "rogue agent" story is really an engine story in disguise: an agent that exceeded its budget operated in a system without one, and an agent that double executed a tool did so because the engine let it.
