Prompt and Model Versioning
Or: The day the diff was empty and the behaviour wasn't
Jul 14, 202615 min read
Command Palette
Search for a command to run...
#system-design
Articles tagged with #system-design
Your Eval Lied: The Architecture of a Real LLM Evaluation Program
The new model swept the golden set, the regression suite, and the LLM as judge evals. Then CSAT dropped. The eval did not lie. The eval measured exactly what it was built to measure. That was the problem.
Jul 9, 202615 min read4
The Agent Reasons, the Engine Remembers: Designing an Agentic Workflow Engine
Every "rogue agent" story is really an engine story in disguise: an agent that exceeded its budget operated in a system without one, and an agent that double executed a tool did so because the engine let it.
Jul 7, 202621 min read25
Nothing Scales the Way You Think: Designing an LLM Inference Platform
When the interviewer says "design an ML serving platform at scale," your infra instincts will betray you on every dimension: the bottleneck is GPU memory, output length determines latency, and the batching strategy you learned for image classifiers is worse than useless here.
Jul 2, 202619 min read66
The Intern with a Refund Button: Designing a Customer Support Agent at Scale
A support agent is the only employee you will ever deploy that can issue ten thousand refunds a second, has read every page of your help center, and will cheerfully invent a return policy if the retrieved documents come up short. Scale that carefully.
Jun 30, 202614 min read12
The Retriever Is Your Security Perimeter: Designing a Multi Tenant RAG Knowledge Base
The moment an LLM starts narrating retrieval results, your retriever becomes your access control layer. Filter permissions anywhere above that and you have built a very sophisticated data leak with excellent citations.
Jun 25, 202618 min read18