Your Eval Lied: The Architecture of a Real LLM Evaluation Program
The new model swept the golden set, the regression suite, and the LLM as judge evals. Then CSAT dropped. The eval did not lie. The eval measured exactly what it was built to measure. That was the problem.
Jul 9, 202615 min read4
