Articles tagged “llm-judge”
3 articles
LLM judges grade your agents wrong. Here's how to fix them.
LLM judges are the only practical way to evaluate agents at production scale, but they have predictable biases that give you false confidence. Here's how to calibrate them so the scores you see reflect the quality you actually have.
Eval sampling for production agents: the 100/10/1 playbook
Running LLM-as-judge on every production conversation costs more than the agent itself. Here's the three-tier sampling system -- 100% lightweight heuristics, 10% LLM judge, 1% human review -- that keeps quality high without multiplying your eval bill.
Online vs. Offline Evals: Close the Production Gap
89% of teams have observability but only 37% run online evals. Here's why that gap is where production failures hide, and how to close it with a practical online eval pipeline.
Learn Agentic AI
Weekly. Patterns for shipping agents that work. MCP, scorecards, regression tests, prompts, model comparisons.
