Articles tagged “production-evals”
2 articles
Testing & Evaluation·14 min read
LLM judges grade your agents wrong. Here's how to fix them.
LLM judges are the only practical way to evaluate agents at production scale, but they have predictable biases that give you false confidence. Here's how to calibrate them so the scores you see reflect the quality you actually have.
Read More
Testing & Evaluation·13 min read
Trajectory Eval: Catch Agent Bugs Output Scoring Misses
Final-output scoring misses 20-40% of agent regressions. Trajectory evaluation scores every step an agent takes -- tool calls, reasoning decisions, order of operations -- and catches the bugs that output-only evals can't see.
Read More
El briefing de Signal
Un email por semana. Cómo los equipos líderes de CS, ingresos e IA están convirtiendo conversaciones en decisiones. Benchmarks, playbooks y lo que funciona en producción.
