Articles tagged “reliability”
13 articles
How to Test Agents That Call the Right Tool and Still Get It Wrong
Tool-use benchmarks check whether an agent picks the right tool. They miss agents that call correctly, then mishandle the result. Here's how to test for it.
Why 40% of multi-agent systems fail in production
Nearly half of multi-agent CX pilots fail within six months of production — not because the AI is wrong, but because teams don't understand how multi-agent systems break. Here are the five failure modes and how to diagnose each.
When your LLM provider goes down, your agent shouldn't
LLM providers go down. Anthropic's 90-day uptime is 98.95% -- that's 44 hours of potential outage per year. Here's how to build provider failover into production AI agents so a cloud incident doesn't become a customer experience incident.
Fault Injection Is the Missing Layer in Agent Testing
ReliabilityBench found rate limiting is the most damaging fault for AI agents in production -- more damaging than wrong answers. Here's how to inject chaos into your test suite before your users do.
Design agent tools that won't double-charge on retry
Agents retry when tools fail. Without idempotency, retries create duplicate records, double charges, and corrupted state. Here's the engineering pattern that makes every mutating tool safe to retry.
SRE for AI Agents: SLOs, Error Budgets, and Reliability
Traditional SRE doesn't catch AI agent failures. Here's a practical SRE playbook for agents: the five SLIs that matter, how to set SLOs that are actually useful, and how error budgets control agent autonomy before problems escalate.
How to Build Idempotent Tool Calls for AI Agents
Naive retry logic charges customers twice, sends duplicate emails, and fires double webhooks. Here's how to build idempotent tool calls for AI agents with idempotency keys, deduplication, and safe retries.
Structured Outputs: Make Your AI Agent Stop Guessing
JSON mode isn't enough. Learn how constrained decoding, Zod schema validation, and validator-retry patterns cut agent parsing failures in production.
Your Agent Is Already a State Machine. Make It Explicit.
Every production AI agent is secretly a state machine. Making it explicit gives you checkpointing, testable paths, and observable state transitions -- without rewriting your agent logic.
Circuit Breakers for AI Agents: Stop the 3 AM Meltdown
One retry loop at 11 PM becomes $437 by 7 AM. Here's how to implement circuit breakers for AI agent tool calls, LLM calls, and external APIs, with TypeScript patterns that stop cascading failures before they start.
Agent Drift: Why Your AI Gets Worse the Longer It Runs
AI agents silently degrade over long conversations. Research quantifies three types of drift and shows why point-in-time evals miss them entirely.
Your Agent Is Getting Smarter. It's Not Getting More Reliable.
Reliability improves at half the rate of accuracy. Three 85%+ tools combine to just 74%. Here's the math, the research, and the testing protocols that close the gap.
IA Agentica en Produccion: De Prototipo a Servicio Confiable
Lleva IA agentica a produccion sin que se rompa a las 2 AM. Cubre patrones de orquestacion (ReAct, bucles de planificacion), manejo de errores, circuit breakers, degradacion elegante, observabilidad y escalamiento, con implementaciones en TypeScript que puedes reutilizar.
El briefing de Signal
Un email por semana. Cómo los equipos líderes de CS, ingresos e IA están convirtiendo conversaciones en decisiones. Benchmarks, playbooks y lo que funciona en producción.
