Articles tagged “production”
41 articles
Agentjacking: how attackers hijack production AI agents
Agentjacking is a new attack class that targets the data your agent trusts, not the model itself. Learn how attackers exploit CRM notes, tool responses, and knowledge bases to hijack production CX agents, and how to stop them.
Reasoning models for CX agents: when to think slow, when to move fast
Extended thinking models improve decision quality for complex CX scenarios, but add 5-30 seconds of latency and cost 3-5x more. Here's the planning-execution architecture that gets you the benefit without the tradeoff.
How to debug the agent conversation you can't reproduce
AI agents fail in ways you can't reproduce by resending the same message. Deterministic replay captures what the agent saw and did so you can step through the failure exactly as it happened.
What the July 28 MCP spec means for your stateful agents
The July 28 MCP spec is the largest protocol revision since launch. Sessions are removed. Stateless request handling replaces the initialization handshake. Here's what breaks and how to fix it before the deadline.
Fault Injection Is the Missing Layer in Agent Testing
ReliabilityBench found rate limiting is the most damaging fault for AI agents in production -- more damaging than wrong answers. Here's how to inject chaos into your test suite before your users do.
Your CX agent needs an autonomy level before it goes live
Not all CX agents should be equally autonomous. Here's a practical L0-to-L4 framework for deciding how much independence to give your agent for each action type, and what eval thresholds to hit before promoting.
Tool receipts: verifiable proof of what your agent actually did
When your AI agent claims it called a tool and got a result, can you verify that? Tool receipts are structured execution proofs that catch fabricated tool outputs before they damage customer trust.
Eval sampling for production agents: the 100/10/1 playbook
Running LLM-as-judge on every production conversation costs more than the agent itself. Here's the three-tier sampling system -- 100% lightweight heuristics, 10% LLM judge, 1% human review -- that keeps quality high without multiplying your eval bill.
Agents that learn while they sleep: async memory consolidation
Post-session async consolidation is the missing layer in most agent memory stacks. Here's how Anthropic's Dreaming primitive works, why Harvey saw a 6x jump, and how to implement the same pattern without Anthropic's API.
Prompt caching: the cost cut most agent teams skip
Prompt caching cuts API costs 41-80% and TTFT by up to 31%. Learn where to put the cache breakpoint, when it backfires, and how to monitor hit rates in production.
Build agents that fix themselves when they fail
Most agents fail silently in production. Self-healing agents detect their own failures, classify what went wrong, and take a different path rather than retrying blindly.
Managed Agents in 2026: Three Runtimes, Three Trade-Offs
Google, Anthropic, and OpenAI all shipped 'managed agents' in May 2026, and they mean completely different things. Here's what each runtime trades away for CX teams.
How to Migrate Your MCP Server to Stateless Mode
The MCP 2026 release candidate makes stateless the recommended default. Your MCP server can now scale behind any load balancer without sticky routing. Here's how to migrate and use the new Tasks extension for async CX work.
How to Build a Regression Test Suite for AI Agents
Your CI/CD pipeline catches code regressions. But who catches it when a prompt change breaks your agent's compliance behavior? Here's how to build behavioral regression testing for non-deterministic AI agents.
Your Agent Is Already a State Machine. Make It Explicit.
Every production AI agent is secretly a state machine. Making it explicit gives you checkpointing, testable paths, and observable state transitions -- without rewriting your agent logic.
AI Agent KPIs: What to Measure Before You Ship
Only 31% of teams have a measurement framework for their AI agents. Here's how to define task completion rate, escalation rate, cost per outcome, and CSAT delta before your first production interaction.
MCP Auth in Production: Scopes, Tokens, and Tenant Isolation
Most MCP servers ship with no auth. Here's how to add OAuth 2.0 scopes, per-tenant tool sets, and client isolation before your MCP server becomes load-bearing production infrastructure.
Circuit Breakers for AI Agents: Stop the 3 AM Meltdown
One retry loop at 11 PM becomes $437 by 7 AM. Here's how to implement circuit breakers for AI agent tool calls, LLM calls, and external APIs, with TypeScript patterns that stop cascading failures before they start.
How Much Testing Is Enough for Your AI Agent?
Code coverage doesn't apply to AI agents. Here's a framework for thinking about evaluation coverage: how many scenarios you need, what distribution to target, and how to know when you've tested enough.
MCP SSE Is Deprecated. Here's How to Migrate
SSE transport is being deprecated across major MCP platforms in 2026. Here's a practical migration guide from HTTP+SSE to Streamable HTTP, with TypeScript examples and a phased rollout strategy.
7 FastMCP mistakes that break your agent in production
FastMCP servers that work locally often fail at scale. Seven common mistakes, from missing annotations to monolithic tool sets, and how to fix each one.
MCP Is Now Open Infrastructure: Build for What's Next
MCP was donated to the Linux Foundation and the AAIF just held its first summit. What does the protocol becoming open infrastructure mean for what you build on top of it?
The 17x error trap in multi-agent systems
Multi-agent systems amplify errors 17x, not reduce them. We compare CrewAI, LangGraph, and Autogen failure modes with concrete fixes and a decision tree.
The no-code ceiling: when agent builders hit production
Visual agent builders get you to 80% fast. The last 20%, telephony, monitoring, testing, and memory, requires infrastructure they never intended to provide.
Online vs. Offline Evals: Close the Production Gap
89% of teams have observability but only 37% run online evals. Here's why that gap is where production failures hide, and how to close it with a practical online eval pipeline.
LLM-as-a-Judge: Build a Production Eval Pipeline
Build a production LLM-as-a-judge eval pipeline step by step. Covers judge selection, rubric design, CI integration, and sampling strategies that scale.
MCP Servers in Production: Observability from Day One
Instrument your MCP servers with OpenTelemetry for production-grade observability. Covers tracing tool calls, detecting loops, cost attribution, and alerting.
Build an AI Agent Observability Pipeline from Scratch
Build a production observability pipeline for AI agents using TypeScript and the Chanl SDK. Covers metrics, traces, quality scoring, drift detection, and alerting.
Production Agent Evals: Catch Score Drift, Ship Confidently
Your evals pass in staging but miss production failures. Build three eval pipelines with the Chanl SDK: automated scorecards, scenario regression, and drift detection that catches quality degradation before customers do.
74% of Production Agents Still Rely on Human Evaluation
A survey of 306 practitioners reveals most production agents are far simpler than expected. The eval gap isn't a tooling problem. It's a trust problem.
Your Agent Is Getting Smarter. It's Not Getting More Reliable.
Reliability improves at half the rate of accuracy. Three 85%+ tools combine to just 74%. Here's the math, the research, and the testing protocols that close the gap.
Your Agent Aced the Benchmark. Production Disagreed.
We scored 92% on GAIA. Production CSAT: 64%. Here's which AI agent benchmarks actually predict deployed performance, why most don't, and what to measure instead.
What to Trace When Your AI Agent Hits Production
OpenTelemetry GenAI conventions are the production standard for agent tracing. What to instrument, what to skip, and what breaks — from a 2 AM debugging war story.
A 7B Domain Model Beat Everything We Tried
Domain-specific language models are beating trillion-parameter generalists on vertical tasks. Here's when a 7B model is the right call, how the training pipeline works, and what production teams are shipping today.
The Multi-Agent Pattern That Actually Works in Production
Gartner reports a 1,445% surge in multi-agent system inquiries. Here are the orchestration patterns that actually work when real customers call -- and why most teams pick the wrong one.
Your AI Agent Has No Guardrails
Air Canada honored a refund its chatbot hallucinated. DPD's bot cursed at customers on camera. One e-commerce agent approved $2.3M in unauthorized refunds at 2:47 AM. Here is the five-layer guardrail architecture that prevents all three.
Your AI Agent Costs $13K/Month. Here's the Fix.
A production customer-service agent burned $13,247 in one month. Prompt caching, model routing, batch processing, and plan-and-execute architecture cut it to $1,100. Real pricing math for every technique.
Part 4: All 7 Extension Points in One Production Codebase
50+ skills, multiple MCP servers, scoped rules, safety hooks — here's how all 7 Claude extension points compose in a real NestJS monorepo with 17 projects. What works, what fights, and what we'd do differently.
Your Agent Passed Every Dev Test. Here's Why It'll Fail in Production
A 4-layer testing framework for AI agents (unit, integration, performance, and chaos testing) so your agent survives real customers, not just controlled demos.
Agentic AI in Production: From Prototype to Reliable Service
Ship agentic AI that doesn't break at 2 AM. Covers orchestration patterns (ReAct, planning loops), error handling, circuit breakers, graceful degradation, observability, and scaling — with TypeScript implementations you can steal.
AI Agent Observability: What to Monitor When Your Agent Goes Live
Build a production observability pipeline for AI agents. Covers latency, token usage, tool success rates, conversation quality, drift detection, structured logging, alerting strategies, and the critical difference between LLM and agent observability.
Learn Agentic AI
Weekly. Patterns for shipping agents that work. MCP, scorecards, regression tests, prompts, model comparisons.
