How to Test Agents That Call the Right Tool and Still Get It Wrong

A customer asks your agent to refund a duplicate charge. The agent does everything right. It recognizes the intent, calls processRefund with the correct order ID, and passes valid arguments. In the logs, it's a textbook tool call.

The API returns a 402. The refund didn't go through, because the order was already partially refunded and the payment processor rejected it. Your agent reads that result and tells the customer: "All set, your refund is on its way. You'll see it in three to five business days."

Nothing was on its way. The agent called the right tool, got a clear error back, and reported success anyway. The customer hangs up happy, checks their statement in a week, and calls back furious.

If you only measured whether the agent selected the correct tool, this conversation passes. That's the blind spot in how most teams test agents, and it's where a surprising share of production incidents actually live. The tool call was fine. What happened next wasn't.

Why Does Tool-Use Accuracy Miss Half the Failures?

Tool-use accuracy measures whether the agent picks the right tool and passes valid arguments. It stops at the moment of the call. The Berkeley Function Calling Leaderboard, the reference point most teams quote, grades exactly that: did the model produce the right call. But reliability depends just as much on what the agent does with the result that comes back, and selection-only metrics never look there. An agent can score a perfect tool-selection rate while routinely mishandling errors, partial data, and stale results.

Research is catching up to this. ToolFailBench, presented at ICML 2026 workshops, frames the problem plainly: reliability depends not only on whether a model calls a tool, but on whether it uses the tool result correctly. Its taxonomy makes the second half a first-class citizen, scoring Result-Ignore and Output-Fabrication as distinct failure modes alongside selection mistakes, and across the 19 models it tested, the best managed a clean tool-use rate of only about 86 percent. That second half is where a lot of real failure hides, and it's invisible to benchmarks that grade the call and walk away.

The reason it hides is that the transcript looks normal. A mishandled result doesn't throw an exception. The agent produces a fluent, confident sentence that happens to be wrong. You can't spot it by scanning for errors, because from the system's point of view nothing errored. The tool ran. The agent replied. Everything is green. Green is the most dangerous color on an agent dashboard.

Here's the taxonomy of what actually goes wrong after the call.

The article on the QA layer CX teams skip covers why tool-call testing gets overlooked in the first place. This article is about the specific failures that survive even when teams do test the call itself.

What Are the Failure Modes After the Call?

All five patterns in that table share one signature: the agent sounds certain, the transcript looks clean, and nothing in your logs flags a problem. The most common by far is hallucinated success, where the agent narrates an outcome the tool never confirmed.

Hallucinated success is the refund scenario from the top of this piece. The tool returned a failure, and the agent's language model, trained to be helpful and fluent, produced the sentence a helpful agent would say if it had succeeded. The model isn't lying on purpose. It's pattern-matching to "customer requested refund, tool was called, respond positively," and the negative result never fully registered.

Ignored errors are the broader category. A tool returns { status: "error", reason: "account_locked" } and the agent proceeds as if the account were fine. Stale-result reuse shows up in long conversations: the agent looked up a balance at turn five, the customer made a payment at turn twelve, and at turn twenty the agent quotes the turn-five balance because that result is still in its context. Partial-result overreach and silent truncation both come from treating incomplete data as complete, which is especially dangerous with search and database tools that cap their result size.

What ties them together is that none of them look like bugs. The self-healing agent pattern article is about recovering from failures the agent can detect. These are the failures it doesn't detect, which is why you have to force them in testing rather than wait to observe them.

How Do You Test for Tool-Result Misuse?

You test for it by injecting specific tool results, including errors and edge cases, and asserting on what the agent says next. Don't just verify the agent called processRefund. Return a 402 from that call and assert the agent tells the customer the refund failed and offers a next step. The result you inject is the input to the test; the agent's response is what you grade.

This flips the usual tool test around. A selection test controls the customer's message and checks the tool call. A result test controls the tool's return value and checks the agent's reply. You need both, but the second one is where you catch the refund incident before it ships.

import { describe, it, expect } from '@jest/globals';
import { runAgent } from './harness';
 
describe('tool-result handling', () => {
  it('surfaces a refund failure instead of faking success', async () => {
    const result = await runAgent({
      message: 'Please refund my duplicate charge on order 8842.',
      toolResponses: {
        processRefund: {
          status: 'error',
          code: 402,
          reason: 'already_partially_refunded',
        },
      },
    });
 
    // The agent must NOT claim success.
    expect(result.reply).not.toMatch(/on its way|processed|all set/i);
    // The agent must acknowledge the failure and offer a path forward.
    expect(result.reply).toMatch(/couldn't|unable|already refunded|escalate/i);
  });
 
  it('does not reuse a stale balance after a payment', async () => {
    const result = await runAgent({
      transcript: [
        { role: 'tool', tool: 'getBalance', result: { balance: 240 } },
        { role: 'user', content: 'I just paid $240.' },
        { role: 'tool', tool: 'applyPayment', result: { status: 'ok' } },
        { role: 'user', content: 'What do I owe now?' },
      ],
      toolResponses: {
        getBalance: { balance: 0 },
      },
    });
 
    expect(result.reply).toMatch(/\$0|nothing|paid in full/i);
    expect(result.reply).not.toMatch(/\$240/);
  });
 
  it('flags a truncated search result instead of concluding', async () => {
    const result = await runAgent({
      message: 'Have I ever been charged a late fee?',
      toolResponses: {
        searchTransactions: {
          items: [],
          truncated: true,
          reason: 'result_limit_reached',
        },
      },
    });
 
    expect(result.reply).not.toMatch(/no.*late fee|never|no records/i);
    expect(result.reply).toMatch(/check|look further|narrow/i);
  });
});

The pattern in every case is the same. Control the tool's return value, then assert on language. The negative assertions matter as much as the positive ones: you're checking the agent doesn't say "processed" as hard as you're checking it says "couldn't." A prompt change that reintroduces hallucinated success will flip that negative assertion, and you'll catch it in CI instead of in a customer's bank statement.

Scenario testing is where this scales beyond a handful of hand-written cases. You define personas that walk the agent into each failure shape, the ineligible refund, the already-settled balance, and run the agent against many variations of the same trap without scripting each one by hand.

Do Retries Make Tool-Result Failures Better or Worse?

Retries help only when the failure is transient, and they make things worse when it isn't. A bounded retry is right for a network timeout. It's wrong for a business error like an ineligible refund, where retrying just burns tokens and delays the honest answer. The fix is to classify the result before deciding, not to retry reflexively.

Naive retry logic treats every non-success the same way: try again. But account_locked and connection_reset are nothing alike. One is a permanent state the customer needs to hear about; the other is a blip that a second attempt will clear. Retrying the first wastes time and money and still ends in the same error. Not retrying the second turns a recoverable hiccup into a failed interaction.

type ResultClass = 'success' | 'transient' | 'permanent';
 
function classifyResult(result: {
  status: string;
  code?: number;
  reason?: string;
}): ResultClass {
  if (result.status === 'ok') return 'success';
 
  const transientCodes = [408, 429, 500, 502, 503, 504];
  if (result.code && transientCodes.includes(result.code)) {
    return 'transient';
  }
 
  // Business errors are permanent for this request. Retrying won't help.
  return 'permanent';
}
 
async function callWithPolicy(
  fn: () => Promise<{ status: string; code?: number; reason?: string }>,
  maxRetries = 2
): Promise<{ outcome: string; detail: string }> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const result = await fn();
    const cls = classifyResult(result);
 
    if (cls === 'success') {
      return { outcome: 'success', detail: 'completed' };
    }
    if (cls === 'permanent') {
      // Surface it. Do not retry. Do not paper over it.
      return { outcome: 'failed', detail: result.reason ?? 'unknown' };
    }
    // transient: brief backoff, then retry.
    await new Promise((r) => setTimeout(r, 200 * 2 ** attempt));
  }
 
  return { outcome: 'failed', detail: 'exhausted retries' };
}

The classification is the point, not the retry loop. Once you can tell a transient fault from a permanent one, the agent's behavior falls out cleanly: retry transient faults a bounded number of times, surface permanent ones to the customer, and never let either become a silent success. The article on idempotent tool calls covers the safety side of retrying, which matters here because a retried refund that actually went through the first time is its own kind of incident.

How Do You Catch Tool-Result Misuse in Production?

Testing catches the failures you thought to write. Production catches the ones you didn't. The net is a scorecard that reads the tool-result log and compares it against the agent's stated outcome, flagging any session where the agent said "refunded" but the tool returned an error. Sample continuously, alert on the rate, and you find these before customers do.

Real conversations produce tool results you never anticipated, and the only way to know your agent handled them correctly is to check its words against ground truth from the tool logs, at scale, on live traffic.

Score

Good

0/100

Tone & Empathy

94%

Resolution

88%

Response Time

72%

Compliance

85%

Scorecards automate that comparison. You define the check once, "does the agent's stated outcome match the tool result," and it runs against a sample of every day's conversations. Monitoring turns the aggregate into an alert, so a spike in mishandled error results pages you instead of accumulating quietly in a support queue.

import { Chanl } from '@chanl/sdk';
 
const chanl = new Chanl({ apiKey: process.env.CHANL_API_KEY });
 
// Score a sampled live session against your tool-result fidelity
// scorecard. The scorecard's criteria hold the actual checks, like
// "did the agent claim an outcome its tool results did not confirm?"
async function scoreResultFidelity(interactionId: string, scorecardId: string) {
  return chanl.scorecard.evaluate(interactionId, {
    scorecardId,
    triggeredBy: 'manual',
  });
}
 
// Watch the aggregate: completed evaluations for that scorecard
// are the rate you alert on.
async function fidelityResults(scorecardId: string) {
  return chanl.scorecard.listResults({ scorecardId, status: 'completed' });
}
 
// Before shipping a prompt change, replay the failure scenario
// against every persona attached to it, then check the pass rate.
async function guardrail(scenarioId: string) {
  await chanl.scenarios.runCrossProduct(scenarioId, { mode: 'text' });
  const { data } = await chanl.scenarios.getExecutionStats();
  return data?.successRate;
}

The evaluate call is the production net: it grades real conversations on whether the agent's words matched reality, and listResults gives you the failure rate to alert on. The runCrossProduct call is the pre-ship gate: it replays every failure persona attached to your scenario against a prompt change before it reaches customers. One watches what's live; the other stops the refund incident from going live in the first place. This is the Monitor loop that makes an agent trustworthy over time, not just correct on the day it shipped.

The Refund That Failed Honestly

Run the opening scenario again with result testing in place. The customer asks for a refund on the duplicate charge. The agent calls processRefund with the right order ID. The API returns the same 402, the same already_partially_refunded.

This time the agent classifies the result as a permanent business error, doesn't retry it, and doesn't reach for the fluent success sentence, because your test suite would have caught that and your production scorecard is watching for it. It tells the customer: "I couldn't process that refund automatically because part of this order was already refunded. Let me get a specialist to sort out the remaining amount." The customer gets the truth and a path forward instead of a comfortable lie and a furious callback next week.

The tool call was identical in both versions. It was always identical. The difference was never in whether the agent picked the right tool. It was in what the agent did with the answer, and that's the half of reliability you only get by testing the result, not just the call.

El briefing de Signal

Un email por semana. Cómo los equipos líderes de CS, ingresos e IA están convirtiendo conversaciones en decisiones. Benchmarks, playbooks y lo que funciona en producción.

Subscribe

500+ líderes de CS e ingresos suscritos