Reasoning models for CX agents: when to think slow, when to move fast

Your agent has to decide whether to issue a refund, escalate to a human, or apply a retention offer. The customer is a six-year premium member, upset about a billing error, already escalated once this quarter, and has mentioned they're considering canceling.

A standard model reads the situation and produces the most probable response. A reasoning model works through it first: what's the retention guideline for customers with this profile? What does the billing error policy say about escalation precedence? If I offer the retention discount first, does that affect the refund eligibility? What's the sequence that maximizes the chance of resolving this without losing the customer?

The reasoning phase is private. The customer sees only the final response. But the response that comes after deliberate planning is meaningfully better than the response that comes from predicting the next most probable token.

Understanding when that quality difference justifies the cost and latency is the engineering decision that most teams are making ad hoc in 2026. Here's a framework for making it deliberately.

What reasoning models actually do

Reasoning models (extended thinking in Claude, OpenAI's o-series) generate private chain-of-thought tokens before producing any output. The model thinks step by step through a problem, catches contradictions, works out multi-step plans, and then produces a response that reflects that prior reasoning.

The key word is private. The thinking phase is not part of the output you send to the customer. Depending on your provider and configuration, you may be able to read the thinking tokens for debugging and evaluation, but they don't appear in the response. The customer receives only the conclusion.

This matters for three reasons. First, the model can consider multiple options explicitly and pick among them, rather than going with whatever prediction had the highest probability. Second, it can work through conditional logic step by step: "if X is true, then Y applies; if Y applies and Z is the case, then the right action is W." Third, it can catch early mistakes: "wait, I was about to recommend the retention offer, but this customer already received one this quarter and is therefore ineligible." Standard models catch that kind of contradiction sometimes; reasoning models catch it more reliably because they've explicitly enumerated the conditions.

The latency reality

Extended thinking adds time before the first token of output. The thinking phase generates 1,000-5,000 tokens of private reasoning before producing the visible response. At standard generation speeds, this takes 3-20 seconds.

Here's the latency profile for three common CX scenarios:

For most CX channels, these numbers have different tolerances:

• Chat/messaging: 12 seconds is borderline but acceptable for a complex question. Users in chat tolerate more latency than voice.
• Email/async: 12 seconds is irrelevant. Processing a support ticket in 12 seconds vs. 1 second is the same experience.
• Voice, real-time: 3.5 seconds is unacceptable. A 3.5-second pause mid-conversation feels like the line dropped.

The conclusion most teams arrive at: reasoning models for CX work in chat and async channels without modification. For voice, they require architectural changes to keep them off the critical path.

The planning-execution architecture

The planning-execution pattern decouples the reasoning step from the execution steps, using different models for each.

A reasoning model runs once to generate a structured plan: which tools to call, in what order, with what arguments, and what to do if each step fails or returns unexpected results. A fast, cheap execution model then follows the plan step by step without re-reasoning at each step.

The reasoning model runs once and pays its latency cost once. The execution model handles each tool call turn, which is fast because it's following explicit instructions rather than reasoning from scratch.

Here's what the plan structure looks like:

import Anthropic from "@anthropic-ai/sdk";
 
interface ActionStep {
  tool: string;
  args: Record<string, unknown>;
  intent: string;
  onFailure: "retry" | "escalate" | "skip" | "abort";
}
 
interface AgentPlan {
  summary: string;
  steps: ActionStep[];
  primaryGoal: string;
  constraints: string[];
}
 
const client = new Anthropic();
 
async function generatePlan(
  customerMessage: string,
  customerContext: string,
  availableTools: string[]
): Promise<AgentPlan> {
  const response = await client.messages.create({
    model: "claude-opus-4-8",
    max_tokens: 8000,
    thinking: {
      type: "enabled",
      budget_tokens: 5000,    // thinking budget before output begins
    },
    messages: [
      {
        role: "user",
        content: `Plan the correct action sequence to resolve this customer request.
 
Customer message: ${customerMessage}
 
Customer context: ${customerContext}
 
Available tools: ${availableTools.join(", ")}
 
Generate a JSON plan with this structure:
{
  "summary": "what this plan accomplishes in one sentence",
  "primaryGoal": "resolved | escalated | information_provided | retained",
  "constraints": ["list any constraints or eligibility limits that affect this plan"],
  "steps": [
    {
      "tool": "tool_name",
      "args": {"key": "value"},
      "intent": "why this step",
      "onFailure": "retry | escalate | skip | abort"
    }
  ]
}`,
      },
    ],
  });
 
  const planText = response.content.find((b) => b.type === "text")?.text;
  if (!planText) throw new Error("No plan generated");
 
  return JSON.parse(planText);
}

The thinking: { budget_tokens: 5000 } parameter tells Claude to use up to 5,000 private thinking tokens before producing output. For simpler scenarios you can reduce this; for complex multi-condition cases you may need more.

Executing the plan with a fast model

Once you have the plan, execute it with a fast model that follows instructions:

async function executePlan(
  plan: AgentPlan,
  conversationHistory: Message[]
): Promise<string> {
  const results: Array<{ step: ActionStep; result: string }> = [];
 
  for (const step of plan.steps) {
    // Call the tool directly rather than asking the model to select it
    let toolResult: string;
    try {
      toolResult = await callTool(step.tool, step.args);
    } catch (error) {
      if (step.onFailure === "escalate") {
        return await escalateToHuman(plan, results);
      }
      if (step.onFailure === "skip") {
        continue;
      }
      throw error;
    }
 
    results.push({ step, result: toolResult });
  }
 
  // Use a fast model for final synthesis
  const response = await client.messages.create({
    model: "claude-haiku-4-5-20251001",   // fast, no thinking needed for synthesis
    max_tokens: 1024,
    messages: [
      ...conversationHistory,
      {
        role: "user",
        content: `Plan goal: ${plan.primaryGoal}
Constraints: ${plan.constraints.join(", ")}
 
Tool results:
${results.map(r => `${r.step.intent}: ${r.result}`).join("\n\n")}
 
Write the customer response that communicates the outcome of these actions naturally.`,
      },
    ],
  });
 
  return (response.content[0] as { text: string }).text;
}

The execution model doesn't reason about which steps to take. It follows the plan and synthesizes the results into a natural response. This is fast because synthesis is simpler than planning.

The combined latency for this pattern: 5-15 seconds for the plan (reasoning model, runs once), plus 0.5-1 second per tool call (up to the plan length), plus 1-2 seconds for final synthesis. For a 4-step plan, that's 8-20 seconds total. Slower than a single standard model call, but much faster than running a reasoning model at every step.

Where reasoning models improve outcomes in CX

The quality benefit shows up most in three scenario types: complex eligibility decisions with multiple concurrent conditions, multi-step action sequencing where order matters, and high-stakes retention decisions that require comparing options against each other. These are the cases where predicting the next token produces worse decisions than explicitly reasoning through the alternatives first.

Complex eligibility decisions. Your CX agent needs to determine whether a customer qualifies for a specific offer given five concurrent policy rules. Standard models often get this wrong when multiple conditions overlap or when one condition has an exception. Reasoning models work through each condition explicitly and catch the interaction effects.

A real example pattern: "customer is premium AND has been with us 3+ years AND signed up before the price change AND is in a state with a regional promotion AND their last order was in the promotion period." A standard model might apply the first three conditions and miss the last two. A reasoning model enumerates all five before deciding.

Multi-step action sequencing. When resolving an issue requires a specific sequence (check inventory before promising availability, verify refund eligibility before issuing it, confirm account status before applying a discount), getting the order wrong causes downstream failures. Reasoning models are better at planning sequences with correct dependencies.

Retention decisions. Retention scenarios are high-stakes, multi-factor decisions. The reasoning model considers the customer's history, the available offers, the eligibility constraints, and the probability of retention under each option. Standard models produce a response that sounds reasonable; reasoning models are more likely to pick the response that's actually optimal given all the factors.

Where reasoning models actively hurt CX

Reasoning models actively hurt performance in four scenarios: real-time voice (3-30 second latency breaks natural conversation), high-volume routing decisions (cost increases with no quality gain), simple information retrieval, and streaming interfaces where users see a long silence before the first token. In these cases, the latency and cost of the thinking phase outweigh any quality improvement.

Real-time voice. Reiterate because it matters: don't use reasoning models in the synchronous voice call loop. The latency is incompatible with natural conversation. Instead, use reasoning models for voice agent configuration (planning the decision tree the voice agent follows) and offline analysis (reviewing completed calls and generating improvement recommendations). The voice agent itself stays on fast models.

High-volume routing decisions. Classifying 10,000 customer messages into categories doesn't benefit from reasoning. The routing decision is simple enough that the fast model gets it right at high accuracy. Adding reasoning triples the cost and adds latency for no quality gain.

Simple information retrieval. "What's my account balance?" doesn't benefit from thinking. The retrieval is straightforward, the response is determined by the data, not by complex reasoning. Save the reasoning budget for scenarios that require it.

Streaming responses. If you're streaming tokens to the UI as they're generated (common in chat interfaces), reasoning models add an awkward silent pause before the first token appears. Standard models start streaming within milliseconds. For streaming interfaces, either suppress the stream and show a spinner during thinking, or use a standard model for streaming and reserve reasoning for the final confirmation step.

Cost model and budgeting

Thinking tokens cost roughly the same as standard input tokens, but you generate many more of them per request. A typical reasoning call generates 2,000-5,000 thinking tokens before a 500-1,000 token output. Compare this to a standard model generating a 500-token output directly.

For a conversation that uses 8 turns of a fast model at $0.015 each, the total cost is $0.12. Adding one reasoning model planning call at $0.06 per call brings the total to $0.18 for the session, a 50% increase. For complex scenarios where the planning call prevents escalation (which costs $4-8 in human agent time), the economics are strongly favorable.

Budget planning: run reasoning models on the subset of conversations that trigger your "complex decision" classifier. For most CX deployments, 15-30% of conversations involve decisions complex enough to benefit from reasoning. Keep the other 70-85% on fast, cheap models. Track cost per session by conversation type to verify the segmentation is working.

The analytics dashboard in Chanl tracks token costs broken down by model and conversation type, which makes this segmentation analysis straightforward in production. Monitoring reasoning model spend is part of Chanl's build-connect-monitor loop for CX agent infrastructure: you build the planning-execution architecture, connect it to your tools, then monitor the cost and quality signals to calibrate the reasoning budget over time.

Evaluating whether reasoning improved the outcome

Evaluate reasoning quality by inspecting thinking content for a sample of calls, comparing trajectory quality against standard models on the same scenario set, and tracking escalation rates by model type over time. One risk with reasoning models: the thinking phase can produce confident-sounding wrong plans. "I thought carefully about this, and I've decided to do the wrong thing" is worse than a standard model hedging correctly.

Evaluate reasoning quality separately from output quality.

Inspect the thinking (if your provider exposes it) for a random sample of reasoning calls. Look for:

• Does the thinking identify all the relevant conditions?
• Does it correctly apply the policy rules to the specific situation?
• Does the plan it generates match the optimal sequence for this scenario type?

Compare trajectory quality between reasoning and standard model on the same scenario set. Run both on 100 complex scenarios and score the action sequences, not just the final responses. The trajectory evaluation approach covers the evaluation methodology. This is especially important for planning-execution architectures where you're evaluating the plan rather than the final response.

Track escalation rates by model type. If your reasoning model is actually making better decisions, you should see lower escalation rates on the complex scenarios it handles. If escalation rates are the same or higher, the reasoning isn't improving outcomes in practice.

In Chanl's scenario testing system, you can run the same complex scenarios through reasoning vs. standard models and compare quality scores automatically. This gives you the data to make the build vs. skip decision for reasoning before you change your production architecture.

The decision framework

The reasoning-model question reduces to four questions for each CX scenario type:

1. Is the decision complex enough? Multi-condition eligibility, multi-step action sequences, high-stakes retention: yes. Simple lookups, routing: no.
2. Is the channel latency-tolerant? Chat, email, async workflows: yes. Voice, streaming: no.
3. Is the volume low enough for the cost? Complex cases (15-30% of conversations): yes. Every conversation: probably not.
4. Is escalation more expensive than reasoning? Usually yes for premium customers and complex issues.

Teams that run this checklist end up with reasoning models on 15-30% of their conversation scenarios and fast models on the rest. This is separate from the framework selection decision covered in AI agent frameworks compared 2026 -- reasoning model architecture sits on top of whichever orchestration framework you chose. The quality improvement on the complex cases is real. The cost and latency on the simple cases make reasoning models the wrong tool.

The planning-execution architecture is the practical middle path: use reasoning for the decision, not for the delivery. One deliberate planning call per complex scenario, fast execution for the steps that follow. The model thinks slowly once and acts fast repeatedly.

Learn Agentic AI

Weekly. Patterns for shipping agents that work. MCP, scorecards, regression tests, prompts, model comparisons.

Subscribe

500+ builders subscribed