Stop Spending Tokens on Agent Routing

Most CX teams end up with a supervisor pattern. One agent receives the call, decides which specialist to route to, hands off, gets the result back, decides what to do next. It works. It's debuggable. And it spends 20-40% of every workflow's tokens on routing decisions that don't actually require reasoning.

The billing call gets routed to billing. The refund call gets routed to refunds. The escalation fires when the tool call fails. None of those decisions needed an LLM. You already knew the answer before you wrote the system prompt.

In May 2026, Microsoft released Conductor, an open-source CLI (MIT license) that makes this explicit. You define your workflow in YAML, write the routing conditions as Jinja2 expressions, and run it. No model in the orchestration loop. No tokens spent deciding which agent goes next. Just a decision table that executes predictably every time.

This post covers when that trade-off makes sense, what Conductor actually does differently from LangGraph and CrewAI, and how the monitoring picture changes when your routing is deterministic.

Why Your Routing Probably Doesn't Need to Think

Most CX workflows have three to five intents at the triage layer. Billing. Refund. Technical. Scheduling. Escalation. The routing logic is "which of these five buckets does this call belong in?" Once the triage agent produces an intent label, the routing decision is trivial -- and it doesn't get less trivial with a better model.

The supervisor pattern makes the routing decision with an LLM call because that's the architecture, not because the decision requires intelligence. You're paying for the model to re-read the full conversation history, decide "this is a billing call," and output the billing agent's name. The decision was the same every time. The LLM just makes it slower and more expensive.

The cost isn't academic. In the supervisor vs. swarm breakdown, we measured the token overhead at 20-40% for typical CX workflows with 4-8 hops. That overhead comes from the router call at every hop: receive context, summarize what you know, decide where to go next, emit output. On Sonnet-class pricing for a team doing 50,000 calls per month at 6 hops average, that's roughly $1,800-$2,400 per month in pure routing overhead. At 5 million calls, it scales linearly to $180,000-$240,000 annually.

For the fraction of your call volume where routing is ambiguous -- where the agent genuinely needs to reason about what to do next -- the LLM router earns its cost. For the 80% where routing follows known patterns, you're paying for a decision you already made when you wrote the workflow.

What Conductor Actually Does

Conductor routes multi-agent AI workflows using YAML definitions and Jinja2 template expressions evaluated against each step's output object. The orchestration layer consumes zero tokens. Routing is deterministic by construction.

A Conductor workflow has two sections. The agents block defines each agent: which model it runs on, what its system prompt is, and which tools it has access to. The steps block defines the workflow: each step names an agent and declares a next block with when/goto condition pairs that control routing after the step completes.

Here's what that looks like for a customer support triage workflow:

name: customer-support
version: "1"
 
agents:
  triage:
    model: claude-sonnet-4-6
    system: |
      Classify the customer request.
      Respond with JSON only:
        intent: "billing" | "refund" | "technical" | "other"
        confidence: 0.0-1.0
        priority: "normal" | "urgent"
 
  billing:
    model: claude-sonnet-4-6
    system: "Handle billing questions. Respond with JSON: resolved (bool), summary (str)."
    tools:
      - name: lookup_account
      - name: update_payment_method
 
  refund:
    model: claude-sonnet-4-6
    system: "Process refund requests. Respond with JSON: approved (bool), amount (number), reason (str)."
    tools:
      - name: check_refund_eligibility
      - name: issue_refund
 
  escalation:
    model: claude-opus-4-8
    system: "Handle complex issues requiring judgment. Respond with JSON: action (str), summary (str)."
    tools:
      - name: lookup_account
      - name: create_support_ticket
      - name: schedule_callback
 
steps:
  - name: classify
    agent: triage
    next:
      - when: "{{ output.intent == 'billing' and output.confidence > 0.8 }}"
        goto: billing
      - when: "{{ output.intent == 'refund' and output.confidence > 0.8 }}"
        goto: refund
      - else: escalation
 
  - name: billing
    agent: billing
    next:
      - when: "{{ output.resolved }}"
        goto: summary
      - else: escalation
 
  - name: refund
    agent: refund
    next:
      - when: "{{ output.approved }}"
        goto: summary
      - else: escalation
 
  - name: escalation
    agent: escalation
    next:
      - goto: summary
 
  - name: summary
    agent: summarizer
    terminal: true

Every routing decision in that file is a boolean expression. If the triage agent outputs { "intent": "billing", "confidence": 0.92 }, the first condition matches and Conductor routes to the billing step. No model was consulted for that routing decision. No tokens were spent.

The else clause catches anything that doesn't match an explicit condition. Low-confidence triage, unexpected output fields, edge cases the designer didn't anticipate -- they all fall to escalation, which is your human-judgment fallback. The fallback is explicit in the file rather than buried in a router agent's system prompt.

How This Compares to LangGraph and CrewAI

If you're already on LangGraph, you've seen that it supports deterministic routing through conditional edges. A StateGraph with conditional edges is functionally equivalent to what Conductor does in YAML -- the routing logic is a Python function that reads state and returns the next node name. Same token savings, more type safety, more integration with the broader LangGraph ecosystem.

# LangGraph conditional routing -- equivalent logic, different ergonomics
def route_after_triage(state: SupportState) -> str:
    if state.intent == "billing" and state.confidence > 0.8:
        return "billing"
    elif state.intent == "refund" and state.confidence > 0.8:
        return "refund"
    else:
        return "escalation"
 
graph.add_conditional_edges("triage", route_after_triage)

The LangGraph version is fine. If you're already using it, keep using it. Conductor's value shows up when your team wants to manage workflows as configuration rather than code. The YAML is diffable, reviewable in a pull request, and readable by product managers who know the business logic but don't write Python. For teams where routing rules change frequently and ownership of those rules belongs to non-engineers, the YAML format earns its weight.

CrewAI sits at the opposite end. Its routing is more LLM-driven by default, with agents assigning tasks through goal-oriented reasoning. That's the right choice when your workflow structure genuinely isn't known in advance -- when an agent needs to discover sub-tasks dynamically, assess what work has been done, and decide what to do next without a predetermined graph to follow. Deterministic routing is too rigid for that use case.

The frameworks comparison post covers when each framework earns its keep across the full picture of agent architectures. Conductor sits in a specific band: workflows where the routing conditions are known, the team wants to manage routing as configuration, and the token savings at scale are worth the rigidity.

What Deterministic Routing Can't Handle

Conductor's declarative model has a hard limit: it can only route based on what the previous step's output contains. If your workflow requires the orchestration layer to reason about novel situations, Conductor doesn't help.

Specifically, it's the wrong tool when:

The set of possible next steps isn't known at design time. If an agent discovers a sub-task mid-workflow that wasn't anticipated when the YAML was written, Conductor has no way to route to it. The else clause sends those cases to your fallback agent, which may not be the right specialist.

Routing decisions require multi-turn context. Conductor routes based on the immediate previous step's output. If the routing decision requires synthesizing context from three earlier steps -- "is this the third time this customer has called about the same issue?" -- that logic has to live inside an agent, not in the YAML routing layer.

The conditions are genuinely ambiguous. Jinja2 expressions are boolean. {{ output.intent == 'billing' }} evaluates to true or false. If intent classification is fuzzy and you want the router to exercise judgment about boundary cases -- weighing tone, history, and urgency together -- you need a model making the routing decision.

Most production CX systems are a mix. Some parts have known structure: intent is one of five values, tool calls succeed or fail, confidence scores cross thresholds. Other parts require genuine reasoning: is this a good customer deserving extra flexibility? does this situation warrant a policy exception? Deterministic routing handles the first half. LLM routing handles the second half.

The practical pattern: put Conductor or LangGraph conditional edges in the outer shell, and let individual agents do the reasoning inside their own contexts. Your triage-to-specialist routing is deterministic. What the specialist does when it gets the call is not.

Running Conductor in Production

Conductor is a CLI, which means you invoke it from your application's subprocess layer or from whatever orchestration infrastructure you already use. The workflow YAML and agent definitions live in your repo. You invoke a call through a workflow like this:

conductor run \
  --workflow support-workflow.yaml \
  --input '{"message": "I was charged twice for my order"}' \
  --output trace.json

The output trace includes every step's input, output, and routing decision. The routing_decision field in each step shows which condition matched and which step it routed to. That trace is your audit log for every call that flows through the workflow.

For production, you'd call Conductor programmatically from your service layer rather than from a terminal:

import { Chanl } from "@chanl/sdk";
import { execFile } from "child_process";
import { promisify } from "util";
 
const execFileAsync = promisify(execFile);
const chanl = new Chanl({ apiKey: process.env.CHANL_API_KEY });
 
async function runSupportCall(message: string, callId: string) {
  const input = JSON.stringify({ message });
 
  const { stdout } = await execFileAsync("conductor", [
    "run",
    "--workflow", "support-workflow.yaml",
    "--input", input,
  ]);
 
  const trace = JSON.parse(stdout);
 
  // Log to Chanl monitoring for branch distribution tracking
  await chanl.calls.logTrace({
    callId,
    trace: trace.steps,
    metadata: {
      workflow: "customer-support",
      finalBranch: trace.terminal_step,
      totalSteps: trace.steps.length,
      routingPath: trace.steps.map((s: { name: string }) => s.name),
    },
  });
 
  return trace.output;
}

Monitoring Deterministic Workflows

Moving to deterministic routing changes what you monitor. A supervisor leaves behind a chain of router outputs that explain why each routing decision was made -- the reasoning is in the trace. Conductor leaves behind condition evaluation logs that show what condition matched. That's less rich for individual debugging, and it creates a specific blindspot.

If the triage agent starts misclassifying calls -- routing billing questions to refund because a model update changed its output JSON schema -- Conductor will correctly follow the YAML rules and send those misclassified calls to the refund agent. The routing is doing exactly what it's supposed to do. The problem is upstream, in the triage agent, and the only way to catch it is to monitor branch distribution.

Branch distribution is the key observability primitive for deterministic workflows. You want to know what fraction of calls take each branch, and alert when that distribution shifts materially. If billing usually handles 45% of calls and drops to 22%, something changed -- either call patterns shifted, or triage started classifying differently. Neither is obvious from per-call traces.

You also want step latency percentiles per branch. The escalation branch runs on Opus-class models and costs more per call -- a spike in escalation volume is both a quality signal and a cost signal. If billing latency p95 jumps, a tool call started timing out. These breakdowns live in Chanl's monitoring dashboards when you pass the Conductor trace through the integration above.

The other signal to watch: what fraction of calls land on else at each decision point. The else clause is your catch-all for cases the routing designer didn't anticipate. A spike in else calls means your conditions aren't covering the actual call distribution -- either a new intent type emerged, or triage started producing output fields that don't match your Jinja2 conditions. Run a scenario suite against the current YAML to find which call types are landing on else when they shouldn't be.

When to Introduce Deterministic Routing

Conductor is the right default for workflows where the routing conditions are already known. That's most CX workflows at some point in their maturity: the handful of intents your triage agent can produce, the success or failure signal from each tool call, the confidence threshold below which you escalate.

The supervisor pattern remains the right starting point when you don't yet know what your routing conditions are, or when workflows are genuinely exploratory. Build supervisor first, run it in production, watch what it routes and why, then crystallize those routing patterns into YAML when you're confident they're stable.

That's the sequence: supervisor while you're learning the call distribution, deterministic routing once you've mapped it. The transition is a documentation exercise more than an engineering exercise -- you're reading your supervisor agent's routing decisions and writing them down as Jinja2 conditions. When the YAML matches what the supervisor was already doing, you've eliminated the cost of having the supervisor make those decisions at runtime.

The state machines for deterministic production piece covers the broader case for determinism in agent architecture -- Conductor is one implementation of a wider principle. If your agent needs to be auditable, reproducible, and cost-predictable, the argument for pushing as much of its behavior into deterministic code or configuration as possible applies well beyond routing.

Learn Agentic AI

Weekly. Patterns for shipping agents that work. MCP, scorecards, regression tests, prompts, model comparisons.

Subscribe

500+ builders subscribed