Enforcing runtime policies on production CX agents

Your CX agent has guardrails: no profanity, no competitor mentions, no personally identifiable data in responses. You've tested them. They hold.

Last week, it agreed to a $2,400 refund for a customer whose policy only covers $150. The conversation sounded completely reasonable. The agent wasn't tricked. It followed the conversation to a logical conclusion that your system prompt technically addressed but didn't prevent.

Guardrails and runtime policies solve different problems, and treating them as the same tool is what causes this failure mode.

What guardrails and runtime policies actually protect

A guardrail filters content. It looks at what the agent produced and checks it against a list of banned outputs: profanity, competitor names, personally identifiable information, toxic language. If the output matches, it's blocked or rewritten. Guardrails are reactive -- they intercept bad outputs after the LLM generates them.

A runtime policy constrains what paths the agent is allowed to take during execution. It's not about what the agent says; it's about what the agent does. Which tools it's permitted to call. Which commitments it's allowed to make. When it must escalate instead of resolve. Runtime policies are proactive -- they prevent certain actions from happening at all, before the LLM gets a chance to take them.

Your system prompt tries to play the role of a runtime policy ("do not discuss refunds over $500"), but it can't enforce anything. Language models drift under conversational pressure, and a natural, reasonable multi-turn interaction has a way of gradually moving the LLM past the line it was told to hold. The more persuasive the customer's framing, the more likely the model is to drift.

If you want real enforcement, the constraint has to live somewhere the LLM can't override it.

What "policies on paths" means

A policy on paths is a constraint on which sequences of actions an agent is permitted to take, evaluated at runtime against the agent's current execution state. The core insight is that enforcement lives in the execution layer, not in the model.

Instead of "do not discuss refunds over $500," a policy on paths says: if the agent has called get_order_value and the return is greater than 500, the agent is not permitted to call issue_refund in this session -- it must call escalate_to_human. The policy doesn't depend on the LLM remembering an instruction. It intercepts the execution at the tool call boundary.

This framing comes from a March 2026 paper, "Runtime Governance for AI Agents: Policies on Paths," which treats agent governance as a constraint-satisfaction problem at the execution layer rather than a natural-language instruction problem at the prompt layer. It's a small conceptual shift with large practical consequences.

The LLM is not the enforcement mechanism. The execution layer is.

The four policy categories your CX agent needs

CX agents need runtime constraints across four distinct categories: topic scope, action permissions, escalation triggers, and data access. Each targets a different failure mode and requires a different enforcement approach.

Topic scope policies define what subjects the agent can engage with -- not as banned phrases, but as a structural definition of what's in and out of scope. A support agent handling order inquiries shouldn't be negotiating service contract terms. A scheduling agent shouldn't be discussing open litigation. These policies are enforced by scoping which tool categories and knowledge base collections the agent is permitted to query.

Action permission policies define what the agent can commit to. Issue refunds up to $X. Schedule appointments in the next 14 days. Waive fees within a defined category. Each action permission is enforced at the tool call level: before issue_refund executes, the policy layer checks whether the amount is within the permitted range for this agent role and conversation context.

Escalation policies define when the agent must hand off rather than continue. Specific language patterns (legal threat, regulatory complaint, crisis signals), more than N unsuccessful resolution attempts, conversation duration exceeding a threshold without closure -- these all trigger mandatory escalation. Enforced by monitoring conversation state and blocking further agent steps until the handoff is complete.

Data access policies define what customer data the agent can read during a session. A billing agent can read payment history but not medical records. A scheduling agent can read appointment history but not purchase history. Enforced by scoping which data sources each agent role is permitted to query.

Each of these is a different thing that can go wrong in production, and each requires a different kind of fix when it does.

The three implementation layers

You can implement runtime policies at three layers, with increasing enforcement strength.

System prompt instructions are the weakest layer. Fast to implement, easy to update, but dependent on the LLM following natural language under adversarial conditions. Appropriate for soft guidance, not hard enforcement.

Tool-level gates are stronger. Wrap each constrained tool with a policy check that runs before the tool executes. The LLM calls the tool normally, but the wrapper intercepts the call and evaluates it against policy state before allowing it through.

import Chanl from "@chanl/sdk";
 
const chanl = new Chanl({ apiKey: process.env.CHANL_API_KEY });
 
async function withPolicyGate<T>(
  toolName: string,
  args: Record<string, unknown>,
  sessionContext: SessionContext,
  toolFn: (args: Record<string, unknown>) => Promise<T>
): Promise<T> {
  const policy = await chanl.tools.checkPolicy({
    toolName,
    args,
    agentRole: sessionContext.agentRole,
    sessionState: {
      customerTier: sessionContext.customerTier,
      previousActions: sessionContext.actionLog,
      conversationDuration: sessionContext.duration,
    },
  });
 
  if (!policy.permitted) {
    if (policy.requiredAction === "escalate") {
      await triggerEscalation(sessionContext, policy.reason);
      throw new PolicyViolationError(
        `Policy requires escalation: ${policy.reason}`
      );
    }
    throw new PolicyViolationError(
      `Tool ${toolName} blocked: ${policy.reason}`
    );
  }
 
  return toolFn(args);
}

Execution-level interceptors are the strongest layer. These operate at the agent loop level, evaluating the full conversation state before each step -- not just the individual tool call. Before the agent takes any action, the interceptor checks whether it's permitted to proceed given everything that's happened so far.

interface PolicySet {
  evaluateState(state: AgentState): PolicyDecision;
}
 
interface PolicyDecision {
  mustEscalate: boolean;
  reason?: string;
  restrictedTopics: string[];
}
 
async function runAgentWithPolicyInterceptor(
  conversation: Conversation,
  policies: PolicySet
): Promise<AgentResult> {
  const agent = createAgent(conversation);
 
  while (!agent.isDone()) {
    const state = agent.getCurrentState();
    const decision = policies.evaluateState(state);
 
    if (decision.mustEscalate) {
      return handleEscalation(conversation, decision.reason);
    }
 
    if (decision.restrictedTopics.length > 0) {
      agent.addConstraint(
        `Do not continue on: ${decision.restrictedTopics.join(", ")}`
      );
    }
 
    await agent.step();
  }
 
  return agent.getResult();
}

Most production CX agents need both layers. Tool-level gates catch individual action violations. Execution-level interceptors catch state-based violations -- where no single action was wrong, but the combination of actions or the conversation trajectory as a whole has crossed a boundary.

Deploy Gate

Pre-deploy quality checks

Score > 80%

92%

Latency < 500ms

234ms

Error Rate < 2%

3.1%

Deploy Blocked

Monitoring policy compliance in production

Policies are only as good as your ability to detect when they drift. Production CX agents handle thousands of conversations, and manual review doesn't scale to that volume.

Automated scoring with scorecards handles this. You define policy compliance as a scoreable criterion -- did the agent stay within its action permission boundaries, did it escalate when required, did it access only the data its role permits -- and run it automatically against every conversation. The results feed into your monitoring dashboard so you can track compliance rate over time and catch regressions quickly.

The metrics to track:

Near-misses are especially useful for policy tuning. If 18% of conversations reach the boundary of your refund limit but stop correctly, your limit might be calibrated well. If 2% cross it anyway, your enforcement has a gap. If 0% reach it at all, your limit might be too conservative for your actual case distribution.

import Chanl from "@chanl/sdk";
 
const chanl = new Chanl({ apiKey: process.env.CHANL_API_KEY });
 
async function getPolicyComplianceReport(timeWindow: string) {
  const metrics = await chanl.calls.getMetrics({
    timeWindow,
    groupBy: "policy_category",
    include: ["violation_rate", "near_miss_rate", "escalation_compliance"],
  });
 
  const violations = await chanl.calls.getViolationPatterns({
    timeWindow,
    limit: 10,
  });
 
  return {
    overallCompliance: metrics.overall.complianceRate,
    byCategory: metrics.byPolicyCategory,
    topPatterns: violations,
  };
}

Testing your policies before they go live

Policy coverage should be tested the same way you test any other agent behavior: with scenarios that specifically target policy boundaries.

For each policy, write three scenario types:

In-policy scenarios cover conversations that should resolve without hitting a policy boundary. These confirm that normal interactions flow correctly. An agent that triggers escalation on a $100 refund when the limit is $500 has a false-positive problem.

Boundary scenarios approach the policy limit but stay within it. A conversation that works up to exactly $499. These verify the agent handles edge cases correctly without unnecessary escalation.

Out-of-policy scenarios are designed to trigger the constraint. A customer asking for a $2,400 refund. A conversation that should escalate because of legal language. These verify the enforcement fires when it should -- and that the fallback behavior (escalation, decline, redirect) happens correctly.

The production guardrails work covers why input-level filtering fails under adversarial inputs. The same logic applies here: untested policies are intentions, not guarantees. And if you're running multi-agent CX systems where containment matters, each agent in the chain needs independently tested policies, because a containment failure in one agent propagates to all the agents downstream of it.

Using the tools layer to scope permissions

One pattern that works well in practice is encoding action permissions directly in your tool definitions. Rather than having a single issue_refund tool that accepts any amount and relying on a policy check to gate it, define tiered tools that express the permission boundary in their structure.

const tools = [
  {
    name: "issue_standard_refund",
    description:
      "Issue a refund for orders under $500. Use for standard return and satisfaction cases.",
    parameters: {
      order_id: { type: "string" },
      amount: { type: "number", maximum: 500 },
      reason: { type: "string" },
    },
  },
  {
    name: "request_escalated_refund_review",
    description:
      "Request a refund review for amounts over $500. Submits to the billing team for approval. Do not promise a specific outcome to the customer.",
    parameters: {
      order_id: { type: "string" },
      requested_amount: { type: "number" },
      context: { type: "string" },
    },
  },
];

When the LLM sees these two tools, the permission boundary is built into the tool choice itself. The agent can't call issue_standard_refund with $2,400 because the schema rejects it. The escalation path is the only tool available for large amounts. This doesn't replace the policy layer -- you still need execution-level enforcement for conversation-state policies -- but it reduces the surface area the policy layer has to cover.

Governance is the production bottleneck, not model capability

Model capability isn't what's holding back enterprise CX agent deployments in 2026. The models are capable enough for the conversations. What's failing is governance: the ability to deploy agents at scale while maintaining confidence that they'll stay within the bounds you've set.

A governance gap isn't a theoretical compliance risk. It's a concrete operational failure: an agent committing to resolutions it can't deliver, accessing data it shouldn't see, or skipping escalations that should have happened. These don't show up in CSAT until customers start noticing a pattern.

Runtime policies are the engineering answer to that gap. They're the layer between "the LLM should follow these instructions" and "the system enforces these constraints regardless of what the LLM decides." Building that layer isn't the interesting part of building a CX agent. It is, though, what makes the difference between a pilot you can demo and a fleet you can trust at production scale.

El briefing de Signal

Un email por semana. Cómo los equipos líderes de CS, ingresos e IA están convirtiendo conversaciones en decisiones. Benchmarks, playbooks y lo que funciona en producción.

Subscribe

500+ líderes de CS e ingresos suscritos