Articles tagged “cost-optimization”
11 articles
How LLM Routing Cuts Agent Costs 40-85% in Production
Most CX agents send every request to their priciest model. Routing simple queries to a cheap model cuts costs 40-85%, at 90-95% of single-model quality.
Most of your agent traffic doesn't need your best model
Routing 100% of agent traffic to your most capable model is costing you 6-10x what it should. Here's how to build a cascade router that handles simple tasks cheaply and escalates to stronger models only when the task demands it.
Reasoning models for CX agents: when to think slow, when to move fast
Extended thinking models improve decision quality for complex CX scenarios, but add 5-30 seconds of latency and cost 3-5x more. Here's the planning-execution architecture that gets you the benefit without the tradeoff.
Eval sampling for production agents: the 100/10/1 playbook
Running LLM-as-judge on every production conversation costs more than the agent itself. Here's the three-tier sampling system -- 100% lightweight heuristics, 10% LLM judge, 1% human review -- that keeps quality high without multiplying your eval bill.
Tool result caching: the latency and cost wins hiding in your stack
Your agent re-fetches the same data on every call. Tool result caching cuts latency by up to 70% and inference costs by 40-60% with changes that take days, not weeks. Here's how to classify, implement, and measure it.
Everyone Benchmarks Opus. Your Chatbot Runs on Haiku.
Haiku 4.5, GPT-5 Mini, Gemini Flash at the $1/MTok tier that powers CX. Tool-call accuracy, first-token latency, structured-output reliability, blended cost math.
Your Agent Should Use Three Models, Not One
Production CX agents route tasks by difficulty, not brand loyalty. The planner/router/summarizer pattern, a concrete rubric, support-deflection cost math, and the failure modes nobody warns you about.
Reasoning Tokens Are Showing Up on the Bill
GPT-5 and Claude thinking tokens bill as output and stay invisible. A 200-token reply can hide 8,000 billable ones. How to measure, cap, and budget.
A 1B Model Just Matched the 70B. Here's How.
How to distill frontier LLMs into small, cheap models that retain 98% accuracy on agent tasks. The teacher-student pattern, NVIDIA's data flywheel, and the Plan-and-Execute architecture that cuts agent costs by 90%.
Why Your AI Bill Is 30x Too High
Small language models match GPT-3.5 at 2% of the size and 95% less cost. Benchmarks, code, and a migration story from $13K/month to $400.
Your AI Agent Costs $13K/Month. Here's the Fix.
A production customer-service agent burned $13,247 in one month. Prompt caching, model routing, batch processing, and plan-and-execute architecture cut it to $1,100. Real pricing math for every technique.
El briefing de Signal
Un email por semana. Cómo los equipos líderes de CS, ingresos e IA están convirtiendo conversaciones en decisiones. Benchmarks, playbooks y lo que funciona en producción.
