How to Build an AI Voicemail Agent That Gets a Callback (AMD, TCPA)

Eighty to ninety percent of B2B cold calls hit voicemail. The standard response is to hang up. The AI-dialer response is to drop a generic synthetic message that gets ignored 90 percent of the time. Both are bad.

There's a third option worth building. A pipeline that detects voicemail in under a second, leaves a personalized message that sounds human, captures the callback even when the lead dials in from a different number, and resumes the conversation with full context. Doable. But you have to assemble four moving parts in the right order, with TCPA and FCC compliance wired in from the start.

The shape of it: outbound dialer triggers a call. Twilio's Answering Machine Detection decides human or machine. If machine, we play a pre-rendered voicemail with three personalization variables. We write a pending-callback record keyed by the lead's phone. The lead calls our toll-free number back, possibly from a different number. The inbound webhook hydrates the agent with the voicemail context. The agent picks up where the voicemail left off.

Each piece on its own is a 30-line problem. Glued together, it's a pipeline. Most teams ship the first piece and call it done.

Detect Machine vs Human in Under 500 ms

The first decision is whether you wait for AMD before doing anything. If you don't, your agent answers the voicemail beep, says hello to a recorded greeting, and sounds insane.

Twilio's AMD has two modes that matter. Enable returns AnsweredBy as soon as it decides, with values human, machine_start, fax, or unknown. Use this when you want speed. DetectMessageEnd waits for the greeting to finish and returns machine_end_beep, machine_end_silence, or machine_end_other. Use this when you actually want to leave a voicemail, because you need to know the greeting is over.

The tradeoff is latency. DetectMessageEnd can take up to seven seconds in worst cases. That's an eternity if a human picked up. The fix is conditional branching: start with DetectMessageEnd and short-circuit on a human determination, which Twilio returns as soon as it's confident.

import twilio from 'twilio';
 
const client = twilio(process.env.TWILIO_SID!, process.env.TWILIO_TOKEN!);
 
await client.calls.create({
  to: lead.phone,
  from: process.env.TWILIO_OUTBOUND_NUMBER!,
  url: `${process.env.APP_URL}/twiml/dispatch`,
  machineDetection: 'DetectMessageEnd',
  machineDetectionTimeout: 5,
  machineDetectionSpeechEndThreshold: 1200,
  asyncAmd: 'true',
  asyncAmdStatusCallback: `${process.env.APP_URL}/twiml/amd-result`,
  statusCallback: `${process.env.APP_URL}/twiml/status`,
  statusCallbackEvent: ['initiated', 'answered', 'completed'],
});

The async AMD callback decouples detection from the call lifecycle. While the call is ringing, you can start synthesizing the voicemail in parallel so it's ready the instant machine_end_beep fires.

When the AMD result arrives, branch on AnsweredBy. The TwiML response either hands control to the live agent or plays the pre-rendered voicemail.

app.post('/twiml/amd-result', async (req, res) => {
  const { CallSid, AnsweredBy } = req.body;
 
  if (AnsweredBy === 'human') {
    return res.type('text/xml').send(`
      <Response>
        <Connect><Stream url="wss://app.example.com/agent/${CallSid}"/></Connect>
      </Response>
    `);
  }
 
  if (['machine_end_beep', 'machine_end_silence', 'machine_end_other'].includes(AnsweredBy)) {
    const audioUrl = await getRenderedVoicemail(CallSid);
    await registerPendingCallback({ callSid: CallSid, leadPhone: req.body.To });
    return res.type('text/xml').send(`
      <Response>
        <Play>${audioUrl}</Play>
      </Response>
    `);
  }
 
  return res.type('text/xml').send('<Response><Hangup/></Response>');
});

unknown is a real outcome. About 5 to 10 percent of AMD attempts return it on noisy lines or short greetings. Hang up rather than guessing. Cold-leaving a voicemail to a human is worse than missing one to a machine.

Pre-Render the Voicemail, Don't Stream It

Voicemail systems hate streaming TTS. Carrier voicemail expects a continuous audio stream and any sub-second gap looks like end-of-message silence, which can truncate your recording. The fix is to render the voicemail once, store it, and play the file.

A voicemail with three variables, name, company pain, callback number, hits the personalization bar without the latency cost of full LLM generation per call. Synthesize once during dial, cache by lead ID, and reuse if you re-attempt.

import { ElevenLabsClient } from 'elevenlabs';
import { uploadToCdn } from './storage';
 
const eleven = new ElevenLabsClient({ apiKey: process.env.ELEVEN_API_KEY! });
 
export async function renderVoicemail(lead: Lead, callSid: string): Promise<string> {
  const script = [
    `Hey ${lead.firstName}, this is Sarah from Acme.`,
    `I noticed your team is hiring for ${lead.role}, and we help reduce time-to-fill by about 40 percent.`,
    `If you'd like to chat, give me a call back at ${lead.callbackNumber}.`,
    `This is an automated message. To opt out, reply STOP to any text from this number.`,
    `Thanks ${lead.firstName}.`,
  ].join(' ');
 
  const audio = await eleven.textToSpeech.convert('21m00Tcm4TlvDq8ikWAM', {
    text: script,
    modelId: 'eleven_multilingual_v2',
    outputFormat: 'mp3_44100_128',
  });
 
  return uploadToCdn(`voicemail/${callSid}.mp3`, audio);
}

Quick notes on that script. The opt-out instruction isn't optional. The "automated message" line is required in California and prudent everywhere else. Total runtime synthesizes to about 22 seconds, which keeps you under the 30-second soft limit most carriers enforce before they truncate. ElevenLabs' Multilingual v2 model gives the most natural cadence we've found; if you care more about latency than cadence, Flash v2.5 lands around 75 ms.

The Compliance Bar Is Non-Negotiable

Before you ship any of this, the legal piece. The FCC issued a Declaratory Ruling on February 8, 2024 that AI-generated voices fall under TCPA's "artificial or prerecorded voice" restriction. That means three things for every call:

1. Prior express consent. The lead must have agreed to receive automated calls from you. Cold-dialing a list you bought is illegal. Consent has to be specific, traceable, and revocable.
2. Caller identification. The voicemail must state the calling party's name, the business name, and a callback number where the recipient can reach you to opt out.
3. Opt-out mechanism. Every voicemail needs a working unsubscribe path. STOP-to-text is the cheapest. A pressable digit during a live call works too.

Enforcement is real. In 2024 the FCC fined Steve Kramer $6 million for the AI-deepfake Biden robocalls during the New Hampshire primary. Lingo Telecom paid $1 million and signed a first-of-its-kind compliance plan covering STIR/SHAKEN, KYC, and Know Your Upstream Provider.

The FCC's July 2024 NPRM (FCC 24-59) proposes adding an explicit in-call disclosure that AI is being used. Read that as: the floor is rising.

State law adds another layer. California's AB 2905, effective 2025, requires AI disclosure at the start of every call with a $500 fine per undisclosed call. Florida's Telephone Solicitation Act restricts calling hours to 8 AM to 9 PM and applies to B2B. New York and a growing list of states are following. Build the disclosure once, key it by the lead's state of record, and inject it into every voicemail script.

Match Callbacks Across Phone Numbers

Roughly 30 percent of callbacks don't come from the number you dialed. The lead calls from their cell instead of the office. A spouse picks up the message and calls back from a personal line. The lead's assistant returns the call.

A naive findOne({ phone }) lookup misses all of those. You need a matching strategy.

type PendingCallback = {
  leadId: string;
  leadPhone: string;
  altPhones: string[];
  voicemailContext: string;
  expiresAt: Date;
};
 
export async function findPendingCallback(callerPhone: string): Promise<PendingCallback | null> {
  const e164 = normalizeE164(callerPhone);
  const last7 = e164.slice(-7);
 
  let match = await db.collection('pending_callbacks').findOne({
    $or: [{ leadPhone: e164 }, { altPhones: e164 }],
    expiresAt: { $gt: new Date() },
  });
  if (match) return match;
 
  match = await db.collection('pending_callbacks').findOne({
    $or: [
      { leadPhone: { $regex: `${last7}$` } },
      { altPhones: { $elemMatch: { $regex: `${last7}$` } } },
    ],
    expiresAt: { $gt: new Date() },
  });
  if (match) return match;
 
  const recentLead = await db.collection('leads').findOne({
    company: await reverseLookupCompany(callerPhone),
    voicemailDroppedAt: { $gt: hoursAgo(72) },
  });
  return recentLead ? toPendingCallback(recentLead) : null;
}

Three tiers: exact E.164 match, last-seven-digit fuzzy match, and a fallback to company-based reverse lookup. Walk them in order, take the first hit. Set a 72-hour TTL on pending callbacks because anything older than that and the lead has forgotten what your voicemail was about.

The Inbound Resume Is the Part Nobody Else Builds

When the inbound call hits Twilio, your webhook has under 200 ms to decide what agent to spawn and what context to give it. Look up the pending callback, hydrate the system prompt, and connect.

app.post('/twiml/inbound', async (req, res) => {
  const { From, CallSid } = req.body;
 
  const pending = await findPendingCallback(From);
 
  const sessionId = await createAgentSession({
    callSid: CallSid,
    systemPrompt: pending
      ? buildResumePrompt(pending)
      : buildGenericInboundPrompt(),
    metadata: { pendingCallbackId: pending?.leadId },
  });
 
  if (pending) {
    await markPendingCallbackResolved(pending.leadId);
  }
 
  return res.type('text/xml').send(`
    <Response>
      <Connect>
        <Stream url="wss://app.example.com/agent/${sessionId}"/>
      </Connect>
    </Response>
  `);
});
 
function buildResumePrompt(p: PendingCallback): string {
  return `
You called this lead recently and left a voicemail. Here's what you said:
"${p.voicemailContext}"
 
The lead is calling you back. Acknowledge the prior voicemail naturally,
do not pretend it didn't happen. Pick up the conversation where you left off.
 
If the lead seems to have forgotten the voicemail, briefly remind them.
Always offer to send a calendar link or transfer to a human if asked.
  `.trim();
}

That buildResumePrompt call is the whole ballgame. Without it the agent answers like a generic inbound ("Hi, how can I help?") and the lead has to re-explain who they are and why they called. With it the agent opens with "Hey, thanks for calling back, I left you a voicemail yesterday about Q2 budget." That's the difference between a meeting booked and a hangup.

Sentiment Analysis

Last 7 days

Positive 68%

Neutral 24%

Negative 8%

Top Topics

Billing342

Support281

Onboarding197

Upgrade156

The annotations on a real callback transcript make the value of this hydration obvious. The agent's first 10 seconds reference the prior voicemail, name the topic, and offer the next step, all from context that wouldn't exist without the registry lookup.

Doing This With Chanl

The pipeline above is roughly 200 lines across four files. Chanl's @chanl/sdk collapses most of it because the agent platform itself owns memory, scenarios, and scorecards.

import { Chanl } from '@chanl/sdk';
 
const chanl = new Chanl({ apiKey: process.env.CHANL_API_KEY! });
 
// 1. Outbound dial. AMD branching to a voicemail-specific agent
//    is on the roadmap; today the dialer-side AMD logic still lives
//    in your TwiML layer. Flagged below.
await chanl.calls.initiate({
  agentId: 'agent_outbound_sdr',
  phone: lead.phone,
  customerName: lead.firstName,
  scorecardId: 'scorecard_voicemail_compliance',
});
 
// 2. After voicemail drops, write the context to customer memory.
//    This is what the inbound agent will pull from on callback.
await chanl.memory.create({
  entityType: 'customer',
  entityId: lead.phone,
  content: `Left voicemail about Q2 hiring budget. Mentioned ${lead.role} role and 40% time-to-fill reduction. Callback number: ${lead.callbackNumber}.`,
});
 
// 3. On inbound, the agent boots, reads memory by caller phone,
//    and seeds its system prompt automatically.
const recent = await chanl.memory.search({
  entityType: 'customer',
  entityId: callerPhone,
  query: 'recent voicemail',
});
 
// 4. Score the resulting call against voicemail-specific axes.
await chanl.scorecards.evaluate(callId, {
  scorecardId: 'scorecard_voicemail_compliance',
});
 
// 5. Test the inbound resume flow with a simulated returning lead.
await chanl.scenarios.run('scenario_busy_cmo_callback', {
  agentId: 'agent_inbound_resume',
});

The premise is agents that remember each customer. The voicemail context isn't a side store, it's the same memory layer the agent uses for every other interaction with that lead. Email exchange, prior call, support ticket, voicemail, all in one timeline keyed by entity. The inbound agent doesn't ask the SDK for "voicemail context", it asks for "everything about this lead in the last 72 hours", and the voicemail naturally falls out.

A few SDK gaps worth flagging because they're the difference between five lines and twenty:

• AMD-aware initiate. Today calls.initiate doesn't expose AMD branching to a different agent. Every outbound voice customer needs options: { amd: true, voicemailAgentId } on day one.
• Pending-callback registry. A first-class chanl.callbacks.expect({ phone, agentId, contextMemoryId, ttl }) would let inbound calls auto-route without you writing the lookup code.
• State-aware disclosure helper. A chanl.calls.disclose({ template: 'state-aware' }) that injects the right "this is an automated call" line based on the lead's state would shrink compliance review from a meeting to a checkbox.

The scorecards layer is also where compliance gets enforced. Add axes like compliance_disclosure_made, callback_number_stated, and voicemail_under_30s to your voicemail scorecard. Run them on every call. A pattern of failures triggers an alert before a regulator does. Use scenarios to stress-test the inbound resume flow with simulated callbacks from leads who pretend not to remember the voicemail.

The Shipping Checklist

If you only build one thing from all this, make it the inbound resume. It's the piece nobody else ships and it's the actual reason your callback-to-meeting rate moves. AMD, synthesis, the registry: those are the cost of admission, not the prize.

The hard part isn't any single piece. It's that all four have to be in place before the system actually gets picked up. Skip the disclosure and you eat a $500-per-call fine. Drop the pre-render step and the voicemail truncates mid-sentence. Forget the registry and the inbound agent sounds amnesic. And without AMD, you'll cheerfully tell a voicemail beep that you'd love to chat about Q2 budget.

When the pieces line up, the first thing the lead hears on callback is "Hey, thanks for calling me back." The conversation already has context. The deal moves forward.

The Signal Briefing

One email a week. How leading CS, revenue, and AI teams are turning conversations into decisions. Benchmarks, playbooks, and what's working in production.

Subscribe

500+ CS and revenue leaders subscribed