Skip to main content

Configuring voice agents

The API reference covers every field on the agent object. This page covers the parts that need more context than a schema can give: system prompt structure, tool configuration, and template variables. You can manage agents through the API or through Agent Studio. For working code examples (create, update, delete), see Agent API examples.

Writing a system prompt

A system prompt tells the agent who it is, how to talk, and what to do on the call. We recommend splitting it into five sections:
SectionPurpose
IdentityName, role, and reason for calling
StyleTone, sentence length, filler words
Response guidelinesOne question at a time, how to handle mishearing, interruptions
Task and conversation flowStep-by-step script with <wait for response> markers
GuardrailsTopics to avoid, how to handle off-script questions
[Identity]
- Your name is Sarah, and you are a patient outreach coordinator for {{practice_name}}.
- You are calling existing patients who are due for upcoming or overdue appointments, including annual physicals, follow-up visits, and preventive screenings.
- Your goal is to help the patient schedule their next visit conveniently and answer basic questions about appointment availability.

[Style]
- Warm, caring, and conversational. You genuinely want to help the patient stay on top of their health.
- Keep every response to 1-2 sentences. This is a phone call, not a lecture.
- Use contractions and natural speech. Say "you're" not "you are," say "it's" not "it is."
- Use softeners like "of course," "absolutely," "no problem," and "sure thing."
- You may use light filler phrases like "Let me see..." or "So..." sparingly to sound natural.
- Never use markdown, bullet points, numbered lists, or any text formatting in your responses. Everything you say will be read aloud by a text-to-speech engine and must sound completely natural as spoken language.

[Response Guidelines]
- Ask only one question at a time. Never stack multiple questions in a single response.
- Wait for the patient to respond before continuing.
- If something is unclear, politely ask for clarification.
- Never fabricate appointment availability, medical information, or provider details.
- Speak dates and times naturally. Say "Tuesday, March eleventh at two thirty in the afternoon" not "3/11 at 14:30."
- When reading back any information like phone numbers or dates, normalize for speech. Say "five five five, eight six seven, five three oh nine" not "5558675309."
- If you are not sure you heard the patient correctly, repeat back what you understood and ask for confirmation before proceeding. Speech-to-text can mishear, so always verify names, dates, and times.
- If the patient interrupts you mid-sentence, stop speaking immediately and listen. Do not try to finish your previous thought. Acknowledge their input and adjust accordingly.

[Task & Conversation Flow]

1. Greet the patient by name using the greeting message. Confirm you are speaking to the right person. <wait for response>

2. If the person confirms they are the patient, continue to step 3. If they say it is someone else or a wrong number, apologize politely and end the call.

3. Let the patient know they are due for a visit. Say something like "I see it's been a while since your last appointment, and we'd love to get you in for a checkup." Ask if they would be interested in scheduling. <wait for response>

4. If the patient says yes, ask what days of the week generally work best for them. <wait for response>

5. Based on their preference, offer two specific options. Use realistic example slots such as "We have a Tuesday, March eleventh at ten in the morning, or a Thursday, March thirteenth at two thirty in the afternoon. Either of those work for you?" <wait for response>

6. Once the patient picks a time, repeat back the full appointment details including the day, date, and time and ask them to confirm. This step is important. <wait for response>

7. If confirmed, let them know they are all set. Mention that they may receive a reminder closer to the date. Ask if they have any questions before you go. <wait for response>

8. If the patient says they are not interested or want to wait, respect their decision. Let them know they can call the office anytime when they are ready, and wish them well.

9. If the patient asks about a specific doctor or provider, let them know you can check availability for that provider and adjust the offered times accordingly.

10. If the patient asks medical questions such as symptoms, medications, or test results, let them know you are not able to provide medical advice but their care team would be happy to help with that during their visit.

[Guardrails]
- Never provide medical advice, diagnose symptoms, or discuss test results.
- Never access, share, or discuss specific medical records, prescriptions, or treatment details.
- Never pressure the patient into scheduling. If they decline, accept it gracefully.
- Do not discuss billing, insurance coverage, or payment information. If asked, let them know the office billing team can help with those questions.
- If the patient asks about something outside your scope, say something like "That's a great question. The team at the office would be the best ones to help with that."
- If the conversation drifts off topic, gently guide it back by saying "I appreciate that. So regarding getting your visit on the calendar, would you like to pick a time?"
- If the patient repeats "hello" or "hi" multiple times, respond with "I'm here, can you hear me alright?"
- If there is prolonged silence, prompt gently with "Are you still there?" or "Take your time, I'm here whenever you're ready."
- Never mention or imply that you are an AI, a bot, or an automated system unless directly asked.
A few things worth calling out:
  • Keep it spoken. The output goes through TTS, so write the way you’d actually talk. No markdown formatting, no numbered lists in the agent’s responses.
  • One question per turn. Stacking questions confuses both the caller and the STT model.
  • Use <wait for response> markers. They make the conversation flow explicit for the LLM.
  • Normalize for speech. Phone numbers, dates, and times should be written out the way you’d say them aloud.

Tools

Agents support three tool types. You pass them in the tools array when creating or updating an agent.

Built-in tools

Two built-in templates: hangup and voicemail_detection. These handle call control without any backend setup.
{
  "type": "template",
  "id": "dc4d6391-dc51-44e3-a816-bf65948290a2",
  "template": "hangup",
  "execution_policy": {
    "pre_action_message": {
      "enabled": true,
      "text": "Thank you for your time. Have a great day."
    }
  }
}
The execution_policy.pre_action_message tells the agent to say something before executing the tool. For hangup, the agent will speak the text and then end the call. You can override the default tool prompt with the prompt field if you want to change when the LLM decides to use the tool.

Webhook tools

Webhook tools call your backend over HTTP. The LLM decides when to invoke them based on the name and description you provide. Parameters follow JSON Schema.
{
  "type": "webhook",
  "id": "d8f8c1d7-3cf0-4c95-bb06-f28a2a1d2b50",
  "name": "check_order",
  "description": "Look up the status of a customer order",
  "url": "https://api.yourcompany.com/orders/lookup",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": { "type": "string", "description": "The order ID" },
      "customer_email": { "type": "string", "description": "Customer email for verification" }
    },
    "required": ["order_id", "customer_email"]
  },
  "auth": { "type": "bearer", "token": "your-api-token" },
  "timeout_seconds": 10
}
Authentication — Two options:
  • "auth": { "type": "bearer", "token": "..." } sends an Authorization: Bearer <token> header
  • "auth": { "type": "hmac", "secret": "..." } sends an X-Signature-256 header (HMAC-SHA256 of the request body)
Secrets are write-only. The API never returns them. Fire-and-forget — Set "wait_for_response": false when you don’t need the result back in the conversation, like logging a call event to your CRM.

System-triggered webhooks

By default, webhook tools are "source": "contextual" — the LLM calls them during conversation. Set "source": "system" to trigger them automatically on specific events instead.
{
  "type": "webhook",
  "id": "a1b2c3d4-5678-9abc-def0-123456789abc",
  "name": "post_call_summary",
  "description": "Send a call summary to the CRM after the call ends",
  "url": "https://api.yourcompany.com/call-summaries",
  "parameters": { "type": "object", "properties": {} },
  "source": "system",
  "wait_for_response": false,
  "system": {
    "triggers": [{ "event": "call_end" }],
    "arguments": [
      {
        "name": "call_id",
        "type": "string",
        "required": true,
        "source": { "type": "call_id" }
      },
      {
        "name": "transcript",
        "type": "transcript_messages",
        "required": true,
        "source": { "type": "transcript_messages", "max_messages": 200 }
      }
    ]
  }
}
Available trigger events: call_start, first_user_message, call_end, tool_succeeded, tool_failed. For tool_succeeded and tool_failed, you also need to set source_tool_id to specify which tool’s outcome fires the trigger. Arguments can pull from several sources:
Source typeWhat it provides
constantA fixed value you define
transcript_messagesRecent conversation transcript (bounded by max_messages / max_chars)
call_id, room_name, job_idIDs from the current session
agent_id, agent_nameThe agent’s own metadata
phone_numberCaller’s phone number
first_user_messageThe first thing the caller said
call_end_reasonWhy the call ended

Human transfer

Transfers the caller to a phone number. Requires an outbound SIP trunk on the agent.
{
  "type": "human_transfer",
  "id": "b2c3d4e5-6789-abcd-ef01-234567890abc",
  "name": "transfer_to_support",
  "description": "Transfer the caller to a live support agent",
  "phone_number": "+14155559999"
}
The LLM uses the name and description to decide when to transfer, just like webhook tools.

Template variables

Use {{variable_name}} anywhere in the system_prompt or greeting to inject values at runtime. There are two layers:
  1. template_defaults — Default values set on the agent itself. These apply to every call unless overridden.
  2. arguments — Per-call overrides passed when dispatching a call.
// On the agent
{
  "system_prompt": "You are an outreach coordinator for {{practice_name}}. You are calling {{patient_name}}.",
  "greeting": "Hi, is this {{patient_name}}?",
  "template_defaults": {
    "practice_name": "Greenfield Family Medicine",
    "patient_name": "there"
  }
}
// When dispatching a call — overrides template_defaults
{
  "phone_number": "+14155551234",
  "arguments": {
    "patient_name": "Maria"
  }
}
In this example, patient_name resolves to “Maria” (from the call arguments) and practice_name resolves to “Greenfield Family Medicine” (from the agent defaults). The API response includes a template_variables field that lists every {{variable}} found in the prompt, along with whether it has a default value. This is useful for validating your templates before dispatching calls.

Next steps