Conversational AI

Audio-based conversational AI for interviews and free-form conversations. Real-time voice conversations via WebRTC audio.

Privacy-first design

CanaryCoders AI stores only template configuration and session metadata (duration, status, cost). No transcripts, participant data, or conversation content is stored. All content data is relayed to your own webhook URL.

How it works

Create a conversation template with system prompt, voice settings, and optional interview questions
CanaryCoders AI provisions the voice agent
Start a session to get a signed URL for WebRTC connection
Connect from the browser using the CanaryCoders AI client SDK
During the conversation, CanaryCoders AI webhooks relay data to your webhook URL

Templates

Create Template

bash

curl -X POST /api/convagents/templates \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Customer Interview",
    "type": "interview",
    "systemPrompt": "You are a friendly interviewer conducting a customer feedback session.",
    "firstMessage": "Hello! Thank you for joining. I have a few questions for you.",
    "voiceId": "JBFqnCBsd6RMkjVDRZzb",
    "voiceSettings": { "stability": 0.3, "speed": 1.0, "similarityBoost": 0.8 },
    "language": "en",
    "maxDurationSeconds": 600,
    "tag": "my_project",
    "clientWebhookUrl": "https://your-app.com/webhook/conversation",
    "webhookSecret": "your-secret-key",
    "tools": [
      { "name": "end_call" },
      { "name": "language_detection", "description": "Switch language when user speaks another language" }
    ],
    "questions": [
      { "question": "How did you hear about our product?", "isRequired": true },
      { "question": "What feature do you use the most?", "context": "Focus on daily usage" },
      { "question": "Any suggestions for improvement?", "isRequired": false }
    ]
  }'

Parameter	Type	Required	Description
`name`	string	Yes	Template name
`type`	string	Yes	`"interview"` or `"conversation"`
`systemPrompt`	string	Yes	System prompt for the AI agent
`firstMessage`	string	No	Initial greeting message
`voiceId`	string	No	Voice ID (see Voice Browser)
`voiceModel`	string	No	Voice model
`voiceSettings`	object	No	TTS voice tuning: `stability` (0–1, lower = more expressive, renders audio tags like `[laughs]` stronger), `speed` (0.7–1.2), `similarityBoost` (0–1)
`language`	string	No	Language code (default: `en`)
`llmProvider`	string	No	AI provider (e.g. `anthropic`, `openai`). Required with `llmModel`
`llmModel`	string	No	AI model (e.g. `claude-sonnet-4-5`). Required with `llmProvider`
`tag`	string	No	Custom tag for grouping usage/billing (max 100 chars)
`clientWebhookUrl`	string	No	Your webhook URL for data relay. Must be a public http(s) URL; URLs that resolve to private or internal addresses are rejected
`webhookSecret`	string	No	Secret for webhook auth (sent as `X-Webhook-Secret`). Auto-generated and returned in the create response if omitted. Webhook calls without it get a 401, so store it
`maxDurationSeconds`	number	No	Max duration in seconds (default: 600)
`tools`	array	No	System tools the agent can use. Default: `[{name: "end_call"}]`. Available: `end_call`, `language_detection`, `transfer_to_agent`, `transfer_to_number`, `skip_turn`, `play_keypad_touch_tone`, `voicemail_detection`
`questions`	array	No	Interview questions (for type `"interview"`)
`aiDisclosure`	boolean	No	Default `true`: the agent tells participants at the start that they are talking to an AI, as the EU AI Act (Art. 50) requires. If you set this to `false`, informing your participants becomes your responsibility as deployer

AI transparency (EU AI Act)

From 2 August 2026, Article 50 of the EU AI Act requires that people talking to an AI system are told so. Agents created through this API disclose this by default at the start of every conversation, and never claim to be human. Keep the default unless you handle the disclosure yourself (for example in your own intro flow); disabling it via aiDisclosure: false shifts the legal duty to you.

Other Template Endpoints

Method	Endpoint	Description
`GET`	`/api/convagents/templates`	List all templates
`GET`	`/api/convagents/templates/:id`	Get template details
`PUT`	`/api/convagents/templates/:id`	Update template (auto-syncs voice agent)
`DELETE`	`/api/convagents/templates/:id`	Delete template (removes voice agent)

Sessions

Start a Session

Create a session to get a signed URL for WebRTC connection. The signed URL expires after 15 minutes.

bash

curl -X POST /api/convagents/sessions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "templateId": 1 }'

Response

json

{
  "success": true,
  "data": {
    "session": {
      "id": 1,
      "templateId": 1,
      "status": "pending",
      "createdAt": "2026-02-11T12:00:00.000Z"
    },
    "signedUrl": "wss://...",
    "expiresIn": 900
  }
}

Other Session Endpoints

Method	Endpoint	Description
`GET`	`/api/convagents/sessions`	List sessions (metadata only)
`GET`	`/api/convagents/sessions/:id`	Get session details (no transcript)
`GET`	`/api/convagents/sessions/:id/audio`	Fetch the session recording (single use: deleted from ElevenLabs on retrieval, else purged after 30 days)

Webhook Relay

During a conversation, CanaryCoders AI relays events to your clientWebhookUrl with the X-Webhook-Secret header.

Webhook authentication

Webhook endpoints require the X-Webhook-Secret header matching the template's webhookSecret. Requests without a valid secret receive a 401 response.

Webhook Payload

json

{
  "event": "event | context | complete",
  "sessionId": 1,
  "templateId": 1,
  "templateName": "Customer Interview",
  "timestamp": "2026-02-11T12:05:00.000Z",
  "data": {
    // Event-specific data
  }
}

Webhook	Endpoint	Description
`context`	`POST /api/convagents/webhook/context`	Agent requests context. Your response is forwarded back.
`event`	`POST /api/convagents/webhook/event`	Mid-conversation events (responses, sections completed)
`complete`	`POST /api/convagents/webhook/complete`	Conversation ended. Session status updated, transcript relayed.

Privacy & data handling

When a conversation ends, ElevenLabs sends the full transcript and post-call analysis to CanaryCoders AI via webhook. CanaryCoders AI relays that payload to the template's clientWebhookUrl and immediately discards it. No transcript or conversation content is written to CanaryCoders AI's database. Only session metadata (status, duration, cost) is persisted.

Authentication: every relay call includes the per-template X-Webhook-Secret header. Your endpoint must be HTTPS and must validate this header before processing the payload.
Voice processing: ElevenLabs (US, SCCs) handles the live voice conversation. See the Data Residency section in Providers & Models for the full transfer basis.

No retry, no recovery

The relay is fire-and-forget: there is no retry queue and no dead-letter store. If your webhook endpoint is unreachable when a conversation completes, the transcript is not delivered and cannot be retrieved from CanaryCoders AI (it was never stored here). Treat your webhook endpoint as the system of record and make it reliable.

Pricing

Conversational AI sessions are billed per minute of conversation. When a session completes, the cost is automatically calculated based on duration and added to the standard billing pipeline.

Component	Rate	Details
`Billing`	Per minute	Automatically calculated when session completes

Session costs appear in your usage data: GET /api/llm/usage and GET /api/portal/usage/daily.

Client-Side Integration

Use the @11labs/client SDK (or @11labs/react for React) to connect from the browser.

typescript

import { Conversation } from '@11labs/client';

// 1. Get signed URL from CanaryCoders AI
const res = await fetch('/api/convagents/sessions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ templateId: 1 }),
});
const { data } = await res.json();

// 2. Start WebRTC conversation
// Pass data.dynamicVariables so the post-call webhook can match this
// exact session. Without it, correlation falls back to a time-based guess.
const conversation = await Conversation.startSession({
  signedUrl: data.signedUrl,
  dynamicVariables: data.dynamicVariables,
  onMessage: (msg) => console.log('Agent:', msg),
  onDisconnect: () => console.log('Conversation ended'),
});

// 3. End conversation when done
await conversation.endSession();