> ## Documentation Index > Fetch the complete documentation index at: https://docs.slng.ai/llms.txt > Use this file to discover all available pages before exploring further. # Speech AI Real-time v4 > Stream real-time transcripts from Soniox Speech AI v4 over WebSocket with speaker diarization, language detection, and configurable endpoint detection. ## AsyncAPI ````yaml api-reference/asyncapi/stt-soniox.asyncapi.yaml /v1/stt/soniox/speech-ai:rt-v4 id: /v1/stt/soniox/speech-ai:rt-v4 title: Speech AI Real-time v4 description: >- Stream real-time transcripts from Soniox Speech AI v4 over WebSocket with speaker diarization, language detection, and configurable endpoint detection. servers: - id: production protocol: wss host: api.slng.ai bindings: [] variables: [] - id: staging protocol: wss host: stageapi.slng.ai bindings: [] variables: [] address: /v1/stt/soniox/speech-ai:rt-v4 parameters: [] bindings: - protocol: ws version: latest value: method: GET headers: type: object properties: X-World-Part-Override: type: string description: >- Target world part override. Auto-selected if not provided. Available world parts: `ap`, `eu`, `na`. enum: - ap - eu - na x-parser-schema-id: x-parser-schema-id: WebsocketHeadersSoniox schemaProperties: - name: method type: string description: GET required: false - name: headers type: object required: false properties: - name: X-World-Part-Override type: string description: >- Target world part override. Auto-selected if not provided. Available world parts: `ap`, `eu`, `na`. enumValues: - ap - eu - na required: false operations: - &ref_1 id: sttSonioxSpeechAiRtV4ReceiveInit title: Stt soniox speech ai rt v4 receive init description: Initialize session on Speech AI Real-time v4 type: receive messages: - &ref_8 id: SonioxSttInitRequest contentType: application/json payload: - name: Init Request (Soniox) description: >- Initialize a Soniox STT session with provider-specific recognition configuration. type: object properties: - name: type type: string description: config required: true - name: model type: string description: Real-time model to use. required: false - name: audio_format type: string description: >- Audio format of the stream. Use `auto` to let Soniox detect container formats. enumValues: - pcm_s16le - pcm_f32le - wav - ogg - flac - auto required: false - name: sample_rate type: integer description: Audio sample rate in Hz (required for raw PCM formats). required: false - name: num_channels type: integer description: Number of audio channels (required for raw PCM formats). required: false - name: enable_speaker_diarization type: boolean description: Enable speaker identification and separation. required: false - name: enable_language_identification type: boolean description: Automatically detect spoken language. required: false - name: enable_endpoint_detection type: boolean description: Enable endpoint detection for utterance boundaries. required: false - name: max_endpoint_delay_ms type: integer description: >- Max endpoint detection delay in milliseconds (500–3000). Soniox's native default is 2000ms; this endpoint defaults to 500ms for lower latency. required: false - name: language_hints type: array description: Language hints for transcription. required: false properties: - name: item type: string required: false - name: language_hints_strict type: boolean description: Only transcribe in specified languages. required: false - name: context type: object description: Context information for improved accuracy. required: false properties: - name: general type: array required: false properties: - name: key type: string required: false - name: value type: string required: false - name: text type: string required: false - name: terms type: array required: false properties: - name: item type: string required: false - name: translation type: object description: Translation configuration. required: false properties: - name: type type: string enumValues: - one_way - two_way required: false - name: target_language type: string required: false - name: language_a type: string required: false - name: language_b type: string required: false - name: client_reference_id type: string description: Client reference ID for tracking. required: false headers: [] jsonPayloadSchema: type: object description: >- Initialize a Soniox Speech AI STT session with provider-specific configuration options before streaming audio. All configuration fields are sent at the root of the message alongside `type` (not nested under a `config` key); the gateway translates this SLNG-side `config` message into Soniox's native init payload before forwarding, and only the fields listed below are forwarded — unknown fields are dropped. Soniox's `api_key` is injected by the gateway from the `Authorization: Bearer ` header, so do not send it. Full field semantics: https://soniox.com/docs/stt/api-reference/websocket-api required: - type properties: type: type: string const: config x-parser-schema-id: model: type: string description: Real-time model to use. default: stt-rt-v4 x-parser-schema-id: audio_format: type: string description: >- Audio format of the stream. Use `auto` to let Soniox detect container formats. default: pcm_s16le enum: - pcm_s16le - pcm_f32le - wav - ogg - flac - auto x-parser-schema-id: sample_rate: type: integer description: Audio sample rate in Hz (required for raw PCM formats). default: 16000 x-parser-schema-id: num_channels: type: integer description: Number of audio channels (required for raw PCM formats). default: 1 x-parser-schema-id: enable_speaker_diarization: type: boolean description: Enable speaker identification and separation. x-parser-schema-id: enable_language_identification: type: boolean description: Automatically detect spoken language. x-parser-schema-id: enable_endpoint_detection: type: boolean description: Enable endpoint detection for utterance boundaries. default: true x-parser-schema-id: max_endpoint_delay_ms: type: integer description: >- Max endpoint detection delay in milliseconds (500–3000). Soniox's native default is 2000ms; this endpoint defaults to 500ms for lower latency. minimum: 500 maximum: 3000 default: 500 x-parser-schema-id: language_hints: type: array description: Language hints for transcription. items: type: string x-parser-schema-id: x-parser-schema-id: language_hints_strict: type: boolean description: Only transcribe in specified languages. x-parser-schema-id: context: type: object description: Context information for improved accuracy. properties: general: type: array items: type: object properties: key: type: string x-parser-schema-id: value: type: string x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: text: type: string x-parser-schema-id: terms: type: array items: type: string x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: translation: type: object description: Translation configuration. properties: type: type: string enum: - one_way - two_way x-parser-schema-id: target_language: type: string x-parser-schema-id: language_a: type: string x-parser-schema-id: language_b: type: string x-parser-schema-id: x-parser-schema-id: client_reference_id: type: string maxLength: 256 description: Client reference ID for tracking. x-parser-schema-id: x-parser-schema-id: title: Init Request (Soniox) description: >- Initialize a Soniox STT session with provider-specific recognition configuration. example: |- { "type": "config", "model": "stt-rt-v4", "audio_format": "pcm_s16le", "sample_rate": 16000, "num_channels": 1, "enable_speaker_diarization": true, "enable_endpoint_detection": true, "max_endpoint_delay_ms": 500, "language_hints": [ "en" ] } bindings: [] extensions: - id: x-parser-unique-object-id value: SonioxSttInitRequest bindings: [] extensions: &ref_0 - id: x-parser-unique-object-id value: /v1/stt/soniox/speech-ai:rt-v4 - &ref_2 id: sttSonioxSpeechAiRtV4ReceiveAudio title: Stt soniox speech ai rt v4 receive audio description: Send audio frame to Speech AI Real-time v4 type: receive messages: - &ref_9 id: SonioxSttBinaryAudio contentType: application/octet-stream payload: - type: string format: binary description: >- Stream raw binary audio frames to be transcribed. Sent as binary WebSocket frames (NOT JSON) — no envelope, no base64 encoding. Frame format must match the `audio_format`, `sample_rate`, and `num_channels` declared in the init message (default `pcm_s16le` at 16kHz mono). x-parser-schema-id: name: Binary Audio Frame headers: [] jsonPayloadSchema: type: string format: binary description: >- Raw binary audio data matching the declared `audio_format`. Sent as a binary WebSocket frame — no JSON envelope, no base64 encoding. x-parser-schema-id: title: Binary Audio Frame description: >- Stream raw binary audio frames to be transcribed. Sent as binary WebSocket frames (NOT JSON) — no envelope, no base64 encoding. Frame format must match the `audio_format`, `sample_rate`, and `num_channels` declared in the init message (default `pcm_s16le` at 16kHz mono). example: >- "0000FF00000000FF00000000010101010101010100000000FFFFFFFFFFFFFEFEFDFEFEFEFEFDFDFEFEFEFEFEFEFEFEFEFFFFFFFFFEFEFEFEFF000100001020303030303030303030201010000FFFFFEFDFDFDFDFEFFFFFFFF0001020303020201000000FFFDFCFBFAFAFBFAF9F8F7F7F7F6F6F4F2F2F3F7FC000406090F14191A19181715110E0A05FEF9F6F3F0EEECEBEBECEEF2F6F9FC0005090D0F101010100E0C080401" bindings: [] extensions: - id: x-parser-unique-object-id value: SonioxSttBinaryAudio bindings: [] extensions: *ref_0 - &ref_3 id: sttSonioxSpeechAiRtV4ReceiveFinalize title: Stt soniox speech ai rt v4 receive finalize description: Force-finalize buffered audio on Speech AI Real-time v4 type: receive messages: - &ref_10 id: SttFinalizeMessage contentType: application/json payload: - name: Finalize Message description: >- Force-finalize buffered audio tokens without closing the connection. type: object properties: - name: type type: string description: finalize required: true headers: [] jsonPayloadSchema: type: object description: >- Mid-stream flush — forces Soniox to finalize any buffered audio and return results. The connection remains open for further audio. Server responds with finalized tokens terminated by a `` marker. required: - type properties: type: type: string const: finalize x-parser-schema-id: x-parser-schema-id: title: Finalize Message description: Force-finalize buffered audio tokens without closing the connection. example: |- { "type": "finalize" } bindings: [] extensions: - id: x-parser-unique-object-id value: SttFinalizeMessage bindings: [] extensions: *ref_0 - &ref_4 id: sttSonioxSpeechAiRtV4ReceiveKeepalive title: Stt soniox speech ai rt v4 receive keepalive description: Send keepalive on Speech AI Real-time v4 type: receive messages: - &ref_11 id: KeepaliveMessage contentType: application/json payload: - name: Keepalive Message description: Keep the WebSocket connection alive during silence. type: object properties: - name: type type: string description: keepalive required: true headers: [] jsonPayloadSchema: type: object description: >- Prevents the connection from being closed due to inactivity. Send at least every 20 seconds when not streaming audio to maintain the session. required: - type properties: type: type: string const: keepalive x-parser-schema-id: x-parser-schema-id: title: Keepalive Message description: Keep the WebSocket connection alive during silence. example: |- { "type": "keepalive" } bindings: [] extensions: - id: x-parser-unique-object-id value: KeepaliveMessage bindings: [] extensions: *ref_0 - &ref_5 id: sttSonioxSpeechAiRtV4ReceiveClose title: Stt soniox speech ai rt v4 receive close description: Close audio stream on Speech AI Real-time v4 type: receive messages: - &ref_12 id: SttCloseMessage contentType: application/json payload: - name: Close Message description: Signal end of audio stream. type: object properties: - name: type type: string description: Empty string. Gateway maps this to an empty binary frame. required: true headers: [] jsonPayloadSchema: type: object description: >- Signal end-of-stream. The gateway translates `{"type": ""}` into an empty binary WebSocket frame forwarded to Soniox — Soniox's native end-of-stream signal per its docs. Alternatively, simply close the WebSocket directly from the client. required: - type properties: type: type: string const: '' description: Empty string. Gateway maps this to an empty binary frame. x-parser-schema-id: x-parser-schema-id: title: Close Message description: Signal end of audio stream. example: |- { "type": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: SttCloseMessage bindings: [] extensions: *ref_0 - &ref_6 id: sttSonioxSpeechAiRtV4SendTokensFrame title: Stt soniox speech ai rt v4 send tokens frame description: Receive native Soniox tokens frame from Speech AI Real-time v4 type: send messages: - &ref_13 id: SonioxSttTokensFrame contentType: application/json payload: - name: Tokens Frame (Soniox native) description: >- Native Soniox tokens frame forwarded unchanged from the upstream provider. type: object properties: - name: tokens type: array description: >- Transcription tokens with per-token timing, confidence, speaker and language. Tokens are sub-word chunks (e.g. `" co"`, `"uld"`) rather than whole words — concatenate `text` across tokens to reconstruct the transcript. Empty on the initial warmup frame. required: true properties: - name: text type: string description: >- Token text. Endpoint markers `` / `` may appear here when endpoint detection is enabled. required: false - name: start_ms type: number description: Token start time in milliseconds. required: false - name: end_ms type: number description: Token end time in milliseconds. required: false - name: confidence type: number description: Token confidence score. required: false - name: is_final type: boolean description: Whether this token is finalized. required: false - name: speaker type: string description: Speaker identifier (when diarization is enabled). required: false - name: language type: string description: >- ISO 639-1 two-letter language code for this token (when language identification or translation is enabled). required: false - name: translation_status type: string description: >- Translation state of the token. `none` when translation is disabled or the token is not to be translated; `original` for source-language tokens; `translation` for translated tokens (accompanied by `source_language`). enumValues: - none - original - translation required: false - name: source_language type: string description: >- ISO 639-1 code of the source language. Only present on tokens with `translation_status: translation`. required: false - name: final_audio_proc_ms type: number description: >- Cumulative audio processing time for finalized tokens, in milliseconds. required: false - name: total_audio_proc_ms type: number description: Total audio processing time for the session, in milliseconds. required: false headers: [] jsonPayloadSchema: type: object description: >- Native Soniox Speech AI server frame forwarded unchanged, discriminated by presence of the `tokens` array. Three variants share this shape: a warmup heartbeat emitted right after session init before any audio is transcribed (empty `tokens`, both processing timers `0`); interim frames with `is_final: false`; and final frames with `is_final: true`, optionally terminated by an `` or `` endpoint marker token when `enable_endpoint_detection` is enabled. required: - tokens properties: tokens: type: array description: >- Transcription tokens with per-token timing, confidence, speaker and language. Tokens are sub-word chunks (e.g. `" co"`, `"uld"`) rather than whole words — concatenate `text` across tokens to reconstruct the transcript. Empty on the initial warmup frame. items: type: object properties: text: type: string description: >- Token text. Endpoint markers `` / `` may appear here when endpoint detection is enabled. x-parser-schema-id: start_ms: type: number description: Token start time in milliseconds. x-parser-schema-id: end_ms: type: number description: Token end time in milliseconds. x-parser-schema-id: confidence: type: number description: Token confidence score. x-parser-schema-id: is_final: type: boolean description: Whether this token is finalized. x-parser-schema-id: speaker: type: string description: Speaker identifier (when diarization is enabled). x-parser-schema-id: language: type: string description: >- ISO 639-1 two-letter language code for this token (when language identification or translation is enabled). x-parser-schema-id: translation_status: type: string enum: - none - original - translation description: >- Translation state of the token. `none` when translation is disabled or the token is not to be translated; `original` for source-language tokens; `translation` for translated tokens (accompanied by `source_language`). x-parser-schema-id: source_language: type: string description: >- ISO 639-1 code of the source language. Only present on tokens with `translation_status: translation`. x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: final_audio_proc_ms: type: number description: >- Cumulative audio processing time for finalized tokens, in milliseconds. x-parser-schema-id: total_audio_proc_ms: type: number description: Total audio processing time for the session, in milliseconds. x-parser-schema-id: x-parser-schema-id: title: Tokens Frame (Soniox native) description: >- Native Soniox tokens frame forwarded unchanged from the upstream provider. example: |- { "tokens": [], "final_audio_proc_ms": 0, "total_audio_proc_ms": 0 } bindings: [] extensions: - id: x-parser-unique-object-id value: SonioxSttTokensFrame bindings: [] extensions: *ref_0 - &ref_7 id: sttSonioxSpeechAiRtV4SendErrorFrame title: Stt soniox speech ai rt v4 send error frame description: Receive native Soniox error frame from Speech AI Real-time v4 type: send messages: - &ref_14 id: SonioxSttErrorFrame contentType: application/json payload: - name: Error Frame (Soniox native) description: >- Native Soniox error frame forwarded unchanged from the upstream provider. type: object properties: - name: error_code type: integer description: Soniox numeric error code. required: true - name: error_message type: string description: Human-readable error description from Soniox. required: true headers: [] jsonPayloadSchema: type: object description: >- Native Soniox error frame. Discriminated by presence of `error_code`. required: - error_code - error_message properties: error_code: type: integer description: Soniox numeric error code. x-parser-schema-id: error_message: type: string description: Human-readable error description from Soniox. x-parser-schema-id: x-parser-schema-id: title: Error Frame (Soniox native) description: >- Native Soniox error frame forwarded unchanged from the upstream provider. example: |- { "error_code": 401, "error_message": "Invalid or expired API key" } bindings: [] extensions: - id: x-parser-unique-object-id value: SonioxSttErrorFrame bindings: [] extensions: *ref_0 sendOperations: - *ref_1 - *ref_2 - *ref_3 - *ref_4 - *ref_5 receiveOperations: - *ref_6 - *ref_7 sendMessages: - *ref_8 - *ref_9 - *ref_10 - *ref_11 - *ref_12 receiveMessages: - *ref_13 - *ref_14 extensions: - id: x-parser-unique-object-id value: /v1/stt/soniox/speech-ai:rt-v4 securitySchemes: - id: bearer name: bearer type: http description: > API key issued by SLNG. Pass as `Authorization: Bearer ` in the WebSocket upgrade request headers. scheme: bearer extensions: [] ````