asyncapi: 3.0.0 info: title: SLNG Gateway API - Soniox STT version: 0.1.0 description: SLNG Gateway API contact: name: SLNG Support url: https://slng.ai email: support@slng.ai license: name: Proprietary tags: - name: STT description: Speech-to-Text services - name: Unmute STT Bridge description: SLNG native WebSocket STT protocol bridge. - name: Unmute TTS Bridge description: SLNG native WebSocket TTS protocol bridge. servers: production: host: api.slng.ai protocol: wss description: Production security: - $ref: "#/components/securitySchemes/bearer" staging: host: stageapi.slng.ai protocol: wss description: Staging security: - $ref: "#/components/securitySchemes/bearer" channels: /v1/stt/soniox/speech-ai:rt-v5: address: /v1/stt/soniox/speech-ai:rt-v5 title: Speech AI Real-time v5 summary: Speech AI Real-time v5 description: Stream real-time transcripts from Soniox Speech AI v5 over WebSocket with speaker diarization, language detection, and configurable endpoint detection. tags: - name: Soniox Speech AI Real-time v5 servers: - $ref: "#/servers/production" - $ref: "#/servers/staging" messages: SonioxSttInitRequest: $ref: "#/components/messages/SonioxSttInitRequest" SonioxSttBinaryAudio: $ref: "#/components/messages/SonioxSttBinaryAudio" SttFinalizeMessage: $ref: "#/components/messages/SttFinalizeMessage" KeepaliveMessage: $ref: "#/components/messages/KeepaliveMessage" SttCloseMessage: $ref: "#/components/messages/SttCloseMessage" SonioxSttTokensFrame: $ref: "#/components/messages/SonioxSttTokensFrame" SonioxSttErrorFrame: $ref: "#/components/messages/SonioxSttErrorFrame" bindings: ws: method: GET headers: $ref: "#/components/schemas/WebsocketHeadersSoniox" operations: sttSonioxSpeechAiRtV5ReceiveInit: action: receive channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Initialize session on Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/SonioxSttInitRequest" description: Initialize session on Soniox Speech AI Real-time v5 tags: [] sttSonioxSpeechAiRtV5ReceiveAudio: action: receive channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Send audio frame to Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/SonioxSttBinaryAudio" description: Send audio frame to Soniox Speech AI Real-time v5 tags: [] sttSonioxSpeechAiRtV5ReceiveFinalize: action: receive channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Force-finalize buffered audio on Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/SttFinalizeMessage" description: Force-finalize buffered audio on Soniox Speech AI Real-time v5. Gateway forwards as Soniox native `finalize` — server responds with finalized tokens terminated by a `` marker. tags: [] sttSonioxSpeechAiRtV5ReceiveKeepalive: action: receive channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Send keepalive on Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/KeepaliveMessage" description: Send keepalive on Soniox Speech AI Real-time v5. Gateway forwards as Soniox native `keepalive` to maintain the session during silence. tags: [] sttSonioxSpeechAiRtV5ReceiveClose: action: receive channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Close audio stream on Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/SttCloseMessage" description: Close audio stream on Soniox Speech AI Real-time v5 tags: [] sttSonioxSpeechAiRtV5SendTokensFrame: action: send channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Receive native Soniox tokens frame from Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/SonioxSttTokensFrame" description: | Receive a native Soniox tokens frame. Forwarded unchanged from Soniox — no SLNG normalization. Includes the initial warmup heartbeat (empty `tokens` array), interim frames (tokens with `is_final: false`), and final frames (tokens with `is_final: true`, optionally terminated by an `` / `` endpoint marker when `enable_endpoint_detection` is enabled). tags: [] sttSonioxSpeechAiRtV5SendErrorFrame: action: send channel: $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5" summary: Receive native Soniox error frame from Speech AI Real-time v5 messages: - $ref: "#/channels/~1v1~1stt~1soniox~1speech-ai:rt-v5/messages/SonioxSttErrorFrame" description: Receive a native Soniox error frame. Forwarded unchanged from Soniox — discriminated by presence of `error_code`. tags: [] components: schemas: WebsocketHeadersSoniox: type: object properties: X-World-Part-Override: type: string description: "Target world part override. Auto-selected if not provided. Available world parts: `ap`, `eu`, `na`." enum: - ap - eu - na messages: SonioxSttTokensFrame: name: SonioxSttTokensFrame title: Tokens Frame (Soniox native) summary: Native Soniox tokens frame forwarded unchanged from the upstream provider. contentType: application/json payload: type: object description: "Native Soniox Speech AI server frame forwarded unchanged, discriminated by presence of the `tokens` array. Three variants share this shape: a warmup heartbeat emitted right after session init before any audio is transcribed (empty `tokens`, both processing timers `0`); interim frames with `is_final: false`; and final frames with `is_final: true`, optionally terminated by an `` or `` endpoint marker token when `enable_endpoint_detection` is enabled." required: - tokens properties: tokens: type: array description: Transcription tokens with per-token timing, confidence, speaker and language. Tokens are sub-word chunks (e.g. `" co"`, `"uld"`) rather than whole words — concatenate `text` across tokens to reconstruct the transcript. Empty on the initial warmup frame. items: type: object properties: text: type: string description: Token text. Endpoint markers `` / `` may appear here when endpoint detection is enabled. start_ms: type: number description: Token start time in milliseconds. end_ms: type: number description: Token end time in milliseconds. confidence: type: number description: Token confidence score. is_final: type: boolean description: Whether this token is finalized. speaker: type: string description: Speaker identifier (when diarization is enabled). language: type: string description: ISO 639-1 two-letter language code for this token (when language identification or translation is enabled). translation_status: type: string enum: - none - original - translation description: Translation state of the token. `none` when translation is disabled or the token is not to be translated; `original` for source-language tokens; `translation` for translated tokens (accompanied by `source_language`). source_language: type: string description: "ISO 639-1 code of the source language. Only present on tokens with `translation_status: translation`." final_audio_proc_ms: type: number description: Cumulative audio processing time for finalized tokens, in milliseconds. total_audio_proc_ms: type: number description: Total audio processing time for the session, in milliseconds. examples: - name: sonioxWarmup summary: Warmup heartbeat — sent right after init, before any audio is transcribed payload: tokens: [] final_audio_proc_ms: 0 total_audio_proc_ms: 0 - name: sonioxInterim summary: Interim frame with sub-word non-final tokens (tokens stream as short chunks, not whole words) payload: tokens: - text: " U" start_ms: 18960 end_ms: 19020 confidence: 0.851 is_final: false speaker: "1" - text: m, start_ms: 19080 end_ms: 19140 confidence: 0.999 is_final: false speaker: "1" - text: " co" start_ms: 19200 end_ms: 19260 confidence: 1 is_final: false speaker: "1" - text: uld start_ms: 19260 end_ms: 19320 confidence: 1 is_final: false speaker: "1" - text: " y" start_ms: 19320 end_ms: 19380 confidence: 1 is_final: false speaker: "1" final_audio_proc_ms: 18120 total_audio_proc_ms: 19920 - name: sonioxFinalWithEndpoint summary: Final frame with endpoint marker (``) — tokens finalized, utterance boundary reached payload: tokens: - text: " U" start_ms: 18960 end_ms: 19020 confidence: 0.97 is_final: true speaker: "1" - text: m, start_ms: 19080 end_ms: 19140 confidence: 0.99 is_final: true speaker: "1" - text: " co" start_ms: 19200 end_ms: 19260 confidence: 1 is_final: true speaker: "1" - text: uld start_ms: 19260 end_ms: 19320 confidence: 1 is_final: true speaker: "1" - text: " y" start_ms: 19320 end_ms: 19380 confidence: 1 is_final: true speaker: "1" - text: ou? start_ms: 19380 end_ms: 19560 confidence: 0.98 is_final: true speaker: "1" - text: start_ms: 19560 end_ms: 19560 confidence: 1 is_final: true final_audio_proc_ms: 19920 total_audio_proc_ms: 19920 - name: sonioxDiarization summary: Final frame with speaker diarization — two speakers identified in one utterance payload: tokens: - text: " Hi" start_ms: 18960 end_ms: 19080 confidence: 0.96 is_final: true speaker: "1" - text: " A" start_ms: 19080 end_ms: 19140 confidence: 0.92 is_final: true speaker: "1" - text: lice. start_ms: 19140 end_ms: 19400 confidence: 0.94 is_final: true speaker: "1" - text: " He" start_ms: 19700 end_ms: 19820 confidence: 0.95 is_final: true speaker: "2" - text: llo start_ms: 19820 end_ms: 19960 confidence: 0.97 is_final: true speaker: "2" - text: " B" start_ms: 19960 end_ms: 20020 confidence: 0.93 is_final: true speaker: "2" - text: ob. start_ms: 20020 end_ms: 20180 confidence: 0.95 is_final: true speaker: "2" final_audio_proc_ms: 20200 total_audio_proc_ms: 20200 - name: sonioxTranslation summary: Final frame with one-way translation — English source tokens followed by Spanish translations payload: tokens: - text: " Hel" start_ms: 18960 end_ms: 19080 confidence: 0.97 is_final: true speaker: "1" language: en translation_status: original - text: lo start_ms: 19080 end_ms: 19200 confidence: 0.98 is_final: true speaker: "1" language: en translation_status: original - text: " world" start_ms: 19200 end_ms: 19560 confidence: 0.95 is_final: true speaker: "1" language: en translation_status: original - text: " Ho" is_final: true language: es source_language: en translation_status: translation - text: la is_final: true language: es source_language: en translation_status: translation - text: " mundo" is_final: true language: es source_language: en translation_status: translation final_audio_proc_ms: 19560 total_audio_proc_ms: 19560 SonioxSttErrorFrame: name: SonioxSttErrorFrame title: Error Frame (Soniox native) summary: Native Soniox error frame forwarded unchanged from the upstream provider. contentType: application/json payload: type: object description: Native Soniox error frame. Discriminated by presence of `error_code`. required: - error_code - error_message properties: error_code: type: integer description: Soniox numeric error code. error_message: type: string description: Human-readable error description from Soniox. examples: - name: sonioxError summary: Native Soniox error frame payload: error_code: 401 error_message: Invalid or expired API key SttFinalizeMessage: name: SttFinalizeMessage title: Finalize Message summary: Force-finalize buffered audio tokens without closing the connection. contentType: application/json payload: type: object description: Mid-stream flush — forces Soniox to finalize any buffered audio and return results. The connection remains open for further audio. Server responds with finalized tokens terminated by a `` marker. required: - type properties: type: type: string const: finalize examples: - name: finalize summary: Force-finalize buffered audio payload: type: finalize KeepaliveMessage: name: KeepaliveMessage title: Keepalive Message summary: Keep the WebSocket connection alive during silence. contentType: application/json payload: type: object description: Prevents the connection from being closed due to inactivity. Send at least every 20 seconds when not streaming audio to maintain the session. required: - type properties: type: type: string const: keepalive examples: - name: keepalive summary: Send keepalive payload: type: keepalive SttCloseMessage: name: SttCloseMessage title: Close Message summary: Signal end of audio stream. contentType: application/json payload: type: object description: "Signal end-of-stream. The gateway translates `{\"type\": \"\"}` into an empty binary WebSocket frame forwarded to Soniox — Soniox's native end-of-stream signal per its docs. Alternatively, simply close the WebSocket directly from the client." required: - type properties: type: type: string const: "" description: Empty string. Gateway maps this to an empty binary frame. examples: - name: close summary: Close the audio stream payload: type: "" SonioxSttBinaryAudio: name: SonioxSttBinaryAudio title: Binary Audio Frame summary: Stream raw binary audio frames to be transcribed. Sent as binary WebSocket frames (NOT JSON) — no envelope, no base64 encoding. Frame format must match the `audio_format`, `sample_rate`, and `num_channels` declared in the init message (default `pcm_s16le` at 16kHz mono). contentType: application/octet-stream payload: type: string format: binary description: Raw binary audio data matching the declared `audio_format`. Sent as a binary WebSocket frame — no JSON envelope, no base64 encoding. examples: - name: binaryAudioFrame summary: Raw PCM audio frame (~256ms of 16kHz pcm_s16le speech) payload: 0000FF00000000FF00000000010101010101010100000000FFFFFFFFFFFFFEFEFDFEFEFEFEFDFDFEFEFEFEFEFEFEFEFEFFFFFFFFFEFEFEFEFF000100001020303030303030303030201010000FFFFFEFDFDFDFDFEFFFFFFFF0001020303020201000000FFFDFCFBFAFAFBFAF9F8F7F7F7F6F6F4F2F2F3F7FC000406090F14191A19181715110E0A05FEF9F6F3F0EEECEBEBECEEF2F6F9FC0005090D0F101010100E0C080401 SonioxSttInitRequest: name: SonioxSttInitRequest title: Init Request (Soniox) summary: Initialize a Soniox STT session with provider-specific recognition configuration. contentType: application/json payload: type: object description: "Initialize a Soniox Speech AI STT session with provider-specific configuration options before streaming audio. All configuration fields are sent at the root of the message alongside `type` (not nested under a `config` key); the gateway translates this SLNG-side `config` message into Soniox's native init payload before forwarding, and only the fields listed below are forwarded — unknown fields are dropped. Soniox's `api_key` is injected by the gateway from the `Authorization: Bearer ` header, so do not send it. Full field semantics: https://soniox.com/docs/stt/api-reference/websocket-api" required: - type properties: type: type: string const: config model: type: string description: Real-time model to use. default: stt-rt-v5 audio_format: type: string description: Audio format of the stream. Use `auto` to let Soniox detect container formats. default: pcm_s16le enum: - pcm_s16le - pcm_f32le - wav - ogg - flac - auto sample_rate: type: integer description: Audio sample rate in Hz (required for raw PCM formats). default: 16000 num_channels: type: integer description: Number of audio channels (required for raw PCM formats). default: 1 enable_speaker_diarization: type: boolean description: Enable speaker identification and separation. enable_language_identification: type: boolean description: Automatically detect spoken language. enable_endpoint_detection: type: boolean description: Enable endpoint detection for utterance boundaries. default: true max_endpoint_delay_ms: type: integer description: Max endpoint detection delay in milliseconds (500–3000). Soniox's native default is 2000ms; this endpoint defaults to 500ms for lower latency. minimum: 500 maximum: 3000 default: 500 language_hints: type: array description: Language hints for transcription. items: type: string language_hints_strict: type: boolean description: Only transcribe in specified languages. context: type: object description: Context information for improved accuracy. properties: general: type: array items: type: object properties: key: type: string value: type: string text: type: string terms: type: array items: type: string translation: type: object description: Translation configuration. properties: type: type: string enum: - one_way - two_way target_language: type: string language_a: type: string language_b: type: string client_reference_id: type: string maxLength: 256 description: Client reference ID for tracking. examples: - name: sonioxInit summary: Soniox STT session initialization with diarization payload: type: config model: stt-rt-v5 audio_format: pcm_s16le sample_rate: 16000 num_channels: 1 enable_speaker_diarization: true enable_endpoint_detection: true max_endpoint_delay_ms: 500 language_hints: - en securitySchemes: bearer: type: http scheme: bearer description: | API key issued by SLNG. Pass as `Authorization: Bearer ` in the WebSocket upgrade request headers.