Integration guide

The Context Router is a single endpoint that speaks the OpenAI Chat Completions protocol. Point an OpenAI-compatible client at it, authenticate with your SLNG API key, and every turn gets routing, caching, and in-region execution without changing how you call the model.

Before you start

Three things decide whether your first request works:

The right key. Create the key inside a project from Improve my agent, on its Project Vault > SLNG API key page. A models-only key from Call TTS/STT models API returns 403 project_mode_forbidden.
The project’s region. The key works only in the region the project was created in. Use that region’s code in the base URL, https://{region}.context-router.slng.ai/v1, and in your speech base URL.
An agent ID and a session ID on every request. See Required fields.

Set up a project for your own code walks through all three, and where to choose the router’s model. For a LiveKit or Pipecat agent, start from the LiveKit or Pipecat guide, or the SLNG templates.

Endpoint and authentication

The router exposes two OpenAI-compatible surfaces. Both share the same auth, routing, caching, and features.

POST <your-base-url>/chat/completions
Authorization: Bearer <SLNG_API_KEY>
Content-Type: application/json

POST <your-base-url>/responses
Authorization: Bearer <SLNG_API_KEY>
Content-Type: application/json

The /responses endpoint is a translation layer over the same router: identical auth, routing, caching, and features. Everything in this guide applies to both surfaces unless a section says otherwise.

Authenticate with only your SLNG_API_KEY, passed as a Bearer token. The OpenAI SDK’s api_key parameter sets this header for you, so you never set internal auth headers yourself.

from openai import OpenAI

client = OpenAI(
    # Regional base URL: https://{region}.context-router.slng.ai/v1
    base_url="https://in.context-router.slng.ai/v1",
    api_key="YOUR_SLNG_API_KEY",
)

Making a request

A request is a standard OpenAI Chat Completions call.

resp = client.chat.completions.create(
    model="slng/auto",
    messages=[
        {"role": "system", "content": "You are a helpful clinic assistant."},
        {"role": "user", "content": "What are your opening hours?"},
    ],
)
print(resp.choices[0].message.content)

Streaming

Streaming works exactly as in OpenAI, for every model the router serves (reasoning and non-reasoning alike).

Always set stream_options.include_usage=True so you receive the final usage trailer chunk.

Streaming request

import openai

stream = client.chat.completions.create(
    model="slng/auto",
    messages=[...],
    stream=True,
    stream_options={"include_usage": True},
)
try:
    for chunk in stream:
        ...  # play or forward the text
except openai.APIError as e:
    # The stream ended with an error chunk; e.code holds its status.
    ...

What to expect on a Chat Completions stream:

Reasoning blocks and control tokens are removed for you. If your model emits a visible reasoning block (<think>, <thought>, <thinking>, <reasoning>) or a stray control token (</s>, <|im_end|> and similar), the router strips it before the chunk reaches you, so your TTS never reads it aloud. Setting slng_pure_proxy suspends this.
On a stream that completes, all answer content arrives before the finish_reason: "stop" chunk, so stopping there is safe.
The usage block rides the final chunk. That is why include_usage matters.
A stream that fails says so. If the model fails once the stream is open, even before its first token, the router cannot change the 200 it already sent, so it ends the stream with one error chunk and then data: [DONE]:
Error chunk
data: {"error": {"message": "Upstream error: ...", "type": "upstream_error", "param": null, "code": 502}} data: [DONE]
That stream carries no finish_reason and no usage block. code is the status you would have received if the request had failed before streaming began: 504 for a timeout, the provider’s own status if it rejected the request (for example 429), and 502 otherwise. A failure inside the router itself ends the stream the same way, with "type": "server_error", "message": "Internal error" and "code": 500, so check for both types. The OpenAI SDKs raise an APIError when they read this chunk (in Python, a plain openai.APIError with the status in e.code), so your client sees an error instead of a short answer that looks complete. Treat the text you already received as partial: a voice agent should not present it as a finished reply. On the /responses endpoint the same failure ends the stream with a response.failed event instead of response.completed.

Ignore unknown fields in your client. Nearly every client already does, including the official OpenAI SDKs. If yours uses strict deserialization (deny_unknown_fields in serde, DisallowUnknownFields in Go, or a hand-written parser that rejects extras), turn it off for this endpoint. The router forwards a provider’s own extra fields verbatim, and some are required on the next turn. On a stream a strict decoder fails quietly: it skips the chunks it cannot parse, so you get a truncated answer with a 200.

Available parameters

Field	What it does
`model`	`slng/auto`, the default when omitted. Send it for a project’s Think model. With an inline `slng_config`, you can instead name one of its models to pin it. With two or more tiers, `slng/auto` picks a tier for each request. See Smart routing.
`messages`	Standard OpenAI messages array. `developer` is treated like `system`.
Sampling params	`temperature`, `top_p`, `max_tokens`, `max_completion_tokens`, and so on.
`stream`	`true` or `false`.
`stream_options.include_usage`	Set `true` to get the usage trailer.
`tools` / `tool_choice`	Standard tool-calling, fully supported. An answer containing tool calls is never cached.
`template_variables`	Per-call personalization values for `{{name}}` placeholders. See Per-call values.
`slng_config`	Your models inline, with their endpoints and provider keys, plus any setting you want to change for this request. See BYOK Context Router.
`slng_agent_id`	Required, unless you send the `X-Slng-Agent-Id` header instead.
`slng_session_id`	Required, unless you send the `X-Slng-Session-Id` header instead.
`slng_analytics`	Default `false`. Keeps a copy of the request and answer for your analytics. See Shadow mode.
`slng_pure_proxy`	Default `false`. Returns your model’s answer untouched. See Shadow mode.

`slng/auto` vs a named model

slng/auto

Recommended. The router answers from your configuration, splitting traffic by weight. With two or more tiers, it picks a tier for each request, and it fails over when a model fails.

A named model

The exact name of one model in the slng_config you send. It pins the request to that model and skips per-turn routing, so there is no failover to another model, and its traffic keeps its own cache entries.

Required fields

Your org identity is managed by SLNG. The agent ID and session ID are yours to send, and both are required on every request.

Agent ID names the logical agent a call belongs to, and scopes its cache. Use a stable value per agent. When you change that agent’s prompt in a meaningful way, give it a fresh ID, for example clinic-scheduler-v2, so answers produced under the old prompt are not reused.
Session ID identifies a single call. Use a value unique to each call, like a UUID, and keep it the same across all turns of that call.

Send each one as a header (X-Slng-Agent-Id, X-Slng-Session-Id) or as a top-level body field (slng_agent_id, slng_session_id). For the value format and the errors a missing or malformed ID returns, see Agent and session headers.

Reading the response

Every request returns a set of x-slng-* headers that tell you how the answer was produced.

Header	Present on	Meaning
`x-slng-request-id`	Every response (including errors).	Quote it in a support request.
`x-slng-response-source`	Every successful (2xx) chat completion.	`llm` or `cache`.
`x-slng-cache-layer`	Cache hits only.	Which cache layer matched.
`x-slng-model`	Live LLM answers only.	The model that answered, after tier selection and any failover.

On an error, only x-slng-request-id comes back, because no answer was produced. Two outcomes are possible on a successful request:

Normal LLM response

x-slng-response-source: llm. The request was routed to the LLM, and usage carries the tokens you were billed for.

Cache hit

x-slng-response-source: cache. No LLM ran. usage carries the original answer’s tokens, so it is what the cache saved rather than what this call cost.

Response caching

The Context Router is a PII-aware service that can serve a repeated turn straight from a cache instead of calling the LLM again. Caching is on by default, and a turn that qualifies is served from cache automatically. Cache layers are checked fastest-first; the first hit wins.

Layer	Match style
`l1_full_input`, `l2_full_input`	Whole-request match.
`l1_exact`, `l2_exact`	Exact match.
`l2_norm_exact`	Normalized-exact match.
`l2_canonical`	Canonical-class match.
`prewarm_full_input`, `prewarm_exact`, `prewarm_norm_exact`, `prewarm_canonical`	The same match styles, over answers SLNG pre-loaded for your agent ahead of time.

Pre-warmed answers are set up together with you, per agent.

What SLNG does not store

So one caller’s answer is never replayed to another, some responses are not stored. The caller still gets their answer; only the caching is skipped.

Any reply containing a number, in any script. A number is usually specific to one caller or record, so identifiers, prices and dates are never cached.
Personal information, such as names, addresses and emails.
Any reply carrying a placeholder the router did not fill in itself.

Per-call values (`template_variables`)

Voice agents need per-call values inside the system prompt: the caller’s name, their city, an appointment time. Instead of building those strings yourself before every request, write a placeholder once and let SLNG fill it in.

Do this rather than building the string yourself. The stored answer then holds the placeholder instead of the caller’s data, which is what makes a personalized answer cacheable and lets your callers share one entry. The model sees the same final prompt either way.

# Keep placeholders in the prompt, send the values separately.
{"role": "system", "content": "You are calling {{caller_name}} in {{city}}."}
# ...with, in the same request:
# "template_variables": {"caller_name": "Rajesh", "city": "Mumbai"}

# Build the prompt string yourself per call.
{"role": "system", "content": f"You are calling {caller_name} in {city}."}

The reverse case matters just as much. A value that steers the answer is not a template variable, however much it varies per call.

# Write the steering value into the prompt text.
{"role": "system", "content": "Reply to the caller in Spanish."}

# The reply language decides what the answer IS, not what it says.
{"role": "system", "content": "Reply to the caller in {{session_language}}."}
# "template_variables": {"session_language": "Spanish"}   # <-- wrong field

Syntax. Use double braces around the variable name: {{caller_name}}, {{city}}. Names are letters, digits, and underscore. Single braces ({name}), Jinja, and ${var} are not placeholders. Only {{name}} is. Substitution applies to your prompt messages, the system and developer roles. Your user and assistant message content is never modified, so a literal {{...}} a caller happens to type is left exactly as-is.

Here is a full request using template variables:

resp = client.chat.completions.create(
    model="slng/auto",
    messages=[
        {"role": "system", "content": "You are calling {{caller_name}} in {{city}}."},
        {"role": "user", "content": "hello?"},
    ],
    extra_body={
        "template_variables": {"caller_name": "Rajesh", "city": "Mumbai"},
    },
)

The model receives You are calling Rajesh in Mumbai.

Missing values are rejected. If your request references a {{name}} you did not supply in template_variables, the router returns a 422.

{
  "error": {
    "message": "Missing required template variables: appointment_time, city",
    "type": "invalid_request_error",
    "param": "template_variables",
    "code": "missing_template_variables"
  }
}

What belongs in `template_variables`

Do carry personalization values that get spoken or echoed in the conversation: the customer’s name, the agent’s name, the company name, an appointment detail rendered as text.
Don’t carry values that steer or change the content of the answer itself: the response language, a plan or tier that changes which policy is described, a region that changes which rules apply.
The language case deserves spelling out. A language variable is fine when it is just text to be said, for example the agent confirming “So your preferred language is {{language}}, correct?” during the call. It is not fine when it controls the language the model answers in: two callers who chose different languages would share one cached answer, and one would hear the wrong language.

Limits. Up to 64 variables per request, names up to 64 characters, values up to 4000 characters. Exceeding these returns a 422.

Runtime variables. To leave a placeholder deliberately unfilled, because your own orchestrator fills it mid-call, set template_vars_strict: false on your configuration. The placeholder then passes through untouched, and an answer that echoes one is never cached.

Which configuration answers

Every request is answered from one configuration: its models, in up to three tiers, and its settings, such as caching. Which one depends on how the request reaches the router:

A key created in a project uses that project’s configuration: the model chosen in the project’s Think section.
An agent you deploy on SLNG uses its own configuration, chosen in its own Think section.

Each configuration keeps your organization’s settings, except where a setting was changed for that project or agent. A slng_config in the request applies on top of the configuration that applies, for that request only. See Inline, per request.

BYOK Context Router (`slng_config`)

There are two ways to give the router your models:

Inline, per request

Send slng_config in the request body. It can name any endpoint and key, including ones you never saved in the dashboard.

In the Think section

Choose a model in your project’s Think section. LLM keys you save on the dashboard’s BYOK page are offered there.

A voice platform integration that only takes a base URL, an API key and a model name can use only the Think section.

Inline, per request

Whether your slng_config carries tiers decides which models answer:

With tiers, the models in it answer, on the endpoints and keys it names.
Without tiers, the models of the configuration that applies answer. For a key created in a project, those are the project’s.

Settings you don’t send, such as the prewarm cache or a flow SLNG authored for your agent, come from the configuration that applies. Any top-level field of the configuration can be set inline, not only the ones in the Configuration reference: each field your slng_config sends replaces the configuration’s value for that request only, and every field it leaves out keeps the configuration’s value. So sending your own models with a project’s key keeps the project’s other settings. Your tiers and your default_tier travel together. If you send tiers, the configuration’s own default_tier no longer applies: send yours with them, or it is unset.

slng_config carries your own endpoints and provider keys, so treat it like a credential: send it only over HTTPS and keep it out of logs.

Saved in the dashboard

Every LLM key you save on the BYOK page is offered in your projects’ Think sections, wherever it serves the project’s language and region. Choose it there to use it. A key marked Disabled cannot be chosen or used.

Smart routing

Tiers rank your models: tier 1 is the cheapest, tier 3 the most capable. With "model": "slng/auto" (the default when you omit model) and two or more filled tiers, the router picks a tier for each request: the cheapest one it expects to answer that turn well. There is no separate field: send two or more tiers in slng_config.

Turns that may need a tool call, turns that look demanding, and turns the router cannot judge (an image, say) go to your most capable tier.
default_tier answers an opening line your agent has the model write, before the caller has said anything, whether or not tools are sent with it (most frameworks send an agent’s tools with every request). Only a request that forces a tool call goes to your most capable tier. A fixed greeting your code speaks never reaches the router. Unset, your cheapest tier answers that line. If the tier you set is empty, the nearest filled tier above it answers, or your most capable tier when there is none above.
One tier means no per-turn routing. That tier answers every turn, exactly as before.
Naming a model in model pins it and skips per-turn routing.
With slng_pure_proxy, there is no per-turn pick: your default_tier answers every turn, or your most capable tier when it is unset (see Shadow mode).

On a live answer, the x-slng-model response header names the model that answered, after any failover. A cache hit carries no x-slng-model. Tiers are for per-turn routing, not a backup list. If you have been sending two tiers to mean “preferred model first, backup second”, put both in one tier and give the backup priority: 1. One tier means no per-turn routing, and the backup still answers when the preferred model fails:

slng_config

{
  "tiers": {
    "1": [
      {"model": "gpt-4o", "weight": 100, "endpoint": {"url": "https://api.openai.com/v1", "api_key": "sk-..."}},
      {"model": "gpt-4o-mini", "weight": 100, "priority": 1, "endpoint": {"url": "https://api.openai.com/v1", "api_key": "sk-..."}}
    ]
  }
}

Example: two tiers

This request sends two Gemini models through Google’s OpenAI-compatible endpoint. A short, formulaic turn goes to gemini-2.5-flash-lite. Every other turn, including one that may need a tool call, goes to gemini-2.5-flash.

Two tiers

resp = client.chat.completions.create(
    model="slng/auto",
    messages=[
        {"role": "system", "content": "You are a helpful clinic assistant."},
        {"role": "user", "content": "Yes, that works."},
    ],
    extra_body={
        "slng_agent_id": "clinic-scheduler-v1",
        "slng_session_id": "call-8f2c1a4e",
        "slng_config": {
            "default_tier": 3,
            "tiers": {
                "1": [
                    {
                        "model": "gemini-2.5-flash-lite",
                        "weight": 100,
                        "endpoint": {
                            "url": "https://generativelanguage.googleapis.com/v1beta/openai/",
                            "api_key": "YOUR_GEMINI_API_KEY",
                            "auth_header": "x-goog-api-key",
                        },
                    }
                ],
                "3": [
                    {
                        "model": "gemini-2.5-flash",
                        "weight": 100,
                        "endpoint": {
                            "url": "https://generativelanguage.googleapis.com/v1beta/openai/",
                            "api_key": "YOUR_GEMINI_API_KEY",
                            "auth_header": "x-goog-api-key",
                        },
                    }
                ],
            },
        },
    },
)

Failover

If the chosen model fails (a 5xx, a timeout, a 429 or a dropped connection), the router tries another model when your configuration has one, in this order:

Another model in the same tier.
Otherwise, the next more capable tier.
Otherwise, when the chosen tier is already your most capable, the closest cheaper tier.

How many models. Without priority entries, a request tries at most two models. If the chosen tier has entries at priority 1, 2, …, the router also works through them, in that order, before it moves on to other tiers, trying at most four models.
Brief outages. On a 502, 503 or 504, or a dropped connection, the router retries the same model once before it steps down to a cheaper tier or gives up. A pinned model gets only that retry.
Other errors. Any other 4xx from your provider comes back to you straight away. If every model tried fails, the request fails with the last error.
Streams. Failover happens only before a stream starts. A failure after that ends the stream with an error chunk (see Streaming).

Configuration reference

A configuration has these fields:

Field	What it is
`tiers`	Your models, in up to three tiers: `"1"` (cheapest) to `"3"` (most capable). Leave it out to answer with the models of the configuration that applies.
`default_tier`	`1`, `2` or `3`. Answers an opening line your agent has the model write, before the caller has spoken, whether or not tools are sent with it; a request that forces a tool call, and every other turn the router cannot judge, goes to your most capable tier. Omitted: unset when you send `tiers`, otherwise the configuration’s own value. Unset, your cheapest tier answers that opening line. If the tier you set is empty, the nearest filled tier above it answers, or your most capable tier when there is none above. With `slng_pure_proxy`, it answers every turn (your most capable tier when unset). No effect with one tier.
`template_vars_strict`	Set `false` to leave an unfilled `{{name}}` placeholder in place instead of returning a `422`. Omitted: the configuration’s own value, else `true`. Do not send `null`: it resets to strict.

These are the fields most requests need. Any other top-level setting of the configuration can be sent in slng_config the same way, and replaces that setting for the request. See Inline, per request. Each entry in a tier has:

Field	What it is
`model`	The name you use to refer to this entry, unique within its tier. `x-slng-model` reports this name.
`weight`	This entry’s share of the tier’s traffic (1 to 100). Weights sum to 100 among entries with the same `priority` in a tier.
`priority`	Optional, default `0`. Entries at `0` share the tier’s traffic by weight, and each tier needs at least one. Entries at `1`, `2`, … get no traffic: when the entries at `0` fail, the router tries the highest-weight entry at each level, in order, so give each level one entry.
`endpoint`	Where the model is served, including your credentials for it.
`endpoint.model_id`	The model name to send to the provider, when it differs from your `model` label. Optional except on `bedrock`, where it is required.
`kwargs`	Optional. Request settings for this model only, such as `{"reasoning_effort": "none"}`. A setting the request itself sends wins.

The smallest useful configuration is one tier with one OpenAI-compatible endpoint:

resp = client.chat.completions.create(
    model="slng/auto",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "slng_config": {
            "tiers": {
                "1": [
                    {
                        "model": "gpt-4o-mini",
                        "weight": 100,
                        "endpoint": {
                            "url": "https://api.openai.com/v1",
                            "api_key": "sk-...",   # YOUR provider key, not your SLNG key
                        },
                    }
                ]
            }
        },
    },
)

Your SLNG key stays in the Authorization header as always. The api_key inside the endpoint is your own key for that provider (here, OpenAI). It is used to make that request’s model call and is never echoed back in any error message.

Supported providers

The endpoint object defaults to provider: "openai-compat", which covers any service that speaks the OpenAI Chat Completions API (OpenAI itself, Azure-hosted alternatives, Groq, self-hosted vLLM, and so on). Four more providers are supported; each needs its own fields on the endpoint object.

`provider`	Required endpoint fields
`openai-compat` (default)	`url`, `api_key`
`openai-responses`	`url`, `api_key`
`azure`	`url`, `api_key`, `azure_deployment`, `api_version`
`vertex`	`vertex_credentials`, `vertex_location`
`bedrock`	`aws_access_key_id`, `aws_secret_access_key`, `aws_region`, `model_id`

On azure, url is the resource root, not the full deployment URL. On openai-compat, add auth_header when the provider wants its key in a custom header. Fields belonging to another provider are rejected rather than ignored. For example, an Azure OpenAI entry looks like:

{
  "model": "my-azure-gpt4o",
  "weight": 100,
  "endpoint": {
    "provider": "azure",
    "url": "https://my-resource.cognitiveservices.azure.com/",
    "api_key": "...",
    "azure_deployment": "gpt-4o-deploy",
    "api_version": "2024-12-01-preview"
  }
}

Limits and errors

The slng_config object must stay under 256 KB when serialized; larger returns a 400.
A configuration that fails validation returns a 400 starting with invalid slng_config: and a description of the problem. Your credentials never appear in these messages.
Sending slng_config as anything other than a JSON object (a string, a list) returns a 422 with code invalid_slng_config.

Shadow mode

Two optional flags let you route production traffic through SLNG and measure what it would save, before you let it change anything. Both default to false and need no setup.

Flag	What it does
`slng_analytics`	Keeps a copy of the request and the answer for your analytics, in addition to serving the call normally. It changes nothing about the answer, your latency, which model answers, or caching. Copies stay in your region.
`slng_pure_proxy`	Returns your own model’s answer untouched: no cached answers, no prepared replies, no changes to the text. Your configuration still applies, including failover, but there is no per-turn tier pick: with two or more tiers, your `default_tier` answers every turn (your most capable tier unless you set it). That includes turns that may need a tool call, so if your agent uses tools, set `default_tier` to your most capable tier for the trial (add `"default_tier": 3` to the `slng_config` you send), or run the trial with one tier. If your tiers still mean “preferred model first, backup second”, every trial turn goes to the backup unless `default_tier` is `1`: see the backup setup in Smart routing.

Send both together and you have a shadow trial. You receive exactly what your own model produces, while the router measures what it could have served from cache and what that would have saved. Going live is removing the two flags: same URL, same key, same request body.

A trial measures; it does not speed anything up. A turn the router would have answered from cache goes to your model instead, so trial latency reflects your own model. Send your prompt in template form with template_variables for the measurement to be meaningful.

Get started

Regions

Execution layer

Agent Builder

Models

Integrate

Before you start

Endpoint and authentication

Making a request

Streaming

Available parameters

`slng/auto` vs a named model

slng/auto

A named model

Required fields

Reading the response

Normal LLM response

Cache hit

Response caching

What SLNG does not store

Per-call values (`template_variables`)

What belongs in `template_variables`

Which configuration answers

BYOK Context Router (`slng_config`)

Inline, per request

In the Think section

Inline, per request

Saved in the dashboard

Smart routing

Example: two tiers

Failover

Configuration reference

Supported providers

Limits and errors

Shadow mode

​Before you start

​Endpoint and authentication

​Making a request

​Streaming

​Available parameters

​slng/auto vs a named model

slng/auto

A named model

​Required fields

​Reading the response

Normal LLM response

Cache hit

​Response caching

​What SLNG does not store

​Per-call values (template_variables)

​What belongs in template_variables

​Which configuration answers

​BYOK Context Router (slng_config)

Inline, per request

In the Think section

​Inline, per request

​Saved in the dashboard

​Smart routing

​Example: two tiers

​Failover

​Configuration reference

​Supported providers

​Limits and errors

​Shadow mode

Before you start

Endpoint and authentication

Making a request

Streaming

Available parameters

`slng/auto` vs a named model

Required fields

Reading the response

Response caching

What SLNG does not store

Per-call values (`template_variables`)

What belongs in `template_variables`

Which configuration answers

BYOK Context Router (`slng_config`)

Inline, per request

Saved in the dashboard

Smart routing

Example: two tiers

Failover

Configuration reference

Supported providers

Limits and errors

Shadow mode