> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prismgateway.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Routing and reliability

> How Prism times out, retries, and falls back across providers, and how to read the response headers

# Routing and reliability

Every chat, messages, and embeddings request is routed to a provider from the
model catalog. When that provider fails, Prism can retry it and move to the next
configured provider before returning an error. This page explains what happens,
what you control, and how to read the result.

## Timeouts

Each upstream attempt has a timeout to first byte. The default is 60 seconds.
Send `x-prism-timeout-ms` to change it for one request:

```bash theme={null}
curl -sS "$PRISM_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $PRISM_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-prism-timeout-ms: 15000" \
  --data-binary @request.json
```

Values are clamped to the server's bounds (at least 1000 ms). Anything that is
not a whole number of milliseconds is ignored and the default applies. A timeout
lower than the server default disables the timeout retry for that request, so a
short timeout on a long non-streaming answer cannot cost two generations.

## Retries and fallback

When enabled by the operator, Prism retries the same provider on transient
failures (rate limits, `5xx`, connection errors, one timeout) with exponential
backoff, honouring the provider's `Retry-After`. If the provider still fails and
the model has fallback providers configured, the request moves to the next one.
The whole sequence shares one time budget, so a request never waits on retries
indefinitely.

Prism never retries or falls back on errors caused by the request itself: a
prompt that exceeds the model's context window, an invalid parameter, or a
provider rejecting the request as malformed comes back to you unchanged.

You are billed once, for the attempt that produced the response. Failed
attempts are never charged.

## Conditional routing

An operator can attach rules to a model that send a request to a different
provider, upstream model or credential when its context matches. The context
is the organization's tags (set by the operator), the key's scopes, the model
id, and request metadata: the body's `metadata` object merged over the
`x-prism-metadata` header (a JSON object, up to 4 KB, at most 32 primitive
values). Metadata is only ever compared against; it never names a provider.

```json theme={null}
{ "metadata": { "tier": "gold", "region": "de" } }
```

The first matching rule wins and replaces the model's whole chain, including
its fallbacks, so a rule that pins a region never leaks out of it on a
transient failure. `x-prism-provider` still reports the provider that served
the request.

Metadata is sent by the caller, so any key can send any value: operators use
it for preferences (a region, an experiment), and match on organization tags
or key scopes for anything with a cost or compliance consequence. A missing
metadata key never equals `null`; it does satisfy `$ne`.

## Reading the response headers

| Header                        | Meaning                                                                        |
| ----------------------------- | ------------------------------------------------------------------------------ |
| `x-prism-provider`            | Provider that served the request                                               |
| `x-prism-retry-count`         | Retries before this response, across all providers tried                       |
| `x-prism-attempted-fallbacks` | Comma-separated providers that failed before the serving one; absent when none |
| `x-prism-trace-id`            | Request ID for support and usage lookup                                        |
| `retry-after`                 | Present on `429` when the provider asked for a wait                            |

These headers are also set on error responses, so you can see what Prism tried
before giving up.

## Error codes

When every attempt fails, the error code names what happened last:

| Code                                          | Status | Meaning                                                        |
| --------------------------------------------- | ------ | -------------------------------------------------------------- |
| `runtime_chat_failed:upstream_timeout`        | `502`  | The provider did not answer within the timeout                 |
| `runtime_chat_failed:upstream_rate_limited`   | `429`  | The provider is rate limiting; `retry-after` is set when known |
| `runtime_chat_failed:upstream_server_error`   | `502`  | The provider returned `5xx`                                    |
| `runtime_chat_failed:upstream_unreachable`    | `502`  | The provider could not be reached                              |
| `runtime_chat_failed:context_length_exceeded` | `400`  | The request exceeds the model's context window                 |
| `runtime_chat_failed:upstream_rejected`       | `4xx`  | The provider rejected the request                              |

Embeddings use the same codes with the `runtime_embeddings_failed:` prefix. See
[Errors](/errors) for retry guidance.

[อ่านภาษาไทย](/th/routing)