Routing Rules

A routing rule decides which gateway serves each request for a FreeRouter key. Two strategies: a weighted %-split, or a priority order with failover — plus optional model remaps that rewrite one incoming model to a fixed provider/model, and an optional decisioner that lets Jev pick among the attached targets. Attach one rule per key.

How rules work

Every FreeRouter key is bound to exactly one routing rule. A rule references the providers you've added to the workspace and lists them as targets. When a request arrives, FreeRouter evaluates the rule, picks a target, swaps in that provider's key, and forwards the request. If a model isn't available on the chosen target, FreeRouter skips to the next eligible target.

Split vs. failover

The two strategies are not either-or on reliability: a split only decides which target goes first. After that pick, a split rule fails over through the remaining targets in listed order, exactly like a priority rule (a weight of 0 makes a target failover-only).

Combine them per key: one key splits for bulk jobs, another fails over for production, and you swap a rule without a redeploy.

Percentage split

A split rule distributes requests across targets by weight — a cost/performance blend, an A/B of two gateways, or a gradual shift of traffic.

Example — 70/30 cost blend

Send most traffic to a cheaper gateway, the rest to a faster one. Targets reference your stored provider keys by ID (find them under Providers in the dashboard) — weights are relative and normalized to 100%:

{
  "strategy": "split",
  "targets": [
    { "provider_key_id": "<openrouter-key-id>", "gateway": "openrouter", "weight": 70 },
    { "provider_key_id": "<vercel-key-id>",     "gateway": "vercel",     "weight": 30 }
  ]
}

Roughly 7 in 10 requests go to OpenRouter, 3 in 10 to the Vercel AI Gateway. Selection is per request, so both gateways see live traffic continuously.

Example — canary a new gateway

Start a new provider at a small share, then raise its weight as you gain confidence:

{
  "strategy": "split",
  "targets": [
    { "provider_key_id": "<openrouter-key-id>", "gateway": "openrouter", "weight": 95 },
    { "provider_key_id": "<cloudflare-key-id>", "gateway": "cloudflare", "weight": 5 }
  ]
}

Priority with failover

A priority rule lists targets in order. FreeRouter tries the first; if that attempt fails retryably, the next target down the list gets the same request.

Example — primary + backups

List targets in priority order — the dashboard order is the priority (no numbers needed):

{
  "strategy": "priority",
  "targets": [
    { "provider_key_id": "<openrouter-key-id>", "gateway": "openrouter" },
    { "provider_key_id": "<vercel-key-id>",     "gateway": "vercel" },
    { "provider_key_id": "<cloudflare-key-id>", "gateway": "cloudflare" }
  ]
}

OpenRouter serves normally. If a request to it fails, FreeRouter retries the same request against Vercel, then Cloudflare, before returning an error. The X-FreeRouter-Attempts response header reports how many targets were tried.

What triggers a failover

• Connection errors or timeouts from the upstream gateway.
• On a streaming request (stream: true), no response headers within the failover timeout (default 5s) while another target remains. The last attempt keeps the upstream timeout (default 60s), which is also what every target of a non-streaming request gets.
• 5xx responses from the gateway.
• 429 rate-limit responses.
• 401, 402 or 403 from that gateway: the provider key was rejected, the account is out of credit, or the key can't use the model. Another target may still serve it. If a rejected provider key (401) is the final answer, FreeRouter returns 502 provider_auth_failed naming the gateway, so it can't be mistaken for a problem with your FreeRouter key.
• The requested model isn't offered by that target — either the gateway answers 404, or its catalog lists no spelling of the model and FreeRouter skips it without calling it (see Model availability).

Client errors that would recur everywhere — like a malformed body (400) — are not retried; FreeRouter returns them immediately. One exception: a gateway that says "no such model" with a 400 instead of a 404 (Morph does) fails over like a 404, labelled no-model; every other 400 from it is relayed as-is.

Why did a request fail over?

X-FreeRouter-Failover names each skipped attempt as gateway:reason (e.g. darkbloom:ttfb-timeout); the Logs tab shows the same chain per request, with a remap badge on attempts that served a model remap. The reasons:

• ttfb-timeout — no response headers inside the failover timeout (default 5s, or the key's / target's own setting). The gateway was too slow or hung; if it is reliably slow but worth keeping, give it more time rather than removing it. The Logs chain says which timer fired and whose setting it was — No first byte within 12s (this target's failover timeout).
• timeout — no response within the upstream timeout (default 60s, or the key's own setting). Hung or extremely slow upstream.
• deadline — the request deadline (95s for the whole request) ran out: either this attempt was cut short because less than its own timeout was left, or it was never called because the deadline had already passed. Earlier targets used up the time; give the slow one less, or put a faster target first.
• unreachable — the connection itself failed (DNS, refused, reset). Check the provider key and gateway status; if other gateways work, it is upstream.
• rate-limited — the gateway returned 429. Retry shortly or spread load.
• no-model — the gateway returned 404 (or Morph's "model not served" 400): the model isn't offered there. Remap it or drop that target for the model.
• http-402 / http-403 — the gateway refused this target for billing or entitlement reasons: out of credit, or the provider key can't use the model (Vercel's free tier, say). Any other status without a label of its own shows the same way, as http-<status>.
• not-listed — the gateway was never called: its catalog lists no spelling of the model, so FreeRouter skipped straight to the next target. Not counted in X-FreeRouter-Attempts.
• batch-only — the gateway was never called: the id it would have received is a batch-only :batch variant (see below). Not counted in X-FreeRouter-Attempts.
• no-embeddings — the gateway was never called: the request was POST /v1/embeddings and that gateway has no embeddings route (only chat). When every target on the rule is skipped this way the request answers 501 embeddings_unsupported, naming the gateways that do serve it. Not counted in X-FreeRouter-Attempts.
• server-error — the gateway returned 5xx. Upstream fault; FreeRouter already moved on.
• provider-auth — the gateway returned 401: the provider key stored for that target was rejected (rotated, revoked, or mistyped). Re-paste it under Providers.
• client-closed — your client hung up. If it left before this hop answered, the hop is labelled client-closed and nothing further is tried. If the gateway had already sent headers (often 200) and the client left while the body was still streaming, the hop stays 200 served and the request row is 499: the Status column is the caller's outcome, the hop badge is the gateway's. Either way FreeRouter cancels the in-flight upstream request (streams and MCP tool loops included) and tries no further targets. 499 is Logs only — the client is already gone, so no HTTP 499 body is sent. See Errors.
• unknown — the attempt ended without a status or a named error (a malformed stage record). Rare; quote the request id if you see it.

Timeouts

Two timers decide how long FreeRouter waits for a gateway to start answering (send its response headers), and one deadline bounds the request as a whole. None of them cuts off a response that has started streaming. The failover timeout only applies to streaming requests; a non-streaming request waits the upstream timeout on every target, because most gateways only send headers once the full answer is ready. Connection errors, 429 and 5xx still fail over immediately either way.

Per key. On API keys, open a key's Advanced section to set both values for that key. Leave a field blank to use the default.

Per target. Some gateways are slow to start but worth keeping — cheaper ones can take 5–15s to the first byte. Click the timer on a target's row in the routing rule to give that provider alone more time, without slowing failover for the rest of the rule. A target's own value wins over the key's, which wins over that gateway's learned budget, which wins over the 5s default:

failover timeout = target's own value → key's value → learned per-gateway budget → 5s
upstream timeout = key's value → 60s

The learned budget is a platform default per gateway, recomputed every 4 hours from 14 days of streaming first-byte times. It never goes below 5s, never above 15s, and moves at most 3s per run. A key or target value you set is left alone. The failover timeout never exceeds the upstream timeout. A target's own value only matters while another target follows it; once it is last, the upstream timeout applies. A model remap hop never carries a target override: it gets the key's failover timeout, then that gateway's learned budget, then the 5s default, even when the same provider key has its own ttfb_ms as a routing target.

Why 90s at most? FreeRouter sends nothing to your app until a gateway answers, and the network in front of api.freerouter.com drops a request that has received no bytes for about 100 seconds. A longer upstream timeout could never take effect. The request deadline keeps the sum of hops under that limit too: three slow targets can no longer wait 30s + 30s + 90s only for the connection to be dropped from outside — the request fails at 95s with a reason you can read.

Multi-step tool calls add up. With MCP tools attached, each model round waits on these timers separately, so a slow gateway is paid once per round — all inside the one request deadline. Keep a faster target after a slow one so a stalled round can fail over. A round that cannot start before the deadline ends a stream that already began with the shape's error event (deadline_exceeded), so a client never waits on a turn that will not come.

Set them programmatically with failover_ttfb_ms / upstream_timeout_ms on a key and ttfb_ms on a routing target — see the Management API and the MCP tools.

Chained turns pin to one provider

A client that chains Responses turns sends back the resp_fr_… id from the previous response as previous_response_id. That id names a response stored by one upstream account, so the follow-up is tried against the originating provider and nothing else: a pinned chain does not fail over, and model remaps and the Jev decisioner are skipped on pinned turns — both exist to move a request elsewhere. If the pinned provider is down you get its error, not a silent switch.

The setup consequence: if your client chains turns — Codex CLI does, as does any Responses client that reuses response ids — the key must include a gateway that can anchor a chain. Verified chain-capable: Ramp Router, Cloudflare AI Gateway, and LLM Gateway. A key routed only at other gateways serves the first turn and fails the follow-ups: OpenRouter, Baseten and Sail reject chaining outright, and FreeRouter refuses to chain off gateways that silently ignore it (Requesty, Darkbloom, Vercel) rather than answer as if the first turn never happened.

When a chain breaks, the error says which part moved: response_chain_target_unavailable (409) means the originating provider is no longer a target on the key — resend the full input to start a new chain. Chained ids are bound to the API key that created them and expire after 30 days. Full per-gateway matrix at Chaining.

Model availability across targets

A routing rule can mix gateways that spell the same model differently — OpenRouter's anthropic/claude-opus-5.5 is Requesty's anthropic/claude-opus-5-5 or vertex/claude-opus-5-5. FreeRouter translates in both directions: whichever spelling you send, each target receives its own (see Model IDs across gateways). Your exact id is kept wherever the target lists it, so a Requesty vertex/… route stays on Vertex when Requesty serves the request.

When a target's catalog lists no spelling of the model — a Requesty region pin like vertex/claude-opus-5-5@eu sent to OpenRouter, say — FreeRouter skips that target without calling it and moves to the next one, under priority and split alike. The skip shows up as gateway:not-listed in X-FreeRouter-Failover and in the Logs chain. Region pins are never dropped to make a request fit: a pinned id only goes to gateways that list that exact pin.

If none of the key's targets carries the model, nothing is sent upstream and the request answers 404 with code: "model_not_available", naming the providers your routing rules picked and the gateways that do list the model:

{
  "error": {
    "message": "The model “vertex/claude-opus-5-5@eu” isn’t available on openrouter, the provider this key’s routing rules picked to serve it. It is listed on: requesty. Add a provider that serves it to this key’s routing rules, or add a model remap.",
    "type": "invalid_request_error",
    "code": "model_not_available"
  }
}

Batch-only models. Some catalogs list a :batch variant next to the regular model — OpenRouter's openai/gpt-4o:batch is the half-price entry served only by its asynchronous Batch API. A :batch id is never sent to a synchronous endpoint: the request answers 400 with code: "batch_only_model" and nothing is spent. Submit it to the Batch API (POST /v1/batches) instead, or send the id without :batch for a synchronous call. A model remap from a :batch id to a regular model still serves, and a remap onto a :batch id is skipped as gateway:batch-only. :batch ids are also left out of GET /v1/models, the Playground picker and the Jev model picker.

Availability comes from FreeRouter's daily catalog crawl. A gateway whose catalog is missing or more than 72 hours old is never assumed to lack a model — it is tried as before — and a model id no catalog knows yet (a same-day launch) is forwarded untouched.

Model remaps

A remap rewrites one incoming model to a fixed provider/model before the rule above runs — no client changes needed. Matching is exact (case-insensitive), * catches everything without its own rule, and a percent (1–100, default 100) canaries the rewrite while the rest falls through to the normal targets. If the destination fails retryably — the same failover triggers as any target: 401/402/403/404/429/5xx, a timeout or a connection error — FreeRouter fails over to the normal targets; any other answer is returned as-is. Full guide: Model Remaps.

Decisioner (Jev pre-router)

On a chat-shaped key you can add one decisioner: before the rule above runs, Jev picks which of the attached targets should serve the request (and, optionally, which model), and the static rule is the fail-open fallback whenever Jev cannot. The verdict rides every response as the X-FreeRouter-Decisioner header. Operators gate it with FR_DECISIONER_ENABLED. Full guide: Decisioner.

What a rule may contain

Every writer — the dashboard, PUT /api/v1/keys/:id/routing and the MCP set_routing tool — runs the same checks. A rule that breaks one is refused with 400 and nothing is saved.

A write replaces the whole rule: leave model_remaps or decisioner out of the body and they are cleared, not kept. Read the rule first (GET …/routing or get_routing) and send it back whole when changing one part.

One rule per key

Because each FreeRouter key carries its own rule, you can run different strategies side by side without touching code. A common setup:

• Production key — priority (failover) for maximum uptime.
• Batch/offline key — split weighted toward the cheapest gateway.
• Evaluation key — split 50/50 to compare two gateways head-to-head.