Models & routing

Every model in UberLLM can be served by several providers — curated upstreams and marketplace seller offers. For each request, UberLLM builds an ordered list of healthy candidates and routes to the best one, failing over automatically if the first attempt stumbles before returning any content.

Model ids

Use the canonical model id, e.g. deepseek/deepseek-v3.1. List the full catalog (with per-provider prices) at:

• Authenticated: GET https://api.uberllm.dev/v1/models
• Public: GET https://api.uberllm.dev/api/v1/public/models
• Web: uberllm.dev/models

Routing suffixes

Append a suffix to the model id to set the sort preference:

deepseek/deepseek-v3.1:floor   # cheapest
deepseek/deepseek-v3.1:nitro   # fastest

Default routing

Without a suffix or provider object, UberLLM:

1. Filters to candidates that can serve the request — active, context window ≥ your prompt, required capabilities (tools / JSON schema) supported, seller offers under their daily cap, and price within any max_price you set.
2. Deprioritizes (does not exclude) any provider with a significant outage in the last 30 seconds — it moves to the back of the line.
3. Load-balances among the healthy front group with a weighted random pick proportional to 1 / price², so cheaper providers win most traffic while the load still spreads. Blended price = input + 3×output per million tokens.

Cheapest-healthy-provider behavior emerges from this: low-priced seller offers sit in the same list as curated endpoints and win on price when healthy, and fall behind when they degrade.

The provider object

For fine control, pass a provider object in the request body (OpenRouter-style):

{
  "model": "deepseek/deepseek-v3.1",
  "messages": [ ... ],
  "provider": {
    "order": ["together", "fireworks"], // try these first, in this order
    "only": ["together", "fireworks"],  // never use anything else
    "ignore": ["novita"],               // exclude these providers
    "allow_fallbacks": true,             // false = no failover, fail on first miss
    "sort": "price",                     // "price" | "latency" | "throughput"
    "max_price": { "input": 1.0, "output": 3.0 } // USD per million tokens ceiling
  }
}

• order — preferred providers first; others follow unless only/allow_fallbacks restrict them.
• only — hard allowlist.
• ignore — hard denylist.
• allow_fallbacks — false pins you to the first candidate; if it fails you get an error rather than a failover.
• sort — price, latency, or throughput (percentiles from the rolling 5-minute health window).
• max_price — drop candidates above these per-MTok prices.

The :floor / :nitro suffixes are shorthands for sort: "price" and sort: "throughput" respectively.

Fallbacks

If a chosen candidate fails to start streaming — connection error, HTTP 5xx, 429, or no first byte within the time budget — UberLLM records the failure and retries the next candidate, up to 3 attempts. Failover only happens before the first content byte reaches you; once streaming starts, a mid-stream upstream failure ends the stream (and you are billed only for what was produced). If every candidate fails, you get 502 no_available_provider and any reservation is fully released.

Set allow_fallbacks: false (or provider.only with a single provider) to opt out of failover.

BYOK routing

If a key is configured for bring-your-own-key (or you pass BYOK per request), UberLLM proxies straight to the provider with your own credentials — no credit reserve/settle, cost_usd = 0, and the request is still logged. See Pricing.