A gateway should add operational value—not just resell a request.
Direct provider integrations are simple until an application supports several model families, regions, credentials, retry policies, price tables, and incident paths. Each client service then reimplements the same adapter and can make different decisions during an outage.
The API Many gateway stores provider routes separately from public model identities. It reserves account balance before dispatch, records every attempt, settles confirmed usage at one published logical-model rate, and exposes a stable OpenAI-compatible boundary to your servers.
The first release intentionally excludes blind retries and unverified model substitutions. Operational reliability comes from clear state and reconciliation, not from hiding uncertain provider outcomes behind another automatic request.
What the gateway owns
Before dispatch
Authentication, payload limits, model eligibility, maximum-route pricing, balance reserve, and idempotency.
During dispatch
Route choice, timeout boundary, explicit failure classification, attempt receipt, and health update.
After dispatch
Usage settlement, refund of unused reserve, stored response, or a reconciliation hold for uncertainty.