Route OpenAI-compatible traffic across providers and API keys with automatic failover, layered limits, usage tracking, and policy-based routing—all from one Go binary.
modelmux start demo bootmodelmux start~/.config/modelmux/config.yamlloaded4 adaptersready10 configuredhealthyhttp://127.0.0.1:8787listeningModelMux evaluates provider and key health before every request, applies configured limits, and moves traffic to the next eligible target when an upstream becomes unavailable.
modelmux providers --status simulated runtime viewYour application should not need to understand provider outages, exhausted keys, cooldown timers, or retry policy. ModelMux absorbs those failures before they become client errors.
modelmux ai explain --request-id request_7f2a↵modelmux replay request_7f2ademo tracePOST /v1/chat/completions
mimo-primary · priority 1
rate limit exceeded
mimo-primary · retry after 300s
claude-team · priority 2
response streamed to client
No application retry. No leaked provider error. One compatible response.
exit 0Move to the next healthy key or provider without changing application code or asking the client to retry.
429 · 5xx · timeoutApply model, key, token, daily quota, and concurrency limits before an exhausted upstream rejects traffic.
RPM · TPM · quotaTrace every selection, retry, cooldown, latency measurement, request ID, and final route from one terminal.
logs · metrics · traceConfigure ModelMux once, validate the route, then keep your application focused on product logic instead of provider-specific recovery code.
modelmux init --guidedsetup sessionAnswer three configuration stages. ModelMux validates each stage before continuing.
Register the provider adapter and upstream endpoint.
Connect routable models, credentials, and recovery policy.
Keep the OpenAI client and replace only its base URL.
curl localhost:8787/v1/models200 OKFollow one request from authentication to the upstream stream. ModelMux keeps the routing path explicit, measurable, and independent from your application code.
modelmux logs --json --limit 1illustrated traceBearer token verified before proxy work begins.
passedTarget group and request policy resolved.
productionModel RPM, key quota, and concurrency checked locally.
42/60 RPMHealth, priority, cooldown, and strategy evaluated.
mimo-primaryCompatible payload forwarded and response streamed.
streamingSelected route stayed observable from local checks through upstream delivery.
200 · 842msThis is the real ModelMux dashboard—not a decorative terminal mockup. Configuration, health, logs, and routing tools stay available without opening a browser control plane.
Start locally with Go, add a provider and keys, then point your existing OpenAI-compatible client at ModelMux. No external control plane is required.
install modelmux3 commandsgo install github.com/livingdolls/yute-modelmux/cmd/modelmux@latestmodelmux config init && modelmux config validatemodelmux startUse config-defined model IDs from any OpenAI-compatible SDK.
exit 0