Quick start

Create a minimal provider, model, and key configuration and send a request through ModelMux.

Result

You will run ModelMux on 127.0.0.1:8787 with one OpenAI-compatible upstream and one API key loaded from an environment variable.

1. Initialize the configuration

Terminal

modelmux config init
${EDITOR:-vi} ~/.config/modelmux/config.yaml

config init creates an example file at the default configuration path. Existing files should be reviewed before replacement.

2. Configure a provider, model, and key

~/.config/modelmux/config.yaml

app:
  name: modelmux
  log_level: info

server:
  host: "127.0.0.1"
  port: 8787
  require_auth: false
  admin:
    require_auth: true

providers:
  - id: example
    name: Example Provider
    type: openai-compatible
    base_url: https://api.example.com/v1
    auth_type: bearer
    timeout_seconds: 120
    enabled: true

models:
  - id: example-chat
    provider_id: example
    model_name: upstream-model-name
    strategy: failover
    enabled: true

keys:
  - id: example-primary
    provider_id: example
    model_id: example-chat
    value_env: EXAMPLE_API_KEY
    status: active
    priority: 1

retry:
  max_retry_per_key: 1
  max_total_attempts: 3
  backoff_milliseconds: [300, 700, 1500]

cooldown:
  rate_limit_seconds: 300
  server_error_seconds: 60
  timeout_seconds: 60

Do not commit plaintext credentials

Use value_env or secret_ref. Plaintext value is intended only for controlled development environments.

3. Validate the configuration

Terminal

export EXAMPLE_API_KEY="your-provider-key"
modelmux config validate

# Optional: also contact configured providers
modelmux config validate --check-provider

Validation catches malformed YAML and invalid configuration relationships before the proxy starts. Use --json when validation output is consumed by scripts.

4. Start the proxy

Terminal

modelmux start

The service listens at http://127.0.0.1:8787. The OpenAI-compatible API base URL is http://127.0.0.1:8787/v1.

5. Send the first request

curl

curl http://127.0.0.1:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "example-chat",
    "messages": [
      {"role": "user", "content": "Explain reliable LLM routing."}
    ]
  }'

The request uses the ModelMux model ID example-chat. ModelMux translates that ID to the configured upstream model name.

6. Connect an OpenAI-compatible SDK

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://127.0.0.1:8787/v1",
  apiKey: "local-development-token",
});

const response = await client.chat.completions.create({
  model: "example-chat",
  messages: [{ role: "user", content: "Hello from ModelMux" }],
});

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8787/v1",
    api_key="local-development-token",
)

response = client.chat.completions.create(
    model="example-chat",
    messages=[{"role": "user", "content": "Hello from ModelMux"}],
)

When server.require_auth is false, the client API key is not used by ModelMux. Once authentication is enabled, pass the configured ModelMux bearer token instead.

7. Inspect the router

Terminal

modelmux tui

# In another terminal
modelmux logs --limit 20
curl http://127.0.0.1:8787/metrics

The TUI exposes providers, models, groups, key health, logs, configuration, metrics, chat sessions, and AI diagnostics.