Configuration reference

Configure the server, storage, providers, models, groups, keys, retries, cooldowns, health checks, and AI features.

File location

The default configuration file is ~/.config/modelmux/config.yaml. Create an example with modelmux config init. Use the global --config flag to point commands at another file.

YAML is the source of truth for declarative routing. Runtime admin actions that enable or disable keys update both runtime state and the configuration file.

Complete example

config.yaml

app:
  name: modelmux
  log_level: info

server:
  host: "127.0.0.1"
  port: 8787
  require_auth: true
  auth_token_env: MODELMUX_AUTH_TOKEN
  max_request_body_mb: 10
  admin:
    require_auth: true

storage:
  type: sqlite
  path: ~/.local/share/modelmux/modelmux.db

providers:
  - id: provider-a
    name: Provider A
    type: openai-compatible
    base_url: https://api.example.com/v1
    auth_type: bearer
    timeout_seconds: 120
    enabled: true

models:
  - id: chat-primary
    provider_id: provider-a
    model_name: upstream-chat-model
    strategy: failover
    enabled: true
    requests_per_minute: 120
    max_concurrent_requests: 10
    capabilities:
      tools: true
      json_mode: true

model_groups:
  - id: production-chat
    name: Production Chat
    strategy: weighted
    enabled: true
    members:
      - model_id: chat-primary
        priority: 1
        weight: 3
        enabled: true

keys:
  - id: provider-a-primary
    provider_id: provider-a
    model_id: chat-primary
    secret_ref: provider-a-primary
    status: active
    priority: 1
    requests_per_minute: 60
    tokens_per_minute: 30000
    max_concurrent_requests: 3
    daily_request_limit: 1000
    daily_token_limit: 500000

health_check:
  enabled: true
  interval_seconds: 300
  timeout_seconds: 15

cooldown:
  rate_limit_seconds: 300
  server_error_seconds: 60
  timeout_seconds: 60

retry:
  max_retry_per_key: 1
  max_total_attempts: 5
  backoff_milliseconds: [300, 700, 1500]

ai:
  enabled: false

Server settings

Field	Description
`server.host`	Listening interface. Keep 127.0.0.1 for local-only access.
`server.port`	HTTP listening port. The documented default is 8787.
`server.require_auth`	Requires bearer authentication for proxy requests.
`server.auth_token_env`	Environment variable holding the ModelMux bearer token.
`server.max_request_body_mb`	Rejects request bodies above the configured size.
`server.admin.require_auth`	Requires authentication for /admin/* endpoints. Keep enabled.

Storage settings

Field	Description
`storage.type`	Leave empty for in-memory state or set to sqlite for persistence.
`storage.path`	SQLite file path. Parent directories are created as needed.

SQLite persists request logs, key runtime state, daily counters, quotas, cooldown timers, and metrics source data. Minute rolling windows remain in memory.

Providers, models, groups, and keys

These four sections form the routing graph:

providers[] defines upstream protocols and base URLs.
models[] defines client-visible model IDs and their provider mapping.
model_groups[] defines aliases that can select across models or exact keys.
keys[] defines credentials, priorities, state, and per-key limits.

IDs must be unique within their respective section. References such as provider_id, model_id, and key_id must resolve to existing enabled configuration entries.

Health checks, cooldowns, and retry

Section	Important fields
`health_check`	`enabled`, `interval_seconds`, `timeout_seconds`
`cooldown`	`rate_limit_seconds`, `server_error_seconds`, `timeout_seconds`
`retry`	`max_retry_per_key`, `max_total_attempts`, `backoff_milliseconds`

Total attempts should be sized to the available key pool. Excessive retries increase latency and can multiply upstream load during provider incidents.

Credential sources

Key field	Use
`value_env`	Recommended for environment-managed credentials.
`secret_ref`	Recommended when using the encrypted ModelMux secret store.
`value`	Plaintext development fallback. Avoid in committed configuration.

Encrypted secrets require MODELMUX_MASTER_KEY. The master key is not stored in the encrypted file and must be supplied whenever the store is opened.

Validation and reload

Terminal

modelmux config validate
modelmux config validate --json
modelmux config validate --check-provider

# Reload a running instance through the protected admin API
curl -X POST http://127.0.0.1:8787/admin/reload \
  -H "Authorization: Bearer $MODELMUX_AUTH_TOKEN"

Validate configuration before deployment. Provider checks perform network calls and therefore require credentials and upstream availability.