Providers

Configure OpenAI-compatible, Anthropic, Gemini, and custom upstream APIs.

Provider types

Type	Supported behavior	Conversion
`openai-compatible`	Chat, completions, streaming, and tools supported by the upstream	Direct pass-through
`anthropic`	Chat and streaming	OpenAI request/response conversion
`gemini`	Chat and streaming	OpenAI request/response conversion
`custom`	Same routing behavior as OpenAI-compatible	Direct pass-through

Conversion provides a common client-facing protocol, but provider-specific features still depend on what the upstream API supports.

OpenAI-compatible providers

config.yaml

providers:
  - id: local-vllm
    name: Local vLLM
    type: openai-compatible
    base_url: http://127.0.0.1:8000/v1
    auth_type: bearer
    timeout_seconds: 120
    enabled: true

This type is suitable for OpenAI-style hosted APIs, local inference servers, and gateways that expose compatible routes. ModelMux forwards supported request structures without provider conversion.

Anthropic providers

config.yaml

providers:
  - id: anthropic
    name: Anthropic
    type: anthropic
    base_url: https://api.anthropic.com
    auth_type: bearer
    timeout_seconds: 120
    enabled: true

ModelMux accepts the OpenAI-compatible client request and converts supported chat and streaming data for Anthropic. Use a model entry whose model_name matches the upstream Anthropic model identifier.

Gemini providers

config.yaml

providers:
  - id: gemini
    name: Google Gemini
    type: gemini
    base_url: https://generativelanguage.googleapis.com
    auth_type: bearer
    timeout_seconds: 120
    enabled: true

Gemini chat and streaming responses are converted to the compatible response shape. Test the exact endpoint and authentication behavior required by the configured Gemini API environment.

Custom providers

Use custom when the upstream follows the same request and response conventions as an OpenAI-compatible service but should be identified separately in configuration, logs, or operations.

ModelMux does not make an incompatible API compatible merely by selecting custom. The upstream still needs to understand the forwarded request format.

Testing providers and keys

Terminal

modelmux config validate --check-provider
modelmux key test --id provider-a-primary

A key test verifies connectivity with the key's configured provider. Authentication failures normally move a key toward an invalid state, while transient provider failures trigger cooldown behavior.