AI routing and diagnostics

Classify requests, apply guardrails, trace routing decisions, and select models with policy rules.

Disabled by default

AI features default to off for backward compatibility. Enable only the components needed by your deployment.

Enable AI features

config.yaml

ai:
  enabled: true
  classifier:
    enabled: true
    mode: heuristic
  guardrails:
    enabled: true
  route_trace:
    enabled: true
    include_response_header: true
  routing_rules:
    - when:
        task: coding
      use_model: deepseek-coder

Request classifier

The heuristic classifier can identify task categories including coding, reasoning, summarization, translation, JSON extraction, tool use, vision, and long-context requests.

Classification is designed for routing diagnostics and policy decisions. Review behavior against your own prompts before relying on it for production-critical selection.

Guardrails

Built-in guardrails can reject prompts containing common API-key patterns such as sk-, sk-ant-, AIza, and Bearer. They can also enforce a configured maximum prompt size.

These checks reduce accidental credential leakage but do not replace a complete data-loss-prevention or content-safety system.

Route traces

Route tracing records classification and routing decisions. With SQLite enabled, traces are persisted and can be inspected through the admin API.

HTTP

X-ModelMux-Route-Trace-ID: <trace-id>
GET /admin/traces/<trace-id>

Use route trace IDs with the normal X-ModelMux-Request-ID when debugging unexpected model selection.

Policy routing

Routing rules can redirect requests based on detected task type and request capabilities such as tools, vision, or streaming.

config.yaml

ai:
  enabled: true
  routing_rules:
    - when:
        task: coding
      use_model: coding-model
    - when:
        vision: true
      use_model: vision-model
    - when:
        tools: true
      use_group: tool-capable-models

Ensure every policy target exists, is enabled, and has eligible keys. Use diagnostics to understand which rule matched.

CLI diagnostics

Terminal

modelmux ai classify --file request.json
modelmux ai route --file request.json --json
modelmux ai explain --request-id <request-id>
modelmux ai doctor-config

doctor-config checks AI-related configuration relationships. route previews routing behavior without requiring the application to issue a normal production request.