Observability

Inspect metrics, structured logs, request IDs, key state, and stored request history.

Request IDs and structured logging

Every proxied response includes an X-ModelMux-Request-ID header. Use it to correlate client failures with ModelMux logs and, when available, AI route traces.

Application logs are emitted with Go's structured log/slog package and include request method, path, remote address, and latency.

Metrics endpoints

HTTP

GET /metrics
GET /metrics?format=prometheus

The default response is JSON. Add format=prometheus for Prometheus exposition format.

Metric	Meaning
`modelmux_requests_total`	Requests by model, key, and group.
`modelmux_errors_total`	Observed error counts.
`modelmux_rate_limits_total`	Local or upstream rate-limit events.
`modelmux_latency_ms`	Latency histogram with predefined millisecond buckets.
`modelmux_status_total`	Response status classes such as 2xx, 4xx, and 5xx.
`modelmux_active_keys`	Currently active-key gauge.
`cooldown_keys`	Keys currently cooling down.
`invalid_keys`	Keys marked invalid.
`limited_keys`	Keys unavailable due to local limits.

Request logs

Request history can include the selected group, model, provider, key, status code, latency, input/output token counts, estimated cost when available, and timestamp.

In-memory mode retains a bounded recent buffer. SQLite mode keeps persistent history and allows later queries through the CLI and HTTP endpoint.

Query request history

CLI

modelmux logs --limit 50
modelmux logs --model-id example-chat
modelmux logs --status-code 429
modelmux logs --json --limit 100

HTTP

GET /logs?limit=50
GET /logs?model_id=example-chat
GET /logs?status_code=429

JSON CLI output is useful for scripts, incident reports, and ingestion into other tooling.

What to monitor

Error and status rate: watch 429 and 5xx growth by model and provider.
Latency: alert on sustained p95 or p99 changes, not isolated slow requests.
Key capacity: monitor active, cooldown, invalid, and limited key counts.
Quota headroom: inspect daily request and token consumption before exhaustion.
Failover frequency: repeated failover can indicate provider degradation or incorrect local limits.
Storage growth: manage request-log retention and database size in long-running deployments.