Metrics and Logs

AISIX AI Gateway exposes aggregate metrics, per-request logs, response headers, and exportable usage events. Together, these signals show service health and connect a caller-visible response to the model, route, and policy outcome that produced it.

Choose a Telemetry Source

Start with the source that matches the operational question, then correlate signals by request, model, or provider when an issue needs deeper investigation.

Scrape Prometheus Metrics

Prometheus metrics are suited for monitoring traffic trends, latency, token counters, cost counters, rate-limit outcomes, cache behavior, and exporter delivery health.

AISIX serves Prometheus metrics on the dedicated metrics listener at /metrics by default. Change the path or disable the endpoint through the startup observability settings.

This endpoint is unauthenticated by design. Keep the dedicated metrics listener private.

Configure Prometheus exposure in the startup configuration:

observability:
  metrics:
    prometheus:
      enabled: true
      path: "/metrics"

The dedicated listener binds to 0.0.0.0:9090 by default. Set a different listener address when Prometheus should scrape another interface or port.

Scrape the default metrics endpoint:

curl -sS "http://127.0.0.1:9090/metrics"

AISIX emits native metric names with the aisix_ prefix. Use the histogram series for latency percentiles across gateway instances and the request counters for success-rate and routing analysis. For exact metric names, label scope, and PromQL examples, see Metrics Reference.

Select the labels of individual metric families with observability.metrics.labels. For configuration examples, defaults, and the complete supported-variable list, see Metric Labels and Variables.

Import the Grafana Overview Dashboard

A prebuilt AISIX AI Gateway dashboard provides a starting view of gateway traffic, latency, token throughput, spend, governance (rate limits, guardrails, and inbound authentication), and upstream health. It uses dashboard ID 25746 and only built-in Grafana panel types. The published JSON declares Grafana 11.0.0; validate the dashboard before using it on an older Grafana release.

In Grafana, select Dashboards, then New, then Import. Enter 25746 and select the Prometheus data source that scrapes the gateway. To fetch the dashboard JSON directly instead:

curl -sS "https://grafana.com/api/dashboards/25746/revisions/latest/download" -o aisix-dashboard.json

The dashboard is an operational overview, not the complete metric catalog. It has no dedicated panels for cache, budget, exporter, MCP, A2A, or configuration-status metrics. Use the Metrics Reference for the complete list and task-specific PromQL.

The provider and model variables filter only panels whose metrics carry those labels. The dashboard expects the default provider and model labels on aisix_requests_total and on the metric families used by its filtered panels. If you remove either label with observability.metrics.labels, update the dashboard variables and affected queries. See Metric Labels and Variables.

Panels for optional features remain empty until those features produce data. The spend panels reflect locally calculated gateway spend, which currently means Realtime sessions. For an open-source AISIX gateway, Realtime spend requires model cost metadata. AISIX Enterprise calculates other request costs, including ordinary Chat Completions, Messages, and Responses requests, in the control plane. Those values do not populate this gateway metric; use Model Pricing and Request Logs for that spend.

Guardrail panels require an attached guardrail. Client-cancellation data appears only when a caller disconnects before response headers are sent.

This dashboard reads gateway metrics with the aisix_ prefix. It does not read the Dev-only aisix_cp_ control-plane metrics.

Count Requests and Attempts Separately

A request and an upstream attempt are different units, and mixing them is the most common source of apparently contradictory numbers. One client request can place several upstream calls: a same-target retry, or a failover to the next target in a Model Group. AISIX counts both units, in different places.

The consequence worth internalizing: a failed attempt that a fallback rescued is invisible in the request counters. The caller received a 200, so the request counter records status="200" and nothing else. The 502 the first target returned lives in the deployment counters and in the usage log, as its own attempt.

So a gateway fronting one broken target in an otherwise healthy Model Group can legitimately show tens of thousands of 5xx rows in the usage log while sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h])) returns a few hundred. The log is counting attempts that failed; the metric is counting callers who saw a failure. Neither is wrong, and they are not expected to converge.

Use these queries for the questions they actually answer:

# Requests that ended in a server error — what callers experienced.
sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h]))

# Requests that ended in a server error even after failing over.
sum(increase(aisix_proxy_requests_total{status=~"5..", is_fallback="true"}[24h]))

# Upstream attempts that failed, by target — including the ones a fallback
# rescued. This is the attempt-level view of a 5xx usage-log row count,
# restricted to the endpoints that dispatch through a Model Group.
sum(increase(aisix_deployment_failure_responses_total[24h])) by (model)

# Fallbacks that rescued a request, by group and by the target reached.
sum(increase(aisix_routing_successful_fallbacks_total[24h])) by (model, fallback_model)

# Usage-event emission attempts, and those the handoff queue rejected.
sum(increase(aisix_usage_events_emitted_total{status_code="5xx"}[24h]))
sum(increase(aisix_usage_event_drops_total[24h]))

# One member's rate-limit rejections. `status` carries the raw HTTP code,
# so a single failure mode is addressable without scanning a whole family.
sum(increase(aisix_usage_events_emitted_total{user_id="<member-id>", status="429"}[24h]))

# Whose queue handoffs were rejected. Both counters carry the same model and
# provider-key labels, so the difference holds per model, not only in total.
sum(increase(aisix_usage_event_drops_total[24h])) by (model, provider_key_name)

Before concluding that two sources disagree, confirm they cover the same population:

• Scrape coverage. Request counters carry no environment label, so a per-environment figure from usage records is not comparable with a gateway-wide PromQL result unless every gateway instance serving that environment is scraped. sum by (job, instance) (...) shows which instances contributed; compare that against the instances actually running.
• Window coverage. increase(...[24h]) measures only the samples present in the range. If instances restarted partway through the window, or Prometheus retention is shorter than the window, the result covers less time than the usage-log filter does. Graphing the raw counter over the same range makes both gaps visible.
• Delivery. aisix_usage_event_drops_total measures events rejected at the producer-to-worker queue handoff. After the queue accepts an event, delivery can still fail without incrementing this counter, so it does not bound the difference between emission attempts and records in final storage. Group queue drops by model. For later failures, monitor telemetry batch failed (events dropped) for control-plane delivery and sink delivery dropped after retries for exporter delivery.
• Attempts that never left the gateway. The deployment families count upstream calls, so an attempt that produced no upstream call is deliberately absent from them: one the target's own rate limits refused, and one rejected while the request was still being assembled — an unusable credential, a missing model_name, a missing or malformed api_base. Those attempts still appear in the usage log and in aisix_usage_events_emitted_total. This is therefore one of the reasons a deployment attempt count runs below a usage-log attempt count, not the whole of it: the scope noted above is another (the deployment families cover only endpoints dispatched through a Model Group, while usage events also cover direct models), as are the scrape, window, and delivery gaps in this list. Isolate one contributor at a time rather than reading the whole difference as gateway-side rejections. The exclusion does have one unambiguous signature: a misconfigured target produces failed requests with no deployment failures at all, because the provider was never asked anything.

Collect Access Logs

Access logs describe an individual proxy request. AISIX writes them through the process logger to the standard error stream, so they appear in the container or process logs collected by your runtime.

Configure process logging in the startup configuration:

observability:
  log_level: "info"

The RUST_LOG environment variable can override the configured log level.

One access log entry is written per proxy request, whether it succeeded, failed, or stopped early. It includes fields such as method, path, status, latency and duration, provider, model, API key ID, request ID, token counts, cache verdict, and routing outcome when those values are available.

A non-streamed request is written when the request ends, so its entry carries everything the gateway resolved. A streamed request writes its entry when the stream ends rather than when the response opens, so the entry carries the same final status as the rest of the request and includes the token counts and provider response ID once the upstream has produced them — a gateway older than 1.3.0 wrote the streamed entry at the response head instead, before those fields existed, which is why an upgrade changes what a streamed line reports even though the request itself did not change.

Read the Two Timing Fields

Every entry carries two durations that answer different questions:

latency_ms is never greater than duration_ms, and the two coincide on a buffered response, where nothing follows the head. Their difference is the length of the stream: use latency_ms for the caller's experience and duration_ms for how long a connection was held open. /a2a and the audio transcription relays measure latency_ms over the whole call, so the two agree there by design; /v1/audio/speech and /v1/videos/{id}/content meter at their tail and relay bytes afterward, so duration_ms ends at the response head instead of the last byte.

Identify the Dispatched Target and the Cache Verdict

Once the gateway selects a target, the entry carries upstream_model and provider_key_id beside model. model keeps its existing meaning — the entry the caller addressed, which is the group name on a routing request — while the new pair names the target actually dispatched to and the provider key used to reach it. Both are omitted when no target was selected, such as a request refused before dispatch or abandoned before routing resolved.

On /v1/chat/completions, the entry also reports how the response cache answered: cache_status (disabled, miss, hit, or bypass) and, on a hit, cache_hit_layer (exact or semantic). A cache hit dispatches to nothing, so what upstream_model, provider_key_id, and provider report on a hit depends on what the addressed entry can claim without a dispatch: a direct model reports its own static mapping, but a Model Group hit — routing or semantic — has no mapping of its own and no record of which target produced the stored response, so upstream_model and provider_key_id are omitted and provider reports unknown rather than whichever candidate the strategy happened to rank first. This also applies to the Prometheus labels built from those fields, so a per-provider panel stops crediting a group's cache hits to a provider that never served them. A gateway at 1.2.0 or earlier writes no cache fields at all, and attributed a Model Group's cache hit to its first-ranked candidate.

When the gateway has it by the time the entry is written, the entry also carries provider_request_id: the response object ID the provider returned, such as an OpenAI chat.completion.id, an Anthropic message id, or a Responses API resp_…. This is the ID a provider's own console and support channel index a call by. The field is omitted rather than blank whenever there is none to record, so you can filter on its presence. Besides the streamed case above, that covers a rejection before dispatch such as a guardrail block, a response served from cache, and the endpoints whose provider response carries no ID at all, including embeddings, audio, and image generation.

For the calls whose ID cannot reach the access log entry, the gateway emits a separate provider call completed entry, carrying request_id, attempt_index, attempt_kind, and provider_request_id. One is written per provider call that returned an ID — so a streamed response produces one, and a request that failed over part-way through a stream produces one per provider call it made. Join them to the access log entry by request_id, and tell one call apart from another within a retried or failed-over request by attempt_index.

Two further fields describe the connection a request arrived on rather than the request itself. They are attached to the request as a whole, so they appear on the access log entry, on the provider call completed entry, and on every diagnostic line the request emits between them:

Both are recorded rather than declared: a request that carried neither logs neither field instead of an empty one, so you can filter on presence.

peer is not the caller's IP address, and substituting one for the other loses the point of both. The gateway resolves the caller's address separately, following proxy.real_ip when it sits behind trusted proxies, and uses it for access control and usage records; that value carries no port. peer is the remote end of the TCP connection the request physically arrived on, and its port is the whole of its value. Behind a layer-4 load balancer, with the gateway on the host network, peer is the only field that joins a gateway log line to the fronting proxy's own record of the same connection.

A streamed response is the one case where that attachment could quietly lapse: its body is written after the point at which a request's log context normally ends. The streaming paths carry that context forward explicitly, so the lines written at the end of a stream — provider call completed among them — carry the same request_id, peer, and downstream_request_id as the rest of the request.

Diagnose a Cancelled or Disconnected Request

A caller that hangs up before the request completes still produces an access log entry and a usage event, with status 499, error_kind="client_disconnected" (error_class on the usage event), zero tokens, and zero cost — on every metered endpoint, not only /v1/chat/completions. The message distinguishes where the disconnect happened:

Only the first two cases are counted by aisix_proxy_client_cancelled_requests_total, which records what the request-level guard observed; a disconnect during an in-flight stream does not raise that counter, though it still files its own 499 entry and usage event. A retried or failed-over request that a caller abandons also files one terminal 499 event alongside one event per attempt that had already failed, naming the attempt that was in flight.

These usage rows did not exist on a gateway older than 1.3.0, which left an access log entry but nothing in the usage log for an abandoned request. Because AISIX Enterprise's request count and success rate are both derived from usage events, upgrading to 1.3.0 makes an environment's request count rise and its success rate fall — the new rows carry zero cost and are filterable by error_class = "client_disconnected", and latency percentiles are unaffected because they already exclude non-2xx rows.

A request refused for exceeding proxy.request_body_limit_bytes produces a second entry on the aisix::body_limit target, carrying declared_content_length, configured_limit_bytes, drained_bytes, and drain_outcome. Join it to the access log entry by request_id.

drain_outcome reports how the gateway finished reading the body it refused, which determines what the caller saw. completed means the caller sent everything it declared and could read the 413. cap_reached, timeout, and client_read_error mean the read stopped early, so the caller usually sees a closed connection instead. The first is logged at info; the other three are logged at warn and are rate limited to one entry per outcome per second, so use aisix_proxy_request_body_limit_rejections_total for volume.

The access_log field is currently reserved and has no effect. Proxy handlers still emit structured access logs, and there is no separate access-log format or sink setting. Collect the standard error stream with your runtime log pipeline when logs need to leave the gateway host.

Correlate Responses with Telemetry

Response headers provide caller-visible correlation and routing hints. They can identify the request, cache outcome, retry timing, or selected target on supported paths.

Use the request ID and other supported response headers to join a caller-visible response to access logs or exported records. For the header scope on each proxy route, see Headers and Error Codes.

Up to three IDs travel with a call, and none of them substitutes for another:

A caller reporting a problem normally has request_id, which the gateway returns on every response. Look that request up, read provider_request_id from the attempt that served it, and take that to the provider. A caller who kept the whole response body may be able to read the provider's ID out of it directly, but only on the endpoints that return one.

downstream_request_id is recorded, never adopted. Whether a caller-supplied ID becomes the gateway's own request_id is a separate decision, governed by proxy.request_id.accept_headers — which by default accepts only x-aisix-request-id. So a gateway behind an ingress normally logs two IDs that mean different things, and neither can be mistaken for the other. It is screened by the same rule as an adopted ID (see Accepted Values) and omitted when the header is absent or unusable.

Reuse Your Own Request ID

If your service already generates a request ID for the business call, send it and AISIX adopts it instead of generating one. That ID then becomes the request's identity everywhere: the x-aisix-request-id response header, the request_id on the access log and on every usage event the request produces, and the x-aisix-request-id the provider receives.

Every trail you use to investigate a request is then keyed by an ID your own application logs already carry. That covers the gateway access log, exported usage events, and AISIX Enterprise Request Logs, with no second mapping to maintain.

Send it in x-aisix-request-id:

# AISIX_PROXY is the gateway origin; omit a trailing slash and endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

curl "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-aisix-request-id: req_abc123-orders-svc" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}' -i

The response echoes the same value:

HTTP/1.1 200 OK
x-aisix-request-id: req_abc123-orders-svc

A request that retries or fails over emits one usage event per attempt. All carry your ID, so filtering on it returns the whole chain rather than only the attempt that succeeded.

Accepted Values

An ID is used as sent when it is 1–256 bytes and contains only visible ASCII characters (! through ~): no spaces, no control characters, no non-ASCII. Qualifying shapes include a UUID, a ULID, a prefixed ID such as req_abc123, and the hexadecimal ID nginx puts in $request_id.

A value outside that range is ignored and AISIX generates a UUID instead, which is the behavior when no ID is sent at all. The request itself is never rejected over its correlation ID.

AISIX does not require IDs to be unique. Sending the same ID for two different requests makes them indistinguishable in every trail that keys on it, so generate a fresh one per request.

Choose the Accepted Headers

By default AISIX reads only its own x-aisix-request-id. Set proxy.request_id.accept_headers to change that:

proxy:
  request_id:
    accept_headers: ["x-aisix-request-id", "x-request-id"]

Headers are consulted in the order listed, and the first acceptable value wins, so the list is also a priority order. Environment-only deployments set the list comma-separated as AISIX_PROXY__REQUEST_ID__ACCEPT_HEADERS.

x-request-id is not accepted by default on purpose. Reverse proxies, ingress controllers, and load balancers stamp that header on every request they forward. Enabling it behind one of those means the correlation ID comes from your infrastructure rather than from the calling service. Add it when AISIX is the first hop, or when the ID your proxy assigns is the one you want to trace by.

Accepting it is not what makes it visible, though. An acceptable x-request-id is recorded as downstream_request_id on the request's log lines either way, so the infrastructure's own ID is available to search on without changing this setting. Listing it here does something further: it makes that value the request's identity everywhere — the response header, the access log's request_id, and every usage event — at which point request_id and downstream_request_id hold the same value, which is the honest report of what happened.

Set accept_headers: [] to ignore caller-supplied IDs entirely and always generate one.

A header name that is not a valid HTTP header name fails startup rather than being silently skipped.

Export Usage Events

Usage events are per-attempt records emitted by supported proxy paths. A request that retries or fails over emits multiple events with the same request_id, ordered by attempt_index. Counting these records therefore counts attempts, not requests — see Count Requests and Attempts Separately before comparing a record count against a request metric. Each event includes its outcome, consumption details, the model alias the caller requested, and the resolved model that served the attempt when the gateway can observe those values. A response served from the response cache records the matching layer in cache_hit_layer (exact or semantic) and, for semantic hits, the matched similarity in cache_similarity.

Latency fields distinguish provider time from caller-visible time:

A caller that disconnects part-way through a streamed response still produces a usage event, because the upstream did the work and may have charged for it. That event carries status 499 rather than 200, and its token counts cover only what arrived before the disconnect. Count streamed successes with status_code = 200 to keep abandoned responses out of the total.

Tell Request Kinds Apart

Every event carries operation, the kind of work the request asked for. Without it, one field says which protocol addressed the gateway and another says which model answered. Neither says whether the call was a conversation, an image, or a video. Every OpenAI-compatible endpoint reports the same inbound_protocol, so a chat completion, an image generation, and a video submission arrive as one undifferentiated stream.

The value comes from the endpoint the request matched, so it is a small fixed set safe to index, group, and chart:

Three points are worth knowing before you write a query against it:

• It describes the request, not the outcome. A request that failed, or that a guardrail refused, carries the same value a successful one would. It is the only field on such a record that names the endpoint at all.
• It is request-scoped. A request that retries or fails over emits one event per attempt, and every attempt carries the same value, so counting events per operation counts attempts. See Count Requests and Attempts Separately.
• Polling a video job is not video generation. Only the submission (POST /v1/videos) produces a usage event; retrieving the job's status or downloading its result does not. A count of video_generation is therefore a count of videos asked for, not of requests made about them.

The field is part of the record's metadata, so it is present under both content modes. An exporter configured for metadata_only — which never receives a prompt — can still separate traffic by kind, which content inspection could not do there at all.

In Alibaba Cloud SLS, the field arrives as its own column and needs no parsing. Enable analytics for the fields a query names, since SQL analysis reads indexed fields only:

* | SELECT operation, COUNT(*) AS calls, SUM(prompt_tokens + completion_tokens) AS tokens GROUP BY operation ORDER BY calls DESC

To pull one kind of traffic, filter on it directly — operation: video_generation — instead of matching a model name or searching the prompt. A model name is a poor substitute here. One model can serve several endpoints, and callers address models through aliases and model groups, so requested_model answers "which configured entry," not "what kind of call."

Attribute Events to a Member

Each event carries user_id: the organization member who owned the API key the request authenticated with, assigned on the key itself. It is absent for keys that belong to no member — ownership is set explicitly when a key is created or edited, not inferred from whoever created it.

One member usually holds several credentials, and this field is what makes them one identity. A JWT-authenticated request runs as the API key its identity resolves to, so a member calling through both an API key and an OIDC token produces events under two different api_key_id values and the same user_id. Filtering on the member returns the whole picture; filtering per key returns a slice of it.

The value is a snapshot taken when the request ran, not a lookup performed when you read the event. Reassigning a key to a different member changes who later requests are attributed to and leaves earlier events attributed to the previous owner, which is what makes a historical query answerable. Deleting the key does not erase the attribution of the events it already produced.

Events written before a data plane that records this field carry no member, so a member filter covers traffic from that upgrade forward.

In the dashboard, Logs offers this as the Member filter, alongside a Status filter that accepts a family (4xx), an exact code (429), or a range (500-599). Combining the two answers questions like "which of this member's requests were rate-limited in the last 24 hours" in one query. The CSV export carries user_id and the member's name.

Usage events are consumed through a sink rather than read from a local endpoint. Configure an observability exporter to deliver them to OTLP/HTTP, object storage, Alibaba Cloud SLS, or Datadog.

OpenAI Cache-Write Tokens

OpenAI can report cache_write_tokens inside Chat Completions usage.prompt_tokens_details or Responses usage.input_tokens_details. AISIX preserves that raw value as the optional cache_write_tokens field of the usage event, including requests bridged through /v1/messages or /v1/responses and supported protocol-aware passthrough routes. This requires a gateway newer than 1.1.0.

{
  "prompt_tokens": 101,
  "completion_tokens": 11,
  "cached_prompt_tokens": 19,
  "cache_write_tokens": 37
}

The field is omitted when the upstream did not report it; an explicit 0 remains zero. The AISIX Enterprise Logs detail, usage-events API, and JSON/CSV export preserve that distinction. CSV uses an empty cell for an absent value. Logs exported through a backend with prefixed field names use its usual prefix, such as aisix.cache_write_tokens in Datadog.

This is separate from Anthropic's additive cache_creation_tokens. It does not increase input/output totals or change the existing cost calculation. In the example above, input plus output is still 112. No existing cache counter is renamed or merged.

Next Steps

Configure Observability Exporters to send usage events to an external collector, log destination, object store, or warehouse workflow. Use the Metrics Reference when building Prometheus dashboards or alerts.