Observability runbook
Operate Apache APISIX with metrics, logs, traces, and alerts across the gateway, NGINX, Lua, and etcd.
This runbook maps APISIX production observability to the golden signals: request rate, errors, latency, saturation, and configuration sync health. Use it to build dashboards, write alerts, and triage incidents.
Golden metrics
[TODO] Define the minimal set of metrics every APISIX operator should monitor (request rate, error rate, p99 latency, upstream latency, CPU/memory, etcd health).
APISIX and NGINX metrics
[TODO] List key metrics from apisix and nginx metric families (requests, upstream, ssl, etc.).
Lua and plugin metrics
[TODO] Document how to expose and interpret Lua-level metrics, including plugin execution time and custom metrics.
etcd metrics
[TODO] List etcd metrics relevant to APISIX (leader, raft, disk, network, watch, health), and their alert thresholds.
Common alerts
5xx rate
[TODO] Add PromQL example and threshold guidance for gateway-level 5xx.
Latency
[TODO] Add latency alerts by percentile and by upstream.
Connection and configuration sync
[TODO] Add alerts for connection exhaustion and configuration synchronization failures between APISIX and etcd.
Logs and traces
[TODO] Describe how to correlate log fields with trace IDs and metrics.
PromQL examples
[TODO] Add PromQL examples for the alerts above, with versioned metric names.