Log Requests to Elasticsearch

Send an allowlisted Apache APISIX access-log record to a pinned local Elasticsearch server, verify the indexed document, and clean up.

Deployments:DockerRoles:Platform engineerTime:20 minutesImpact:Changes live traffic

Goal

Enable elasticsearch-logger on one isolated Route and index an explicitly allowlisted four-field access-log document in a controlled Elasticsearch node. Validate the exact Route ID in the indexed document, then remove every resource.

Access logs can contain personal data, credentials, or regulated identifiers. Keep request and response bodies disabled, allowlist only necessary fields, and apply encryption, least privilege, retention, and deletion controls before adapting this local example.

Before you begin

Complete Get APISIX with the Docker quickstart.
Install Docker, cURL, and jq.
Use a shell that supports Bash or Zsh heredocs.
Reserve enough local memory for a disposable single-node Elasticsearch server. This is not a production topology.

Set the endpoints and read secrets without placing them in shell history:

export APISIX_ADMIN_URL="http://127.0.0.1:9180"
export GATEWAY_URL="http://127.0.0.1:9080"
export GUIDE_DIR="$(pwd)"
read -rsp "Admin API key (press Enter for an unchanged quickstart): " APISIX_ADMIN_KEY && export APISIX_ADMIN_KEY && echo
read -rsp "Temporary Elasticsearch password: " ELASTIC_PASSWORD && export ELASTIC_PASSWORD && echo
test -n "${ELASTIC_PASSWORD}"

Protect the fixed Route, containers, and local file used by this guide:

test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' "${APISIX_ADMIN_URL}/apisix/admin/routes/elasticsearch-guide-route" -H "X-API-KEY: ${APISIX_ADMIN_KEY}")" = 404 || { echo 'Stop: route elasticsearch-guide-route already exists'; exit 1; }
for container in elasticsearch-guide-server elasticsearch-guide-upstream; do
  test -z "$(docker ps -aq --filter "name=^/${container}$")" || { echo "Stop: container ${container} already exists"; exit 1; }
done
test ! -e elasticsearch-guide-nginx.conf || { echo 'Stop: elasticsearch-guide-nginx.conf already exists'; exit 1; }

Steps

Step 1: Start a pinned Elasticsearch server

Start a disposable Elasticsearch 9.4.2 node on the APISIX quickstart network. Its API is published only on the host loopback interface:

docker run -d --name elasticsearch-guide-server \
  --network apisix-quickstart-net \
  -p 127.0.0.1:9200:9200 \
  -e discovery.type=single-node \
  -e xpack.security.enabled=true \
  -e xpack.security.http.ssl.enabled=false \
  -e ELASTIC_PASSWORD="${ELASTIC_PASSWORD}" \
  -e ES_JAVA_OPTS="-Xms512m -Xmx512m" \
  docker.elastic.co/elasticsearch/elasticsearch:9.4.2
for attempt in {1..60}; do
  curl --fail --silent --show-error --user "elastic:${ELASTIC_PASSWORD}" \
    --connect-timeout 2 --max-time 5 "http://127.0.0.1:9200" >/dev/null 2>&1 && break
  sleep 2
done
curl --fail --silent --show-error --user "elastic:${ELASTIC_PASSWORD}" \
  --connect-timeout 3 --max-time 10 "http://127.0.0.1:9200" | \
  jq -e '.tagline == "You Know, for Search"'

HTTP TLS is disabled only for this isolated local network. Use HTTPS in production and leave the plugin's ssl_verify option enabled with a trusted certificate chain.

What changed: a fresh authenticated Elasticsearch node is reachable by APISIX and from host loopback, with no persistent data volume.

Step 2: Start a controlled upstream

Create a deterministic NGINX response and start it without publishing an upstream port:

cat > elasticsearch-guide-nginx.conf <<'NGINX'
events {}
http {
  access_log off;
  server {
    listen 80;
    location = /elasticsearch-guide { return 200 '{"source":"elasticsearch-guide-upstream"}'; }
  }
}
NGINX
docker run -d --name elasticsearch-guide-upstream \
  --network apisix-quickstart-net \
  -v "${GUIDE_DIR}/elasticsearch-guide-nginx.conf:/etc/nginx/nginx.conf:ro" \
  nginx:1.29.1-alpine
for attempt in {1..15}; do
  docker exec apisix-quickstart bash -c '</dev/tcp/elasticsearch-guide-upstream/80' >/dev/null 2>&1 && break
  sleep 1
done
docker exec apisix-quickstart bash -c '</dev/tcp/elasticsearch-guide-upstream/80'

What changed: APISIX has one controlled local endpoint for generating a log document.

Step 3: Create the logging Route

Use jq to insert the password into one authenticated fixed-ID Route. A route-local log_format avoids changing other logger instances. batch_max_size: 1 is for deterministic local validation, not a production throughput recommendation:

jq -n --arg password "${ELASTIC_PASSWORD}" '{
  name:"elasticsearch-guide-route",
  uri:"/elasticsearch-guide",
  methods:["GET"],
  plugins:{
    "elasticsearch-logger":{
      endpoint_addrs:["http://elasticsearch-guide-server:9200"],
      field:{index:"apisix-guide"},
      auth:{username:"elastic",password:$password},
      log_format:{host:"$host",client_ip:"$remote_addr",route_id:"$route_id",timestamp:"$time_iso8601"},
      batch_max_size:1,
      inactive_timeout:1,
      buffer_duration:10,
      max_retry_count:2,
      retry_delay:1
    }
  },
  upstream:{type:"roundrobin",nodes:{"elasticsearch-guide-upstream:80":1}}
}' | \
curl -sS --connect-timeout 3 --max-time 15 \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/elasticsearch-guide-route" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H 'Content-Type: application/json' \
  --data-binary @- | \
jq -e '.value.id == "elasticsearch-guide-route" and .value.plugins["elasticsearch-logger"].field.index == "apisix-guide" and .value.plugins["elasticsearch-logger"].batch_max_size == 1'

APISIX encrypts the plugin's password field before storage, but operators with authorized configuration access can still retrieve effective settings. Restrict Admin API access and use an APISIX Secret or environment reference for long-lived production credentials.

What changed: only this Route sends an allowlisted record to the isolated Elasticsearch index.

Validate

Require the controlled upstream response, then poll until Elasticsearch returns exactly one matching document:

curl --fail-with-body -sS --connect-timeout 3 --max-time 10 \
  "${GATEWAY_URL}/elasticsearch-guide" | \
  jq -e '. == {source:"elasticsearch-guide-upstream"}'
for attempt in {1..15}; do
  count="$(curl --fail --silent --show-error --user "elastic:${ELASTIC_PASSWORD}" \
    --connect-timeout 3 --max-time 10 \
    "http://127.0.0.1:9200/apisix-guide/_count?q=route_id:elasticsearch-guide-route" | jq -r '.count')"
  test "${count}" = 1 && break
  sleep 1
done
curl --fail --silent --show-error --user "elastic:${ELASTIC_PASSWORD}" \
  --connect-timeout 3 --max-time 10 \
  "http://127.0.0.1:9200/apisix-guide/_search?q=route_id:elasticsearch-guide-route&size=1" | \
jq -e '.hits.total.value == 1 and .hits.hits[0]._source.route_id == "elasticsearch-guide-route" and (.hits.hits[0]._source.host | length > 0) and (.hits.hits[0]._source.client_ip | length > 0) and (.hits.hits[0]._source.timestamp | length > 0)'

Both assertions must pass. They prove that the intended upstream responded and that the plugin indexed one document with all four allowlisted fields.

Roll back / Clean up

Delete the Route before removing its logging destination and upstream, then remove the local file and secrets from the shell:

curl --fail-with-body -sS --connect-timeout 3 --max-time 15 -X DELETE \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/elasticsearch-guide-route" \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}"
docker rm -f elasticsearch-guide-upstream elasticsearch-guide-server
rm -f elasticsearch-guide-nginx.conf
unset APISIX_ADMIN_URL APISIX_ADMIN_KEY GATEWAY_URL GUIDE_DIR ELASTIC_PASSWORD count

Expected: the fixed Route, both containers, Elasticsearch index data, local file, and shell variables are absent.

Troubleshoot

Symptom	Cause	Fix
Elasticsearch does not become ready.	The image cannot start with the available memory, or the password does not meet local requirements.	Inspect `docker logs elasticsearch-guide-server`; correct the local resource or password issue before creating the Route.
APISIX reports an indexing error.	Authentication failed, the endpoint is unreachable, or Elasticsearch rejected the bulk request.	Verify the exact readiness command, container network, and APISIX error log; recreate the fixed Route from the current password.
The upstream response succeeds but the document does not appear.	The batch worker has not flushed or exhausted its retries.	Inspect APISIX errors and Elasticsearch logs; keep the bounded poll and do not increase the in-memory buffer before finding the delivery failure.
Logging creates excessive load.	A batch size of one sends every entry immediately.	Size `batch_max_size`, `inactive_timeout`, `buffer_duration`, and retry limits from measured production traffic and failure budgets.

Elasticsearch Logger (opens in Plugin Hub docs) — schema, batching, custom formats, and encrypted fields.
APISIX Secrets — reference long-lived logger credentials without embedding them in Route payloads.
Elasticsearch Docker installation — official image and local evaluation guidance.