Proxy Transport Layer (L4) Traffic

Configure an Apache APISIX TCP stream route, then extend it with PROXY protocol, upstream mTLS, or weighted stream traffic.

Deployments:DockerRoles:Platform engineerTime:25 minutesImpact:Restarts the data plane

Goal

Enable APISIX transport-layer proxying on a disposable quickstart, expose a MySQL server through a TCP Stream Route, and verify the proxied connection. Optional sections show supported TCP and UDP listener shapes, per-listener PROXY protocol controls, client-certificate authentication to a TLS upstream, and deterministic Stream traffic splitting.

APISIX can serve Layer 4 and Layer 7 traffic at the same time. A Stream Route selects a TCP or UDP listener by server_port; its upstream scheme controls the protocol APISIX uses toward the upstream.

Before you begin

Complete Get APISIX on a machine with Docker. This procedure is limited to a dedicated apache/apisix:3.18.0-ubuntu quickstart container because it temporarily edits and reloads that container's configuration.
Install a MySQL client, cURL, jq, and OpenSSL.
Stop if this is a shared or production gateway. Merge the listener through that deployment's configuration-management workflow instead of editing a running container.

Set the variables used in this guide. Generate a password instead of placing a reusable password in shell history:

export APISIX_ADMIN_URL="http://127.0.0.1:9180"
export STREAM_GATEWAY_HOST="127.0.0.1"
export STREAM_GATEWAY_PORT="9100"
export STREAM_MYSQL_PASSWORD="$(openssl rand -hex 18)"
read -rsp "Admin API key (press Enter for an unchanged quickstart): " APISIX_ADMIN_KEY && export APISIX_ADMIN_KEY && echo

Confirm the expected quickstart image, TCP publication, Docker network, and Admin API access:

docker inspect apisix-quickstart | jq -e '.[0].Config.Image == "apache/apisix:3.18.0-ubuntu"'
docker network inspect apisix-quickstart-net >/dev/null
docker port apisix-quickstart 9100/tcp | grep -q ':9100$'
test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' "${APISIX_ADMIN_URL}/apisix/admin/stream_routes" -H "X-API-KEY: ${APISIX_ADMIN_KEY}")" = 200

Protect existing APISIX resources, containers, and local files before using the guide's fixed names:

for resource in stream_routes/stream-guide-mysql upstreams/stream-guide-mtls upstreams/stream-guide-canary ssls/stream-guide-client; do
  test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' "${APISIX_ADMIN_URL}/apisix/admin/${resource}" -H "X-API-KEY: ${APISIX_ADMIN_KEY}")" = 404 || { echo "Stop: ${resource} already exists"; exit 1; }
done
for container in stream-guide-mysql stream-guide-mysql-canary stream-guide-mtls-upstream; do
  test -z "$(docker ps -aq --filter "name=^/${container}$")" || { echo "Stop: container ${container} already exists"; exit 1; }
done
test ! -e stream-guide-certs || { echo 'Stop: stream-guide-certs already exists'; exit 1; }
docker exec apisix-quickstart bash -ceu '
  test ! -e /usr/local/apisix/conf/config.yaml.stream-guide-backup
  ! grep -Eq "^[[:space:]]+(proxy_mode|stream_proxy):" /usr/local/apisix/conf/config.yaml
'

The last check intentionally stops if the quickstart already has Layer 4 settings. Preserve and merge existing listeners through the configuration source that owns them.

Steps

Step 1: Enable the transport-layer proxy

Back up the exact quickstart configuration, add one TCP listener, validate the generated NGINX configuration, and reload APISIX:

docker exec -i apisix-quickstart bash -se <<'BASH'
conf=/usr/local/apisix/conf/config.yaml
backup=/usr/local/apisix/conf/config.yaml.stream-guide-backup
cp "$conf" "$backup"
sed -i '/^apisix:$/a\
  proxy_mode: http&stream\
  stream_proxy:\
    tcp:\
      - 9100
' "$conf"
apisix test
apisix reload
BASH

If apisix test fails, restore the backup before continuing:

docker exec apisix-quickstart bash -ceu '
  mv /usr/local/apisix/conf/config.yaml.stream-guide-backup /usr/local/apisix/conf/config.yaml
  apisix test
  apisix reload
'

The tcp and udp arrays also accept bound addresses and port ranges. TCP listeners accept object entries for TLS termination and per-listener PROXY protocol controls:

config.yaml fragment

apisix:
  proxy_mode: http&stream
  stream_proxy:
    tcp:
      - 9100
      - "127.0.0.1:9101"
      - "9200-9210"
      - addr: "127.0.0.1:9300-9310"
        tls: true
    udp:
      - 9400
      - "127.0.0.1:9401"
      - "9500-9510"

Quote ranges so YAML preserves them as strings. Every configured listener must also be published by the container runtime, Kubernetes Service, firewall, and load balancer using the correct protocol. The current quickstart publishes 9100/TCP; it does not publish the example UDP ports. Docker cannot add a missing publication to a running container.

Step 2: Start the MySQL upstream

Start MySQL on the quickstart network and wait up to 60 seconds for it to accept connections:

docker run -d --name stream-guide-mysql --hostname stream-guide-primary \
  --network apisix-quickstart-net \
  -e "MYSQL_ROOT_PASSWORD=${STREAM_MYSQL_PASSWORD}" \
  -e MYSQL_ROOT_HOST=% \
  mysql:8.4
for attempt in {1..30}; do
  docker exec -e "MYSQL_PWD=${STREAM_MYSQL_PASSWORD}" stream-guide-mysql \
    mysqladmin ping --connect-timeout=2 -h 127.0.0.1 -uroot --silent && break
  sleep 2
done
docker exec -e "MYSQL_PWD=${STREAM_MYSQL_PASSWORD}" stream-guide-mysql \
  mysqladmin ping --connect-timeout=2 -h 127.0.0.1 -uroot --silent

Step 3: Create the Stream Route

Create a Stream Route at a fixed resource URL. The listener port selects this route and APISIX reaches MySQL over its Docker network:

curl -sS --connect-timeout 3 --max-time 15 "${APISIX_ADMIN_URL}/apisix/admin/stream_routes/stream-guide-mysql" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H "Content-Type: application/json" \
  -d '{
    "server_port": 9100,
    "upstream": {
      "scheme": "tcp",
      "type": "roundrobin",
      "nodes": {"stream-guide-mysql:3306": 1}
    }
  }'

Step 4: Configure PROXY protocol when required

This step is optional. APISIX can consume a PROXY header from a TCP load balancer and send a new PROXY header to a compatible upstream. Global settings provide defaults; object entries override either direction on one listener:

config.yaml fragment

apisix:
  proxy_mode: http&stream
  proxy_protocol:
    enable_tcp_pp: true
    enable_tcp_pp_to_upstream: true
  stream_proxy:
    tcp:
      - addr: 9100
        proxy_protocol: false
        proxy_protocol_to_upstream: false
      - addr: 9200

nginx_config:
  stream:
    real_ip_from:
      - 192.168.1.0/24
      - 2001:db8:1234::/48

Setting	Direction	Effect
`proxy_protocol`	Load balancer to APISIX	Makes the TCP listener consume the inbound PROXY header.
`proxy_protocol_to_upstream`	APISIX to upstream	Sends a PROXY header before the proxied stream data.

Limit real_ip_from to load balancers you control. Trusting a network lets any peer in that network claim an arbitrary client address. For a trusted peer, $remote_addr, Stream access logs, and address-based plugins use the address from the header; $realip_remote_addr retains the directly connected peer.

These settings apply only to TCP. Do not enable either direction until every affected client, load balancer, and upstream is compatible. A normal MySQL or TLS endpoint usually closes a connection if it receives an unexpected plaintext PROXY line.

Step 5: Authenticate to a TLS upstream with mTLS

This optional extension proves APISIX client authentication to a Stream TLS upstream. Generate a short-lived test CA plus server and client certificates:

mkdir stream-guide-certs
export STREAM_GUIDE_CERT_DIR="$(pwd)/stream-guide-certs"
openssl genrsa -out stream-guide-certs/ca.key 2048
openssl req -x509 -new -sha256 -days 30 -key stream-guide-certs/ca.key -out stream-guide-certs/ca.crt \
  -subj '/CN=Stream Guide CA' \
  -addext 'basicConstraints=critical,CA:TRUE' \
  -addext 'keyUsage=critical,keyCertSign,cRLSign'

openssl genrsa -out stream-guide-certs/server.key 2048
openssl req -new -sha256 -key stream-guide-certs/server.key -out stream-guide-certs/server.csr -subj '/CN=stream-guide-mtls-upstream'
printf 'extendedKeyUsage=serverAuth\nkeyUsage=digitalSignature,keyEncipherment\n' > stream-guide-certs/server.ext
openssl x509 -req -sha256 -days 30 -in stream-guide-certs/server.csr \
  -CA stream-guide-certs/ca.crt -CAkey stream-guide-certs/ca.key -CAcreateserial \
  -extfile stream-guide-certs/server.ext -out stream-guide-certs/server.crt

openssl genrsa -out stream-guide-certs/client.key 2048
openssl req -new -sha256 -key stream-guide-certs/client.key -out stream-guide-certs/client.csr -subj '/CN=apisix-stream-client'
printf 'extendedKeyUsage=clientAuth\nkeyUsage=digitalSignature,keyEncipherment\n' > stream-guide-certs/client.ext
openssl x509 -req -sha256 -days 30 -in stream-guide-certs/client.csr \
  -CA stream-guide-certs/ca.crt -CAkey stream-guide-certs/ca.key -CAcreateserial \
  -extfile stream-guide-certs/client.ext -out stream-guide-certs/client.crt
openssl verify -CAfile stream-guide-certs/ca.crt -purpose sslclient stream-guide-certs/client.crt

Start an upstream that requires a client certificate signed by the test CA:

docker run -d --name stream-guide-mtls-upstream --network apisix-quickstart-net \
  -v "${STREAM_GUIDE_CERT_DIR}:/certs:ro" alpine/openssl \
  s_server -accept 9443 -cert /certs/server.crt -key /certs/server.key \
  -CAfile /certs/ca.crt -Verify 1 -www

Create a client SSL resource, create the TLS Upstream, and temporarily point the Stream Route to it:

jq -n --rawfile cert stream-guide-certs/client.crt --rawfile key stream-guide-certs/client.key \
  '{type: "client", cert: $cert, key: $key}' | \
curl -sS --connect-timeout 3 --max-time 15 "${APISIX_ADMIN_URL}/apisix/admin/ssls/stream-guide-client" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H "Content-Type: application/json" --data-binary @-

curl -sS --connect-timeout 3 --max-time 15 "${APISIX_ADMIN_URL}/apisix/admin/upstreams/stream-guide-mtls" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H "Content-Type: application/json" \
  -d '{
    "scheme": "tls",
    "type": "roundrobin",
    "nodes": {"stream-guide-mtls-upstream:9443": 1},
    "tls": {"client_cert_id": "stream-guide-client"}
  }'

curl -sS --connect-timeout 3 --max-time 15 "${APISIX_ADMIN_URL}/apisix/admin/stream_routes/stream-guide-mysql" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H "Content-Type: application/json" \
  -d '{"server_port": 9100, "upstream_id": "stream-guide-mtls"}'

Send HTTP plaintext to the downstream TCP listener. APISIX creates the upstream TLS connection and presents its client certificate:

test "$(curl -sS --connect-timeout 3 --max-time 15 -o /dev/null -w '%{http_code}' "http://${STREAM_GATEWAY_HOST}:${STREAM_GATEWAY_PORT}/")" = 200

This validates client-certificate authentication only. The generic Stream tls upstream does not use upstream.tls.verify to validate the upstream server certificate; that option currently applies only to Kafka upstreams. Use a trusted network boundary if server identity must also be enforced.

You can configure upstream.tls.client_cert and client_key inline instead. When data encryption is enabled, APISIX encrypts the inline private key before storage. A Stream Route GET response does not return that key in plaintext. A client SSL resource can also hold $secret://... or $env://... references in cert and key and be selected with client_cert_id.

Step 6: Split Stream traffic

This optional extension restores the MySQL upstream and proves deterministic selection of a canary. Start a second MySQL server with the same test credential:

docker run -d --name stream-guide-mysql-canary --hostname stream-guide-canary \
  --network apisix-quickstart-net \
  -e "MYSQL_ROOT_PASSWORD=${STREAM_MYSQL_PASSWORD}" \
  -e MYSQL_ROOT_HOST=% \
  mysql:8.4
for attempt in {1..30}; do
  docker exec -e "MYSQL_PWD=${STREAM_MYSQL_PASSWORD}" stream-guide-mysql-canary \
    mysqladmin ping --connect-timeout=2 -h 127.0.0.1 -uroot --silent && break
  sleep 2
done
docker exec -e "MYSQL_PWD=${STREAM_MYSQL_PASSWORD}" stream-guide-mysql-canary \
  mysqladmin ping --connect-timeout=2 -h 127.0.0.1 -uroot --silent

Create the canary Upstream and replace the Stream Route with a traffic-split rule. A weight-only entry represents the route's default upstream; weights 1:0 make validation deterministic:

curl -sS --connect-timeout 3 --max-time 15 "${APISIX_ADMIN_URL}/apisix/admin/upstreams/stream-guide-canary" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H "Content-Type: application/json" \
  -d '{"scheme":"tcp","type":"roundrobin","nodes":{"stream-guide-mysql-canary:3306":1}}'

curl -sS --connect-timeout 3 --max-time 15 "${APISIX_ADMIN_URL}/apisix/admin/stream_routes/stream-guide-mysql" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H "Content-Type: application/json" \
  -d '{
    "server_port": 9100,
    "plugins": {
      "traffic-split": {
        "rules": [{
          "match": [{"vars": [["route_id", "==", "stream-guide-mysql"]]}],
          "weighted_upstreams": [
            {"upstream_id": "stream-guide-canary", "weight": 1},
            {"weight": 0}
          ]
        }]
      }
    },
    "upstream": {
      "scheme": "tcp",
      "type": "roundrobin",
      "nodes": {"stream-guide-mysql:3306": 1}
    }
  }'

For an approximate 10% canary share, use weights 1 and 9. Finite samples do not guarantee an exact ratio.

Validate

Use the check that matches the last step you completed. If you stopped after Step 3, query the primary MySQL server through APISIX and require its configured hostname:

export MYSQL_PWD="${STREAM_MYSQL_PASSWORD}"
MYSQL_HOSTNAME=$(mysql --connect-timeout=5 --protocol=TCP -h "${STREAM_GATEWAY_HOST}" -P "${STREAM_GATEWAY_PORT}" -u root -Nse 'SELECT @@hostname;')
test "${MYSQL_HOSTNAME}" = "stream-guide-primary"
printf 'Stream Route selected: %s\n' "${MYSQL_HOSTNAME}"
unset MYSQL_PWD

If you completed Step 5 but not Step 6, the HTTP 200 assertion in Step 5 is the mTLS validation; the route no longer targets MySQL, so skip both MySQL checks here.

If you completed Step 6, query MySQL and require the canary:

export MYSQL_PWD="${STREAM_MYSQL_PASSWORD}"
MYSQL_HOSTNAME=$(mysql --connect-timeout=5 --protocol=TCP -h "${STREAM_GATEWAY_HOST}" -P "${STREAM_GATEWAY_PORT}" -u root -Nse 'SELECT @@hostname;')
test "${MYSQL_HOSTNAME}" = "stream-guide-canary"
printf 'traffic-split selected: %s\n' "${MYSQL_HOSTNAME}"
unset MYSQL_PWD

Roll back / Clean up

Delete the Stream Route before its optional Upstreams and client SSL resource, remove disposable containers and certificate files, then restore the exact APISIX configuration backup:

curl -sS --connect-timeout 3 --max-time 15 -X DELETE "${APISIX_ADMIN_URL}/apisix/admin/stream_routes/stream-guide-mysql" -H "X-API-KEY: ${APISIX_ADMIN_KEY}"
for id in stream-guide-canary stream-guide-mtls; do
  curl -sS --connect-timeout 3 --max-time 15 -X DELETE "${APISIX_ADMIN_URL}/apisix/admin/upstreams/${id}" -H "X-API-KEY: ${APISIX_ADMIN_KEY}"
done
curl -sS --connect-timeout 3 --max-time 15 -X DELETE "${APISIX_ADMIN_URL}/apisix/admin/ssls/stream-guide-client" -H "X-API-KEY: ${APISIX_ADMIN_KEY}"

for container in stream-guide-mysql-canary stream-guide-mtls-upstream stream-guide-mysql; do
  if docker container inspect "${container}" >/dev/null 2>&1; then docker rm -f "${container}"; fi
done
rm -rf stream-guide-certs

docker exec apisix-quickstart bash -ceu '
  mv /usr/local/apisix/conf/config.yaml.stream-guide-backup /usr/local/apisix/conf/config.yaml
  apisix test
  apisix reload
'
unset APISIX_ADMIN_URL APISIX_ADMIN_KEY STREAM_GATEWAY_HOST STREAM_GATEWAY_PORT STREAM_MYSQL_PASSWORD STREAM_GUIDE_CERT_DIR MYSQL_PWD MYSQL_HOSTNAME

If you merged additional TCP, UDP, TLS, or PROXY protocol listeners through a managed deployment, remove only the entries created for this guide and verify that deployment's rollout.

Troubleshoot

Symptom	Cause	Fix
The client cannot connect to port 9100.	APISIX did not load the Stream listener, or the TCP port is not published through every network layer.	Run `apisix test`, inspect the running configuration, and verify the container, Service, load balancer, and firewall all expose `9100/TCP`.
A UDP client times out.	A TCP publication was mistaken for UDP, or no UDP Stream Route uses that listener.	Publish the exact port as UDP before creating the container or Service, then bind a UDP upstream and Stream Route to the same `server_port`.
MySQL reports an authentication or connection error.	The password differs, MySQL is not ready, or APISIX cannot resolve the container name.	Repeat the bounded `mysqladmin ping`, inspect `docker logs stream-guide-mysql`, and confirm both containers share `apisix-quickstart-net`.
A listener fails immediately after enabling PROXY protocol.	A direct client or upstream does not send or accept the expected PROXY header.	Revert the per-listener flags and introduce PROXY protocol only across a fully compatible path.
The mTLS request closes without a response.	The client SSL resource is missing, has the wrong type, or the upstream does not trust its issuer.	Inspect both APISIX and `stream-guide-mtls-upstream` logs, verify the client certificate against the CA, and confirm `client_cert_id`.
Traffic splitting still reaches the primary.	The rule did not match or the weight-only default entry is nonzero.	Read back the Stream Route, confirm `route_id` and both weighted entries, and use `1:0` only for deterministic validation.