Upgrade and Roll Back APISIX
Replace a running Apache APISIX release with another one on Docker or Kubernetes, confirm the gateway still serves its routes, and return to the previous release.
Applies to:Apache APISIX 3.18.0
Deployments:DockerKubernetesRoles:Platform engineerSRETime:30 minutesImpact:Restarts the data plane
Goal
Every gateway instance running the APISIX release you chose, with the previous image tag and an etcd snapshot still in hand so that one command returns the deployment to where it started.
This guide replaces the gateway process. It does not migrate your configuration: routes, upstreams, and consumers live in etcd or in a standalone file and are read by the new release as they are. What can change between releases is how the gateway interprets them, which is what Review Changes Before Upgrade is for.
Before you begin
-
The Admin API accepts your key. Check it with:
curl -s -o /dev/null -w '%{http_code}\n' "${APISIX_ADMIN_URL}/apisix/admin/routes" \ -H "X-API-KEY: ${APISIX_ADMIN_KEY}"Expected:
200. In standalone mode there is no Admin API to check — Deployment Modes says which mode you are in. -
The data plane answers on its proxy port and names the release it runs. Check it with:
curl -sI "${GATEWAY_URL}" | grep -i '^server'Expected:
Server: APISIX/3.18.0, or the release your instances run today. -
An etcd snapshot taken from this deployment is on disk, and it is the snapshot a rollback restores. Take it with Back Up and Restore etcd, then check it with:
ls -l ./backup/snapshot.dbExpected: one file, a non-zero size, dated after your last configuration change.
-
You have read the behaviour changes for the release you are moving to: Review Changes Before Upgrade for this documentation set's release, and the Apache APISIX changelog for every release between yours and the target.
Set the variables this guide uses:
export APISIX_ADMIN_URL="http://127.0.0.1:9180"
export GATEWAY_URL="http://127.0.0.1:9080"
export APISIX_TAG="3.18.0-ubuntu" # the target image tag, from hub.docker.com/r/apache/apisix/tags
export APISIX_CONTAINER="apisix-quickstart" # Docker only: the container name your deployment uses
export APISIX_NAMESPACE="apisix" # Kubernetes only: the namespace of the Helm release
export APISIX_RELEASE="apisix" # Kubernetes only: the Helm release name, as `helm ls` prints it
export APISIX_DEPLOYMENT="apisix" # Kubernetes only: the gateway Deployment, as `kubectl -n "${APISIX_NAMESPACE}" get deployment` prints it
export APISIX_SMOKE_PATH="/ip" # a path one of your own routes serves; the quickstart's route serves /ip
read -rsp "Admin API key: " APISIX_ADMIN_KEY && export APISIX_ADMIN_KEY && echoSteps
Three steps: record what a rollback would need, replace the gateway, then count the routes the new release loaded. On a fleet, run steps 2 and 3 on one instance and keep it out of rotation until step 3 passes — Canary Deployment shifts a share of live traffic onto it with the traffic-split (opens in Plugin Hub docs) plugin instead of all of it at once.
Step 1: Record the image the gateway runs today
The tag is the rollback target, so read it from the running deployment rather than from your notes.
export APISIX_PREVIOUS_TAG="$(docker inspect --format '{{.Config.Image}}' "${APISIX_CONTAINER}")" \
&& echo "${APISIX_PREVIOUS_TAG}"apache/apisix:3.18.0-ubuntuexport APISIX_PREVIOUS_TAG="$(kubectl -n "${APISIX_NAMESPACE}" get deployment "${APISIX_DEPLOYMENT}" \
-o jsonpath='{.spec.template.spec.containers[*].image}')" && echo "${APISIX_PREVIOUS_TAG}"apache/apisix:3.18.0-ubuntuBoth commands print a full image reference rather than a bare tag, which is what the rollback needs. 3.18.0-ubuntu is the tag this documentation set's own install commands use; yours names whichever release the instance is on today.
What changed: nothing on the gateway; APISIX_PREVIOUS_TAG now holds the image the rollback goes back to.
Step 2: Replace the gateway with the target release
The gateway process stops and starts, so this instance serves no traffic between the two commands.
Pull the target image first, so the gateway is down only for the restart and not for the download:
docker pull "apache/apisix:${APISIX_TAG}"Stop the running instance gracefully — apisix quit waits for in-flight requests to finish, where apisix stop does not — and remove the container:
docker exec "${APISIX_CONTAINER}" apisix quit
docker rm -f "${APISIX_CONTAINER}"Start the replacement with the same docker run command your deployment already uses, with apache/apisix:${APISIX_TAG} as the image and the same --mount, -p, and -e arguments. Install APISIX with Docker carries that command for standalone and decoupled mode; the mounted config.yaml and, in standalone mode, apisix.yaml are what carry your configuration across the replacement.
Save the overrides the release was installed with, and upgrade with them rather than with --reuse-values:
helm get values "${APISIX_RELEASE}" -n "${APISIX_NAMESPACE}" -o yaml > apisix-values.yamlSet the target image in apisix-values.yaml. The Apache APISIX chart is versioned independently of APISIX, so its default image is not necessarily the release you want:
image:
tag: "3.18.0-ubuntu"Then upgrade the release and wait for the rollout:
helm repo update
helm upgrade "${APISIX_RELEASE}" apisix/apisix -n "${APISIX_NAMESPACE}" -f apisix-values.yaml
kubectl -n "${APISIX_NAMESPACE}" rollout status deployment "${APISIX_DEPLOYMENT}"Expected: rollout status reports the Deployment rolled out, and exits 0.
Helm values are the source of truth for a Kubernetes deployment: an edit made to conf/config.yaml inside a pod, or to the generated ConfigMap, is overwritten by this upgrade. See Helm Chart.
What changed: the instance runs the target release, with the configuration source it had before — the same etcd cluster, or the same mounted apisix.yaml.
Step 3: Count the routes the new release loaded
A gateway that starts is not yet a gateway that kept its routes. Count them through the Admin API:
curl -s "${APISIX_ADMIN_URL}/apisix/admin/routes" -H "X-API-KEY: ${APISIX_ADMIN_KEY}" \
| grep -o '"total":[0-9]*'Expected: "total": and the number of routes the deployment had before the upgrade. The list endpoint answers with that count beside the routes themselves — see the Admin API reference. In standalone mode, read the count from the mounted apisix.yaml instead; there is no Admin API to ask.
What changed: nothing; the count is the check that the new release read the configuration store it was pointed at.
Validate
Send a request through a route of your own and read the status line and the Server header together. This is the check that proves the upgrade from outside the gateway: the proxy path works, and the process answering is the new release.
curl -is "${GATEWAY_URL}${APISIX_SMOKE_PATH}" | sed -n '1p;/^[Ss]erver:/p'HTTP/1.1 200 OK
Server: APISIX/3.18.0The Server header names the release in APISIX_TAG; 3.18.0 is the release this documentation set publishes. A status line that matches what the route returned before the upgrade, with the new release in the header, is the whole outcome.
Roll back / Clean up
Returning to APISIX_PREVIOUS_TAG restores the gateway process. It does not restore configuration: anything written through the Admin API since the snapshot stays in etcd, which is what you want in almost every case — the release changed, the routes did not.
docker exec "${APISIX_CONTAINER}" apisix quit
docker rm -f "${APISIX_CONTAINER}"Start the container again from the same docker run command, with ${APISIX_PREVIOUS_TAG} as the image.
Put ${APISIX_PREVIOUS_TAG}'s tag back into apisix-values.yaml — the values file is what the next upgrade reads, so rolling back anywhere else is undone by it — and upgrade with that file again:
helm upgrade "${APISIX_RELEASE}" apisix/apisix -n "${APISIX_NAMESPACE}" -f apisix-values.yaml
kubectl -n "${APISIX_NAMESPACE}" rollout status deployment "${APISIX_DEPLOYMENT}"Expected: rollout status reports the Deployment rolled out, and exits 0.
The gateway is down for the restart in both paths, exactly as in step 2. Restore the etcd snapshot only when the configuration itself has to go back as well — a migration wrote resources the older release rejects, for example — with Back Up and Restore etcd, and remember that a restore reverts every write made since the snapshot.
Troubleshoot
| Symptom | Cause | Fix |
|---|---|---|
The Server header still names the release you started from. | On Kubernetes, the chart's own default image.tag won: the values file that was applied does not set one, and the chart is versioned independently of APISIX. On Docker, the replacement container was started from a shell where APISIX_TAG was not exported. | Set image.tag in apisix-values.yaml and run helm upgrade again, or re-export APISIX_TAG and repeat step 2. |
The gateway starts, but every route returns 404. | The new instance is reading a different configuration store: an apisix.yaml that was not mounted in standalone mode, or a deployment.etcd.host the replacement command did not carry over. | Compare the mounts and environment of the replacement with the command in Install APISIX with Docker, then repeat step 3's route count. |
| The gateway does not start, and the log names a configuration key. | A key was removed or renamed in a release between yours and the target, and the file mounted from the host still carries it. | Validate the generated configuration with apisix test, then check the key against Configuration Files. Roll back if the key is load-bearing for your deployment. |
Related
- Review Changes Before Upgrade — what this release changed about plugin defaults, metric labels, and authentication checks before you shift traffic.
- Back Up and Restore etcd — how the snapshot this guide requires is taken and how it is restored.
- Deployment Modes — why a standalone deployment has no Admin API to check and where its configuration lives instead.
- APISIX CLI —
apisix quit,apisix reload,apisix test, andapisix versionin full. - Apache APISIX changelog — the upstream release-by-release list of changes (external).