Docs
Apache APISIXManage traffic and APIsAI GatewayImplement Prompt Guardrails

Implement Prompt Guardrails

Enforce allow and deny patterns on one Apache APISIX AI Route, prove rejected prompts never reach the upstream, and clean up.

Applies to:Apache APISIX 3.18.0

Deployments:DockerRoles:Platform engineerSecurity engineerTime:15 minutesImpact:Changes live traffic

Goal

Apply ai-prompt-guard to one isolated Chat Completions-style Route. Prove that an allowed prompt reaches a controlled upstream while a prompt outside the allowlist and a prompt matching the denylist are both rejected before proxying.

Regular expressions are a narrow deterministic control, not a complete AI safety system. They do not understand intent, encoded text, every supported request shape, or model output. Combine them with threat modeling, provider controls, authorization, rate limits, audit logging, testing, and incident response appropriate to the application.

Before you begin

  • Complete Get APISIX with the Docker quickstart.
  • Install Docker, cURL, and jq.
  • Use a shell that supports Bash or Zsh heredocs.

Set the endpoints and protect the fixed Route, container, and local file used by this guide:

export APISIX_ADMIN_URL="http://127.0.0.1:9180"
export GATEWAY_URL="http://127.0.0.1:9080"
export GUIDE_DIR="$(pwd)"
read -rsp "Admin API key (press Enter for an unchanged quickstart): " APISIX_ADMIN_KEY && export APISIX_ADMIN_KEY && echo
test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' "${APISIX_ADMIN_URL}/apisix/admin/routes/prompt-guard-guide-route" -H "X-API-KEY: ${APISIX_ADMIN_KEY}")" = 404 || { echo 'Stop: route prompt-guard-guide-route already exists'; exit 1; }
test -z "$(docker ps -aq --filter 'name=^/prompt-guard-guide-upstream$')" || { echo 'Stop: container prompt-guard-guide-upstream already exists'; exit 1; }
test ! -e prompt-guard-guide-nginx.conf || { echo 'Stop: prompt-guard-guide-nginx.conf already exists'; exit 1; }

Steps

Step 1: Start an auditable upstream

Create a deterministic NGINX response and keep its access log on standard output. Do not publish an upstream port:

cat > prompt-guard-guide-nginx.conf <<'NGINX'
events {}
http {
  log_format guide '$request_method $uri $status';
  access_log /dev/stdout guide;
  server {
    listen 80;
    location = /prompt-guard-guide { return 200 '{"source":"prompt-guard-guide-upstream"}'; }
  }
}
NGINX
docker run -d --name prompt-guard-guide-upstream \
  --network apisix-quickstart-net \
  -v "${GUIDE_DIR}/prompt-guard-guide-nginx.conf:/etc/nginx/nginx.conf:ro" \
  nginx:1.29.1-alpine
for attempt in {1..15}; do
  docker exec apisix-quickstart bash -c '</dev/tcp/prompt-guard-guide-upstream/80' >/dev/null 2>&1 && break
  sleep 1
done
docker exec apisix-quickstart bash -c '</dev/tcp/prompt-guard-guide-upstream/80'

What changed: APISIX has one controlled endpoint whose access log can prove whether a request was proxied.

Step 2: Create the guarded Route

Require the latest user message to mention an invoice or receipt, then reject it if it also includes password or secret. fail_mode: error rejects request shapes the plugin cannot inspect instead of silently bypassing the guard:

jq -n '{
  name:"prompt-guard-guide-route",
  uri:"/prompt-guard-guide",
  methods:["POST"],
  plugins:{
    "ai-prompt-guard":{
      allow_patterns:["(?i)invoice|receipt"],
      deny_patterns:["(?i)password|secret"],
      match_all_roles:false,
      match_all_conversation_history:false,
      fail_mode:"error",
      max_req_body_size:16384
    }
  },
  upstream:{type:"roundrobin",nodes:{"prompt-guard-guide-upstream:80":1}}
}' | \
curl -sS --connect-timeout 3 --max-time 15 \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/prompt-guard-guide-route" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H 'Content-Type: application/json' \
  --data-binary @- | \
jq -e '.value.id == "prompt-guard-guide-route" and .value.plugins["ai-prompt-guard"].fail_mode == "error" and .value.plugins["ai-prompt-guard"].max_req_body_size == 16384'

When both lists are configured, the prompt must first match at least one allow pattern and must then match no deny pattern. Keep patterns narrowly scoped and test evasion, Unicode, and performance behavior before production use.

What changed: the fixed Route checks only the latest user content and fails closed for unrecognized request bodies.

Validate

Send one allowed request and require the upstream response:

curl --fail-with-body -sS --connect-timeout 3 --max-time 10 \
  "${GATEWAY_URL}/prompt-guard-guide" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Review this invoice total."}]}' | \
jq -e '. == {source:"prompt-guard-guide-upstream"}'
for attempt in {1..10}; do
  test "$(docker logs prompt-guard-guide-upstream 2>&1 | grep -c '^POST /prompt-guard-guide 200$')" = 1 && break
  sleep 1
done
test "$(docker logs prompt-guard-guide-upstream 2>&1 | grep -c '^POST /prompt-guard-guide 200$')" = 1

Send one prompt outside the allowlist and one that passes the allowlist but matches the denylist. Both must return 400:

test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' \
  "${GATEWAY_URL}/prompt-guard-guide" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Tell me a joke."}]}')" = 400
test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' \
  "${GATEWAY_URL}/prompt-guard-guide" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"The invoice contains a password."}]}')" = 400
test "$(docker logs prompt-guard-guide-upstream 2>&1 | grep -c '^POST /prompt-guard-guide 200$')" = 1

The final count must remain one. It proves neither rejected prompt reached the upstream.

Roll back / Clean up

Delete the Route before its upstream, then remove the local file and shell variables:

curl --fail-with-body -sS --connect-timeout 3 --max-time 15 -X DELETE \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/prompt-guard-guide-route" \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}"
docker rm -f prompt-guard-guide-upstream
rm -f prompt-guard-guide-nginx.conf
unset APISIX_ADMIN_URL APISIX_ADMIN_KEY GATEWAY_URL GUIDE_DIR

Expected: the fixed Route, container, local file, and shell variables are absent.

Troubleshoot

SymptomCauseFix
Every request returns 400.The body is not a supported AI format, the last user message misses the allow pattern, or the JSON is invalid.Start with the exact allowed request and inspect the APISIX error log before changing patterns.
A denied phrase reaches the upstream.The regular expression does not match the normalized content or only an earlier message contains the phrase.Test the pattern independently and enable match_all_conversation_history only when the increased inspection scope is intended.
A non-JSON request is accepted.fail_mode is not error on the stored Route or another Route matched.Read the fixed Route through the authenticated Admin API and verify the requested URI.
Large requests consume excessive memory.The request-body limit is too high for the deployment's concurrency.Derive max_req_body_size from an explicit payload and concurrency budget; reject oversized input before model processing.

Pre-Define Prompt Templates

Previous Page

Implement Basic Authentication

Next Page