Pre-Define Prompt Templates

Replace client bodies with bounded Chat Completions and Responses API templates, inspect exact upstream payloads, and clean up.

Deployments:DockerRoles:Platform engineerTime:20 minutesImpact:Changes live traffic

Goal

Use ai-prompt-template on one isolated Route to let clients supply only declared values for two administrator-owned request shapes. Send a Chat Completions template and a Responses API template to a controlled local echo server, then assert the exact bodies APISIX forwards upstream.

Template substitution does not make client input trusted. Keep models, tools, endpoints, and privileged instructions fixed in administrator-controlled configuration; validate and authorize supplied values before using them in sensitive workflows.

Before you begin

Complete Get APISIX with the Docker quickstart.
Install Docker, cURL, and jq.
Use a shell that supports Bash or Zsh heredocs.

Set the endpoints and protect the fixed Route, container, and local file used by this guide:

export APISIX_ADMIN_URL="http://127.0.0.1:9180"
export GATEWAY_URL="http://127.0.0.1:9080"
export GUIDE_DIR="$(pwd)"
read -rsp "Admin API key (press Enter for an unchanged quickstart): " APISIX_ADMIN_KEY && export APISIX_ADMIN_KEY && echo
test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' "${APISIX_ADMIN_URL}/apisix/admin/routes/prompt-template-guide-route" -H "X-API-KEY: ${APISIX_ADMIN_KEY}")" = 404 || { echo 'Stop: route prompt-template-guide-route already exists'; exit 1; }
test -z "$(docker ps -aq --filter 'name=^/prompt-template-guide-upstream$')" || { echo 'Stop: container prompt-template-guide-upstream already exists'; exit 1; }
test ! -e prompt-template-guide-server.py || { echo 'Stop: prompt-template-guide-server.py already exists'; exit 1; }

Steps

Step 1: Start a request-body echo server

Create a minimal server that returns the JSON body it receives, then start it without publishing an upstream port:

cat > prompt-template-guide-server.py <<'PYTHON'
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        length = int(self.headers.get("content-length", "0"))
        body = self.rfile.read(length)
        self.send_response(200)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *_):
        pass

ThreadingHTTPServer(("0.0.0.0", 8080), Handler).serve_forever()
PYTHON
docker run -d --name prompt-template-guide-upstream \
  --network apisix-quickstart-net \
  -v "${GUIDE_DIR}/prompt-template-guide-server.py:/srv/server.py:ro" \
  python:3.13.7-alpine python /srv/server.py
for attempt in {1..15}; do
  docker exec apisix-quickstart bash -c '</dev/tcp/prompt-template-guide-upstream/8080' >/dev/null 2>&1 && break
  sleep 1
done
docker exec apisix-quickstart bash -c '</dev/tcp/prompt-template-guide-upstream/8080'

This echo server is for local inspection only. It has no authentication, TLS, request limit, or production hardening, and its port is intentionally reachable only from the Docker network.

What changed: APISIX has a controlled upstream that reveals the resolved template body without calling a model provider.

Step 2: Create two bounded templates

Create one fixed Route with a Chat Completions path and a Responses API path. The template fixes the model and structure; clients supply only topic and length values:

jq -n '{
  name:"prompt-template-guide-route",
  uris:["/template/chat","/template/v1/responses"],
  methods:["POST"],
  plugins:{
    "ai-prompt-template":{
      templates:[
        {
          name:"Guide chat summary",
          template:{
            model:"guide-model",
            messages:[
              {role:"system",content:"Answer in \u007b\u007blength\u007d\u007d."},
              {role:"user",content:"Explain \u007b\u007btopic\u007d\u007d."}
            ]
          }
        },
        {
          name:"Guide response summary",
          template:{
            model:"guide-model",
            instructions:"Answer in \u007b\u007blength\u007d\u007d.",
            input:"Explain \u007b\u007btopic\u007d\u007d."
          }
        }
      ],
      max_req_body_size:16384
    }
  },
  upstream:{type:"roundrobin",nodes:{"prompt-template-guide-upstream:8080":1}}
}' | \
curl -sS --connect-timeout 3 --max-time 15 \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/prompt-template-guide-route" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H 'Content-Type: application/json' \
  --data-binary @- | \
jq -e '.value.id == "prompt-template-guide-route" and (.value.plugins["ai-prompt-template"].templates | map(.name)) == ["Guide chat summary","Guide response summary"] and .value.plugins["ai-prompt-template"].max_req_body_size == 16384'

In the jq program, \u007b and \u007d encode literal braces, so the stored strings are Answer in {{length}}. and Explain {{topic}}.. This avoids shell brace expansion while preserving the plugin's required placeholder syntax. The Responses API path must end in /v1/responses so downstream AI plugins can distinguish its resolved input field from an Embeddings body.

What changed: clients can select one of two fixed request shapes without choosing the model or adding arbitrary body fields.

Validate

Select the Chat Completions template and require the exact resolved body returned by the echo server:

curl --fail-with-body -sS --connect-timeout 3 --max-time 10 \
  "${GATEWAY_URL}/template/chat" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"template_name":"Guide chat summary","topic":"mTLS","length":"one sentence"}' | \
jq -e '. == {
  model:"guide-model",
  messages:[
    {role:"system",content:"Answer in one sentence."},
    {role:"user",content:"Explain mTLS."}
  ]
}'

Select the Responses API template and require its exact resolved fields:

curl --fail-with-body -sS --connect-timeout 3 --max-time 10 \
  "${GATEWAY_URL}/template/v1/responses" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"template_name":"Guide response summary","topic":"rate limiting","length":"two sentences"}' | \
jq -e '. == {
  model:"guide-model",
  instructions:"Answer in two sentences.",
  input:"Explain rate limiting."
}'

Both assertions must pass. They validate the post-template body itself rather than accepting provider prose as indirect evidence.

Roll back / Clean up

Delete the Route before its upstream, then remove the local file and shell variables:

curl --fail-with-body -sS --connect-timeout 3 --max-time 15 -X DELETE \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/prompt-template-guide-route" \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}"
docker rm -f prompt-template-guide-upstream
rm -f prompt-template-guide-server.py
unset APISIX_ADMIN_URL APISIX_ADMIN_KEY GATEWAY_URL GUIDE_DIR

Expected: the fixed Route, container, local file, and shell variables are absent.

Troubleshoot

Symptom	Cause	Fix
The request returns a missing-template error.	`template_name` does not exactly match a configured name.	Copy one of the two stored names and keep template names unique on the Route.
A placeholder remains in the upstream body.	The client omitted the corresponding key or used another spelling.	Compare the client body with every `{{name}}` token in the selected template.
The request returns `400`.	The body is invalid JSON or exceeds `max_req_body_size`.	Validate the JSON and size the limit from an explicit payload and concurrency budget.
The echo server is unreachable.	The containers do not share `apisix-quickstart-net` or the Python process exited.	Inspect `docker logs prompt-template-guide-upstream` and repeat the bounded TCP readiness test.

AI Prompt Template (opens in Plugin Hub docs) — template selection, placeholders, request formats, and size limits.
AI Prompt Decorator (opens in Plugin Hub docs) — add shared instructions without replacing the client's body.
AI Prompt Guard (opens in Plugin Hub docs) — reject disallowed client values before upstream processing.