Docs
Apache APISIXManage traffic and APIsAI GatewayProxy OpenAI Requests

Proxy OpenAI Requests

Configure APISIX to authenticate with OpenAI and verify Chat Completions, Responses API, and Embeddings requests through the ai-proxy plugin.

Applies to:Apache APISIX 3.18.0

Deployments:DockerRoles:Platform engineerTime:20 minutesImpact:Changes live traffic

Goal

Create two isolated APISIX Routes that keep the OpenAI credential and model choice at the gateway. Verify Chat Completions, Responses API, and Embeddings response shapes, then remove the Routes and shell secrets.

The validation requests call the live OpenAI API. They can incur provider charges and send the example prompts to OpenAI under your account's data controls. Use approved test credentials, models, and content.

Before you begin

Set the endpoints, read secrets without echoing them, and protect the fixed Route IDs used by this guide:

export APISIX_ADMIN_URL="http://127.0.0.1:9180"
export GATEWAY_URL="http://127.0.0.1:9080"
read -rsp "Admin API key (press Enter for an unchanged quickstart): " APISIX_ADMIN_KEY && export APISIX_ADMIN_KEY && echo
read -rsp "OpenAI API key: " OPENAI_API_KEY && export OPENAI_API_KEY && echo
read -rp "OpenAI text model: " OPENAI_MODEL && export OPENAI_MODEL
read -rp "OpenAI embedding model: " OPENAI_EMBEDDING_MODEL && export OPENAI_EMBEDDING_MODEL
test -n "${OPENAI_API_KEY}" && test -n "${OPENAI_MODEL}" && test -n "${OPENAI_EMBEDDING_MODEL}"
for route_id in openai-guide-generation openai-guide-embeddings; do
  test "$(curl -sS --connect-timeout 3 --max-time 10 -o /dev/null -w '%{http_code}' \
    "${APISIX_ADMIN_URL}/apisix/admin/routes/${route_id}" \
    -H "X-API-KEY: ${APISIX_ADMIN_KEY}")" = 404 || \
    { echo "Stop: route ${route_id} already exists"; exit 1; }
done

Steps

Step 1: Create the generation Route

Use a Chat Completions path and a path that retains the required /v1/responses suffix. APISIX detects Chat Completions from messages; it detects Responses from input together with that suffix.

jq -n --arg key "${OPENAI_API_KEY}" --arg model "${OPENAI_MODEL}" '{
  name:"openai-guide-generation",
  uris:["/openai-guide/chat","/openai-guide/v1/responses"],
  methods:["POST"],
  plugins:{
    "ai-proxy":{
      provider:"openai",
      auth:{header:{Authorization:("Bearer " + $key)}},
      options:{model:$model}
    }
  }
}' | \
curl --fail-with-body -sS --connect-timeout 3 --max-time 15 \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/openai-guide-generation" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H 'Content-Type: application/json' \
  --data-binary @- | \
jq -e --arg model "${OPENAI_MODEL}" '
  .value.id == "openai-guide-generation" and
  .value.uris == ["/openai-guide/chat","/openai-guide/v1/responses"] and
  .value.plugins["ai-proxy"].provider == "openai" and
  .value.plugins["ai-proxy"].options.model == $model'

What changed: both client paths now use one gateway-managed OpenAI credential and text model. A client-supplied model cannot override options.model.

Step 2: Create the Embeddings Route

Responses and Embeddings bodies both contain input. A separate Route makes the embedding model explicit and prevents a missing Responses suffix from silently selecting the wrong protocol.

jq -n --arg key "${OPENAI_API_KEY}" --arg model "${OPENAI_EMBEDDING_MODEL}" '{
  name:"openai-guide-embeddings",
  uri:"/openai-guide/embeddings",
  methods:["POST"],
  plugins:{
    "ai-proxy":{
      provider:"openai",
      auth:{header:{Authorization:("Bearer " + $key)}},
      options:{model:$model}
    }
  }
}' | \
curl --fail-with-body -sS --connect-timeout 3 --max-time 15 \
  "${APISIX_ADMIN_URL}/apisix/admin/routes/openai-guide-embeddings" -X PUT \
  -H "X-API-KEY: ${APISIX_ADMIN_KEY}" -H 'Content-Type: application/json' \
  --data-binary @- | \
jq -e --arg model "${OPENAI_EMBEDDING_MODEL}" '
  .value.id == "openai-guide-embeddings" and
  .value.uri == "/openai-guide/embeddings" and
  .value.plugins["ai-proxy"].provider == "openai" and
  .value.plugins["ai-proxy"].options.model == $model'

Do not commit credentials in ADC, Kubernetes manifests, or shell history. The Admin API configuration still contains the provider key and must be access-controlled. When data encryption with a keyring is enabled, APISIX encrypts configured ai-proxy authentication fields before storing them in etcd.

Validate

Send a Chat Completions request and require a non-empty assistant message:

curl --fail-with-body -sS --connect-timeout 3 --max-time 90 \
  "${GATEWAY_URL}/openai-guide/chat" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Reply with one short sentence defining an API gateway."}]}' | \
jq -e '.object == "chat.completion" and (.choices[0].message.content | type == "string" and length > 0)'

Send a Responses request and require a completed response containing an assistant message. OpenAI documents output as a heterogeneous array, so the assertion does not assume the message is its first item.

curl --fail-with-body -sS --connect-timeout 3 --max-time 90 \
  "${GATEWAY_URL}/openai-guide/v1/responses" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"input":"Reply with one short sentence defining an API gateway."}' | \
jq -e '.object == "response" and .status == "completed" and any(.output[]?; .type == "message" and .role == "assistant")'

Send an Embeddings request and require at least one non-empty vector:

curl --fail-with-body -sS --connect-timeout 3 --max-time 90 \
  "${GATEWAY_URL}/openai-guide/embeddings" -X POST \
  -H 'Content-Type: application/json' \
  -d '{"input":"APISIX is an API gateway."}' | \
jq -e '.object == "list" and (.data[0].embedding | type == "array" and length > 0)'

All three commands must exit successfully. They validate provider response structure, not exact generated prose.

Roll back / Clean up

Delete both Routes and remove secrets and model names from the shell:

for route_id in openai-guide-generation openai-guide-embeddings; do
  curl --fail-with-body -sS --connect-timeout 3 --max-time 15 -X DELETE \
    "${APISIX_ADMIN_URL}/apisix/admin/routes/${route_id}" \
    -H "X-API-KEY: ${APISIX_ADMIN_KEY}" | jq -e '.key | endswith("/" + $route_id)' --arg route_id "${route_id}"
done
unset APISIX_ADMIN_URL APISIX_ADMIN_KEY GATEWAY_URL OPENAI_API_KEY OPENAI_MODEL OPENAI_EMBEDDING_MODEL

Expected: both fixed Route IDs are deleted and the six shell variables are unset.

Troubleshoot

SymptomCauseFix
The Responses request is handled as Embeddings.The client-facing URI does not end in /v1/responses.Use the exact Responses URI configured in this guide.
OpenAI returns 401.The key is invalid, revoked, or unavailable to this project.Create or select an approved project key, then recreate both Routes.
OpenAI returns 404 or a model access error.The chosen model is unavailable to the project or does not support that API.Select models shown for the same project and recreate the affected Route.
APISIX returns 502 or times out.APISIX cannot reach OpenAI, or the provider exceeded the bounded request time.Check gateway egress, DNS, TLS trust, and provider status; do not remove timeouts while diagnosing.
A validation assertion fails after HTTP 200.The provider response does not match the expected protocol.Read the response without the final jq only in an approved environment, and verify the Route URI and selected model.

Convert JSON to XML

Previous Page

Proxy OpenRouter Requests

Next Page