AI Request Rewrite Configuration

Configuration reference for the AI Request Rewrite plugin (`ai-request-rewrite`): the parameters, each attribute with its type, whether it is required, its default and its accepted values.

Parameters

See plugin common configurations (opens in Apache APISIX docs) for configuration options available to all plugins.

prompt — string · required · since APISIX 3.13.0 · API7 Gateway 3.8.11

The prompt to instruct the LLM on how to rewrite the client request.

provider — string · required · since APISIX 3.13.0 · API7 Gateway 3.8.11

Valid values: openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible

LLM service provider.

When set to openai, the plugin will proxy requests to https://api.openai.com/v1/chat/completions.

When set to deepseek, the plugin will proxy requests to https://api.deepseek.com/chat/completions.

When set to gemini (available from APISIX 3.15.0 and Enterprise 3.9.3), the plugin will proxy requests to https://generativelanguage.googleapis.com/v1beta/openai/chat/completions. If you are proxying requests to an embedding model, you should configure the embedding model endpoint in the override.

When set to vertex-ai (introduced in API7 Enterprise 3.9.3 and APISIX 3.15.0):

In API7 Enterprise, the plugin proxies requests to Google Cloud Vertex AI. For chat completions, it uses https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project_id}/locations/{region}/endpoints/openapi/chat/completions. For embeddings, it uses https://{region}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{region}/publishers/google/models/{model}:predict. Configure provider_conf with project_id and region, or set override.endpoint for a custom endpoint. Authenticate with auth.gcp or auth.header.

In APISIX, configure override.endpoint with the complete Vertex AI endpoint and provide the required Google Cloud access token in auth.header. The plugin schema does not include auth.gcp or provider_conf.

When set to anthropic (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://api.anthropic.com/v1/chat/completions.

When set to openrouter (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://openrouter.ai/api/v1/chat/completions.

When set to aimlapi (available from APISIX 3.14.0 and Enterprise 3.8.17), the plugin uses the OpenAI-compatible driver and proxies the request to https://api.aimlapi.com/v1/chat/completions.

When set to openai-compatible, the plugin proxies requests to the custom endpoint configured in override.

When set to azure-openai, the plugin also proxies requests to the custom endpoint configured in override and additionally removes the model parameter from user requests.

auth — object · required · since APISIX 3.13.0 · API7 Gateway 3.8.11

Authentication configurations.

query — object · optional · since APISIX 3.13.0 · API7 Gateway 3.8.11

Authentication query parameters. At least one of the header and query should be configured.

gcp — object · optional

GCP service account authentication for Vertex AI. Available in API7 Enterprise from 3.9.3 and not in APISIX.

service_account_json — string · optional

GCP service account JSON content used for authentication. This can be configured using this parameter or by setting the GCP_SERVICE_ACCOUNT environment variable.

max_ttl — integer · optional · min: 1

Maximum TTL for GCP access token caching, in seconds.

expire_early_secs — integer · optional · default: 60 · min: 0

Number of seconds to expire the access token before its actual expiration time. This prevents edge cases where tokens expire during active requests.

options — object · optional · since APISIX 3.13.0 · API7 Gateway 3.8.11

Model configurations.

In addition to model, you can configure non-streaming generation parameters, such as temperature and top_p, which are forwarded to the upstream LLM service. The plugin must receive a complete JSON response before it can replace the original request body.

model — string · optional · since APISIX 3.13.0 · API7 Gateway 3.8.11

Name of the LLM model, such as gpt-4 or gpt-3.5. See your LLM provider's API documentation for more available models.

provider_conf — object · optional

Provider-specific configuration. When provider is vertex-ai, one of provider_conf or override should be configured.

Available in API7 Enterprise from 3.9.3 and not in APISIX.

project_id — string · required

Google Cloud Project ID for Vertex AI.

region — string · required

Google Cloud Region for Vertex AI.

override — object · optional · since APISIX 3.13.0 · API7 Gateway 3.8.11

Override setting.

endpoint — string · optional · since APISIX 3.13.0 · API7 Gateway 3.8.11

LLM provider endpoint. Required when provider is openai-compatible. When provider is vertex-ai, APISIX requires this field. API7 Enterprise uses provider_conf instead and does not require override.endpoint.

timeout — integer · optional · default: 30000 · since APISIX 3.13.0 · API7 Gateway 3.8.11

Valid values: between 1 and 60000 inclusive

Request timeout in milliseconds when requesting the LLM service.

keepalive — boolean · optional · default: true · since APISIX 3.13.0 · API7 Gateway 3.8.11

If true, keep the connection alive when requesting the LLM service.

keepalive_pool — integer · optional · default: 30 · since APISIX 3.13.0 · API7 Gateway 3.8.11

Keepalive pool size for when connecting with the LLM service.

ssl_verify — boolean · optional · default: true · since APISIX 3.13.0 · API7 Gateway 3.8.11

If true, verify the LLM service's certificate.

max_req_body_size — integer · optional · default: 67108864 · since APISIX 3.18.0 · API7 Gateway 3.9.17

Valid values: greater than or equal to 1

Maximum request body size in bytes buffered into memory. Larger request bodies are rejected with HTTP 400 before the rewrite LLM is called. Introduced in API7 Enterprise 3.9.17 and 3.10.4, and APISIX 3.18.0.