AI Request Rewrite Configuration

Parameters

See plugin common configurations for configuration options available to all plugins.

• prompt — string · required

  The prompt to instruct the LLM on how to rewrite the client request.

• provider — string · required

  Valid values: openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible

  LLM service provider.

  When set to openai, the plugin will proxy requests to https://api.openai.com/v1/chat/completions.

  When set to deepseek, the plugin will proxy requests to https://api.deepseek.com/chat/completions.

  When set to gemini (available from APISIX 3.15.0 and Enterprise 3.9.3), the plugin will proxy requests to https://generativelanguage.googleapis.com/v1beta/openai/chat/completions. If you are proxying requests to an embedding model, you should configure the embedding model endpoint in the override.

  When set to vertex-ai (introduced in API7 Enterprise 3.9.3 and APISIX 3.15.0):

  In API7 Enterprise, the plugin proxies requests to Google Cloud Vertex AI. For chat completions, it uses https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project_id}/locations/{region}/endpoints/openapi/chat/completions. For embeddings, it uses https://{region}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{region}/publishers/google/models/{model}:predict. Configure provider_conf with project_id and region, or set override.endpoint for a custom endpoint. Authenticate with auth.gcp or auth.header.

  In APISIX, configure override.endpoint with the complete Vertex AI endpoint and provide the required Google Cloud access token in auth.header. The plugin schema does not include auth.gcp or provider_conf.

  When set to anthropic (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://api.anthropic.com/v1/chat/completions.

  When set to openrouter (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://openrouter.ai/api/v1/chat/completions.

  When set to aimlapi (available from APISIX 3.14.0 and Enterprise 3.8.17), the plugin uses the OpenAI-compatible driver and proxies the request to https://api.aimlapi.com/v1/chat/completions.

  When set to openai-compatible, the plugin proxies requests to the custom endpoint configured in override.

  When set to azure-openai, the plugin also proxies requests to the custom endpoint configured in override and additionally removes the model parameter from user requests.

• auth — object · required

  Authentication configurations.

  • header — object · optional

    Authentication headers. At least one of the header and query should be configured.

  • query — object · optional

    Authentication query parameters. At least one of the header and query should be configured.

  • gcp — object · optional

    GCP service account authentication for Vertex AI. Available in API7 Enterprise from 3.9.3 and not in APISIX.

    • service_account_json — string · optional

      GCP service account JSON content used for authentication. This can be configured using this parameter or by setting the GCP_SERVICE_ACCOUNT environment variable.

    • max_ttl — integer · optional

      Maximum TTL for GCP access token caching, in seconds.

    • expire_early_secs — integer · optional · default: 60

      Number of seconds to expire the access token before its actual expiration time. This prevents edge cases where tokens expire during active requests.

• options — object · optional

  Model configurations.

  In addition to model, you can configure non-streaming generation parameters, such as temperature and top_p, which are forwarded to the upstream LLM service. The plugin must receive a complete JSON response before it can replace the original request body.

  • model — string · optional

    Name of the LLM model, such as gpt-4 or gpt-3.5. See your LLM provider's API documentation for more available models.

• provider_conf — object · optional

  Provider-specific configuration. When provider is vertex-ai, one of provider_conf or override should be configured.

  Available in API7 Enterprise from 3.9.3 and not in APISIX.

  • project_id — string · required

    Google Cloud Project ID for Vertex AI.

  • region — string · required

    Google Cloud Region for Vertex AI.

• override — object · optional

  Override setting.

  • endpoint — string · optional

    LLM provider endpoint. Required when provider is openai-compatible. When provider is vertex-ai, APISIX requires this field. API7 Enterprise uses provider_conf instead and does not require override.endpoint.

• timeout — integer · optional · default: 30000

  Valid values: between 1 and 60000 inclusive

  Request timeout in milliseconds when requesting the LLM service.

• keepalive — boolean · optional · default: true

  If true, keep the connection alive when requesting the LLM service.

• keepalive_pool — integer · optional · default: 30

  Keepalive pool size for when connecting with the LLM service.

• ssl_verify — boolean · optional · default: true

  If true, verify the LLM service's certificate.

• max_req_body_size — integer · optional · default: 67108864

  Valid values: greater than or equal to 1

  Maximum request body size in bytes buffered into memory. Larger request bodies are rejected with HTTP 400 before the rewrite LLM is called. Introduced in API7 Enterprise 3.9.17 and 3.10.4, and APISIX 3.18.0.