| 3.10.7 (opens in API7 Gateway docs) | 2026-09-08 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: When an LLM upstream broke the connection after part of a streaming response had already been delivered to the client, the gateway tried to turn the committed HTTP 200 into a… (opens in API7 Gateway docs) |
| 3.10.7 (opens in API7 Gateway docs) | 2026-09-08 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: With the Vertex AI provider, an embeddings request whose model contained a /, a ? or a space was sent to the wrong path or failed outright. A model named models/text-embedding… (opens in API7 Gateway docs) |
| 3.10.7 (opens in API7 Gateway docs) | 2026-09-08 | Fix | Fixed issue: When an LLM upstream broke the connection after part of a streaming response had already been delivered to the client, the gateway tried to turn the committed HTTP 200 into a 500, which is not possible on… (opens in API7 Gateway docs) |
| 3.10.7 (opens in API7 Gateway docs) | 2026-09-08 | Feature | Instance health checks are now reported through the Data Plane control API. GET /v1/healthcheck lists one entry per instance that declares checks, carrying the new plugin and meta fields (meta.instance names the insta… (opens in API7 Gateway docs) |
| 3.10.6 (opens in API7 Gateway docs) | 2026-08-25 | Feature | Added fallback_http_statuses, which lists the HTTP status codes an instance may answer with to make the gateway retry the request against the next instance. Previously a fallback only happened when an instance could n… (opens in API7 Gateway docs) |
| 3.10.6 (opens in API7 Gateway docs) | 2026-08-25 | Fix | Fixed issue: When a request was retried against the next instance, it was built from the body the previous attempt had already rewritten, so the previous instance's options — for example its temperature or max_tokens… (opens in API7 Gateway docs) |
| 3.10.5 (opens in API7 Gateway docs) | 2026-08-11 | Feature | Added the semantic balancer algorithm, which routes each request to the instance whose configured examples are semantically closest to the prompt, instead of by weight or hash. The embedding provider is configured und… (opens in API7 Gateway docs) |
| 3.10.4 (opens in API7 Gateway docs) | 2026-07-27 | Feature | Added max_req_body_size and max_resp_body_size to bound how much of a request or response body a plugin reads into memory, defaulting to 67108864 bytes (64 MiB). A request body larger than the limit is rejected, and a… (opens in API7 Gateway docs) |
| 3.10.4 (opens in API7 Gateway docs) | 2026-07-27 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: On an error response from the LLM, $apisix_upstream_response_time and $llm_time_to_first_token were logged in seconds (for example 0.240) or as 0, while s… (opens in API7 Gateway docs) |
| 3.10.4 (opens in API7 Gateway docs) | 2026-07-27 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: The requests the gateway sent to the LLM provider carried the client's own headers, including Cookie, Authorization, and arbitrary custom headers, exposin… (opens in API7 Gateway docs) |
| 3.10.3 (opens in API7 Gateway docs) | 2026-07-14 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Anthropic Messages requests that contained tool results mixed with other content could be converted into an invalid OpenAI Chat message order, causing OpenAI-compatible upstre… (opens in API7 Gateway docs) |
| 3.10.3 (opens in API7 Gateway docs) | 2026-07-14 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Structured chat content could reach downstream AI plugins as a table and cause request processing errors. Protocol adapters now flatten text content consistently before AI gua… (opens in API7 Gateway docs) |
| 3.10.3 (opens in API7 Gateway docs) | 2026-07-14 | Fix | Fixed issue: A plugin attached through a global rule or a consumer (such as ai-proxy-multi) could fail with HTTP 5xx because the gateway could not fetch the plugin's parent configuration at request time. All plugin-be… (opens in API7 Gateway docs) |
| 3.10.2 (opens in API7 Gateway docs) | 2026-06-29 | Fix | Fixed issue: A failure while constructing the working instance pool could raise a Lua error and destructively clear pool state. The failure path is now nil-safe and non-destructive. (opens in API7 Gateway docs) |
| 3.10.2 (opens in API7 Gateway docs) | 2026-06-29 | Fix | Fixed issue: When the upstream LLM returned an error status (such as HTTP 429 or 5xx), the error response body was discarded and the client received an empty body. The upstream error body and content type are now pres… (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Feature | Added max_retries and retry_on_failure_within_ms to the fallback mechanism. max_retries caps how many additional instances a request retries after a failure; retry_on_failure_within_ms only falls back when the upstrea… (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Fix | Fixed issue: After per-instance health checkers were created, the cached server picker was not rebuilt, so a worker could keep routing part of its traffic to instances already marked unhealthy; active health-check pro… (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Change | forward-auth, ai-proxy, ai-proxy-multi: the body is limited by a new max_req_body_size option (default 67108864 bytes, i.e. 64 MiB); requests over the limit are rejected with HTTP 413. (opens in API7 Gateway docs) |
| 3.10.0 (opens in API7 Gateway docs) | 2026-06-01 | Feature | When an LLM endpoint domain resolves to multiple A records, the plugin now resolves all of them and builds a multi-node upstream, selecting a node per request for better load distribution and failover. The Host header… (opens in API7 Gateway docs) |
| 3.9.20 (opens in API7 Gateway docs) | 2026-09-03 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: With the Vertex AI provider, an embeddings request whose model contained a /, a ? or a space was sent to the wrong path or failed outright. A model named models/text-embedding… (opens in API7 Gateway docs) |
| 3.9.20 (opens in API7 Gateway docs) | 2026-09-03 | Feature | Instance health checks are now reported through the Data Plane control API. GET /v1/healthcheck lists one entry per instance that declares checks, carrying the new plugin and meta fields (meta.instance names the insta… (opens in API7 Gateway docs) |
| 3.9.19 (opens in API7 Gateway docs) | 2026-08-26 | Feature | Added fallback_http_statuses, which lists the HTTP status codes an instance may answer with to make the gateway retry the request against the next instance. Previously a fallback only happened when an instance could n… (opens in API7 Gateway docs) |
| 3.9.19 (opens in API7 Gateway docs) | 2026-08-26 | Fix | Fixed issue: When a request was retried against the next instance, it was built from the body the previous attempt had already rewritten, so the previous instance's options — for example its temperature or max_tokens… (opens in API7 Gateway docs) |
| 3.9.19 (opens in API7 Gateway docs) | 2026-08-26 | Fix | Fixed issue: When a streaming upstream failed part way through — the connection dropping or timing out after some of the stream had already reached the client — the gateway still tried to answer HTTP 500 or 504 on a r… (opens in API7 Gateway docs) |
| 3.9.18 (opens in API7 Gateway docs) | 2026-08-12 | Feature | Added the semantic balancer algorithm, which routes each request to the instance whose configured examples are semantically closest to the prompt, instead of by weight or hash. The embedding provider is configured und… (opens in API7 Gateway docs) |
| 3.9.17 (opens in API7 Gateway docs) | 2026-07-28 | Feature | Added max_req_body_size and max_resp_body_size to bound how much of a request or response body a plugin reads into memory, defaulting to 67108864 bytes (64 MiB). A request body larger than the limit is rejected, and a… (opens in API7 Gateway docs) |
| 3.9.17 (opens in API7 Gateway docs) | 2026-07-28 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: On an error response from the LLM, $apisix_upstream_response_time and $llm_time_to_first_token were logged in seconds (for example 0.240) or as 0, while s… (opens in API7 Gateway docs) |
| 3.9.17 (opens in API7 Gateway docs) | 2026-07-28 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: The requests the gateway sent to the LLM provider carried the client's own headers, including Cookie, Authorization, and arbitrary custom headers, exposin… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: A failure while constructing the ai-proxy-multi working instance pool could raise a Lua error and destructively clear pool state. The failure path is now nil-safe and non-dest… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Anthropic Messages requests that contained tool results mixed with other content could be converted into an invalid OpenAI Chat message order, causing OpenAI-compatible upstre… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Several Anthropic-to-OpenAI conversion details diverged from LiteLLM-compatible behavior, including tool name sanitization, long tool-name collision handling, adaptive thinkin… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Structured chat content could reach downstream AI plugins as a table and cause request processing errors. Protocol adapters now flatten text content consistently before AI gua… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | Fixed issue: A failure while constructing the ai-proxy-multi working instance pool could raise a Lua error and destructively clear pool state. The failure path is now nil-safe and non-destructive. (opens in API7 Gateway docs) |
| 3.9.15 (opens in API7 Gateway docs) | 2026-06-22 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: When an upstream LLM provider returned an HTTP 429 or 5xx response, the gateway returned only the status code with an empty body, discarding the provider's error details (such… (opens in API7 Gateway docs) |
| 3.9.15 (opens in API7 Gateway docs) | 2026-06-22 | Fix | Fixed issue: With consistent-hashing (chash) load balancing, the hash ring was rebuilt from only the currently healthy nodes whenever a node's health changed, which — especially with uneven node weights — reshuffled t… (opens in API7 Gateway docs) |
| 3.9.14 (opens in API7 Gateway docs) | 2026-06-15 | Feature | Added max_retries to bound how many fallback retries are attempted, and retry_on_failure_within_ms so that only a failure occurring within the configured window triggers a fallback. A slow failure is returned to the c… (opens in API7 Gateway docs) |
| 3.9.14 (opens in API7 Gateway docs) | 2026-06-15 | Fix | Fixed issue: Health checks for domain-based upstreams could be unstable — a cached node picker could go stale after health checkers were created, and the health-check configuration could be mutated in place across req… (opens in API7 Gateway docs) |
| 3.9.11 (opens in API7 Gateway docs) | 2026-04-30 | Fix | Fixed issue: After the Control Plane pushed a config update, health check validation logged noisy warnings (unable to construct upstream for plugin: ai-proxy-multi) because DNS resolution state was lost during the con… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | Added override.request_body for per-protocol deep-merge request body overrides and override.llm_options for provider-aware max_tokens mapping. Operators can set protocol-specific parameters (such as max_tokens, stop_s… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Change | AI Proxy Multi: the same fields under instances. (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | AI Proxy, AI Proxy Multi: Added max_stream_duration_ms and max_response_bytes safeguards to prevent unbounded LLM streaming responses from pinning a worker process at high CPU. When either limit is exceeded, the strea… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | AI Proxy, AI Proxy Multi: Added override.request_body for per-protocol deep-merge request body overrides and override.llm_options for provider-aware max_tokens mapping. Operators can set protocol-specific parameters (… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | AI Proxy, AI Proxy Multi: The gateway now detects client disconnection during streaming and immediately stops reading from the LLM upstream, freeing worker resources and avoiding unnecessary API quota consumption. (opens in API7 Gateway docs) |
| 3.9.9 (opens in API7 Gateway docs) | 2026-04-10 | Fix | Fixed issue: The Data Plane compatibility report showed spurious warnings for valid plugin configurations. Plugins using patternProperties, conditional schemas (if/then/else), allOf, dependencies, or additionalPropert… (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Feature | Added full support for the OpenAI Responses API (POST /v1/responses). Both streaming and non-streaming responses are handled, and all downstream plugins (RAG, content moderation, prompt decorator, prompt guard, loggin… (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Feature | AI Proxy, AI Proxy Multi: Added full support for the OpenAI Responses API (POST /v1/responses). Both streaming and non-streaming responses are handled, and all downstream plugins (RAG, content moderation, prompt decor… (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Feature | AI Proxy, AI Proxy Multi: Added native Anthropic Messages API support. Requests to /v1/messages using the Anthropic SDK format are now passed through directly to Anthropic-compatible backends without protocol conversi… (opens in API7 Gateway docs) |
| 3.9.2 (opens in API7 Gateway docs) | 2026-01-19 | Feature | AI Proxy/AI Proxy Multi: Supported four new providers: Gemini, Vertex AI, OpenRouter, and Anthropic. (opens in API7 Gateway docs) |
| 3.9.2 (opens in API7 Gateway docs) | 2026-01-19 | Fix | Fixed issue: The health checker caused the gateway to continuously output error logs after an update to the ai-proxy-multi plugin. (opens in API7 Gateway docs) |
| 3.8.21 (opens in API7 Gateway docs) | 2025-12-23 | Fix | AI Proxy/AI Proxy Multi: Fixed issue: Headers configured in the plugin could not override downstream headers with the same name due to case-insensitivity. (opens in API7 Gateway docs) |
| 3.8.19 (opens in API7 Gateway docs) | 2025-12-09 | Fix | AI Proxy/AI Proxy Multi: Fixed issue: Request headers generated by the client or other plugins were not forwarded to the upstream. (opens in API7 Gateway docs) |
| 3.8.18 (opens in API7 Gateway docs) | 2025-11-25 | Fix | Fixed issue: Health checkers were rebuilt when requests hit different consumers. (opens in API7 Gateway docs) |
| 3.8.17 (opens in API7 Gateway docs) | 2025-11-10 | Feature | AI Proxy/AI Proxy Multi/AI Request Rewrite: Supported the aimlapi provider. (opens in API7 Gateway docs) |
| 3.8.17 (opens in API7 Gateway docs) | 2025-11-10 | Fix | Fixed issue: Multiple worker processes cannot share the health checker of an AI instance. (opens in API7 Gateway docs) |
| 3.8.11 (opens in API7 Gateway docs) | 2025-08-25 | Fix | Fixed issue: Abnormal traffic distribution. (opens in API7 Gateway docs) |
| 3.8.10 (opens in API7 Gateway docs) | 2025-08-25 | Fix | Fixed issue: A panic was triggered by connection timeouts to the upstream service. (opens in API7 Gateway docs) |
| 3.8.2 (opens in API7 Gateway docs) | 2025-05-19 | Feature | Added retry logic for 429/5xx response codes to improve request success rate. (opens in API7 Gateway docs) |
| 3.8.0 (opens in API7 Gateway docs) | 2025-04-22 | Feature | Displayed a record at the warning log level if the fallback was triggered. (opens in API7 Gateway docs) |
| 3.8.0 (opens in API7 Gateway docs) | 2025-04-22 | Feature | Implemented a new identifier field for collecting LLM request and response content. (opens in API7 Gateway docs) |
| 3.8.0 (opens in API7 Gateway docs) | 2025-04-22 | Feature | Supported HTTP POST method for health checks. (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Introduced new AI plugin which enforces token-based rate limiting for requests sent to LLM services. It helps manage API usage by controlling the number of tokens consumed within a specified time frame, ensuring fair… (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Supported active health check. (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Supported bypassing upstream configuration when using the plugin. (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Supported openai-compatible provider. (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Supported proxying embedding model APIs. (opens in API7 Gateway docs) |
