| 3.10.7 (opens in API7 Gateway docs) | 2026-09-08 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: When an LLM upstream broke the connection after part of a streaming response had already been delivered to the client, the gateway tried to turn the committed HTTP 200 into a… (opens in API7 Gateway docs) |
| 3.10.7 (opens in API7 Gateway docs) | 2026-09-08 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: With the Vertex AI provider, an embeddings request whose model contained a /, a ? or a space was sent to the wrong path or failed outright. A model named models/text-embedding… (opens in API7 Gateway docs) |
| 3.10.6 (opens in API7 Gateway docs) | 2026-08-25 | Fix | Fixed issue: When a streaming upstream answered HTTP 200 with Content-Type: text/event-stream and then ended the response without writing anything, the request was neither answered nor terminated, so the client receiv… (opens in API7 Gateway docs) |
| 3.10.4 (opens in API7 Gateway docs) | 2026-07-27 | Feature | Added max_req_body_size and max_resp_body_size to bound how much of a request or response body a plugin reads into memory, defaulting to 67108864 bytes (64 MiB). A request body larger than the limit is rejected, and a… (opens in API7 Gateway docs) |
| 3.10.4 (opens in API7 Gateway docs) | 2026-07-27 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: On an error response from the LLM, $apisix_upstream_response_time and $llm_time_to_first_token were logged in seconds (for example 0.240) or as 0, while s… (opens in API7 Gateway docs) |
| 3.10.4 (opens in API7 Gateway docs) | 2026-07-27 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: The requests the gateway sent to the LLM provider carried the client's own headers, including Cookie, Authorization, and arbitrary custom headers, exposin… (opens in API7 Gateway docs) |
| 3.10.3 (opens in API7 Gateway docs) | 2026-07-14 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Anthropic Messages requests that contained tool results mixed with other content could be converted into an invalid OpenAI Chat message order, causing OpenAI-compatible upstre… (opens in API7 Gateway docs) |
| 3.10.3 (opens in API7 Gateway docs) | 2026-07-14 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Structured chat content could reach downstream AI plugins as a table and cause request processing errors. Protocol adapters now flatten text content consistently before AI gua… (opens in API7 Gateway docs) |
| 3.10.2 (opens in API7 Gateway docs) | 2026-06-29 | Fix | Fixed issue: A request that carried tool_choice but had no usable tools (for example, only a built-in tool that is dropped during conversion) was forwarded with the orphan tool_choice and rejected by the upstream. Suc… (opens in API7 Gateway docs) |
| 3.10.2 (opens in API7 Gateway docs) | 2026-06-29 | Fix | Fixed issue: When the upstream LLM returned an error status (such as HTTP 429 or 5xx), the error response body was discarded and the client received an empty body. The upstream error body and content type are now pres… (opens in API7 Gateway docs) |
| 3.10.2 (opens in API7 Gateway docs) | 2026-06-29 | Fix | Fixed issue: When the upstream returned a tool call whose arguments were not valid JSON, the entire response conversion was aborted and the client received nothing. The malformed tool call now falls back to an empty a… (opens in API7 Gateway docs) |
| 3.10.2 (opens in API7 Gateway docs) | 2026-06-29 | Feature | The structured llm_summary object emitted to logger plugins (when logging.summaries is enabled) now includes additional AI observability fields: stream, tool_count, has_tool_calls, end_user_id, cache_read_input_tokens… (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Feature | Added built-in NGINX variables that describe each LLM request, for use in access_log formats and logger plugins: $llm_total_tokens, $llm_stream, $llm_has_tool_calls, $llm_tool_count, $llm_end_user_id, $llm_cache_read_… (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Fix | Fixed issue: A timeout while reaching an upstream LLM (such as a DNS-resolution timeout) returned HTTP 500 instead of HTTP 504. Upstream LLM timeouts now correctly return HTTP 504. (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Fix | Fixed issue: In passthrough mode, the upstream request was always sent as POST and the client's query string was dropped, breaking providers that require query parameters on other methods (for example, Azure OpenAI's… (opens in API7 Gateway docs) |
| 3.10.1 (opens in API7 Gateway docs) | 2026-06-15 | Change | forward-auth, ai-proxy, ai-proxy-multi: the body is limited by a new max_req_body_size option (default 67108864 bytes, i.e. 64 MiB); requests over the limit are rejected with HTTP 413. (opens in API7 Gateway docs) |
| 3.10.0 (opens in API7 Gateway docs) | 2026-06-01 | Feature | The upstream LLM request body is now JSON-encoded with sorted keys. This produces a stable, byte-identical body for equivalent requests, which improves prompt-cache hit rates on LLM providers that cache by exact reque… (opens in API7 Gateway docs) |
| 3.9.20 (opens in API7 Gateway docs) | 2026-09-03 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: With the Vertex AI provider, an embeddings request whose model contained a /, a ? or a space was sent to the wrong path or failed outright. A model named models/text-embedding… (opens in API7 Gateway docs) |
| 3.9.19 (opens in API7 Gateway docs) | 2026-08-26 | Fix | Fixed issue: When a streaming upstream answered HTTP 200 with Content-Type: text/event-stream and then ended the response without writing anything, the request was neither answered nor terminated, so the client receiv… (opens in API7 Gateway docs) |
| 3.9.19 (opens in API7 Gateway docs) | 2026-08-26 | Fix | Fixed issue: When a streaming upstream failed part way through — the connection dropping or timing out after some of the stream had already reached the client — the gateway still tried to answer HTTP 500 or 504 on a r… (opens in API7 Gateway docs) |
| 3.9.19 (opens in API7 Gateway docs) | 2026-08-26 | Fix | Fixed issue: When the gateway cut a stream short itself — max_response_bytes or max_stream_duration_ms on AI Proxy reaching its limit — these plugins still appended the protocol's completion event to the stream they r… (opens in API7 Gateway docs) |
| 3.9.17 (opens in API7 Gateway docs) | 2026-07-28 | Feature | Added max_req_body_size and max_resp_body_size to bound how much of a request or response body a plugin reads into memory, defaulting to 67108864 bytes (64 MiB). A request body larger than the limit is rejected, and a… (opens in API7 Gateway docs) |
| 3.9.17 (opens in API7 Gateway docs) | 2026-07-28 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: On an error response from the LLM, $apisix_upstream_response_time and $llm_time_to_first_token were logged in seconds (for example 0.240) or as 0, while s… (opens in API7 Gateway docs) |
| 3.9.17 (opens in API7 Gateway docs) | 2026-07-28 | Fix | AI Proxy, AI Proxy Multi, and AI Request Rewrite: Fixed issue: The requests the gateway sent to the LLM provider carried the client's own headers, including Cookie, Authorization, and arbitrary custom headers, exposin… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: A failure while constructing the ai-proxy-multi working instance pool could raise a Lua error and destructively clear pool state. The failure path is now nil-safe and non-dest… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Anthropic Messages requests that contained tool results mixed with other content could be converted into an invalid OpenAI Chat message order, causing OpenAI-compatible upstre… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Several Anthropic-to-OpenAI conversion details diverged from LiteLLM-compatible behavior, including tool name sanitization, long tool-name collision handling, adaptive thinkin… (opens in API7 Gateway docs) |
| 3.9.16 (opens in API7 Gateway docs) | 2026-07-10 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: Structured chat content could reach downstream AI plugins as a table and cause request processing errors. Protocol adapters now flatten text content consistently before AI gua… (opens in API7 Gateway docs) |
| 3.9.15 (opens in API7 Gateway docs) | 2026-06-22 | Fix | AI Proxy and AI Proxy Multi: Fixed issue: When an upstream LLM provider returned an HTTP 429 or 5xx response, the gateway returned only the status code with an empty body, discarding the provider's error details (such… (opens in API7 Gateway docs) |
| 3.9.14 (opens in API7 Gateway docs) | 2026-06-15 | Fix | Fixed issue: An upstream LLM timeout was mapped to HTTP 500. It is now mapped to HTTP 504 Gateway Time-out. (opens in API7 Gateway docs) |
| 3.9.14 (opens in API7 Gateway docs) | 2026-06-15 | Fix | Fixed issue: In passthrough mode, the client's HTTP method and query string were not forwarded to the upstream. They are now preserved. (opens in API7 Gateway docs) |
| 3.9.14 (opens in API7 Gateway docs) | 2026-06-15 | Feature | The upstream LLM request body is now JSON-encoded with sorted keys. This produces a stable, byte-identical body for equivalent requests, which improves prompt-cache hit rates on LLM providers that cache by exact reque… (opens in API7 Gateway docs) |
| 3.9.13 (opens in API7 Gateway docs) | 2026-05-22 | Feature | Added streaming_flush_interval_ms to control periodic flushing for streaming responses. The default is 10 ms, which reduces per-chunk flush overhead while keeping streaming latency bounded. (opens in API7 Gateway docs) |
| 3.9.13 (opens in API7 Gateway docs) | 2026-05-22 | Feature | Improved large JSON request body handling for AI requests that use post_arg.* route variables. In internal benchmarks with 1 MB, 5 MB, and 10 MB OpenAI Chat Completions-compatible request bodies, throughput improved b… (opens in API7 Gateway docs) |
| 3.9.13 (opens in API7 Gateway docs) | 2026-05-22 | Feature | Improved OpenAI-compatible streaming response throughput by reducing per-token flush and SSE parsing overhead. In internal single-worker benchmarks with a mock OpenAI upstream, peak streaming throughput increased from… (opens in API7 Gateway docs) |
| 3.9.12 (opens in API7 Gateway docs) | 2026-05-08 | Feature | Added support for the AWS Bedrock provider using the Converse API. Configure with "provider": "bedrock", AWS IAM credentials in auth.aws, and an AWS region in provider_conf.region. Both non-streaming and streaming (Co… (opens in API7 Gateway docs) |
| 3.9.11 (opens in API7 Gateway docs) | 2026-04-30 | Feature | Added a passthrough protocol adapter that proxies unrecognized API formats (such as /v1/images/generations) through to the upstream without transformation. Previously, requests that did not match a known protocol (Ope… (opens in API7 Gateway docs) |
| 3.9.11 (opens in API7 Gateway docs) | 2026-04-30 | Feature | Rewrote the Anthropic-to-OpenAI protocol converter using a whitelist body construction approach. This prevents Anthropic-specific fields (such as metadata, top_k, thinking, and output_config) from leaking through to O… (opens in API7 Gateway docs) |
| 3.9.11 (opens in API7 Gateway docs) | 2026-04-30 | Feature | Upstream nginx variables ($upstream_status, $upstream_addr, $upstream_response_time, $upstream_header_time, $upstream_connect_time, $upstream_response_length) are now populated in access logs when ai-proxy uses cosock… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | AI Proxy, AI Proxy Multi: Added max_stream_duration_ms and max_response_bytes safeguards to prevent unbounded LLM streaming responses from pinning a worker process at high CPU. When either limit is exceeded, the strea… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | AI Proxy, AI Proxy Multi: Added override.request_body for per-protocol deep-merge request body overrides and override.llm_options for provider-aware max_tokens mapping. Operators can set protocol-specific parameters (… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Feature | AI Proxy, AI Proxy Multi: The gateway now detects client disconnection during streaming and immediately stops reading from the LLM upstream, freeing worker resources and avoiding unnecessary API quota consumption. (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Change | AI Proxy: auth.header, auth.query, auth.gcp.service_account_json (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Fix | Fixed issue: When a protocol converter was active (for example, Anthropic-to-OpenAI) and the upstream returned SSE events in an unexpected format, the gateway returned a 500 error instead of 502. The gateway now corre… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Fix | Fixed issue: When an LLM provider emitted many small SSE chunks per second (such as single-character reasoning tokens), a single streaming request could monopolize a worker process at 100% CPU, degrading availability… (opens in API7 Gateway docs) |
| 3.9.10 (opens in API7 Gateway docs) | 2026-04-22 | Fix | Fixed issue: When LLM providers returned JSON null for nullable response fields (such as prompt_tokens_details or usage), the gateway crashed because the JSON null sentinel value passed Lua truthiness checks but could… (opens in API7 Gateway docs) |
| 3.9.9 (opens in API7 Gateway docs) | 2026-04-10 | Fix | Fixed issue: The Data Plane compatibility report showed spurious warnings for valid plugin configurations. Plugins using patternProperties, conditional schemas (if/then/else), allOf, dependencies, or additionalPropert… (opens in API7 Gateway docs) |
| 3.9.9 (opens in API7 Gateway docs) | 2026-04-10 | Fix | Fixed issue: When using AI Proxy with protocol conversion (e.g., Anthropic client to OpenAI provider), stream_options.include_usage=true was injected into the pre-conversion request body instead of the post-conversion… (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Feature | AI Proxy, AI Proxy Multi: Added full support for the OpenAI Responses API (POST /v1/responses). Both streaming and non-streaming responses are handled, and all downstream plugins (RAG, content moderation, prompt decor… (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Feature | AI Proxy, AI Proxy Multi: Added native Anthropic Messages API support. Requests to /v1/messages using the Anthropic SDK format are now passed through directly to Anthropic-compatible backends without protocol conversi… (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Fix | Fixed issue: The apisix_llm_active_connections Prometheus gauge was never decremented when a plugin exited early via ngx.exit(), causing the metric to grow indefinitely and report incorrect active connection counts. (opens in API7 Gateway docs) |
| 3.9.8 (opens in API7 Gateway docs) | 2026-04-07 | Feature | Services can now be published without upstream configuration, enabling AI Proxy and other plugin-only scenarios where no backend upstream is needed. (opens in API7 Gateway docs) |
| 3.9.7 (opens in API7 Gateway docs) | 2026-03-25 | Feature | Added bidirectional protocol conversion between Anthropic and OpenAI formats. Users can send requests in Anthropic SDK format to OpenAI-compatible backends (such as DeepSeek or OpenRouter), with the gateway automatica… (opens in API7 Gateway docs) |
| 3.9.7 (opens in API7 Gateway docs) | 2026-03-25 | Fix | Fixed issue: AI request token usage statistics (prompt_tokens, completion_tokens) were inaccurate when HTTP chunk boundaries did not align with SSE event boundaries in upstream responses. (opens in API7 Gateway docs) |
| 3.9.2 (opens in API7 Gateway docs) | 2026-01-19 | Feature | AI Proxy/AI Proxy Multi: Supported four new providers: Gemini, Vertex AI, OpenRouter, and Anthropic. (opens in API7 Gateway docs) |
| 3.8.21 (opens in API7 Gateway docs) | 2025-12-23 | Fix | AI Proxy/AI Proxy Multi: Fixed issue: Headers configured in the plugin could not override downstream headers with the same name due to case-insensitivity. (opens in API7 Gateway docs) |
| 3.8.19 (opens in API7 Gateway docs) | 2025-12-09 | Fix | AI Proxy/AI Proxy Multi: Fixed issue: Request headers generated by the client or other plugins were not forwarded to the upstream. (opens in API7 Gateway docs) |
| 3.8.17 (opens in API7 Gateway docs) | 2025-11-10 | Feature | AI Proxy/AI Proxy Multi/AI Request Rewrite: Supported the aimlapi provider. (opens in API7 Gateway docs) |
| 3.8.8 (opens in API7 Gateway docs) | 2025-07-28 | Fix | Fixed issue: Fix upstream_response_time being empty by adding apisix_upstream_response_time variable to measure the time APISIX takes to communicate with the upstream and receive the response. (opens in API7 Gateway docs) |
| 3.8.2 (opens in API7 Gateway docs) | 2025-05-19 | Fix | Fixed issue: AI Proxy plugin would accept invalid endpoint URLs (e.g. missing colon in "http//localhost"). (opens in API7 Gateway docs) |
| 3.8.0 (opens in API7 Gateway docs) | 2025-04-22 | Feature | Implemented new identifier fields logging and summaries for collecting LLM request and response content. (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Supported openai-compatible provider. (opens in API7 Gateway docs) |
| 3.7.0 (opens in API7 Gateway docs) | 2025-03-10 | Feature | Supported proxying embedding model APIs. (opens in API7 Gateway docs) |
| 3.4.0 (opens in API7 Gateway docs) | 2025-01-07 | Fix | Fixed issue: Query parameters from override.endpoint are not sent to LLM. (opens in API7 Gateway docs) |
Docs