AISIX 0.8.0

Release date: August 6, 2026

This release makes rate limiting expressive enough to describe a real quota policy. A single policy now decides which traffic it applies to through a condition tree, splits its counters along any dimension of that traffic, caps seven different quantities, and can suspend itself on a recurring schedule such as weekday off-peak hours, whole weekends, or named holidays. MCP servers gain an endpoint that serves one server and keeps its original tool names, so a client written against that server works unchanged. Token, spend, and request metrics now cover every endpoint that reports usage rather than only chat and messages, and audio transcription is priced by the duration of the audio.

Behavior Changes

• The gateway now treats api_base as the upstream root on every OpenAI-family endpoint. Previously /v1/chat/completions appended the endpoint to the configured base verbatim, while /v1/responses, /v1/rerank, /v1/audio/*, /v1/realtime, /v1/files, /v1/batches, and /v1/fine_tuning/jobs inserted a /v1 segment unless the base already ended in one, so a key whose upstream root was not /v1 served chat correctly and returned 404 everywhere else. Both paths now read the base the same way. A base carrying a path but no version segment builds https://proxy.corp/openai-shim/responses where it previously built .../openai-shim/v1/responses; move the /v1 into api_base if you relied on the old behavior. A bare host still has /v1 added, a base ending in /v1 is unchanged, and Anthropic is unchanged in every base shape. See OpenAI-Compatible Vendors.
• Histogram bucket boundaries changed for two metrics. aisix_request_ttft_seconds drops its two edges below 50 ms, because it measures the upstream time to first token and never observes values there, and aisix_request_e2e_latency_seconds gains edges at 420 and 600 seconds so histogram_quantile() interpolates past 300 seconds instead of pinning to it. Dashboards and recording rules that depend on the previous edges need updating. Both sets are now configurable per metric under observability.metrics.buckets. See Metrics Reference.
• Time to first token now counts a reasoning model's first reasoning delta. Only ordinary content stamped the measurement before, so a model that reasons before it answers reported a later first token than the caller observed. Reported values for those models drop accordingly. See Metrics and Logs.
• GET /v1/models now lists model groups alongside direct models, semantic routers, and ensembles. A model group was the only virtual alias filtered out of the listing, so a caller key scoped to a group as its public entry point received an empty list. No authorization changed, since those names were always callable by those keys. Only clients that assert an exact model list are affected. See Model Aliases.

New Features

Rate Limiting

• A rate limit policy can now be written in a conditional form that carries a tree of conditions instead of a single scope. Conditions match on team, member, caller API key, model, model name, and provider, combine through explicit and and or groups up to three levels deep, support negation, and match strings by equality, list membership, or regular expression. One resource can therefore hold different quotas for different traffic instead of one flat limit. See Rate Limit Policies.
• A conditional policy splits its counters with group_by, which buckets the counters per team, per member, per caller key, per model, or any combination of those. A single policy now enforces a per-tenant quota that previously required one policy per tenant.
• A policy can cap seven quantities in any combination: requests per second, minute, hour, and day, tokens per minute and per day, and concurrent requests.
• A policy that matches on a model property reserves its quota where the concrete model is known, which is per target for a routing or ensemble parent. An over-limit target is treated as a failed attempt and fails over to the next one, rather than consuming the parent alias's own budget.
• A rate limit rejection now identifies the policy that produced it. The 429 response body carries the policy's id and name, and aisix_ratelimit_rejections_total counts rejections on every endpoint and is labeled by the enforcing layer and policy. With several policies live, an unattributed rejection could not be traced to its cause.
• A policy can carry recurring suspension windows during which it is not enforced. Each window selects days either by weekday or by explicit dates, takes wall-clock start and end times in its own IANA time zone, and treats an end at or before the start as crossing midnight and belonging to the day it starts on. Multiple windows form a union, enforcement resumes automatically, and the counters are not reset by the transition, so suspending and resuming inside one rate window cannot be used to clear consumed quota. In AISIX Cloud, the Dashboard exposes these as a schedule list on the rate limit policy form. See Rate Limit Policies.

MCP Gateway

• A new per-server endpoint at /mcp/{server} serves a single registered MCP server and lists its tools under their original names, without the <server>__ prefix the aggregate endpoint applies. Both the bare and the prefixed spelling are accepted when calling a tool, and tool access control is always evaluated on the prefixed form. A client written against one server's own tool names therefore works through the gateway unchanged. See MCP Gateway Overview.

Deployment

• A new proxy.url_rewrites setting rewrites the request path at the gateway entry, matched by regular expression with capture-group templates in the replacement. Patterns are validated at startup and can be supplied through an environment variable for deployments that ship no configuration file. This lets a client whose base URL cannot be changed reach a different gateway path. See URL Rewriting.

Pricing

• Audio models can now be priced by the duration of the audio rather than by tokens. A transcription request reports the audio's length as its cost basis, and the model's price is expressed per minute of audio. In AISIX Cloud, the Dashboard accepts these rates on the model pricing form. See Audio and Model Pricing.

Improvements

• The detailed request metrics and the token and spend metrics now cover every endpoint that reports usage, including responses, embeddings, rerank, audio, images, and the realtime surface. They previously covered only chat completions and messages, so usage on the other endpoints was recorded in the usage log but missing from the metrics. See Metrics Reference.
• Configuration reads are now forward compatible. A resource document carrying a field this gateway version does not recognize loads and serves, with the unknown fields ignored and reported through partially_compatible on the configuration status endpoint, a deduplicated warning, and a new gauge. Previously an unknown field rejected the whole document, which turned every additive change on the control plane into a breaking one for a gateway that had not yet been upgraded — a caller API key carrying a new field stopped authenticating. Write paths remain strict and still reject unknown fields. See Configuration Status.
• A rejected configuration update no longer costs the last good value. The gateway keeps serving the value it previously loaded for that key across a resynchronization and a restart, instead of dropping the row. See Configuration Propagation.
• Errors that a provider reports inside an already committed stream are now surfaced with the provider's own status and message, across OpenAI-family, Anthropic, Gemini, and Amazon Bedrock streams. An Anthropic error event arriving mid-stream was previously discarded and the truncated stream closed as though it had completed, so a client could not tell a cut-off response from a finished one. Such a stream now ends with an error frame and no completion marker. See Streaming.
• A request refused for exceeding the body limit now reports how the reading of its body ended, through a dedicated metric, so a refusal whose client disconnected is distinguishable from one whose body was read to completion. See Metrics and Logs.
• The gateway warns once when it reaches a credentialed MCP, OpenAPI, or A2A upstream over cleartext http, deduplicated per server and address, so the mistake is visible without filling the log. See MCP Upstream Authentication.
• A request to /v1/realtime that is not a WebSocket upgrade is now recorded and answered in the gateway's own error format, and path rejections elsewhere in the proxy go through the same path as other early refusals, so they appear in the access log and the request count. See Realtime.
• The AISIX Cloud Admin API reference now documents every operation and schema it exposes, and its base URL is an editable field rather than a fixed relative path. The reference is served from the API7 documentation site, not from the reader's control plane. Readers can therefore set the base URL to their Hybrid Cloud or On-Premises control plane so the examples use the correct endpoint.

Fixes

• A streaming transcription now records the usage it reports. Such a request logged zero tokens and therefore cost nothing, while the same transcription made without streaming was billed normally. See Audio.
• The offline package's quickstart script now prints a Dashboard address that works. It previously printed the container's internal port, so following the printed URL failed on the default installation path. See On-Premises Quickstart.
• The Dashboard's model pricing form now accepts audio rates below one cent per minute, which the audio pricing of most providers requires.
• Budget threshold guidance in the Dashboard no longer suggests that a hard stop can overshoot its limit by an arbitrary amount. See Budgets.