AISIX 0.5.0

Release date: July 24, 2026

This release adds a unified video generation endpoint, budget threshold alerts that notify an external system, and automatic prompt caching for Anthropic models. It also corrects several cases where retries, timeouts, rate limits, and usage accounting did not apply on every request path.

New Features

Video Generation

• A new video generation endpoint accepts prompt-to-video tasks on /v1/videos, polls their status, and returns the finished video, following the same three-stage shape as the OpenAI video API. Providers currently mapped are Alibaba Cloud Model Studio, Zhipu AI CogVideoX, and Volcengine Ark Seedance.
• Video requests now pass through the same gateway controls as chat traffic: model aliases, caller API key access checks, client IP allowlists, model-level rate limits, and input guardrail scanning of the prompt. Previously video traffic could only reach a provider through passthrough, which applies none of these per-model controls.
• Task state is not stored by the gateway. The returned video ID carries the routing information, so status and download calls work against any gateway instance.
• Video submissions are recorded in usage logs with zero tokens. Duration-based cost accounting is not yet applied, so video traffic does not consume budgets in this release.

Guardrails

• A new Alibaba Cloud AI Guardrails kind calls the MultiModalGuard service and acts on its suggestion verdicts. When the service returns masked content, the gateway writes the masked text back into the request instead of rejecting it outright.
• The gateway records aisix_guardrail_latency_seconds, a per-execution latency histogram that separates guardrail overhead from upstream model latency. See Metrics.
• Responses from Alibaba guardrail services keep their upstream request ID, and the gateway correlates it with its own request ID for cross-system troubleshooting.

Budgets and Alerts

• Budget threshold alerts notify an external system when spending crosses a configured percentage of a budget. Notification channels support generic webhooks and Slack.
• The dashboard budgets page now manages budgets at every scope from one place, and budgets are part of the AISIX Cloud Admin API contract.

Prompt Caching

• Models can enable automatic prompt caching so the gateway inserts Anthropic cache breakpoints into qualifying requests. Callers get prompt-cache savings without changing their client code, and the option is configurable per model from the dashboard.

Usage Reporting

• When an upstream returns no usage block, the gateway now estimates token counts locally rather than recording zero. Estimated records are flagged as such in usage reporting and on the dashboard logs page, so estimated and reported usage stay distinguishable.
• The aisix_llm_tokens_by_client_total metric gained a model label, and the metric is now recorded for the responses endpoint.
• Built-in client type detection covers more coding agents, and operators can add their own user-agent mapping rules for clients the gateway does not recognize.

Deployment

• A new aisix export command writes a resources.yaml file from a running etcd store, which produces a starting configuration for standalone mode from an existing deployment.
• The gateway can run with the admin listener disabled through the admin.enabled setting, for deployments that configure the gateway declaratively and want no write API exposed.
• The status listener reports per-model runtime health, so upstream reachability can be checked without sending a model request.
• The gateway reports the hash of the configuration it has applied in its heartbeat, and the control plane exposes gateway nodes and rejected resources in its public spec. This makes it possible to confirm which gateways have taken up a configuration change. See Configuration Propagation.

Improvements

• The upstream secret of a provider key can be rotated in place without recreating the key or re-pointing the models that use it.
• The dashboard provider picker is searchable and lists providers under their proper display names.
• The dashboard logs page shows the whole upstream error message instead of truncating it, and its upstream filter no longer offers model groups, which are not upstreams.
• The logs query window follows the organization's usage retention setting instead of a fixed range.
• The control plane reports why a request was rejected instead of returning a generic failure.
• The Admin API write path is deprecated in favor of declarative configuration. It still works in this release. Declare dynamic resources in a resources.yaml file instead, or manage them through the AISIX Cloud Admin API.

Fixes

• The retries setting is now honored on streaming chat requests, which previously ran a separate code path that did not retry.
• On Azure models, stream_timeout applies to the gap between chunks as documented, instead of capping the whole response, so long but healthy streams are no longer cut off.
• Connection failures report the underlying transport cause rather than a generic upstream error, and the connection layer now applies explicit bounds instead of library defaults.
• Client-supplied cache_control markers survive the OpenAI-to-Anthropic bridge, so callers using an OpenAI client against an Anthropic model keep their prompt-cache savings.
• Ensemble panel and judge sub-calls estimate their usage when the backend reports none, so ensemble requests no longer under-report tokens.
• The passthrough tunnel enforces the rate limit of the model named in the request body.
• Group dispatch applies each routing target's own model rate limit and client IP allowlist, instead of only checking the group entry point.
• Per-attempt error messages are no longer clipped at 256 characters, so upstream failures stay readable in logs.
• Minted-token caches for providers that exchange credentials for short-lived tokens are keyed on the whole credential, so a key rotation takes effect immediately instead of after the old token expires.
• The dashboard pod mounts a writable cache directory, which fixes image requests that could hang indefinitely in Helm deployments.
• The playground accepts only dashboard sessions. A personal access token can no longer spend provider quota through it.