AI Rate Limiting Configuration
Configuration reference for the AI Rate Limiting plugin (`ai-rate-limiting`): the parameters, each attribute with its type, whether it is required, its default and its accepted values.
Parameters
See plugin common configurations (opens in Apache APISIX docs) for configuration options available to all plugins.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), you should configure one of the following parameter sets, but not both:
rules- Any combination of
limitandtime_windowand/orinstances
limit — integer | string · optional · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: greater than 0
The maximum number of tokens allowed to consume within a given time interval.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables (opens in Apache APISIX docs) prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
time_window — integer | string · optional · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: greater than 0
The time interval corresponding to the rate limiting limit in seconds.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables (opens in Apache APISIX docs) prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
show_limit_quota_header — boolean · optional · default: true · since APISIX 3.12.0 · API7 Gateway 3.7.0
If true, includes the rate limiting response headers. Specifically, when using limit/time_window or instances, the headers include the instance name as a suffix:
X-AI-RateLimit-Limit-{name}shows the total quota.X-AI-RateLimit-Remaining-{name}shows the remaining quota.X-AI-RateLimit-Reset-{name}shows the number of seconds until the counter resets.
When rules is set, the headers use a prefix instead. See rules.header_prefix for details.
limit_strategy — string · optional · default: total_tokens · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: total_tokens, prompt_tokens, completion_tokens, or expression
Type of token to apply rate limiting. total_tokens, prompt_tokens, and completion_tokens values are returned in each model response, where total_tokens is the sum of prompt_tokens and completion_tokens.
When set to expression, rate limiting cost is calculated using a custom Lua arithmetic expression defined in cost_expr. Available in API7 Enterprise from version 3.9.8 and APISIX from version 3.17.0.
cost_expr — string · optional · since APISIX 3.17.0 · API7 Gateway 3.9.8
Valid values: any non-empty string (must be a valid Lua arithmetic expression)
Lua arithmetic expression for dynamic token cost calculation. Variables are injected from the LLM provider's raw usage response fields (e.g., input_tokens, output_tokens, cache_creation_input_tokens). Missing variables default to 0. Only math functions (abs, ceil, floor, max, min) and arithmetic operators are allowed. Expression syntax is validated when limit_strategy is expression, where this field is required.
Example: input_tokens + cache_creation_input_tokens computes cost from Anthropic Claude's cache-aware token usage.
Available in API7 Enterprise from version 3.9.8 and APISIX from version 3.17.0.
instances — array[object] · optional · since APISIX 3.12.0 · API7 Gateway 3.7.0
LLM instance rate limiting configurations.
name — string · required · since APISIX 3.12.0 · API7 Gateway 3.7.0
Name of the LLM service instance.
limit — integer | string · required · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: greater than 0
The maximum number of tokens allowed to consume within a given time interval.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables (opens in Apache APISIX docs) prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
time_window — integer | string · required · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: greater than 0
The time interval corresponding to the rate limiting limit in seconds.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables (opens in Apache APISIX docs) prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
rejected_code — integer · optional · default: 503 · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: between 200 and 599 inclusive
The HTTP status code returned when a request exceeding the quota is rejected.
rejected_msg — string · optional · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: any non-empty string
The response body returned when a request exceeding the quota is rejected.
policy — string · optional · default: local · since APISIX 3.12.0 · API7 Gateway 3.7.0
Valid values: local, redis, redis-cluster, or redis-sentinel
The policy for rate limiting counters. API7 Gateway requires this field; use local for configurations that do not use Redis. APISIX uses local when the field is omitted.
Redis-backed policies are available in API7 Enterprise from version 3.8.19 and in APISIX from version 3.18.0.
Set to local to store the counter in memory locally.
Set to redis to store the counter on a Redis instance.
Set to redis-cluster to store the counter in a Redis cluster.
Set to redis-sentinel to store the counter on the Redis primary node managed by Redis Sentinel, which ensures high availability by automatically promoting a replica to primary in case of failure. Redis Sentinel provides high availability for Redis when not using Redis Cluster.
redis_host — string · optional · since APISIX 3.12.0
The address of the Redis node. Required when policy is redis.
redis_port — integer · optional · default: 6379 · since APISIX 3.12.0
Valid values: greater than or equal to 1
The port of the Redis node when policy is redis.
redis_username — string · optional · since APISIX 3.12.0
The username for Redis if Redis ACL is used. If you use the legacy authentication method requirepass, configure only redis_password. Used when policy is redis, and with redis-sentinel in API7 Enterprise 3.10.5 and APISIX 3.18.0.
redis_password — string · optional · since APISIX 3.12.0
Password of the Redis node when policy is redis or redis-cluster, and with redis-sentinel in API7 Enterprise 3.10.5 and APISIX 3.18.0.
In API7 Gateway 3.10.2 or later in the 3.10 release series, and 3.9.16 or later in the 3.9 release series, the value is encrypted with AES256 before being saved to the database.
In APISIX 3.18.0 or later, the value is encrypted with AES before being stored in etcd.
redis_database — integer · optional · default: 0 · since APISIX 3.12.0
Valid values: greater than or equal to 0
The database number in Redis when policy is redis or redis-sentinel.
redis_ssl — boolean · optional · default: false · since APISIX 3.12.0
If true, use SSL to connect to Redis when policy is redis.
redis_ssl_verify — boolean · optional · default: false · since APISIX 3.12.0
If true, verify the server SSL certificate when policy is redis.
redis_timeout — integer · optional · default: 1000 · since APISIX 3.12.0
Valid values: greater than or equal to 1
The Redis timeout value in milliseconds when policy is redis or redis-cluster.
redis_cluster_nodes — array[string] · optional · since APISIX 3.12.0
List of Redis cluster nodes with at least one address. Required when policy is redis-cluster.
redis_cluster_name — string · optional · since APISIX 3.12.0
The name of the Redis cluster. Required when policy is redis-cluster.
redis_cluster_ssl — boolean · optional · default: false · since APISIX 3.12.0
If true, use SSL to connect to Redis cluster when policy is redis-cluster.
redis_cluster_ssl_verify — boolean · optional · default: false · since APISIX 3.12.0
If true, verify the server SSL certificate when policy is redis-cluster.
redis_sentinels — array[object] · optional
An array of Redis Sentinel nodes (host and port). Required when policy is redis-sentinel.
redis_master_name — string · optional
The name of the Redis master group that Sentinels are monitoring. Required when policy is redis-sentinel.
redis_role — string · optional · default: master
Valid values: master or slave
The Redis node role to connect to. Configurable when policy is redis-sentinel. Set to master to connect to the current Redis master, and set to slave to connect to a Redis replica.
redis_connect_timeout — integer · optional · default: 1000
Valid values: greater than or equal to 1
Timeout in milliseconds for establishing a connection to a Redis node. Configurable when policy is redis-sentinel.
redis_read_timeout — integer · optional · default: 1000
Valid values: greater than or equal to 1
Timeout in milliseconds for reading data from a Redis node. Configurable when policy is redis-sentinel.
redis_keepalive_timeout — integer · optional · default: 10000 for redis or redis-cluster; 60000 for redis-sentinel · since APISIX 3.15.0
Valid values: redis and redis-cluster: greater than or equal to 1000; redis-sentinel: greater than or equal to 1
Time in milliseconds that an idle Redis connection is kept alive in the connection pool before being closed. Used by all Redis-backed policies in APISIX 3.18.0. In API7 Enterprise, it is used by redis-sentinel; redis and redis-cluster support it from version 3.9.16 on the 3.9 release series and version 3.10.3 on the 3.10 release series.
redis_keepalive_pool — integer · optional · default: 100 · since APISIX 3.15.0
Valid values: greater than or equal to 1
Maximum number of idle Redis connections in the keepalive pool. Used when policy is redis or redis-cluster. Available in API7 Enterprise from version 3.9.16 on the 3.9 release series and version 3.10.3 on the 3.10 release series, and in APISIX from version 3.18.0.
sentinel_username — string · optional
Username used to authenticate with the Redis Sentinel instance. Configurable when policy is redis-sentinel.
sentinel_password — string · optional
Password used to authenticate with the Redis Sentinel instance. Configurable when policy is redis-sentinel.
In API7 Gateway 3.10.2 or later in the 3.10 release series, and 3.9.16 or later in the 3.9 release series, the value is encrypted with AES256 before being saved to the database.
In APISIX 3.18.0 or later, the value is encrypted with AES before being stored in etcd.
allow_degradation — boolean · optional · default: false · since APISIX 3.12.0 · API7 Gateway 3.7.0
If true, allow the gateway to continue handling requests without the plugin when the plugin or its dependencies become unavailable.
Available in API7 Enterprise from version 3.8.19 and in APISIX from version 3.18.0.
rules — array[object] · optional · since APISIX 3.16.0 · API7 Gateway 3.8.17
An array of rate-limiting rules that are applied sequentially.
Available in API7 Enterprise from 3.8.17 and in APISIX from 3.16.0.
count — integer | string · required · since APISIX 3.16.0 · API7 Gateway 3.8.17
Valid values: greater than 0
The maximum number of tokens allowed to consume within a given time interval.
This parameter also supports the string data type and allows the use of built-in variables (opens in Apache APISIX docs) prefixed with a dollar sign ($).
time_window — integer | string · required · since APISIX 3.16.0 · API7 Gateway 3.8.17
Valid values: greater than 0
The time interval corresponding to the rate limiting count in seconds.
This parameter also supports the string data type and allows the use of built-in variables (opens in Apache APISIX docs) prefixed with a dollar sign ($).
key — string · required · since APISIX 3.16.0 · API7 Gateway 3.8.17
The key to count requests by. If the configured key does not exist, the rule will not be executed.
The key is interpreted as a variable. The variable does not need to be prefixed by a dollar sign ($). See built-in variables (opens in Apache APISIX docs) for available variables.
header_prefix — string · optional · since APISIX 3.16.0 · API7 Gateway 3.8.19
Prefix for all rate limiting response headers. Available in API7 Enterprise from version 3.8.19 and in APISIX from version 3.17.0.
When configured, the prefix is inserted after X-AI- in the header name. For example, with header_prefix set to test, the headers become X-AI-Test-RateLimit-Limit, X-AI-Test-RateLimit-Remaining, and X-AI-Test-RateLimit-Reset.
When not configured, the index of the rule in the rules array is used as the prefix. For example, headers for the first rule will be X-AI-1-RateLimit-Limit, X-AI-1-RateLimit-Remaining, and X-AI-1-RateLimit-Reset.