Upstreams and Load Balancing
Definitions of backend targets, including load balancing algorithms and health checks.
An upstream defines the backend target(s) for a service. It specifies the destination nodes, how the gateway distributes traffic among them (load balancing), and how it monitors their availability (health checks).
In API7 Enterprise, an upstream is configured through its owning service — the Admin API exposes it at /apisix/admin/services/{service_id}/upstreams, not as a standalone top-level resource. To reuse the same backend across several services, define an equivalent upstream under each service.
Load balancing algorithms
API7 Gateway supports multiple algorithms to distribute traffic across backend nodes.
| Algorithm | Description | Best for... |
|---|---|---|
roundrobin | Distributes requests sequentially across nodes based on their weights. | Most general-purpose applications. |
chash | Uses consistent hashing (by URI, cookie, etc.) to map requests to the same node. | Stateful applications requiring session affinity. |
least_conn | routes requests to the node with the fewest active connections. | Workloads with varying request processing times. |
ewma | Exponentially Weighted Moving Average (Peak EWMA). Selects nodes with the lowest predicted latency. | Dynamically responding to backend performance fluctuations. |
Consistent hashing keys
When type is set to chash, the gateway hashes an attribute of each request and uses that hash to pick a backend node. Requests that produce the same value are consistently routed to the same node. Two fields control this:
hash_on: where the value is read from. Defaults tovars.key: which field to read from that source.
The format that key accepts depends on hash_on. The gateway validates the combination when the upstream is saved and rejects an invalid key with a 400 error.
hash_on | key refers to | Accepted key values | Example |
|---|---|---|---|
vars | An NGINX variable | Only one of: uri, server_name, server_addr, request_uri, remote_port, remote_addr, query_string, host, hostname, mqtt_client_id, or a query argument in the form arg_<name>. Write the name without the leading $. | remote_addr, arg_device_id |
header | A request header | The header name only, without the http_ prefix (letters, digits, -, and _; case-insensitive). | X-Api-Key |
cookie | A request cookie | The cookie name only, without the cookie_ prefix (letters, digits, -, and _). | session_id |
consumer | The authenticated consumer | Not applicable — omit key. The gateway hashes on the consumer's name. | — |
vars_combinations | One or more variables | Any string of $-prefixed variable references, resolved per request. | $request_uri$remote_addr |
hash_on: vars accepts only the fixed set of variables listed above — not the full list of built-in variables. To hash on a variable outside that set, or on a combination of values, use hash_on: vars_combinations. Reference each variable with a $ prefix, for example $request_uri$remote_addr.
If key cannot be resolved for a request (for example, the named header or argument is absent), the gateway falls back to hashing on remote_addr.
Key fields
| Field | Type | Description |
|---|---|---|
type | string | Load balancing algorithm: roundrobin (default), chash, least_conn, or ewma. |
nodes | array | List of {host, port, weight, priority} objects for static backend targets. |
scheme | string | Protocol for communicating with the backend: http, https, grpc, or grpcs. |
retries | integer | Number of retries when a request fails. Range: 0-65535. |
timeout | object | Timeout settings: {connect, send, read} in seconds. Default: 60 for each. |
retry_timeout | number | Timeout to continue retries. Default: 0 (disabled). |
pass_host | string | pass (service host), node (node host), or rewrite (custom host). |
keepalive_pool | object | Connection pool settings: {size, idle_timeout, requests}. |
hash_on | string | Used with chash to select the hash source: vars (default), header, cookie, consumer, or vars_combinations. See Consistent Hashing Keys. |
key | string | The value to hash on when type is chash. Required unless hash_on is consumer; its accepted format depends on hash_on. See Consistent Hashing Keys. |
Each timeout field accepts fractional seconds (0.5, for example) as well as whole numbers. The schema only requires a positive value; it does not enforce or round to a 0.001 (one-millisecond) floor, so a smaller fraction such as 0.0001 is also accepted and stored as given.
Service discovery
Instead of defining static nodes, use service discovery to fetch backend targets from a registry dynamically.
| Field | Description |
|---|---|
service_name | The service identifier in the registry (e.g., <registry_id>/public/group/my-service). |
discovery_type | The discovery mechanism: kubernetes, nacos, or consul. |
Health checks
The gateway performs health checks to identify and remove unhealthy nodes from the load balancing rotation.
- Active health checks: The gateway periodically sends heartbeat requests (HTTP or TCP) to the backend.
- Passive health checks: The gateway monitors real-time traffic to identify failing nodes (e.g., observing consecutive 5xx errors).
TLS to upstream
If your backend requires secure communication, configure the following fields:
client_certificate: ID of the certificate used for client authentication (mTLS).ca_certificates: List of CA certificate IDs to verify the backend's certificate.tls_verify: If true, the gateway verifies the backend's certificate against theca_certificates.
Related
Tasks that use this concept
- Configure Upstream Health Checks — monitor upstream availability with active and passive health checks.
- Upstream mTLS — present a client certificate to an upstream that requires one, and verify the upstream's own certificate.
Related concepts
- Service Discovery — dynamically resolve backend endpoints from service registries.
- Consumers and Credentials — manage API caller identities and authentication.