Create Model

POST /environments/{env_id}/models

Create Model

Create a model alias in an environment. The AISIX Cloud control plane creates a direct model unless another kind is selected.

Parameters

• env_id (path, string, required): Environment ID that scopes the request.

Request body

Content type: application/json

• kind (string, optional): Model kind to create. New models are direct models unless another kind is selected.

• display_name (string, required): Model alias that callers use in requests.

• model_name (string, optional): Direct model upstream identifier.

• provider_key_id (string, optional): Provider key ID that supplies the upstream credential for direct models.

• rate_limit (object, optional): Per-model rate limit.

• allOf variant 1

• rate_limit.rps (integer, optional): Requests per second.

• rate_limit.rpm (integer, optional): Requests per minute.

• rate_limit.rph (integer, optional): Requests per hour.

• rate_limit.rpd (integer, optional): Requests per day.

• rate_limit.tpm (integer, optional): Tokens per minute.

• rate_limit.tpd (integer, optional): Tokens per day.

• rate_limit.concurrency (integer, optional): Maximum in-flight requests.

• background_model_check (object, optional): Periodic upstream health probe configuration for direct models.

• background_model_check.enabled (boolean, required): Whether background health checks are active.

• background_model_check.interval_seconds (integer, required): Probe interval in seconds.

• background_model_check.timeout_seconds (integer, required): Probe timeout in seconds.

• background_model_check.prompt (string, required): Prompt sent by the background health probe.

• background_model_check.max_tokens (integer, required): Maximum tokens requested for the probe response.

• background_model_check.ignore_statuses (array, required): Upstream status codes that count as "still healthy" even when returned during a probe (e.g. 429, 408).

• background_model_check.stale_after_seconds (integer, required): How long an unhealthy probe result is honored before it expires and the model re-enters rotation. The value must be greater than or equal to interval_seconds.

• cooldown (object, optional): Per-request cooldown settings for direct models. When a target returns a configured error, times out, or has a transport error, the gateway skips it until the cooldown expires.

• cooldown.enabled (boolean, optional): Whether cooldown is active for this model.

• cooldown.default_seconds (integer, optional): Hold duration, in seconds, applied when a cooldown is triggered and no upstream Retry-After is honored.

• cooldown.max_seconds (integer, optional): Upper bound, in seconds, on any single hold. This value caps an honored upstream Retry-After header.

• cooldown.honor_retry_after (boolean, optional): Whether an upstream Retry-After response header can set the hold duration, capped at max_seconds.

• cooldown.trigger_statuses (array, optional): HTTP status codes that trigger a cooldown.

• cooldown.trigger_on_timeout (boolean, optional): Whether a request timeout triggers a cooldown.

• cooldown.trigger_on_transport (boolean, optional): Whether a transport-level error (connection failure, TLS error) triggers a cooldown.

• auto_prompt_caching (object, optional): Automatic prompt-cache breakpoint injection for normalized requests to direct Anthropic models. When enabled, the gateway adds cache-control markers to /v1/chat/completions and /v1/responses requests that carry none of their own, so callers can receive provider-side prompt-cache discounts without adding markers. The native /v1/messages route preserves caller-supplied markers but does not add them automatically.

• auto_prompt_caching.enabled (boolean, optional): Whether automatic prompt-cache injection is active for this model.

• auto_prompt_caching.ttl (string, optional): Cache lifetime for injected breakpoints. 5m is written at 1.25x the base input rate and breaks even after one cache read; 1h is written at 2x and pays off only when the cached prefix is reused across a longer session.

• timeout (integer, optional): Non-streaming request timeout in milliseconds. Accepted on every model kind except ensemble (whose own deadline is ensemble.timeout_ms); on a routing model or semantic router it is the group/router-level slot. Unset values inherit that value, then the gateway default; 0 disables the timeout.

• stream_timeout (integer, optional): Streaming read timeout in milliseconds. Accepted on every model kind except ensemble. 0 or unset falls back to the group/router stream_timeout, then to timeout, then to gateway defaults.

• retries (integer, optional): Retry attempts after a retryable upstream failure. Accepted on direct, embedding and semantic models — not ensemble, and a routing model rejects the top-level field (set routing.retries instead). Unset falls back to the group slot (routing.retries for a model group, the top-level value for a semantic router), then to the gateway default; 0 disables retrying for this model.

• allowed_cidrs (array, optional): Client-IP allowlist in CIDR notation for direct and routing models. Empty values allow all client IPs. Requests outside the allowlist receive 403 with code: ip_restricted.

• routing (object, optional): Virtual-router configuration for routing models. Shape is identical between request and response.

• routing.strategy (string, required): Target-selection strategy, applied within each priority tier (see targets[].priority). round_robin is smooth WEIGHTED round-robin over targets[].weight — equal or absent weights degrade to a plain declaration-order cycle. consistent_hash hashes a per-request key (see hash_on) over a ketama-style ring so the same key keeps landing on the same target, with weight scaling each target's share of the ring; on failure the walk follows the ring, so only the failed target's keys move. failover always starts at the tier's first target (declaration order) and moves on only after failure. The runtime-signal strategies re-rank every target best-first on each request: least_cost by the target model's combined input+output price, least_latency by a moving average of recent upstream latency, least_busy by in-flight requests divided by weight. weight is accepted with every strategy; failover, least_cost, and least_latency do not use it.

• routing.targets (array, required): Candidate direct models used by the routing model.

• routing.targets[].model_id (string, required): UUID of a direct model in the same environment.

• routing.targets[].weight (integer, optional): Target weight, default 1, accepted with every strategy. Used by round_robin (rotation share), consistent_hash (share of the hash ring), and least_busy (in-flight divided by weight); failover, least_cost, and least_latency do not use it.

• routing.targets[].priority (integer, optional): Priority tier, default 0; a HIGHER value is preferred (the APISIX node-priority convention — give backup targets -1). The strategy orders targets within each tier; a lower tier only receives traffic when every higher-tier target failed or was removed from rotation by health/cooldown filtering. Accepted with every strategy.

• routing.targets[].tags (array, optional): Tags for tag/metadata-conditional routing, orthogonal to strategy. When a request carries routing tags (via the x-aisix-routing-tags header), only targets whose tags intersect the request's are eligible; a target tagged default is the fallback used when nothing matches and for untagged requests. Omitted/empty means the target opts out of tag filtering. The configured strategy then orders whatever target set survives filtering.

• routing.retries (integer, optional): Number of retry attempts before trying another target.

• routing.max_fallbacks (integer, optional): Maximum number of fallback targets tried for one request.

• routing.retry_on_429 (boolean, optional): Whether upstream 429 responses are retryable.

• routing.when_all_unavailable (string, optional): What the router does when every target has been removed from rotation by runtime health/cooldown filtering and none remain to receive the request. fail (default) returns an error to the caller; try_anyway ignores the filtering for this request and attempts the targets regardless. Optional; omitted = fail.

• routing.hash_on (array, optional): Where the consistent_hash key comes from: an ordered chain of sources, the first one yielding a non-empty value wins. Omitted = the x-aisix-routing-key request header, falling back to the caller's API key id. Only valid with strategy: consistent_hash — sending it with any other strategy returns 400 INVALID_REQUEST. When updating a model, the routing object is a full replacement — omitting this field restores the default chain.

• routing.hash_on[].type (string, required): Which request attribute supplies the hash key: a request header or cookie (both named by name), the caller's api_key id, or the caller's resolved client_ip.

• routing.hash_on[].name (string, optional): The header or cookie name to read. Required for header and cookie sources; must be absent for api_key and client_ip — invalid combinations return 400 INVALID_REQUEST.

• routing.fallback_on_statuses (array, optional): Upstream HTTP status codes that trigger retries and failover in addition to the defaults. Out of the box, a response with a status other than 429 in the 400-499 range is treated as a request problem and returned to the caller without trying other targets. Some model services use such codes for transient conditions — overload, queue saturation, quota exhaustion — and listing those codes here (for example [408, 409]) lets the router try the remaining targets instead. Codes in the 500-599 range already trigger failover, so listing them changes nothing. Duplicate entries return 400 INVALID_REQUEST. Optional; omitted keeps the default behavior. When updating a model, the routing object is a full replacement — omitting this field clears it.

• ensemble (object, optional): Ensemble configuration for models that fan out to a panel and synthesize the result with a judge model. Panel members and the judge reference direct models in the same environment.

• ensemble.panel (array, required): Direct models that produce candidate responses.

• ensemble.panel[].model_id (string, required): UUID of a direct model in the same environment.

• ensemble.panel[].temperature (number, optional): Temperature override for this panel member.

• ensemble.panel[].seed (integer, optional): Seed passed to this panel member.

• ensemble.panel[].weight (integer, optional): Member weight used in the synthesis context.

• ensemble.judge (object, required): Direct model that synthesizes an ensemble's candidate responses.

• ensemble.judge.model_id (string, required): UUID of a direct model in the same environment.

• ensemble.judge.synthesis_prompt (string, optional): Custom instruction used by the judge during synthesis.

• ensemble.min_responses (integer, optional): Minimum successful panel responses before synthesis runs. The value must be within the panel size.

• ensemble.timeout_ms (integer, optional): Per-panel-member upstream timeout in milliseconds.

• semantic (object, optional): Semantic-router configuration. The router embeds each request, compares it with route examples, and selects the best matching direct model. If no route clears the threshold, the request uses the default model.

• semantic.embedding_model_id (string, required): UUID of an embedding model in the same environment.

• semantic.default_model_id (string, required): UUID of the direct model used when no route matches.

• semantic.threshold (number, required): Default similarity threshold for routes without their own.

• semantic.embedding_timeout_ms (integer, optional): Per-call embedding deadline in milliseconds.

• semantic.on_embedding_failure (object, optional): Behavior when the embedding call fails or times out. Use default to route to the default model, fail to reject with 503, or target to route to target_model_id.

• semantic.on_embedding_failure.mode (string, required): Fallback behavior when embedding fails.

• semantic.on_embedding_failure.target_model_id (string, optional): UUID of a direct model; set only when mode is target.

• semantic.routes (array, required): Semantic routes evaluated for each request.

• semantic.routes[].name (string, required): Route label, surfaced in the x-aisix-route header.

• semantic.routes[].target_model_id (string, required): UUID of a direct model in the same environment.

• semantic.routes[].description (string, optional): Human-readable description of this semantic route.

• semantic.routes[].examples (array, required): Example utterances matched against the request.

• semantic.routes[].threshold (number, optional): Per-route threshold override; falls back to the router threshold.

• embedding (object, optional): Embedding-modality metadata for models that point at an OpenAI-compatible /v1/embeddings endpoint.

• embedding.dimensions (integer, required): Output vector dimensionality.

• embedding.normalize (boolean, optional): Whether the endpoint already returns L2-normalized vectors.

Responses

• 201: Created

• model (object, required): Model resource. The kind value determines which variant block is present:

• direct: uses model_name and provider_key_id.

• routing: uses routing.

• ensemble: uses ensemble.

• semantic: uses semantic.

• embedding: uses embedding, plus model_name and provider_key_id.

• model.id (string, required): Model ID.

• model.env_id (string, required): Environment ID that owns this model.

• model.kind (string, required): Model kind. The value is fixed at creation.

• model.display_name (string, required): Model alias that callers use in requests.

• model.model_name (string, optional): Upstream model identifier for direct models, such as gpt-4o-2024-08-06.

• model.provider_key_id (string, optional): Provider key ID that supplies the upstream credential for direct models.

• model.rate_limit (object, optional): Per-model rate limit. null means no model-level limit.

• allOf variant 1

• model.rate_limit.rps (integer, optional): Requests per second.

• model.rate_limit.rpm (integer, optional): Requests per minute.

• model.rate_limit.rph (integer, optional): Requests per hour.

• model.rate_limit.rpd (integer, optional): Requests per day.

• model.rate_limit.tpm (integer, optional): Tokens per minute.

• model.rate_limit.tpd (integer, optional): Tokens per day.

• model.rate_limit.concurrency (integer, optional): Maximum in-flight requests.

• model.background_model_check (object, optional): Periodic upstream health probe configuration for direct models.

• model.background_model_check.enabled (boolean, required): Whether background health checks are active.

• model.background_model_check.interval_seconds (integer, required): Probe interval in seconds.

• model.background_model_check.timeout_seconds (integer, required): Probe timeout in seconds.

• model.background_model_check.prompt (string, required): Prompt sent by the background health probe.

• model.background_model_check.max_tokens (integer, required): Maximum tokens requested for the probe response.

• model.background_model_check.ignore_statuses (array, required): Upstream status codes that count as "still healthy" even when returned during a probe (e.g. 429, 408).

• model.background_model_check.stale_after_seconds (integer, required): How long an unhealthy probe result is honored before it expires and the model re-enters rotation. The value must be greater than or equal to interval_seconds.

• model.cooldown (object, optional): Per-request cooldown settings for direct models. When a target returns a configured error, times out, or has a transport error, the gateway skips it until the cooldown expires.

• model.cooldown.enabled (boolean, optional): Whether cooldown is active for this model.

• model.cooldown.default_seconds (integer, optional): Hold duration, in seconds, applied when a cooldown is triggered and no upstream Retry-After is honored.

• model.cooldown.max_seconds (integer, optional): Upper bound, in seconds, on any single hold. This value caps an honored upstream Retry-After header.

• model.cooldown.honor_retry_after (boolean, optional): Whether an upstream Retry-After response header can set the hold duration, capped at max_seconds.

• model.cooldown.trigger_statuses (array, optional): HTTP status codes that trigger a cooldown.

• model.cooldown.trigger_on_timeout (boolean, optional): Whether a request timeout triggers a cooldown.

• model.cooldown.trigger_on_transport (boolean, optional): Whether a transport-level error (connection failure, TLS error) triggers a cooldown.

• model.auto_prompt_caching (object, optional): Automatic prompt-cache breakpoint injection for normalized requests to direct Anthropic models. When enabled, the gateway adds cache-control markers to /v1/chat/completions and /v1/responses requests that carry none of their own, so callers can receive provider-side prompt-cache discounts without adding markers. The native /v1/messages route preserves caller-supplied markers but does not add them automatically.

• model.auto_prompt_caching.enabled (boolean, optional): Whether automatic prompt-cache injection is active for this model.

• model.auto_prompt_caching.ttl (string, optional): Cache lifetime for injected breakpoints. 5m is written at 1.25x the base input rate and breaks even after one cache read; 1h is written at 2x and pays off only when the cached prefix is reused across a longer session.

• model.timeout (integer, optional): Non-streaming request timeout in milliseconds. Accepted on every model kind except ensemble (whose own deadline is ensemble.timeout_ms). On a routing model or semantic router the value is the group/router-level slot applied to every target that does not set its own; on a direct or embedding model it applies to that model's own upstream call. Unset values inherit that group/router value, then the gateway default; 0 disables the timeout.

• model.stream_timeout (integer, optional): Streaming read timeout in milliseconds for first-token and inter-chunk waits. Accepted on every model kind except ensemble. 0 or unset falls back to the group/router stream_timeout, then to timeout, then to gateway defaults.

• model.retries (integer, optional): Retry attempts after a retryable upstream failure before the request gives up, or fails over to the next target inside a model group or semantic router. Accepted on direct, embedding and semantic models — not ensemble, and a routing model rejects the top-level field (its group slot is routing.retries). On a semantic router the top-level value is the router slot. Unset values fall back to that slot, then to the gateway's deployment-wide default; 0 disables retrying for this model.

• model.allowed_cidrs (array, optional): Client-IP allowlist in CIDR notation for direct and routing models. Empty values allow all client IPs. Requests outside the allowlist receive 403 with code: ip_restricted.

• model.routing (object, optional): Virtual-router configuration for routing models. Shape is identical between request and response.

• model.routing.strategy (string, required): Target-selection strategy, applied within each priority tier (see targets[].priority). round_robin is smooth WEIGHTED round-robin over targets[].weight — equal or absent weights degrade to a plain declaration-order cycle. consistent_hash hashes a per-request key (see hash_on) over a ketama-style ring so the same key keeps landing on the same target, with weight scaling each target's share of the ring; on failure the walk follows the ring, so only the failed target's keys move. failover always starts at the tier's first target (declaration order) and moves on only after failure. The runtime-signal strategies re-rank every target best-first on each request: least_cost by the target model's combined input+output price, least_latency by a moving average of recent upstream latency, least_busy by in-flight requests divided by weight. weight is accepted with every strategy; failover, least_cost, and least_latency do not use it.

• model.routing.targets (array, required): Candidate direct models used by the routing model.

• model.routing.targets[].model_id (string, required): UUID of a direct model in the same environment.

• model.routing.targets[].weight (integer, optional): Target weight, default 1, accepted with every strategy. Used by round_robin (rotation share), consistent_hash (share of the hash ring), and least_busy (in-flight divided by weight); failover, least_cost, and least_latency do not use it.

• model.routing.targets[].priority (integer, optional): Priority tier, default 0; a HIGHER value is preferred (the APISIX node-priority convention — give backup targets -1). The strategy orders targets within each tier; a lower tier only receives traffic when every higher-tier target failed or was removed from rotation by health/cooldown filtering. Accepted with every strategy.

• model.routing.targets[].tags (array, optional): Tags for tag/metadata-conditional routing, orthogonal to strategy. When a request carries routing tags (via the x-aisix-routing-tags header), only targets whose tags intersect the request's are eligible; a target tagged default is the fallback used when nothing matches and for untagged requests. Omitted/empty means the target opts out of tag filtering. The configured strategy then orders whatever target set survives filtering.

• model.routing.retries (integer, optional): Number of retry attempts before trying another target.

• model.routing.max_fallbacks (integer, optional): Maximum number of fallback targets tried for one request.

• model.routing.retry_on_429 (boolean, optional): Whether upstream 429 responses are retryable.

• model.routing.when_all_unavailable (string, optional): What the router does when every target has been removed from rotation by runtime health/cooldown filtering and none remain to receive the request. fail (default) returns an error to the caller; try_anyway ignores the filtering for this request and attempts the targets regardless. Optional; omitted = fail.

• model.routing.hash_on (array, optional): Where the consistent_hash key comes from: an ordered chain of sources, the first one yielding a non-empty value wins. Omitted = the x-aisix-routing-key request header, falling back to the caller's API key id. Only valid with strategy: consistent_hash — sending it with any other strategy returns 400 INVALID_REQUEST. When updating a model, the routing object is a full replacement — omitting this field restores the default chain.

• model.routing.hash_on[].type (string, required): Which request attribute supplies the hash key: a request header or cookie (both named by name), the caller's api_key id, or the caller's resolved client_ip.

• model.routing.hash_on[].name (string, optional): The header or cookie name to read. Required for header and cookie sources; must be absent for api_key and client_ip — invalid combinations return 400 INVALID_REQUEST.

• model.routing.fallback_on_statuses (array, optional): Upstream HTTP status codes that trigger retries and failover in addition to the defaults. Out of the box, a response with a status other than 429 in the 400-499 range is treated as a request problem and returned to the caller without trying other targets. Some model services use such codes for transient conditions — overload, queue saturation, quota exhaustion — and listing those codes here (for example [408, 409]) lets the router try the remaining targets instead. Codes in the 500-599 range already trigger failover, so listing them changes nothing. Duplicate entries return 400 INVALID_REQUEST. Optional; omitted keeps the default behavior. When updating a model, the routing object is a full replacement — omitting this field clears it.

• model.ensemble (object, optional): Ensemble configuration for models that fan out to a panel and synthesize the result with a judge model. Panel members and the judge reference direct models in the same environment.

• model.ensemble.panel (array, required): Direct models that produce candidate responses.

• model.ensemble.panel[].model_id (string, required): UUID of a direct model in the same environment.

• model.ensemble.panel[].temperature (number, optional): Temperature override for this panel member.

• model.ensemble.panel[].seed (integer, optional): Seed passed to this panel member.

• model.ensemble.panel[].weight (integer, optional): Member weight used in the synthesis context.

• model.ensemble.judge (object, required): Direct model that synthesizes an ensemble's candidate responses.

• model.ensemble.judge.model_id (string, required): UUID of a direct model in the same environment.

• model.ensemble.judge.synthesis_prompt (string, optional): Custom instruction used by the judge during synthesis.

• model.ensemble.min_responses (integer, optional): Minimum successful panel responses before synthesis runs. The value must be within the panel size.

• model.ensemble.timeout_ms (integer, optional): Per-panel-member upstream timeout in milliseconds.

• model.semantic (object, optional): Semantic-router configuration. The router embeds each request, compares it with route examples, and selects the best matching direct model. If no route clears the threshold, the request uses the default model.

• model.semantic.embedding_model_id (string, required): UUID of an embedding model in the same environment.

• model.semantic.default_model_id (string, required): UUID of the direct model used when no route matches.

• model.semantic.threshold (number, required): Default similarity threshold for routes without their own.

• model.semantic.embedding_timeout_ms (integer, optional): Per-call embedding deadline in milliseconds.

• model.semantic.on_embedding_failure (object, optional): Behavior when the embedding call fails or times out. Use default to route to the default model, fail to reject with 503, or target to route to target_model_id.

• model.semantic.on_embedding_failure.mode (string, required): Fallback behavior when embedding fails.

• model.semantic.on_embedding_failure.target_model_id (string, optional): UUID of a direct model; set only when mode is target.

• model.semantic.routes (array, required): Semantic routes evaluated for each request.

• model.semantic.routes[].name (string, required): Route label, surfaced in the x-aisix-route header.

• model.semantic.routes[].target_model_id (string, required): UUID of a direct model in the same environment.

• model.semantic.routes[].description (string, optional): Human-readable description of this semantic route.

• model.semantic.routes[].examples (array, required): Example utterances matched against the request.

• model.semantic.routes[].threshold (number, optional): Per-route threshold override; falls back to the router threshold.

• model.embedding (object, optional): Embedding-modality metadata for models that point at an OpenAI-compatible /v1/embeddings endpoint.

• model.embedding.dimensions (integer, required): Output vector dimensionality.

• model.embedding.normalize (boolean, optional): Whether the endpoint already returns L2-normalized vectors.

• model.created_at (string, required): Time when the model was created.

• model.updated_at (string, required): Time when the model was last updated.

• warnings (array, optional): Present on save operations when some data plane nodes in the environment run a gateway version that will not execute part of the saved model. Never present on reads.

• warnings[].code (string, required): How affected nodes treat the saved resource. field_ignored: the resource stays in service on those nodes but the named field has no effect there — beware when the field is restrictive, since the restriction is silently not enforced. row_rejected: those nodes refuse the whole resource, so it stops serving there entirely (or, if a previously accepted version exists, they keep serving the old values).

• warnings[].field (string, required): Dot-separated path of the field that triggered the warning, as delivered to gateway nodes.

• warnings[].min_dp_version (string, required): Oldest gateway version that fully executes the field. The literal value unreleased means no released gateway version executes it yet.

• warnings[].affected_dp_count (integer, required): Number of currently registered nodes in the environment running an older gateway version than min_dp_version.

• warnings[].affected_dp_versions (array, required): Distinct gateway versions reported by the affected nodes, capped at five entries. Nodes that report no version appear as unknown.

• 400: Request body or parameters failed validation.

• error (object, required): Error details.

• error.code (string, required): Machine-readable identifier. Examples: UNAUTHENTICATED, INVALID_REQUEST, MASTER_KEY_UNSET, INTERNAL.

• error.message (string, required): Human-readable explanation; safe to surface to operators.

• 401: Missing or invalid bearer. Response fields (application/json): identical to POST /environments/{env_id}/models, response 400 above.

• 403: Authenticated request is not permitted. Response fields (application/json): identical to POST /environments/{env_id}/models, response 400 above.

• 404: Resource not found. Response fields (application/json): identical to POST /environments/{env_id}/models, response 400 above.

• 409: Request conflicts with current resource state. Response fields (application/json): identical to POST /environments/{env_id}/models, response 400 above.

cURL

curl -X POST '$AISIX_CP/environments/{env_id}/models' -H 'Authorization: Bearer $AISIX_TOKEN'