Batch, Files, and Fine-Tuning

AISIX AI Gateway exposes the OpenAI-compatible Files, Batch, and Fine-tuning APIs as first-class proxy routes. Clients keep the standard OpenAI SDK calls while the gateway manages caller authentication, provider credentials, and usage attribution for completed batches.

In this guide, you will upload a batch input file, create and track a batch, and see how the gateway routes each call to the right provider.

Supported Proxy Routes

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, resource-scoped routes with api-key authentication). Vertex AI, Bedrock, and Anthropic-native batch flows use different wire and storage models and are not served by these routes yet.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by an OpenAI-compatible or Azure OpenAI provider.

Export the gateway connection and request values:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

How AISIX Routes Files and Jobs

A batch create request references a previously uploaded file id. There is no model field in the body for the gateway to route on, so AISIX uses gateway-encoded resource ids:

1. When you upload a file, name the routing model once: a model multipart field, a ?model= query parameter, or an x-aisix-model header.
2. The file id returned by the gateway (aisix-…) embeds that model. Any later call that references the id routes automatically, including batch create, file retrieve or download, and fine-tuning training_file.
3. Ids returned by create and retrieve responses (batch ids, output file ids, fine-tuning job ids) are encoded the same way, so follow-up calls need no extra hints.

Raw provider ids keep working: the gateway falls back to an explicit model query parameter or header, and then to the first OpenAI-compatible model the caller key can access. Pass an explicit model for deterministic routing.

Upload a File and Run a Batch

Upload the batch input file through the gateway, naming the routing model on the request:

curl -sS -X POST "${AISIX_PROXY}/v1/files" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "x-aisix-model: ${AISIX_MODEL}" \
  -F purpose=batch \
  -F file=@batch-input.jsonl

The response id starts with aisix- and embeds the routing model. Create the batch with that id:

curl -sS -X POST "${AISIX_PROXY}/v1/batches" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "aisix-…",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Track the batch with the batch ID returned by the create request:

curl -sS "${AISIX_PROXY}/v1/batches/aisix-…" \
  -H "Authorization: Bearer ${AISIX_API_KEY}"

When the batch completes, download the result file with the output_file_id from the batch response:

curl -sS "${AISIX_PROXY}/v1/files/aisix-…/content" \
  -H "Authorization: Bearer ${AISIX_API_KEY}"

The official OpenAI SDKs work unchanged. Point base_url at the gateway and pass the routing hint as a header on files.create.

Fine-Tuning Jobs

Fine-tuning jobs route through the encoded training_file id. The model field in the job body is the provider's base model to fine-tune and is forwarded verbatim; it is not a gateway model alias. The returned job object also names a provider base model rather than a gateway alias: the gateway does not translate this field in either direction, so the job object reports whatever the provider records for the model you submitted:

curl -sS -X POST "${AISIX_PROXY}/v1/fine_tuning/jobs" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini-2024-07-18",
    "training_file": "aisix-…"
  }'

Usage and Cost Attribution

File and job management calls, such as upload, create, list, and cancel, record zero-token usage events for visibility in logs.

When a batch retrieve first observes status: "completed", the gateway downloads the batch output file and aggregates per-line token usage by provider-billed model. AISIX then emits usage events with real token counts. These events carry a deterministic request id (batch-<id>), so repeated retrieves and gateway restarts do not double-count.

Gateway Policy Behavior

The full policy envelope applies to every route in this family, with one exception on the guardrail side. Caller API key authentication and model access lists, per-model client IP restrictions, and rate limits apply everywhere. Input and output guardrail scans run on /v1/batches and /v1/fine_tuning/jobs, over the JSON body the caller submits and the JSON the provider returns — not over the file a job points at. The five Files routes run no guardrail check in either direction, so an uploaded or downloaded file is relayed unscreened whatever is attached in the request's scope, and the JSONL a batch job processes is therefore screened nowhere in this family. See The Files Routes Are Not Screened. Matching AISIX Cloud budgets also apply. Upstream calls honor the model's request timeout, and transport failures count toward the model's cooldown when the model enables it.

Next Steps

You have now run file uploads, batches, and fine-tuning jobs through the gateway. Next, review Passthrough Routes for provider-native endpoints outside this surface.