Route Enterprise AI Traffic to AWS Bedrock

Amazon Bedrock provides access to foundation models from Anthropic, Meta, Mistral, Amazon and others behind a single AWS-native API, with IAM authentication and regional endpoints. This guide shows how to route traffic to Bedrock through API7 Gateway using the ai-proxy (opens in Plugin Hub docs) plugin.

The gateway talks to Bedrock's Converse API, so clients keep sending OpenAI-style chat completion requests and the gateway performs the translation and the SigV4 signing.

Prerequisites

• Install cURL to send requests to the services for validation.

• Have a running API7 Gateway instance.

• Have an AWS account with model access granted for the models you intend to use, in the region you intend to use them.

• Create a token from the Dashboard and save it to an environment variable:

  export API_KEY=your-dashboard-token   # replace with your Dashboard token

• Replace {gateway_group_id} with your gateway group ID. Use default if you are following the quickstart.

• If you are following the Admin API examples, create or reuse a service in API7 Gateway. If you do not have one yet, follow Create or Reuse a Service, then save its ID to an environment variable:

  export SERVICE_ID=your-service-id         # replace with your service ID

Configure AWS Credentials

Create an IAM identity whose policy allows bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream on the models you will call, then save its access key to environment variables:

export AWS_ACCESS_KEY_ID=your-access-key-id
export AWS_SECRET_ACCESS_KEY=your-secret-access-key

The gateway signs every request with SigV4 using these credentials.

"auth": { "aws": { "access_key_id": "...", "secret_access_key": "...", "session_token": "..." } }

Configure the AI Proxy for Bedrock

Create a route with the ai-proxy (opens in Plugin Hub docs) plugin:

curl -k "https://localhost:7443/apisix/admin/routes?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  --data-binary @- <<REQ
  {
    "id": "bedrock-route",
    "service_id": "$SERVICE_ID",
    "paths": ["/bedrock"],
    "plugins": {
      "ai-proxy": {
        "provider": "bedrock",
        "provider_conf": {
          "region": "us-east-1"
        },
        "auth": {
          "aws": {
            "access_key_id": "$AWS_ACCESS_KEY_ID",
            "secret_access_key": "$AWS_SECRET_ACCESS_KEY"
          }
        },
        "options": {
          "model": "anthropic.claude-3-5-sonnet-20240620-v1:0"
        }
      }
    }
  }
REQ

services:
  - name: Bedrock Service
    routes:
      - uris:
          - /bedrock
        name: bedrock-route
        plugins:
          ai-proxy:
            provider: bedrock
            provider_conf:
              region: us-east-1
            auth:
              aws:
                access_key_id: ${AWS_ACCESS_KEY_ID}
                secret_access_key: ${AWS_SECRET_ACCESS_KEY}
            options:
              model: anthropic.claude-3-5-sonnet-20240620-v1:0

adc sync -f adc.yaml

Choosing the model

options.model accepts either form Bedrock recognizes:

Unlike most providers, options.model is not required. If you leave it out, the model comes from each request's model field, which lets one route serve several models. When options.model is set it wins, and the client's model is ignored. If neither is present at request time, the gateway answers HTTP 400.

A colon in a model ID is not a problem: the gateway URL-encodes the model into a single path segment before signing.

Validate the Configuration

Send a chat completion request:

curl "http://127.0.0.1:9080/bedrock" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a mathematician." },
      { "role": "user", "content": "What is 1+1?" }
    ]
  }'

You should receive a response similar to the following:

{
  "object": "chat.completion",
  "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "1 + 1 = 2"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 16,
    "completion_tokens": 9,
    "total_tokens": 25
  }
}

To enable streaming responses, set "stream": true in the request body. The gateway switches to Bedrock's converse-stream endpoint and converts its response for you.

Troubleshooting

Next Steps

You have learned how to route traffic to AWS Bedrock through API7 Gateway. See the Bedrock documentation and the supported models for more details.

• Multi-LLM Routing and Fallback — Route across regions or failover to other providers.
• Anthropic — Reach the same Claude models through Anthropic's own API instead.
• AI Observability and Cost Tracking — Track token usage per model.