Manage and Secure AI Traffic
Centralize LLM access on API7 Gateway. Route traffic to multiple providers, enforce guardrails, and control costs from one platform.
API7 Gateway manages LLM traffic with the same data plane, control plane and policies it uses for every other API. A set of AI plugins turns a route into a unified control point between your applications and LLM providers, so platform engineers can centralize authentication, enforce guardrails, control costs, and gain observability across all AI traffic — without modifying application code.
This document provides an overview of those capabilities, the architecture, and the supported providers.
The AI plugins on this page run under your existing API7 Gateway license — no separate authorization is required. If you're comparing this against AISIX (API7's dedicated AI gateway product) or Apache APISIX's own AI plugins, see Choose the right AI gateway (opens in AISIX AI Gateway docs) to pick the right scope for your use case.
Architecture
The AI plugins run on the API7 data plane and are configured through the control plane, exactly like the rest of the plugin set. Requests flow through a processing pipeline before reaching the LLM provider:
The pipeline stages correspond to AI plugins running on the data plane. Each stage is optional — configure only what you need.
Key Capabilities
| Capability | Description | Plugin |
|---|---|---|
| LLM Proxying | Unified API to 8+ providers, including any OpenAI-compatible endpoint | ai-proxy (opens in Plugin Hub docs) |
| Multi-Model Routing | Load balancing, failover, health checks, and priority-based routing across models and providers | ai-proxy-multi (opens in Plugin Hub docs) |
| Protocol Conversion | Anthropic to OpenAI protocol translation with automatic response conversion, transparent to clients | ai-proxy (opens in Plugin Hub docs) |
| Prompt Engineering | Reusable templates and system prompt injection at the gateway layer | ai-prompt-decorator (opens in Plugin Hub docs), ai-prompt-template (opens in Plugin Hub docs) |
| Content Guardrails | PCRE-based prompt filtering and external toxicity detection | ai-prompt-guard (opens in Plugin Hub docs), ai-aws-content-moderation (opens in Plugin Hub docs), ai-aliyun-content-moderation (opens in Plugin Hub docs) |
| PII Redaction | Mask sensitive data in prompts before they reach LLM providers | ai-request-rewrite (opens in Plugin Hub docs) |
| Token Rate Limiting | Budget control per consumer, route, or model instance based on token counts | ai-rate-limiting (opens in Plugin Hub docs) |
| RAG Integration | Retrieval-Augmented Generation with Azure AI Search and Azure OpenAI embeddings | ai-rag (opens in Plugin Hub docs) |
| AI-Driven Transformation | Intelligent request rewriting powered by an LLM | ai-request-rewrite (opens in Plugin Hub docs) |
| AI Observability | Built-in logging (summaries and payloads) and Prometheus metrics for token accounting | ai-proxy (opens in Plugin Hub docs) |
| MCP Gateway | Expose OpenAPI services as MCP tools for AI agents | openapi-to-mcp (opens in Plugin Hub docs) |
Supported LLM Providers
The following LLM providers have dedicated driver implementations:
| Provider | Provider Key | Guide |
|---|---|---|
| OpenAI | openai | Route Traffic to OpenAI |
| Azure OpenAI | azure-openai | Integrate Azure OpenAI |
| Anthropic | anthropic | Connect to Anthropic Claude |
| DeepSeek | deepseek | Route Traffic to DeepSeek |
| Google Gemini | gemini | Integrate Google Gemini |
| Vertex AI | vertex-ai | Route Traffic to Vertex AI |
| AWS Bedrock | bedrock | Route Traffic to AWS Bedrock |
| OpenRouter | openrouter | Access LLMs via OpenRouter |
| OpenAI-Compatible | openai-compatible | Connect Any OpenAI-Compatible LLM |
Any LLM API that follows the OpenAI chat completion format can be proxied using the openai-compatible provider type.
Next Steps
- Get Started — Proxy your first LLM request in 5 minutes.
- LLM Providers — Configure a specific provider.
- Use Cases and How-To Guides — Explore advanced capabilities like routing, guardrails, and cost control.