Manage and Secure AI Traffic
Centralize LLM access with API7 AI Gateway. Route traffic to multiple providers, enforce guardrails, and control costs from one platform.
API7 AI Gateway is the AI traffic management layer of API7 Gateway. It provides a unified control point between your applications and LLM providers, enabling platform engineers to centralize authentication, enforce guardrails, control costs, and gain observability across all AI traffic — without modifying application code.
This document provides an overview of AI Gateway capabilities, architecture, and supported providers.
Architecture
AI Gateway runs as a set of plugins on the API7 Data Plane, configured through the Control Plane. Requests flow through a processing pipeline before reaching the LLM provider:
The pipeline stages correspond to AI plugins running on the Data Plane. Each stage is optional — configure only what you need.
Key Capabilities
| Capability | Description | Plugin |
|---|---|---|
| LLM Proxying | Unified API to 8+ providers, including any OpenAI-compatible endpoint | ai-proxy |
| Multi-Model Routing | Load balancing, failover, health checks, and priority-based routing across models and providers | ai-proxy-multi |
| Protocol Conversion | Anthropic to OpenAI protocol translation with automatic response conversion, transparent to clients | ai-proxy |
| Prompt Engineering | Reusable templates and system prompt injection at the gateway layer | ai-prompt-decorator, ai-prompt-template |
| Content Guardrails | PCRE-based prompt filtering and external toxicity detection | ai-prompt-guard, ai-aws-content-moderation, ai-aliyun-content-moderation |
| PII Redaction | Mask sensitive data in prompts before they reach LLM providers | ai-request-rewrite |
| Token Rate Limiting | Budget control per consumer, route, or model instance based on token counts | ai-rate-limiting |
| RAG Integration | Retrieval-Augmented Generation with Azure AI Search and Azure OpenAI embeddings | ai-rag |
| AI-Driven Transformation | Intelligent request rewriting powered by an LLM | ai-request-rewrite |
| AI Observability | Built-in logging (summaries and payloads) and Prometheus metrics for token accounting | ai-proxy |
| MCP Gateway | Expose OpenAPI services as MCP tools for AI agents | openapi-to-mcp |
Supported LLM Providers
API7 AI Gateway supports the following LLM providers with dedicated driver implementations:
| Provider | Provider Key | Guide |
|---|---|---|
| OpenAI | openai | Route Traffic to OpenAI |
| Azure OpenAI | azure-openai | Integrate Azure OpenAI |
| Anthropic | anthropic | Connect to Anthropic Claude |
| DeepSeek | deepseek | Route Traffic to DeepSeek |
| Google Gemini | gemini | Integrate Google Gemini |
| Vertex AI | vertex-ai | Route Traffic to Vertex AI |
| OpenRouter | openrouter | Access LLMs via OpenRouter |
| OpenAI-Compatible | openai-compatible | Connect Any OpenAI-Compatible LLM |
Any LLM API that follows the OpenAI chat completion format can be proxied using the openai-compatible provider type.
AWS Bedrock support with native SigV4 signing is planned for a future release.
Next Steps
- Get Started — Proxy your first LLM request in 5 minutes.
- LLM Providers — Configure a specific provider.
- Use Cases and How-To Guides — Explore advanced capabilities like routing, guardrails, and cost control.