Higress
Open-source AI gateway for routing, securing and observing traffic to LLM APIs, MCP servers and inference workloads.
Higress is an open-source AI gateway, hosted as a CNCF Sandbox project, that sits between applications and large language models. Higress gives one calling interface for many model providers and adds fallback, content filtering, caching and token quotas. Developers deploy Higress locally with Docker or use the managed cloud edition on Alibaba Cloud.
Best for Engineering teams that call several model providers or expose MCP tools and need central routing, quotas and security.
- Listed in
- Developer APIDevOps & Monitoring
- Platforms
- API
- Pricing model
- Free
- Free plan
- Yes
- API
- Yes
- Added
Last verified: . Information taken from higress.ai.
What is Higress?
Higress is an open-source AI gateway built on Istio and Envoy and maintained as a Sandbox project under the Cloud Native Computing Foundation (CNCF). It handles three kinds of AI traffic through one entry point: calls to model APIs, requests to MCP tools and inference workloads running on Kubernetes. For model access it converts between the protocols of more than a hundred common models, so applications use a single request format, and it can fall back to another model when one fails. Further gateway functions include content safety checks on prompts and responses, exact and semantic caching of results, and counting of tokens per consumer with limits. Existing services can be exposed as MCP servers, and behaviour is extended with Wasm plugins. A commercial, fully managed edition is offered as Alibaba Cloud AI Gateway.
Key Features
Unified Model Access
Translates one request format to the protocols of 100+ models and supports model-level fallback.
MCP Server Hosting
Receives MCP requests and adds authentication, traffic scheduling, parameter mapping and audit to existing services.
Content Safety
Filters data sent to external models and screens output returned to users.
Result Caching
Stores exact and semantically similar answers to cut token use and latency on repeated questions.
Token Quotas
Tracks token consumption per consumer and restricts callers that exceed their allowance.
Wasm Plugins
Extends the gateway with plugins, including many ready-made ones from the community.
Use Cases
- Multi-Provider Routing
- A platform team puts OpenAI, Claude and local models behind one internal endpoint.
- Turning APIs into MCP Tools
- A company publishes existing HTTP services as MCP servers for its agents.
- Cost Control
- An administrator sets token limits per team and reviews usage data.
- Prompt and Output Screening
- A security engineer adds safety checks without changing application code.
- Local Evaluation
- A developer runs the gateway in Docker to test features in a few minutes.
Things to know
- Two editions
- The open-source gateway is self-managed, while Alibaba Cloud AI Gateway is a managed commercial service with serverless and enterprise tiers.
- Cloud pricing in yuan
- The managed serverless tier is advertised with usage-based billing quoted in Chinese yuan.
- Documentation license
- The project's documentation is distributed under CC-BY-4.0.
FAQs
What is Higress?
Higress is an open-source AI gateway that manages traffic between applications and LLM APIs, AI agents and MCP servers.
Is Higress free?
Yes. Higress is open source and free to deploy yourself; a separate commercial cloud edition is sold through Alibaba Cloud.
How do I try Higress?
Higress can be started locally with a one-click Docker deployment that requires no signup, and the site says core features can be verified in about five minutes.
Which models does Higress support?
Higress states that it performs unified protocol conversion for more than 100 common models, including OpenAI, Claude and locally hosted LLMs.
Does Higress support MCP?
Yes. Higress can host MCP servers and convert existing services into them, handling authentication and traffic control at the gateway.
Who is behind Higress?
Higress is a Sandbox project of the Cloud Native Computing Foundation (CNCF); its commercial edition is offered on Alibaba Cloud.
Who it's for
- Platform engineers
- Backend developers
- DevOps teams
- AI application teams
Used in these workflows
- Building an AI-powered app: the tool stack, layer by layerStage 3: Model APIs, gateways and inference hosting
Each guide lists the stages of one job and the tools that help at each stage. All guides
Comments
No comments yet. If you have used Higress, tell other readers how it went.
Alternatives to Higress
| Tool | Free plan | Starting price | API |
|---|---|---|---|
 Kong KonnectKong's unified platform for managing and governing API, AI, MCP and event traffic through gateways. |
No | $25/mo | Yes |
 LangChainOpen source agent frameworks plus the LangSmith platform for tracing, evaluating and deploying AI agents. |
Yes | — | Yes |
 TrueFoundryEnterprise AI gateway for routing, governing and observing LLM, MCP and agent traffic in your own infrastructure. |
Yes | — | Yes |
 VercelCloud platform for deploying web apps and AI agents, with an AI gateway, sandboxes, workflows and a global CDN. |
Yes | $20/mo | Yes |
 APIParkUnified gateway for calling, authenticating, billing and monitoring AI models, APIs and MCP services. |
— | — | Yes |
Listed in the same categories as Higress. A starting price appears only where we checked it on the maker’s pricing page; a dash means not checked or not published. All Developer API tools