Skip to content
Gateways and proxies
- LiteLLM - SDK and proxy exposing 100+ providers behind the OpenAI API, with cost tracking, logging, and callbacks to most platforms above; open-core, MIT outside the
enterprise/ directory.
- Portkey AI Gateway - Routing gateway with retries, fallbacks, caching, and inline guardrails.
- Bifrost - Go gateway that fronts multiple LLM providers behind one OpenAI-compatible API, with key load balancing, fallbacks, and Prometheus and OpenTelemetry telemetry; guardrails are enterprise-only.
- Agentgateway - Linux Foundation proxy for agent-to-LLM, agent-to-MCP, and agent-to-agent (A2A) traffic with routing, guardrails, and RBAC, which, with tracing configured, exports an OTLP span per upstream MCP call carrying
mcp.method.name and gen_ai.tool.name, plus per-tool call counters as Prometheus metrics.
- Traceloop Hub - Small Rust LLM gateway from the OpenLLMetry authors, with OTel emission built in.
- Kong - API gateway whose AI plugins add LLM routing, token metrics, and request logging to an existing gateway deployment.
- Recued - Self-hosted MCP gateway between AI agents and the owner’s mail, calendar, files and CRM that checks each tool call against a per-agent contract and holds writes for the owner’s approval by default, and documents no OpenTelemetry export; AGPL-3.0.