Best AI and LLM Gateways (2026): 9 Compared on Real Cost, Licence and Risk

An LLM gateway sits between your application and the model providers. Your code calls one endpoint, and the gateway handles routing, provider keys, fallbacks when a provider fails, caching, spend limits and logs. Pick the wrong one and you either pay a fee on every dollar of inference you buy or you run a proxy that holds every provider key your company owns.

Search for the best LLM gateways and you will find a problem before you find an answer. Every list in the top results that we read was written by a company that sells a gateway or an AI platform, and each one puts its own product first. This guide compares nine gateways on what their vendors publish: licence, hosting model, the way they charge, and what happened to them in 2026, which was a busy year.

How we put this together. We have not used these gateways ourselves; this is a research-based comparison. Every price, limit and feature below comes from the vendor’s own pricing page, documentation or GitHub repository, read on 22 October 2026, and we link to each source. GitHub star counts and release dates come from GitHub’s public API on the same day. Where a vendor publishes a latency figure, we report it as that vendor’s claim, because none of them publishes a benchmark run by someone else. Nothing here is a test result, and no vendor paid to appear on this page or to be ranked in any position.

Who writes the LLM gateway rankings

Before the lineup, it helps to know where the existing rankings come from. On 22 October 2026 we read the “best LLM gateway” lists that ranked for the query.

Four vendor lists, four different winners

Maxim AI’s list, published 26 September 2026, puts Bifrost first, and Bifrost is Maxim’s own gateway. Braintrust’s list, from 16 May 2026, ranks Braintrust Gateway first. Future AGI’s list names its own Agent Command Center as the top pick. Opper’s list, updated 7 October 2026, ranks Opper first and does disclose that Opper is its product. The other results we found came from Dataiku, which sells its own LLM platform, and a GPU hosting company.

None of this means those products are bad. It means the order on those pages tells you who wrote the page. We do not sell a gateway, and we ranked by fit for a specific need, which we explain under each entry.

What the vendor lists leave out

Several of those lists still describe the market as it looked in 2025. Braintrust’s list includes Helicone without mentioning that Helicone was acquired this year. Only Opper’s notes that Palo Alto Networks now owns Portkey, and none of the four mentions, in its LiteLLM entry, that LiteLLM had its release pipeline compromised twice in 2026. Those facts matter more to a buyer than a feature grid, so they are in this guide.

How LLM gateways actually charge

Gateway pricing pages are hard to compare because each one meters something different. Here are the six models we found, all as published on 22 October 2026.

A fee on credit purchases

You prepay credits and the gateway adds a percentage when you buy them. Model prices then pass through at the provider’s list price. OpenRouter charges 5.5% on its Standard plan and 8% on Business. Cloudflare’s Unified Billing adds 5%, so $100 of credits costs $105. On a $20,000 monthly inference bill, a 5.5% fee is $1,100 a month.

A markup on model cost

Requesty adds a 5% markup to model costs on its pay-as-you-go plan, and its own example shows a model priced at $10 per million tokens costing $10.50.

Zero markup

Vercel AI Gateway states that it “charges no markup and no platform fee on tokens”, including when you bring your own provider key. You do pay any card processing fees, and some optional features are metered separately.

Per log or per request

Portkey charges by recorded logs: 10,000 a month free and 100,000 a month on its $49 Production plan. TrueFoundry and Kong meter requests inside a subscription.

Per model or per seat

Kong charges $100 a month for each model proxied on its Plus plan. TrueFoundry charges $25 per user per month on Pro.

Free software you run yourself

LiteLLM and Bifrost are open source and free to self-host. The cost moves to your own servers, database, upgrades and security work, and both sell an enterprise tier with no published price.

LLM gateways compared

All figures read on the vendors’ own pages on 22 October 2026. Model and provider counts are each vendor’s own description, counted in different ways, so they are not directly comparable.

How we chose and ordered these

Every gateway here had to offer one API across several model providers, publish its pricing or licence, and show signs of active development in October 2026. We ordered them by how well each serves a clear, common need, how much of its pricing is public, and how stable its ownership and release process looked on 22 October 2026. A higher position is not a claim that a gateway is faster or more reliable than the ones below it, because no independent benchmark exists to support that. We dropped gateways whose product or ownership we could not confirm, and we list them at the end.

1. OpenRouter: one hosted key for hundreds of models

OpenRouter is a hosted gateway and marketplace. You buy credits, call one OpenAI-compatible endpoint, and pick from models served by many providers. Its pricing page lists “500+ models” and “80+ providers” on paid plans, with automatic fallbacks to route around outages and rate limits.

The cost model is a fee on credit purchases, not a per-token markup. On 22 October the Standard pay-as-you-go plan carried a 5.5% fee and Business carried 8%, with inference billed at list price. Bring your own provider keys and the first $25,000 of list-price inference each month carries no fee, then 5% after that, charged against your credits. The free plan gives access to “25+ free models” with a limit of 50 requests a day, rising to 1,000 a day once you have bought at least $10 of credits. Paid models have no plan-based rate limit; OpenRouter passes through the provider’s limits.

The trade-off is control. Your prompts pass through OpenRouter’s infrastructure, it is not open source, and you cannot run it in your own cloud. Its pricing FAQ says zero data retention routing is available on every plan.

Choose it if you want to try many models fast without setting up an account with each provider. Skip it if traffic has to stay inside your own network.

2. LiteLLM: the most-starred open-source gateway here, with a security caveat

LiteLLM is an open-source proxy you run yourself. Its site describes “One API for 100+ LLMs, with spend tracking, auth, and guardrails” and counts “140+ LLM providers”. Its GitHub repository had 61,825 stars on 22 October 2026, with release v1.106.2 published the day before. The code is MIT-licensed, except for an enterprise folder that carries its own licence.

The free tier on its homepage lists 140+ provider integrations, virtual keys, budgets and teams, load balancing, request and token rate limits, LLM guardrails and logging integrations such as Langfuse and OpenTelemetry. Its GitHub README also describes retry and fallback logic across deployments and an MCP gateway. LiteLLM Enterprise adds SSO and SCIM, audit logs, air-gapped deployment and 24/7 support, with custom pricing and a 30-day trial key.

Two release compromises in 2026

LiteLLM’s own security blog documents two supply chain incidents this year. On 24 March 2026, it reported that “the compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8”, live for about 40 minutes and carrying code that stole credentials; the official proxy Docker image was not affected because it pins dependencies. On 21 October 2026, the day before we wrote this, it reported that “LiteLLM’s release pipeline was compromised and pushed v1.107.0 to the docker registry and PyPi.” That release reached PyPI and the Docker Hub tags v1.107.0, latest, main-latest and main-stable, plus the matching GitHub Container Registry images, for about 1 hour 45 minutes on PyPI and about 2.5 hours on Docker Hub. LiteLLM says earlier versions are safe, names 1.106.2 as the version to pin, and says customers on its private enterprise release were not affected.

LiteLLM disclosed both incidents publicly with times, file names and clean-up steps, which is what you want from a vendor. But a gateway holds every provider key you route through it, so a compromised build is a serious event. If you self-host LiteLLM, pin an exact version, never deploy the latest tag automatically, and keep provider keys in a secret manager you can rotate quickly.

Choose it if you need a gateway inside your own VPC and have someone to own upgrades. Skip it if nobody on your team will watch its security notices.

3. Vercel AI Gateway: zero markup on tokens

Vercel AI Gateway is a hosted gateway that you can call from any server, not just from apps deployed on Vercel. Its pricing documentation says it “charges no markup and no platform fee on tokens”, and that the same applies when you bring your own key. You buy credits, Vercel deducts each request at the provider’s list price, and you pay any payment processing fees.

There is a free tier with a monthly credit, limited to a subset of models with lower rate limits. Once you buy credits, the free credit stops and your rate limits rise. Some features cost extra when switched on, all on the pricing page as of 22 October: custom reporting at $0.075 per 1,000 writes and $5 per 1,000 queries, a team-wide provider allowlist at $0.10 per 1,000 successful requests, team-wide zero data retention at $0.10 per 1,000 requests, and trace drains at $0.05 per 1,000 traces plus $0.50 per GB.

The gateway handles provider and model fallbacks, request logs with cost and latency, and budgets for a team, project, API key or team member. Read the budget rules closely: budgets cover spend on Vercel’s own credentials, BYOK spend is metered separately, and if a BYOK request fails the gateway can retry with Vercel’s credentials and charge your credits.

Choose it if you want a hosted gateway with no fee on token spend. Skip it if you need to self-host.

4. Cloudflare AI Gateway: free caching, logs and rate limits

Cloudflare AI Gateway sits in front of your existing provider calls and adds analytics, logging, caching, rate limiting, retries and model fallback. Its pricing page, updated 15 October 2026, says the core features “are offered for free” on all plans. For accounts that created their first gateway before 24 September 2026, the log allowance is 100,000 logs in total on the Workers Free plan and 10 million logs per gateway on Workers Paid; newer accounts follow Workers Logs pricing.

You can keep using your own provider keys, or switch to Unified Billing, where you buy Cloudflare credits and get one bill. That carries a 5% fee on credit purchases, with provider prices passed through unchanged. Unified Billing currently covers OpenAI, Anthropic, Google AI Studio, Google Vertex AI, xAI and Groq through their native endpoints. Data loss prevention scanning is free; guardrails are billed as Workers AI inference.

Choose it if you want logging, caching and rate limits in front of your existing keys at no cost. Skip it if you need deep governance such as SSO-based team budgets.

5. Portkey: governance and guardrails, now owned by Palo Alto Networks

Portkey offers a hosted control plane and an open-source gateway. Its site claims access to “1,600+ LLMs via a unified API” and lists guardrails with PII redaction, caching, routing, observability, role-based access, budget limits, SSO and an MCP gateway. On 22 October its homepage banner read: “Portkey is now PRISMA AIRS AI Gateway, generally available for all enterprises.” Palo Alto Networks announced its plan to buy Portkey on 30 April 2026 and said it had closed the acquisition on 29 May 2026.

Its pricing page lists an open-source option you host yourself; a free Developer plan with 10,000 recorded logs a month and 3 days of log retention, which Portkey says is not suitable for production; a Production plan at $49 a month with 100,000 recorded logs, 30 days of log retention and $9 for each extra 100,000; and custom Enterprise pricing. Portkey says going over a log limit does not block requests, only the logging.

The open-source gateway repository is MIT-licensed with 13,250 stars, but on 22 October 2026 its last release was v1.15.2 from 12 January 2026 and its last commit was on 25 May 2026. If you plan to self-host Portkey’s gateway, ask the vendor where open-source development now happens.

Choose it if you want hosted guardrails and governance, and a security vendor behind the product is a plus for your buyers. Skip it if you want an actively updated open-source gateway to self-host.

6. Bifrost: open-source gateway with an MCP gateway built in

Bifrost is an open-source gateway from Maxim AI, under the Apache 2.0 licence. Its GitHub repository had 9,020 stars on 22 October 2026 and was updated that day. Its documentation lists semantic caching, weighted load balancing across API keys, automatic fallbacks between providers and models, hierarchical budgets and rate limits per virtual key, team and customer, and an MCP gateway that acts as both client and server. An enterprise edition has a 14-day trial and no published price.

Bifrost’s marketing leads on speed, and its numbers do not agree with each other. The product page FAQ claims “11 microseconds of overhead at 5k RPS”, a stats block on the same page says 20 microseconds, and the GitHub description says “under 200µs of overhead at 5k RPS”. The page also claims Bifrost is “50x faster than LiteLLM” at P99 latency, from Maxim’s own benchmark. Provider counts vary too: “8+ providers” on the product page and “20+ providers” in the docs. Gateway overhead is small next to model response times either way, so treat these as Maxim’s claims and measure on your own traffic.

Remember that Maxim also publishes one of the ranking lists described above, with Bifrost in first place.

Choose it if you want a self-hosted gateway with an Apache licence and MCP tool routing. Skip it if you want a hosted service with no servers to run.

7. Kong AI Gateway: for teams already on Kong

Kong AI Gateway extends Kong’s API gateway to LLM, MCP and agent-to-agent traffic. Its product page lists semantic caching, routing and load balancing, PII sanitisation, semantic prompt guards, MCP server generation and authentication, and quotas on token spend per user, model and time period. The core Kong gateway is Apache 2.0 with 44,289 GitHub stars and a 3.9.4 release on 12 October 2026, although the product page does not say which AI features are open source.

Its pricing page shows how differently Kong charges. The Plus plan starts at $25 a month plus usage, then costs $100 a month for each model proxied, up to 5 models, with 1 million requests a month included and $200 for each extra million, capped at 10 million. Serverless AI gateways cost $25 a month per control plane and dedicated cloud gateways $500 a month per control plane plus bandwidth. There is a 30-day free trial and custom Enterprise pricing.

Choose it if your platform team already runs Kong for APIs and wants AI traffic under the same rules. Skip it if you want to try many models cheaply, because the per-model fee adds up quickly.

8. TrueFoundry: enterprise deployment in your own cloud

TrueFoundry pitches its AI gateway at enterprises that need to keep data in their own environment. Its gateway page says it can run in your VPC, on-premises, air-gapped or across clouds, and that “No data leaves your domain.” Its pricing page lists VPC, on-prem and data residency under the Enterprise plan. Features include latency-based routing, weighted load balancing, fallbacks, token and cost quotas, role-based access, guardrails for PII and toxicity, MCP integration and support for self-hosted models served through vLLM, SGLang, KServe and Triton.

Pricing on 22 October: a free Developer plan for up to 3 users with 10,000 requests and tool calls per user a month; Pro at $25 per user a month with 20,000 requests per user pooled across the team and $20 per extra 100,000; and Enterprise on a custom annual contract. TrueFoundry claims “sub-3ms internal latency” and more than 10 billion requests a month; both are its own figures. Its gateway page gives both “1,600+ models” and “250+ LLMs” without explaining the difference.

Choose it if you need an air-gapped or on-prem gateway with vendor support. Skip it if you are a small team that wants pay-as-you-go access to models.

9. Requesty: a simple markup and EU data residency

Requesty is a hosted router with “600+ AI models” from “20+ providers”, per its pricing page. The free plan covers free models with 200 requests a day and includes routing, caching, fallbacks, spend tracking and EU data residency. Pay as you go adds a 5% markup on model costs, plus bring-your-own-key support, budget caps and an MCP gateway. Enterprise adds SSO, role-based access with audit logs, and guardrails with PII detection.

Choose it if you want an OpenRouter-style hosted router with EU data residency on the free plan. Skip it if you need to self-host or want zero markup.

What we left out, and why

Helicone appears on most vendor lists. Mintlify acquired it in March 2026 and said at the time that Helicone would continue in maintenance mode. On 22 October 2026 its homepage and pricing page said: “Helicone is no longer part of Mintlify. Details to come soon.” Its main repository still receives commits, but its latest GitHub release is from August 2025. Pricing is still listed, from a free Hobby plan to $799 a month for Team, but with ownership unclear we would wait for those details before building on it.

Martian and Unify were early model routers. On 22 October Martian’s homepage named no router or gateway product, and Unify’s homepage now sells “Continual learning for AI teams”. Braintrust, Future AGI and Opper bundle gateways into wider AI platforms; each ranks itself first on its own list, and we did not research them in depth for this edition.

Hosted or self-hosted: which do you need?

Hosted gateways

OpenRouter, Vercel, Cloudflare, Requesty and Portkey’s cloud plans run the infrastructure for you. You trade a fee or markup, plus the fact that prompts pass through a third party, for zero servers to manage. Check each one’s data retention settings: Vercel charges for team-wide zero data retention, and OpenRouter says zero data retention routing is available on every plan.

Self-hosted gateways

LiteLLM, Bifrost and Portkey’s open-source gateway run in your own cloud. Prompts never leave your network on the way to the provider, and there is no fee on inference. The cost is operational: a database, upgrades, monitoring and security patching. LiteLLM’s two incidents this year show what that last item means in practice.

Enterprise platforms

Kong and TrueFoundry sell governance to platform teams: SSO, role-based access, quotas and private deployment, with prices that rise per model, per seat or per request rather than with token spend.

How to choose an LLM gateway

Start from your monthly inference spend

Multiply your expected monthly model spend by each gateway’s fee. A 5% or 5.5% fee costs little at low volume and a lot at high volume. Zero-markup and self-hosted options look better the more you spend.

Decide where prompts may go

If prompts contain customer data that may not leave your network, rule out hosted gateways early and compare LiteLLM, Bifrost, Portkey’s self-hosted gateway, Kong hybrid and TrueFoundry Enterprise.

Check what is metered besides tokens

Logs, traces, team-wide policies and per-model fees all appear on these pricing pages. Portkey meters logs, Vercel meters some add-ons, and Kong charges per model.

Read the security and ownership history

A gateway holds your provider keys. Read the vendor’s security notices, pin versions if you self-host, and check who owns the product today. In 2026 alone, one gateway was acquired and later said it was no longer part of its buyer, and another was bought by a security company and renamed.

Ignore benchmark headlines you cannot reproduce

Every latency figure on these pages comes from the vendor that sells the gateway. Gateway overhead is usually small next to model response time. Run your own traffic through a short trial before deciding on speed.

A gateway gives you logs and spend data, but it is not a full tracing or testing tool. Our guides to AI agent observability tools and LLM evaluation tools cover the next layer, and if you self-host workflow tools that call models, see our comparison of n8n hosting.

Sources

• OpenRouter: website, pricing and FAQ.
• LiteLLM: website, enterprise page, GitHub repository, March 2026 security update, 21 October 2026 incident report, security blog index.
• Vercel: AI Gateway documentation (updated 14 September 2026), AI Gateway pricing (updated 8 September 2026).
• Cloudflare: AI Gateway documentation, pricing (updated 15 October 2026), Unified Billing (updated 30 September 2026).
• Portkey: website, pricing, gateway repository, Palo Alto Networks press release on the closed deal.
• Bifrost: product page, documentation, GitHub repository.
• Kong: AI Gateway page, pricing, GitHub repository.
• TrueFoundry: AI Gateway page, pricing.
• Requesty: website, pricing.
• Helicone: website, pricing, GitHub repository, Mintlify’s acquisition post. Martian: website. Unify: website.
• Vendor ranking lists: Maxim AI, Braintrust, Future AGI, Opper.
• All pages and GitHub data read 22 October 2026.

Written by

Faz

Faz is the founder of AIToolsBakery. Some tools here are tested hands on. Others are assessed from vendor documentation and pricing verified on the live page, and every review says which one it is. Sponsors can buy a position in a guide. They cannot buy the score, the criticism, or silence about a better option.

How we test and how we make money →

Related Posts

LangSmith Review (2026): The LangChain-Native LLM Observability Platform

Best AI Agent Observability Tools (2026): Tested for Tracing Multi-Step Agents

Langfuse vs LangSmith (2026): Which LLM Observability Platform to Actually Use