Doubleword Open Sources the World’s Fastest AI Gateway

Today, we’re open-sourcing the Doubleword Control Layer - the world’s fastest open-source AI Gateway that delivers 15× higher throughput and 450× lower latency overhead than LiteLLM. 

The Doubleword Control Layer provides a single, secure interface for routing, managing, and governing inference activity across models - whether open-source or proprietary.

With the Control Layer, enterprises can:

• Seamlessly switch between models - open or proprietary, hosted or self-hosted
• Turn any model into a production-ready API - with built-in auth, rate limits, and user controls
• Centrally monitor and audit all inference activity - for governance, compliance, and optimization

Try it out via the Github repo here.

Why an AI Gateway Matters

In today’s fast-moving AI landscape, relying on a single model or provider is a liability - new, faster, and more capable models are released constantly. Teams must juggle shifting APIs, compliance requirements, and performance constraints. A well-designed gateway sits between applications and model endpoints, offering a single, stable interface while giving teams granular control over how inference traffic flows.

And because every query to every model passes through the gateway, performance and reliability are critical. It has to feel invisible to users - while giving administrators total control.

“Building AI applications at scale means balancing flexibility, speed and control - and too often teams have to choose.” said Meryem Arik, CEO of Doubleword “By open-sourcing the fastest and most reliable AI gateway, we’re giving teams a foundation they can own - open, performant, and fully in their control.” 

A High-Performance Gateway, Built for Scale

The Control Layer is the world’s fastest AI gateway - engineered for enterprise-grade performance and developer-first usability:

• 15.3× faster throughput than LiteLLM
• 450× lower latency overhead than LiteLLM, adding just 2ms at 100 concurrent users
• Unified API for routing across multiple model providers
• Native compatibility with Doubleword’s Inference Stack for deploying open source models
• Built-in observability & control - logging, metrics, and request tracing
• Extensible and lightweight, with minimal configuration

As Fergus Finn, Doubleword’s CTO, explains:

“AI gateways should be invisible to the user - adding control without adding latency. When we found existing options fell short, we built the Doubleword Control Layer from the ground up for uncompromising performance, safety, and reliability, powered by Rust.”

The Doubleword Control Layer is now open-source - enabling any team to run world-class inference infrastructure entirely within their own environment. It’s a core part of our InferenceOps mission: to make AI inference centralized, governed, and lightning-fast - without compromise.

👉 Explore the repo: https://github.com/doublewordai/control-layer

👉 Check out the docs, demo video, and full benchmarking write-up: https://docs.doubleword.ai/control-layer/

‍