Skip to main content
Version: next

Routerly

One gateway. Any AI model. Total control.

Routerly is the self-hosted decision and control layer for your LLM providers. It sits between your applications and OpenAI, Anthropic, Google Gemini, Ollama and more, decides which model serves each request through composable routing policies with explainable decisions, and controls who may send what through roles, permissions, budgets, guardrails and audit.

Wire-format transparent. Compatible with the OpenAI and Anthropic SDKs out of the box: change the base URL in your client, nothing else. Routerly adds no custom headers and no non-standard fields, and returns the provider's response as received. Payloads change only for features you switch on yourself (PII scrubbing, guardrail steering, token optimizers).


Key Features​

FeatureDescription
Routers, orchestrators, passthroughA router scores models; an orchestrator forwards to a priority-ordered list of routers; a passthrough forwards the client's own credential to OpenAI or Anthropic
11 routing policiesllm, semantic-intent, model-preference, cheapest, health, performance, capability, context, budget-remaining, rate-limit, fairness, combined into one ranking, plus reusable routing profiles (auto, cheap, fast, coding)
Explainable decisionsRouting trace per request, Playground with trace and cost breakdown, per-call usage records
Budgets and cost trackingEvery call priced per token; hard limits (hourly to yearly, rolling windows) at global, router and token level, enforced before the provider is called
Guardrails and PII scrubbingRule- and judge-based content guardrails and PII policies on requests, responses, or both
8 token optimizerssession-dedup, ccr, rtk, headroom, json-table, relevance, caveman, llmlingua-2; off by default, enabled per router
ExperimentsA/B test whole routers against each other with no client change
OpenAI + Anthropic compatible/v1/chat/completions, /v1/responses and /v1/messages
Roles and auditCustom roles over 26 granular permissions, two-factor authentication, audit log
NotificationsIn-app inbox, email (SMTP, SES, SendGrid, Azure, Google) and webhooks (generic, Slack, Teams, PagerDuty, Discord)
MCP serverPersonal MCP tokens let Claude Code, Cursor, Cline and other MCP clients inspect and adjust Routerly with your permissions
Zero infrastructureNo database, no Redis: configuration lives in JSON files
Dashboard, CLI, APIEvery function is available in all three
Self-updatingUpdate check against GitHub Releases every 24 h. Channels: latest, current, next, develop. Update from the dashboard or with routerly update run

How It Works​

Point your app — or any AI tool like Cursor, Open WebUI, OpenClaw, or LangChain — at localhost:3000 instead of the provider's URL. From that moment, Routerly takes over: it picks the best available model for each request, enforces your budget, tracks every token spent, and automatically reroutes if a provider fails.

Any Client                 Routerly             Providers
────────── ──────── ─────────
Your App ──▶ Authenticate OpenAI
Cursor Policy scoring ──▶ Anthropic
Open WebUI POST /v1/ Select model Gemini
OpenClaw chat/ Forward request Ollama
LangChain completions Track cost ◀── ...
◀── Return response
  1. Your app sends a request to Routerly using a router token as the Bearer header
  2. Routerly authenticates the token and resolves the router
  3. Enabled routing policies run in parallel and score each candidate model
  4. The highest-scoring model within budget receives the request
  5. Response is forwarded back; tokens, cost, and latency are recorded
  6. On error or timeout, the next candidate is tried automatically

Use Cases​

ScenarioHow Routerly helps
SaaS & multi-tenantOne router per tenant, hard spend caps, automatic cheapest-model routing
Local-first developmentDevelop against Ollama locally, promote to a hosted model in production — same API, same code
Automatic cost reductionRoute simple tasks to cheap models, complex ones to capable models
Resilience & failoverRegister the same capability across OpenAI, Claude, and Gemini — Routerly detects failures and reroutes in real time
AI tools & chat UIsCursor, Open WebUI, OpenClaw, LibreChat — any tool that speaks OpenAI or Anthropic format
Model evaluation / A/B testingRun an experiment between routers; compare cost, latency, and error rate in the dashboard

Quick Start​

macOS / Linux:

# Install v0.5.3 (the default channel still resolves to an older release)
curl -fsSL https://www.routerly.ai/install.sh | bash -s -- --version v0.5.3

Windows (PowerShell):

# Install v0.5.3 (the default channel still resolves to an older release)
powershell -c "& ([scriptblock]::Create((irm https://www.routerly.ai/install.ps1))) -Version v0.5.3"

Then create the admin account at http://localhost:3000/dashboard, log in from the CLI, register a model and create a router:

routerly auth login --url http://localhost:3000 --email [email protected]
routerly model add --id gpt-5-mini --provider openai --api-key sk-YOUR_KEY
routerly router create --name "My App"
routerly router model add "My App" gpt-5-mini

Open http://localhost:3000/dashboard to manage everything from the web UI.

→ See the Installation guide and Quick Start tutorial for full details.


Comparison​

Last verified: 2026-10-21, against each project's public README and documentation. "n/v" means not verifiable from public documentation.

Routerly (0.5.x)LiteLLM proxyBifrostPortkey OSS gatewayOpenRouter
Self-hosted✅✅✅✅❌ hosted service
Open sourceAGPL-3.0open core, enterprise features under a commercial licenseApache-2.0 (enterprise tier exists)MIT (hosted/enterprise tier exists)n/a
OpenAI-compatible API✅✅✅✅✅
Anthropic Messages API (/v1/messages)✅✅✅ (/anthropic)n/vn/v
Database for core usenone (JSON files)PostgreSQL recommended for production; virtual keys and spend tracking use itn/vn/vn/a
Routing11 composable policies, profiles, orchestratorsfallback, weighted, latency, cost, usage, least-busy strategiesrouting rules, key/provider load balancing, fallbacksfallbacks, retries, weighted load balancingn/v
LLM- or intent-based routing✅ (llm, semantic-intent)n/vn/vn/vn/v
Budgets / spend control✅ global, router, token✅ spend tracking and limits per key/team/user✅ virtual keys, teams, customersn/vn/v
Guardrails✅ built in (rules, judge, PII)✅enterprise tier✅ input/output checks (PII redaction listed as enterprise)n/v
Token optimizers✅ 8, opt-inn/vn/vn/vn/v
MCPmanagement MCP server (not a tool gateway)MCP gatewayMCP client and serverMCP gateway (separate product)n/v
Roles and permissions✅ custom roles, 26 permissionsn/v (SSO listed as enterprise)enterprise tiern/vn/v
SSO (OIDC/LDAP)not yetenterpriseenterprisen/vn/v
Web UI✅ dashboard✅ admin UI✅ built-inlocal log consolehosted dashboard

These gateways overlap with Routerly and are not mutually exclusive: anything that speaks the OpenAI or Anthropic API can be registered behind Routerly as a custom provider.


Routerly dashboard overview showing cost cards, chart, and per-model breakdown

Next Steps​