Skip to main content
Version: next

Management API

The management API is used by the dashboard and CLI. All endpoints require a JWT session token.

Base URL: http://localhost:3000/api

Authentication: Authorization: Bearer <jwt>

Obtain a JWT via POST /api/auth/login.


Authentication​

Login​

POST /api/auth/login
{ "email": "[email protected]", "password": "your-password" }

Response:

{
"token": "eyJ...",
"refreshToken": "a3f8c2...",
"user": { "id": "uuid", "email": "[email protected]", "role": "admin", "permissions": [] }
}
  • token - short-lived JWT (1 hour). Use as Authorization: Bearer <token> on all other endpoints.
  • refreshToken - opaque token used to obtain new access tokens without re-entering credentials. Store securely; see POST /api/auth/refresh. Rotates on every use.

Refresh​

POST /api/auth/refresh

This endpoint is public (no Authorization header required).

{ "refreshToken": "a3f8c2..." }

Response:

{
"token": "eyJ...",
"refreshToken": "b9d4e1...",
"user": { "id": "uuid", "email": "[email protected]", "role": "admin", "permissions": [] }
}

Issues a new 1-hour access token and a new refresh token (rotation). The previous refresh token is immediately invalidated - replace it with the value returned in the response. Returns 401 if the token is invalid or has already been used/revoked.

note

The CLI and dashboard perform this refresh automatically - the CLI tries silently when the token expires or is within 5 minutes of expiry; the dashboard retries on any 401 response. Both clients persist the new refresh token automatically.


Setup​

Check Setup Status​

GET /api/setup/status

Returns { "configured": false } if no admin account exists yet; { "configured": true } otherwise.

Create First Admin​

POST /api/setup/first-admin

Only available when configured: false.

{ "email": "[email protected]", "password": "secure-password" }

Me (Current User)​

Get Profile​

GET /api/me

Returns id, email, roleId, permissions (the role's current permissions, resolved on every call) and language when set. The dashboard reads it when restoring a session so a role change applies after a reload.

Set Language​

PATCH /api/me/language
{ "language": "it" }

Sets the caller's own dashboard language (1-10 characters). No extra permission: a user can change only their own. GET /api/me and the login response return language.

Update Profile​

PUT /api/me
{ "email": "[email protected]", "currentPassword": "old", "newPassword": "new" }

Models​

A model listed here (ModelConfig-shaped) is always backed by a Model Instance bound to a Connection under the hood; these endpoints are a convenience layer over that pair. Each model in the response carries a connectionId pointing at the connection it resolves against. See Connections and Dashboard: Models: Preconfigured vs Custom Connection for the higher-level explanation.

Model Catalog​

GET /api/models/catalog

Auth: Authorization: Bearer <jwt> (requires model:read; settings:read is not needed)

Returns the known models per provider, each flagged isConfigured when already registered. It feeds the model picker, so it follows the model permissions, unlike the catalog repository settings routes (/api/catalog/*), which need settings:read or settings:write.

List Models​

GET /api/models

Auth: Authorization: Bearer <jwt> (requires model:read)

Response 200: array of models, including those on disabled connections. Each entry includes connectionId; secret fields (apiKey, cfClearance, and the other provider-specific credential fields) are always redacted, use Get Model API Key to read the plaintext key.

Create Model​

POST /api/models

Auth: Authorization: Bearer <jwt> (requires model:write)

{
"id": "gpt-5-mini",
"provider": "openai",
"apiKey": "sk-...",
"inputPerMillion": 0.25,
"outputPerMillion": 2.0,
"contextWindow": 128000,
"capabilities": { "functionCalling": true, "json": true },
"fieldOverrides": { "inputPerMillion": true }
}

Request fields:

  • connectionId (optional): bind the model to an existing connection instead of supplying credentials inline. When present, apiKey/endpoint/cfClearance and the other credential fields are ignored; the connection's provider must match provider (400 otherwise, 404 if connectionId does not exist).
  • When connectionId is absent, apiKey/endpoint/cfClearance (and any provider-specific credential fields) are used to create a dedicated, single-model connection with id conn-for-<id>, and the model binds to it. This is the same wire shape the endpoint accepted before the connections cutover, so existing integrations are unaffected.
  • timeout (optional): upstream request timeout in milliseconds, positive integer (default 60000 when omitted). Non-integer or non-positive values return 400. Returned on the model. On PUT, omitting it keeps the current value and null resets it to the default.
  • fieldOverrides (optional): object mapping field names to true to lock them against catalog sync. Supported fields: inputPerMillion, outputPerMillion, cachePerMillion, cacheWritePerMillion, pricingTiers, contextWindow, capabilities.

Response 201: the created model (redacted), including connectionId.

  • catalogDefaults (if model is in catalog): last known catalog values for each tracked field
  • fieldOverrides (if any): which fields are locked against auto-sync

Errors: 409 a model with this id already exists · 400 connectionId given but its provider does not match provider · 404 connectionId given but not found · 403 insufficient permissions

Get Model​

GET /api/models/:id

Response includes:

  • connectionId: the connection this model currently resolves against
  • catalogDefaults (if model is in catalog): object with keys: inputPerMillion, outputPerMillion, cachePerMillion, cacheWritePerMillion, pricingTiers, contextWindow, capabilities (whichever were synced from the catalog)
  • fieldOverrides (if any): object with field names as keys, all values set to true

Update Model​

PUT /api/models/:id

Request body (all fields optional):

{
"inputPerMillion": 0.5,
"fieldOverrides": { "inputPerMillion": true, "contextWindow": false }
}

Changing a field value automatically sets fieldOverrides[fieldName] = true. To unlock a field for auto-sync, send "fieldOverrides[fieldName] = false". When all overrides are cleared, the fieldOverrides object is removed from the model config.

connectionId rebinding:

  • Send connectionId to rebind the model to a different existing connection (same provider-match / 400/404 rules as create). If the model was previously on its own dedicated conn-for-<id> connection and no other model still references it, that dedicated connection is deleted as part of the rebind.
  • Omit connectionId: if the model is currently on a shared connection and the body carries no apiKey/cfClearance, the shared binding is left untouched (a plain field edit does not silently detach the model from its shared connection). If the model is on its own dedicated connection, or the body includes new credentials, the dedicated conn-for-<id> connection is created/updated from them (empty apiKey/cfClearance keeps the currently stored credentials).
  • Renaming a model's id (via body id) also renames its dedicated connection from conn-for-<old-id> to conn-for-<new-id> when that connection is still single-model.

Delete Model​

DELETE /api/models/:id

Errors: 404 model not found · 409 { error, usedBy } the model is still referenced by a router or router token (usedBy = router ids; nothing is deleted) · 403 insufficient permissions

If the model owned its dedicated conn-for-<id> connection and no other model instance references it, the connection is deleted along with the model.

Get Model API Key​

GET /api/models/:id/apikey

Auth: Authorization: Bearer <jwt> (requires model:write)

Returns the plaintext credential of the connection the model is currently bound to. To rotate the key, use Update Model (or, for a model on a preconfigured connection, Update Connection) with a new apiKey.

Response 200:

{ "apiKey": "sk-..." }

apiKey is null for oauth/web-session providers (anthropic-oauth, openai-web, etc.), whose credentials are never returned in plaintext.

Errors: 404 model not found · 403 insufficient permissions


Provider Descriptors​

List Provider Descriptors​

GET /api/providers/descriptors

Auth: Authorization: Bearer <jwt> (requires connections:read)

Returns the static registry of provider types Routerly knows how to connect to (distinct from the model catalog - see Catalog). Used by the dashboard to populate the provider dropdown when creating a connection.

Response 200:

[
{
"id": "openai",
"label": "OpenAI",
"protocol": "openai",
"supportLevel": "native",
"nativeCapabilities": { "vision": true, "functionCalling": true, "json": true }
},
{
"id": "anthropic-oauth",
"label": "Anthropic (OAuth)",
"protocol": "anthropic",
"supportLevel": "oauth",
"nativeCapabilities": { "thinking": true, "vision": true, "functionCalling": true, "json": true }
}
]

Fields:

  • id - provider identifier, used as providerId on a connection
  • protocol - wire protocol the connection speaks: openai, anthropic, gemini, or custom
  • supportLevel - how credentials are supplied: native (plain API key), compatible (OpenAI-compatible custom endpoint, plain API key), oauth (OAuth access/refresh token pair, encrypted at rest), web (browser session cookie, encrypted at rest)
  • nativeCapabilities - capability flags (thinking, vision, functionCalling, json, embedding) the provider natively supports, used as defaults for model instances

Errors: 403 insufficient permissions


Connections​

A connection stores credentials for one account with one provider (see providerId, from Provider Descriptors). A connection does not expose any models by itself - create Model Instances on top of it to make models routable.

List Connections​

GET /api/connections

Auth: Authorization: Bearer <jwt> (requires connections:read)

Response 200: array of connections with credentials always undefined (never returned by any connections endpoint, on any provider supportLevel).

[
{ "id": "conn-uuid", "providerId": "openai", "label": "Main OpenAI", "enabled": true }
]

Errors: 403 insufficient permissions

Create Connection​

POST /api/connections

Auth: Authorization: Bearer <jwt> (requires connections:manage)

{
"providerId": "openai",
"label": "Main OpenAI",
"credentials": { "apiKey": "sk-..." },
"endpoint": "https://api.openai.com/v1",
"enabled": true
}

Fields:

  • providerId - must match a known provider id from Provider Descriptors (required)
  • providerName - upstream provider behind a custom connection, e.g. deepseek (optional). Free text: it names the service the endpoint belongs to, and models created on the connection use it as their ID prefix (deepseek/deepseek-r1). Routing still dispatches on providerId
  • label - unique name for the connection (optional). Blank or absent means the server generates one from the provider (openai, then openai-2, openai-3, ...), or from providerName for a custom connection. A name another connection already uses is rejected
  • credentials - arbitrary key-value object; shape depends on supportLevel (required, may be {}). See Credential encryption below
  • endpoint - override base URL, e.g. for custom/Azure-style deployments (optional)
  • enabled - whether the connection is usable by routing (required)

Credential encryption (oauth/web providers):

For providers with supportLevel: "oauth" or "web", plaintext credential fields are encrypted server-side before being written to disk, and the plaintext keys are stripped from the stored config. The API never returns credentials on any response, on any provider.

supportLevelPlaintext input fieldStored (encrypted) fieldRequired
oauthoauthPlainoauthEncyes
oauthrefreshPlainrefreshEncno
webcookiePlaincookieEncyes
webcfClearancePlaincfClearanceEncno (openai-web only)

Any other field on credentials (e.g. expiresAt for oauth providers) passes through untouched. For native/compatible providers (e.g. plain apiKey), credentials passes through entirely untouched - plaintext-at-rest is intentional for those providers.

Response 200: the created connection, credentials omitted (see List Connections above).

Errors: 400 invalid body / unknown providerId / label_taken ({"error":"label_taken","message":"Label \"<name>\" is already used by another connection"}) · 403 insufficient permissions

Update Connection​

PATCH /api/connections/:id

Auth: Authorization: Bearer <jwt> (requires connections:manage)

Request body (all fields optional, same shape as create):

{ "label": "Renamed", "credentials": { "oauthPlain": "new-access-token" } }

Sending "label": "" regenerates the name from the provider; sending a name another connection already uses returns 400 label_taken.

When credentials is present in the body, it replaces the stored credentials object wholesale (not a deep merge) - resend every field you want to keep, following the same oauthPlain/refreshPlain/cookiePlain/cfClearancePlain convention as create. Only the fields present in the submitted credentials object are encrypted; omitted plaintext fields simply don't produce an encrypted counterpart.

Response 200: the updated connection, credentials omitted.

Errors: 400 invalid body · 404 connection not found · 403 insufficient permissions

Delete Connection​

DELETE /api/connections/:id

Auth: Authorization: Bearer <jwt> (requires connections:manage)

Response: 204 No Content

Errors: 404 connection not found · 409 { error, usedBy } the connection is still used by one or more models (usedBy = model ids; nothing is deleted) · 403 insufficient permissions


Model Instances​

A model instance exposes one upstream model on top of an existing connection, with its own pricing, context window, limits, and capability overrides. This is what shows up as a routable model.

List Instances​

GET /api/instances

Auth: Authorization: Bearer <jwt> (requires connections:read)

Response 200: array of instances.

[
{
"id": "inst-uuid",
"connectionId": "conn-uuid",
"upstreamModelId": "gpt-5-mini",
"cost": { "inputPerMillion": 0.25, "outputPerMillion": 2.0 },
"contextWindow": 128000,
"limits": [],
"capabilities": { "functionCalling": true, "json": true }
}
]

Errors: 403 insufficient permissions

Create Instance​

POST /api/instances

Auth: Authorization: Bearer <jwt> (requires connections:manage)

{
"connectionId": "conn-uuid",
"upstreamModelId": "gpt-5-mini",
"cost": { "inputPerMillion": 0.25, "outputPerMillion": 2.0 },
"contextWindow": 128000,
"limits": [],
"capabilities": { "functionCalling": true, "json": true }
}

Fields:

  • connectionId - id of an existing connection (required)
  • upstreamModelId - the provider's model id, e.g. gpt-5-mini (required)
  • cost - { inputPerMillion, outputPerMillion, cachePerMillion?, cacheWritePerMillion?, pricingTiers? } (required). When cachePerMillion is omitted, cache-read tokens are billed at inputPerMillion (and cacheWritePerMillion falls back the same way); the service logs a one-time warning per model on the first cache hit. Set cachePerMillion to price cache reads correctly.
  • contextWindow - token limit (required)
  • limits - array of usage limit objects, same shape as router token limits (optional)
  • capabilities - { thinking?, vision?, functionCalling?, json?, embedding? }, overrides the connection provider's nativeCapabilities (optional)

Response 200: the created instance.

Errors: 400 invalid body · 403 insufficient permissions

Update Instance​

PATCH /api/instances/:id

Auth: Authorization: Bearer <jwt> (requires connections:manage)

Request body (all fields optional, same shape as create).

Response 200: the updated instance.

Errors: 400 invalid body · 404 instance not found · 403 insufficient permissions

Delete Instance​

DELETE /api/instances/:id

Auth: Authorization: Bearer <jwt> (requires connections:manage)

Response: 204 No Content

Errors: 404 instance not found · 403 insufficient permissions


Profiles​

A profile bundles one area of a router's configuration into a reusable, named unit. Its kind decides which fields it carries:

kindFields
routingpolicies, selector, fallbackStrategy
optimizeroptimizers
securityguardrails, pii

Every profile also has id, kind, label, version, builtin and, when it was produced by Clone Profile, baseId.

Routerly ships read-only built-ins for two kinds (auto, cheap, fast, coding; optimizer-safe, optimizer-balanced, optimizer-aggressive). Security ships none: guardrails and PII policies rewrite the request, so a router never inherits them from a preset it did not choose, and every security profile is user-created. A router either keeps its own inline configuration for a kind or is assigned a profile of that kind via Assign Router Profiles; the three kinds are assigned independently.

See Concepts: Routing: Routing Profiles for the selector and fallback strategy reference. Both fields are optional on write and default to argmax and next-best; the API is the only surface that sets them, since the dashboard and the CLI do not expose either.

List Profiles​

GET /api/profiles
GET /api/profiles?kind=routing

Auth: Authorization: Bearer <jwt> (requires profiles:read)

Returns the built-ins followed by any user-created profiles. kind filters the result to one kind; omit it to get all three.

Response 200:

[
{
"id": "auto",
"kind": "routing",
"version": 1,
"label": "Auto",
"policies": [
{ "type": "health", "enabled": true },
{ "type": "performance", "enabled": true },
{ "type": "cheapest", "enabled": true },
{ "type": "capability", "enabled": true }
],
"selector": "argmax",
"fallbackStrategy": "next-best",
"builtin": true
},
{
"id": "optimizer-safe",
"kind": "optimizer",
"version": 1,
"label": "Safe",
"optimizers": { "steps": [{ "id": "session-dedup", "enabled": true }] },
"builtin": true
},
{
"id": "b1e7c0f2-9a4d-4c31-8f0a-2d6b5e7c1a90",
"kind": "security",
"version": 1,
"label": "My Security Profile",
"guardrails": { "detectInjection": true, "rules": [] },
"pii": { "policies": [{ "target": "request", "entities": ["EMAIL", "PHONE", "CREDIT_CARD", "SSN", "IBAN"] }] },
"builtin": false
}
]

Errors: 400 invalid_kind · 403 insufficient permissions

Create Profile​

POST /api/profiles

Auth: Authorization: Bearer <jwt> (requires profiles:manage)

{
"kind": "routing",
"label": "My Routing Profile",
"policies": [{ "type": "cheapest", "enabled": true }],
"selector": "cheapest",
"fallbackStrategy": "abort"
}

Fields:

  • kind: routing | optimizer | security (required, decides which other fields are accepted)
  • label: display name (required, non-empty)
  • everything else is optional and defaults to an empty configuration, so { "kind": "security", "label": "Empty" } is a valid body that can be filled in later with Update Profile

Response 201: the created profile (builtin: false, version: 1).

Errors: 400 invalid body · 403 insufficient permissions

Clone Profile​

POST /api/profiles/clone

Auth: Authorization: Bearer <jwt> (requires profiles:manage)

{ "baseId": "auto", "label": "My Auto Profile" }

Fields:

  • baseId: id of the built-in profile to clone (user-owned profiles cannot be cloned) (required)
  • label: display name for the new profile (required, non-empty)

Response 200: the created profile, same kind and configuration as the base (builtin: false, version: 1, baseId set to the source profile's id).

Errors: 400 invalid body or unknown baseId · 403 insufficient permissions

Update Profile​

PATCH /api/profiles/:id

Auth: Authorization: Bearer <jwt> (requires profiles:manage)

Request body: any subset of the fields that belong to the target profile's kind, at least one required. kind itself cannot be changed.

{
"label": "Renamed Profile",
"policies": [{ "type": "cheapest", "enabled": true }],
"selector": "cheapest",
"fallbackStrategy": "abort"
}

Only user-created profiles can be updated. Every successful update bumps the profile's version field by 1.

Response 200: the updated profile.

Errors: 400 invalid body (including a field that does not belong to this profile's kind) · 404 profile not found · 409 immutable_builtin_profile (target id matches a built-in) · 403 insufficient permissions

Delete Profile​

DELETE /api/profiles/:id

Auth: Authorization: Bearer <jwt> (requires profiles:manage)

Response: 204 No Content

Errors: 404 profile not found · 409 immutable_builtin_profile (target id matches a built-in) · 409 profile_in_use (a router still references it, unassign it first) · 403 insufficient permissions

Assign Router Profiles​

PUT /api/routers/:id/profiles

Auth: Authorization: Bearer <jwt> (requires router:write)

{ "routing": "auto", "optimizer": null }

Fields (at least one required, each string | null):

  • routing, optimizer, security: id of an existing profile of that kind, or null to clear the assignment and fall back to the router's own inline configuration for that kind

Kinds omitted from the body keep their current assignment.

Response 200: the updated router (same shape as List Routers, tokens present with token values stripped).

Errors: 400 invalid body · 404 router not found or profile_not_found (with the offending kind) · 403 insufficient permissions


Optimizers​

Prompt/context optimizers reduce a request's token footprint before it is forwarded to a provider. All 8 ship disabled by default; a router opts in per-optimizer via its optimizers field. See Concepts: Optimizers for what each optimizer does, its class (lossless / recoverable / lossy), the safety gate, and which steps are language-bound.

List Optimizers​

GET /api/optimizers

Auth: Authorization: Bearer <jwt> (requires optimizers:read)

Read-only catalog of the optimizers installed in the running service, resolved from the in-memory registry populated at module bootstrap.

Response 200:

[
{ "id": "session-dedup", "klass": "lossless", "installed": true },
{ "id": "ccr", "klass": "recoverable", "installed": true },
{ "id": "rtk", "klass": "recoverable", "installed": true },
{ "id": "headroom", "klass": "lossless", "installed": true },
{ "id": "json-table", "klass": "recoverable", "installed": true },
{ "id": "relevance", "klass": "lossy", "installed": true },
{ "id": "caveman", "klass": "lossy", "installed": true },
{ "id": "llmlingua-2", "klass": "lossy", "installed": true }
]

Returns [] if the optimizer modules were not bootstrapped.

Errors: 403 insufficient permissions

Preview Optimizers​

POST /api/optimizers/preview

Auth: Authorization: Bearer <jwt> (requires optimizers:read)

Pure dry-run: applies the given steps to sample messages and reports the token delta per step. No upstream call is made and no router config is written.

{
"routerId": "proj-uuid",
"model": "openai/gpt-4o-mini",
"sampleMessages": [{ "role": "user", "content": "Hello, please help me with this." }],
"steps": [
{ "id": "caveman", "enabled": true },
{ "id": "rtk", "enabled": true }
]
}

Fields:

  • sampleMessages: message array to run the pipeline over, same shape as an LLM Proxy request's messages (required, min 1)
  • steps: the optimizer steps to dry-run, same shape as the optimizers router field. Optional when routerId is given: omitted, the preview runs the pipeline the router really uses, which is its assigned optimizer profile or, with none, its saved custom steps. Sent, exactly these steps run (use it to preview unsaved edits)
  • routerId: optional. When given, the preview runs "as" that router (its other config is read; nothing is written); 404 if unknown. Without steps, routerId is required (400 otherwise)
  • model: optional model id the sample is addressed to. headroom sizes its budget on the requested model's context window, so without one it reports a skip reason instead of trimming, exactly as it would on a request naming a model Routerly has no window for

Response 200 (pipeline.kind is profile, with profileId and label, when the router's assigned profile ran; custom otherwise):

{
"estimatedTokensBefore": 55,
"estimatedTokensAfter": 30,
"pipeline": { "kind": "custom", "label": "Custom" },
"messages": [{ "role": "user", "content": "Hello, help me this." }],
"perStep": [
{
"id": "caveman",
"before": 55,
"after": 30,
"messages": [{ "role": "user", "content": "Hello, help me this." }]
},
{
"id": "rtk",
"before": 30,
"after": 30,
"messages": [{ "role": "user", "content": "Hello, help me this." }],
"skipReason": "No redundant whitespace or repeated block found in the message text."
}
]
}
FieldDescription
estimatedTokensBefore / estimatedTokensAfterToken estimate of the whole prompt before the first step and after the last one
messagesThe prompt as the pipeline left it, ready to diff against sampleMessages
perStep[].before / perStep[].afterToken estimate around that single step
perStep[].messagesThe prompt as that step left it. Always present, so step n's diff anchors on step n-1's output; unchanged when the step did nothing
perStep[].rolledBacktrue when the step produced a result the safety gate rejected, so its change was discarded. Absent otherwise
perStep[].skipReasonWhy an enabled step declined to run, in the optimizer's own words. Absent when the step ran

A lossy step whose result fails the safety gate reports rolledBack: true and is otherwise unchanged (before === after), mirroring what happens on a live request. rolledBack is what separates "was rejected" from "had nothing to do", which the token delta alone cannot say. A step that declined to run at all reports skipReason instead: no context window for the model, no repeated message, text that is not English, no JSON array long enough, the llmlingua-2 checkpoint not downloaded.

Errors: 400 invalid body · 404 routerId given but not found · 403 insufficient permissions

LLMLingua-2 Checkpoints​

GET /api/optimizers/llmlingua2/model

Auth: Authorization: Bearer <jwt> (requires optimizers:read)

State of the optional llmlingua-2 install on this service host. Not router-scoped: the runtime and the downloaded checkpoints are shared by every router that names one. Carries no prompt content.

Response 200:

{
"runtimeInstalled": true,
"checkpoints": [
{
"key": "bert-multilingual-q8",
"label": "BERT multilingual, quantized",
"repo": "ldenoue/llmlingua-2-bert-base-multilingual-cased-meetingbank",
"dtype": "q8",
"sizeMb": 182,
"license": "The export repo declares no license; the upstream weights are Apache-2.0.",
"note": "The default. Smallest and fastest, and enough for prose in the 104 languages BERT multilingual covers.",
"isDefault": true,
"state": "downloading",
"progress": 42,
"loadedBytes": 76000000,
"totalBytes": 182000000
},
{
"key": "xlm-roberta-large-int8",
"label": "XLM-RoBERTa large, int8",
"repo": "atjsh/llmlingua-2-js-xlm-roberta-large-meetingbank",
"dtype": "int8",
"sizeMb": 579,
"license": "MIT.",
"note": "Better compression quality and a declared license, at roughly three times the disk and noticeably slower inference.",
"isDefault": false,
"state": "absent"
}
]
}
FieldDescription
runtimeInstalledWhether the optional @huggingface/transformers dependency is present. false means no checkpoint can be downloaded or used
checkpoints[].keyThe value a step's model field takes (optimizers router field)
checkpoints[].isDefaultThe checkpoint a step with no model runs on. Reported rather than derived, because ROUTERLY_LLMLINGUA_MODEL can move it
checkpoints[].stateabsent, downloading or ready
checkpoints[].progress0–100 while downloading, absent otherwise. loadedBytes / totalBytes accompany it when the host reports sizes
checkpoints[].errorWhy the last download attempt failed. The checkpoint goes back to absent and can be retried

Errors: 403 insufficient permissions

POST /api/optimizers/llmlingua2/model

Auth: Authorization: Bearer <jwt> (requires optimizers:manage)

Starts one checkpoint's download on the service host and returns immediately: it is hundreds of megabytes, so the caller polls the GET above rather than holding a request open.

{ "key": "xlm-roberta-large-int8" }

key is optional; omitted, the default checkpoint is downloaded. It must be one this host publishes, and free text is refused rather than fetched, since fetching is what costs the disk.

Response 202: the same body as the GET, with the requested checkpoint already in downloading.

Errors: 400 unknown checkpoint key · 409 @huggingface/transformers is not installed on the service host · 403 insufficient permissions


Experiments​

A/B tests that route each call to one of several routers. A variant is an existing router taken whole, so the whole of what a router expresses (its models, routing profile, optimizer pipeline, guardrails) becomes comparable. See Concepts: Experiments for how a test lives, the rotations and how the numbers are computed.

Tokens are returned with token stripped on every read: the raw value exists only in the 201 of Create Experiment and the response of Create Experiment Token, the same rule router tokens follow.

List Experiments​

GET /api/experiments

Auth: Authorization: Bearer <jwt> (requires experiments:read)

Response 200:

[
{
"id": "8f2c1d64-2f1e-4c0a-9a1b-6b5c2d0e7f31",
"name": "Cheap vs premium",
"description": "Is the cheap model good enough for support replies?",
"rotation": "sticky",
"stickyKey": "auto",
"variants": [
{ "id": "4d3b2a10-8c7e-4f21-9b0d-1e2f3a4b5c6d", "routerId": "proj-cheap", "name": "Cheap" },
{ "id": "6a1c9e07-5b3d-42f8-8e10-7c4d9f2b0a35", "routerId": "proj-premium", "name": "Premium" }
],
"tokens": [
{
"id": "b7e5c3a1-9f2d-4e60-a8b3-0c1d2e3f4a5b",
"tokenSnippet": "sk-rt-2246",
"createdAt": "2026-08-01T10:12:03.000Z",
"lastUsedAt": "2026-08-01T10:41:55.000Z"
}
],
"judge": { "enabled": true, "modelId": "gpt-4o", "criteria": ["Answers the question asked"], "sampleRate": 0.2 },
"createdAt": "2026-08-01T10:12:03.000Z"
}
]
FieldDescription
rotationsticky (default), weighted or round-robin
stickyKeyauto (default), end-user, conversation or client. Read only by sticky
variants[].routerIdThe router this arm routes to
variants[].nameDisplay label. Absent means the router's own name is shown
variants[].weightShare of traffic under weighted. Normalised against the other weights, so 1/1 and 50/50 are the same split. A missing weight counts as 1
judgeQuality scoring. Absent when never configured, enabled: false when turned off
judge.sampleRateFraction of calls scored, 0-1 (the dashboard field and the CLI flag take a percentage)
judgeScoresVerdict tally per variant id: { count, totalScore, lastAt }. Absent until the first verdict
minSamplesPerVariantOverrides the default of 30

Errors: 403 insufficient permissions

Get Experiment​

GET /api/experiments/:id

Auth: Authorization: Bearer <jwt> (requires experiments:read)

Response 200: one experiment, same shape as the list entries.

Errors: 404 not found · 403 insufficient permissions

Experiment Metrics​

GET /api/experiments/:id/metrics?from=<iso>&to=<iso>

Auth: Authorization: Bearer <jwt> (requires experiments:read)

The comparison, one entry per variant. Read straight off the usage log, which the experiment stamps with its id on every call it routes, so the numbers agree with Usage by construction and the experiment keeps no counters of its own.

Query: from and to, both optional ISO 8601 values. Neither given measures the whole history. An explicit window, not the period vocabulary /api/usage uses: the caller already knows the window it wants. A date-only bound (2026-08-01) covers the whole day, exactly as /api/usage reads it; a full instant is taken as given.

Response 200:

{
"experimentId": "8f2c1d64-2f1e-4c0a-9a1b-6b5c2d0e7f31",
"minSamplesPerVariant": 30,
"totalCalls": 214,
"ready": false,
"variants": [
{
"variantId": "4d3b2a10-8c7e-4f21-9b0d-1e2f3a4b5c6d",
"routerId": "proj-cheap",
"name": "Cheap",
"calls": 118,
"errors": 2,
"errorRate": 0.017,
"cost": 0.4212,
"avgCostPerCall": 0.00357,
"inputTokens": 91240,
"outputTokens": 30118,
"avgLatencyMs": 812,
"p95LatencyMs": 1904,
"avgTtftMs": 240,
"judgedCalls": 24,
"avgScore": 7.4,
"enoughSamples": true
}
]
}
FieldDescription
readyEvery variant reached minSamplesPerVariant. Until then the comparison is premature
enoughSamplesThe same test for one variant
errors / errorRateCalls whose outcome was neither success nor blocked
p95LatencyMsNearest-rank, the same method the usage route uses
avgTtftMsOver the streamed calls only. Absent when none streamed
judgedCalls / avgScoreJudge verdicts, 0-10. avgScore is absent until the first one. Both come from the running tally on the experiment, so they cover its whole life and ignore from/to

Only the client's own completion calls are counted. Router decision calls, guardrail passes and the judge's own verdicts are gateway overhead: counting them would make the cheap variant look expensive for a reason the operator cannot act on.

Errors: 404 not found · 403 insufficient permissions

Create Experiment​

POST /api/experiments

Auth: Authorization: Bearer <jwt> (requires experiments:manage)

{
"name": "Cheap vs premium",
"description": "Is the cheap model good enough for support replies?",
"rotation": "weighted",
"variants": [
{ "routerId": "proj-cheap", "name": "Cheap", "weight": 80 },
{ "routerId": "proj-premium", "name": "Premium", "weight": 20 }
],
"judge": { "enabled": true, "modelId": "gpt-4o", "criteria": ["Answers the question asked"], "sampleRate": 0.2 },
"minSamplesPerVariant": 50
}

Fields: name (required) · description · rotation (default sticky) · stickyKey · variants (default [], each needs routerId; id is generated when omitted) · judge · minSamplesPerVariant (integer >= 1).

The experiment is created with its first token and routes traffic straight away. Variants can be left empty here and added by PATCH later.

Response 201: the experiment, plus a top-level token holding the raw value. It is returned here and nowhere else.

Errors: 400 invalid body · 404 router_not_found (with the offending routerIds) · 409 name already taken · 403 insufficient permissions · 403 insufficient permissions

Update Experiment​

PATCH /api/experiments/:id

Auth: Authorization: Bearer <jwt> (requires experiments:manage)

Same fields as create, all optional, at least one required. variants and judge.criteria replace the whole list rather than merging into it.

Every field stays editable for the whole life of the experiment, including one already serving traffic. Redesigning a test that already has traffic mixes two different measurements under one set of numbers, so narrow the metrics window to the period after the change.

Response 200: the updated experiment.

Errors: 400 invalid body · 404 not found · 404 router_not_found · 403 insufficient permissions

Create Experiment Token​

POST /api/experiments/:id/tokens

Auth: Authorization: Bearer <jwt> (requires experiments:manage)

An experiment can hold several tokens, which is how one test is handed to several clients and one of them revoked later. No body.

Response 200:

{
"token": "sk-rt-91b0d4e7c2a58f36b1d09e4c7a2f5b8d3e6c1a0f9b4d7e2c5a8f1b6d3e0c9a4",
"tokenInfo": {
"id": "c8f6d4b2-0a3e-4f71-b9c4-1d2e3f4a5b6c",
"tokenSnippet": "sk-rt-91b0",
"createdAt": "2026-08-01T11:02:17.000Z"
}
}

A client uses it exactly like a router token: same base URL, this value in place of the router's. Each request lands on one variant and is billed to that variant's router. Nothing about the test reaches the wire.

Errors: 404 not found · 403 insufficient permissions

Revoke Experiment Token​

DELETE /api/experiments/:id/tokens/:tokenId

Auth: Authorization: Bearer <jwt> (requires experiments:manage)

Response 204: no content. Any client still using the token starts failing immediately.

Errors: 404 not found · 404 token not found · 403 insufficient permissions · 403 insufficient permissions

Delete Experiment​

DELETE /api/experiments/:id

Auth: Authorization: Bearer <jwt> (requires experiments:manage)

Response 204: no content.

The experiment's tokens go with it, so every client still calling one starts getting 401. Move those clients to a router token first.

Errors: 404 not found · 403 insufficient permissions


Routers​

List Routers​

GET /api/routers

The list is not filtered by router membership: every caller holding router:read sees every router (name, kind, token snippets, never full tokens). Membership (/api/routers/:id/members) records who administers a router and scopes MCP access; it does not hide routers. To keep a router out of someone's view, do not grant them router:read.

There is no single-router route: read a router from this list. Routers are addressed by id, never by name or slug.

Create Router​

POST /api/routers
{
"name": "My App",
"timeoutMs": 2000,
"models": [{ "modelId": "gpt-5-mini", "prompt": "Short factual answers" }],
"autoRouting": true,
"routingModelId": "ollama/qwen3.5:9b"
}

Only name is required. models holds target model references, not bare id strings: each entry is { modelId, prompt? }. Response 201: the created router plus a token field with the first router token in clear text, which is the only time it is readable.

Model ids that are not in the model registry are accepted (a model may be unmapped or managed elsewhere) but listed in a warnings string array on the response, e.g. Model "nope/x" is not in the model registry. The field is absent when every model is registered. PUT /api/routers/:id behaves the same.

Errors: 400 empty name · 400 invalid timeoutMs · 409 a router with that name already exists · 403 insufficient permissions

For a router created with kind: "passthrough" (see Concepts: Architecture), no request or response field changes: tokens[] is still returned as-is. But a token is now issued for it under the same condition as any other kind — a real target model present in the submitted models, alongside the fixed pass-through entry — instead of tokens[] being left empty unconditionally. A passthrough router created with only the pass-through entry, the previous default, still gets no token.

Update Router​

PUT /api/routers/:id

On PUT, the guardrails and pii fields are optional: omit a field to leave it unchanged, send null to clear it, or send an object to replace it.

FieldTypeDescription
namestringRouter name
timeoutMsnumberUpstream timeout for this router's requests
traceContentbooleanTrace content opt-in. Default false: traces record metadata only. Set to true to also store prompts and answers on the trace (entry.content) — they then reach the trace stream, the usage record and any integration exporting traces.
candidatesarrayOnly on kind: "orchestrator". Request body: { routerId, limits? }[]. Response also resolves each candidate's name: { routerId, name, limits? }[]. Array order is the priority order (index 0 = highest) — there is no weight field.

For a router with kind: "passthrough", no request or response field changes here either — toRouterResponse still returns tokens[] as-is. PUT now issues a token (tokens[] gains an entry) the moment the submitted models gains its first real target model, and revokes it (tokens: []) the moment the last real target model is removed, leaving only the fixed pass-through entry. See Concepts: Architecture for the ordering rule that decides which of a passthrough router's real models get scored versus the pass-through entry's own raw-forward, and for the fallback's forwarded-credential caveat.

Content Guardrails and PII (router fields)​

A router may carry two optional security blocks, accepted by both POST /api/routers and PUT /api/routers/:id and validated server-side. Use PATCH /api/routers/:id/guardrails for partial updates (guardrails or PII only).

Guardrails​

Guardrails evaluate each request and/or response against an ordered list of independent rules; each enabled rule is evaluated in sequence and triggers its configured actions independently.

{
"guardrails": {
"detectInjection": true,
"rules": [
{
"type": "regex",
"enabled": true,
"target": "request",
"block": true,
"log": false,
"config": { "patterns": ["competitor", "rival\\s+product"] }
},
{
"type": "semantic",
"enabled": true,
"target": "both",
"block": true,
"log": true,
"config": {
"embeddingModelId": "text-embedding-3-small",
"fallbackModelIds": ["text-embedding-3-large"],
"examples": ["example of blocked content"],
"threshold": 0.82
}
},
{
"type": "topic",
"enabled": true,
"target": "request",
"inject": true,
"config": {
"modelId": "claude-haiku-4-5",
"fallbackModelIds": ["gpt-4-mini"],
"allowedTopics": "Customer support questions about our product only",
"threshold": 0.5
}
},
{
"type": "topic",
"enabled": true,
"target": "response",
"config": {
"modelId": "claude-haiku-4-5",
"fallbackModelIds": ["gpt-4-mini"],
"allowedTopics": "Customer support questions about our product only",
"threshold": 0.5
}
},
{
"type": "moderation",
"enabled": true,
"target": "request",
"inject": false,
"config": {
"modelId": "claude-haiku-4-5",
"fallbackModelIds": ["gpt-4-mini"],
"threshold": 0.5,
"systemPrompt": "You are a content safety classifier for customer support conversations."
}
}
]
}
}

Rule types:

TypeTargetConfig fieldsDescription
regexrequest / response / bothpatterns: string[]Block text matching any regex pattern (case-insensitive)
semanticrequest / response / bothembeddingModelId, fallbackModelIds?: string[], examples: string[], threshold?: number (default 0.82)Block semantically similar content using embedding cosine similarity; fallback models tried in order if primary fails
topicrequest / response / both (or omitted for inject-only)modelId (optional for inject-only), fallbackModelIds?: string[], allowedTopics: string, threshold?: number (default 0.5)LLM judge: block content not matching the allowed topics description; fallback judges tried in order if primary fails. When target is omitted and inject: true, injects only (no judge)
moderationrequest / response / both (or omitted for inject-only)modelId (optional for inject-only), fallbackModelIds?: string[], threshold?: number (default 0.5), systemPrompt?: stringLLM judge: block harmful content (hate, violence, sexual, self-harm); fallback judges tried in order if primary fails. When target is omitted and inject: true, injects only (no judge)

Rule fields:

  • enabled?: boolean: default true when absent; when false, the rule is skipped
  • target?: 'request' | 'response' | 'both': which side(s) the judge evaluates. Required for regex/semantic. Optional for topic/moderation (omit to enable inject-only). When set, a judge model is required (config.modelId).
  • inject?: boolean: (topic/moderation only) append this rule's instruction to the request system prompt. Independent of target. When true on a topic/moderation rule, the rule steers the model without calling the judge (soft enforcement).
  • block?: boolean: auto-managed when target is set (always true for judged rules); accepted on input for backward compatibility but ignored.
  • log?: boolean: auto-managed when target is set (always true for judged rules); accepted on input for backward compatibility but ignored.
  • blockMessage?: string: the block message now comes from the judge's reason field. This field is still accepted on input for backward compatibility but is no longer surfaced by the dashboard or CLI.
  • useJudgeResponse?: boolean: auto-managed (always true for judged rules); accepted on input for backward compatibility but ignored.

Fallback models (topic/moderation/semantic only):

  • fallbackModelIds?: string[]: ordered list of model IDs to try if the primary model is unavailable or returns an error (other than budget/usage exceeded). Fallbacks are tried in order. If a model returns a usage or budget-exceeded error, the rule fails immediately without trying further fallbacks (fail-closed on budget).

Judge scoring (topic/moderation with judge):

  • The judge responds with {"reason": "<explanation>", "score": <0.00-10.00>}. The reason is in the user's language and becomes the block message. The score (0.00-10.00) is normalized to 0-1 before threshold comparison. Topic rules trigger when normalized score < threshold (off-topic); moderation rules trigger when normalized score > threshold (harmful).

detectInjection flag:

  • When true, run a built-in prompt-injection detector on every request before rule evaluation. A hit blocks and is logged. Injection detection does not support custom messages.

Target values: request evaluates the user messages; response evaluates the model output; both evaluates both sides; omitted (for topic/moderation) skips the judge entirely when inject: true.

Streaming interaction: When any enabled rule has target as response or both (judge on response), the entire response must be buffered before the block decision is made. In this case, streaming is disabled for the request, and the client receives the full response as a single chunk.

PII​

{
"pii": {
"policies": [
{
"enabled": true,
"target": "both",
"entities": ["EMAIL", "PHONE", "CREDIT_CARD", "SSN", "IBAN"],
"outputBufferSize": 30
},
{
"enabled": true,
"target": "request",
"entities": ["EMAIL", "PHONE"],
"customPatterns": ["\\b[A-Z]{2}[0-9]{6,8}\\b"]
}
]
}
}

PII configuration contains a list of policies. All enabled policies are merged per-direction at scrub time (request scrubbing merges policies with target: request or target: both; response scrubbing merges policies with target: response or target: both).

Policy fields:

  • enabled?: boolean: default true when absent; when false, this policy is skipped
  • target: 'request' | 'response' | 'both': which side(s) to scrub
  • entities?: PiiEntity[]: entity types to detect (EMAIL, PHONE, CREDIT_CARD, SSN, IBAN). Defaults to all types when omitted.
  • customPatterns?: string[]: additional regex patterns to scrub (case-insensitive)
  • outputBufferSize?: number: (response scrubbing only) suffix buffer size in characters (10-500, default 30). Used to catch patterns spanning chunk boundaries when streaming. When multiple response policies are active, the largest value wins.

Matched values in message content are replaced with typed placeholders ([EMAIL], [PHONE_NUMBER], [CREDIT_CARD], [SSN], [IBAN]) before the request is forwarded to the provider and before the response is returned to the caller.

When a guardrail triggers or PII is redacted, the usage record gains guardrailTriggered (the rule type) and/or piiRedacted (the list of redacted entity types).

Guardrail judge calls and usage attribution​

Security rules that call a model (semantic embedding, topic judge, moderation judge) are tracked as separate usage records with callType: "guardrail". These records are attributed to the same router and token as the originating request and are subject to the same budget limits - an over-budget judge call fails the same as an over-budget completion. The records appear in GET /api/usage alongside completion and routing records and are broken out in the usage summary (see Query Usage Records).

Consumer Impact​

PII scrubbing modifies request content in-flight. The model receives placeholders instead of original sensitive values and responds based on the modified message. Disable specific entity types if your application requires the model to see the original values.

A rule with block: true returns a wire-faithful HTTP 200 response when it triggers. It does not return HTTP 400. See LLM Proxy: Guardrail block wire format.

A rule with log: true (and block unset) forwards the request to the model with no consumer-visible impact; the match is recorded on the usage record for audit purposes. block and log are independent, so a rule may do both: block the request and record the match.

Optimizers (router field)​

A router may carry an optional optimizers block, accepted by both POST /api/routers and PUT /api/routers/:id. On PUT, the field follows the same undefined/null/object convention as guardrails and pii: omit it to leave the pipeline unchanged, send null to clear it, or send an object to validate and replace it. Setting or clearing it requires optimizers:manage in addition to router:write. A caller with router:write but not optimizers:manage gets 403 on any request whose body includes a non-undefined optimizers field, even if every other field is otherwise valid.

{
"optimizers": {
"steps": [
{ "id": "session-dedup", "enabled": true },
{ "id": "ccr", "enabled": true },
{ "id": "caveman", "enabled": true },
{ "id": "relevance", "enabled": true, "threshold": 0.3 },
{ "id": "llmlingua-2", "enabled": true, "model": "xlm-roberta-large-int8" }
]
}
}

Fields:

  • steps: array of optimizer steps. Array order is execution order, steps run top to bottom in the request.preprocess pipeline phase (required, may be empty)
  • Each step: id (one of session-dedup, ccr, rtk, headroom, json-table, relevance, caveman, llmlingua-2; each id may appear at most once, and a duplicate id is rejected), enabled: boolean, threshold?: number (optional; a positive number, capped at 1 for relevance and llmlingua-2, unbounded for ccr, headroom and json-table; unused for session-dedup, rtk and caveman)
  • model?: string: llmlingua-2 only, the key of the checkpoint that step runs on, from LLMLingua-2 Checkpoints. Unset means the host's default checkpoint. Set on any other id it is rejected with 400

See Concepts: Optimizers for what each id does and its class. See Concepts: Optimizers, Threshold Range for what each id's threshold means (ccr: turn count, default 3; headroom: reserved token budget, default 1024; json-table: minimum rows, default 5; relevance and llmlingua-2: a 0–1 ratio, default 0.1 and 0.5; the rest unused). Leave threshold unset to use the built-in default.

Errors: 400 invalid optimizers config (bad shape, out-of-range threshold, duplicate step id, unknown checkpoint key, or model on a step that runs on none) · 403 insufficient permissions (optimizers:manage required to set/clear)

Delete Router​

DELETE /api/routers/:id

Router Tokens​

List Tokens​

GET /api/routers/:id/tokens

Create Token​

POST /api/routers/:id/tokens
{
"name": "production",
"labels": ["prod"],
"scopes": ["batch", "internal"],
"tags": {
"environment": "prod",
"team": "backend"
},
"limits": [
{
"metric": "cost",
"limit": 10.00,
"window": "monthly",
"mode": "extend"
}
]
}

Fields:

  • name - token name (required)
  • labels: array of free-text labels shown next to the token in the dashboard (optional)
  • scopes: free-form scopes stored with the token (optional). Routerly does not interpret them; they are yours for bookkeeping. The MCP server is authenticated by personal MCP tokens, not by these scopes
  • tags - arbitrary key-value metadata attached to the token (optional). Tags are included in every usage record created with this token.
  • limits - array of per-token spending limits (optional)

Response includes the token value in plain text - returned once only. The response also includes the labels, scopes, and tags fields.

Update Token​

PUT /api/routers/:id/tokens/:tokenId
{
"scopes": ["batch"],
"tags": {
"environment": "staging"
}
}

Fields:

  • labels: replace the token's labels (optional)
  • scopes: replace the token's access scopes (optional)
  • tags - replace the token's tags. Pass an empty object {} to clear all tags (optional).

Any field omitted from the request body is left unchanged (partial update).

Delete Token​

DELETE /api/routers/:id/tokens/:tokenId

Router Members​

List Members​

GET /api/routers/:id/members

Add Member​

POST /api/routers/:id/members
{ "userId": "user-uuid", "role": "viewer" }

Update Member Role​

PUT /api/routers/:id/members/:userId
{ "role": "editor" }

Remove Member​

DELETE /api/routers/:id/members/:userId

Users​

List Users​

GET /api/users

Create User​

POST /api/users
{ "email": "[email protected]", "password": "password", "role": "operator" }

Get User​

GET /api/users/:id

Update User​

PUT /api/users/:id

Delete User​

DELETE /api/users/:id

Roles​

List Roles​

GET /api/roles

Create Role​

POST /api/roles
{
"name": "billing_reviewer",
"permissions": ["router:read", "report:read"]
}

Update Role​

PUT /api/roles/:name

Delete Role​

DELETE /api/roles/:name

Spend Groups​

Org- and team-level budget containers for the hierarchical spend-limit cascade (organisation -> team -> API key). A router belongs to a group via its spendGroupId; groups may nest via parentGroupId. When a request runs through a router that belongs to a group, the group chain's limits are enforced in addition to the per-model/router/token limits. Child limits cannot exceed parent limits (validated on create/update).

A group object:

{
"id": "uuid",
"name": "Engineering",
"limits": [{ "metric": "cost", "windowType": "period", "period": "monthly", "value": 1000 }],
"routerIds": ["proj-1"],
"tokenIds": [],
"parentGroupId": "org-uuid"
}

List Spend Groups​

GET /api/spend-groups

Requires report:read. Returns each group with a usage array (current and remaining consumption per limit, aggregating its own and descendant groups' routers) for the consumption tree view and end-of-period forecast.

Create Spend Group​

POST /api/spend-groups

Requires router:write. Body validated with Zod (name required; limits, routerIds, tokenIds, parentGroupId optional). Returns 400 if a child limit exceeds the parent's matching limit or the parent does not exist.

Update Spend Group​

PUT /api/spend-groups/:id

Requires router:write. Same validation as create. A group cannot be its own parent.

Delete Spend Group​

DELETE /api/spend-groups/:id

Requires router:write. Returns 409 if the group still has child groups.


Usage​

Query Usage Records​

GET /api/usage

Query parameters:

ParameterTypeDescription
fromISO dateStart of range
toISO dateEnd of range
routerstringFilter by router slug
routerIdsstringComma-separated router IDs to filter by
modelstringFilter by model ID
modelIdsstringComma-separated model IDs to filter by
tokenIdsstringComma-separated router token IDs to filter by. Matches the token the call authenticated with, so a router's traffic can be narrowed to one client. Records written before 0.4.0 carry no tokenId and are excluded whenever this filter is set
callTypestringWho made the call: completion (the client), routing, or guardrail. completion also matches legacy records with no callType field
requestTypestringWhat was asked for, from the endpoint the client hit: chat, completion, embedding, rerank, image, audio. chat also matches records written before 0.4.0, which had no requestType field
outcomestringsuccess, error, budget_exceeded, timeout, blocked. error matches records that are neither success nor blocked
limitnumberMax records to return (default: 100)
offsetnumberPagination offset
savingsstring1 adds the savings block to the response. Off by default: the computation is only paid for by the callers that show it
seriesstring1 adds the savings series to the response. Same cost as savings, and equally off by default

callType and requestType are two different questions about the same record. A semantic-intent embedding fired by the router is callType: "routing", requestType: "embedding"; a plain chat request from a client is callType: "completion", requestType: "chat". Calls the gateway forwards through the pass-through proxy (embeddings, images, audio) are recorded with their requestType and zero tokens, since their body is streamed to the client rather than parsed.

All filters are applied server-side. routerIds, modelIds and tokenIds accept comma-separated values for multi-value filtering; they combine with (AND) the single-value router and model parameters when both are provided, narrowing the result to records that match every active filter.

Response summary object:

{
"summary": {
"totalCost": 0.1234,
"totalCalls": 200,
"successCalls": 188,
"blockedCalls": 3,
"errorCalls": 9,
"completionCalls": 180,
"routingCalls": 8,
"guardrailCalls": 12,
"completionCost": 0.1200,
"routingCost": 0.0011,
"guardrailCost": 0.0023,
"latencyMedianMs": 780,
"latencyP95Ms": 2450,
"ttftMedianMs": 210,
"ttftP95Ms": 900,
"ttftSamples": 176
},
"byModel": {
"openai/gpt-5-mini": {
"calls": 120,
"cost": 0.08,
"success": 115,
"avgLatencyMs": 820,
"p95LatencyMs": 1540
}
},
"byCallType": { "completion": 180, "routing": 8, "guardrail": 12 },
"byRequestType": { "chat": 176, "embedding": 24 },
"timeline": [],
"records": [],
"pagination": {}
}

The summary object breaks down calls and cost by sub-activity type:

FieldDescription
completionCalls / completionCostMain model inference calls
routingCalls / routingCostLLM policy routing calls (e.g. the llm routing policy)
guardrailCalls / guardrailCostModel calls made by security rules (semantic embedding, topic judge, moderation judge)
blockedCallsRequests blocked by a guardrail rule before reaching any model
errorCallsFailed calls -- does not include blocked calls
latencyMedianMs / latencyP95MsResponse time distribution over successful client calls. Routing and guardrail calls are excluded: they are gateway overhead, not client-visible time
ttftMedianMs / ttftP95MsTime-to-first-token distribution over the same calls
ttftSamplesCalls backing the TTFT figures. ttftMs is optional on the record, so this can be lower than successCalls

byCallType and byRequestType count the records of each kind in the window. Both are counted before the callType and requestType filters are applied, so they stay stable while a type filter is active: a client can build its filter controls from them and still show every value available. The other filters (period, router, model, outcome, tags) do narrow them. Records written before these fields existed are counted as completion and chat.

Each byModel entry includes:

FieldDescription
callsTotal requests for this model in the period
costTotal cost in USD
successNumber of successful calls
avgLatencyMsMean response time in milliseconds
p95LatencyMs95th-percentile response time in milliseconds

Every record in the records array carries tokenId, the router token the call authenticated with, taken from the bearer token that was presented and never from the payload. Records written before 0.4.0 have no such field. Internal calls that no client token stands behind, such as an experiment rotation firing on its own, are recorded without one.

Guardrail judge call records appear in the records array with callType: "guardrail". Blocked request records appear with outcome: "blocked" and callType: "guardrail". The errorCalls counter excludes blocked requests -- a block is a normal guardrail outcome, not a model error.

The outcome filter on GET /api/usage accepts blocked in addition to success, error, and budget_exceeded.

Savings block​

Add savings=1 to GET /api/usage to get a savings object alongside the rest of the response. It answers two questions over the same filtered records: what the routed traffic actually cost and took, and what it would have cost and taken had every client call gone to one fixed model instead.

{
"savings": {
"comparedCalls": 180,
"comparedCost": 0.12,
"comparedLatencyMs": 148000,
"comparedInputTokens": 420000,
"comparedOutputTokens": 96000,
"cache": { "inputTokens": 180000, "cost": 0.0162 },
"baselines": [
{
"modelId": "openai/gpt-5-mini",
"cost": 0.14,
"costDelta": 0.02,
"costDeltaPercent": 14.28,
"latencyMs": 132000,
"latencyDeltaMs": -16000,
"latencySamples": 120,
"tokensEstimated": 541800,
"tokenDelta": 25800
}
],
"optimizers": [
{ "id": "ccr", "calls": 84, "tokensSaved": 51200, "costSaved": 0.0128, "rolledBack": 0 },
{ "id": "caveman", "calls": 12, "tokensSaved": 940, "costSaved": 0.0002, "rolledBack": 3 }
]
}
}
FieldDescription
comparedCallsClient calls the counterfactual covers: successful, carrying tokens, priced against a known model
comparedCostWhat those calls actually cost, in USD
comparedLatencyMsWhat those calls actually took, in milliseconds, summed
comparedInputTokens / comparedOutputTokensToken totals of the compared calls
cache.inputTokensInput tokens served from prompt cache instead of being charged at full input price
cache.costUSD those cached tokens saved against the same model's full input price
baselinesOne counterfactual per baseline model, cheapest first
optimizersOne entry per optimizer that changed at least one compared call, most tokens saved first

Each optimizer entry:

FieldDescription
idThe optimizer id
callsCompared calls where it ran and changed the prompt
tokensSavedPrompt tokens it removed, summed over those calls
costSavedUSD those tokens would have cost at the serving model's input price
rolledBackCalls where its output was rejected by the safety gate and discarded

Unlike the baselines, these are measured, not counterfactual: the service records each optimizer's own before/after token estimate on the usage record as the request is served. The array is empty when no record in the window carries optimizer stats, which is the case for every record written before 0.4.0.

Each baseline entry:

FieldDescription
modelIdThe baseline model
costThe compared calls repriced at this model's rates, in USD
costDeltacost - comparedCost: money saved against this baseline. Negative means routing cost more
costDeltaPercentcostDelta as a percentage of the baseline cost
latencyMsEstimated total time, from this model's own median milliseconds per output token over the same window. Absent when the model produced no output token in the window
latencyDeltaMslatencyMs - comparedLatencyMs: time saved against this baseline. Negative means routing was slower. Absent together with latencyMs
latencySamplesCalls of this model in the window backing the time estimate. 0 means there is no estimate, only the cost figure
tokensEstimatedInput plus output tokens the same conversations are estimated to take on this model, from the ratio between its tokenizer family and the family of the model that served each call
tokenDeltatokensEstimated - (comparedInputTokens + comparedOutputTokens): tokens saved against this baseline. Positive means routing moved fewer

Baselines depend on the scope of the query. A query scoped with routerId counterfactuals exactly that router's enabled target models. An unscoped query counterfactuals the paid models in play in the window: every enabled target model of the routers that produced a record in it, plus every model that served a client call in it. Free models are excluded because they make the cost comparison meaningless, and embedding models are excluded because they cannot answer a completion call. A model that only served routing or guardrail calls is not in play either. This is the set the dashboard Overview compares against.

The token counterfactual is a ratio, not a re-tokenization

tokensEstimated is derived from a fixed ratio per tokenizer family (o200k, cl100k, Claude, Llama). Routerly does not retain prompts, so nothing is re-tokenized. Every surface that shows this figure declares it as an estimate.

Routing and guardrail calls are excluded from the comparison: they are the gateway's own overhead, not the client's workload, and the summary object already reports them. Pass-through records carry no tokens and are excluded for the same reason.

Repricing is an estimate, not a replay

The cost figure is exact arithmetic on the observed token counts, but a different model tokenizes the same text slightly differently and may answer at a different length. Read a baseline as "the same conversation, priced elsewhere". Measuring the real difference needs a live comparison on production traffic.

The same block is rendered by routerly report savings and by the router's Dashboard tab.

Savings series​

Add series=1 to GET /api/usage to get the same comparison spread over time. It covers the same records as the savings block, cut into buckets: one per hour when period=daily, one per day otherwise, the same granularity as timeline.

{
"series": {
"bucket": "day",
"baselineModelId": "openai/gpt-5",
"baselineModelIds": ["openai/gpt-5-mini", "openai/gpt-5"],
"points": [
{
"bucket": "2026-07-31",
"calls": 96,
"cost": 0.062,
"baselineCost": 0.184,
"baselineCosts": { "openai/gpt-5-mini": 0.071, "openai/gpt-5": 0.184 },
"inputTokens": 210000,
"outputTokens": 48000,
"cachedInputTokens": 90000,
"latencyMs": 74000,
"baselineLatencyMs": 96000
}
]
}
}
FieldDescription
buckethour or day: the width of each point
baselineModelIdModel the baselineCost and baselineLatencyMs figures are priced against: the costliest baseline, the worst case routing avoided. Absent when no baseline costs anything
baselineModelIdsEvery paid baseline baselineCosts is keyed by, cheapest first. Empty when no baseline costs anything
pointsOldest first, capped at the most recent 60 buckets

Each point:

FieldDescription
bucketYYYY-MM-DD for a day bucket, YYYY-MM-DDTHH (UTC) for an hour bucket
callsCompared client calls in the bucket, the same set the savings block counts
costUSD those calls actually cost
baselineCostThe same calls repriced at baselineModelId. 0 when there is no baseline
baselineCostsThe same calls repriced at every paid baseline, keyed by model id. Empty when no baseline costs anything; the costliest entry equals baselineCost
inputTokens / outputTokens / cachedInputTokensToken totals of the bucket
latencyMsSummed end-to-end latency of those calls
baselineLatencyMsEstimated summed latency on the baseline, from its throughput over the whole window. 0 when it has no sample to estimate from

Every field is a total rather than an average, so a consumer can re-bucket or average them itself. Buckets with no compared call are left out rather than zero filled: a gap means there was no traffic, which a zero would misreport as free traffic.

The series is rendered by the dashboard Overview and by routerly report savings --trend.

Get a Single Usage Record​

GET /api/usage/:id

Requires report:read. Returns the full usage record.

:id is matched against the record id first, then against its traceId. That second lookup is what makes a notification carrying only a trace id link straight to the request that produced it.

Errors: 404 record not found · 403 missing report:read

Individual usage records for blocked requests carry guardrailTriggered (the rule identifier, e.g. regex:pattern or injection:dan-mode) and blockedBy (same value; present only when the outcome is blocked). Records where PII was redacted carry piiRedacted with an array of redacted entity types. Records where a guardrail triggered on the flag or log path carry guardrailTriggered but not blockedBy.

Wire format unchanged

The block response sent to the API client is standard and unchanged: HTTP 200, empty content, finish_reason: "content_filter" (OpenAI) or stop_reason: "refusal" (Anthropic). Only observability around the block changed - the usage record is now written and the summary counts it separately.

Get Routing Trace​

GET /api/traces/:id

Returns the routing trace ({ trace: [...] }) of a request still in the in-memory buffer (5 minutes). All trace entries are stored out-of-band; the wire response sent to your API client is never modified. The durable copy is on the request's usage record.

Every entry carries where it came from, so a consumer can group a request by phase or by module without knowing who produced it:

FieldDescription
phasePipeline phase that was running: ingress, protocol.decode, request.preprocess, routing.prepare, routing.execute, upstream.prepare, upstream.execute, response.postprocess, protocol.encode, egress, finalize
moduleModule that reported it: router, policy, model, pii, guardrail, budget, resilience, … (read from the module:event message)
messageThe event, e.g. router:result, policy:result:cheapest, pii:scrubbed
panelWhich side of the call the entry belongs to: request/response for the client's own call, router-request/router-response for the calls Routerly made on its behalf. This is what keeps router overhead out of the request's own cost
detailsEvent-specific metadata (see below)
atEpoch milliseconds
contentPrompts and answers. Present only for routers with traceContent: true (see Update Router); otherwise the field never exists, in the buffer or on disk
EntryWhen emitteddetails shape
guardrail:evaluatedAfter every guardrail check on each target, whether or not any rule fires{ target: "request"|"response", rules: RuleEval[] }. One object per configured rule, inject-only and skipped rules included. RuleEval is { rule, outcome, index?, type?, target?, score?, threshold?, ms?, injects?, reason?, judgeMessage?, judgeRaw?, usage? }. outcome is passed, triggered, or skipped; score/threshold are reported whatever the outcome, so a rule that passed at 0.49 against 0.50 is visible; reason carries the hit string on a trigger and the cause on a skip (model-not-found, embedding-failed, judge-failed); usage is the judge call's tokens. The built-in prompt-injection check appears as rule: "injection".
guardrail:injectedA rule added steering text to the request{ target: "request", rules: string[], chars }. The injected text itself is on content.injection, and only for routers with traceContent: true.
guardrail:triggeredEmitted whenever a rule triggers (block or log) on the request side{ rule, target, block, log, blockMessage }
guardrail:response-triggeredEmitted whenever a rule triggers (block or log) on the response side{ rule, target, block, log, blockMessage }
pii:evaluatedAfter every PII scrubbing pass, on the request and on the response, whether or not anything was redacted{ target, mode?, policies: { configured, active }, entities: string[], customPatterns, scanned, redacted: string[], counts: Record<string, number>, ms }. entities is what was looked for, redacted what was found, counts how many of each. scanned is how many message contents were scanned: multimodal parts are skipped, so a clean pass still says what it looked at.
pii:scrubbedWhen at least one PII entity was detected and replaced{ entities: string[], counts }. Entity types that were replaced. Also emitted alongside pii:evaluated on a hit.
optimizer:stepOnce per configured optimizer step, whatever it did{ id, outcome, klass?, tokensBefore?, tokensAfter?, saved?, ms?, reason? }. outcome is applied, unchanged, skipped or rolled-back; reason says why (disabled, not-registered, unsupported-request, safety-gate, invalid-result, threw: …).
budget:checkedBefore every upstream attempt, per candidate model{ model, allowed, ms, violated?: [{ metric, window, limit, current }] }. A candidate refused here is dropped and the router moves to the next one.
egress:sentLast entry of the request: what was written back to the client{ protocol, kind, status?, encoding?, frames?, bytes?, error? }. kind is json, stream, block or passthrough.
trace:recapOnce, when the request finishesThe aggregate of everything above: { outcome, durationMs, model?, provider?, attempts, tokens, costUsd, latencyMs, ttftMs?, tokensPerSec?, overhead?, guardrails?, pii?, optimizers?, errors? }. outcome is ok, blocked, error or incomplete; overhead is what Routerly's own model calls cost, kept out of the request's numbers. Purely derived, so it never disagrees with the entries it summarises.
curl -s http://localhost:3000/api/traces/$TRACE_ID \
-H "Authorization: Bearer <jwt>"

Stream Traces​

GET /api/traces/stream

Server-sent events, one trace event per entry, as the entries happen. Requires report:read.

Query parameterDescription
correlationIdOnly requests that carried this x-routerly-trace value
routerIdOnly requests of this router
traceIdOnly this one request

Without filters the stream carries every request the service handles. Filters combine (AND).

curl -N "http://localhost:3000/api/traces/stream?correlationId=my-id" \
-H "Authorization: Bearer <jwt>"
: open

event: trace
data: {"traceId":"018f3c2a-...","entry":{"phase":"routing.prepare","module":"router","message":"router:result","panel":"router-response","details":{...},"at":1785700142000},"routerId":"...","correlationId":"my-id","topic":"trace/routing.prepare/router/result"}

: ping

This is how the Playground follows a request live: it picks a correlation id, sends it on the proxy request as x-routerly-trace, and reads it back here. The proxied request and its response are untouched — the id never reaches the provider and no header is added to the answer.

A comment line (: ping) is written every 15 seconds so idle streams survive proxies.


Settings​

Get Settings​

GET /api/settings

Response 200:

{
"port": 3000,
"host": "0.0.0.0",
"logLevel": "info",
"publicUrl": "https://routerly.example.com",
"requireMfa": false,
"listeningAddresses": ["http://127.0.0.1:3000", "http://192.168.1.116:3000"],
"localAddresses": ["192.168.1.116"],
"providerRepos": [
{
"url": "https://raw.githubusercontent.com/Inebrio/Routerly-Providers/main/",
"enabled": true,
"channel": null
}
]
}

listeningAddresses and localAddresses are derived at runtime from the bind host and are not stored in settings.json: a wildcard bind expands to every IPv4 interface, any other host resolves to that single address. They are read-only and ignored on PUT.

Update Settings​

PUT /api/settings

defaultLanguage (1-10 characters) sets the instance default dashboard language; other values return 400. channel sets the update channel the service checks against (PUT takes effect immediately, no restart — the running update checker is updated in place); an unrecognized value returns 400.

{
"port": 3000,
"logLevel": "info",
"publicUrl": "https://routerly.example.com",
"channel": "develop",
"providerRepos": [
{
"url": "https://raw.githubusercontent.com/Inebrio/Routerly-Providers/main/",
"enabled": true
},
{
"url": "https://your-org.com/catalog/",
"enabled": true
}
]
}

Fields:

  • logLevel must be one of fatal, error, warn, info, debug, trace; anything else answers 400.
  • port, logLevel, publicUrl, requireMfa - service configuration (optional). publicUrl must be an http:///https:// URL (trailing slash removed) or "" to clear it; usageRetention.maxAgeDays/maxSizeMb must be positive integers; otherwise 400. When requireMfa is true, every management route except /api/me, /api/me/language, POST /api/auth/2fa/setup and POST /api/auth/2fa/confirm answers 403 {"error":"mfa_required"} to a user who has not enabled 2FA
  • channel - update channel: latest, current, next, stable, develop, or a version tag such as v0.4.0 (optional)
  • providerRepos - array of provider repository objects (optional)

ProviderRepo object:

  • url - repository endpoint (required)
  • enabled - whether the repo is active (optional, default true)
  • channel - named channel to prefer (optional, e.g. stable, latest) — this is the provider catalog's own channel concept, unrelated to the update channel above

Errors: 400 { "error": "Invalid channel. Accepted values: latest, current, next, stable, develop, or a version tag such as v0.4.0." } when channel is not one of the five names above or a version tag.

Breaking change

develop is not an alias for next: it is its own channel, the bleeding-edge line. An instance that had develop persisted under the old meaning now tracks the bleeding-edge line instead of next. No migration runs on disk. A persisted "channel": "stable" stays valid and is never rewritten.

Get Available Releases​

GET /api/system/releases

Requires a valid dashboard session (any role). Used by the dashboard's channel selector.

Response 200:

{ "channels": ["latest", "current", "next", "develop"], "versions": [] }

channels always includes the four channel names above, plus any non-semver GitHub tag found among the repository's releases. versions is currently always empty.


Modules​

Every module is core and always active; there is no endpoint to enable or disable one. A stored configuration that marks a module disabled is cleared at service start.

List Modules​

GET /api/modules

Auth: Authorization: Bearer <jwt> (requires modules:read)

Response 200:

[
{
"id": "guardrails",
"version": "0.4.0",
"enabled": true,
"alwaysOn": false,
"dependsOn": []
},
{
"id": "reverse-proxy",
"version": "0.4.0",
"enabled": true,
"alwaysOn": true,
"dependsOn": ["provider", "routing"]
}
]

Response fields:

FieldTypeDescription
idstringModule identifier
versionstringModule version. Always equal to the running product's version, not versioned independently per module
enabledbooleanAlways true
alwaysOnbooleanAlways true
dependsOnarrayList of module IDs this module depends on; empty array if no dependencies

Errors: 403 insufficient permissions


Catalog​

Manage provider and model catalog repositories and cache.

Get Provider Catalog​

GET /api/providers

Auth: Authorization: Bearer <jwt> (requires model:read)

Returns the full provider catalog fetched from configured repositories.

Response 200:

{
"providers": [
{
"id": "openai",
"name": "OpenAI",
"models": [
{
"id": "gpt-5.2",
"name": "GPT-5.2",
"contextWindow": 128000,
"inputPrice": 1.75,
"outputPrice": 14.0,
"capabilities": ["functionCalling", "json", "vision"]
}
]
},
{
"id": "anthropic",
"name": "Anthropic",
"models": [
{
"id": "claude-opus-4-6",
"name": "Claude Opus 4.6",
"contextWindow": 200000,
"inputPrice": 5.0,
"outputPrice": 25.0,
"capabilities": ["functionCalling", "json"]
}
]
}
]
}

Errors: 403 insufficient permissions

Get Catalog Status​

GET /api/catalog/status

Auth: Authorization: Bearer <jwt> (requires settings:read)

Returns per-repository status information.

Response 200:

{
"repos": [
{
"url": "https://raw.githubusercontent.com/Inebrio/Routerly-Providers/main/",
"enabled": true,
"resolvedFile": "providers/providers.20260630120000.json",
"updatedAt": "2026-06-30T12:00:00.000Z",
"lastChecked": "2026-06-30T14:30:00.000Z",
"error": null
},
{
"url": "https://your-org.com/catalog/",
"enabled": true,
"resolvedFile": null,
"updatedAt": null,
"lastChecked": "2026-06-30T14:30:00.000Z",
"error": "HTTP 404: Not Found"
}
],
"cachedAt": "2026-06-30T14:30:00.000Z",
"expiresAt": "2026-06-30T14:35:00.000Z"
}

Fields:

  • url - repository endpoint
  • enabled - whether this repo is active
  • resolvedFile - filename of the last successfully fetched catalog (null if never fetched)
  • updatedAt - timestamp from the catalog registry (null if never fetched)
  • lastChecked - when Routerly last attempted to fetch from this repo
  • error - error message if the last fetch failed (null on success)
  • cachedAt - when the current catalog was loaded into memory
  • expiresAt - when the 5-minute cache expires

Errors: 403 insufficient permissions

Refresh Catalog Cache​

POST /api/catalog/refresh

Auth: Authorization: Bearer <jwt> (requires settings:write)

Invalidate the in-memory cache and fetch all enabled repositories immediately.

Response 200:

{ "ok": true, "message": "Catalog refreshed successfully" }

Response 200 (partial failure - some repos errored):

{
"ok": false,
"message": "Catalog refresh completed with errors",
"errors": [
{
"url": "https://your-org.com/catalog/",
"error": "HTTP 404: Not Found"
}
]
}

Errors: 403 insufficient permissions


Notifications​

List Channels​

GET /api/notifications/channels

Auth: Authorization: Bearer <jwt> (requires user:write)

Returns the configured notification channels array.

Create Channel​

POST /api/notifications/channels

Auth: Authorization: Bearer <jwt> (requires user:write)

Request body

{
"provider": "dashboard",
"name": "Budget Alerts",
"events": ["budget.*"],
"targets": {
"roles": ["admin"],
"permissions": [],
"users": ["user-uuid"]
}
}
FieldTypeRequiredDescription
providerstringyesChannel type: smtp, ses, sendgrid, azure, google, webhook, slack, teams, pagerduty, discord, dashboard
namestringnoFriendly label shown in the UI
eventsstring[]noEvent patterns routed to this channel (empty = all). Supports exact names, *, and prefix globs like budget.*
targetsobjectno{ roles, permissions, users } - who receives (empty = everyone). Controls inbox visibility for dashboard and recipient resolution for email channels; ignored for webhook/native channels

Provider-specific fields (e.g. host, apiKey, botToken) pass through alongside these base fields.

Response 201

{
"id": "abc-uuid",
"provider": "dashboard",
"name": "Budget Alerts",
"events": ["budget.*"],
"targets": { "roles": ["admin"] }
}

Errors: 400 invalid body (unknown provider, bad targets shape) · 403 insufficient permissions

Delete Channel​

DELETE /api/notifications/channels/:id

Auth: Authorization: Bearer <jwt> (requires user:write)

Response: 204 No Content

Errors: 404 channel not found · 403 insufficient permissions

Get Channel​

GET /api/notifications/channels/:id

Auth: Authorization: Bearer <jwt> (requires user:write)

Response 200

{
"id": "abc-uuid",
"provider": "smtp",
"name": "Admin Alerts",
"host": "smtp.example.com",
"port": 587,
"fromAddress": "[email protected]",
"fromName": "Routerly",
"events": ["provider.error", "budget.exhausted"],
"targets": { "roles": ["admin"] }
}

Secrets (API keys, passwords, tokens) are masked and shown as null when present in the channel config.

Errors: 404 channel not found · 403 insufficient permissions

Update Channel​

PATCH /api/notifications/channels/:id

Auth: Authorization: Bearer <jwt> (requires user:write)

Request body (all fields optional):

{
"name": "Updated Name",
"events": ["budget.*"],
"targets": { "roles": ["admin"], "permissions": [], "users": [] },
"host": "new-smtp.example.com",
"port": 587,
"password": "new_secret_password",
"apiKey": "new_api_key"
}
  • To clear event filters, send an empty array: "events": []
  • To clear targets, send an empty object: "targets": {}
  • Sending null for any non-secret optional field (e.g. "targets": null, "username": null) removes the field from the stored channel.
  • Secret fields (passwords, API keys, tokens) are only updated when explicitly provided and non-empty. Omitting a secret field leaves it unchanged.

Response 200

{
"id": "abc-uuid",
"provider": "smtp",
"name": "Updated Name",
"host": "new-smtp.example.com",
"port": 587,
"fromAddress": "[email protected]",
"events": ["budget.*"],
"targets": { "roles": ["admin"] }
}

Errors: 404 channel not found · 400 invalid body · 403 insufficient permissions

Test Channel​

POST /api/notifications/channels/:id/test

Auth: Authorization: Bearer <jwt> (requires user:write)

Request body (optional)

{ "to": "[email protected]" }

Response 200

{ "ok": true, "message": "Test notification sent" }

Errors: 404 channel not found · 400 send failed (details in message)

Notifications Inbox​

The in-app notification inbox is per-user, available to any authenticated dashboard user (no special permission required). Three gates decide whether an item reaches the caller:

  1. Audience - the targets of the dashboard channel that created the item; no targeting means everyone.
  2. Permissions - auth.* events need audit:read, config.model_* needs model:read, config.router_* needs router:read, system.* needs settings:read. Routing, provider and budget events are not gated.
  3. Routers - an item whose details.routerId points at a router the caller cannot reach (by routerIds scope or membership) is hidden. Callers without a router scope see every router.

Users can also dismiss items individually (soft delete), which removes them from their personal inbox only.

List Inbox Items​

GET /api/notifications/inbox?limit=50&page=1&pageSize=20&severity=all&category=&event=&unreadOnly=false&from=&to=

Query params:

  • limit - max items in legacy flat-list response (1–200, default 50). When page is omitted, activates flat-list mode; presence of page switches to paginated mode.
  • page - page number for paginated response (1-indexed, default 1)
  • pageSize - items per page (1–100, default 20)
  • severity - filter by severity: info, warning, critical, or all (default all)
  • category - filter by event category: routing, provider, budget, config, security, system. Omit for all categories
  • event - filter by event name substring (case-insensitive)
  • unreadOnly - when true, returns only items the current user has not read
  • from - start date (YYYY-MM-DD or ISO 8601 timestamp); when date-only, spans from 00:00
  • to - end date (YYYY-MM-DD or ISO 8601 timestamp); when date-only, spans to 23:59:59

Response 200 (flat-list mode when page omitted):

{
"items": [
{
"id": "8f3c…",
"event": "provider.error",
"severity": "critical",
"timestamp": "2026-06-24T12:00:00.000Z",
"details": { "modelId": "openai/gpt-4o", "traceId": "e7b1…" },
"read": false,
"traceId": "e7b1…",
"eventCount": 3
}
],
"unreadCount": 5,
"enabled": true
}

Events emitted while serving the same request share a trace id and are folded into a single item: traceId carries that id and eventCount how many events it represents. eventCount is omitted when the item stands alone. The list never carries the folded sequence - fetch the item to get it.

Response 200 (paginated mode when page is provided):

{
"items": [ /* … */ ],
"pagination": {
"page": 1,
"pageSize": 20,
"totalRecords": 127,
"totalPages": 7
},
"unreadCount": 5,
"enabled": true
}
  • items - notifications newest-first
  • unreadCount - total unread count for the current user (independent of filters)
  • enabled - true when at least one dashboard-provider channel is configured; false when the inbox is not yet initialized
  • pagination - only in paginated mode (when page query param is present)

Get Single Notification​

GET /api/notifications/inbox/:id

Response 200:

{
"id": "8f3c…",
"event": "provider.error",
"severity": "critical",
"timestamp": "2026-06-24T12:00:00.000Z",
"details": { "modelId": "openai/gpt-4o", "latencyMs": 5000, "traceId": "e7b1…" },
"read": false,
"traceId": "e7b1…",
"eventCount": 2,
"events": [
{
"event": "routing.fallback_used",
"severity": "info",
"timestamp": "2026-06-24T11:59:58.000Z",
"details": { "primaryModelId": "openai/gpt-4o", "fallbackModelId": "ollama/qwen2.5:3b" }
},
{
"event": "provider.error",
"severity": "critical",
"timestamp": "2026-06-24T12:00:00.000Z",
"details": { "modelId": "openai/gpt-4o", "latencyMs": 5000 }
}
]
}
  • eventCount - number of events folded into this item; 1 for a lone event
  • events - the incident's sequence, oldest first. Present only when a second event correlated on the same traceId. The top-level event, severity and details mirror the most severe entry
  • Only this endpoint returns events; the list endpoint stops at traceId and eventCount

Errors: 404 notification not found (does not exist, is not in the current user's inbox, or the caller lacks the permission or router scope the item requires)

Mark Inbox Items as Read​

POST /api/notifications/inbox/read

Marks inbox items as read for the current user. Provide either ids or all:

{ "ids": ["8f3c…", "1a2b…"] }
{ "all": true }

Returns 400 if neither is provided.

Response 200:

{ "updated": 2 }

All read operations are logged to the audit log as notification:read actions.

Mark Inbox Items as Unread​

POST /api/notifications/inbox/unread

Clears the current user's read mark on one or more notifications. Inverse of /read. Provide either ids or all:

{ "ids": ["8f3c…"] }
{ "all": true }

Returns 400 if neither is provided.

Response 200:

{ "updated": 1 }

All unread operations are logged to the audit log as notification:unread actions.

Delete (Dismiss) Inbox Items​

POST /api/notifications/inbox/delete

Dismisses (soft-deletes) one or more items from the current user's inbox only. Other users' copies of the same notification remain unaffected. Provide either ids or all:

{ "ids": ["8f3c…", "1a2b…"] }
{ "all": true }

Returns 400 if neither is provided.

Response 200:

{ "deleted": 2 }

Important: Deletion is per-user only. Global deletion is never performed. All delete operations are logged to the audit log as notification:delete actions.

Test (legacy endpoint)​

POST /api/notifications/test
{ "channelId": "abc-uuid" }

Returns 200 OK on success or an error with details. Prefer POST /api/notifications/channels/:id/test.


Integrations​

Export Routerly metrics to external monitoring systems (Prometheus, OpenTelemetry, Datadog, Grafana Cloud, InfluxDB, Webhooks).

List Integrations​

GET /api/integrations

Auth: Authorization: Bearer <jwt> (requires settings:write)

Response 200

{
"integrations": [
{
"id": "int-uuid",
"type": "prometheus",
"enabled": true,
"name": "Prometheus",
"authToken": null
},
{
"id": "int-uuid2",
"type": "otel",
"enabled": true,
"name": "OpenTelemetry",
"endpoint": "http://localhost:4318",
"protocol": "http",
"headers": {}
}
]
}

Secret fields (authToken, apiKey, token, secret) are masked and shown as null when present.

Errors: 403 insufficient permissions

Get Integration​

GET /api/integrations/:id

Auth: Authorization: Bearer <jwt> (requires settings:write)

Response 200 (example: Datadog integration)

{
"id": "int-uuid",
"type": "datadog",
"enabled": true,
"name": "Datadog Production",
"apiKey": null,
"site": "datadoghq.com"
}

Secret fields are masked and shown as null when present.

Errors: 404 integration not found · 403 insufficient permissions

Create Integration​

POST /api/integrations

Auth: Authorization: Bearer <jwt> (requires settings:write)

Request body - per-type examples:

Prometheus (pull, optional auth)

{
"type": "prometheus",
"name": "Prometheus",
"authToken": "secret-token-123"
}

OpenTelemetry (push)

{
"type": "otel",
"name": "OpenTelemetry Collector",
"endpoint": "http://localhost:4318",
"protocol": "http",
"headers": {
"Authorization": "Bearer otel-token"
}
}

Datadog (push)

{
"type": "datadog",
"name": "Datadog",
"apiKey": "dd_api_key_123",
"site": "datadoghq.com"
}

Site options: datadoghq.com (US East), datadoghq.eu (EU), us3.datadoghq.com, us5.datadoghq.com, ddog-gov.com (GovCloud).

Grafana Cloud (push, Prometheus remote_write)

{
"type": "grafana",
"name": "Grafana Cloud",
"url": "https://prometheus-blocks-prod-us-central1.grafana.net/api/prom/push",
"username": "123456",
"apiKey": "glc_api_key_123"
}

InfluxDB (push, v2)

{
"type": "influxdb",
"name": "InfluxDB",
"url": "http://localhost:8086",
"token": "my-token",
"org": "routerly",
"bucket": "metrics"
}

Webhook (push)

{
"type": "webhook",
"name": "Webhook",
"url": "https://example.com/metrics",
"secret": "signing-secret",
"headers": {
"X-Custom-Header": "value"
}
}

Webhook requests are signed with HMAC-SHA256 using the secret field; the signature is sent in the X-Routerly-Signature header (format: sha256=<hex>).

Response 201

{
"id": "int-uuid",
"type": "otel",
"enabled": true,
"name": "OpenTelemetry Collector",
"endpoint": "http://localhost:4318",
"protocol": "http",
"headers": {}
}

Errors: 400 invalid body · 403 insufficient permissions

Update Integration​

PATCH /api/integrations/:id

Auth: Authorization: Bearer <jwt> (requires settings:write)

Request body (all fields optional):

{
"name": "Updated Name",
"enabled": false,
"apiKey": "new-api-key"
}
  • To enable/disable: set "enabled": true or "enabled": false
  • Secret fields (authToken, apiKey, token, secret) are only updated when explicitly provided and non-empty. Omitting a secret field leaves it unchanged.
  • Per-type config fields (endpoint, protocol, url, etc.) can be updated individually

Response 200

{
"id": "int-uuid",
"type": "otel",
"enabled": false,
"name": "Updated Name",
"endpoint": "http://localhost:4318",
"protocol": "http"
}

Errors: 404 integration not found · 400 invalid body · 403 insufficient permissions

Delete Integration​

DELETE /api/integrations/:id

Auth: Authorization: Bearer <jwt> (requires settings:write)

Response: 204 No Content

Errors: 404 integration not found · 403 insufficient permissions

Test Integration​

POST /api/integrations/:id/test

Auth: Authorization: Bearer <jwt> (requires settings:write)

Tests connectivity to the external system. For Prometheus (pull-based), the test is a no-op (always succeeds). For push-based integrations, sends a real metrics push and returns the result.

Response 200

{ "ok": true, "message": "Connection successful" }

Response 200 (on push failure)

{ "ok": false, "message": "HTTP 401: Unauthorized" }

Errors: 404 integration not found · 403 insufficient permissions

Metrics Pushed​

All push-type integrations (OpenTelemetry, Datadog, Grafana, InfluxDB, Webhook) send the following metrics every 60 seconds:

MetricTypeLabels/DimensionsDescription
routerly_requests_totalCounterrouter, modelTotal request count
routerly_tokens_totalCountertype (input/output), router, modelTotal tokens consumed
routerly_cost_usd_totalGaugerouter, modelEstimated USD cost
routerly_request_duration_p50_msGaugerouterMedian request latency
routerly_request_duration_p95_msGaugerouter95th percentile latency
routerly_budget_used_ratioGaugerouterBudget consumption (0–1)

Each integration type sends these metrics in its native format (OTLP, Datadog Series API, Prometheus remote_write, InfluxDB line protocol, JSON webhook).

Trace Export​

Metrics answer "how much"; traces answer "why this model". An otel or webhook integration can also receive one export per proxied request:

{
"traces": { "enabled": true, "sampleRate": 0.1 }
}
FieldTypeDescription
enabledbooleanOff by default. Unlike the 60s metric push, trace export costs one outbound request per proxied request.
sampleRatenumber0–1, share of traces exported. Omit for every trace.

Accepted by POST /api/integrations and PATCH /api/integrations/:id, on otel and webhook only: Datadog, Grafana and InfluxDB are metric-only write paths and reject the field. A settings write takes effect immediately (the export path caches the integration list for 30 seconds otherwise).

OpenTelemetry — OTLP/HTTP JSON to <endpoint>/v1/traces, native spans:

  • one routerly.request server span per request, carrying routerly.trace_id and routerly.router_id
  • one routerly.<phase> internal child span per pipeline phase (routerly.routing.prepare, routerly.routing.execute, …)
  • every trace entry as a span event on its phase, named after the entry (router:result), with routerly.panel, routerly.module and the entry details as attributes (values over 4096 chars are truncated)

Span ids are derived from the trace id, so a retried export overwrites instead of duplicating.

Webhook — the signed envelope used by the metric push, with the trace as body:

{
"source": "routerly",
"type": "trace",
"timestamp": "2026-08-02T19:37:33.485Z",
"trace": {
"id": "018f3c2a-4b5d-7e8f-9012-34567890abcd",
"routerId": "7a1f9a3b-...",
"entries": [
{ "phase": "routing.prepare", "module": "router", "panel": "router-response", "message": "router:result", "details": { }, "at": 1785700142000 }
]
}
}

Prompts and answers are included only for routers with traceContent: true. An export failure is never surfaced on the proxy path: the request that produced the trace is unaffected.


Audit Log​

List Audit Entries​

GET /api/audit

Auth: Authorization: Bearer <jwt> (requires audit:read)

Returns paginated audit log entries. Requires audit:read permission.

Query parameters:

ParameterTypeDescription
userIdstringFilter by user ID or email (substring match on email)
actionstringFilter by action substring (e.g. model:create)
resultstringFilter by result: success, forbidden, or error
fromstringStart timestamp (ISO 8601)
tostringEnd timestamp (ISO 8601)
pagenumberPage number (default: 1)
pageSizenumberEntries per page (default: 50, max: 200)

Response 200

{
"entries": [
{
"id": "uuid",
"timestamp": "2026-06-25T13:03:26.767Z",
"userId": "uuid",
"email": "[email protected]",
"endpoint": "POST /api/models",
"action": "model:create",
"result": "success",
"details": { "id": "my-model" }
}
],
"pagination": {
"page": 1,
"pageSize": 50,
"totalRecords": 142,
"totalPages": 3
}
}

Errors: 403 insufficient permissions


System​

Get System Info​

GET /api/system/info

No authentication required.

Response:

{
"version": "0.2.0",
"nodeVersion": "v22.0.0",
"platform": "darwin",
"configDir": "/Users/you/.routerly/config",
"dataDir": "/Users/you/.routerly/data",
"uptimeSeconds": 3600,
"channel": "current",
"rawChannel": "current",
"isDocker": false,
"updateInfo": {
"available": true,
"currentVersion": "0.2.0",
"latestVersion": "0.3.0",
"channel": "current",
"releaseUrl": "https://github.com/Inebrio/Routerly/releases/tag/v0.3.0",
"checkedAt": "2026-06-09T10:00:00.000Z"
}
}

updateInfo is null if no check has completed yet (first 24 hours after boot). isDocker is true when the service is running inside a Docker container. channel is the stored value as-is; rawChannel is kept for compatibility and always equals channel — neither is ever silently substituted, so a persisted value is reported back verbatim rather than corrected (see Update Settings).


Force Update Check​

GET /api/system/update-check

Requires authentication. Forces an immediate check against the GitHub Releases API and returns the result. This also updates the cached value returned by GET /api/system/info.

Response: same shape as updateInfo above (UpdateInfo object).

{
"available": false,
"currentVersion": "0.2.0",
"latestVersion": "0.2.0",
"channel": "current",
"checkedAt": "2026-06-09T12:34:56.000Z",
"status": "ok"
}

status is ok when the channel was compared, disabled when update checking is off on this instance (non-production run or ROUTERLY_DISABLE_UPDATE_CHECK=true; no request is made and checkedAt is empty), or error when the request failed (the reason is in error, latestVersion is empty and available is false). currentVersion is always the installed version.


Trigger In-App Update​

POST /api/system/update

Admin only. Downloads and installs the latest version on the active channel and restarts the service. The response is returned immediately (202); the update runs in the background.

Constraints:

  • Returns 403 if the caller is not an admin.
  • Returns 403 if the service is running inside Docker (ROUTERLY_DOCKER=1). Pull the new image instead.
  • Returns 400 if the service is running on Windows. Run the installer manually.

Response 202:

{ "message": "Update started. The service will restart shortly." }

Poll GET /health to detect when the service has restarted. The CLI command routerly update run does this automatically.


Personal MCP Surface​

Every user manages their own MCP tokens and sees the tools those tokens expose. These routes live under /api/me: they act on the caller and require no permission beyond a valid dashboard session. This is distinct from the /mcp protocol surface itself (see Service: MCP Server), which is authenticated by an MCP token rather than a JWT.

List My Tools​

GET /api/me/mcp-tools

Auth: Authorization: Bearer <jwt>

Returns the tools the caller's own MCP tokens expose: the registry filtered by the caller's permissions. A tool the caller cannot call is never listed.

Response 200:

[
{
"name": "list_models",
"scope": "read",
"description": "List the models configured on this Routerly gateway (id, provider, context window). No secrets are returned.",
"sourceModule": "catalog.registry",
"permission": "model:read"
},
{
"name": "toggle_model",
"scope": "write",
"description": "Enable or disable one of a router's model refs (flips its `enabled` flag). The flag is persisted but not yet honored by the routing engine. Requires the 'router:write' permission.",
"sourceModule": "config.store",
"permission": "router:write"
}
]

sourceModule is the internal DI token key of the module backing the tool (e.g. catalog.registry, config.store). A tool is only present when its backing module is bootstrapped on this instance; a module that is not running removes its tools from the list entirely.

List My Tokens​

GET /api/me/mcp-tokens

Auth: Authorization: Bearer <jwt>

Response 200:

[
{
"id": "2f1c0b8a-6d4e-4a2f-9f10-0b3f1c8e77aa",
"name": "laptop",
"tokenSnippet": "sk-rt-mcp-8f3c",
"createdAt": "2026-07-01T09:12:00.000Z",
"lastUsedAt": "2026-07-31T18:40:12.000Z",
"expiresAt": "2027-01-01T00:00:00.000Z"
}
]

The stored SHA-256 hash never leaves the service: tokenSnippet is the display identity. lastUsedAt and expiresAt are absent when unset.

Create My Token​

POST /api/me/mcp-tokens

Auth: Authorization: Bearer <jwt>

{
"name": "laptop",
"expiresAt": "2027-01-01T00:00:00.000Z"
}

Fields:

  • name - 1 to 60 characters, unique among the caller's tokens (required)
  • expiresAt - ISO 8601 instant (optional). Omit for a token that never expires

Response 201: the token record plus token, the raw value. This is the only time it is returned; only its hash is stored.

{
"id": "2f1c0b8a-6d4e-4a2f-9f10-0b3f1c8e77aa",
"name": "laptop",
"tokenSnippet": "sk-rt-mcp-8f3c",
"createdAt": "2026-07-01T09:12:00.000Z",
"expiresAt": "2027-01-01T00:00:00.000Z",
"token": "sk-rt-mcp-8f3c1d…"
}

Errors: 400 invalid body · 409 An MCP token named "<name>" already exists

Revoke My Token​

DELETE /api/me/mcp-tokens/:id

Auth: Authorization: Bearer <jwt>

Response 204, no body. Revocation is immediate: a client using that token fails its next call. Only the caller's own tokens are visible here, so another user's id is reported as not found.

Errors: 404 { "error": "Token not found" }