Profile pin · TypeScript

llm-gateway

LLM proxy with routing, guardrails, atomic per-key budgets, and failover across OpenAI, Anthropic, Gemini, and Ollama.

177 backend + 41 dashboard tests. Benchmark timings are single-process, not throughput figures.

Language: TypeScript Updated: 2026-06-16 Visibility: public

Without a proxy control point, every app talks to providers directly—routing, budgets, guardrails, and fallback live in scattered code instead of one enforceable layer you can audit and change once.

The proxy is the control point.

Offline demo:

make demo-ui

Walkthrough

Sample requests show fallback, budget stop, and guardrail block in one log. Sample data.

Proxy console with provider health and budget enforcement. Sample data.

Why it matters

LLM proxy with routing, guardrails, atomic per-key budgets, and failover across OpenAI, Anthropic, Gemini, and Ollama.

Repo facts

Default branch: main. Stars: 0. Forks: 0. Created: 2026-05-31.