How Gauge works

Five steps from connected
to fully automated.

No new SDK. No changes to your prompts. Gauge sits above your existing gateway, learns from your real traffic, and tells your gateway which model to use — based on quality scores and customer margin, not intuition.

Zero lock-in. Gauge never sits in your critical path. If you disconnect, your traffic keeps flowing exactly as it did before — nothing breaks, nothing changes. Worst case: you spend an afternoon connecting it and decide it's not for you.
01
Day 1 · ~30 minutes
Connect your stack
Three connections. No SDK, no code changes on your side. One gateway webhook covers every model you're already running — GPT-4o, Claude, Gemini, your own fine-tuned models, all of it.
✓GatewayLiteLLM · Bifrost · OpenRouter
✓RevenueStripe · Chargebee · Maxio · Recurly · Paddle · CSV
✓BillingOpenAI · Anthropic · Google · AWS · RunPod
Once connected, Gauge starts learning from your real traffic immediately — no configuration required.
02
Continuous · out of band
Map every request to a paying customer
Every completion event flowing through your gateway carries a customer_id. Gauge maps that ID to their billing platform subscription (Stripe, Chargebee, Maxio, or others) and starts computing a running AI cost per customer in real time — giving you a per-customer view of AI spend that connects directly to revenue.
Inbound completion event // example · Acme Legal Tech is Gauge's customer · Kirkland & Ellis is one of Acme's law firm clients
{
  "customer_id": "cus_kirkland_0012",
  "model": "gpt-4o",
  "prompt_tokens": 3200,
  "feature": "contract_summary",
  "cost_usd": 0.00832
}
→ Kirkland & Ellis · MRR $24,000 · AI cost this month: $5,820 · AI margin: 75.8%
03
Days 2–5 · continuous · out of band
Validate every model decision with real traffic — not intuition
Gauge mirrors 5% of live requests to candidate models in parallel — your users see nothing. An LLM-as-judge scores both outputs for semantic similarity. Only when a task type clears the quality floor does Gauge instruct the gateway to switch. But the mirroring itself is the easy part — it's what surrounds it that makes the decision trustworthy.
What takes ongoing investment to maintain
·Sample size validation before trusting a result
·Per-customer variance — same task, different quality by customer
·Domain-calibrated judge — generic similarity ≠ domain quality
·Drift detection as models and prompts change over time
·Cross-customer benchmarks that improve with every new dataset
What Gauge does
✓Minimum 200 samples before any decision is trusted
✓Separate quality floor per customer, per task type
✓Judge calibrated to your domain on first connect
✓Re-evaluates every 500 requests automatically
✓Pre-populated benchmarks from 50+ companies on day one
Live validation results · contract_summary · Llama 3.1 8B
Customer
Samples
Score
Floor
Decision
Kirkland & Ellis
482
0.88
0.82
✓ validated
Latham & Watkins
314
0.86
0.82
✓ validated
Davis Polk
201
0.79
0.82
— held
// same task type · different result per customer · this is why per-customer validation matters
⚡ Quality validation is an ongoing commitment — models change, prompts evolve, customer behaviour shifts. Gauge keeps this current automatically so your team doesn't have to.
04
Ongoing · automatic
Tell your gateway which model to use — based on true cost, not just the token price
Once quality is proven, Gauge sends a model override instruction to your gateway. But before choosing a provider, it calculates the effective cost per token — not just the headline rate. For self-hosted models, Gauge factors in your GPU lease cost and current utilization to compute what each token is actually costing you. A $0.14/1M Llama on RunPod at 40% utilization is actually more expensive than Fireworks at $0.20/1M.
Managed inference
Fireworks AI · Llama 3.1 8B
Token price$0.20 / 1M
Infra costbundled
Effective cost$0.20 / 1M ✓
Self-hosted · RunPod
Your cluster · Llama 3.1 8B
Lease cost$3,100 / mo
Utilization40%
Effective cost$0.31 / 1M ✗
⚡ Gauge instructs your gateway to use Fireworks — even though RunPod's headline rate is cheaper — because the effective cost at current utilization is higher.
Routing override · outbound webhook sent
{
  "action": "model_instruction",
  "customer_id": "cus_kirkland_0012",
  "task_type": "contract_summary",
  "model": "meta-llama/Llama-3.1-8B",
  "provider": "fireworks",
  "effective_cost_per_1m": 0.20,
  "reason": "quality_cleared · lowest_effective_cost"
}
To disconnect: remove the gateway webhook and revoke your billing platform OAuth. Your traffic was never routed through Gauge — it flows through your own gateway exactly as before. Nothing breaks.
05
Always on
Monitor AI margin, cloud compute costs, and when to switch
Gauge tracks three cost layers simultaneously: per-token inference costs from your gateway, cloud compute lease costs from RunPod/Lambda/AWS, and revenue per customer from your billing platform. The margin matrix shows AI unit margin per customer. The GPU crossover engine computes the exact date when your self-hosted fleet becomes cheaper than managed inference at your current growth rate. PR guardrails block features that would break margins before they ship.
AI Margin
75.8%
per customer · live
GPU crossover
Day 14
self-host beats API
PRs blocked
3
this month · $4,280 saved
Cloud compute sources Gauge monitors
Managed inference (Fireworks, Baseten, OpenRouter) per-token · bundled
Neocloud GPU lease (RunPod, Lambda, CoreWeave) monthly lease + utilization
Hyperscaler (AWS Bedrock, GCP Vertex, Azure) API + reserved capacity
Projected outcome · customer example

What this looks like after 30 days.

// Acme Legal Tech · an AI-native legal SaaS company · running contract summaries, clause extraction, and document Q&A on behalf of law firm clients · $50k/mo AI spend · not an actual customer result

AI margin
34% → 71%
quality unchanged
Monthly AI spend
$50k → $17.5k
same traffic volume
Recovered / month
$32,500
zero user complaints
Common questions
Does Gauge sit in the critical path of my requests?
No. Gauge operates out of band — it reads your gateway logs and sends model override instructions via webhook. If Gauge goes down, your traffic flows normally through your existing gateway, using whatever model it was last instructed to use. Zero single point of failure.
What if the cheaper model starts degrading in quality?
Gauge continuously re-evaluates quality scores — not just once at setup. Every 500 requests per task/customer pair, it re-samples and recalculates. If quality drifts below the floor, it automatically pulls traffic back to the frontier model.
Do I need to change my prompts or code?
No. Your prompts stay exactly as they are. Gauge intercepts at the gateway layer and routes transparently. The only setup required is connecting your gateway and billing platform — no SDK, no code changes on your side.
Which gateways does Gauge work with?
LiteLLM, Bifrost, and OpenRouter are all supported. If you're running a different gateway, reach out — we're actively expanding integrations based on design partner needs.
Where does MRR actually come from?
From your billing platform via a read-only connection — Gauge supports Stripe, Chargebee, Maxio, Recurly, Paddle, and CSV import. Gauge reads active subscriptions and maps each customer to their MRR. It never writes to your billing system.
What does it cost?
We're in early design-partner mode — pricing isn't public yet. Design partners get direct access to the roadmap and early pricing. If you're spending $20k+/mo on AI inference and want to be involved, reach out.

Ready to connect?

10 design-partner spots. Direct line to the roadmap. No payment required.

Get access → See Gauge in action →