Zero-Overshoot Guarantee

The Concurrency Burst Problem

When autonomous multi-agent systems run parallel workloads (Fan-Out or Map-Reduce patterns), dozens or hundreds of requests hit the gateway simultaneously.

In traditional cloud gateways (such as Cloudflare AI Gateway):

1. The gateway checks spend against budget at ingress.
2. The check uses an eventually consistent or cached counter.
3. Because previous requests are still generating tokens on GPUs (1–3 seconds latency), the counter shows $0.00 spend.
4. All concurrent requests are admitted to the upstream provider!
5. Spend is only updated post-completion, creating an Overshoot Window.

---

Empirical Benchmark (Cloudflare vs AgentRail)

We tested both platforms under identical conditions:

• Configured Budget Limit: $0.050000 USD
• Concurrency Burst: 180 concurrent requests fired in the same event tick
• Models: Llama 3.1 8B + Llama 3.3 70B

Metric	Cloudflare AI Gateway	AgentRail Gateway
Hard Budget Limit	$0.050000 USD	$0.050000 USD
Requests Fired Simultaneously	180 requests	180 requests
Admitted (Dispatched)	180 (100.0%)	50 (27.8%)
Blocked at Ingress (HTTP 429)	0 (0.0%)	130 (72.2%)
Total Incurred Cost	$0.103409 USD	$0.050000 USD
Budget Overshoot ($)	+$0.053409 USD	$0.000000 USD
Budget Overshoot (%)	+106.8% (206.8% total)	0.00% (Zero-Overshoot)
Accounting Model	Post-Hoc / Eventual	Pre-Flight ACID

---

How AgentRail Guarantees 0.00% Overshoot

AgentRail uses Pre-Flight ACID Reservations in SQLite Durable Objects:

1. Worst-Case Cost Ceiling: Before dispatching, the gateway calculates the maximum possible cost based on max_tokens (Worst-Case BPE).
2. Atomic Ingress Lock (<10ms): Inside an ACID transaction, AgentRail verifies: $$\text{reservation} \le \text{budget} - \text{spent} - \text{active_reservations}$$
3. Instant Rejection: If capacity is exhausted, excess requests are rejected in memory at the edge with HTTP 429 BUDGET_EXCEEDED before sending a single network packet to OpenAI or Anthropic.
4. Dynamic Settlement: When the provider returns exact usage, the gateway settles the exact microdollar cost and immediately releases unspent reservations back into the pool.