bot-bottle/docs/prds/prd-new-egress-metering-budgets-cutoff.md

PRD prd-new: Egress metering, budgets, and forced cutoff

• Status: Draft
• Author: didericis
• Created: 2026-06-25
• Issue: #251

Summary

Add out-of-band cost enforcement on top of the egress plane: meter every agent's authoritative token usage at the egress proxy, decrement per-scope budgets without the agent's cooperation, and forcibly cut a bottle's egress when a budget is exhausted. The trigger (usage threshold) and the action (route-drop / freeze / kill) both run with no agent in the loop — this is enforcement, distinct from the supervise sidecar (PRD 0013), which is agent-initiated and so cannot stop a runaway agent.

This builds on infrastructure that already exists and is not re-designed here:

• The egress chokepoint — the MITM proxy every agent's traffic flows through (PRD 0001 / 0006 / 0017) — is where metering reads and cutoff acts.
• The host SQLite store (PRD 0067) is where usage, budget, and enforcement state persist, behind its existing repository API.
• Observability — the host dashboard and egress traffic logging (PRD 0055) — already surfaces what agents are doing.

This PRD adds only the three missing pieces: metering (authoritative accounting), budgets, and forced cutoff.

Problem

bot-bottle can't meter agent token usage or enforce a cost limit. When an agent crosses a token threshold, there is no way to kill its egress automatically — no human in the loop.

The existing supervise sidecar (PRD 0013) is entirely agent-initiated: every action begins with the agent voluntarily calling an MCP tool and an operator approving it. A runaway or expensive agent — exactly the cost-overrun case — will never call egress-block on itself. Supervision is therefore a collaborative recovery mechanism, not an enforcement mechanism; making it mandatory (#249) would not deliver forced cost-cutoff.

The requirement forces a distinction the current design blurs:

• Plane A — enforcement (this PRD). System → infrastructure. Meter usage, cut egress on threshold, account for cost. Out-of-band; independent of the agent. Unconditional — an enforcement plane you can opt out of isn't enforcement.
• Plane B — agent-facing recovery (the existing supervise sidecar). Agent → operator, approval-gated. Useful interactively; meaningless for a headless agent with no operator watching its queue. Remains optional.

This PRD builds the enforcement slice of Plane A — metering, budgets, and forced cutoff. The chokepoint that makes interception possible and the observability surfaces that display usage already exist; what's missing is turning measured usage into an enforced budget. It reframes the "always-on control" invariant of #249 as "the egress enforcement plane is always present" — a more defensible property than "every agent runs the agent-facing supervisor." Unsupervised (headless/CI/ephemeral) agents stay first-class: still subject to the mandatory meter + kill switch, they simply lack the agent-facing proposal tools they couldn't use anyway.

Goals / Success Criteria

• The egress proxy meters every request to a metered API host (e.g. api.anthropic.com) and records authoritative token usage per bottle and per agent provider, with no agent cooperation.
• A budget can be set at four scopes with deterministic precedence (agent → bottle → parent bottle → global host budget); the most-specific applicable budget governs.
• When usage crosses a budget, the bottle's configured cutoff policy (cutoff | freeze | kill) fires automatically, executed host-side on the egress plane — never via the supervise queue. An operator can also trigger the same cutoff on demand through the existing host controller/dashboard surface (that surface is not built here).
• Host budgets, default cutoff policy, and per-provider limits are declared in a new host-level ~/.bot-bottle/settings.yml, parseable by yaml_subset.py.
• Usage, budget state, and enforcement actions persist in the existing host SQLite store (PRD 0067) behind its repository API.

Non-goals

• The egress chokepoint — already implemented. The MITM egress proxy (PRD 0001 / 0006 / 0017) is the interception point; this PRD hooks metering and cutoff into it, it does not build or modify the proxy.
• Observability / dashboard — already implemented. Live usage display, the host dashboard, and egress traffic logging (PRD 0055) exist; this PRD writes the usage/enforcement data they surface but adds no new dashboard and no new visibility surface.
• The SQLite store foundation — already implemented (PRD 0067). This PRD adds metering/budget/enforcement tables behind the existing repository API; it does not introduce the store.
• Remote control / cross-host control plane. Web + mobile remote control, cross-host budgets, and the authn/transport they require are explicitly deferred. This is host-only.
• Dollar-denominated budgets. Budgets are token counts keyed by agent provider, not currency. Price tables are out of scope.
• Migrating existing flat-file state into SQLite. Resume metadata.json, transcripts, Dockerfile overrides, the supervise queue, and audit logs stay on the filesystem. Only the new metering/budget/enforcement ledger is SQL.
• Making the supervise sidecar (Plane B) mandatory. Out of scope here; this PRD is the answer to "what should be unconditional" (Plane A enforcement), leaving #249's Plane-B question open.
• Per-request hard pre-send blocking as the primary mechanism. The gate is budget-crossing detected at/after metering; a pre-flight estimator (below) is a refinement, not the core enforcement path.

Design

Two measurements: gate vs. account

There are two distinct needs, and they want different signals:

• Account (authoritative). Decrement the real budget from the API response, which already carries authoritative usage (Anthropic input_tokens / output_tokens, OpenAI usage). The egress addon already has a response(flow) hook (bot_bottle/egress_addon.py:460), so the real number is available with no extra network call. Caveat: agent traffic is mostly streaming SSE, so the response path must tail the stream for the final usage event rather than parse a single JSON body — scoped explicitly as work.
• Gate (estimate). To block before sending, only the request is available, so an estimator / provider count_tokens endpoint is the only option.

Calling count_tokens for accounting would be both less accurate and an extra metered egress call per request, so accounting uses response usage and the estimator is reserved for the optional pre-flight gate.

count_tokens on agent providers

Add an abstract count_tokens(request) -> int to the AgentProvider abstraction (bot_bottle/agent_provider.py):

• Default is a good-enough stdlib estimator. Prefer stdlib only; a small pip dependency for the sidecar is acceptable for the fallback if stdlib proves too inaccurate (this does not relax the package's stdlib-first stance — it would be a sidecar-only dep, like the bundle already carries).
• Built-in claude uses Anthropic's token-counting endpoint; built-in codex uses OpenAI's. These are exact for the gate but cost a metered call, so they are gate-only; accounting still comes from the response.

Budgets and precedence

Budgets are token counts keyed by agent provider name (the same names bottles already use). Four scopes, most-specific wins:

agent  →  bottle  →  parent bottle  →  global (host)

The global host budget is the highest-priority feature to ship; per-agent and per-bottle budgets override it for finer control. A budget can also be supplied at bottle launch (--budget or equivalent), overriding the settings.yml defaults for that run. Enforcement evaluates the effective budget as the nearest-defined scope at decrement time.

~/.bot-bottle/settings.yml

New host-level settings file (the ~/.bot-bottle/ root, not the per-repo .bot-bottle/ — host budgets must not be committed per-repo). Parsed by yaml_subset.py, so it must stay within that bounded subset (flat mappings, scalars; no anchors, no multi-line block scalars). Shape:

budget:
  claude: 5000000      # token budget keyed by agent provider
  codex: 2000000
shutdown: cutoff       # default cutoff policy: cutoff | freeze | kill

Forced cutoff and cutoff policy

On budget exhaustion (or an operator command via the existing surface), the configured per-bottle cutoff policy fires. The three policies map onto primitives that already exist:

• cutoff (default) — drop the bottle's routes.yaml to empty and reload (or isolate the bottle from the egress network); the agent/bottle keeps running but can no longer reach metered hosts. This is the route-drop already available on the egress plane (bot_bottle/backend/egress_apply.py).
• freeze — commit/snapshot state, then kill the agent/bottle; resumable later via bot_bottle/backend/freeze.py.
• kill — tear the bottle down without saving state (backend teardown).

The trigger lives in the metering path and the action in the egress/backend plane; neither touches the supervise proposal queue (design constraint from #251).

Persistence

Usage, budget state, and enforcement-audit rows live in the existing host SQLite store (PRD 0067) at ~/.bot-bottle/bot-bottle.db, behind its repository API — this PRD adds tables, not a store. SQLite is the right fit for the concurrency this introduces: a global token budget decremented by N egress sidecars in parallel is a read-modify-write race that atomic transactions + WAL handle for free, and the per-scope precedence rollup plus "sum across all bottles" is a GROUP BY rather than an N-directory rescan. The new tables carry a schema_version-managed migration like the rest of the store.

Where enforcement runs

Metering, budget evaluation, and the cutoff actions run out-of-band in the host-level controller — cross-bottle, independent of any agent, not a per-bottle daemon. The controller writes usage/enforcement state that the already-implemented observability surfaces (dashboard, traffic log) read; this PRD does not build or modify those surfaces.

Implementation chunks

Ordered, individually mergeable:

1. Metering at the egress proxy. Parse authoritative response usage (including SSE final-usage tailing) in the egress addon response hook; write per-bottle / per-provider usage rows to the ledger (new tables in the existing PRD 0067 store).
2. settings.yml + budget model. Host-level ~/.bot-bottle/settings.yml parsed by yaml_subset.py; budget precedence (agent → bottle → parent → global) and the --budget launch flag.
3. Forced cutoff + cutoff policy. Wire the threshold trigger to the cutoff / freeze / kill primitives on the egress/backend plane; record enforcement actions to the audit ledger.
4. count_tokens pre-flight gate (optional refinement). Abstract method + stdlib estimator default; Anthropic/OpenAI endpoints for built-in claude/codex; optional pre-send block.

Open questions

• SSE usage tailing robustness. Buffering streamed responses to extract the final usage event without breaking the agent's own stream consumption — how much of the body must the addon hold, and what's the failure mode if the stream is interrupted mid-flight?
• Crossing mid-request. A single response can push usage past budget only after it's already been delivered. Is post-hoc cutoff (next request blocked) sufficient, or is a pre-flight estimator gate (chunk 4) required for v1?
• Provider name ↔ metered host mapping. How does the proxy attribute a flow to an agent-provider budget key — by destination host, by bottle identity, or both?
• Parent-bottle budget semantics. For bottle extends (PRD 0025 / 0065) chains, does "parent bottle" mean the manifest parent, the launching bottle, or the full ancestry summed?