diff --git a/docs/prds/prd-new-egress-control-plane.md b/docs/prds/prd-new-egress-control-plane.md new file mode 100644 index 0000000..f7bb414 --- /dev/null +++ b/docs/prds/prd-new-egress-control-plane.md @@ -0,0 +1,247 @@ +# PRD prd-new: Egress control plane — metering, budgets, and forced cutoff + +- **Status:** Draft +- **Author:** didericis +- **Created:** 2026-06-25 +- **Issue:** #251 + +## Summary + +Add an **out-of-band egress enforcement & observability plane**: meter every +agent's token usage at the egress proxy, decrement budgets without the agent's +cooperation, and forcibly cut a bottle's egress when a budget is exhausted — +either automatically or on command from a host-level dashboard. The trigger +(usage threshold) and the action (route-drop / freeze / kill) both live in the +egress plane and run with no agent in the loop. This is distinct from the +supervise sidecar (PRD 0013), which is agent-initiated and therefore cannot +enforce a cost cutoff on a runaway agent. State (usage ledger, budgets, audit) +moves into a host-level SQLite database behind a thin repository API, the first +SQL store in an otherwise flat-file repo. + +## Problem + +bot-bottle can't currently do two things the cost-overrun case demands: + +1. **Forced egress shutdown on limit.** When an agent crosses a token + threshold, kill its egress automatically — no human in the loop. +2. **Remote (host-level) management.** Drive agents from a single surface: + see usage, cut egress, stop bottles, to prevent cost overruns. + +The existing supervise sidecar (PRD 0013) is **entirely agent-initiated**: every +action begins with the agent voluntarily calling an MCP tool and an operator +approving it. A runaway or expensive agent — exactly the cost-overrun case — +will never call `egress-block` on itself. Supervision is therefore a +**collaborative recovery** mechanism, not an **enforcement** mechanism; making +it mandatory (#249) would not deliver forced cost-cutoff. + +The requirement forces a distinction the current design blurs: + +- **Plane A — enforcement / observability (this PRD).** System → infrastructure. + Meter usage, cut egress on threshold or command, account for cost. + Out-of-band; independent of the agent. **Unconditional** — an enforcement + plane you can opt out of isn't enforcement. +- **Plane B — agent-facing recovery (the existing supervise sidecar).** + Agent → operator, approval-gated. Useful interactively; meaningless for a + headless agent with no operator watching its queue. Remains optional. + +This PRD builds Plane A. It reframes the "always-on control" invariant of #249 +as "the egress control plane is always present" — a more defensible property +than "every agent runs the agent-facing supervisor." Unsupervised +(headless/CI/ephemeral) agents stay first-class: still subject to the mandatory +meter + kill switch, they simply lack the agent-facing proposal tools they +couldn't use anyway. + +## Goals / Success Criteria + +- The egress proxy meters every request to a metered API host (e.g. + `api.anthropic.com`) and records authoritative token usage per bottle and per + agent provider, with no agent cooperation. +- A budget can be set at four scopes with deterministic precedence + (**agent → bottle → parent bottle → global host budget**); the + most-specific applicable budget governs. +- When usage crosses a budget, the bottle's configured **cutoff policy** + (`cutoff` | `freeze` | `kill`) fires automatically, executed host-side on the + egress plane — never via the supervise queue. +- An operator can, from a single **host-level TUI dashboard**, see live per-bottle + usage against budget and command a cutoff/stop on demand. +- Host budgets, default cutoff policy, and per-provider limits are declared in a + new host-level `~/.bot-bottle/settings.yml`, parseable by `yaml_subset.py`. +- All usage, budget state, and enforcement actions persist in a host-level + SQLite DB behind a thin repository API, so the store can later be swapped for + a cross-host cloud service. + +## Non-goals + +- **Remote control / cross-host control plane.** Web + mobile remote control, + cross-host budgets, and the authn/transport they require are explicitly + deferred. v1 is a **host-only TUI** with no remote surface. +- **Dollar-denominated budgets.** Budgets are token counts keyed by agent + provider, not currency. Price tables are out of scope. +- **Migrating existing flat-file state into SQLite.** Resume `metadata.json`, + transcripts, Dockerfile overrides, the supervise queue, and audit logs stay on + the filesystem. Only the *new* metering/budget/enforcement ledger is SQL. +- **Making the supervise sidecar (Plane B) mandatory.** Out of scope here; this + PRD is the answer to "what should be unconditional" (Plane A), leaving #249's + Plane-B question open. +- **Per-request hard pre-send blocking as the primary mechanism.** The gate is + budget-crossing detected at/after metering; a pre-flight estimator (below) is a + refinement, not the core enforcement path. + +## Design + +### Two measurements: gate vs. account + +There are two distinct needs, and they want different signals: + +- **Account (authoritative).** Decrement the real budget from the API + **response**, which already carries authoritative usage (Anthropic + `input_tokens` / `output_tokens`, OpenAI `usage`). The egress addon already + has a `response(flow)` hook (`bot_bottle/egress_addon.py:460`), so the real + number is available with no extra network call. **Caveat:** agent traffic is + mostly streaming SSE, so the response path must tail the stream for the final + usage event rather than parse a single JSON body — scoped explicitly as work. +- **Gate (estimate).** To block *before* sending, only the request is available, + so an estimator / provider `count_tokens` endpoint is the only option. + +Calling `count_tokens` for accounting would be both less accurate *and* an extra +metered egress call per request, so accounting uses response `usage` and the +estimator is reserved for the optional pre-flight gate. + +### `count_tokens` on agent providers + +Add an abstract `count_tokens(request) -> int` to the `AgentProvider` +abstraction (`bot_bottle/agent_provider.py`): + +- **Default** is a good-enough stdlib estimator. Prefer stdlib only; a small + pip dependency *for the sidecar* is acceptable for the fallback if stdlib + proves too inaccurate (this does not relax the package's stdlib-first stance — + it would be a sidecar-only dep, like the bundle already carries). +- **Built-in `claude`** uses Anthropic's token-counting endpoint; + **built-in `codex`** uses OpenAI's. These are exact for the gate but cost a + metered call, so they are gate-only; accounting still comes from the response. + +### Budgets and precedence + +Budgets are token counts keyed by **agent provider name** (the same names +bottles already use). Four scopes, most-specific wins: + +``` +agent → bottle → parent bottle → global (host) +``` + +The global host budget is the highest-priority feature to ship (the cross-host +control plane will eventually consume it); per-agent and per-bottle budgets +override it for finer control. A budget can also be supplied **at bottle +launch** (`--budget` or equivalent), overriding the settings.yml defaults for +that run. Enforcement evaluates the effective budget as the +nearest-defined scope at decrement time. + +### `~/.bot-bottle/settings.yml` + +New **host-level** settings file (the `~/.bot-bottle/` root, *not* the per-repo +`.bot-bottle/` — host budgets must not be committed per-repo). Parsed by +`yaml_subset.py`, so it must stay within that bounded subset (flat mappings, +scalars; no anchors, no multi-line block scalars). Shape: + +```yaml +budget: + claude: 5000000 # token budget keyed by agent provider + codex: 2000000 +shutdown: cutoff # default cutoff policy: cutoff | freeze | kill +``` + +### Forced cutoff and cutoff policy + +On budget exhaustion (or an operator command), the configured per-bottle cutoff +policy fires. The three policies map onto primitives that already exist: + +- **`cutoff`** (default) — drop the bottle's `routes.yaml` to empty and reload + (or isolate the bottle from the egress network); the agent/bottle keeps + running but can no longer reach metered hosts. This is the route-drop already + available on the egress plane (`bot_bottle/backend/egress_apply.py`). +- **`freeze`** — commit/snapshot state, then kill the agent/bottle; resumable + later via `bot_bottle/backend/freeze.py`. +- **`kill`** — tear the bottle down without saving state (backend teardown). + +The trigger lives in the metering path and the action in the egress/backend +plane; **neither touches the supervise proposal queue** (design constraint from +#251). + +### Host-level SQLite store + +**Decision: introduce SQLite now, narrowly.** + +- **The dependency objection doesn't apply.** `sqlite3` is in the Python stdlib, + so it does not break the AGENTS.md stdlib-first / no-runtime-pip stance — same + category as the hand-rolled `yaml_subset.py`, except the stdlib already ships + the whole engine. +- **It fits the problem.** A *global* token budget decremented concurrently by N + egress sidecars (today `~/.bot-bottle/` already has `state/`, `audit/`, + `queue/` written by parallel bottles) is a read-modify-write race. Over JSON + that means hand-rolled file locking; SQLite gives atomic transactions + WAL for + free. The per-agent/per-bottle precedence rollup plus "sum across all bottles" + is a `GROUP BY`, not an N-directory rescan. +- **It rehearses the cloud swap.** "Wrap operations in an API so we can swap to a + cloud service" maps directly onto a thin repository/DAO over SQLite → Postgres + later. A JSON-file store is a worse rehearsal than SQL. + +**Costs (real but bounded):** a new paradigm in a flat-file repo needs a +`schema_version` table + idempotent startup migrations; SQLite serializes +writers, so WAL mode + `busy_timeout` are required (a non-issue at a handful of +bottles); test fixtures need temp DBs. + +**Scope of the store:** one DB at `~/.bot-bottle/bot-bottle.db` behind a thin +repository API. Only the **new** metering/budget/enforcement-audit ledger lives +there. Existing per-bottle blobs (resume `metadata.json`, transcripts, +Dockerfile overrides, supervise queue) stay on the filesystem — migrating them +now is churn for no benefit and they lack the concurrency/aggregation problem. + +### Host-level controller + dashboard + +A single **host-level controller** owns the meter, budget evaluation, and the +cutoff actions across all bottles (cf. `bot_bottle/cli/supervise.py`'s +cross-bottle view), rather than a per-bottle daemon. v1 ships one host-level +**TUI dashboard** that reads live usage-vs-budget from the SQLite store and +offers on-demand cutoff/stop. The existing supervisor UI should eventually fold +into this same dashboard; this PRD lays the host-level surface it will move to. + +## Implementation chunks + +Ordered, individually mergeable: + +1. **SQLite repository foundation.** `~/.bot-bottle/bot-bottle.db`, schema + + `schema_version` migrations, WAL + `busy_timeout`, thin repository API, + temp-DB test fixtures. No behavior wired yet. +2. **Metering at the egress proxy.** Parse authoritative response `usage` + (including SSE final-usage tailing) in the egress addon `response` hook; + write per-bottle / per-provider usage rows to the ledger. +3. **`settings.yml` + budget model.** Host-level `~/.bot-bottle/settings.yml` + parsed by `yaml_subset.py`; budget precedence (agent → bottle → parent → + global) and the `--budget` launch flag. +4. **Forced cutoff + cutoff policy.** Wire the threshold trigger to the + `cutoff` / `freeze` / `kill` primitives on the egress/backend plane; record + enforcement actions to the audit ledger. +5. **Host-level TUI dashboard.** Live usage-vs-budget view + on-demand + cutoff/stop, reading the store. +6. **`count_tokens` pre-flight gate (optional refinement).** Abstract method + + stdlib estimator default; Anthropic/OpenAI endpoints for built-in + claude/codex; optional pre-send block. + +## Open questions + +- **SSE usage tailing robustness.** Buffering streamed responses to extract the + final usage event without breaking the agent's own stream consumption — how + much of the body must the addon hold, and what's the failure mode if the + stream is interrupted mid-flight? +- **Crossing mid-request.** A single response can push usage past budget only + *after* it's already been delivered. Is post-hoc cutoff (next request blocked) + sufficient, or is a pre-flight estimator gate (chunk 6) required for v1? +- **Provider name ↔ metered host mapping.** How does the proxy attribute a + flow to an agent-provider budget key — by destination host, by bottle + identity, or both? +- **Parent-bottle budget semantics.** For `bottle extends` (PRD 0025 / 0065) + chains, does "parent bottle" mean the manifest parent, the launching bottle, + or the full ancestry summed? +- **Dashboard ↔ controller transport (even host-only).** In-process, a local + socket, or polling the SQLite store directly? Picks the seam the future remote + control plane will extend.