diff --git a/docs/prds/0081-reprovision-gateway-state-on-bringup.md b/docs/prds/0081-reprovision-gateway-state-on-bringup.md index 58e88a06..81756c75 100644 --- a/docs/prds/0081-reprovision-gateway-state-on-bringup.md +++ b/docs/prds/0081-reprovision-gateway-state-on-bringup.md @@ -49,29 +49,40 @@ destroy (the original #450 failure mode). running agents by the same reconcile. - Per-bottle failures are tolerated: one unreachable or malformed bottle does not block the others or the gateway coming up. +- **All backends** (Firecracker, docker, macOS) reconcile through the *same* + contract — an abstract method on the backend base class, so a new backend + cannot forget to implement it and none drifts onto a bespoke mechanism. ## Non-goals - Deliberate mid-session CA rotation *without* a gateway restart — `rotate_ca` stays for that operator action. -- The docker / macOS backends. This lands the reconcile for Firecracker (the - backend that boots a fresh gateway rootfs each time); docker keeps its existing - host-bind-mounted CA until the same seam is extended there (follow-up). The - unification target is *per-service*, not *per-backend*. - Changing source-IP attribution, `/resolve`, or the plane split (#469). ## Design -**Trigger — the gateway startup flow.** `FirecrackerGateway.connect_to_orchestrator` -is the cold-boot path: `FirecrackerInfraService.ensure_running` calls it only when -it boots a fresh pair, and skips it when it adopts a healthy, current gateway -(state intact). So reconcile fires exactly when the gateway was (re)booted — -including orchestrator restarts, since the pair boots together — and never on an -adopt. There is no bare-restart path that bypasses `ensure_running`. +**The contract — `attach_bottled_agents_to_gateway()` on the backend ABC.** +Reconciling running bottles against the current gateway is a backend +responsibility (only the backend can enumerate its agents and reach them — +firecracker over SSH, docker/macOS over `exec`/`cp`), so it is an +`@abc.abstractmethod` on `BottleBackend` (`backend/base.py`). Every backend +implements it; the host calls it whenever the gateway is (re)brought up. This is +what makes the fix cross-backend by construction rather than a per-backend +follow-up. It reprovisions **all** registered bottles' gateway-dependent state: +CA, git-gate, and egress tokens. -**`reconcile_running_bottles(gateway, orchestrator_url)`** (new -`firecracker/reconcile.py`), called at the end of `connect_to_orchestrator` once -the gateway VM is up and its CA is available: +**Trigger — the gateway bring-up path.** The host calls +`attach_bottled_agents_to_gateway()` only when the gateway was actually +(re)brought up — the cold-boot branch of the infra bring-up (e.g. +`FirecrackerInfraService.ensure_running` after it boots a fresh pair), never on an +adopt of a healthy, current gateway (state intact). So it fires exactly when the +gateway was (re)booted — including orchestrator restarts, since the pair boots +together — and there is no bare-restart path that bypasses bring-up. + +**Per-backend implementation.** Each backend's `attach_bottled_agents_to_gateway` +enumerates its live bottles and, for each, reconciles the three services against +the current gateway. The firecracker implementation, once the gateway VM is up +and its CA is available: 1. Map each live bottle's guest IP → `bottle_id` from the orchestrator registry (`list_bottles`). @@ -100,6 +111,7 @@ as a CA rotation with zero extra machinery. ## Open questions -- Docker / macOS parity: extend the same reconcile seam (docker `exec` instead of - SSH) and drop docker's CA bind-mount, or leave docker on persistence? Deferred - to a follow-up; not blocking the Firecracker fix. +- Docker's existing host-bind-mounted CA (`host_gateway_ca_dir`): once docker's + `attach_bottled_agents_to_gateway` pushes the CA to running agents on bring-up, + the bind-mount is redundant — drop it (so docker rotates like firecracker) or + keep it as belt-and-suspenders? Leaning drop, for one behaviour across backends.