diff --git a/docs/prds/prd-new-consolidate-docker-infra-backend.md b/docs/prds/prd-new-consolidate-docker-infra-backend.md new file mode 100644 index 0000000..e50eb92 --- /dev/null +++ b/docs/prds/prd-new-consolidate-docker-infra-backend.md @@ -0,0 +1,155 @@ +# PRD prd-new: Consolidate infra backend for Docker + +- **Status:** Draft +- **Author:** Claude +- **Created:** 2026-07-20 +- **Issue:** #431 + +## Summary + +The Docker backend runs two containers — `bot-bottle-orch-gateway` (gateway +data plane) and `bot-bottle-orchestrator` (control plane) — where the +macOS and Firecracker backends already run a single combined infra +unit. This PRD collapses Docker to the same model: one `bot-bottle-infra` +container running both processes under the `gateway_init` supervise tree, a +restructured `Dockerfile.infra` as the shared gateway+orchestrator base, +and a handful of extracted shared utilities (CA cert polling, teardown +sequence, launch skeleton) that are currently duplicated across all three +`consolidated_launch.py` files. + +## Goals / success criteria + +- Docker backend starts exactly one infra container instead of two. +- `Dockerfile.infra` is the shared base image (gateway + orchestrator, no + buildah); the Firecracker image layers buildah on top of it. +- The orchestrator process runs under the `gateway_init` supervise tree + inside the combined container (one PID-1, one restart/health surface). +- CA cert polling, the teardown sequence, and the shared launch skeleton + (ensure-infra → register → provision → return context) live in a single + shared module; all three backends import from it. +- No functional change to macOS or Firecracker launch paths. + +## Non-goals + +- Changing per-bottle isolation — agents stay one-VM/container-each. +- Consolidating transport implementations (`DockerGatewayTransport`, + `AppleGatewayTransport`, `SshGatewayTransport`) — these are already the + right abstraction boundary. +- macOS DHCP-inversion of registration order — irreducible backend + difference, stays as-is. +- Any changes to the orchestrator RPC protocol or the attribution model. + +## Design + +### Dockerfile restructuring + +**Current shape:** + +- `Dockerfile.gateway` — data plane (mitmproxy, gitleaks, git, openssh, + supervise daemons) +- `Dockerfile.orchestrator` — control plane (python:3.12-slim + bot_bottle + package; stdlib-only, no third-party deps) +- `Dockerfile.infra` — Firecracker only: `FROM bot-bottle-gateway` + + buildah + `COPY --from bot-bottle-orchestrator` + +**New shape:** + +- `Dockerfile.gateway` — unchanged +- `Dockerfile.orchestrator` — unchanged (single definition of orchestrator + content; both Docker infra and Firecracker infra `COPY --from` it) +- `Dockerfile.infra` — **shared base**: `FROM bot-bottle-gateway` + `COPY + --from bot-bottle-orchestrator` (no buildah — Docker infra image) +- `Dockerfile.infra.fc` — Firecracker only: `FROM bot-bottle-infra` + + buildah/crun/netavark/aardvark-dns (layered on the shared base, same net + result as today) + +The comment in `Dockerfile.infra` that says "the docker backend keeps +orchestrator + gateway as separate images; this combined image exists only +for the Firecracker single-VM cut" is removed. + +### Orchestrator in the supervise tree + +`gateway_init` already supervises the data-plane daemons (egress, git-http, +supervise-MCP). The orchestrator control plane is added as another supervised +process: `python3 -m bot_bottle.orchestrator --host 0.0.0.0 --port +--broker stub`. + +The orchestrator source is bind-mounted (`/app` → repo root, as today) so +dev live-reload still works. `source_hash`-based container recreation in +`OrchestratorService.ensure_running` continues to apply — a code change +recreates the combined infra container, which bounces both gateway and +orchestrator. This is acceptable: the docker backend is a dev/legacy target +where in-flight egress connections across a code deploy are not a hard +requirement. + +### `OrchestratorService` changes + +`OrchestratorService` currently starts two containers in sequence: gateway +first (`DockerGateway.ensure_running`), then orchestrator. After this PRD: + +- Single `docker run` of `bot-bottle-infra:latest` +- Container name: `bot-bottle-infra` (replaces `bot-bottle-orch-gateway` + + `bot-bottle-orchestrator`) +- Published ports: `127.0.0.1:{port}:{port}` for the control plane (same as + today) +- Bind mounts: repo root + host root (same as today) +- `DockerGateway` becomes an implementation detail of `OrchestratorService` + rather than a separately started container; the gateway image name + (`GATEWAY_IMAGE`) is no longer referenced at runtime, only at build time + for the `Dockerfile.infra` base + +The `_gateway()` / `ensure_running` two-step in `OrchestratorService` is +replaced by a single `_run_infra_container()`. + +### Shared backend utilities + +Three items are duplicated across +`backend/docker/consolidated_launch.py`, +`backend/macos_container/consolidated_launch.py`, and +`backend/firecracker/consolidated_launch.py`: + +1. **CA cert polling loop** — `deadline = time.monotonic() + timeout; while + ...: try fetch CA; sleep` — extracted to + `backend/consolidated_util.py:poll_ca_cert(transport, *, timeout)`. + +2. **Teardown sequence** — `OrchestratorClient(url).teardown_bottle(id)` + + `deprovision_git_gate(transport, id)` — extracted to + `backend/consolidated_util.py:teardown_consolidated(url, transport, + bottle_id)`. + +3. **Launch skeleton** — all three follow: ensure-infra → allocate/register + → provision git-gate → fetch CA cert → return launch context. The macOS + inversion (agent starts before registration, source IP from DHCP) is the + only deviation. Extract a shared `_provision_bottle(transport, bottle_id, + plan, orchestrator_url)` helper covering the register → provision → + return-token steps; the backends keep their own `launch_consolidated` + wrappers for the before/after (infra-ensure + agent-start + IP + allocation), calling the shared helper. + +The new `backend/consolidated_util.py` module holds only backend-neutral, +transport-agnostic logic. All three backends import from it. + +## Implementation chunks + +1. **(this PR)** Dockerfile restructuring: rename current `Dockerfile.infra` + content to `Dockerfile.infra.fc`; write new `Dockerfile.infra` as + gateway+orchestrator base. Update Firecracker image-build references from + `Dockerfile.infra` → `Dockerfile.infra.fc`. + +2. Add orchestrator process to `gateway_init` supervise tree. + +3. Collapse `OrchestratorService` to a single-container start; rename + container from `bot-bottle-orch-gateway`/`bot-bottle-orchestrator` → + `bot-bottle-infra`; update image name constant. + +4. Extract `backend/consolidated_util.py` with `poll_ca_cert`, + `teardown_consolidated`, and `_provision_bottle`; update all three + `consolidated_launch.py` files to import from it. + +5. Update tests that reference the old container names or two-container + startup sequence. + +## Open questions + +None — the supervise-tree approach and shared Dockerfile layering were +confirmed in issue #431.