docs(prd): consolidate infra backend for docker
Adds PRD for collapsing the Docker backend from two containers (gateway + orchestrator) to a single bot-bottle-infra container, restructuring Dockerfile.infra as the shared base, and extracting duplicated CA polling / teardown / provision helpers into a shared backend utility module. Closes #431
This commit is contained in:
@@ -0,0 +1,155 @@
|
||||
# PRD prd-new: Consolidate infra backend for Docker
|
||||
|
||||
- **Status:** Draft
|
||||
- **Author:** Claude
|
||||
- **Created:** 2026-07-20
|
||||
- **Issue:** #431
|
||||
|
||||
## Summary
|
||||
|
||||
The Docker backend runs two containers — `bot-bottle-orch-gateway` (gateway
|
||||
data plane) and `bot-bottle-orchestrator` (control plane) — where the
|
||||
macOS and Firecracker backends already run a single combined infra
|
||||
unit. This PRD collapses Docker to the same model: one `bot-bottle-infra`
|
||||
container running both processes under the `gateway_init` supervise tree, a
|
||||
restructured `Dockerfile.infra` as the shared gateway+orchestrator base,
|
||||
and a handful of extracted shared utilities (CA cert polling, teardown
|
||||
sequence, launch skeleton) that are currently duplicated across all three
|
||||
`consolidated_launch.py` files.
|
||||
|
||||
## Goals / success criteria
|
||||
|
||||
- Docker backend starts exactly one infra container instead of two.
|
||||
- `Dockerfile.infra` is the shared base image (gateway + orchestrator, no
|
||||
buildah); the Firecracker image layers buildah on top of it.
|
||||
- The orchestrator process runs under the `gateway_init` supervise tree
|
||||
inside the combined container (one PID-1, one restart/health surface).
|
||||
- CA cert polling, the teardown sequence, and the shared launch skeleton
|
||||
(ensure-infra → register → provision → return context) live in a single
|
||||
shared module; all three backends import from it.
|
||||
- No functional change to macOS or Firecracker launch paths.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Changing per-bottle isolation — agents stay one-VM/container-each.
|
||||
- Consolidating transport implementations (`DockerGatewayTransport`,
|
||||
`AppleGatewayTransport`, `SshGatewayTransport`) — these are already the
|
||||
right abstraction boundary.
|
||||
- macOS DHCP-inversion of registration order — irreducible backend
|
||||
difference, stays as-is.
|
||||
- Any changes to the orchestrator RPC protocol or the attribution model.
|
||||
|
||||
## Design
|
||||
|
||||
### Dockerfile restructuring
|
||||
|
||||
**Current shape:**
|
||||
|
||||
- `Dockerfile.gateway` — data plane (mitmproxy, gitleaks, git, openssh,
|
||||
supervise daemons)
|
||||
- `Dockerfile.orchestrator` — control plane (python:3.12-slim + bot_bottle
|
||||
package; stdlib-only, no third-party deps)
|
||||
- `Dockerfile.infra` — Firecracker only: `FROM bot-bottle-gateway` +
|
||||
buildah + `COPY --from bot-bottle-orchestrator`
|
||||
|
||||
**New shape:**
|
||||
|
||||
- `Dockerfile.gateway` — unchanged
|
||||
- `Dockerfile.orchestrator` — unchanged (single definition of orchestrator
|
||||
content; both Docker infra and Firecracker infra `COPY --from` it)
|
||||
- `Dockerfile.infra` — **shared base**: `FROM bot-bottle-gateway` + `COPY
|
||||
--from bot-bottle-orchestrator` (no buildah — Docker infra image)
|
||||
- `Dockerfile.infra.fc` — Firecracker only: `FROM bot-bottle-infra` +
|
||||
buildah/crun/netavark/aardvark-dns (layered on the shared base, same net
|
||||
result as today)
|
||||
|
||||
The comment in `Dockerfile.infra` that says "the docker backend keeps
|
||||
orchestrator + gateway as separate images; this combined image exists only
|
||||
for the Firecracker single-VM cut" is removed.
|
||||
|
||||
### Orchestrator in the supervise tree
|
||||
|
||||
`gateway_init` already supervises the data-plane daemons (egress, git-http,
|
||||
supervise-MCP). The orchestrator control plane is added as another supervised
|
||||
process: `python3 -m bot_bottle.orchestrator --host 0.0.0.0 --port <port>
|
||||
--broker stub`.
|
||||
|
||||
The orchestrator source is bind-mounted (`/app` → repo root, as today) so
|
||||
dev live-reload still works. `source_hash`-based container recreation in
|
||||
`OrchestratorService.ensure_running` continues to apply — a code change
|
||||
recreates the combined infra container, which bounces both gateway and
|
||||
orchestrator. This is acceptable: the docker backend is a dev/legacy target
|
||||
where in-flight egress connections across a code deploy are not a hard
|
||||
requirement.
|
||||
|
||||
### `OrchestratorService` changes
|
||||
|
||||
`OrchestratorService` currently starts two containers in sequence: gateway
|
||||
first (`DockerGateway.ensure_running`), then orchestrator. After this PRD:
|
||||
|
||||
- Single `docker run` of `bot-bottle-infra:latest`
|
||||
- Container name: `bot-bottle-infra` (replaces `bot-bottle-orch-gateway` +
|
||||
`bot-bottle-orchestrator`)
|
||||
- Published ports: `127.0.0.1:{port}:{port}` for the control plane (same as
|
||||
today)
|
||||
- Bind mounts: repo root + host root (same as today)
|
||||
- `DockerGateway` becomes an implementation detail of `OrchestratorService`
|
||||
rather than a separately started container; the gateway image name
|
||||
(`GATEWAY_IMAGE`) is no longer referenced at runtime, only at build time
|
||||
for the `Dockerfile.infra` base
|
||||
|
||||
The `_gateway()` / `ensure_running` two-step in `OrchestratorService` is
|
||||
replaced by a single `_run_infra_container()`.
|
||||
|
||||
### Shared backend utilities
|
||||
|
||||
Three items are duplicated across
|
||||
`backend/docker/consolidated_launch.py`,
|
||||
`backend/macos_container/consolidated_launch.py`, and
|
||||
`backend/firecracker/consolidated_launch.py`:
|
||||
|
||||
1. **CA cert polling loop** — `deadline = time.monotonic() + timeout; while
|
||||
...: try fetch CA; sleep` — extracted to
|
||||
`backend/consolidated_util.py:poll_ca_cert(transport, *, timeout)`.
|
||||
|
||||
2. **Teardown sequence** — `OrchestratorClient(url).teardown_bottle(id)` +
|
||||
`deprovision_git_gate(transport, id)` — extracted to
|
||||
`backend/consolidated_util.py:teardown_consolidated(url, transport,
|
||||
bottle_id)`.
|
||||
|
||||
3. **Launch skeleton** — all three follow: ensure-infra → allocate/register
|
||||
→ provision git-gate → fetch CA cert → return launch context. The macOS
|
||||
inversion (agent starts before registration, source IP from DHCP) is the
|
||||
only deviation. Extract a shared `_provision_bottle(transport, bottle_id,
|
||||
plan, orchestrator_url)` helper covering the register → provision →
|
||||
return-token steps; the backends keep their own `launch_consolidated`
|
||||
wrappers for the before/after (infra-ensure + agent-start + IP
|
||||
allocation), calling the shared helper.
|
||||
|
||||
The new `backend/consolidated_util.py` module holds only backend-neutral,
|
||||
transport-agnostic logic. All three backends import from it.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **(this PR)** Dockerfile restructuring: rename current `Dockerfile.infra`
|
||||
content to `Dockerfile.infra.fc`; write new `Dockerfile.infra` as
|
||||
gateway+orchestrator base. Update Firecracker image-build references from
|
||||
`Dockerfile.infra` → `Dockerfile.infra.fc`.
|
||||
|
||||
2. Add orchestrator process to `gateway_init` supervise tree.
|
||||
|
||||
3. Collapse `OrchestratorService` to a single-container start; rename
|
||||
container from `bot-bottle-orch-gateway`/`bot-bottle-orchestrator` →
|
||||
`bot-bottle-infra`; update image name constant.
|
||||
|
||||
4. Extract `backend/consolidated_util.py` with `poll_ca_cert`,
|
||||
`teardown_consolidated`, and `_provision_bottle`; update all three
|
||||
`consolidated_launch.py` files to import from it.
|
||||
|
||||
5. Update tests that reference the old container names or two-container
|
||||
startup sequence.
|
||||
|
||||
## Open questions
|
||||
|
||||
None — the supervise-tree approach and shared Dockerfile layering were
|
||||
confirmed in issue #431.
|
||||
Reference in New Issue
Block a user