didericis-claude
8e43c26ab4
Extract the shared CA cert polling loop into `backend/util.poll_ca_cert`
(firecracker and macos backends were duplicating deadline/sleep/raise logic).
Each caller now wraps a fetch lambda and converts TimeoutError to its own
error type. Also corrects the PRD port publication line from {port}:{port}
to {host_port}:8099.
157 lines
6.8 KiB
Markdown
157 lines
6.8 KiB
Markdown
# PRD prd-new: Consolidate infra backend for Docker
|
|
|
|
- **Status:** Active
|
|
- **Author:** Claude
|
|
- **Created:** 2026-07-20
|
|
- **Issue:** #431
|
|
|
|
## Summary
|
|
|
|
The Docker backend runs two containers — `bot-bottle-orch-gateway` (gateway
|
|
data plane) and `bot-bottle-orchestrator` (control plane) — where the
|
|
macOS and Firecracker backends already run a single combined infra
|
|
unit. This PRD collapses Docker to the same model: one `bot-bottle-infra`
|
|
container running both processes under the `gateway_init` supervise tree, a
|
|
restructured `Dockerfile.infra` as the shared gateway+orchestrator base,
|
|
and a handful of extracted shared utilities (CA cert polling, teardown
|
|
sequence, launch skeleton) that are currently duplicated across all three
|
|
`consolidated_launch.py` files.
|
|
|
|
## Goals / success criteria
|
|
|
|
- Docker backend starts exactly one infra container instead of two.
|
|
- `Dockerfile.infra` is the shared base image (gateway + orchestrator, no
|
|
buildah); the Firecracker image layers buildah on top of it.
|
|
- The orchestrator process runs under the `gateway_init` supervise tree
|
|
inside the combined container (one PID-1, one restart/health surface).
|
|
- CA cert polling, the teardown sequence, and the shared launch skeleton
|
|
(ensure-infra → register → provision → return context) live in a single
|
|
shared module; all three backends import from it.
|
|
- No functional change to macOS or Firecracker launch paths.
|
|
|
|
## Non-goals
|
|
|
|
- Changing per-bottle isolation — agents stay one-VM/container-each.
|
|
- Consolidating transport implementations (`DockerGatewayTransport`,
|
|
`AppleGatewayTransport`, `SshGatewayTransport`) — these are already the
|
|
right abstraction boundary.
|
|
- macOS DHCP-inversion of registration order — irreducible backend
|
|
difference, stays as-is.
|
|
- Any changes to the orchestrator RPC protocol or the attribution model.
|
|
|
|
## Design
|
|
|
|
### Dockerfile restructuring
|
|
|
|
**Current shape:**
|
|
|
|
- `Dockerfile.gateway` — data plane (mitmproxy, gitleaks, git, openssh,
|
|
supervise daemons)
|
|
- `Dockerfile.orchestrator` — control plane (python:3.12-slim + bot_bottle
|
|
package; stdlib-only, no third-party deps)
|
|
- `Dockerfile.infra` — Firecracker only: `FROM bot-bottle-gateway` +
|
|
buildah + `COPY --from bot-bottle-orchestrator`
|
|
|
|
**New shape:**
|
|
|
|
- `Dockerfile.gateway` — unchanged
|
|
- `Dockerfile.orchestrator` — unchanged (single definition of orchestrator
|
|
content; both Docker infra and Firecracker infra `COPY --from` it)
|
|
- `Dockerfile.infra` — **shared base**: `FROM bot-bottle-gateway` + `COPY
|
|
--from bot-bottle-orchestrator` (no buildah — Docker infra image)
|
|
- `Dockerfile.infra.fc` — Firecracker only: `FROM bot-bottle-infra` +
|
|
buildah/crun/netavark/aardvark-dns (layered on the shared base, same net
|
|
result as today)
|
|
|
|
The comment in `Dockerfile.infra` that says "the docker backend keeps
|
|
orchestrator + gateway as separate images; this combined image exists only
|
|
for the Firecracker single-VM cut" is removed.
|
|
|
|
### Orchestrator in the supervise tree
|
|
|
|
`gateway_init` already supervises the data-plane daemons (egress, git-http,
|
|
supervise-MCP). The orchestrator control plane is added as another supervised
|
|
process: `python3 -m bot_bottle.orchestrator --host 0.0.0.0 --port <port>
|
|
--broker stub`.
|
|
|
|
The orchestrator source is bind-mounted (`/app` → repo root, as today) so
|
|
dev live-reload still works. `source_hash`-based container recreation in
|
|
`OrchestratorService.ensure_running` continues to apply — a code change
|
|
recreates the combined infra container, which bounces both gateway and
|
|
orchestrator. This is acceptable: the docker backend is a dev/legacy target
|
|
where in-flight egress connections across a code deploy are not a hard
|
|
requirement.
|
|
|
|
### `OrchestratorService` changes
|
|
|
|
`OrchestratorService` currently starts two containers in sequence: gateway
|
|
first (`DockerGateway.ensure_running`), then orchestrator. After this PRD:
|
|
|
|
- Single `docker run` of `bot-bottle-infra:latest`
|
|
- Container name: `bot-bottle-infra` (replaces `bot-bottle-orch-gateway` +
|
|
`bot-bottle-orchestrator`)
|
|
- Published ports: `127.0.0.1:{host_port}:8099` for the control plane
|
|
(`gateway_init` listens on a fixed internal port 8099; the caller-chosen
|
|
host port maps to it)
|
|
- Bind mounts: repo root + host root (same as today)
|
|
- `DockerGateway` becomes an implementation detail of `OrchestratorService`
|
|
rather than a separately started container; the gateway image name
|
|
(`GATEWAY_IMAGE`) is no longer referenced at runtime, only at build time
|
|
for the `Dockerfile.infra` base
|
|
|
|
The `_gateway()` / `ensure_running` two-step in `OrchestratorService` is
|
|
replaced by a single `_run_infra_container()`.
|
|
|
|
### Shared backend utilities
|
|
|
|
Three items are duplicated across
|
|
`backend/docker/consolidated_launch.py`,
|
|
`backend/macos_container/consolidated_launch.py`, and
|
|
`backend/firecracker/consolidated_launch.py`:
|
|
|
|
1. **CA cert polling loop** — `deadline = time.monotonic() + timeout; while
|
|
...: try fetch CA; sleep` — extracted to
|
|
`backend/consolidated_util.py:poll_ca_cert(transport, *, timeout)`.
|
|
|
|
2. **Teardown sequence** — `OrchestratorClient(url).teardown_bottle(id)` +
|
|
`deprovision_git_gate(transport, id)` — extracted to
|
|
`backend/consolidated_util.py:teardown_consolidated(url, transport,
|
|
bottle_id)`.
|
|
|
|
3. **Launch skeleton** — all three follow: ensure-infra → allocate/register
|
|
→ provision git-gate → fetch CA cert → return launch context. The macOS
|
|
inversion (agent starts before registration, source IP from DHCP) is the
|
|
only deviation. Extract a shared `_provision_bottle(transport, bottle_id,
|
|
plan, orchestrator_url)` helper covering the register → provision →
|
|
return-token steps; the backends keep their own `launch_consolidated`
|
|
wrappers for the before/after (infra-ensure + agent-start + IP
|
|
allocation), calling the shared helper.
|
|
|
|
The new `backend/consolidated_util.py` module holds only backend-neutral,
|
|
transport-agnostic logic. All three backends import from it.
|
|
|
|
## Implementation chunks
|
|
|
|
1. **(this PR)** Dockerfile restructuring: rename current `Dockerfile.infra`
|
|
content to `Dockerfile.infra.fc`; write new `Dockerfile.infra` as
|
|
gateway+orchestrator base. Update Firecracker image-build references from
|
|
`Dockerfile.infra` → `Dockerfile.infra.fc`.
|
|
|
|
2. Add orchestrator process to `gateway_init` supervise tree.
|
|
|
|
3. Collapse `OrchestratorService` to a single-container start; rename
|
|
container from `bot-bottle-orch-gateway`/`bot-bottle-orchestrator` →
|
|
`bot-bottle-infra`; update image name constant.
|
|
|
|
4. Extract `backend/consolidated_util.py` with `poll_ca_cert`,
|
|
`teardown_consolidated`, and `_provision_bottle`; update all three
|
|
`consolidated_launch.py` files to import from it.
|
|
|
|
5. Update tests that reference the old container names or two-container
|
|
startup sequence.
|
|
|
|
## Open questions
|
|
|
|
None — the supervise-tree approach and shared Dockerfile layering were
|
|
confirmed in issue #431.
|