9 Commits

Author	SHA1	Message	Date
didericis-claude	1a4b390e8a	fix: make installed wheel self-contained + harden install.sh prereqs ... Addresses the review on PR #481. Self-contained wheel (review point 1): the gateway/infra/orchestrator images build from a context that must hold bot_bottle/, pyproject.toml, and the root-level Dockerfiles. Modules previously located these by walking __file__ to the repo root, so an installed wheel (package in site-packages, no repo root) passed `doctor` but failed `start`. - Add bot_bottle/resources.py: build_root() returns the repo root in a checkout (unchanged) or a staged copy from the wheel's bundled _resources/ otherwise; dockerfile()/nix_netpool_module()/ netpool_script() derive from it. - setup.py bundles the root Dockerfiles, nix module, netpool script, and pyproject.toml into bot_bottle/_resources/ at build; MANIFEST.in ships them in the sdist. - Route every _REPO_ROOT/_REPO_DIR call site (docker/macos launch, macos infra, firecracker infra_vm/infra_artifact/setup, orchestrator lifecycle/gateway) through resources. Checkout behavior is unchanged. install.sh prerequisites (review point 2): check for git when installing a git+ spec, and — before the pip fallback — that pip is usable and the interpreter isn't externally managed (PEP 668), pointing at pipx. Tests: test_resources covers checkout + staged-wheel layouts; test_wheel_install builds the wheel, installs it into an isolated venv, and asserts `doctor` runs and build_root() yields a valid context. Running `start` end-to-end still needs a Docker/KVM host (CI). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 22:34:25 -04:00
didericis	e3376b8809	chore(backend): drop dead code left by the InfraService refactor ... Remove symbols with no remaining consumer: - `gateway_name` property on Docker/MacosInfraService — nothing uses it; the launch flow reads `gateway().name` off the Gateway service. (+ its docker test.) - firecracker `infra_vm` EGRESS_PORT / SUPERVISE_PORT / GIT_HTTP_PORT — dead duplicates; launch.py uses supervisor.types / docker.egress / a local const. - macOS `INFRA_DB_VOLUME` back-compat alias — unused since the DB-volume test moved to `ORCHESTRATOR_DB_VOLUME`. - docker `infra.py`: the unused `ORCHESTRATOR_SOURCE_HASH_LABEL` import and the `__all__` re-exports of the orchestrator constants (consumers import them from `docker.orchestrator`, not `docker.infra`). - macOS `infra.py`: the unused `GatewayError` import + re-export. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 16:47:18 -04:00
didericis	afb92ca155	refactor(backend): InfraService ABC + FirecrackerInfraService ... Add a backend-neutral `InfraService` ABC (backend/infra_service.py) so all three backends present the same composer contract: `orchestrator()` / `gateway()` accessors, `ensure_running(*, startup_timeout) -> str` (the host control-plane URL), `stop()`, plus concrete `url()` / `is_healthy()` that delegate to the orchestrator. `DockerInfraService` and `MacosInfraService` now subclass it. Give firecracker the missing class: `FirecrackerInfraService` (backend/firecracker/infra.py) wraps the `infra_vm` substrate as the pair coordinator (adopt-or-boot-both under the singleton flock). `infra_vm` keeps only the plane-agnostic substrate; its `ensure_running` / `_adopt` / `InfraEndpoint` move to the class, and the coordinator helpers it now calls cross-module go public (`singleton_lock` / `expected_version` / `adoptable` / `record_booted_version`). Unify the return type on the way: `ensure_running` returns the URL string everywhere (was `str` for docker, two different `InfraEndpoint`s for macOS/firecracker), and those `InfraEndpoint`s are deleted — the services are the source of truth (callers read `gateway().address()` / `orchestrator().url()`). docker's `url` property becomes the inherited `url()`. backend.py / consolidated_launch / image_builder + tests updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 16:10:21 -04:00
didericis	cc4e29c3da	refactor(orchestrator): introduce the Orchestrator service ABC (docker impl) ... Mirror the Gateway work for the control plane. Add the backend-neutral `Orchestrator` service ABC in orchestrator/lifecycle.py — build the image/rootfs, `ensure_running` (start + health-gate), `is_running`/`is_healthy`/`stop`, `url()` (host-facing) vs `gateway_url()` (what the gateway resolves against), and a concrete `mint_gateway_token()` (the orchestrator holds the signing key, so it issues the gateway's token — #469). `DockerOrchestrator` (backend/docker/orchestrator.py) is the first impl, extracted out of `DockerInfraService`: the control-plane container run, the `--internal` control network, the source-hash recreate gate, health polling, and the orchestrator constants (name/label/network/image/dockerfile/hash-label). `DockerInfraService` now composes `orchestrator()` + `gateway()` — each builds its own image via `ensure_built`, the orchestrator comes up first, then the gateway is connected with the orchestrator-minted token. Its `url`/`is_healthy` delegate to the orchestrator service. Split the container-lifecycle tests into test_docker_orchestrator; test_docker_infra now covers the composition. macOS + firecracker follow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 14:36:35 -04:00
didericis	05df21f210	refactor(gateway): introduce the Gateway service ABC (docker impl) ... Replace the docker backend's ad-hoc gateway wiring with the shared `Gateway` service ABC (in the gateway package), the first of the two per-host service classes that supersede the per-backend infra glue. The contract is backend-neutral: `connect_to_orchestrator(url, gateway_token)` binds the gateway to its control plane and brings it up, `address()` reports the agent-facing proxy target, `ca_cert_pem()` vends the mitmproxy CA, and `provisioning_transport()` hands back the exec/cp seam git-gate provisioning stages per-bottle repos + deploy keys through. `GatewayProvisionError` + `GatewayTransport` move to the ABC so every backend shares them. Key trust-model change: the gateway no longer mints its own token. It never holds the signing key (#469), so the orchestrator mints the role-scoped `gateway` JWT and hands it in via `connect_to_orchestrator`; the gateway only injects it. `DockerGateway.ensure_running` becomes `connect_to_orchestrator` (stashing the URL + token as instance state), and the docker launch flow reads `address()` / `provisioning_transport()` off the service rather than re-deriving the container IP + transport itself. macOS + firecracker gateways move under the ABC in following commits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 12:58:00 -04:00
didericis	d62d19a2e7	feat(docker): split orchestrator and gateway into separate containers (PRD 0070) ... Now that #469 got the DB off the data plane, the docker backend runs the control plane and data plane as two containers instead of the combined `bot-bottle-infra` container: * bot-bottle-orchestrator — the lean control-plane container (image Dockerfile.orchestrator, `-m bot_bottle.orchestrator`). Joins the new `bot-bottle-orchestrator` control network (`--internal`) only, plus a host-loopback publish for the CLI. Sole opener of bot-bottle.db; holds the signing key; no CA, no gateway daemons. * bot-bottle-orch-gateway — the data-plane container (DockerGateway), dual-homed on the agent `bot-bottle-gateway` network AND the control network, so it resolves the orchestrator by name (http://bot-bottle-orchestrator:8099) while agents — never on the control network — have no route to the control plane (the L3 block, not just the JWT). Holds the gateway JWT + the mitmproxy CA. DockerInfraService now orchestrates the pair (builds gateway + orchestrator images, brings up the orchestrator then the gateway) and exposes gateway_name; the launch flow attributes agents against the gateway container. DockerGateway gains a control_network it joins after run. rotate_ca / the CA read target the gateway container. The combined Dockerfile.infra is no longer built by docker (firecracker/macOS still use it until their splits). pyright 0 errors; unit suite green (2273). Integration tests updated for the two-container shape but need a real-docker run to validate the networking. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-25 02:32:05 -04:00
didericis	ca1d341d4f	refactor: unify component naming — control_plane/control_auth -> orchestrator ... The codebase used "control plane" both as an architectural role term AND as an identifier alias for the orchestrator component, producing duplicate names for one thing (control_plane_url vs orchestrator_url, CONTROL_PLANE_PORT, host_control_plane_token, …). Going forward the concrete component is always named for what it is — Gateway or Orchestrator — and the plane vocabulary is reserved for prose (module descriptions, the security argument). Renamed (identifiers + the in-repo env/wire/file string values, all setters/getters are in this repo so the change is atomic): ControlPlaneServer -> OrchestratorServer control_plane_url -> orchestrator_url probe_control_plane_url -> probe_orchestrator_url host_control_plane_token -> host_orchestrator_token CONTROL_PLANE_PORT -> ORCHESTRATOR_PORT CONTROL_PLANE_TOKEN_ENV/FILE -> ORCHESTRATOR_TOKEN_ENV/FILENAME BOT_BOTTLE_CONTROL_PLANE_TOKEN-> BOT_BOTTLE_ORCHESTRATOR_TOKEN control-plane-token (file) -> orchestrator-token control_auth (module) -> orchestrator_auth (stays top-level; the gateway imports it and must not import the orchestrator/ package) CONTROL_AUTH_HEADER -> ORCHESTRATOR_AUTH_HEADER x-bot-bottle-control-auth -> x-bot-bottle-orchestrator-auth CONTROL_AUTH_JWT_ENV -> ORCHESTRATOR_AUTH_JWT_ENV BOT_BOTTLE_CONTROL_AUTH_JWT -> BOT_BOTTLE_ORCHESTRATOR_AUTH_JWT _control_auth_headers -> _orchestrator_auth_headers Prose plane-terms ("control plane", "data plane") are preserved, including the test name test_data_plane_daemons_get_jwt_not_key (it names the security invariant). Gateway and orchestrator verified to agree on the renamed wire header; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:17:48 -04:00
didericis	eab9d15130	refactor(docker): fold run_docker into backend/docker/util; drop the docker_cmd shim ... docker_cmd.py existed as a top-level module solely so the orchestrator's docker components could share run_docker without dragging the (then-heavy) backend layer in. Now that backend/__init__ and backend/docker/__init__ are thin, importing backend.docker.util costs 6 modules instead of 76, so the shim's whole reason to exist is gone. Move run_docker into backend/docker/util.py (where the other docker subprocess primitives live) and delete docker_cmd.py. Backend siblings import it via `from .util import run_docker`; the two docker-specific orchestrator modules (docker_broker, rotate_ca) import `from ..backend.docker.util import run_docker`. No import cycle (backend.docker.util pulls nothing from orchestrator); orchestrator.__main__ stays lean at 28 modules. run_docker patch targets are unchanged (tests patch it in the importing module). Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:58:23 -04:00
didericis	b96a8b44e0	refactor(docker): rename OrchestratorService -> DockerInfraService, move to backend/docker ... OrchestratorService wasn't the orchestrator — it's the host-side lifecycle of the docker infra *container* (the one that runs the Orchestrator). It read as "the orchestrator as a service" and lived in orchestrator/lifecycle.py, while its siblings (macOS MacosInfraService, Firecracker infra_vm) live under their backend package. Rename it DockerInfraService and move it to backend/docker/infra.py alongside the docker backend, with its docker-only constants (INFRA_*/ORCHESTRATOR_* image + container names, daemon list, mount paths). orchestrator/lifecycle.py keeps only the backend-neutral pieces the other infra services share — DEFAULT_PORT, DEFAULT_STARTUP_TIMEOUT_SECONDS, OrchestratorStartError, source_hash — which macOS / firecracker / client still import from there. backend/docker/infra.py imports those (backend -> orchestrator is an allowed direction). Renamed the unit test to test_docker_infra.py. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:26:52 -04:00