Addresses the third review round on PR #481.
- `bot-bottle doctor` now checks `is_backend_ready()` (a full backend
status() probe: daemon reachable, network pool present, KVM usable)
instead of the cheap PATH-only `is_backend_available()`. A host with a
stopped Docker daemon or half-configured Firecracker no longer reports
`ok: backend` / exit 0 when `start` can't actually work; each not-ready
backend prints its own diagnostics, and doctor passes only if at least
one backend is ready.
- `install.sh` resolves the pip `--user` scripts directory from the
interpreter (`sysconfig.get_path("scripts", get_preferred_scheme("user"))`)
instead of hardcoding `~/.local/bin`, which is wrong on a python.org
macOS interpreter (`~/Library/Python/<X.Y>/bin`). The PATH guidance now
prints the actual directory.
Tests: doctor tests mock `is_backend_ready` (the readiness contract) and
cover the not-ready → fail path; a new install-script test drives the
macOS `osx_framework_user` scheme and asserts it resolves a
non-~/.local/bin directory.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the review on PR #481.
Self-contained wheel (review point 1): the gateway/infra/orchestrator
images build from a context that must hold bot_bottle/, pyproject.toml,
and the root-level Dockerfiles. Modules previously located these by
walking __file__ to the repo root, so an installed wheel (package in
site-packages, no repo root) passed `doctor` but failed `start`.
- Add bot_bottle/resources.py: build_root() returns the repo root in a
checkout (unchanged) or a staged copy from the wheel's bundled
_resources/ otherwise; dockerfile()/nix_netpool_module()/
netpool_script() derive from it.
- setup.py bundles the root Dockerfiles, nix module, netpool script, and
pyproject.toml into bot_bottle/_resources/ at build; MANIFEST.in ships
them in the sdist.
- Route every _REPO_ROOT/_REPO_DIR call site (docker/macos launch, macos
infra, firecracker infra_vm/infra_artifact/setup, orchestrator
lifecycle/gateway) through resources. Checkout behavior is unchanged.
install.sh prerequisites (review point 2): check for git when installing
a git+ spec, and — before the pip fallback — that pip is usable and the
interpreter isn't externally managed (PEP 668), pointing at pipx.
Tests: test_resources covers checkout + staged-wheel layouts;
test_wheel_install builds the wheel, installs it into an isolated venv,
and asserts `doctor` runs and build_root() yields a valid context.
Running `start` end-to-end still needs a Docker/KVM host (CI).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Give bot-bottle a real distribution path so new users can install
without cloning the repo:
- pyproject.toml: full project metadata, a `bot-bottle` console-script
entry point (bot_bottle.cli:main), and package-data for the runtime
assets (Dockerfiles, egress entrypoint, netpool defaults, macos init).
Still zero runtime pip dependencies.
- install.sh: POSIX, sudo-free, idempotent bootstrapper — checks Python
>= 3.11, creates ~/.bot-bottle/{agents,bottles,contrib}, installs via
pipx (pip --user fallback), then runs `bot-bottle doctor`.
- `bot-bottle doctor`: new store-free subcommand reporting Python
version, backend availability (reuses is_backend_available rather than
hardcoding Docker), and config-dir presence. Exits non-zero when a hard
prerequisite is unmet.
- PRD prd-new-install-script and unit tests for doctor, the packaging
contract, and the install script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the empty-key opt-out codex flagged: ControlPlaneProvisioning
no longer lets the orchestrator start without a signing key when a
backend declares an "isolated" topology. Host separation is not the
safety condition — the gateway (and any other caller) must reach the
control-plane listener for /resolve, so an open orchestrator would
still grant them full `cli`. The signing key is now mandatory for
every backend.
Drops the now-purposeless Topology/COLOCATED machinery (no backend
declared a non-default topology) and the contractual
test_orchestrator_key_allows_empty_when_isolated. Updates the PRD's
invariant 4 and lifecycle docstring accordingly. Docs/behaviour only
otherwise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hoist control-plane auth provisioning out of the per-backend launchers into
one shared contract, parameterized per trust domain (#476). Every blocking
finding in PR #471 was the same integration-bug class: each launcher
re-derived, by hand, how to generate the signing key, scope it to the
orchestrator, mint the gateway JWT, and keep the host key canonical.
Introduces `trust_domain.py`:
* `TrustDomain` — one credential boundary (host-canonical key file + role
set + env vars). `mint`/`verify` are scoped to the domain's roles, so a
future host-controller domain (#468) uses its own key/verifier/roles
rather than a `host` role on the control plane's frozenset (which the
orchestrator key could then forge).
* `ControlPlaneProvisioning` — the single seam answering the four
invariants: host-canonical key, split key-vs-token credential, CLI token
valid across co-running backends, and fail-closed (no open mode) for any
co-located topology.
* `Topology` — the backend declares what it is; the default is co-located +
fail-closed, so a backend need not redeclare it.
The `Orchestrator` ABC gets `control_plane_key()` (fail-closed) and routes
`mint_gateway_token()` through the contract; docker/macOS/firecracker
orchestrators, the server (verify), and the host CLI client (mint cli) all go
through the domain instead of reading the host key directly. `orchestrator_auth`
gains an optional `roles=` arg (default unchanged) so a domain scopes its own
role set; `paths.host_signing_key(filename)` generalizes host_orchestrator_token.
Adds unit coverage for the domain boundary + provisioning invariants and a PRD
capturing the durable rationale. No change to the auth primitive's HMAC, the
plane split, or the server's documented open-mode fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes#476
`_supervise_token_block` and `_await_token_response` called the synchronous
`PolicyResolver` directly from mitmproxy's async request handling. Those RPCs
funnel through `PolicyResolver._post_json`, which blocks on
`urllib.request.urlopen`. Because the poll loop runs for the entire
operator-approval window, a slow or unreachable orchestrator would repeatedly
freeze the proxy event loop — stalling every other bottle's traffic — despite
`_await_token_response`'s contract of not blocking the loop (#471 review,
review #443).
Dispatch both the propose and poll RPCs via `asyncio.to_thread` so the
blocking urllib call runs in a worker thread and the event loop keeps serving
other flows. Fail-closed semantics are unchanged: `PolicyResolveError` still
propagates through the await and is caught exactly as before.
Regression test records the thread each RPC executes on and asserts it is not
the loop thread; it fails against the pre-fix inline calls and passes with the
dispatch. Full egress-addon + policy-resolver + supervise-server suites green
(155 tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous fix interpolated the real per-bottle identity token into the
`warn()` message shown when `claude mcp add supervise` fails, so a
registration failure would leak the credential into host terminal output
and any collected launch logs (#476 review, PR #471 comment).
Render a `<bottle-identity-token>` placeholder instead. The recovery hint
still shows the required `--header x-bot-bottle-identity: …` shape, and the
operator substitutes the value from inside the bottle (it rides in the
agent's HTTPS_PROXY credentials) — so the token never reaches the host log.
The token truthiness still gates whether the header is shown at all.
Test updated to assert the header name and placeholder are present and the
raw token is absent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The manual registration command shown in the fallback warning when
`claude mcp add supervise` fails was missing the `--header
x-bot-bottle-identity: <token>` option. Since (source_ip,
identity_token) attribution is now mandatory on the supervise server,
the missing header caused every subsequent supervise request to fail
closed as unattributed — defeating supervision for the entire bottle.
Reuse the already-constructed `token` variable to build the same
`manual_header` string the automatic path uses; include it in the
printed command. Tests verify both the happy path and the fallback
include (or correctly omit) the header.
Addresses P2 finding in #471 (comment 7, didericis-codex 2026-07-25).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove symbols with no remaining consumer:
- `gateway_name` property on Docker/MacosInfraService — nothing uses it; the
launch flow reads `gateway().name` off the Gateway service. (+ its docker test.)
- firecracker `infra_vm` EGRESS_PORT / SUPERVISE_PORT / GIT_HTTP_PORT — dead
duplicates; launch.py uses supervisor.types / docker.egress / a local const.
- macOS `INFRA_DB_VOLUME` back-compat alias — unused since the DB-volume test
moved to `ORCHESTRATOR_DB_VOLUME`.
- docker `infra.py`: the unused `ORCHESTRATOR_SOURCE_HASH_LABEL` import and the
`__all__` re-exports of the orchestrator constants (consumers import them from
`docker.orchestrator`, not `docker.infra`).
- macOS `infra.py`: the unused `GatewayError` import + re-export.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a backend-neutral `InfraService` ABC (backend/infra_service.py) so all three
backends present the same composer contract: `orchestrator()` / `gateway()`
accessors, `ensure_running(*, startup_timeout) -> str` (the host control-plane
URL), `stop()`, plus concrete `url()` / `is_healthy()` that delegate to the
orchestrator. `DockerInfraService` and `MacosInfraService` now subclass it.
Give firecracker the missing class: `FirecrackerInfraService`
(backend/firecracker/infra.py) wraps the `infra_vm` substrate as the pair
coordinator (adopt-or-boot-both under the singleton flock). `infra_vm` keeps only
the plane-agnostic substrate; its `ensure_running` / `_adopt` / `InfraEndpoint`
move to the class, and the coordinator helpers it now calls cross-module go
public (`singleton_lock` / `expected_version` / `adoptable` /
`record_booted_version`).
Unify the return type on the way: `ensure_running` returns the URL string
everywhere (was `str` for docker, two different `InfraEndpoint`s for
macOS/firecracker), and those `InfraEndpoint`s are deleted — the services are the
source of truth (callers read `gateway().address()` / `orchestrator().url()`).
docker's `url` property becomes the inherited `url()`. backend.py /
consolidated_launch / image_builder + tests updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sweep for vestiges of the old combined-plane model and the pre-split shared
rootfs. Two are load-bearing, the rest are stale docs/comments:
- Bug: macOS `enumerate_active` only excluded the gateway container from the
agent list, so after the split the orchestrator container
(`bot-bottle-mac-orchestrator`, also `bot-bottle-`-prefixed) was enumerated as
a phantom agent. Exclude both infra containers; test covers it.
- Dead code: the gateway `bootstrap.py` still carried an `orchestrator` daemon
spec + `_OPT_IN_DAEMONS` + a signing-key/JWT env branch, all for the old
combined container where the gateway process could also run the control plane.
No backend ever requests it now — removed; the key-stripping stays as
defense-in-depth.
Stale-comment reframes: "the/single infra container" -> the orchestrator +
gateway pair (or the specific plane); "shared rootfs / bb_role init / one
published rootfs" -> the per-plane rootfs + `role_init`; the deleted
Dockerfile.infra references in Dockerfile.orchestrator/.gateway; and the macOS
"one infra container ... same address" docstring + its now-false
share-one-address test (the planes are distinct containers with distinct
addresses).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each infra VM now boots its own rootfs instead of a shared combined image, so
the exposed gateway VM no longer carries buildah + the control-plane code it
never runs (a fatter, less-isolated exposed surface — the opposite of the plane
split's intent). The combined image bought only "one artifact"; each VM already
kept a full copy of the shared rootfs, so nothing was saved at boot.
* orchestrator rootfs — control plane + buildah (Dockerfile.orchestrator.fc,
FROM orchestrator); the slim gateway rootfs boots bot-bottle-gateway:latest
directly. Dockerfile.infra.fc (the combined image) is deleted.
* `_infra_init` splits into `_orchestrator_init` / `_gateway_init` (shared
preamble via `_init_head`); each per-plane rootfs bakes only its role init,
so the `bb_role` cmdline branch is gone.
* infra_artifact is role-parametrized: a per-role package
(bot-bottle-firecracker-<role>), version hash, URL, cache dir, and candidate
subdir. `ensure_built` pulls both; `_expected_version` combines both markers.
* publish_infra builds the images once, then builds + publishes an
orchestrator and a gateway artifact under DIR/<role>/.
coverage.sh's candidate-dir plumbing is layout-agnostic (publish_infra --output
now fills DIR/<role>/, which ensure_artifact_gz reads). Full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull the control plane's host-side logic out of `infra_vm` into
`FirecrackerOrchestrator` (backend/firecracker/orchestrator.py): booting the
orchestrator microVM on its link with the persistent registry volume (/dev/vdb),
seeding the host-canonical signing key over SSH, waiting for /health, and the
buildah SSH target. `url()` == `gateway_url()` (the gateway resolves the
orchestrator by guest IP via bb_orch). This completes the trio — docker, macOS,
firecracker — behind both the Gateway and Orchestrator ABCs.
`infra_vm` stays the pair coordinator + shared substrate: `ensure_running` now
mints nothing itself — it constructs both services (lazy import to break the
cycle), calls `orchestrator.ensure_running()` first, then
`gateway.connect_to_orchestrator(orch.gateway_url(), orch.mint_gateway_token())`.
The plane-agnostic boot primitives graduate to public API (`_boot_vm`/
`_push_secret` -> `boot_vm`/`push_secret`) now that they're consumed only across
modules; `InfraVm` loses its `orchestrator_url` (moved to the service) and
`InfraEndpoint.orchestrator` is now the `FirecrackerOrchestrator` service.
Container-lifecycle tests split into test_firecracker_orchestrator; the infra_vm
tests keep the substrate + pair-coordinator coverage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the Apple-Container control-plane lifecycle out of `MacosInfraService`
into `MacosOrchestrator` (backend/macos_container/orchestrator.py): the
container run on the host-only control network, the container-only DB volume,
the source-hash recreate gate, health polling against the resolved
control-network address, and `probe_orchestrator_url`. `url()` == `gateway_url()`
here (Apple has no container DNS, so one resolved address serves the CLI and the
gateway).
`MacosInfraService` now composes `orchestrator()` + `gateway()`: ensure the
networks, build both images (each service self-builds via `ensure_built` — added
to `MacosGateway` too), bring the orchestrator up first, then connect the gateway
with the orchestrator-minted token. `ORCHESTRATOR_IMAGE` + the DB-volume constant
move to the orchestrator module (with back-compat aliases where imported).
Container-lifecycle tests split into test_macos_orchestrator; test_macos_infra
now covers the composition.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The two-VM stop() only knew the new orchestrator/gateway config paths
(<infra>/{orchestrator,gateway}/config.json), so a host still running the
OLD single combined-VM (booted from <infra>/config.json on the orchestrator
link) kept it alive across the migration — and it held the orchestrator TAP,
so the first split boot died with "Open tap device failed: Resource busy"
(seen on the CI runner's bborchci0 pool).
stop() now also reaps that legacy singleton (both its drifted pidfile and any
firecracker whose --config-file is the legacy path), so the cutover is
self-healing with no manual per-host teardown. Scoped to the legacy infra
config path, so pool agent VMs are untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirror the Gateway work for the control plane. Add the backend-neutral
`Orchestrator` service ABC in orchestrator/lifecycle.py — build the image/rootfs,
`ensure_running` (start + health-gate), `is_running`/`is_healthy`/`stop`,
`url()` (host-facing) vs `gateway_url()` (what the gateway resolves against),
and a concrete `mint_gateway_token()` (the orchestrator holds the signing key,
so it issues the gateway's token — #469).
`DockerOrchestrator` (backend/docker/orchestrator.py) is the first impl,
extracted out of `DockerInfraService`: the control-plane container run, the
`--internal` control network, the source-hash recreate gate, health polling,
and the orchestrator constants (name/label/network/image/dockerfile/hash-label).
`DockerInfraService` now composes `orchestrator()` + `gateway()` — each builds
its own image via `ensure_built`, the orchestrator comes up first, then the
gateway is connected with the orchestrator-minted token. Its `url`/`is_healthy`
delegate to the orchestrator service.
Split the container-lifecycle tests into test_docker_orchestrator; test_docker_infra
now covers the composition. macOS + firecracker follow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Free the `Orchestrator` name for the incoming host-side control-plane service
ABC (parallel to `Gateway`). The in-guest control-plane core (registry + broker,
in orchestrator/service.py) becomes `OrchestratorCore`; server.py, __main__.py,
the lazy facade, and the two service/server tests updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the module files to match their functions' verb-first names:
`bottle_provision.py` -> `provision_bottle.py`,
`gateway_provision.py` -> `provision_gateway.py` (and the test file to match).
Imports, docstrings, and the git_gate/service.py pointer updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The backends' public teardown entry point now pairs with `launch_consolidated`
under the provision/deprovision verb used everywhere else
(`deprovision_bottle`, `deprovision_git_gate`). Renamed across all three
backends' consolidated_launch + launch modules and the tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename `consolidated_util` -> `bottle_provision` and align it with
`gateway_provision` as the two provisioning domains: `bottle_provision` owns
the bottle lifecycle (`provision_bottle` + `deprovision_bottle`, the latter
renamed from `teardown_consolidated`), `gateway_provision` owns the git-gate
state pushed through the transport (`provision_git_gate`/`deprovision_git_gate`).
The three backends' consolidated_launch modules import from the new module;
their own public `teardown_consolidated` wrappers (called by launch.py) keep
their names.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`backend/docker/gateway_provision.py` had no docker-specific code left once
`DockerGatewayTransport` moved out — `provision_git_gate` / `deprovision_git_gate`
drive any `GatewayTransport`, and the guest-side paths they write
(`/git-gate/creds/<id>`, `/git/<id>`, `/etc/git-gate/...`) are identical inside
every backend's gateway. Yet the neutral `backend/consolidated_util.py` reached
into the docker package to import them.
Move it to `backend/gateway_provision.py`, a sibling of its only importer. The
gateway *package* can't host it (git_gate already imports gateway.git_gate,
so gateway importing git_gate would cycle), but the backend layer uses git_gate
freely. Docstrings + the git_gate/service.py pointer updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Give every backend a `gateway_transport.py` holding just its `GatewayTransport`
impl, named consistently with the backend: `DockerGatewayTransport`,
`MacosGatewayTransport` (was `AppleGatewayTransport`), and
`FirecrackerGatewayTransport` (was `SshGatewayTransport`).
- docker: split `DockerGatewayTransport` out of `gateway_provision.py`, which
keeps only the backend-neutral provisioning logic (provision_git_gate /
deprovision_git_gate) the other backends share.
- macOS: `gateway_provision.py` held only the transport, so it's replaced by
`gateway_transport.py`.
- firecracker: move the transport out of `gateway.py`.
Behaviour-preserving; importers + tests updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull the gateway's host-side logic out of `infra_vm` and into
`FirecrackerGateway`'s own methods, so the gateway is self-contained and
`infra_vm` shrinks toward being just the pair coordinator (a step toward
removing it). Now living in the gateway service: the gateway VM boot +
`bb_orch` cmdline, the pre-minted JWT push, the mitmproxy CA fetch over SSH,
the `SshGatewayTransport` exec/cp, and the lifecycle predicates
(is_running/stop/address). `ensure_running` / `_adopt` construct the gateway
and call `connect_to_orchestrator` (lazy import to break the cycle); the
InfraEndpoint's gateway is now the `FirecrackerGateway` service.
What deliberately stays shared in `infra_vm` (documented at the top of
gateway.py): the plane-agnostic VM substrate the orchestrator VM also needs
(`_boot_vm`, the stable SSH keypair, the secret-push retry, PID lifecycle), the
pair coordinator (`ensure_running`: orchestrator-first health gate + singleton
lock + adoption/version marker), and the single shared `bb_role`-branched init
baked into the one published rootfs both VMs boot — the guest-side gateway
startup can't live in a host method. These are the seam that moves to a neutral
module when the Orchestrator service lands and `infra_vm` dissolves.
CA fetch now raises GatewayError (was die/SystemExit) to match the ABC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `FirecrackerGateway`, the Firecracker implementation of the shared
`Gateway` service — completing the trio (docker, macOS, firecracker) behind one
uniform contract. The gateway here is a whole microVM, so the adapter composes
`infra_vm`'s VM primitives (boot, SSH secret push, netpool links) into the ABC
surface: `connect_to_orchestrator(url, token)` parses the orchestrator guest IP
off the URL and boots the gateway VM seeding the pre-minted token; `address()`
is the gateway link's guest IP; `ca_cert_pem()` fetches over SSH (adapting the
running VM when this process didn't boot it); `provisioning_transport()` is the
SSH exec/cp transport.
Trust-model alignment with the other backends: minting moves out of
`_push_gateway_jwt` to the host composition point — `ensure_running` mints the
role-scoped `gateway` JWT and threads it through `boot_gateway(orch_ip, token)`,
so the push step only writes the token the gateway presents (#469).
`infra_vm` keeps driving the orchestrator+gateway pair as a singleton and gains
gateway-only helpers (`gateway_running`/`stop_gateway`/`gateway_guest_ip`/
`adopted_gateway_vm`); the consolidated launch reads the gateway's transport +
CA off the service.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the Apple-Container gateway data plane out of `MacosInfraService` into
`MacosGateway`, the macOS implementation of the shared `Gateway` service.
`connect_to_orchestrator(url, gateway_token)` runs the triple-homed gateway
container (egress + agent + control networks) carrying the mitmproxy CA and the
pre-minted `gateway` token; `address()` / `ca_cert_pem()` /
`provisioning_transport()` round out the contract.
The infra service now composes the gateway: `ensure_running` mints the
role-scoped `gateway` JWT (it holds the signing key; the gateway never does —
#469) and hands it to `gateway().connect_to_orchestrator(...)`, and
`ca_cert_pem` delegates to the service. The inlined `_ensure_gateway_container`
+ CA-read are gone. `AppleGatewayTransport` + the gateway container name/label
now live with the gateway module, breaking the old infra→provision coupling.
Splits the gateway-run + CA tests out of test_macos_infra into a dedicated
test_macos_gateway; the infra tests mock `svc.gateway`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the docker backend's ad-hoc gateway wiring with the shared `Gateway`
service ABC (in the gateway package), the first of the two per-host service
classes that supersede the per-backend infra glue.
The contract is backend-neutral: `connect_to_orchestrator(url, gateway_token)`
binds the gateway to its control plane and brings it up, `address()` reports
the agent-facing proxy target, `ca_cert_pem()` vends the mitmproxy CA, and
`provisioning_transport()` hands back the exec/cp seam git-gate provisioning
stages per-bottle repos + deploy keys through. `GatewayProvisionError` +
`GatewayTransport` move to the ABC so every backend shares them.
Key trust-model change: the gateway no longer mints its own token. It never
holds the signing key (#469), so the orchestrator mints the role-scoped
`gateway` JWT and hands it in via `connect_to_orchestrator`; the gateway only
injects it. `DockerGateway.ensure_running` becomes `connect_to_orchestrator`
(stashing the URL + token as instance state), and the docker launch flow reads
`address()` / `provisioning_transport()` off the service rather than
re-deriving the container IP + transport itself.
macOS + firecracker gateways move under the ABC in following commits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
These four ensure_running() tests patched container_mod but not
ensure_networks (imported into infra.py from .gateway), so they shelled out
to the real Apple `container` CLI and errored on Linux
(FileNotFoundError: 'container') — the dev + CI platform for this branch.
Mock it so they're true unit tests, green on any platform.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Now that #469 got the DB off the data plane, the Firecracker infra runs as
two microVMs instead of one — mirroring the docker/macos plane split:
* orchestrator VM (ORCH_IFACE) — control plane + buildah image builds; sole
DB opener; host-seeded signing key. No gateway daemons.
* gateway VM (new GW_IFACE) — egress / git-http / supervise data plane;
mitmproxy CA + a host-minted `gateway` JWT (never the key). Reaches the
orchestrator only over the one nft forward rule its link allows.
Both boot the SAME shared infra rootfs; a `bb_role=` kernel-cmdline arg
selects which plane a VM's PID-1 init starts, so there is still one published
artifact. The gateway learns the orchestrator's address via `bb_orch=` on the
cmdline (no IP baked into the artifact).
Isolation is nearly free: agents were already nft-dropped except the DNAT'd
gateway ports, so re-pointing that single DNAT rule at the gateway VM
(`dnat to gw_guest`) severs every agent's L3 route to the control plane. The
only added nft is the second infra link's mirror block (masquerade egress +
forward accept, which subsumes gateway->orchestrator) in the shared shell
script and the NixOS module.
netpool gains GW_IFACE + gw_slot() (the /31 above the orch link);
firecracker_vm.boot gains extra_boot_args for the role cmdline; infra_vm
ensure_running() boots + adopts the pair (orchestrator first, then the gateway
that resolves policy against it) and returns an InfraEndpoint mirroring the
docker/macos shape. Builds stay in the orchestrator (PRD 0070 v1); the gateway
is the slim unit.
Unit-tested (test_firecracker_infra_vm rewritten for two VMs; gw_slot helper
test added); the KVM boot / L3-isolation checks are validated on a Firecracker
host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The single Apple container existed only because two guests writing one
bot-bottle.db over virtiofs would race incoherent fcntl locks; #469 removed
that (the data plane no longer opens the DB), so the macOS backend now runs
the planes as two containers like docker:
* bot-bottle-mac-orchestrator — lean control plane on the host-only
`bot-bottle-mac-control` network only (image Dockerfile.orchestrator,
`-m bot_bottle.orchestrator`). Sole mounter of the container-only DB
volume; holds the signing key. The CLI + gateway reach it at its
control-network address.
* bot-bottle-mac-infra — the gateway, triple-homed on the NAT egress net,
the host-only agent net, and the control net. Resolves the orchestrator by
IP (Apple has no container DNS) via BOT_BOTTLE_ORCHESTRATOR_URL; holds the
CA + gateway JWT.
Agents sit on the agent network only, so they have no route to the control
plane. ensure_networks gains the control network; MacosInfraService brings up
the orchestrator then the gateway; probe_orchestrator_url + ca_cert_pem target
the right containers. pyright 0 errors; unit suite green (2274). Needs a real
Apple-container host to validate the networking.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
didericis-codex
didericis-claude
didericis-claude
didericis