13 Commits

Author	SHA1	Message	Date
didericis-claude	aac27d8a40	fix(firecracker): provision role-scoped control-plane auth in the infra VM ... The Firecracker infra VM started the control plane with no signing key, so it ran open and granted every unauthenticated caller the `cli` role. The nft boundary only fences off the separate agent VM — the egress / supervise / git-http daemons run in this same VM and reach the orchestrator over 127.0.0.1, so a compromised data-plane daemon could still drive the operator routes (approve its own supervise proposals, rewrite policy, read injected tokens). Generate the signing key on the persistent volume, hand it only to the orchestrator process, mint a `gateway` JWT for the data-plane daemons, and mirror the key back to the host token file so the CLI signs `cli` tokens the VM verifies — the same per-process scoping the docker / macOS launchers use. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 07:03:05 +00:00
didericis-claude	b25cd72fc3	test(backend): cover poll_ca_cert timeout paths for diff-coverage gate ... Add tests for the three uncovered paths introduced by the poll_ca_cert extraction: the timeout + sleep branches in backend/util, and the TimeoutError → GatewayError and TimeoutError → die() conversions in the macOS and Firecracker callers.	2026-07-21 04:54:40 +00:00
didericis	d589c08d9d	test(firecracker): pin INFRA_BUILD off in the default-pull test ... The coverage CI job exports BOT_BOTTLE_INFRA_BUILD=local so the integration suite builds infra/agent images from source. But that job also runs the unit suite, and the ambient =local leaked into test_default_pulls_artifact_without_ docker — it drove ensure_built() down the local Docker-build branch, so the "build_image not called / artifact pulled" assertion failed (build_image called 3 times). Pin the var off in that test so it exercises the default pull path regardless of the runner's ambient env (mirrors test_local_mode_builds_deps_before_infra, which pins it on). Full unit suite verified green under BOT_BOTTLE_INFRA_BUILD =local. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR	2026-07-19 18:03:11 -04:00
didericis	0adbf25977	test(firecracker): satisfy pyright on the new infra-VM tests ... Two lint fixes, both test-only (no effect on the infra artifact version): - annotate the TestAdoptable / TestKillInfraFirecrackers helper params (reportMissingParameterType). - test_docker_test_helpers: read __unittest_skip__ via getattr on the dynamically-built Case type, matching the sibling assertion — pyright can't see the attribute the skip decorator adds (reportAttributeAccessIssue). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR	2026-07-19 01:31:06 -04:00
didericis	4c01e31e96	fix(firecracker): version-aware infra-VM adoption + robust teardown ... The infra VM is a per-host singleton that outlives short-lived launchers, so `ensure_running` adopts it when its control plane is healthy. But it adopted ANY healthy VM regardless of the code that built it — so after an infra-code change the old VM kept being adopted and the new code never booted. The only way to dislodge it was an out-of-band `kill`, which then raced whatever launched next. On CI this meant every infra change needed a manual VM kill. Make adoption version-aware: * boot records the infra-artifact version it booted from in a `booted-version` marker beside the singleton; `stop` clears it. * `ensure_running` adopts only when the marker matches the current version (`_adoptable`); a missing/mismatched marker falls through to stop + reboot. So a stale VM is replaced automatically on the next launch — no manual kill, and it's concurrency-safe (the reboot happens under the singleton flock). Also harden teardown: the PID file drifts after crashes / out-of-band kills, so `stop` now also reaps any orphaned firecracker still bound to the infra config path (scoped to that path, so interactive-pool VMs are untouched) — otherwise a survivor holds the orchestrator TAP and the fresh boot dies with "tap … Resource busy". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR	2026-07-19 01:03:30 -04:00
didericis	6f885af4b4	fix(firecracker): make guest /tmp world-writable so agents can use it ... The rootless agent rootfs build can land /tmp as 0755/root-owned, so the agent (uid 1000 node) can't create scratch dirs there. The sandbox-escape suite's README-push test does `cd /tmp && git init sandbox-escape-repo` and died with "cannot mkdir sandbox-escape-repo: Permission denied" — before the git-gate gitleaks hook could run — so the test read it as a missing hook. On docker the agent inherits node:22-slim's 1777 /tmp, which is why only the Firecracker path was affected (and only now that the suite runs end-to-end). Set /tmp to 1777 in the guest PID-1 init, so every agent VM boots with a usable /tmp regardless of rootfs perm drift. Also update the infra-init unit test to assert the gateway launches via the `bot_bottle.gateway_init` module (matching the prior fix), not a file path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR	2026-07-19 00:49:00 -04:00
didericis	18f190b7e3	feat(firecracker): pull the infra rootfs as a prebuilt artifact (PRD 0069 Stage 2) ... Stage 2 of the docker-free Firecracker backend (#348): stop building the fixed infra image on the launch host. The infra VM's rootfs is host- and bottle-agnostic (authorized_keys + guest IP ride the kernel cmdline, not the rootfs), so it's built once off-host and published as a versioned, ready-to- boot ext4; the launch host downloads + verifies + boots it — no Docker, no image tooling, just HTTP + gunzip. - infra_artifact.py: version = content hash of the rootfs inputs (the shipped bot_bottle package + the three Dockerfiles + the init), so a launch host pulls the artifact matching its code and a content change can't silently boot a stale rootfs. Pull + sha256-verify (fail-closed) + gunzip from a Gitea generic package; base/owner/token configurable, default this Gitea. - infra_vm.ensure_built/boot default to the pull path; BOT_BOTTLE_INFRA_BUILD= local keeps the docker build-from-source path for iterating on Dockerfiles. - publish_infra.py: the off-host half — builds the images with Docker, mke2fs the rootfs (with buildah slack), gzips, and PUTs it to the generic package. Rollout note: default=pull means a launch 404s until an artifact is published; until the Gitea packages endpoint is enabled + an artifact published, use BOT_BOTTLE_INFRA_BUILD=local. Freeze/migrate's remaining docker use is a separate PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ	2026-07-16 23:41:53 -04:00
didericis	c60e6b7e9f	refactor(firecracker): single infra VM builds too — buildah in one image (PR #354 review) ... Addresses the review finding that buildah lived only in the orchestrator image, so the persistent infra VM wasn't the builder — a separate throwaway builder VM contended with it for the orchestrator TAP. Consolidate: - **Rebase the gateway (and thus infra) on `python:3.12-slim` = Debian trixie**, pip-installing mitmproxy instead of `FROM mitmproxy/mitmproxy` (Debian bookworm). trixie ships buildah 1.39, which can build agent Dockerfiles that use heredocs; bookworm's 1.28 can't (`Unknown instruction: "{"`). CA path is unchanged (set via `--set confdir=`). - **buildah lives only in `Dockerfile.infra`** now (removed from the orchestrator image, which is lean/stdlib-only again). - **Shared orchestrator content**: `Dockerfile.orchestrator` is the single definition of the control-plane payload; the infra image `COPY --from`s it (same trixie base → clean copy, and future deps like iroh are added once). The docker backend runs the orchestrator image directly. - **`image_builder` builds inside the infra VM** (which now has buildah) over SSH — no throwaway builder VM, so the `bborch0` contention is gone. `ensure_built` builds orchestrator + gateway before infra (FROM gateway, COPY --from orchestrator). Verified on a KVM host: images build (buildah 1.39 in infra), the agent image builds *inside* the infra VM (heredoc Dockerfile and all), the infra VM stays healthy, the agent boots and `claude --version` = 2.1.172. The rebased gateway still starts as a docker container and generates its CA (docker backend unaffected). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 15:23:36 -04:00
didericis	4873030550	feat(firecracker): git-gate over SSH into the gateway VM (Stage B, 7/n) ... git-gate now works end-to-end for git-upstream bottles on the infra VM. Three fixes surfaced by driving a real clone through git-http: - `SshGatewayTransport.cp_into` preserves the source file mode (docker cp does). The access-hook is staged 0700 and git-http execs it directly; a plain `cat >` landed it 0644 -> EACCES. Keys stay 0600. - the infra init runs `BOT_BOTTLE_GATEWAY_DAEMONS=egress,git-http,supervise`: the VM backend reaches git over git-http (9420), so the git:// daemon (git-gate, whose /git-gate-entrypoint.sh the consolidated model doesn't stage) is left out instead of crash-looping. - `build_infra_rootfs_dir` folds the init's content-hash into the rootfs cache key, so an init change actually rebuilds the rootfs (the base image digest alone wouldn't catch it). Verified on a KVM host: launch_consolidated provisions a git-upstream bottle's repo + creds into the gateway VM over SSH; an agent VM clones via git-http (source-IP attributed) and the access-hook resolves the provisioned key + known_hosts and attempts the upstream SSH fetch — failing closed only because the test used a dummy key + fake upstream ("refusing to serve stale data"). With real creds the fetch serves. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 14:36:54 -04:00
didericis	b93b14f5c2	feat(firecracker): persistent registry volume for the infra VM (Stage B, 6/n) ... The infra VM's rootfs is ephemeral (rebuilt each boot), so the bottle registry DB needs durable storage across restarts. Give the infra VM a firecracker analogue of a docker volume: a host-side ext4 file attached as a second virtio-block device (guest /dev/vdb), mounted at the control plane's DB dir (/var/lib/bot-bottle, where host_db_path lives at db/bot-bottle.db). - firecracker_vm.boot/_config take an optional `data_drive` (a non-root, RW second drive). - infra_vm creates the volume on first use (`mke2fs` an empty ext4 at <fc-cache>/infra/registry.ext4) and mounts /dev/vdb in the PID-1 init before the control plane starts. It's a plain ext4 file, so `sudo mount -o loop <path>` (VM stopped) inspects bot-bottle.db directly. Verified on a KVM host: register a bottle, restart the infra VM (fresh rootfs, same volume) — /dev/vdb re-mounts and the registry `db/` dir + a marker file survive the restart. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 14:36:54 -04:00
didericis	bb434b14d7	feat(firecracker): persistent infra-VM singleton lifecycle (Stage B, 4/n) ... The infra VM must outlive the short-lived `start` launcher and be reused across launches. Add an idempotent singleton: - `ensure_running()` adopts the infra VM when its control plane is already healthy (a prior launcher booted it), else clears any stale VM and boots a fresh one. Returns a handle usable for CA fetch / git-gate provisioning whether we booted it or adopted it. - boot is `detached` (firecracker in its own session via start_new_session) so it survives the launcher exiting; its PID is recorded so a later process can `stop()` it. `_kill_pidfile` SIGTERM/SIGKILLs but only if the PID is still a firecracker process (guards a recycled PID). - a STABLE SSH key (generated once under the infra cache dir, re-injected each boot via the cmdline) so any launcher can SSH in to fetch the CA / provision, not just the one that booted the VM. `InfraVm.vm` is None in the adopted case; teardown then goes through the PID file. Verified on a KVM host: first ensure_running boots; a second adopts it (same PID, no reboot) and can still reach /health and fetch the gateway CA over SSH; stop() tears it down (control plane then unreachable). Next: git-gate provisioning into the VM over SSH (today docker exec/cp), then swap consolidated_launch.py onto the infra VM. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 14:36:54 -04:00
didericis	e1610121c0	feat(firecracker): run the gateway data plane in the infra VM too (Stage B, 2/n) ... The single infra VM now runs BOTH the orchestrator control plane and the gateway data plane (egress / supervise / git-http), multi-tenant against the local control plane — the single-VM shape from the Stage B design. - Dockerfile.infra: the firecracker infra image = the gateway image + the baked control-plane source (FROM bot-bottle-gateway, COPY bot_bottle). Reuses the gateway payload rather than copying mitmproxy/ gitleaks into a third image; the docker backend keeps its two separate images. - infra_vm: build from source (gateway then infra image), and the PID-1 init now also launches `gateway_init` with BOT_BOTTLE_ORCHESTRATOR_URL=http://127.0.0.1:8099. Adds `gateway_ca_pem` (fetch the mitmproxy CA over SSH) and the agent-facing port constants. - init exports PATH — a bare-init shell resolves its own execs via a built-in default path, but that isn't in the environment, so gateway_init's `python3 ...` daemons would otherwise fail to spawn. Verified on a KVM host: infra VM boots, control plane /health -> 200, and egress:9099 / supervise:9100 / git-http:9420 all listen and are reachable from the host over the TAP link; the gateway CA is retrievable. (git-gate stays down until a bottle provisions its per-bottle entrypoint/creds, same as a fresh docker gateway — non-fatal, the supervisor keeps the rest up.) Next: agent->gateway VM-to-VM routing so bbfc* VMs reach these ports at the infra VM and nowhere else. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 14:36:54 -04:00
didericis	1614172423	feat(firecracker): run the orchestrator control plane as a VM (Stage B, 1/n) ... First slice of Stage B: the orchestrator control plane runs as a persistent Firecracker infra VM instead of a Docker container. The host CLI reaches it over HTTP at the orchestrator link's guest IP; agent VMs will reach its gateway ports (added next) over VM-to-VM routing. - Dockerfile.orchestrator bakes the stdlib-only control-plane source (COPY bot_bottle) so the image is self-contained and runs from a built image with no runtime bind-mount — a guest VM can't bind-mount host source. (Build-from-source stays the default; a pull-from-registry mode lands later. The docker backend's dev bind-mount still overlays this.) - util.build_base_rootfs_dir / inject_guest_boot take a `variant` + `init_script`, so the same orchestrator image is prepared two ways without a cache collision: the builder VM keeps the SSH-only agent init; the infra VM gets a control-plane PID-1 init. - new firecracker/infra_vm.py: boot the infra VM on the orchestrator link, run `python -m bot_bottle.orchestrator` as PID 1, and poll /health. Verified on a KVM host: infra VM boots, control plane answers `GET /health -> 200 {"status":"ok"}` from the host over the TAP link. Next: fold gateway_init (egress/git-gate/supervise) into the same VM, then agent->gateway routing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 14:36:54 -04:00