refactor(firecracker): single infra VM builds too — buildah in one image (PR #354 review)

Addresses the review finding that buildah lived only in the orchestrator
image, so the persistent infra VM wasn't the builder — a separate throwaway
builder VM contended with it for the orchestrator TAP. Consolidate:

- **Rebase the gateway (and thus infra) on `python:3.12-slim` = Debian
  trixie**, pip-installing mitmproxy instead of `FROM mitmproxy/mitmproxy`
  (Debian bookworm). trixie ships buildah 1.39, which can build agent
  Dockerfiles that use heredocs; bookworm's 1.28 can't (`Unknown
  instruction: "{"`). CA path is unchanged (set via `--set confdir=`).
- **buildah lives only in `Dockerfile.infra`** now (removed from the
  orchestrator image, which is lean/stdlib-only again).
- **Shared orchestrator content**: `Dockerfile.orchestrator` is the single
  definition of the control-plane payload; the infra image `COPY --from`s
  it (same trixie base → clean copy, and future deps like iroh are added
  once). The docker backend runs the orchestrator image directly.
- **`image_builder` builds inside the infra VM** (which now has buildah)
  over SSH — no throwaway builder VM, so the `bborch0` contention is gone.
  `ensure_built` builds orchestrator + gateway before infra (FROM gateway,
  COPY --from orchestrator).

Verified on a KVM host: images build (buildah 1.39 in infra), the agent
image builds *inside* the infra VM (heredoc Dockerfile and all), the infra
VM stays healthy, the agent boots and `claude --version` = 2.1.172. The
rebased gateway still starts as a docker container and generates its CA
(docker backend unaffected).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck

Dockerfile.gateway

@@ -39,28 +39,32 @@
 # with Dockerfile.git-gate's prior base (now deleted at chunk 3).
 FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
 
-# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
+# Stage 2: assembly. Based on `python:3.12-slim` (Debian trixie) rather
-# Python + mitmdump pre-installed — heavier than the others, so
+# than the `mitmproxy/mitmproxy` image (Debian bookworm) so the whole
-# this stage starts there and pulls the standalone binaries in.
+# stack — gateway here, and the firecracker infra image that builds
-FROM mitmproxy/mitmproxy:11.1.3
+# FROM this — lands on trixie, whose buildah (1.39) can build agent
-
+# Dockerfiles that use heredocs. mitmproxy is pip-installed to the same
-# Run as root inside the bundle. The bundle is the isolation
+# effect as the upstream image. (bookworm's buildah is 1.28, which can't
-# boundary; per-daemon user separation inside it is not load-bearing
+# parse `RUN ... <<EOF`; see the infra image + PR discussion.)
-# and complicates the supervisor's spawn path.
+FROM python:3.12-slim
-USER root
 
 # Runtime system deps:
 #   git supplies the `git daemon` subcommand (no separate package)
 #     plus the core `git` binary the pre-receive hook invokes.
 #   openssh-client supplies the upstream SSH transport the
 #     pre-receive hook uses to forward accepted refs.
-#   ca-certificates is needed for mitmdump upstream TLS (the
+#   ca-certificates is needed for mitmdump upstream TLS.
-#     base image already has it; listed for explicitness).
 RUN apt-get update \
  && apt-get install -y --no-install-recommends \
       git openssh-client ca-certificates \
  && rm -rf /var/lib/apt/lists/*
 
+# mitmdump (the egress data plane). The upstream mitmproxy image baked
+# this in; on the plain python base we pip-install the same pinned
+# version. Its CA dir is set explicitly via `--set confdir=` in
+# egress-entrypoint.sh, so it doesn't depend on a `mitmproxy` home user.
+RUN pip install --no-cache-dir mitmproxy==11.1.3
+
 # Pull the standalone binaries into the final image.
 COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
 

Dockerfile.infra

@@ -0,0 +1,45 @@
+# Firecracker single infra-VM image (PRD 0070 Stage B).
+#
+# The per-host infra VM runs the orchestrator control plane, the gateway
+# data plane, AND builds agent images (buildah) — all in one microVM (see
+# backend/firecracker/infra_vm.py). It composes:
+#   * FROM the gateway image (mitmproxy / git / gitleaks / supervise + the
+#     flat daemon modules) — now trixie-based, so buildah 1.39 is available;
+#   * `COPY --from` the orchestrator image's content (the single definition
+#     of the control-plane payload — see Dockerfile.orchestrator), so this
+#     VM and the docker backend share one orchestrator definition; and
+#   * buildah, installed HERE only (the docker orchestrator/gateway images
+#     never carry it).
+#
+# multi-`FROM` can't union two bases (that's multi-stage, not multiple
+# inheritance), so the orchestrator content is pulled in via `COPY --from`
+# rather than a second base. Both images share the trixie `python:3.12-slim`
+# base, so the copy is clean (same python; future installed deps copy too).
+#
+# The docker backend keeps orchestrator + gateway as separate images; this
+# combined image exists only for the Firecracker single-VM cut. Splitting a
+# service back into its own VM later is a routing change, not a repackaging
+# (PRD 0070's "secret concentration"; a disposable builder can boot from
+# this same image on its own TAP).
+FROM bot-bottle-gateway:latest
+
+# --- in-VM agent-image builder (PRD 0069 Stage 3) -------------------
+# The Firecracker backend builds users' agent Dockerfiles *inside this VM*
+# with buildah (rootless, daemonless) instead of on the host — no host
+# Docker daemon, no root-equivalent `docker` group. `crun` is the OCI
+# runtime; `netavark` + `aardvark-dns` are the network backend for `FROM`
+# pulls + `RUN` egress. Requires the trixie base (buildah 1.39: bookworm's
+# 1.28 can't parse Dockerfile heredocs that agent images use).
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+      buildah crun netavark aardvark-dns \
+ && rm -rf /var/lib/apt/lists/*
+# vfs + chroot: buildah works as root in the bare microVM (no
+# fuse-overlayfs / overlay module / subuid maps). Matches image_builder.
+ENV STORAGE_DRIVER=vfs \
+    BUILDAH_ISOLATION=chroot
+
+# The orchestrator content, pulled from its single definition. The gateway
+# image already has the flat daemon modules under /app; this adds the full
+# `bot_bottle` package so `python3 -m bot_bottle.orchestrator` resolves.
+COPY --from=bot-bottle-orchestrator:latest /app/bot_bottle /app/bot_bottle

Dockerfile.orchestrator

@@ -1,27 +1,36 @@
 # Orchestrator control-plane image (PRD 0070, #384).
 #
-# The per-host orchestrator runs `python3 -m bot_bottle.orchestrator`.
+# This is the **single definition of the orchestrator's content** — the
-# The `bot_bottle` package is **stdlib-only** by design, so the control
+# `bot_bottle` package baked onto a Python runtime — referenced by BOTH:
-# plane needs nothing but a Python runtime — none of the gateway's
+#   * the docker backend, which runs this image directly as the lean
-# mitmproxy / git / gitleaks payload (that is the separate
+#     control-plane container; and
-# `bot-bottle-gateway` image, Dockerfile.gateway). Splitting them keeps
+#   * the firecracker infra image (Dockerfile.infra), which `COPY --from`s
-# the secret-dense control plane (it concentrates every bottle's egress
+#     this image's `/app/bot_bottle` so the single infra VM runs the same
-# tokens — see PRD 0070's "secret concentration") on a minimal
+#     control plane. Keeping it in one place means future orchestrator deps
-# dependency surface.
+#     (e.g. iroh) are added here once, not duplicated per backend.
 #
-# The repo is bind-mounted read-only into the container at run time (see
+# It stays deliberately lean: the control plane is **stdlib-only** today, so
-# `orchestrator/lifecycle.py`), so the source is NOT copied in here: the
+# no third-party payload — none of the gateway's mitmproxy/git/gitleaks
-# image is just the runtime. `ensure_running` recreates the container
+# (that's Dockerfile.gateway) and no buildah (that's the firecracker
-# only when the bind-mounted source hash changes (#381), which is why
+# builder, and lives only in Dockerfile.infra). Keeping the secret-dense
-# the code stays a mount rather than a baked layer.
+# control plane on a minimal dependency surface is the point (PRD 0070's
+# "secret concentration").
+#
+# Shares the trixie `python:3.12-slim` base with the gateway image, so when
+# the orchestrator grows real deps they can be `COPY --from`'d into the
+# infra image cleanly (same base/python — installed packages copy safely).
 
 FROM python:3.12-slim
 
-# No third-party deps to install — stdlib only. Kept as an explicit,
-# self-documenting stage so a future confinement step (baking the
-# package, dropping the bind mount) has an obvious home.
 WORKDIR /app
 
+# The orchestrator content. Baked so the image is self-contained (runs from
+# a built image, no runtime bind-mount); the docker backend may still
+# bind-mount /app for dev live-reload, which simply overlays this copy.
+# `.dockerignore` keeps .git/docs/*.md out of the context. (Future deps like
+# iroh go here too — a shared requirements installed on this same base.)
+COPY bot_bottle /app/bot_bottle
+
 # Documentation only; lifecycle.py overrides the entrypoint to
 # `python3 -m bot_bottle.orchestrator` with the runtime flags.
 ENTRYPOINT ["python3", "-m", "bot_bottle.orchestrator"]

bot_bottle/agent_provider.py

@@ -61,6 +61,13 @@ class AgentProviderRuntime:
     prompt_mode: PromptMode
     bypass_args: tuple[str, ...]
     resume_args: tuple[str, ...]
+    # argv run inside a throwaway container of a freshly built agent
+    # image, right after `build_image()`, to catch a build that
+    # exited 0 but produced a broken CLI (e.g. an npm
+    # optionalDependencies fetch for a platform-native binary that
+    # silently no-ops on a transient failure). Empty tuple skips the
+    # check — not every provider has opted in yet.
+    smoke_test: tuple[str, ...] = ()
 
 
 @dataclass(frozen=True)

bot_bottle/backend/docker/consolidated_launch.py

@@ -29,7 +29,11 @@ from ...orchestrator.gateway import GATEWAY_NAME, GATEWAY_NETWORK
 from ...orchestrator.lifecycle import OrchestratorService
 from ...orchestrator.registration import registration_inputs
 from .gateway_net import next_free_ip
-from .gateway_provision import deprovision_git_gate, provision_git_gate
+from .gateway_provision import (
+    DockerGatewayTransport,
+    deprovision_git_gate,
+    provision_git_gate,
+)
 
 
 class ConsolidatedLaunchError(RuntimeError):
@@ -121,7 +125,8 @@ def launch_consolidated(
         metadata=inputs.metadata, tokens=tokens,
     )
     try:
-        provision_git_gate(gateway_name, reg.bottle_id, git_gate_plan)
+        provision_git_gate(
+            DockerGatewayTransport(gateway_name), reg.bottle_id, git_gate_plan)
     except Exception:
         # Roll the registration back so a provisioning failure leaves no orphan.
         client.teardown_bottle(reg.bottle_id)
@@ -142,7 +147,7 @@ def teardown_consolidated(
     """Deregister the bottle and remove its git-gate state from the gateway.
     Both steps are idempotent so this is safe from a cleanup trap."""
     OrchestratorClient(orchestrator_url).teardown_bottle(bottle_id)
-    deprovision_git_gate(gateway_name, bottle_id)
+    deprovision_git_gate(DockerGatewayTransport(gateway_name), bottle_id)
 
 
 __all__ = [

bot_bottle/backend/docker/gateway_provision.py

@@ -15,14 +15,15 @@ bottle's push credentials out of another's repos on the shared gateway.
 from __future__ import annotations
 
 import re
+from typing import Protocol
 
 from ...docker_cmd import run_docker
 from ...git_gate import GitGatePlan, git_gate_render_provision
 
 # bottle ids index the gateway's per-bottle repo + creds dirs; they land in
-# `docker cp`/`rm` path arguments, so validate before any path is built (a
+# exec/cp path arguments, so validate before any path is built (a traversal
-# traversal id like "../etc" must never reach the container). Registry ids are
+# id like "../etc" must never reach the gateway). Registry ids are token_hex —
-# token_hex — this is defense in depth at the docker boundary.
+# this is defense in depth at the transport boundary.
 _SAFE_BOTTLE_ID = re.compile(r"[A-Za-z0-9_-]+")
 
 
@@ -30,6 +31,42 @@ class GatewayProvisionError(RuntimeError):
     """A git-gate provisioning step against the running gateway failed."""
 
 
+class GatewayTransport(Protocol):
+    """How the launcher stages files + runs commands in the running gateway.
+    Backend-neutral so the same provisioning logic serves the docker gateway
+    (exec/cp over the docker socket) and the firecracker gateway VM (over
+    SSH)."""
+
+    def exec(self, argv: list[str]) -> None:
+        """Run `argv` in the gateway, raising `GatewayProvisionError` on
+        failure."""
+
+    def cp_into(self, src: str, dest: str) -> None:
+        """Copy host file `src` to `dest` in the gateway, raising on
+        failure."""
+
+
+class DockerGatewayTransport:
+    """`GatewayTransport` for the docker gateway container (exec/cp)."""
+
+    def __init__(self, gateway: str) -> None:
+        self.gateway = gateway
+
+    def exec(self, argv: list[str]) -> None:
+        proc = run_docker(["docker", "exec", self.gateway, *argv])
+        if proc.returncode != 0:
+            raise GatewayProvisionError(
+                f"gateway exec {argv!r} failed: {proc.stderr.strip()}"
+            )
+
+    def cp_into(self, src: str, dest: str) -> None:
+        proc = run_docker(["docker", "cp", src, f"{self.gateway}:{dest}"])
+        if proc.returncode != 0:
+            raise GatewayProvisionError(
+                f"gateway cp {src} -> {dest} failed: {proc.stderr.strip()}"
+            )
+
+
 def _require_safe(bottle_id: str) -> None:
     if not _SAFE_BOTTLE_ID.fullmatch(bottle_id):
         raise GatewayProvisionError(f"unsafe bottle id {bottle_id!r}")
@@ -39,27 +76,11 @@ def _creds_dir(bottle_id: str) -> str:
     return f"/git-gate/creds/{bottle_id}"
 
 
-def _exec(gateway: str, argv: list[str]) -> None:
+def provision_git_gate(
-    """`docker exec` a command in the gateway, raising on non-zero exit."""
+    transport: GatewayTransport, bottle_id: str, plan: GitGatePlan,
-    proc = run_docker(["docker", "exec", gateway, *argv])
+) -> None:
-    if proc.returncode != 0:
+    """Place `bottle_id`'s git-gate credentials into the running gateway and
-        raise GatewayProvisionError(
+    init its bare repos under `/git/<bottle_id>/`.
-            f"gateway exec {argv!r} failed: {proc.stderr.strip()}"
-        )
-
-
-def _cp_into(gateway: str, src: str, dest: str) -> None:
-    """`docker cp` a host file into the gateway, raising on non-zero exit."""
-    proc = run_docker(["docker", "cp", src, f"{gateway}:{dest}"])
-    if proc.returncode != 0:
-        raise GatewayProvisionError(
-            f"gateway cp {src} -> {dest} failed: {proc.stderr.strip()}"
-        )
-
-
-def provision_git_gate(gateway: str, bottle_id: str, plan: GitGatePlan) -> None:
-    """Place `bottle_id`'s git-gate credentials into the running `gateway`
-    container and init its bare repos under `/git/<bottle_id>/`.
 
     Copies each upstream's identity key (and known_hosts, when present) into
     `/git-gate/creds/<bottle_id>/`, then runs the namespaced provisioning
@@ -70,31 +91,36 @@ def provision_git_gate(gateway: str, bottle_id: str, plan: GitGatePlan) -> None:
     # The pre-receive + access hooks are bottle-agnostic and shared by every
     # bottle's repos; install them into the gateway (idempotent — same content
     # each time). The per-bottle model cp'd these into each bundle at start.
-    _exec(gateway, ["mkdir", "-p", "/etc/git-gate"])
+    transport.exec(["mkdir", "-p", "/etc/git-gate"])
-    _cp_into(gateway, str(plan.hook_script), "/etc/git-gate/pre-receive")
+    transport.cp_into(str(plan.hook_script), "/etc/git-gate/pre-receive")
-    _cp_into(gateway, str(plan.access_hook_script), "/etc/git-gate/access-hook")
+    transport.cp_into(str(plan.access_hook_script), "/etc/git-gate/access-hook")
     creds = _creds_dir(bottle_id)
-    _exec(gateway, ["mkdir", "-p", creds])
+    transport.exec(["mkdir", "-p", creds])
     for u in plan.upstreams:
         if u.identity_file:
-            _cp_into(gateway, u.identity_file, f"{creds}/{u.name}-key")
+            transport.cp_into(u.identity_file, f"{creds}/{u.name}-key")
         known_hosts = str(u.known_hosts_file)
         if known_hosts and known_hosts != ".":
-            _cp_into(gateway, known_hosts, f"{creds}/{u.name}-known_hosts")
+            transport.cp_into(known_hosts, f"{creds}/{u.name}-known_hosts")
     # Init the bare repos + per-repo credential config for this namespace.
     script = git_gate_render_provision(bottle_id, plan.upstreams)
-    _exec(gateway, ["sh", "-c", script])
+    transport.exec(["sh", "-c", script])
 
 
-def deprovision_git_gate(gateway: str, bottle_id: str) -> None:
+def deprovision_git_gate(transport: GatewayTransport, bottle_id: str) -> None:
     """Remove a bottle's repos + creds from the gateway on teardown. Idempotent
     — an already-absent namespace is a clean no-op (best effort; a stray dir
     can't leak, since attribution is by source IP and the bottle is gone)."""
     _require_safe(bottle_id)
-    run_docker([
+    try:
-        "docker", "exec", gateway, "rm", "-rf",
+        transport.exec([
-        f"/git/{bottle_id}", _creds_dir(bottle_id),
+            "rm", "-rf", f"/git/{bottle_id}", _creds_dir(bottle_id),
-    ])
+        ])
+    except GatewayProvisionError:
+        pass  # best-effort teardown; absent namespace is success
 
 
-__all__ = ["provision_git_gate", "deprovision_git_gate", "GatewayProvisionError"]
+__all__ = [
+    "provision_git_gate", "deprovision_git_gate",
+    "GatewayProvisionError", "GatewayTransport", "DockerGatewayTransport",
+]

bot_bottle/backend/docker/launch.py

@@ -36,6 +36,7 @@ from contextlib import ExitStack, contextmanager
 from pathlib import Path
 from typing import Callable, Generator
 
+from ...agent_provider import runtime_for
 from ...egress import egress_resolve_token_values
 from ...git_gate import (
     provision_git_gate_dynamic_keys,
@@ -110,6 +111,9 @@ def launch(
                 plan.image, _REPO_DIR,
                 dockerfile=plan.dockerfile_path,
             )
+            docker_mod.verify_agent_image(
+                plan.image, runtime_for(plan.agent_provider_template).smoke_test,
+            )
 
         # Step 2: mint the git-gate dynamic (gitea) deploy keys, if any, before
         # provisioning the bottle's repos into the shared gateway.

bot_bottle/backend/docker/util.py

@@ -4,11 +4,13 @@ existence, and building images."""
 
 from __future__ import annotations
 
+import os
 import re
 import shutil
 import subprocess
 from typing import Iterable, Iterator
 
+from ...docker_cmd import run_docker
 from ...log import die, info
 # from ...workspace import WorkspacePlan
 
@@ -88,6 +90,29 @@ def docker_exec_root(container: str, argv: list[str]) -> None:
     )
 
 
+def docker_exec(container: str, argv: list[str], *, user: str = "") -> None:
+    """Run `docker exec` in the named container, dying with the
+    command's own stderr on failure. Pass `user=\"0\"` to run as root."""
+    cmd = ["docker", "exec"]
+    if user:
+        cmd += ["-u", user]
+    cmd += [container, *argv]
+    result = run_docker(cmd)
+    if result.returncode != 0:
+        die(
+            f"docker exec in {container} failed: "
+            f"{(result.stderr or '').strip() or '<no stderr>'}"
+        )
+
+
+def docker_cp(src: str, dest: str) -> None:
+    """Run `docker cp`, dying with the command's own stderr on failure."""
+    result = run_docker(["docker", "cp", src, dest])
+    if result.returncode != 0:
+        die(f"docker cp {src} -> {dest} failed: "
+            f"{(result.stderr or '').strip() or '<no stderr>'}")
+
+
 _SLUG_RE = re.compile(r"[^a-z0-9]+")
 
 
@@ -108,15 +133,40 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
 
     `dockerfile` is an optional path (relative to `context`, or
     absolute) for callers that need to build from a non-default
-    Dockerfile in the same context — e.g. `Dockerfile.git-gate`."""
+    Dockerfile in the same context — e.g. `Dockerfile.git-gate`.
+
+    Set `BOT_BOTTLE_NO_CACHE=1` (the `start --no-cache` flag) to force
+    `--no-cache`. The npm/curl installers some provider Dockerfiles
+    shell out to can silently no-op on a transient network failure —
+    e.g. an `optionalDependencies` fetch for a platform-native binary —
+    and Docker will then cache that broken layer indefinitely."""
     info(f"building image {ref} from {context} (layer cache keeps repeat builds fast)")
     args = ["docker", "build", "-t", ref]
+    if os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
+        args.append("--no-cache")
     if dockerfile:
         args.extend(["-f", dockerfile])
     args.append(context)
     subprocess.run(args, check=True)
 
 
+def verify_agent_image(image: str, argv: tuple[str, ...]) -> None:
+    """Run `argv` inside a throwaway container of a freshly built agent
+    image and die loudly if it fails, instead of shipping an image
+    whose CLI only breaks at first real use. No-op when the provider
+    hasn't declared a smoke test (`AgentProviderRuntime.smoke_test`)."""
+    if not argv:
+        return
+    result = run_docker(["docker", "run", "--rm", "--entrypoint", argv[0], image, *argv[1:]])
+    if result.returncode != 0:
+        detail = (result.stderr or result.stdout or "").strip()
+        die(
+            f"agent image {image!r} failed its post-build smoke test "
+            f"({' '.join(argv)}): {detail}\n"
+            f"Try rebuilding from scratch: bot-bottle start --no-cache"
+        )
+
+
 # def build_image_with_cwd(
 #     derived: str,
 #     base: str,

bot_bottle/backend/firecracker/consolidated_launch.py

@@ -0,0 +1,109 @@
+"""Consolidated bottle launch sequence for the Firecracker backend
+(PRD 0070, Stage B).
+
+The shared gateway + orchestrator control plane run in a single persistent
+per-host **infra VM** (`infra_vm.py`), not Docker containers. Agent VMs reach
+the gateway's egress / supervise / git-http ports at the infra VM via a
+PREROUTING DNAT on their own host-side TAP IP (see
+`scripts/firecracker-netpool.sh`), and the host CLI reaches the control plane
+over HTTP at the infra VM's guest IP.
+
+Attribution is by the agent VM's guest IP, unspoofable by construction: the
+/31 point-to-point TAP + the `bot_bottle_fc` nft table ensure only the
+expected VM can source-IP that address.
+
+Sequence:
+  1. ensure the infra VM (control plane + gateway) is up (a singleton — a
+     prior launcher may already have booted it);
+  2. register the bottle by its guest IP (attribution key) → bottle id +
+     identity token;
+  3. provision its git-gate repos/creds into the gateway VM (over SSH);
+  4. fetch the shared gateway CA for the provisioner to install in the rootfs.
+
+The TAP slot allocation, rootfs build, and VM boot are the caller's job.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from ...egress import EgressPlan
+from ...git_gate import GitGatePlan
+from ...orchestrator.client import OrchestratorClient
+from ...orchestrator.lifecycle import (
+    OrchestratorStartError,  # re-exported so callers can catch it
+)
+from ...orchestrator.registration import registration_inputs
+from ..docker.gateway_provision import deprovision_git_gate, provision_git_gate
+from . import infra_vm
+
+
+class ConsolidatedLaunchError(RuntimeError):
+    """The consolidated register/provision sequence could not complete."""
+
+
+@dataclass(frozen=True)
+class LaunchContext:
+    """What the Firecracker launch needs from the consolidated sequence."""
+
+    bottle_id: str
+    identity_token: str
+    source_ip: str          # the VM's guest IP — the attribution key
+    gateway_ca_pem: str     # the shared gateway CA the provisioner installs
+    orchestrator_url: str
+
+
+def launch_consolidated(
+    egress_plan: EgressPlan,
+    git_gate_plan: GitGatePlan,
+    *,
+    guest_ip: str,
+    image_ref: str = "",
+    tokens: dict[str, str] | None = None,
+) -> LaunchContext:
+    """Ensure the infra VM is up, register the bottle by its guest IP, and
+    provision its git-gate state into the gateway VM. Returns the context the
+    agent-VM launch needs. Raises on failure — the caller tears down."""
+    infra = infra_vm.ensure_running()
+    url = infra.control_plane_url
+    client = OrchestratorClient(url)
+
+    inputs = registration_inputs(egress_plan)
+    reg = client.register_bottle(
+        guest_ip, image_ref=image_ref, policy=inputs.policy,
+        metadata=inputs.metadata, tokens=tokens,
+    )
+    try:
+        provision_git_gate(
+            infra_vm.gateway_transport(), reg.bottle_id, git_gate_plan)
+    except Exception:
+        client.teardown_bottle(reg.bottle_id)
+        raise
+
+    # The shared gateway CA every agent on this host trusts for TLS
+    # interception — fetched from the infra VM over SSH.
+    return LaunchContext(
+        bottle_id=reg.bottle_id,
+        identity_token=reg.identity_token,
+        source_ip=guest_ip,
+        gateway_ca_pem=infra.gateway_ca_pem(),
+        orchestrator_url=url,
+    )
+
+
+def teardown_consolidated(bottle_id: str, *, orchestrator_url: str) -> None:
+    """Deregister the bottle and remove its git-gate state from the gateway
+    VM. Both steps are idempotent so this is safe from a cleanup trap. Does
+    NOT stop the infra VM — it's a persistent per-host singleton shared by
+    every bottle."""
+    OrchestratorClient(orchestrator_url).teardown_bottle(bottle_id)
+    deprovision_git_gate(infra_vm.gateway_transport(), bottle_id)
+
+
+__all__ = [
+    "LaunchContext",
+    "launch_consolidated",
+    "teardown_consolidated",
+    "ConsolidatedLaunchError",
+    "OrchestratorStartError",
+]

bot_bottle/backend/firecracker/firecracker_vm.py

@@ -78,20 +78,32 @@ def _config(
     vcpus: int,
     mem_mib: int,
     guest_mac: str,
+    data_drive: Path | None = None,
 ) -> dict[str, object]:
+    drives: list[dict[str, object]] = [
+        {
+            "drive_id": "rootfs",
+            "path_on_host": str(rootfs),
+            "is_root_device": True,
+            "is_read_only": False,
+        }
+    ]
+    # A second virtio-block device (guest /dev/vdb) — the infra VM's
+    # persistent registry "volume", a host-side ext4 file that outlives the
+    # ephemeral rootfs across VM restarts.
+    if data_drive is not None:
+        drives.append({
+            "drive_id": "data",
+            "path_on_host": str(data_drive),
+            "is_root_device": False,
+            "is_read_only": False,
+        })
     return {
         "boot-source": {
             "kernel_image_path": str(util.kernel_path()),
             "boot_args": _boot_args(guest_ip, host_ip, pubkey),
         },
-        "drives": [
+        "drives": drives,
-            {
-                "drive_id": "rootfs",
-                "path_on_host": str(rootfs),
-                "is_root_device": True,
-                "is_read_only": False,
-            }
-        ],
         "network-interfaces": [
             {
                 "iface_id": "eth0",
@@ -118,9 +130,16 @@ def boot(
     vcpus: int = 2,
     mem_mib: int = 2048,
     guest_mac: str = "06:00:AC:10:00:02",
+    detached: bool = False,
+    data_drive: Path | None = None,
 ) -> VmHandle:
     """Write the config and launch the VMM. Returns once the process is
-    spawned; callers wait for SSH readiness separately."""
+    spawned; callers wait for SSH readiness separately.
+
+    `detached` starts the VMM in its own session (`start_new_session`) so it
+    survives the launcher exiting — used for the persistent per-host infra
+    VM, which must outlive the short-lived `start` process (agent VMs stay
+    attached and are torn down with the launcher)."""
     run_dir.mkdir(parents=True, exist_ok=True)
     config_path = run_dir / "config.json"
     console_log = run_dir / "console.log"
@@ -128,6 +147,7 @@ def boot(
         _config(
             rootfs=rootfs, tap=tap, guest_ip=guest_ip, host_ip=host_ip,
             pubkey=pubkey, vcpus=vcpus, mem_mib=mem_mib, guest_mac=guest_mac,
+            data_drive=data_drive,
         ),
         indent=2,
     ))
@@ -137,6 +157,7 @@ def boot(
     process = subprocess.Popen(
         ["firecracker", "--no-api", "--config-file", str(config_path)],
         stdout=log_fh, stderr=subprocess.STDOUT, stdin=subprocess.DEVNULL,
+        start_new_session=detached,
     )
     return VmHandle(process=process, guest_ip=guest_ip, console_log=console_log)
 

bot_bottle/backend/firecracker/image_builder.py

@@ -0,0 +1,170 @@
+"""Docker-free agent-image builds for the Firecracker backend (PRD 0069 Stage 3).
+
+Agent Dockerfiles build **inside the persistent per-host infra VM**
+(`infra_vm.py`), which carries buildah (rootless, daemonless): no host Docker
+daemon, no root-equivalent `docker` group. The build runs over SSH against the
+infra VM and its rootfs streams back to the host, where the existing
+`mke2fs -d` path (`util.build_rootfs_ext4`) turns it into a bootable ext4.
+
+Building in the infra VM — rather than a throwaway builder VM — means there is
+one buildah image (`bot-bottle-infra`) and no contention for the orchestrator
+TAP. Tradeoff: an untrusted Dockerfile's `RUN` steps share the VM with the
+control plane + gateway (buildah `--isolation chroot` isn't a hard boundary) —
+the accepted single-VM blast-radius tradeoff, re-splittable into a disposable
+builder (booted from this same image on its own TAP) later.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import shutil
+import subprocess
+from pathlib import Path
+
+from ...log import die, info
+from . import infra_vm, util
+
+# vfs + chroot: buildah works as root in the microVM (no fuse-overlayfs /
+# overlay module / subuid maps). `--isolation` is a build/run-only flag;
+# `from`/`mount` take just the store.
+_BUILD_FLAGS = "--isolation chroot --storage-driver vfs"
+_STORE_FLAG = "--storage-driver vfs"
+
+_BUILD_TIMEOUT_SECONDS = 900.0
+
+
+def _dockerfile_hash(dockerfile: Path) -> str:
+    """Cache key: the Dockerfile's content. The shipped agent Dockerfiles
+    COPY nothing from the build context (see .dockerignore), so their content
+    fully determines the image; a Dockerfile that adds COPY will want the
+    context folded in here too."""
+    return hashlib.sha256(dockerfile.read_bytes()).hexdigest()[:16]
+
+
+def build_agent_rootfs_dir(
+    dockerfile: Path, *, image_tag: str, smoke_test: tuple[str, ...] = (),
+) -> Path:
+    """Build `dockerfile` in the infra VM (buildah, no host docker), export its
+    rootfs, inject the guest boot bits, and return the cached base dir — the
+    same shape `util.build_rootfs_ext4` consumes. Cached by Dockerfile content,
+    so a repeat launch skips the rebuild.
+
+    `smoke_test` (the provider's declared argv, e.g. `("claude","--version")`)
+    is run in the freshly built image before export, catching an npm
+    silent-failure image at build time rather than at first agent use."""
+    digest = _dockerfile_hash(dockerfile)
+    base = util.cache_dir() / "rootfs" / f"agent-{digest}"
+    ready = base / ".bb-ready"
+    if ready.is_file():
+        info(f"using cached agent rootfs {base.name}")
+        return base
+
+    if base.exists():
+        shutil.rmtree(base, ignore_errors=True)
+    base.mkdir(parents=True)
+
+    info(f"building agent image {image_tag!r} in the infra VM")
+    _build_in_infra(dockerfile, base, smoke_test, digest)
+    util.inject_guest_boot(base)
+    ready.write_text("ok\n")
+    return base
+
+
+def _build_in_infra(
+    dockerfile: Path, base: Path, smoke_test: tuple[str, ...], digest: str,
+) -> None:
+    """Ensure the infra VM is up, `buildah build` the Dockerfile in it, smoke
+    test the image, and stream its rootfs into `base`. The infra VM persists;
+    only the per-build container/image/context are cleaned up."""
+    infra = infra_vm.ensure_running()
+    key, ip = infra.private_key, infra.guest_ip
+    tag = f"bot-bottle-agent-build-{digest}"
+    ctx = f"/tmp/agent-build-{digest}"
+    try:
+        prep = _ssh(key, ip, f"rm -rf {ctx}; mkdir -p {ctx}/ctx")
+        if prep.returncode != 0:
+            die(f"preparing build dir in the infra VM failed: {prep.stderr.strip()}")
+        _send_dockerfile(key, ip, dockerfile, ctx)
+        _buildah_build(key, ip, ctx, tag)
+        _smoke_test(key, ip, tag, smoke_test)
+        _stream_rootfs(key, ip, tag, base)
+    finally:
+        # Keep the ephemeral infra rootfs from accumulating build state. Builds
+        # are serial per launch, so removing all working containers is safe.
+        _ssh(key, ip,
+             f"buildah rm {_STORE_FLAG} -a >/dev/null 2>&1; "
+             f"buildah rmi {_STORE_FLAG} {tag} >/dev/null 2>&1; rm -rf {ctx}",
+             timeout=60)
+
+
+def _ssh(private_key: Path, guest_ip: str, script: str,
+         *, timeout: float = 60.0) -> subprocess.CompletedProcess[str]:
+    return subprocess.run(
+        util.ssh_base_argv(private_key, guest_ip) + [script],
+        capture_output=True, text=True, timeout=timeout, check=False,
+    )
+
+
+def _send_dockerfile(private_key: Path, guest_ip: str, dockerfile: Path, ctx: str) -> None:
+    proc = subprocess.run(
+        util.ssh_base_argv(private_key, guest_ip) + [f"cat > {ctx}/Dockerfile"],
+        input=dockerfile.read_bytes(), capture_output=True, timeout=30, check=False,
+    )
+    if proc.returncode != 0:
+        die(f"sending Dockerfile to the infra VM failed: "
+            f"{proc.stderr.decode(errors='replace').strip()}")
+
+
+def _buildah_build(private_key: Path, guest_ip: str, ctx: str, tag: str) -> None:
+    build = _ssh(
+        private_key, guest_ip,
+        f"buildah build {_BUILD_FLAGS} -t {tag} -f {ctx}/Dockerfile {ctx}/ctx",
+        timeout=_BUILD_TIMEOUT_SECONDS,
+    )
+    if build.returncode != 0:
+        tail = "\n".join((build.stdout + build.stderr).strip().splitlines()[-20:])
+        die(f"buildah build in the infra VM failed:\n{tail}")
+
+
+def _smoke_test(private_key: Path, guest_ip: str, tag: str, argv: tuple[str, ...]) -> None:
+    """Run the provider's smoke argv inside the freshly built image
+    (`buildah run`, which uses the image's own PATH), failing the build
+    loudly if the CLI is a broken stub. No-op without a declared test."""
+    if not argv:
+        return
+    cmd = (
+        f"set -e; ctr=$(buildah from {_STORE_FLAG} {tag}); "
+        f"buildah run {_BUILD_FLAGS} \"$ctr\" -- {' '.join(argv)}; rc=$?; "
+        f"buildah rm \"$ctr\" >/dev/null 2>&1 || true; exit $rc"
+    )
+    result = _ssh(private_key, guest_ip, cmd, timeout=120)
+    if result.returncode != 0:
+        detail = (result.stdout + result.stderr).strip().splitlines()[-10:]
+        die(f"agent image failed its post-build smoke test "
+            f"({' '.join(argv)}):\n" + "\n".join(detail))
+
+
+def _stream_rootfs(private_key: Path, guest_ip: str, tag: str, base: Path) -> None:
+    """`buildah mount` the built image in the infra VM and pipe its rootfs tar
+    straight into `base` on the host (extracted as the non-root host user, so
+    uid 0 isn't preserved — the guest init restores /root ownership)."""
+    export = (
+        f"set -e; ctr=$(buildah from {_STORE_FLAG} {tag}); "
+        f"mnt=$(buildah mount {_STORE_FLAG} \"$ctr\"); "
+        f"tar -C \"$mnt\" -cf - ."
+    )
+    ssh_proc = subprocess.Popen(
+        util.ssh_base_argv(private_key, guest_ip) + [export],
+        stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+    )
+    assert ssh_proc.stdout is not None
+    untar = subprocess.run(
+        ["tar", "-x", "-C", str(base)], stdin=ssh_proc.stdout, check=False,
+    )
+    ssh_proc.stdout.close()
+    ssh_err = (ssh_proc.stderr.read().decode(errors="replace")
+               if ssh_proc.stderr else "")
+    rc = ssh_proc.wait()
+    if rc != 0 or untar.returncode != 0:
+        die(f"exporting the built rootfs from the infra VM failed: "
+            f"{ssh_err.strip() or '<no stderr>'}")

bot_bottle/backend/firecracker/infra_vm.py

@@ -0,0 +1,391 @@
+"""The per-host infra VM for the Firecracker backend (PRD 0070 Stage B).
+
+A single persistent microVM that runs the orchestrator **control plane** (and,
+in a following step, the gateway **data plane**) — the trusted per-host service
+the docker backend runs as containers. It boots on the NAT'd orchestrator link
+(`netpool.orch_slot()`): the host CLI reaches its control plane over HTTP at the
+guest IP, and agent VMs reach its gateway ports over VM-to-VM routing.
+
+Build-from-source (the default while the design churns): the rootfs is exported
+from the locally built orchestrator image, which bakes the stdlib-only
+control-plane source. A pull-from-registry mode (Gitea's OCI registry) becomes
+the default later.
+
+SSH is left enabled for debugging; the control plane is the load-bearing
+surface.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import os
+import shlex
+import signal
+import stat
+import subprocess
+import time
+import urllib.error
+import urllib.request
+from dataclasses import dataclass
+from pathlib import Path
+
+from ...log import die, info
+from ..docker import util as docker_mod
+from ..docker.gateway_provision import GatewayProvisionError
+from . import firecracker_vm, netpool, util
+
+# The single infra-VM image: gateway data plane + baked control-plane source
+# (Dockerfile.infra FROM the gateway image). Built from source by default;
+# a pull-from-registry mode lands later.
+_INFRA_IMAGE = "bot-bottle-infra:latest"
+_GATEWAY_IMAGE = "bot-bottle-gateway:latest"
+_ORCHESTRATOR_IMAGE = "bot-bottle-orchestrator:latest"
+_REPO_ROOT = Path(__file__).resolve().parents[3]
+
+CONTROL_PLANE_PORT = 8099
+# Gateway data-plane ports (agent-facing): egress proxy, supervise MCP,
+# git-http. Reached by agent VMs over VM-to-VM routing (added next).
+EGRESS_PORT = 9099
+SUPERVISE_PORT = 9100
+GIT_HTTP_PORT = 9420
+# mitmproxy writes its CA here a beat after start; agents install it to trust
+# the gateway's TLS interception.
+_GATEWAY_CA_PATH = "/home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem"
+
+# The infra VM makes direct upstream connections (gateway egress, and buildah
+# during builds), and the kernel `ip=` cmdline sets no resolver. Public for
+# now; routing DNS through a filtered path is a later refinement.
+_INFRA_RESOLVER = "1.1.1.1"
+
+_HEALTH_TIMEOUT_SECONDS = 45.0
+_HEALTH_POLL_SECONDS = 0.5
+_CA_TIMEOUT_SECONDS = 30.0
+
+
+@dataclass
+class InfraVm:
+    """A handle to the per-host infra VM: its guest IP and the stable SSH key
+    used to fetch the gateway CA / provision git-gate. `vm` is the live VMM
+    handle when this process booted it, and None when adopting a singleton a
+    prior launcher started (teardown then goes through the PID file)."""
+
+    guest_ip: str
+    private_key: Path
+    vm: firecracker_vm.VmHandle | None = None
+
+    @property
+    def control_plane_url(self) -> str:
+        return f"http://{self.guest_ip}:{CONTROL_PLANE_PORT}"
+
+    def terminate(self) -> None:
+        """Stop the infra VM — via the live handle if we booted it, else the
+        PID file (adopting-process case)."""
+        if self.vm is not None:
+            self.vm.terminate()
+        else:
+            _kill_pidfile()
+        _pid_file().unlink(missing_ok=True)
+
+    def gateway_ca_pem(self, *, timeout: float = _CA_TIMEOUT_SECONDS) -> str:
+        """The gateway's mitmproxy CA (PEM) that agents install to trust its
+        TLS interception. Generated a moment after boot, so this polls over
+        SSH until it appears (mirrors DockerGateway.ca_cert_pem)."""
+        deadline = time.monotonic() + timeout
+        while True:
+            proc = subprocess.run(
+                util.ssh_base_argv(self.private_key, self.guest_ip)
+                + [f"cat {_GATEWAY_CA_PATH}"],
+                capture_output=True, text=True, timeout=15, check=False,
+            )
+            if proc.returncode == 0 and "BEGIN CERTIFICATE" in proc.stdout:
+                return proc.stdout
+            if time.monotonic() >= deadline:
+                die(f"gateway CA not available after {timeout:g}s: "
+                    f"{proc.stderr.strip() or 'empty'}")
+            time.sleep(_HEALTH_POLL_SECONDS)
+
+
+def ensure_built() -> None:
+    """Build the infra image from source (bootstrap via host docker). The
+    infra image `COPY --from`s the orchestrator image and is `FROM` the
+    gateway image, so both must exist first. A pull-from-registry mode
+    replaces this later."""
+    docker_mod.build_image(
+        _ORCHESTRATOR_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.orchestrator")
+    docker_mod.build_image(
+        _GATEWAY_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.gateway")
+    docker_mod.build_image(
+        _INFRA_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.infra")
+
+
+def build_infra_rootfs_dir() -> Path:
+    """The infra VM's base rootfs: the infra image prepared with the
+    control-plane + gateway init as PID 1. The init's content is folded into
+    the cache key so an init change rebuilds the rootfs (the base image digest
+    alone wouldn't catch it)."""
+    init = _infra_init()
+    tag = hashlib.sha256(init.encode()).hexdigest()[:8]
+    return util.build_base_rootfs_dir(
+        _INFRA_IMAGE, variant=f"-infra-{tag}", init_script=init,
+    )
+
+
+def ensure_running() -> InfraVm:
+    """Idempotent per-host singleton. Adopt the infra VM if its control plane
+    is already healthy (a prior launcher booted it — it outlives short-lived
+    `start` processes); otherwise clear any stale VM and boot a fresh one.
+    Returns a handle usable for CA fetch / git-gate provisioning."""
+    slot = netpool.orch_slot()
+    url = f"http://{slot.guest_ip}:{CONTROL_PLANE_PORT}"
+    key = _infra_dir() / "id_ed25519"
+    if key.exists() and _health_ok(url):
+        info(f"adopting running infra VM at {url}")
+        return InfraVm(guest_ip=slot.guest_ip, private_key=key)
+
+    stop()  # clear a stale/hung VM holding the link before booting fresh
+    ensure_built()
+    infra = boot()
+    wait_for_health(infra)
+    return infra
+
+
+def stop() -> None:
+    """Stop the infra VM singleton (idempotent — absent is success)."""
+    _kill_pidfile()
+    _pid_file().unlink(missing_ok=True)
+
+
+def boot() -> InfraVm:
+    """Boot the infra VM (detached, so it outlives the launcher) on the
+    orchestrator link, recording its PID. Prefer `ensure_running`."""
+    slot = netpool.orch_slot()
+    if not netpool.tap_present(slot.iface):
+        die(f"orchestrator link {slot.iface} not present.\n"
+            f"    ./cli.py backend setup --backend=firecracker")
+
+    base = build_infra_rootfs_dir()
+    run_dir = _infra_dir()
+    rootfs = run_dir / "rootfs.ext4"
+    util.build_rootfs_ext4(base, rootfs, slack_mib=8192)
+    private_key, pubkey = _stable_keypair()
+
+    info(f"booting infra VM on {slot.iface} (guest {slot.guest_ip})")
+    vm = firecracker_vm.boot(
+        name="bot-bottle-infra", rootfs=rootfs, tap=slot.iface,
+        guest_ip=slot.guest_ip, host_ip=slot.host_ip, pubkey=pubkey,
+        run_dir=run_dir, mem_mib=4096, detached=True,
+        data_drive=_ensure_registry_volume(),
+    )
+    _pid_file().write_text(str(vm.process.pid))
+    return InfraVm(guest_ip=slot.guest_ip, private_key=private_key, vm=vm)
+
+
+def _infra_dir() -> Path:
+    d = util.cache_dir() / "infra"
+    d.mkdir(parents=True, exist_ok=True)
+    return d
+
+
+def _pid_file() -> Path:
+    return _infra_dir() / "vm.pid"
+
+
+# The registry "volume": a host-side ext4 file attached to the infra VM as a
+# second virtio-block device (guest /dev/vdb), mounted at the control plane's
+# DB dir. It outlives the ephemeral rootfs, so the bottle registry survives an
+# infra-VM restart — the firecracker analogue of a docker volume. It is a
+# plain ext4 file: `sudo mount -o loop <path>` on the host (with the VM
+# stopped) to inspect bot-bottle.db directly.
+_REGISTRY_SIZE = "512M"
+
+
+def registry_volume_path() -> Path:
+    return _infra_dir() / "registry.ext4"
+
+
+def _ensure_registry_volume() -> Path:
+    """Create the empty ext4 registry volume on first use; reuse it after."""
+    vol = registry_volume_path()
+    if vol.exists():
+        return vol
+    info(f"creating infra registry volume {vol} ({_REGISTRY_SIZE})")
+    proc = subprocess.run(
+        ["mke2fs", "-q", "-t", "ext4", "-F", str(vol), _REGISTRY_SIZE],
+        capture_output=True, text=True, check=False,
+    )
+    if proc.returncode != 0:
+        vol.unlink(missing_ok=True)
+        die(f"creating registry volume failed: {proc.stderr.strip()}")
+    return vol
+
+
+def _stable_keypair() -> tuple[Path, str]:
+    """The infra VM's SSH keypair — generated once and reused, so any later
+    launcher can SSH in (fetch CA / provision) even though a different process
+    booted the VM. The pubkey is re-injected on every boot via the cmdline."""
+    d = _infra_dir()
+    key, pub = d / "id_ed25519", d / "id_ed25519.pub"
+    if key.exists() and pub.exists():
+        return key, pub.read_text().strip()
+    key.unlink(missing_ok=True)
+    pub.unlink(missing_ok=True)
+    subprocess.run(
+        ["ssh-keygen", "-t", "ed25519", "-N", "", "-q", "-f", str(key),
+         "-C", "bot-bottle-infra"],
+        check=True,
+    )
+    return key, pub.read_text().strip()
+
+
+def _kill_pidfile() -> None:
+    """SIGTERM (then SIGKILL) the recorded infra VMM, if it's still ours.
+    Guards against a recycled PID by checking the process is firecracker."""
+    try:
+        pid = int(_pid_file().read_text().strip())
+    except (OSError, ValueError):
+        return
+    try:
+        comm = Path(f"/proc/{pid}/comm").read_text().strip()
+    except OSError:
+        return  # already gone
+    if comm != "firecracker":
+        return  # PID recycled by an unrelated process
+    try:
+        os.kill(pid, signal.SIGTERM)
+        for _ in range(50):
+            if not Path(f"/proc/{pid}").exists():
+                return
+            time.sleep(0.1)
+        os.kill(pid, signal.SIGKILL)
+    except OSError:
+        pass
+
+
+def _health_ok(url: str) -> bool:
+    try:
+        with urllib.request.urlopen(f"{url}/health", timeout=1.0) as resp:
+            return resp.status == 200
+    except (urllib.error.URLError, TimeoutError, OSError):
+        return False
+
+
+class SshGatewayTransport:
+    """`GatewayTransport` for the gateway running in the infra VM — the docker
+    exec/cp equivalents over SSH (dropbear + the stable infra key)."""
+
+    def __init__(self, private_key: Path, guest_ip: str) -> None:
+        self._key = private_key
+        self._ip = guest_ip
+
+    def exec(self, argv: list[str]) -> None:
+        proc = subprocess.run(
+            util.ssh_base_argv(self._key, self._ip) + [shlex.join(argv)],
+            capture_output=True, text=True, timeout=60, check=False,
+        )
+        if proc.returncode != 0:
+            raise GatewayProvisionError(
+                f"infra gateway exec {argv!r} failed: {proc.stderr.strip()}")
+
+    def cp_into(self, src: str, dest: str) -> None:
+        # Preserve the source mode (docker cp does): the access-hook is staged
+        # 0700 and git-http execs it directly — a plain `cat >` would land it
+        # 0644 and the exec fails with EACCES; keys stay 0600.
+        mode = stat.S_IMODE(os.stat(src).st_mode)
+        q = shlex.quote(dest)
+        proc = subprocess.run(
+            util.ssh_base_argv(self._key, self._ip)
+            + [f"cat > {q} && chmod {mode:o} {q}"],
+            input=Path(src).read_bytes(), capture_output=True, timeout=30, check=False,
+        )
+        if proc.returncode != 0:
+            raise GatewayProvisionError(
+                f"infra gateway cp {src} -> {dest} failed: "
+                f"{proc.stderr.decode(errors='replace').strip()}")
+
+
+def gateway_transport() -> SshGatewayTransport:
+    """git-gate provisioning transport for the gateway in the infra VM, built
+    from the stable key + the orchestrator link's guest IP. Needs no live VM
+    handle, so teardown can use it too."""
+    return SshGatewayTransport(
+        _infra_dir() / "id_ed25519", netpool.orch_slot().guest_ip)
+
+
+def wait_for_health(
+    infra: InfraVm, *, timeout: float = _HEALTH_TIMEOUT_SECONDS,
+) -> None:
+    """Poll the control plane's /health until it answers 200 or the deadline
+    passes. Dies (with the console tail) if the VMM exits early."""
+    url = f"{infra.control_plane_url}/health"
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        if infra.vm is not None and not infra.vm.is_alive():
+            die(f"infra VM exited during boot (rc={infra.vm.process.returncode}).\n"
+                f"{firecracker_vm._console_tail(infra.vm.console_log)}")
+        try:
+            with urllib.request.urlopen(url, timeout=1.0) as resp:
+                if resp.status == 200:
+                    info(f"infra control plane healthy at {infra.control_plane_url}")
+                    return
+        except (urllib.error.URLError, TimeoutError, OSError):
+            pass
+        time.sleep(_HEALTH_POLL_SECONDS)
+    tail = (firecracker_vm._console_tail(infra.vm.console_log)
+            if infra.vm is not None else "")
+    die(f"infra control plane at {url} did not become healthy within "
+        f"{timeout:.0f}s.\n{tail}")
+
+
+def _infra_init() -> str:
+    """PID-1 init for the infra VM: mount the pseudo-filesystems, wire a
+    resolver, start dropbear (debug SSH), then launch the control plane and
+    the gateway data plane (multi-tenant against the local control plane)."""
+    return f"""#!/bin/sh
+# bot-bottle Firecracker infra VM init (PID 1).
+mount -t proc proc /proc 2>/dev/null
+mount -t sysfs sys /sys 2>/dev/null
+mount -t devtmpfs dev /dev 2>/dev/null
+mkdir -p /dev/pts && mount -t devpts devpts /dev/pts 2>/dev/null
+mount -o remount,rw / 2>/dev/null
+
+# Export a real PATH: a bare-init shell resolves its own execs via a
+# built-in default path, but that isn't in the *environment*, so
+# gateway_init's subprocess daemons (spawned as `python3 ...`) would
+# inherit no PATH and fail to find python3. Export it for all children.
+export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+
+# Direct upstream resolver (control-plane / gateway egress + buildah).
+printf 'nameserver {_INFRA_RESOLVER}\\n' > /etc/resolv.conf 2>/dev/null
+
+# Debug SSH: install the per-boot pubkey from the kernel cmdline.
+KEY=$(sed -n 's/.*bb_pubkey=\\([^ ]*\\).*/\\1/p' /proc/cmdline | base64 -d 2>/dev/null)
+if [ -n "$KEY" ]; then
+    mkdir -p /root/.ssh
+    printf '%s\\n' "$KEY" > /root/.ssh/authorized_keys
+    chmod 700 /root/.ssh && chmod 600 /root/.ssh/authorized_keys
+fi
+chown -R 0:0 /root 2>/dev/null || true
+mkdir -p /etc/dropbear /run /var/lib/bot-bottle
+
+# Persistent registry volume (second virtio-block device, /dev/vdb) mounted
+# at the control plane's DB dir, so bot-bottle.db survives infra-VM restarts.
+mount -t ext4 /dev/vdb /var/lib/bot-bottle 2>/dev/null || true
+
+/bb-dropbear -R -E -p 22 &
+
+# Control plane. Source is baked at /app; the package is stdlib-only.
+cd /app
+BOT_BOTTLE_ROOT=/var/lib/bot-bottle python3 -m bot_bottle.orchestrator \\
+    --host 0.0.0.0 --port {CONTROL_PLANE_PORT} --broker stub &
+
+# Gateway data plane, multi-tenant: each request resolves source-IP ->
+# policy against the local control plane. The VM backend reaches git over
+# git-http (9420), so the git:// daemon (git-gate, needs a per-bottle
+# entrypoint the consolidated model doesn't use) is left out.
+BOT_BOTTLE_GATEWAY_DAEMONS=egress,git-http,supervise \\
+BOT_BOTTLE_ORCHESTRATOR_URL=http://127.0.0.1:{CONTROL_PLANE_PORT} \\
+    python3 /app/gateway_init.py &
+
+# Reap as PID 1; children are backgrounded, so `wait` blocks.
+while : ; do wait ; done
+"""

bot_bottle/backend/firecracker/launch.py

@@ -1,25 +1,63 @@
-"""Launch flow for the Firecracker backend — temporarily disabled (#385).
+"""Launch flow for the Firecracker backend (PRD 0070, consolidated).
 
-The firecracker backend launched a per-bottle companion container (the
+Per bottle:
-egress / git-gate / supervise data plane) alongside each microVM. That
+  1. build the agent image (docker), export it to a cached ext4 rootfs;
-per-bottle-companion architecture was removed in the companion-container removal;
+  2. ensure the per-host orchestrator + shared gateway are up;
-firecracker's replacement — the consolidated per-host gateway — lands in
+  3. claim a free TAP pool slot (rootless flock);
-its own cutover (#354).
+  4. register the bottle on the orchestrator by the VM's guest IP (the
+     attribution key) and provision its git-gate state into the gateway;
+  5. boot the microVM on that TAP; wait for SSH;
+  6. provision (shared gateway CA, prompt, skills, workspace, git, supervise)
+     over SSH.
 
-Until that lands, launching a firecracker bottle fails closed rather than
+The per-bottle Docker sidecar bundle is gone. The shared gateway handles
-silently running the removed path. `prepare` / `status` / cleanup still
+egress / git-gate / supervise for every VM; Docker's PREROUTING DNAT routes
-work, so `backend status --backend=firecracker` and orphan cleanup are
+the VMs' traffic to it, and the nft table's `ct status dnat accept` rule
-unaffected.
+in the forward chain lets it pass. The VM still sends to `host_tap_ip:PORT`
+— the address its world is, by nft design, limited to.
+
+Isolation is enforced by the operator-provisioned nft table (checked
+fail-closed in preflight): a VM reaches only the sidecar (DNAT'd from
+the host TAP IP) and nothing else.
 """
 
 from __future__ import annotations
 
-from contextlib import contextmanager
+import dataclasses
+import os
+from contextlib import ExitStack, contextmanager
+from pathlib import Path
 from typing import Callable, Generator
 
-from ...log import die
+from ...agent_provider import runtime_for
+from ...bottle_state import (
+    egress_state_dir,
+    git_gate_state_dir,
+    read_committed_image,
+)
+from ...egress import (
+    egress_agent_env_entries,
+    egress_resolve_token_values,
+)
+from ...git_gate import (
+    provision_git_gate_dynamic_keys,
+    revoke_git_gate_provisioned_keys,
+)
+from ...log import info, warn
+from ...supervise import SUPERVISE_PORT
+from ..docker import util as docker_mod
+from ..docker.egress import EGRESS_PORT
+from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
+from . import firecracker_vm, image_builder, isolation_probe, netpool, util
 from .bottle import FirecrackerBottle
 from .bottle_plan import FirecrackerBottlePlan
+from .consolidated_launch import (
+    launch_consolidated,
+    teardown_consolidated,
+)
+
+
+_GIT_HTTP_PORT = 9420
 
 
 @contextmanager
@@ -28,12 +66,188 @@ def launch(
     *,
     provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None],
 ) -> Generator[FirecrackerBottle, None, None]:
-    """Fail closed: the firecracker backend is disabled while its
+    """Build, launch, and provision a Firecracker bottle via the consolidated
-    consolidated (gateway-backed) launch is built in #354."""
+    orchestrator. Teardown on exit."""
-    del plan, provision
+    stack = ExitStack()
-    die(
+    bottle_for_revoke = plan.manifest.bottle
-        "the firecracker backend is temporarily disabled during the "
+    git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
-        "companion-container removal (#385); its consolidated relaunch "
+
-        "lands in #354. Use --backend=docker for now."
+    def teardown() -> None:
+        teardown_exc: BaseException | None = None
+        try:
+            stack.close()
+        except BaseException as exc:  # noqa: W0718 - teardown must continue
+            teardown_exc = exc
+            warn(f"firecracker teardown failed: {exc!r}")
+        revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
+        if teardown_exc is not None:
+            raise teardown_exc
+
+    try:
+        # Step 1: agent rootfs. Built from the Dockerfile inside a Firecracker
+        # builder VM (buildah, no host docker); a committed snapshot is reused
+        # when present. Returns the base dir the per-bottle ext4 is made from.
+        plan, agent_base = _build_agent_base(plan)
+
+        # Step 2: mint the git-gate dynamic (gitea) deploy keys, if any.
+        git_gate_plan = plan.git_gate_plan
+        if git_gate_plan.upstreams:
+            git_gate_plan = provision_git_gate_dynamic_keys(
+                plan.manifest.bottle, git_gate_plan, git_gate_state_dir(plan.slug),
+            )
+
+        # Step 3: claim a TAP slot; the flock is held until teardown.
+        slot, lock = netpool.allocate(plan.slug)
+        stack.callback(lock.close)
+        info(f"firecracker slot {slot.iface}: host={slot.host_ip} "
+             f"guest={slot.guest_ip}")
+
+        # Step 4: register on the orchestrator + provision this bottle's
+        # git-gate state into the shared gateway. The per-bottle egress tokens
+        # are resolved from the host env now and handed to the orchestrator
+        # (in memory) for the gateway to inject — the agent never sees them.
+        # Attribution is by the VM's guest IP (unspoofable via /31 TAP + nft).
+        effective_env = {**os.environ, **plan.agent_provision.provisioned_env}
+        token_values = egress_resolve_token_values(
+            plan.egress_plan.token_env_map, effective_env,
+        )
+        ctx = launch_consolidated(
+            plan.egress_plan, git_gate_plan,
+            guest_ip=slot.guest_ip,
+            image_ref=plan.image,
+            tokens=token_values,
+        )
+        stack.callback(
+            teardown_consolidated, ctx.bottle_id,
+            orchestrator_url=ctx.orchestrator_url,
+        )
+
+        # Step 5: install the SHARED gateway CA (replaces the per-bottle CA).
+        # Write it to a stable host path so the provisioner can copy it over SSH.
+        ca_dir = egress_state_dir(plan.slug) / "gateway-ca"
+        ca_dir.mkdir(parents=True, exist_ok=True)
+        ca_file = ca_dir / "gateway-ca.pem"
+        ca_file.write_text(ctx.gateway_ca_pem)
+        egress_plan = dataclasses.replace(
+            plan.egress_plan,
+            mitmproxy_ca_host_path=ca_file,
+            mitmproxy_ca_cert_only_host_path=ca_file,
+        )
+        # Point the agent's git-gate insteadOf rewrites and supervise MCP URL
+        # at the shared gateway (reached at the slot's host TAP IP — the VM
+        # sends there and Docker DNAT routes to the gateway container).
+        git_gate_url = (
+            f"http://{slot.host_ip}:{_GIT_HTTP_PORT}" if git_gate_plan.upstreams else ""
+        )
+        supervise_url = (
+            f"http://{slot.host_ip}:{SUPERVISE_PORT}/"
+            if plan.supervise_plan is not None else ""
+        )
+        plan = dataclasses.replace(
+            plan,
+            git_gate_plan=git_gate_plan,
+            egress_plan=egress_plan,
+            agent_proxy_url=f"http://{slot.host_ip}:{EGRESS_PORT}",
+            agent_git_gate_url=git_gate_url,
+            agent_supervise_url=supervise_url,
+        )
+
+        # Step 6: build the per-bottle rootfs + SSH key, then boot.
+        run_dir = util.cache_dir() / "run" / plan.slug
+        run_dir.mkdir(parents=True, exist_ok=True)
+        rootfs = run_dir / "rootfs.ext4"
+        util.build_rootfs_ext4(agent_base, rootfs)
+        private_key, pubkey = util.generate_keypair(run_dir)
+
+        vm = firecracker_vm.boot(
+            name=plan.container_name,
+            rootfs=rootfs,
+            tap=slot.iface,
+            guest_ip=slot.guest_ip,
+            host_ip=slot.host_ip,
+            pubkey=pubkey,
+            run_dir=run_dir,
+        )
+        stack.callback(vm.terminate)
+        firecracker_vm.wait_for_ssh(vm, private_key)
+
+        # Authoritative fail-closed egress-boundary check, before the agent
+        # runs: prove the VM cannot reach the host directly.
+        isolation_probe.verify_isolation(private_key, slot.guest_ip)
+
+        bottle = FirecrackerBottle(
+            plan.container_name,
+            private_key=private_key,
+            guest_ip=slot.guest_ip,
+            guest_env=_agent_guest_env(plan, slot.host_ip),
+            agent_command=plan.agent_command,
+            agent_prompt_mode=plan.agent_prompt_mode,
+            agent_provider_template=plan.agent_provider_template,
+            terminal_title=(
+                f"{plan.spec.label} ({plan.spec.agent_name})"
+                if plan.spec.label else plan.spec.agent_name
+            ),
+            terminal_color=plan.spec.color,
+            agent_workdir=plan.workspace_plan.workdir,
+        )
+        bottle.prompt_path = provision(plan, bottle)
+
+        yield bottle
+    finally:
+        teardown()
+
+
+def _build_agent_base(
+    plan: FirecrackerBottlePlan,
+) -> tuple[FirecrackerBottlePlan, Path]:
+    """Produce the agent's base rootfs dir. Primary path: build the Dockerfile
+    inside a Firecracker builder VM (buildah, no host docker), smoke-testing
+    the image before export. A committed snapshot (freeze/migrate) is still
+    exported via the host docker path until that is ported too."""
+    committed = read_committed_image(plan.slug)
+    if committed and docker_mod.image_exists(committed):
+        info(f"using committed image {committed!r}")
+        plan = dataclasses.replace(
+            plan,
+            agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
+        )
+        return plan, util.build_base_rootfs_dir(committed)
+    base = image_builder.build_agent_rootfs_dir(
+        Path(plan.dockerfile_path),
+        image_tag=plan.image,
+        smoke_test=runtime_for(plan.agent_provider_template).smoke_test,
     )
-    yield  # unreachable — `die` raises; keeps this a generator/contextmanager
+    return plan, base
+
+
+# --- agent guest env -------------------------------------------------
+
+def _agent_guest_env(plan: FirecrackerBottlePlan, host_ip: str) -> dict[str, str]:
+    """Env injected into every agent/exec call over SSH. The VM has no
+    baked process env (it just runs init), so the proxy/CA/git/supervise
+    wiring is applied per-invocation."""
+    proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
+    no_proxy = f"localhost,127.0.0.1,{host_ip}"
+    env: dict[str, str] = {
+        "HTTPS_PROXY": proxy_url, "HTTP_PROXY": proxy_url,
+        "https_proxy": proxy_url, "http_proxy": proxy_url,
+        "NO_PROXY": no_proxy, "no_proxy": no_proxy,
+        "NODE_EXTRA_CA_CERTS": AGENT_CA_PATH,
+        "SSL_CERT_FILE": AGENT_CA_BUNDLE,
+        "REQUESTS_CA_BUNDLE": AGENT_CA_BUNDLE,
+    }
+    if plan.agent_git_gate_url:
+        env["GIT_GATE_URL"] = plan.agent_git_gate_url
+    if plan.agent_supervise_url:
+        env["MCP_SUPERVISE_URL"] = plan.agent_supervise_url
+    for entry in egress_agent_env_entries(plan.egress_plan):
+        key, _, value = entry.partition("=")
+        env[key] = value
+    env.update(plan.agent_provision.guest_env)
+    # Forwarded (bare-name) env: resolve host values now, since the VM
+    # can't inherit them from a `docker run --env NAME`.
+    for name in plan.forwarded_env:
+        value = os.environ.get(name)
+        if value is not None:
+            env[name] = value
+    return env

bot_bottle/backend/firecracker/netpool.defaults.env

@@ -15,3 +15,11 @@ BOT_BOTTLE_FC_POOL_SIZE=8
 BOT_BOTTLE_FC_IP_BASE=10.243.0.0
 BOT_BOTTLE_FC_IFACE_PREFIX=bbfc
 BOT_BOTTLE_FC_NFT_TABLE=bot_bottle_fc
+# The orchestrator/gateway VM's own TAP — a dedicated link OUTSIDE the
+# bbfc* agent pool. Unlike agent VMs (which reach only their gateway),
+# the orchestrator is trusted infra that needs real NAT'd internet
+# egress: to FROM-pull + apt/npm during in-VM agent-image builds
+# (buildah) and to forward agent egress upstream (Stage B gateway). Its
+# /31 is the top of the IP_BASE /16 (host x.y.255.0, guest x.y.255.1),
+# clear of the pool near the bottom of the block.
+BOT_BOTTLE_FC_ORCH_IFACE=bborch0

bot_bottle/backend/firecracker/netpool.py

@@ -79,6 +79,12 @@ def _cfg(key: str) -> str:
 IFACE_PREFIX = _cfg("BOT_BOTTLE_FC_IFACE_PREFIX")
 NFT_TABLE = _cfg("BOT_BOTTLE_FC_NFT_TABLE")
 
+# The orchestrator/gateway VM's dedicated TAP — outside the bbfc* agent
+# pool and, unlike it, NAT'd to the internet (see `orch_slot`). The
+# orchestrator is trusted infra: it builds agent images in-VM (buildah
+# needs to FROM-pull + apt/npm) and forwards agent egress upstream.
+ORCH_IFACE = _cfg("BOT_BOTTLE_FC_ORCH_IFACE")
+
 
 def pool_size() -> int:
     return int(_cfg("BOT_BOTTLE_FC_POOL_SIZE"))
@@ -123,6 +129,25 @@ def all_slots() -> list[Slot]:
     return [slot(i) for i in range(pool_size())]
 
 
+def orch_slot() -> Slot:
+    """The orchestrator/gateway VM's dedicated link — its own TAP
+    (`ORCH_IFACE`) on a /31 at the TOP of the IP_BASE /16 (host
+    x.y.255.0, guest x.y.255.1), well clear of the agent pool near the
+    bottom of the block. Unlike a pool `Slot`, this link is NAT'd out to
+    the internet by the setup (the orchestrator is trusted infra), so it
+    is deliberately *not* one of the isolated `bbfc*` slots.
+
+    `index` is -1 (sentinel: not a pool index)."""
+    base16 = int(ipaddress.IPv4Address(ip_base())) & 0xFFFF0000
+    host = base16 + 0xFF00
+    return Slot(
+        index=-1,
+        iface=ORCH_IFACE,
+        host_ip=str(ipaddress.IPv4Address(host)),
+        guest_ip=str(ipaddress.IPv4Address(host + 1)),
+    )
+
+
 # --- fail-closed verification ---------------------------------------
 
 def _run_ok(argv: list[str]) -> bool:

bot_bottle/backend/firecracker/util.py

@@ -159,15 +159,22 @@ def docker_image_id(ref: str) -> str:
     return result.stdout.strip().replace("sha256:", "")[:16]
 
 
-def build_base_rootfs_dir(image_ref: str) -> Path:
+def build_base_rootfs_dir(
-    """Export the agent image's filesystem and inject the guest init +
+    image_ref: str, *, variant: str = "", init_script: str | None = None,
-    static dropbear. Cached by image digest — the per-bottle bits
+) -> Path:
+    """Export the image's filesystem and inject the guest init + static
+    dropbear. Cached by image digest — the per-bottle bits
     (authorized_keys, IP) are passed at boot via the kernel cmdline, so
     this tree carries nothing bottle-specific and is safely shared.
 
+    `variant` suffixes the cache key so the same image can be prepared
+    with a different `init_script` (e.g. the infra VM boots the same
+    orchestrator image as the builder but runs the control plane as
+    PID 1, not the SSH-only agent init) without a cache collision.
+
     Returns the prepared directory (read as the `mke2fs -d` source)."""
     digest = docker_image_id(image_ref)
-    base = cache_dir() / "rootfs" / digest
+    base = cache_dir() / "rootfs" / f"{digest}{variant}"
     ready = base / ".bb-ready"
     if ready.is_file():
         return base
@@ -200,17 +207,19 @@ def build_base_rootfs_dir(image_ref: str) -> Path:
             stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
         )
 
-    _inject_guest_boot(base)
+    inject_guest_boot(base, init_script=init_script)
     ready.write_text("ok\n")
     return base
 
 
-def _inject_guest_boot(rootfs: Path) -> None:
+def inject_guest_boot(rootfs: Path, init_script: str | None = None) -> None:
-    """Drop the static dropbear and the PID-1 init into the rootfs."""
+    """Drop the static dropbear and the PID-1 init into the rootfs.
+    `init_script` defaults to the SSH-only agent init; the infra VM
+    passes its own (control plane + gateway) init."""
     shutil.copy2(dropbear_path(), rootfs / "bb-dropbear")
     os.chmod(rootfs / "bb-dropbear", 0o755)
     init = rootfs / "bb-init"
-    init.write_text(_GUEST_INIT)
+    init.write_text(init_script or _GUEST_INIT)
     os.chmod(init, 0o755)
 
 

bot_bottle/backend/macos_container/util.py

@@ -60,13 +60,21 @@ def dns_server() -> str:
 
 
 def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
-    """Build an OCI image with Apple's BuildKit-backed `container build`."""
+    """Build an OCI image with Apple's BuildKit-backed `container build`.
+
+    Set `BOT_BOTTLE_NO_CACHE=1` (the `start --no-cache` flag) to force
+    `--no-cache`. The npm/curl installers some provider Dockerfiles
+    shell out to can silently no-op on a transient network failure —
+    e.g. an `optionalDependencies` fetch for a platform-native binary —
+    and the builder will then cache that broken layer indefinitely."""
     info(
         f"building image {ref} from {context} with Apple Container "
         "(layer cache keeps repeat builds fast)"
     )
     _ensure_builder_dns()
     args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
+    if os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
+        args.append("--no-cache")
     if dockerfile:
         # `container build` resolves -f relative to the current working
         # directory, not the build context. Anchor a relative Dockerfile to
@@ -78,6 +86,28 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
     subprocess.run(args, check=True)
 
 
+def verify_agent_image(image: str, argv: tuple[str, ...]) -> None:
+    """Run `argv` inside a throwaway container of a freshly built agent
+    image and die loudly if it fails, instead of shipping an image
+    whose CLI only breaks at first real use. No-op when the provider
+    hasn't declared a smoke test (`AgentProviderRuntime.smoke_test`)."""
+    if not argv:
+        return
+    result = subprocess.run(
+        [_CONTAINER, "run", "--rm", "--entrypoint", argv[0], image, *argv[1:]],
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    if result.returncode != 0:
+        detail = (result.stderr or result.stdout or "").strip()
+        die(
+            f"agent image {image!r} failed its post-build smoke test "
+            f"({' '.join(argv)}): {detail}\n"
+            f"Try rebuilding from scratch: bot-bottle start --no-cache"
+        )
+
+
 def commit_container(container_name: str, image_tag: str) -> None:
     """Snapshot a running Apple Container as a local image.
 

bot_bottle/cli/start.py

@@ -46,6 +46,17 @@ def cmd_start(argv: list[str]) -> int:
     parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
     parser.add_argument("--dry-run", action="store_true")
     parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
+    parser.add_argument(
+        "--no-cache",
+        action="store_true",
+        help=(
+            "rebuild agent/sidecar images from scratch, bypassing the "
+            "build layer cache. Use when an image looks broken after a "
+            "dependency bump — e.g. an installer's optionalDependencies "
+            "fetch silently no-op'd on a transient failure and got baked "
+            "into a cached layer."
+        ),
+    )
     parser.add_argument(
         "--backend",
         choices=known_backend_names(),
@@ -97,6 +108,11 @@ def cmd_start(argv: list[str]) -> int:
     args = parser.parse_args(argv)
 
     dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
+    if args.no_cache or os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
+        # Read by build_image() in each backend's util module — set here
+        # so both the interactive and --headless paths pick it up without
+        # threading a no_cache field through every backend's plan dataclass.
+        os.environ["BOT_BOTTLE_NO_CACHE"] = "1"
 
     manifest = ManifestIndex.resolve(USER_CWD)
     backend_name: str | None = args.backend

bot_bottle/contrib/claude/agent_provider.py

@@ -91,6 +91,7 @@ _RUNTIME = AgentProviderRuntime(
     prompt_mode="append_file",
     bypass_args=("--dangerously-skip-permissions",),
     resume_args=("--continue",),
+    smoke_test=("claude", "--version"),
 )
 
 

bot_bottle/contrib/codex/agent_provider.py

@@ -61,6 +61,7 @@ _RUNTIME = AgentProviderRuntime(
     prompt_mode="read_prompt_file",
     bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
     resume_args=("resume", "--last"),
+    smoke_test=(_CODEX_CLI, "--version"),
 )
 
 

bot_bottle/orchestrator/gateway.py

@@ -100,6 +100,7 @@ class DockerGateway(Gateway):
         orchestrator_url: str = "",
         build_context: Path | None = None,
         dockerfile: str | None = GATEWAY_DOCKERFILE,
+        host_port_bindings: tuple[int, ...] = (),
     ) -> None:
         self.image_ref = image_ref
         self.name = name
@@ -110,6 +111,10 @@ class DockerGateway(Gateway):
         self._orchestrator_url = orchestrator_url
         self._build_context = build_context or _REPO_ROOT
         self._dockerfile = dockerfile
+        # Ports published on the host (0.0.0.0). Used by the Firecracker
+        # backend's dev-harness gateway so VMs can reach it via their TAP link;
+        # Docker's DNAT + the nft `ct status dnat accept` rule handle the rest.
+        self._host_port_bindings = host_port_bindings
 
     def image_exists(self) -> bool:
         return run_docker(["docker", "image", "inspect", self.image_ref]).returncode == 0
@@ -188,6 +193,8 @@ class DockerGateway(Gateway):
             # trust it) — see GATEWAY_CA_VOLUME.
             "--volume", f"{GATEWAY_CA_VOLUME}:{MITMPROXY_HOME}",
         ]
+        for port in self._host_port_bindings:
+            argv += ["--publish", f"0.0.0.0:{port}:{port}"]
         if self._orchestrator_url:
             # Makes the gateway's egress / git / supervise daemons multi-tenant:
             # each request resolves source-IP -> policy against the control plane.

bot_bottle/orchestrator/lifecycle.py

@@ -78,7 +78,13 @@ def _source_hash(repo_root: Path) -> str:
 
 class OrchestratorService:
     """Manages the orchestrator control-plane container + the shared gateway.
-    Callers only need `ensure_running()` + `url`."""
+    Callers only need `ensure_running()` + `url`.
+
+    `orchestrator_name` / `orchestrator_label` let backends run independent
+    orchestrators on the same host without name collisions (e.g. the
+    Firecracker backend uses `bot-bottle-fc-orchestrator` alongside the Docker
+    backend's `bot-bottle-orchestrator`). Subclass and override `_gateway()`
+    to supply a backend-specific gateway variant."""
 
     def __init__(
         self,
@@ -89,6 +95,8 @@ class OrchestratorService:
         gateway_image: str = GATEWAY_IMAGE,
         repo_root: Path = _REPO_ROOT,
         host_root: Path | None = None,
+        orchestrator_name: str = ORCHESTRATOR_NAME,
+        orchestrator_label: str = ORCHESTRATOR_LABEL,
     ) -> None:
         self.port = port
         self.network = network
@@ -100,6 +108,8 @@ class OrchestratorService:
         self._gateway_image = gateway_image
         self._repo_root = repo_root
         self._host_root = host_root or bot_bottle_root()
+        self._orchestrator_name = orchestrator_name
+        self._orchestrator_label = orchestrator_label
 
     @property
     def url(self) -> str:
@@ -111,7 +121,7 @@ class OrchestratorService:
         """Control-plane URL as the gateway container reaches it — by name over
         docker DNS on the shared network. This is the gateway's
         BOT_BOTTLE_ORCHESTRATOR_URL."""
-        return f"http://{ORCHESTRATOR_NAME}:{self.port}"
+        return f"http://{self._orchestrator_name}:{self.port}"
 
     def is_healthy(self, *, timeout: float = _HEALTH_REQUEST_TIMEOUT_SECONDS) -> bool:
         try:
@@ -129,11 +139,11 @@ class OrchestratorService:
         fixed-name container first). Register-only broker → no docker socket.
         Labels the container with `source_hash` so a later `ensure_running`
         can detect a real code change (see `_source_hash`)."""
-        run_docker(["docker", "rm", "--force", ORCHESTRATOR_NAME])
+        run_docker(["docker", "rm", "--force", self._orchestrator_name])
         proc = run_docker([
             "docker", "run", "--detach",
-            "--name", ORCHESTRATOR_NAME,
+            "--name", self._orchestrator_name,
-            "--label", ORCHESTRATOR_LABEL,
+            "--label", self._orchestrator_label,
             "--label", f"{ORCHESTRATOR_SOURCE_HASH_LABEL}={source_hash}",
             "--network", self.network,
             # Host CLI reaches the control plane here; bound to loopback so it
@@ -185,12 +195,12 @@ class OrchestratorService:
         *current* bind-mounted source. Mirrors `DockerGateway`'s
         image-staleness check, but by content hash rather than image id since
         the orchestrator runs bind-mounted source, not a built image."""
-        if not self._container_running(ORCHESTRATOR_NAME):
+        if not self._container_running(self._orchestrator_name):
             return False
         proc = run_docker([
             "docker", "inspect", "--format",
             "{{ index .Config.Labels \"" + ORCHESTRATOR_SOURCE_HASH_LABEL + "\" }}",
-            ORCHESTRATOR_NAME,
+            self._orchestrator_name,
         ])
         if proc.returncode != 0:
             return True  # can't compare -> don't churn a working container
@@ -219,7 +229,10 @@ class OrchestratorService:
             return self.url
 
         self._ensure_orchestrator_image()
-        log.info("starting orchestrator container", context={"name": ORCHESTRATOR_NAME})
+        log.info(
+            "starting orchestrator container",
+            context={"name": self._orchestrator_name},
+        )
         self._run_orchestrator_container(current_hash)
 
         deadline = time.monotonic() + startup_timeout
@@ -234,7 +247,7 @@ class OrchestratorService:
 
     def stop(self) -> None:
         """Remove the orchestrator + gateway containers (idempotent)."""
-        run_docker(["docker", "rm", "--force", ORCHESTRATOR_NAME])
+        run_docker(["docker", "rm", "--force", self._orchestrator_name])
         self._gateway().stop()
 
 

nix/firecracker-netpool.nix

@@ -49,10 +49,12 @@ let
   # /31 alignment == an even final octet (only bit 0 matters for base+2i).
   lastOctet = lib.toInt (lib.last (lib.splitString "." cfg.ipBase));
 
-  # The script needs ip/nft/sysctl + the usual coreutils. It gets every
+  # The script needs ip/nft/sysctl + the usual coreutils, plus iptables
-  # pool value via the unit's Environment=, so it never reads the shared
+  # for the orchestrator link's DOCKER-USER accept (best-effort; skipped
-  # defaults file (which isn't beside it once copied to the store).
+  # when Docker is absent). It gets every pool value via the unit's
-  runtimePath = with pkgs; [ iproute2 nftables procps coreutils gnused ];
+  # Environment=, so it never reads the shared defaults file (which isn't
+  # beside it once copied to the store).
+  runtimePath = with pkgs; [ iproute2 nftables iptables procps coreutils gnused ];
 
   ownEnv =
     if cfg.group != null
@@ -64,6 +66,7 @@ let
     BOT_BOTTLE_FC_IP_BASE = cfg.ipBase;
     BOT_BOTTLE_FC_IFACE_PREFIX = cfg.ifacePrefix;
     BOT_BOTTLE_FC_NFT_TABLE = cfg.tableName;
+    BOT_BOTTLE_FC_ORCH_IFACE = cfg.orchIface;
   } // ownEnv;
 in
 {
@@ -127,6 +130,19 @@ in
       description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
     };
 
+    orchIface = lib.mkOption {
+      type = lib.types.str;
+      default = defaults.BOT_BOTTLE_FC_ORCH_IFACE;
+      defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
+      description = ''
+        TAP name for the orchestrator/gateway VM's dedicated link. Unlike
+        the isolated bbfc* agent pool, this link is NAT'd to the internet
+        (the orchestrator is trusted infra that builds agent images in-VM
+        and forwards agent egress upstream). Must match
+        BOT_BOTTLE_FC_ORCH_IFACE / netpool.ORCH_IFACE.
+      '';
+    };
+
     writeEnvFile = lib.mkOption {
       type = lib.types.bool;
       default = false;
@@ -176,6 +192,7 @@ in
         BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
         BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
         BOT_BOTTLE_FC_NFT_TABLE=${cfg.tableName}
+        BOT_BOTTLE_FC_ORCH_IFACE=${cfg.orchIface}
       '';
     };
   };

scripts/firecracker-netpool.sh

@@ -63,12 +63,13 @@ POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-$(_default BOT_BOTTLE_FC_POOL_SIZE)}"
 IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-$(_default BOT_BOTTLE_FC_IP_BASE)}"
 PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-$(_default BOT_BOTTLE_FC_IFACE_PREFIX)}"
 TABLE="${BOT_BOTTLE_FC_NFT_TABLE:-$(_default BOT_BOTTLE_FC_NFT_TABLE)}"
+ORCH_IFACE="${BOT_BOTTLE_FC_ORCH_IFACE:-$(_default BOT_BOTTLE_FC_ORCH_IFACE)}"
 OWNER="${BOT_BOTTLE_FC_OWNER:-${SUDO_USER:-$USER}}"
 GROUP="${BOT_BOTTLE_FC_GROUP:-}"
 
 # Fail loudly rather than provisioning a half/empty range if a value
 # resolved to nothing (env unset AND the shared file unreadable).
-for _v in POOL_SIZE IP_BASE PREFIX TABLE; do
+for _v in POOL_SIZE IP_BASE PREFIX TABLE ORCH_IFACE; do
     [ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; }
 done
 
@@ -89,6 +90,13 @@ host_ip() { _int_to_ip $(( $(_ip_to_int "$IP_BASE") + 2*$1 )); }
 guest_ip() { _int_to_ip $(( $(_ip_to_int "$IP_BASE") + 2*$1 + 1 )); }
 iface()   { echo "${PREFIX}$1"; }
 
+# Orchestrator/gateway VM link: a /31 at the TOP of the IP_BASE /16
+# (host x.y.255.0, guest x.y.255.1), well clear of the agent pool near
+# the bottom of the block. Must match netpool.py:orch_slot().
+_orch_base() { echo $(( ($(_ip_to_int "$IP_BASE") & 0xFFFF0000) + 0xFF00 )); }
+orch_host()  { _int_to_ip "$(_orch_base)"; }
+orch_guest() { _int_to_ip $(( $(_orch_base) + 1 )); }
+
 require_root() {
     if [ "$(id -u)" -ne 0 ]; then
         echo "error: '$1' needs root (run under sudo)" >&2
@@ -125,8 +133,21 @@ cmd_up() {
         echo "  $dev  host=$host  guest=$(guest_ip "$i")  $own_desc"
     done
 
+    # The orchestrator/gateway VM's dedicated link. Same rootless-open
+    # ownership as the pool, but NAT'd to the internet (below) — it is
+    # trusted infra, not an isolated agent slot.
+    ip link show "$ORCH_IFACE" >/dev/null 2>&1 \
+        || ip tuntap add dev "$ORCH_IFACE" mode tap "${own_args[@]}"
+    ip addr replace "$(orch_host)/31" dev "$ORCH_IFACE"
+    ip link set "$ORCH_IFACE" up
+    echo "  $ORCH_IFACE  host=$(orch_host)  guest=$(orch_guest)  (NAT'd egress)  $own_desc"
+
     _install_nft
     echo "nftables table inet $TABLE installed (fail-closed boundary)"
+    _install_orch_egress
+    echo "orchestrator egress installed ($ORCH_IFACE -> NAT out)"
+    _install_gateway_route
+    echo "agent->gateway route installed (${PREFIX}* :$GATEWAY_PORTS -> $(orch_guest))"
     echo "done."
 }
 
@@ -142,7 +163,12 @@ _install_nft() {
     #   input: a VM never needs host-local delivery (its gateway is
     #     reached via DNAT->forward), so drop all direct input from VMs
     #     -> host services bound on 0.0.0.0 are unreachable from the VM.
+    # Delete-first (create empty, delete, recreate) so a re-applied `up`
+    # lands identical state instead of appending rules / erroring on the
+    # existing base chains — the setup is idempotent regardless of history.
     nft -f - <<EOF
+table inet $TABLE {}
+delete table inet $TABLE
 table inet $TABLE {
     chain forward {
         type filter hook forward priority -10; policy accept;
@@ -161,8 +187,87 @@ table inet $TABLE {
 EOF
 }
 
+# Give the orchestrator/gateway VM real internet egress (agent VMs get
+# none — that's the isolation table above). Three parts, because the
+# path must work both during bootstrap (Docker still present) and after
+# Docker is removed:
+#   * masquerade   — SNAT the orch guest /31 out the host uplink so its
+#                    RFC-1918 address can reach the internet.
+#   * nft forward  — accept the orch link's forward path (load-bearing
+#                    on a pure-nft host whose FORWARD policy drops; a
+#                    harmless no-op where forwarding is already open).
+#                    It never drops, so it can't weaken the isolation
+#                    table's agent drops.
+#   * DOCKER-USER  — during bootstrap Docker's FORWARD chain policy is
+#                    DROP; its sanctioned DOCKER-USER hook is the only
+#                    place a user ACCEPT survives. Best-effort + guarded
+#                    (skipped once Docker is gone).
+_install_orch_egress() {
+    nft -f - <<EOF
+table inet ${TABLE}_nat {}
+delete table inet ${TABLE}_nat
+table inet ${TABLE}_nat {
+    chain forward {
+        type filter hook forward priority -10; policy accept;
+        iifname "$ORCH_IFACE" accept
+        oifname "$ORCH_IFACE" ct state established,related accept
+    }
+    chain postrouting {
+        type nat hook postrouting priority 100; policy accept;
+        ip saddr $(orch_guest) oifname != "$ORCH_IFACE" masquerade
+    }
+}
+EOF
+    _docker_user_orch add
+}
+
+# Insert (add) or delete (del) the DOCKER-USER ACCEPT rules for the
+# orchestrator link, idempotently, only when the chain exists.
+_docker_user_orch() {
+    local op="$1" flag
+    command -v iptables >/dev/null 2>&1 || return 0
+    iptables -t filter -L DOCKER-USER >/dev/null 2>&1 || return 0
+    for flag in "-i" "-o"; do
+        if [ "$op" = add ]; then
+            iptables -C DOCKER-USER "$flag" "$ORCH_IFACE" -j ACCEPT 2>/dev/null \
+                || iptables -I DOCKER-USER "$flag" "$ORCH_IFACE" -j ACCEPT
+        else
+            iptables -D DOCKER-USER "$flag" "$ORCH_IFACE" -j ACCEPT 2>/dev/null || true
+        fi
+    done
+}
+
+# Agent -> gateway VM routing. The shared gateway (egress / supervise /
+# git-http) runs in the orchestrator/infra VM at $(orch_guest). Agents keep
+# addressing their own host-side TAP IP on the gateway ports; a PREROUTING
+# DNAT redirects that to the infra VM, and the isolation table's
+# `ct status dnat accept` forward rule lets it through — every other agent
+# egress stays dropped. Source IP is deliberately NOT masqueraded: the
+# gateway attributes each request to the originating bottle by its (nft +
+# /31 unspoofable) guest IP.
+_install_gateway_route() {
+    nft -f - <<EOF
+table ip ${TABLE}_gw {}
+delete table ip ${TABLE}_gw
+table ip ${TABLE}_gw {
+    chain prerouting {
+        type nat hook prerouting priority -100; policy accept;
+        iifname "${PREFIX}*" tcp dport { $GATEWAY_PORTS } dnat to $(orch_guest)
+    }
+}
+EOF
+}
+
 cmd_down() {
     require_root down
+    _docker_user_orch del
+    nft delete table ip "${TABLE}_gw" 2>/dev/null || true
+    nft delete table inet "${TABLE}_nat" 2>/dev/null || true
+    if ip link show "$ORCH_IFACE" >/dev/null 2>&1; then
+        ip link set "$ORCH_IFACE" down 2>/dev/null || true
+        ip tuntap del dev "$ORCH_IFACE" mode tap 2>/dev/null || true
+        echo "  removed $ORCH_IFACE"
+    fi
     nft delete table inet "$TABLE" 2>/dev/null || true
     for i in $(seq 0 $((POOL_SIZE-1))); do
         local dev ; dev="$(iface "$i")"
@@ -178,6 +283,10 @@ cmd_down() {
 cmd_status() {
     echo "table inet $TABLE:"
     nft list table inet "$TABLE" 2>/dev/null || echo "  (absent)"
+    echo "table inet ${TABLE}_nat (orchestrator egress):"
+    nft list table inet "${TABLE}_nat" 2>/dev/null || echo "  (absent)"
+    echo "table ip ${TABLE}_gw (agent->gateway route):"
+    nft list table ip "${TABLE}_gw" 2>/dev/null || echo "  (absent)"
     echo "taps:"
     for i in $(seq 0 $((POOL_SIZE-1))); do
         local dev ; dev="$(iface "$i")"
@@ -185,6 +294,9 @@ cmd_status() {
             ip -brief addr show "$dev" | sed 's/^/  /'
         fi
     done
+    if ip -brief addr show "$ORCH_IFACE" >/dev/null 2>&1; then
+        ip -brief addr show "$ORCH_IFACE" | sed 's/^/  /'
+    fi
 }
 
 case "${1:-}" in

tests/unit/test_docker_launch_committed_image.py

@@ -91,6 +91,7 @@ class TestLaunchCommittedImage(unittest.TestCase):
         with mock.patch.object(launch_mod, "read_committed_image", return_value=committed_tag), \
                 mock.patch.object(launch_mod.docker_mod, "image_exists", return_value=image_present), \
                 mock.patch.object(launch_mod.docker_mod, "build_image", side_effect=_build), \
+                mock.patch.object(launch_mod.docker_mod, "verify_agent_image"), \
                 mock.patch.object(launch_mod, "launch_consolidated", return_value=_CTX), \
                 mock.patch.object(launch_mod, "teardown_consolidated"), \
                 mock.patch.object(launch_mod, "DockerGateway", return_value=gw), \

tests/unit/test_docker_launch_teardown.py

@@ -94,6 +94,7 @@ class TestTeardownWarning(unittest.TestCase):
         )
 
         with mock.patch.object(launch_mod.docker_mod, "build_image"), \
+             mock.patch.object(launch_mod.docker_mod, "verify_agent_image"), \
              mock.patch.object(launch_mod, "launch_consolidated", return_value=ctx), \
              mock.patch.object(launch_mod, "teardown_consolidated"), \
              mock.patch.object(launch_mod, "DockerGateway", return_value=gw), \

tests/unit/test_firecracker_backend.py

@@ -334,6 +334,18 @@ class TestBootArgs(unittest.TestCase):
         self.assertEqual("/run/rootfs.ext4", cfg["drives"][0]["path_on_host"])
         self.assertFalse(cfg["drives"][0]["is_read_only"])
         self.assertEqual("bbfc0", cfg["network-interfaces"][0]["host_dev_name"])
+        self.assertEqual(1, len(cfg["drives"]))  # no data drive by default
+
+    def test_config_adds_data_drive(self):
+        cfg = cast(Any, firecracker_vm._config(
+            rootfs=Path("/run/rootfs.ext4"), tap="bbfc0",
+            guest_ip="100.64.0.1", host_ip="100.64.0.0", pubkey="k",
+            vcpus=2, mem_mib=2048, guest_mac="06:00:AC:10:00:02",
+            data_drive=Path("/run/registry.ext4"),
+        ))
+        self.assertEqual(2, len(cfg["drives"]))
+        self.assertFalse(cfg["drives"][1]["is_root_device"])
+        self.assertEqual("/run/registry.ext4", cfg["drives"][1]["path_on_host"])
 
 
 class TestBottleExecClose(unittest.TestCase):

tests/unit/test_firecracker_helpers.py

@@ -64,6 +64,21 @@ class TestNetpoolProbes(unittest.TestCase):
              patch.object(netpool, "tap_present", side_effect=[True, False]):
             self.assertEqual(["bbfc1"], netpool.missing_taps())
 
+    def test_orch_slot_is_top_of_ip_base_16(self):
+        # Dedicated orchestrator link: /31 at the top of the IP_BASE /16,
+        # clear of the pool (bottom of the block). Must match the shell
+        # script's orch_host()/orch_guest() and be a non-pool index.
+        with patch.dict("os.environ", {"BOT_BOTTLE_FC_IP_BASE": "10.243.0.0"}):
+            s = netpool.orch_slot()
+        self.assertEqual(netpool.ORCH_IFACE, s.iface)
+        self.assertEqual("10.243.255.0", s.host_ip)
+        self.assertEqual("10.243.255.1", s.guest_ip)
+        self.assertEqual(-1, s.index)
+        # Never collides with a pool slot's guest address.
+        with patch.dict("os.environ", {"BOT_BOTTLE_FC_IP_BASE": "10.243.0.0"}):
+            pool_guests = {sl.guest_ip for sl in netpool.all_slots()}
+        self.assertNotIn(s.guest_ip, pool_guests)
+
 
 class TestConsoleTail(unittest.TestCase):
     def test_reads_tail(self):

tests/unit/test_firecracker_image_builder.py

@@ -0,0 +1,74 @@
+"""Unit tests for the docker-free Firecracker agent-image builder.
+
+The VM boot / SSH / buildah plumbing (`_build_in_infra`) is integration-tested
+on a KVM host; here we cover the cache decision, the boot-bit injection, and
+the smoke-test no-op — the logic that must hold without a VM.
+"""
+
+from __future__ import annotations
+
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+from bot_bottle.backend.firecracker import image_builder
+
+
+class TestBuildAgentRootfsDir(unittest.TestCase):
+    def setUp(self):
+        self._tmp = tempfile.TemporaryDirectory()
+        self.cache = Path(self._tmp.name)
+        self.dockerfile = self.cache / "Dockerfile"
+        self.dockerfile.write_text("FROM node:22-slim\n")
+        self.addCleanup(self._tmp.cleanup)
+
+    def test_cache_hit_skips_rebuild(self):
+        digest = image_builder._dockerfile_hash(self.dockerfile)
+        base = self.cache / "rootfs" / f"agent-{digest}"
+        base.mkdir(parents=True)
+        (base / ".bb-ready").write_text("ok\n")
+        with patch.object(image_builder.util, "cache_dir", return_value=self.cache), \
+             patch.object(image_builder, "_build_in_infra") as build:
+            out = image_builder.build_agent_rootfs_dir(
+                self.dockerfile, image_tag="t:latest")
+        build.assert_not_called()
+        self.assertEqual(base, out)
+
+    def test_cache_miss_builds_injects_and_marks_ready(self):
+        with patch.object(image_builder.util, "cache_dir", return_value=self.cache), \
+             patch.object(image_builder, "_build_in_infra") as build, \
+             patch.object(image_builder.util, "inject_guest_boot") as inject:
+            out = image_builder.build_agent_rootfs_dir(
+                self.dockerfile, image_tag="t:latest", smoke_test=("claude", "--version"))
+        build.assert_called_once()
+        # smoke_test threads through to the VM build.
+        self.assertEqual(("claude", "--version"), build.call_args.args[2])
+        inject.assert_called_once_with(out)
+        self.assertTrue((out / ".bb-ready").is_file())
+
+    def test_content_addressed_cache_key(self):
+        other = self.cache / "Dockerfile2"
+        other.write_text("FROM python:3.12-slim\n")
+        self.assertNotEqual(
+            image_builder._dockerfile_hash(self.dockerfile),
+            image_builder._dockerfile_hash(other),
+        )
+
+
+class TestSmokeTest(unittest.TestCase):
+    def test_empty_argv_is_noop(self):
+        with patch.object(image_builder, "_ssh") as ssh:
+            image_builder._smoke_test(Path("/k"), "10.0.0.1", "tag", ())
+        ssh.assert_not_called()
+
+    def test_failed_smoke_dies(self):
+        import subprocess
+        result = subprocess.CompletedProcess([], 1, stdout="broken", stderr="")
+        with patch.object(image_builder, "_ssh", return_value=result), \
+             self.assertRaises(SystemExit):
+            image_builder._smoke_test(Path("/k"), "10.0.0.1", "tag", ("claude", "--version"))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/test_firecracker_infra_vm.py

@@ -0,0 +1,179 @@
+"""Unit tests for the Firecracker infra VM (control-plane VM boot).
+
+The KVM boot / HTTP reachability is integration-tested on a KVM host; here
+we cover the URL shape, the rootfs-variant wiring, and the health-poll
+decisions that must hold without a VM.
+"""
+
+from __future__ import annotations
+
+import unittest
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+from bot_bottle.backend.firecracker import infra_vm
+
+
+class TestControlPlaneUrl(unittest.TestCase):
+    def test_url_uses_guest_ip_and_port(self):
+        infra = infra_vm.InfraVm(
+            vm=MagicMock(), guest_ip="10.243.255.1", private_key=Path("/k"))
+        self.assertEqual(
+            f"http://10.243.255.1:{infra_vm.CONTROL_PLANE_PORT}",
+            infra.control_plane_url,
+        )
+
+
+class TestBuildInfraRootfs(unittest.TestCase):
+    def test_uses_infra_variant_and_init(self):
+        with patch.object(infra_vm.util, "build_base_rootfs_dir") as build:
+            build.return_value = Path("/cache/rootfs/x-infra")
+            infra_vm.build_infra_rootfs_dir()
+        build.assert_called_once()
+        self.assertEqual(infra_vm._INFRA_IMAGE, build.call_args.args[0])
+        # variant is "-infra-<init-hash>" so an init change rebuilds the rootfs.
+        self.assertTrue(build.call_args.kwargs["variant"].startswith("-infra-"))
+        # The init runs BOTH the control plane and the gateway data plane,
+        # and exports PATH so gateway_init's subprocess daemons find python3.
+        init = build.call_args.kwargs["init_script"]
+        self.assertIn("bot_bottle.orchestrator", init)
+        self.assertIn("gateway_init.py", init)
+        self.assertIn("export PATH=", init)
+        # Persistent registry volume mounted at the DB dir before the CP starts.
+        self.assertIn("/dev/vdb", init)
+        # VM backend uses git-http (9420); the git:// daemon is left out.
+        self.assertIn("BOT_BOTTLE_GATEWAY_DAEMONS=egress,git-http,supervise", init)
+
+
+class TestSshGatewayTransport(unittest.TestCase):
+    def test_cp_into_preserves_source_mode(self):
+        import os
+        import tempfile
+        from subprocess import CompletedProcess
+        with tempfile.NamedTemporaryFile() as f:
+            os.chmod(f.name, 0o700)  # like the staged access-hook
+            t = infra_vm.SshGatewayTransport(Path("/k"), "10.0.0.1")
+            with patch.object(infra_vm.subprocess, "run",
+                              return_value=CompletedProcess([], 0)) as run:
+                t.cp_into(f.name, "/etc/git-gate/access-hook")
+        remote_cmd = run.call_args.args[0][-1]
+        self.assertIn("chmod 700", remote_cmd)  # exec bit preserved over SSH
+
+    def test_exec_raises_on_failure(self):
+        from subprocess import CompletedProcess
+        t = infra_vm.SshGatewayTransport(Path("/k"), "10.0.0.1")
+        with patch.object(infra_vm.subprocess, "run",
+                          return_value=CompletedProcess([], 1, stderr="nope")), \
+             self.assertRaises(infra_vm.GatewayProvisionError):
+            t.exec(["mkdir", "-p", "/git-gate"])
+
+
+class TestRegistryVolume(unittest.TestCase):
+    def test_reuses_existing_volume(self):
+        import tempfile
+        with tempfile.TemporaryDirectory() as td:
+            vol = Path(td) / "registry.ext4"
+            vol.write_bytes(b"")  # already present
+            with patch.object(infra_vm, "registry_volume_path", return_value=vol), \
+                 patch.object(infra_vm.subprocess, "run") as run:
+                out = infra_vm._ensure_registry_volume()
+            run.assert_not_called()  # no mke2fs when it exists
+            self.assertEqual(vol, out)
+
+    def test_creates_volume_when_missing(self):
+        import tempfile
+        from subprocess import CompletedProcess
+        with tempfile.TemporaryDirectory() as td:
+            vol = Path(td) / "registry.ext4"
+            with patch.object(infra_vm, "registry_volume_path", return_value=vol), \
+                 patch.object(infra_vm.subprocess, "run",
+                              return_value=CompletedProcess([], 0)) as run:
+                infra_vm._ensure_registry_volume()
+            argv = run.call_args.args[0]
+            self.assertIn("mke2fs", argv)
+            self.assertIn(str(vol), argv)
+
+
+class TestEnsureBuilt(unittest.TestCase):
+    def test_builds_deps_before_infra(self):
+        with patch.object(infra_vm.docker_mod, "build_image") as build:
+            infra_vm.ensure_built()
+        tags = [c.args[0] for c in build.call_args_list]
+        # infra is FROM gateway and COPY --from orchestrator, so both first.
+        self.assertEqual(infra_vm._INFRA_IMAGE, tags[-1])
+        self.assertIn(infra_vm._ORCHESTRATOR_IMAGE, tags[:-1])
+        self.assertIn(infra_vm._GATEWAY_IMAGE, tags[:-1])
+
+
+class TestWaitForHealth(unittest.TestCase):
+    def _infra(self, alive: bool = True) -> infra_vm.InfraVm:
+        vm = MagicMock()
+        vm.is_alive.return_value = alive
+        return infra_vm.InfraVm(vm=vm, guest_ip="10.0.0.1", private_key=Path("/k"))
+
+    def test_returns_on_200(self):
+        infra = self._infra()
+        cm = MagicMock()
+        cm.__enter__.return_value.status = 200
+        with patch.object(infra_vm.urllib.request, "urlopen", return_value=cm):
+            infra_vm.wait_for_health(infra, timeout=5)  # must not raise
+
+    def test_dies_when_vm_exits(self):
+        infra = self._infra(alive=False)
+        assert infra.vm is not None  # narrow for the type checker (it's a mock)
+        infra.vm.process.returncode = 1
+        with patch.object(infra_vm.firecracker_vm, "_console_tail", return_value=""), \
+             self.assertRaises(SystemExit):
+            infra_vm.wait_for_health(infra, timeout=5)
+
+
+class TestEnsureRunningSingleton(unittest.TestCase):
+    def test_adopts_when_healthy(self):
+        # A healthy control plane + existing key -> adopt (no boot), vm=None.
+        with patch.object(infra_vm, "_health_ok", return_value=True), \
+             patch.object(infra_vm, "_infra_dir") as d, \
+             patch.object(infra_vm, "boot") as boot:
+            keydir = MagicMock()
+            (keydir / "id_ed25519").exists.return_value = True
+            d.return_value = keydir
+            infra = infra_vm.ensure_running()
+        boot.assert_not_called()
+        self.assertIsNone(infra.vm)
+
+    def test_boots_when_unhealthy(self):
+        with patch.object(infra_vm, "_health_ok", return_value=False), \
+             patch.object(infra_vm, "stop") as stop, \
+             patch.object(infra_vm, "ensure_built") as built, \
+             patch.object(infra_vm, "boot") as boot, \
+             patch.object(infra_vm, "wait_for_health") as wait:
+            boot.return_value = infra_vm.InfraVm(
+                guest_ip="10.243.255.1", private_key=Path("/k"), vm=MagicMock())
+            infra_vm.ensure_running()
+        stop.assert_called_once()      # clear a stale VM first
+        built.assert_called_once()
+        boot.assert_called_once()
+        wait.assert_called_once()
+
+
+class TestKillPidfile(unittest.TestCase):
+    def test_noop_when_no_pidfile(self):
+        import tempfile
+        with tempfile.TemporaryDirectory() as td:
+            with patch.object(infra_vm, "_pid_file", return_value=Path(td) / "vm.pid"), \
+                 patch.object(infra_vm.os, "kill") as kill:
+                infra_vm._kill_pidfile()  # must not raise
+            kill.assert_not_called()
+
+    def test_skips_dead_or_recycled_pid(self):
+        import tempfile
+        with tempfile.TemporaryDirectory() as td:
+            pidf = Path(td) / "vm.pid"
+            pidf.write_text("999999")  # a PID that isn't a live firecracker
+            with patch.object(infra_vm, "_pid_file", return_value=pidf), \
+                 patch.object(infra_vm.os, "kill") as kill:
+                infra_vm._kill_pidfile()
+            kill.assert_not_called()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/test_gateway_provision.py

@@ -8,6 +8,7 @@ from unittest.mock import Mock, patch
 
 from bot_bottle.backend.docker.gateway_provision import (
     GatewayProvisionError,
+    DockerGatewayTransport,
     deprovision_git_gate,
     provision_git_gate,
 )
@@ -54,7 +55,7 @@ class TestProvisionGitGate(unittest.TestCase):
     def test_copies_creds_and_runs_namespaced_init(self) -> None:
         calls: list[list[str]] = []
         with patch(_RUN, side_effect=_recorder(calls)):
-            provision_git_gate("gw", "bottle1", _plan(_up("foo", known_hosts="/host/kh")))
+            provision_git_gate(DockerGatewayTransport("gw"), "bottle1", _plan(_up("foo", known_hosts="/host/kh")))
 
         cps = [c for c in calls if c[:2] == ["docker", "cp"]]
         self.assertIn(["docker", "cp", "/host/keys/id", "gw:/git-gate/creds/bottle1/foo-key"], cps)
@@ -71,32 +72,32 @@ class TestProvisionGitGate(unittest.TestCase):
     def test_omits_known_hosts_copy_when_absent(self) -> None:
         calls: list[list[str]] = []
         with patch(_RUN, side_effect=_recorder(calls)):
-            provision_git_gate("gw", "b1", _plan(_up("foo")))  # no known_hosts
+            provision_git_gate(DockerGatewayTransport("gw"), "b1", _plan(_up("foo")))  # no known_hosts
         creds_cps = [c for c in calls if c[:2] == ["docker", "cp"] and "/git-gate/creds/" in c[3]]
         self.assertEqual(1, len(creds_cps))  # only the key, not known_hosts
         self.assertTrue(creds_cps[0][3].endswith("/foo-key"))
 
     def test_no_upstreams_is_noop(self) -> None:
         with patch(_RUN) as m:
-            provision_git_gate("gw", "b1", _plan())
+            provision_git_gate(DockerGatewayTransport("gw"), "b1", _plan())
         m.assert_not_called()
 
     def test_raises_on_docker_failure(self) -> None:
         with patch(_RUN, return_value=_proc(returncode=1, stderr="boom")):
             with self.assertRaises(GatewayProvisionError):
-                provision_git_gate("gw", "b1", _plan(_up("foo")))
+                provision_git_gate(DockerGatewayTransport("gw"), "b1", _plan(_up("foo")))
 
     def test_rejects_unsafe_bottle_id_before_any_docker(self) -> None:
         with patch(_RUN) as m:
             with self.assertRaises(GatewayProvisionError):
-                provision_git_gate("gw", "../etc", _plan(_up("foo")))
+                provision_git_gate(DockerGatewayTransport("gw"), "../etc", _plan(_up("foo")))
         m.assert_not_called()  # rejected before a single docker call
 
 
 class TestDeprovision(unittest.TestCase):
     def test_removes_repo_and_creds(self) -> None:
         with patch(_RUN, return_value=_proc()) as m:
-            deprovision_git_gate("gw", "b1")
+            deprovision_git_gate(DockerGatewayTransport("gw"), "b1")
         argv = m.call_args.args[0]
         self.assertEqual(["docker", "exec", "gw", "rm", "-rf"], argv[:5])
         self.assertIn("/git/b1", argv)
@@ -105,7 +106,7 @@ class TestDeprovision(unittest.TestCase):
     def test_rejects_unsafe_bottle_id(self) -> None:
         with patch(_RUN) as m:
             with self.assertRaises(GatewayProvisionError):
-                deprovision_git_gate("gw", "a/b")
+                deprovision_git_gate(DockerGatewayTransport("gw"), "a/b")
         m.assert_not_called()