feat(smolmachines): end-to-end launch + Bottle.exec + smoke + probes (PRD 0023 chunk 2d)

End-to-end launch flow for the smolmachines backend. Brings up
the per-bottle docker bridge + sidecar bundle, creates and
starts the smolvm guest pointed at the bundle's pinned IP via
TSI's `--allow-cidr <bundle-ip>/32`, yields a SmolmachinesBottle
handle that routes exec/cp through `smolvm machine exec / cp`,
tears everything down on context exit.

launch.py:
- ExitStack-managed: create_bundle_network → start_bundle →
  machine_create → machine_start (each registered for reverse
  teardown).
- daemons_csv="" for chunk 2d — bundle init logs "no daemons
  selected" and idles. Real daemon bringup with inner-Plan-driven
  env + volumes lands in chunk 4.

bottle.py:
- SmolmachinesBottle.exec → smolvm.machine_exec (captured).
- SmolmachinesBottle.exec_claude → direct subprocess.run with
  inherited TTY for interactive sessions.
- SmolmachinesBottle.cp_in → smolvm.machine_cp.

Architecture pivots forced by smolvm 0.8.0's CLI shape:
1. `--from <smolmachine>` and `--smolfile <toml>` are MUTUALLY
   EXCLUSIVE in smolvm 0.8.0. We need --from to avoid the
   registry-pull race that bit us on machine_start (libkrun
   agent's network attempt got refused by macOS with
   "connect: permission denied" on IPv6). So Smolfile is dropped
   entirely; per-bottle env + allow_cidrs flow as CLI flags
   (`--allow-cidr CIDR`, `-e K=V`) directly to machine_create.
2. `smolvm pack create --image` doesn't pull from the local
   docker daemon — only OCI registries via crane. The real
   claude-bottle:latest image lives in the local docker daemon
   and isn't reachable that way. Chunk 2d ships with an alpine
   placeholder; the agent-image-conversion gap belongs to
   chunk 4 (push the image to a registry, or smolvm grows a
   docker-daemon transport).

Other changes:
- machine_create grew `image=` / `from_path=` / `allow_cidrs=`
  / `env=` kwargs; smolfile= dropped.
- bottle_plan: smolfile_path → agent_from_path + guest_env.
- prepare: pack_create against `alpine:latest`, cached under
  ~/.cache/claude-bottle/smolmachines/ keyed by image ref.
- Deleted smolfile.py + test_smolfile.py (dead code now).

Tests:
- Unit: 540 passing (smolvm wrapper grew 4 new flag forms; one
  test renamed to reflect --from + --allow-cidr + -e combo).
- Integration: 3 new cases in tests/integration/
  test_smolmachines_launch.py, gated on Darwin + smolvm on PATH
  + docker + not GITEA_ACTIONS:
    * smoke: bottle.exec("echo hello-from-vm") round-trips with
      the correct stdout + returncode.
    * localhost-reach probe: agent dials 127.0.0.1:9 → connect
      refused (TSI's <bundle-ip>/32 allowlist doesn't include
      loopback). The regression test for the gap the PRD design
      pivot was about.
    * egress-port-bypass probe: agent dials <bundle-ip>:9099
      (egress's port) → connect refused. Chunk 2d has no
      daemons running so nothing's listening anyway; chunk 3
      will preserve this property once egress is up but bound
      to 127.0.0.1 inside the bundle.

End-to-end smoke + both probes green locally on macOS with
smolvm 0.8.0.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

claude_bottle/backend/smolmachines/backend.py

@@ -1,6 +1,5 @@
 """SmolmachinesBottleBackend — the smolmachines implementation of
-BottleBackend (PRD 0023). Chunk 1 ships prepare-only; launch raises
-NotImplementedError until chunk 2."""
+BottleBackend (PRD 0023)."""
 
 from __future__ import annotations
 
@@ -9,6 +8,7 @@ from pathlib import Path
 from typing import Generator
 
 from .. import BottleBackend, BottleSpec
+from . import launch as _launch
 from . import prepare as _prepare
 from .bottle import SmolmachinesBottle
 from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
@@ -32,18 +32,11 @@ class SmolmachinesBottleBackend(
     def launch(
         self, plan: SmolmachinesBottlePlan
     ) -> Generator[SmolmachinesBottle, None, None]:
-        del plan
-        raise NotImplementedError(
-            "smolmachines launch is implemented in PRD 0023 chunk 2; "
-            "chunk 1 ships prepare-only (the Smolfile + gvproxy "
-            "config are written, but no VM is brought up)."
-        )
-        # The generator never gets here, but the type checker wants
-        # to see the yield:
-        yield  # type: ignore[unreachable]
+        with _launch.launch(plan) as bottle:
+            yield bottle
 
     # The four `provision_*` methods land in chunk 4 alongside the
-    # `smolvm machine exec`-based copy-in flow. Stubs raise so any
+    # `smolvm machine cp`-based copy-in flow. Stubs raise so any
     # caller that reaches them before chunk 4 gets a clear pointer.
     def provision_prompt(
         self, plan: SmolmachinesBottlePlan, target: str
@@ -65,12 +58,12 @@ class SmolmachinesBottleBackend(
 
     def cleanup(self, plan: SmolmachinesBottleCleanupPlan) -> None:
         del plan
-        # Nothing to clean in chunk 1 — see SmolmachinesBottleCleanupPlan
-        # docstring.
+        # Nothing to clean in chunks 1-3 — see
+        # SmolmachinesBottleCleanupPlan docstring.
 
     def list_active(self) -> None:
         from ...log import info
         info(
             "smolmachines list_active: not implemented (chunk 4 wires "
-            "it to `smolvm machine list`)"
+            "it to `smolvm machine ls --json`)"
         )

claude_bottle/backend/smolmachines/bottle.py

@@ -1,41 +1,66 @@
-"""SmolmachinesBottle — runtime handle stub (PRD 0023 chunk 1).
+"""SmolmachinesBottle — running-instance handle (PRD 0023 chunk 2d).
 
-The chunk-1 backend doesn't launch VMs yet, so this class only
-exists to make `SmolmachinesBottleBackend.launch` resolvable at
-import time. Every method raises NotImplementedError; chunk 2
-gives it real `smolvm machine exec` plumbing."""
+Routes `exec_claude` / `exec` / `cp_in` through `smolvm machine
+exec` / `smolvm machine cp`. The handle is yielded by `launch`
+and torn down via the surrounding ExitStack on context exit;
+`close` is a no-op idempotent alias so the BottleBackend ABC's
+context-manager contract is satisfied."""
 
 from __future__ import annotations
 
+import subprocess
+import sys
+
 from .. import Bottle, ExecResult
+from . import smolvm as _smolvm
 
 
 class SmolmachinesBottle(Bottle):
-    """Stub. Real impl lands in chunk 2."""
+    """Handle returned by `SmolmachinesBottleBackend.launch`. The
+    underlying VM lifecycle (create / start / stop / delete) lives
+    on the launch ExitStack — this class only routes runtime
+    operations to the right `smolvm machine ...` subcommand."""
 
-    def __init__(self, name: str) -> None:
-        self.name = name
+    def __init__(self, machine_name: str) -> None:
+        self.name = machine_name
 
     def exec_claude(self, argv: list[str], *, tty: bool = True) -> int:
-        del argv, tty
-        raise NotImplementedError(
-            "smolmachines backend chunk 1 ships prepare-only; "
-            "exec_claude lands in chunk 2"
-        )
+        """Run `claude` interactively inside the VM. Inherits the
+        operator's terminal (stdin / stdout / stderr) so the
+        session feels native. Blocks until claude exits; returns
+        the in-VM exit code.
+
+        We bypass the captured-output `machine_exec` helper here
+        because that one wraps stdout/stderr in pipes — fine for
+        scripted exec, wrong for an interactive shell. Drop down
+        to `subprocess.run` with the TTY inherited."""
+        flags = ["smolvm", "machine", "exec", "--name", self.name]
+        if tty:
+            flags += ["-i", "-t"]
+        flags += ["--", "claude", *argv]
+        result = subprocess.run(flags, check=False)
+        return result.returncode
 
     def exec(self, script: str) -> ExecResult:
-        del script
-        raise NotImplementedError(
-            "smolmachines backend chunk 1 ships prepare-only; "
-            "exec lands in chunk 2"
+        """Run a POSIX shell script and capture the result. The
+        script runs under `/bin/sh -c`, matching what the docker
+        backend's `exec` does — callers can write shell-y test
+        helpers without worrying about argv splitting."""
+        r = _smolvm.machine_exec(
+            self.name,
+            ["/bin/sh", "-c", script],
+        )
+        return ExecResult(
+            returncode=r.returncode,
+            stdout=r.stdout,
+            stderr=r.stderr,
         )
 
     def cp_in(self, host_path: str, container_path: str) -> None:
-        del host_path, container_path
-        raise NotImplementedError(
-            "smolmachines backend chunk 1 ships prepare-only; "
-            "cp_in lands in chunk 2"
-        )
+        """Copy a host path into the guest at `container_path`."""
+        _smolvm.machine_cp(host_path, f"{self.name}:{container_path}")
 
     def close(self) -> None:
+        # Real teardown lives on the launch ExitStack; this is just
+        # the idempotent alias the BottleBackend ABC expects.
         pass

claude_bottle/backend/smolmachines/bottle_plan.py

@@ -1,10 +1,10 @@
 """SmolmachinesBottlePlan — concrete BottlePlan for the smolmachines
 backend (PRD 0023).
 
-Chunk 1 + 2a fields: slug, smolfile_path, bundle docker subnet /
-gateway / pinned IP. VM lifecycle + provisioning fields (machine
-name, `.smolmachine` artifact path, etc.) land in later chunks
-as the launch flow grows."""
+Slug + bundle docker subnet / gateway / pinned IP + smolvm
+machine name + agent `.smolmachine` artifact + per-bottle guest
+env. Provisioning fields (CA cert path, prompt path, etc.) land
+in chunk 4."""
 
 from __future__ import annotations
 
@@ -24,13 +24,34 @@ class SmolmachinesBottlePlan(BottlePlan):
     Inherits `spec` and `stage_dir` from BottlePlan."""
 
     slug: str
-    smolfile_path: Path
     # Per-bottle docker subnet for the sidecar bundle container.
     # The bundle runs at `bundle_ip` (always `.2`); the gateway is
     # at `.1`. smolvm's TSI allowlist is set to `bundle_ip/32`.
     bundle_subnet: str
     bundle_gateway: str
     bundle_ip: str
+    # smolvm machine name + agent image source. machine_create
+    # boots from a packed `.smolmachine` artifact (pre-baked at
+    # prepare time via `smolvm pack create`); using `--from`
+    # instead of `--image` avoids the registry-pull race we hit
+    # when machine_start tried to fetch on-demand and the libkrun
+    # agent's network attempt got refused by macOS.
+    #
+    # Chunk 2d ships with a public placeholder image (alpine)
+    # since claude-bottle:latest lives in the operator's local
+    # docker daemon and smolvm's crane backend can't read from
+    # there; chunk 4 resolves the agent-image-conversion gap
+    # (push to a registry first, or smolvm grows a docker-daemon
+    # transport).
+    machine_name: str
+    agent_from_path: Path
+    # In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
+    # the guest has no DNS resolver inside the TSI allowlist.
+    # Passed to `smolvm machine create` as `-e K=V` flags.
+    # Smolfile-rendering is gone (smolvm 0.8.0's
+    # `--smolfile` is mutually exclusive with `--from`, and
+    # `--from` is the path that avoids the registry-pull race).
+    guest_env: dict[str, str]
 
     def print(self, *, remote_control: bool) -> None:
         """Compact y/N preflight. Same shape as the Docker

claude_bottle/backend/smolmachines/launch.py

@@ -0,0 +1,73 @@
+"""End-to-end launch flow for the smolmachines backend
+(PRD 0023 chunk 2d).
+
+Brings up the per-bottle docker bridge + sidecar bundle, creates
++ starts the smolvm guest pointed at the bundle's pinned IP via
+the Smolfile's TSI allowlist, yields a `SmolmachinesBottle`
+handle, tears everything down on context exit.
+
+Chunk-2d scope: smoke-test plumbing for the launch + exec round
+trip. The bundle daemons aren't supplied with config files yet
+(pipelock.yaml, routes.yaml, etc.); the bundle's init supervisor
+exits cleanly when nothing is configured. Real provisioning + CA
+install + the inner Plan plumbing land in chunk 4."""
+
+from __future__ import annotations
+
+from contextlib import ExitStack, contextmanager
+from typing import Generator
+
+from . import smolvm as _smolvm
+from . import sidecar_bundle as _bundle
+from .bottle import SmolmachinesBottle
+from .bottle_plan import SmolmachinesBottlePlan
+
+
+@contextmanager
+def launch(
+    plan: SmolmachinesBottlePlan,
+) -> Generator[SmolmachinesBottle, None, None]:
+    """Build + run the bottle and yield a handle; tear everything
+    down on exit. Errors during bringup unwind any partial state
+    via the ExitStack."""
+    stack = ExitStack()
+    try:
+        # 1. Per-bottle docker bridge + bundle container.
+        network = _bundle.bundle_network_name(plan.slug)
+        _bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
+        stack.callback(_bundle.remove_bundle_network, network)
+
+        bundle_spec = _bundle.BundleLaunchSpec(
+            slug=plan.slug,
+            network_name=network,
+            subnet=plan.bundle_subnet,
+            gateway=plan.bundle_gateway,
+            bundle_ip=plan.bundle_ip,
+            # Chunk 2d: empty daemon set — the init supervisor
+            # logs "no daemons selected" and idles. Real daemon
+            # bringup with inner-Plan-driven env + volumes lands
+            # in chunk 4 alongside provisioning.
+            daemons_csv="",
+        )
+        _bundle.start_bundle(bundle_spec)
+        stack.callback(_bundle.stop_bundle, plan.slug)
+
+        # 2. smolvm VM. --from carries the pre-packed
+        # .smolmachine artifact (built by prepare); --allow-cidr
+        # + -e carry the per-bottle TSI allowlist + env. Smolfile
+        # isn't usable here — smolvm 0.8.0 makes `--from` and
+        # `--smolfile` mutually exclusive.
+        _smolvm.machine_create(
+            plan.machine_name,
+            from_path=plan.agent_from_path,
+            allow_cidrs=[f"{plan.bundle_ip}/32"],
+            env=plan.guest_env,
+        )
+        stack.callback(_smolvm.machine_delete, plan.machine_name)
+        _smolvm.machine_start(plan.machine_name)
+        stack.callback(_smolvm.machine_stop, plan.machine_name)
+
+        # 3. Yield the handle.
+        yield SmolmachinesBottle(plan.machine_name)
+    finally:
+        stack.close()

claude_bottle/backend/smolmachines/prepare.py

@@ -1,8 +1,9 @@
-"""smolmachines `_resolve_plan` (PRD 0023 chunk 2a).
+"""smolmachines `_resolve_plan` (PRD 0023 chunk 2d).
 
-Resolves the per-bottle docker subnet + bundle IP and writes the
-Smolfile to the stage dir. No VM bringup. The plan it returns is
-enough for the y/N preflight to render."""
+Resolves the per-bottle docker subnet + bundle IP, pre-packs the
+agent's `.smolmachine` artifact (cached under
+`~/.cache/claude-bottle/smolmachines/`), and assembles the guest
+env. No VM bringup — that's `launch.launch`'s job."""
 
 from __future__ import annotations
 
@@ -15,11 +16,18 @@ from ...backend.docker.bottle_state import (
     bottle_identity,
     write_metadata,
 )
+from . import smolvm as _smolvm
 from .bottle_plan import SmolmachinesBottlePlan
-from .smolfile import smolfile_build, smolfile_write
 from .util import smolmachines_bundle_subnet, smolmachines_preflight
 
 
+# Per-host cache for `smolvm pack create` outputs. Keyed by the
+# image ref so re-prepares for the same image hit the cache
+# (pack create is idempotent on the smolvm side but takes several
+# seconds even when no layer is fetched).
+_SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "claude-bottle" / "smolmachines"
+
+
 # Gateway ports the bundle exposes inside its container — pipelock
 # HTTPS proxy, git-gate's git-daemon, supervise's MCP. The agent
 # inside the smolvm guest dials these on the bundle's pinned IP.
@@ -31,10 +39,13 @@ _BUNDLE_SUPERVISE_PORT = 9100
 def resolve_plan(
     spec: BottleSpec, *, stage_dir: Path
 ) -> SmolmachinesBottlePlan:
-    """Materialize the smolmachines plan. The Smolfile lands at
-    `<stage>/smolfile.toml`; the bundle's docker subnet + pinned
-    IP are derived from the slug and carried on the plan for
-    launch to consume."""
+    """Materialize the smolmachines plan. The bundle's docker
+    subnet + pinned IP are derived from the slug; the agent's
+    `.smolmachine` artifact is built (or cache-hit) here so
+    launch's `machine create --from` boots without a registry
+    pull. Per-bottle guest env + the TSI allow_cidrs land on the
+    plan for launch to pass straight through to
+    `machine create` flags."""
     smolmachines_preflight()
 
     manifest = spec.manifest
@@ -74,18 +85,43 @@ def resolve_plan(
             f"http://{bundle_ip}:{_BUNDLE_SUPERVISE_PORT}"
         )
 
-    smolfile_path = stage_dir / "smolfile.toml"
-    smolfile_write(
-        smolfile_build(env=guest_env, bundle_ip=bundle_ip),
-        smolfile_path,
-    )
+    machine_name = f"claude-bottle-{slug}"
+    # Chunk 2d placeholder until chunk 4's agent-image work lands.
+    # alpine pulls cleanly from docker.io via smolvm's crane
+    # backend; the real claude-bottle image lives in the local
+    # docker daemon and isn't reachable that way.
+    agent_image_ref = "alpine:latest"
+    agent_from_path = _ensure_smolmachine(agent_image_ref)
 
     return SmolmachinesBottlePlan(
         spec=spec,
         stage_dir=stage_dir,
         slug=slug,
-        smolfile_path=smolfile_path,
         bundle_subnet=subnet,
         bundle_gateway=gateway,
         bundle_ip=bundle_ip,
+        machine_name=machine_name,
+        agent_from_path=agent_from_path,
+        guest_env=guest_env,
     )
+
+
+def _ensure_smolmachine(image_ref: str) -> Path:
+    """Cache `smolvm pack create --image <image_ref>` output under
+    `~/.cache/claude-bottle/smolmachines/<slug>`. Returns the
+    `.smolmachine.smolmachine` sidecar path — that's the file
+    `machine create --from` consumes (pack create produces a
+    launcher binary at `.smolmachine` plus the sidecar alongside
+    it; the sidecar is the actual artifact).
+
+    Re-runs of pack create against the same image hit smolvm's
+    layer cache; we still skip the call entirely when the
+    sidecar is already on disk, since each invocation costs
+    several seconds even on a hot cache."""
+    _SMOLMACHINE_CACHE_DIR.mkdir(parents=True, exist_ok=True)
+    slug = image_ref.replace(":", "_").replace("/", "_")
+    binary = _SMOLMACHINE_CACHE_DIR / f"{slug}.smolmachine"
+    sidecar = _SMOLMACHINE_CACHE_DIR / f"{slug}.smolmachine.smolmachine"
+    if not sidecar.is_file():
+        _smolvm.pack_create(image_ref, binary)
+    return sidecar

claude_bottle/backend/smolmachines/smolfile.py

@@ -1,81 +0,0 @@
-"""Smolfile (TOML) renderer for the smolmachines backend (PRD 0023).
-
-The Smolfile shape comes from `smolvm machine create --smolfile`
-in smolvm 0.8.0. The renderer emits only the runtime overrides
-that vary per bottle:
-
-  - `env = ["K=V", ...]` — agent's HTTPS_PROXY / NO_PROXY /
-    NODE_EXTRA_CA_CERTS, all using IP literals pointing at the
-    per-bottle sidecar bundle's pinned docker IP.
-
-  - `[network] allow_cidrs = ["<bundle-ip>/32"]` — TSI's single-IP
-    allowlist. With this and no other `allow_*` or
-    `outbound-localhost-only`, the agent can dial exactly one IP:
-    the bundle. Host loopback, LAN, and the public internet
-    directly are all refused at the VMM layer.
-
-What the renderer does NOT emit:
-
-  - `image` / `entrypoint` / `cmd` — those come from the
-    `.smolmachine` artifact (produced by `smolvm pack create
-    --image claude-bottle:latest`) and don't vary across bottles
-    of the same agent image.
-
-  - `cpus` / `memory` — left at smolvm defaults until the
-    operator surfaces a need to override per bottle (the manifest
-    has no such field today).
-
-Pure function; disk writes happen via `smolfile_write`."""
-
-from __future__ import annotations
-
-from pathlib import Path
-from typing import Any, Mapping
-
-
-def smolfile_build(
-    *,
-    env: Mapping[str, str],
-    bundle_ip: str,
-) -> dict[str, Any]:
-    """Build the Smolfile config dict. `env` is `{NAME: VALUE}` for
-    the guest's process env (IP-literal HTTPS_PROXY etc.).
-    `bundle_ip` is the pinned docker IP of the per-bottle sidecar
-    bundle; it lands in `[network] allow_cidrs` as a /32."""
-    return {
-        "env": [f"{k}={v}" for k, v in sorted(env.items())],
-        "network": {
-            "allow_cidrs": [f"{bundle_ip}/32"],
-        },
-    }
-
-
-def smolfile_render(cfg: dict[str, Any]) -> str:
-    """Render the Smolfile dict as TOML. Schema is narrow (string
-    list + one table with a string list) so we render by hand and
-    stay stdlib-only."""
-    lines: list[str] = []
-    lines.append(f'env = {_toml_array(cfg["env"])}')
-    lines.append("")
-    lines.append("[network]")
-    lines.append(f'allow_cidrs = {_toml_array(cfg["network"]["allow_cidrs"])}')
-    return "\n".join(lines) + "\n"
-
-
-def smolfile_write(cfg: dict[str, Any], path: Path) -> Path:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    path.write_text(smolfile_render(cfg))
-    path.chmod(0o600)
-    return path
-
-
-def _toml_str(value: Any) -> str:
-    """TOML basic string: double-quoted with backslash + double-quote
-    escapes."""
-    s = str(value)
-    s = s.replace("\\", "\\\\").replace('"', '\\"')
-    return f'"{s}"'
-
-
-def _toml_array(values: list[Any]) -> str:
-    return "[" + ", ".join(_toml_str(v) for v in values) + "]"

claude_bottle/backend/smolmachines/smolvm.py

@@ -98,17 +98,36 @@ def pack_create(image: str, output: Path) -> None:
 def machine_create(
     name: str,
     *,
+    image: str | None = None,
     from_path: Path | None = None,
-    smolfile: Path | None = None,
+    allow_cidrs: Sequence[str] = (),
+    env: Mapping[str, str] | None = None,
 ) -> None:
-    """`smolvm machine create NAME [--from PATH] [--smolfile PATH]`.
-    NAME is positional (the CLI's exception to the `--name`
-    pattern other subcommands use)."""
+    """`smolvm machine create NAME [--image IMG | --from PATH]
+    [--allow-cidr CIDR ...] [-e K=V ...]`. NAME is positional
+    (the CLI's exception to the `--name` pattern other
+    subcommands use).
+
+    `image` (registry ref like `alpine:latest`) and `from_path`
+    (a `.smolmachine` artifact) are mutually exclusive — one or
+    the other tells smolvm what to boot. The wrapper doesn't
+    enforce exclusivity; smolvm errors clearly enough.
+
+    `allow_cidrs` and `env` are passed as CLI flags instead of a
+    Smolfile because `--from` and `--smolfile` are themselves
+    mutually exclusive in smolvm 0.8.0 — and we want `--from`'s
+    no-pull-at-start property. The flag form gives the same
+    result without the Smolfile complication."""
     args: list[str] = ["machine", "create"]
+    if image is not None:
+        args += ["--image", image]
     if from_path is not None:
         args += ["--from", str(from_path)]
-    if smolfile is not None:
-        args += ["--smolfile", str(smolfile)]
+    for cidr in allow_cidrs:
+        args += ["--allow-cidr", cidr]
+    if env:
+        for k, v in env.items():
+            args += ["-e", f"{k}={v}"]
     args.append(name)
     _smolvm(*args)