bot-bottle/docs/prds/0023-smolmachines-backend.md

# PRD 0023: smolmachines bottle backend

- **Status:** Draft
- **Author:** didericis
- **Created:** 2026-05-26

## Summary

Ship a second concrete `BottleBackend` — `SmolmachinesBottleBackend`,
selected via `CLAUDE_BOTTLE_BACKEND=smolmachines` — that runs a
bottle inside a per-agent libkrun microVM on macOS (and KVM on Linux,
opportunistically). The egress topology moves out of an internal
Docker network and onto libkrun's TSI ("Transport Socket Interface")
allowlist plus a host-side pipelock/egress/git-gate/supervise stack
listening on per-bottle loopback ports. The Docker backend ships
unchanged; this is opt-in via the existing env-var selector.

The acceptance gate is PRD 0022's `tests/integration/test_sandbox_escape.py`
running green against `CLAUDE_BOTTLE_BACKEND=smolmachines`.

## Problem

`agent-vm-isolation.md` argues for hardware-isolated microVMs over
container-based bottles on macOS; `smolmachines-as-vm-backend.md`
concludes that smolmachines is the most plausible concrete VMM for
this project. Today, the only backend in the registry is Docker
(`claude_bottle/backend/__init__.py:_BACKENDS = {"docker": ...}`),
and three things motivate a second one now:

- **Isolation ceiling.** On macOS the Docker backend's agent
  container shares Docker Desktop's host VM with every other bottle.
  Container escape from claude-code lands the agent inside that
  shared VM. A per-bottle libkrun microVM gets hardware page tables
  via `Hypervisor.framework`; cross-bottle isolation becomes
  enforced by the CPU's MMU instead of namespace bookkeeping.
- **PRD 0022 is backend-agnostic by design** but currently only
  exercises the Docker backend. The suite was written with
  `CLAUDE_BOTTLE_BACKEND` selection in mind precisely so the
  smolmachines path could be validated against the same five
  attacks. Until a second backend exists, the abstraction is
  unproven.
- **CI carve-outs.** Most bottle-bringup integration tests skip
  under `GITEA_ACTIONS=true` because act_runner shares the host
  Docker socket but not the host filesystem. A smolmachines path
  doesn't share that constraint shape (it has its own, but
  different), so adding the backend forces the abstraction to be
  clean in places where Docker-specific assumptions have been
  tolerated.

The smolmachines research note's `## Recommendation` ("adopt
smolmachines as the bottle VM backend on macOS; keep pipelock DIY")
is the design hypothesis under test here.

## Goals / Success Criteria

The feature works when all of the following are observable on a
macOS host with smolmachines installed:

- `CLAUDE_BOTTLE_BACKEND=smolmachines python3 cli.py start <agent>`
  brings up a microVM, runs claude-code inside it, and tears it
  down on exit. Same y/N preflight UX as Docker — only the
  resolved-runtime line differs.
- The sandbox-escape suite in `tests/integration/test_sandbox_escape.py`
  runs green against the smolmachines backend (all five attack
  categories blocked).
- Selecting the backend on a host without `smolvm` installed dies
  at startup with an install pointer; no silent fall-through to
  Docker.
- Active bottles show up under
  `python3 cli.py list-bottles` regardless of backend.
- `python3 cli.py stop <bottle>` and orphan cleanup work for both
  Docker bottles and smolmachines bottles via the same CLI surface.

The feature is **done** when all of the following ship:

- A new `claude_bottle/backend/smolmachines/` subpackage exists,
  mirroring the layout of `claude_bottle/backend/docker/`
  (`backend.py`, `bottle.py`, `bottle_plan.py`,
  `bottle_cleanup_plan.py`, `prepare.py`, `launch.py`,
  `cleanup.py`, `util.py`, and a `provision/` subpackage for the
  five `provision_*` methods).
- `SmolmachinesBottleBackend` registered under the
  `"smolmachines"` key in `claude_bottle/backend/__init__.py:_BACKENDS`.
- Per-bottle Smolfile generation: a runtime-rendered TOML written
  to the bottle's stage dir, analogous to the compose file the
  Docker backend writes today. The Smolfile pins `command`,
  `env`, `--outbound-localhost-only`, and the per-bottle DNS
  allowlist.
- Host-side sidecar relocation: pipelock, egress, git-gate, and
  supervise each run as host processes (one set per bottle),
  bound to `127.0.0.1` on per-bottle dynamically-allocated ports.
  The agent's environment carries the resolved URLs (e.g.
  `HTTPS_PROXY=http://127.0.0.1:<pipelock-port>`).
- The agent guest image is produced from the existing `Dockerfile`
  (or a thin variant), exported as an OCI archive, and consumed by
  `smolvm machine create`. The image build step is part of `prepare`,
  analogous to `docker_mod.build_image`.
- The PRD 0022 sandbox-escape suite, run with
  `CLAUDE_BOTTLE_BACKEND=smolmachines`, passes locally on a
  smolmachines-capable host. The suite is updated to skip cleanly
  on hosts that can't reach smolmachines (same shape as the
  existing `GITEA_ACTIONS == "true"` skip), not to fail.
- README + `CLAUDE.md` updated to document the env-var selection,
  the macOS-only scope for v1, and the `smolvm` install
  prerequisite.

## Non-goals

- **No Linux KVM support shipped in this PRD.** smolmachines works
  on Linux via KVM, but the abstraction win is biggest on macOS
  where Docker's shared-VM topology hurts most. Linux can come
  later behind the same selector.
- **No removal of the Docker backend.** Both backends ship side by
  side. Selection stays env-driven; the manifest does not gain a
  `backend` field.
- **No default-backend change.** `docker` remains the default
  value of `CLAUDE_BOTTLE_BACKEND`; smolmachines is strictly
  opt-in until it has been load-bearing on at least one operator's
  workflow for a release cycle.
- **No host bind mounts.** The smolmachines research note flagged
  that `-v HOST:GUEST` mounts via virtiofs would defeat the
  isolation goal. The manifest already has no concept of host
  mounts; this PRD does not introduce one. If a future PRD wants
  agent-side access to host files, it must come through a
  controlled channel (vsock relay, OCI overlay, supervise sidecar
  endpoint).
- **No HTTP API mode.** `smolvm serve` is the long-term-clean
  control plane, but v1 drives smolmachines via CLI subprocess
  invocations — the lower-overhead first iteration the research
  note already endorses.
- **No custom kernel / initrd.** smolmachines uses libkrunfw
  only; the agent image is an OCI ref, not a kernel + rootfs pair.
- **No warm-pool or snapshot/restore.** Each bottle gets a fresh
  microVM; cold-start cost is paid up front.
- **No supervise/agent-credential rewrites for the new backend.**
  Provisioning logic ports as-is; only the *transport* (host-side
  port URLs instead of in-network DNS names) changes.

## Scope

### In scope

- New `claude_bottle/backend/smolmachines/` subpackage with the
  full set of `BottleBackend` overrides.
- Smolfile generator (TOML), analogous to
  `backend/docker/compose.py`'s `bottle_plan_to_compose`.
- A host-side sidecar process manager that owns the lifecycle of
  pipelock + egress + git-gate + supervise for one bottle, binding
  them to per-bottle loopback ports and tearing them down with the
  bottle. This is the smolmachines-specific replacement for
  `docker compose up`/`down`.
- Per-bottle CA install path: the egress sidecar's CA cert lands
  inside the microVM via `smolvm machine exec` after start
  (analogous to the existing `provision_ca` for Docker).
- DNS allowlist plumbing: every host in `bottle.egress.allowlist`
  goes into the Smolfile's DNS filter section (vsock port 6002),
  so the VMM-layer DNS filter and the bottle's policy stay in
  sync — agent can't `dig` its way out via raw IP literals (TSI
  + CIDR allowlist enforces this; DNS filter denies hostname
  resolution).
- Preflight `smolvm` check: if the user selects this backend and
  `smolvm` isn't on `$PATH`, die with an install pointer (brew tap
  + version pin TBD in implementation; see open question 3).
- Manifest validation: refuse any bottle field this backend can't
  honor (today there are none, since the Docker backend already
  rejects host mounts; this is a forward-compat check).
- Tests:
  - Smoke unit-level test: Smolfile renderer produces the
    expected TOML for a fixture bottle.
  - Integration test: `prepare → launch → exec("echo hi") →
    teardown` on a smolmachines-capable host (skips otherwise
    via the same env/platform gate the Docker integration tests
    use).
  - PRD 0022 suite, re-run with the env var flipped, passes.

### Out of scope

- VM image caching across bottles (each prepare rebuilds from the
  OCI archive; layer reuse is whatever smolmachines provides).
- Cross-host bottle relocation (the OCI archive is local-only).
- Operator-facing knobs for vCPU / memory / overlay size (use
  sensible defaults; expose as manifest fields in a later PRD if
  needed).
- Integration with the `supervise` plane's permission-prompt UX
  beyond port plumbing — supervise already speaks HTTP and binds
  to whatever loopback the backend hands it.

## Proposed Design

### Backend layout

```
claude_bottle/backend/smolmachines/
  __init__.py            re-exports SmolmachinesBottleBackend
  backend.py             SmolmachinesBottleBackend façade
  bottle.py              SmolmachinesBottle (exec_claude / exec / cp_in / close)
  bottle_plan.py         SmolmachinesBottlePlan + .print()
  bottle_cleanup_plan.py SmolmachinesBottleCleanupPlan
  prepare.py             resolve_plan(spec, stage_dir, ...) -> SmolmachinesBottlePlan
  launch.py              @contextmanager launch(plan) -> SmolmachinesBottle
  cleanup.py             prepare_cleanup / cleanup / list_active
  smolfile.py            bottle_plan_to_smolfile(...) -> dict + render
  sidecars.py            host-side pipelock/egress/git-gate/supervise lifecycle
  smolvm.py              thin subprocess wrapper: machine create/start/exec/stop
  util.py                slugify, port allocation, OCI archive helpers
  provision/             ca.py, prompt.py, skills.py, git.py, supervise.py
```

### Network + egress topology

```
  ┌── macOS host ─────────────────────────────────────────────┐
  │                                                           │
  │  ┌── per-bottle host sidecars (one set per microVM) ─┐    │
  │  │ pipelock     127.0.0.1:<p1>                       │    │
  │  │ egress       127.0.0.1:<p2>                       │    │
  │  │ git-gate     127.0.0.1:<p3>                       │    │
  │  │ supervise    127.0.0.1:<p4>                       │    │
  │  └───────────────────────────────────────────────────┘    │
  │                          ▲                                │
  │                          │ TSI passthrough (localhost)    │
  │                          │                                │
  │  ┌── libkrun microVM (per bottle) ───────────────────┐    │
  │  │ env: HTTPS_PROXY=http://127.0.0.1:<p1>            │    │
  │  │      EGRESS_URL=http://127.0.0.1:<p2>             │    │
  │  │      GIT_GATE_URL=http://127.0.0.1:<p3>           │    │
  │  │      MCP_SUPERVISE_URL=http://127.0.0.1:<p4>      │    │
  │  │ --outbound-localhost-only                         │    │
  │  │ DNS filter (vsock:6002) → host allowlist          │    │
  │  └───────────────────────────────────────────────────┘    │
  │                                                           │
  └───────────────────────────────────────────────────────────┘
```

Two changes vs. the Docker backend:

1. **Sidecars are host processes, not sibling containers.** No
   internal Docker network; isolation comes from TSI plus the
   per-bottle loopback port set.
2. **The "internal" allowlist becomes localhost-only.** Egress out
   to the public internet still happens through pipelock + egress
   — the same scanning + DLP + auth-injection chain — but the
   agent's first hop is `127.0.0.1:<p1>` reached via TSI, not a
   sidecar's IP on a Docker-managed bridge.

### Lifecycle

`SmolmachinesBottleBackend.prepare(spec, stage_dir)`:

1. Cross-backend validation via `BottleBackend._validate` (skills,
   git identity files).
2. Allocate four loopback ports (bind, get free port, release;
   record on plan).
3. Resolve the agent OCI archive path (build if missing, cache by
   Dockerfile + agent-name hash).
4. Render the per-bottle Smolfile to `stage_dir/smolfile.toml`,
   pinning command/env/`--outbound-localhost-only` + DNS allowlist.
5. Resolve the in-VM CA paths so launch knows where to copy
   pipelock's CA after start.
6. Return a `SmolmachinesBottlePlan` carrying the slug, port map,
   OCI archive path, Smolfile path, and host sidecar specs.

`SmolmachinesBottleBackend.launch(plan)`:

1. Start the four host sidecars in dependency order (pipelock →
   egress → git-gate → supervise), bound to the plan's allocated
   ports. Register teardown callbacks in reverse order.
2. `smolvm machine create --smolfile <path>` and
   `smolvm machine start <name>`.
3. Provisioning: CA install → prompt → skills → git → supervise
   config, each via `smolvm machine exec` (analogous to
   `docker exec`).
4. Yield a `SmolmachinesBottle` whose `exec_claude` / `exec` /
   `cp_in` all funnel through `smolvm machine exec` /
   `smolvm machine cp`.
5. Teardown: stop and remove the VM, then stop the sidecars (in
   reverse start order).

### Data model

No manifest schema change. `bottles[]` continues to carry
`egress.allowlist`, `env`, `git`, `skills` references, etc.; the
smolmachines backend reads the same fields as the docker backend.
The DNS allowlist plumbed into the Smolfile is just
`bottle.egress.allowlist` re-encoded as TOML.

The `BottleSpec` dataclass and the `Bottle` ABC do not change.

### Selection wiring

In `claude_bottle/backend/__init__.py`:

```python
from .docker import DockerBottleBackend
from .smolmachines import SmolmachinesBottleBackend

_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
    "docker": DockerBottleBackend(),
    "smolmachines": SmolmachinesBottleBackend(),
}
```

The existing "unknown backend" `die()` path stays as-is.

### External dependencies

- `smolvm` CLI binary on `$PATH` (one new external dep, gated by
  the preflight check). Pinned version policy is deferred to the
  open questions; v1 reads `smolvm --version` and refuses to launch
  outside a known-good range.
- No new Python packages. Subprocess + stdlib `tomllib`/`tomli_w`
  for Smolfile authoring. (`tomli_w` is the only candidate
  module; if it's not stdlib in the target Python, render TOML
  by hand from a `dict[str, Any]` — Smolfile shape is small.)

### Acceptance test plan

- **Unit:** `tests/unit/test_smolfile.py` verifies the renderer
  produces the expected TOML for a fixture bottle (allowlist →
  DNS rules, env → `env =`, command line, outbound-localhost
  flag).
- **Integration smoke:** `tests/integration/test_smolmachines_smoke.py`
  with `prepare → launch → exec → teardown`, guarded by a
  `smolvm` presence check + macOS / KVM platform check.
- **PRD 0022 re-run:** with `CLAUDE_BOTTLE_BACKEND=smolmachines`,
  all five attack categories return sandbox-block markers and the
  suite passes. The test code does not change beyond the env-var
  flip — that's the contract the PRD 0022 abstraction was
  designed for.

## Sizing — into chunks

1. **Backend skeleton + selection + Smolfile renderer.** Subpackage
   layout, `_resolve_plan` stub that emits a TOML file but doesn't
   launch anything, `_BACKENDS` registration, preflight `smolvm`
   check. Unit test on the renderer. No VM bringup yet.
2. **VM lifecycle + OCI archive build.** `smolvm.py` subprocess
   wrapper, prepare-time image build (existing Dockerfile → OCI
   archive), launch path that creates + starts + stops a VM with
   no sidecars wired. Smoke integration test: `exec("echo hi")`
   inside a started VM.
3. **Host-side sidecar relocation.** `sidecars.py`: per-bottle
   pipelock + egress + git-gate + supervise as host processes on
   loopback. Port allocator. Teardown ordering. No provisioning
   yet beyond what the sidecars need.
4. **Provisioning parity with Docker.** CA install via
   `smolvm machine exec`, prompt/skills/.git copy-in, supervise
   MCP config. End-to-end `start` works for a real agent manifest.
5. **PRD 0022 sandbox-escape suite green.** Skip-guard update,
   small adjustments to test helpers if any (the test uses
   `bottle.exec(script)` and inspects `returncode` + body for
   sandbox markers — should be transport-agnostic, but verify).
   Document the macOS-only scope in README.

## Open questions

1. **Sidecar locality: host process vs in-VM init.** This PRD
   defaults to host-process sidecars (proposed design above). The
   alternative — bake pipelock + egress + git-gate + supervise
   into the OCI image and start them via init in the same VM —
   would simplify port plumbing (the agent reaches sidecars over
   localhost inside the VM, not over TSI) but expands the trust
   boundary of the agent VM. Default A unless someone identifies
   a TSI loopback edge case during chunk 3.
2. **`smolvm` install policy.** Pin via brew formula version, or
   build-from-source step, or vendored binary checked into the
   repo. v1 most likely runs `smolvm --version` at preflight and
   accepts a documented range; vendoring is heavier but reduces
   "works on my Mac" drift.
3. **CA install inside the OCI overlay.** Two paths: bake at
   prepare time (one OCI archive per CA fingerprint, big cache
   key) vs. inject at start time via `smolvm machine exec` after
   the VM is up. PRD 0006 chose the runtime path for Docker
   (docker-cp + `update-ca-certificates`); smolvm has the same
   shape via `machine exec`. Default to runtime injection unless
   it conflicts with `--outbound-localhost-only` start order.
4. **DNS filter granularity.** smolmachines's vsock-6002 filter
   accepts an allowlist of hostnames; we want to enforce both
   "agent can only resolve names on the bottle's allowlist" *and*
   "agent can only egress via TSI to 127.0.0.1." Confirm
   empirically (smoke test in chunk 2) that the allowlist applies
   to *guest-initiated* DNS only and doesn't accidentally NXDOMAIN
   the host-side pipelock's upstream lookups.
5. **`bottle.exec(script)` exit-code fidelity.** The PRD 0022 test
   suite reads `returncode` + stdout + stderr from
   `ExecResult`. Confirm `smolvm machine exec` propagates exit
   codes and separated streams — the research note's
   "external integration is the CLI" implies yes, but the
   embedded SDK bug it flagged suggests we should verify before
   coding around it.
6. **CI gating.** Gitea's act_runner is Linux without nested KVM,
   so smolmachines integration tests will skip there for the same
   structural reason the Docker bringup tests do (no real
   isolation primitive available on the runner). The skip
   predicate becomes `not (smolvm_available() and
   (platform.system() == "Darwin" or kvm_available()))`. CI
   coverage for this backend will come from local runs on the
   maintainer's macOS host until a Darwin runner is wired up;
   ack that as a known gap.
7. **Active bottle discovery.** Docker uses container labels to
   enumerate active bottles (`list_active` queries the daemon).
   smolmachines's enumeration story is `smolvm machine list`; the
   plan is to mirror the label scheme via Smolfile metadata
   (`labels = { "claude-bottle" = "1" }`-style entries, if the
   format supports it; otherwise via a deterministic name prefix
   `claude-bottle-<slug>`).

## References

- `docs/research/smolmachines-as-vm-backend.md` — primary research
  note recommending this adoption; PRD 0023's design hypothesis.
- `docs/research/agent-vm-isolation.md` — the broader microVM /
  gvproxy / pipelock landscape this PRD lands inside of.
- `docs/research/agent-sandbox-landscape.md` — identifies
  `"runtime": "microvm"`-style opt-in as the borrowable idea;
  smolmachines is the concrete implementation.
- PRD 0003 (`docs/prds/0003-bottle-backend-abstraction.md`) — the
  backend abstraction this PRD is the first non-Docker consumer
  of.
- PRD 0017 (`docs/prds/0017-egress-proxy-via-mitmproxy.md`) — the
  egress sidecar the host-side relocation reuses verbatim, only
  with a different transport.
- PRD 0022
  (`docs/prds/0022-sandbox-escape-integration-test.md`) — the
  acceptance gate for this PRD; the suite already runs through
  `get_bottle_backend()` so the env-var flip is the only change
  needed to exercise the smolmachines path.