docs(prd-0023): make gvproxy the network primitive; reject TSI
TSI's --outbound-localhost-only is permissive on all of 127.0.0.0/8 with no destination-port filter, so any host loopback service (local Postgres, IDE plugins, another bottle's sidecar) is reachable from the guest. That's the wrong default for the malicious-agent threat model. Reworked the network design around gvproxy + VFKT unixgram attachment: the guest gets a virtio-net device, gvproxy is the userspace TCP/IP stack on the host side, and the only thing reachable from the guest is the explicit port-forward list (typically just pipelock). Host LAN, host loopback, and the public internet directly are gone by construction. VMM choice (smolmachines vs PyObjC + Virtualization.framework) is an open question contingent on whether libkrun's virtio-net mode lets us point at a custom unixgram socket. Backend name stays "smolmachines" either way per the original spec. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -6,31 +6,54 @@
|
||||
|
||||
## Summary
|
||||
|
||||
Ship a second concrete `BottleBackend` — `SmolmachinesBottleBackend`,
|
||||
selected via `CLAUDE_BOTTLE_BACKEND=smolmachines` — that runs a
|
||||
bottle inside a per-agent libkrun microVM on macOS (and KVM on Linux,
|
||||
opportunistically). The egress topology moves out of an internal
|
||||
Docker network and onto libkrun's TSI ("Transport Socket Interface")
|
||||
allowlist plus a host-side pipelock/egress/git-gate/supervise stack
|
||||
listening on per-bottle loopback ports. The Docker backend ships
|
||||
unchanged; this is opt-in via the existing env-var selector.
|
||||
Ship a second concrete `BottleBackend` —
|
||||
`SmolmachinesBottleBackend`, selected via
|
||||
`CLAUDE_BOTTLE_BACKEND=smolmachines` — that runs each bottle inside
|
||||
a per-agent microVM on macOS. The egress topology is enforced by
|
||||
**gvproxy** (gvisor-tap-vsock), a userspace TCP/IP stack the guest's
|
||||
virtio-net device is wired into via `VZFileHandleNetworkDeviceAttachment`.
|
||||
gvproxy's only outbound configuration is an explicit per-bottle port
|
||||
forward to a host-side pipelock; everything else — the host's LAN,
|
||||
the host's loopback services, the public internet — is unreachable
|
||||
from the guest by construction. pipelock + egress + git-gate +
|
||||
supervise stay as host-side processes on per-bottle loopback ports,
|
||||
reached *only* through gvproxy's forwarded ports.
|
||||
|
||||
The acceptance gate is PRD 0022's `tests/integration/test_sandbox_escape.py`
|
||||
running green against `CLAUDE_BOTTLE_BACKEND=smolmachines`.
|
||||
This explicitly rejects libkrun's TSI ("Transport Socket Interface")
|
||||
allowlist as the network primitive. TSI's `--outbound-localhost-only`
|
||||
is permissive on the entire `127.0.0.0/8` range with no
|
||||
destination-port filter — the agent can dial any host-side service
|
||||
bound to loopback (a local Postgres, an IDE plugin, a different
|
||||
bottle's pipelock). That's the wrong default for a malicious-agent
|
||||
threat model; see "Why gvproxy, not TSI" below.
|
||||
|
||||
The Docker backend ships unchanged; this is opt-in via the existing
|
||||
env-var selector. The acceptance gate is PRD 0022's
|
||||
`tests/integration/test_sandbox_escape.py` running green against
|
||||
`CLAUDE_BOTTLE_BACKEND=smolmachines`.
|
||||
|
||||
## Problem
|
||||
|
||||
`agent-vm-isolation.md` argues for hardware-isolated microVMs over
|
||||
container-based bottles on macOS; `smolmachines-as-vm-backend.md`
|
||||
concludes that smolmachines is the most plausible concrete VMM for
|
||||
this project. Today, the only backend in the registry is Docker
|
||||
evaluates smolmachines as the lifecycle wrapper. Today, the only
|
||||
backend in the registry is Docker
|
||||
(`claude_bottle/backend/__init__.py:_BACKENDS = {"docker": ...}`),
|
||||
and three things motivate a second one now:
|
||||
and four things motivate a second one now:
|
||||
|
||||
- **Network reach beyond pipelock.** The threat model is a malicious
|
||||
agent attempting to dial something on the operator's *local
|
||||
network* (`192.168.x.x` services, the home router, a coworker's
|
||||
laptop on the same Wi-Fi) or *host's loopback* (a local database,
|
||||
an editor plugin, another bottle's sidecar) without traversing
|
||||
pipelock. The Docker backend's `--internal` network blocks the
|
||||
first; nothing in the current stack blocks the second cleanly.
|
||||
This PRD's gvproxy-based design closes both gaps: the guest can
|
||||
only reach the explicit port-forward list, period.
|
||||
- **Isolation ceiling.** On macOS the Docker backend's agent
|
||||
container shares Docker Desktop's host VM with every other bottle.
|
||||
Container escape from claude-code lands the agent inside that
|
||||
shared VM. A per-bottle libkrun microVM gets hardware page tables
|
||||
container shares Docker Desktop's host VM with every other
|
||||
bottle. Container escape from claude-code lands the agent inside
|
||||
that shared VM. A per-bottle microVM gets hardware page tables
|
||||
via `Hypervisor.framework`; cross-bottle isolation becomes
|
||||
enforced by the CPU's MMU instead of namespace bookkeeping.
|
||||
- **PRD 0022 is backend-agnostic by design** but currently only
|
||||
@@ -41,15 +64,36 @@ and three things motivate a second one now:
|
||||
unproven.
|
||||
- **CI carve-outs.** Most bottle-bringup integration tests skip
|
||||
under `GITEA_ACTIONS=true` because act_runner shares the host
|
||||
Docker socket but not the host filesystem. A smolmachines path
|
||||
Docker socket but not the host filesystem. A microVM path
|
||||
doesn't share that constraint shape (it has its own, but
|
||||
different), so adding the backend forces the abstraction to be
|
||||
clean in places where Docker-specific assumptions have been
|
||||
tolerated.
|
||||
|
||||
The smolmachines research note's `## Recommendation` ("adopt
|
||||
smolmachines as the bottle VM backend on macOS; keep pipelock DIY")
|
||||
is the design hypothesis under test here.
|
||||
## Why gvproxy, not TSI
|
||||
|
||||
libkrun's TSI hijacks guest socket syscalls inside the VMM and
|
||||
opens the actual sockets from the host process, with a CIDR
|
||||
allowlist gate. That works fine for blocking LAN reach (don't
|
||||
allowlist `192.168.0.0/16`, agent can't dial it). But TSI's
|
||||
`--outbound-localhost-only` permits the *entire* `127.0.0.0/8`
|
||||
range across all ports — there is no destination-port filter at
|
||||
the TSI layer (`smolmachines-as-vm-backend.md` flags this in the
|
||||
"`--allow-host` semantics" caveat). For our threat model that
|
||||
means any host-loopback service is reachable from the guest.
|
||||
|
||||
gvproxy implements a full userspace TCP/IP stack on the host side
|
||||
of a `VZFileHandleNetworkDeviceAttachment` unixgram socket. The
|
||||
guest has a real virtio-net device; gvproxy is its gateway. The
|
||||
guest can only reach what gvproxy is configured to forward —
|
||||
typically a single port forward to the per-bottle pipelock —
|
||||
and DNS resolves NXDOMAIN by default. There is no "permissive
|
||||
loopback" mode to mis-configure; if it's not in `port_forwards`,
|
||||
the guest cannot reach it.
|
||||
|
||||
That property — *explicit allowlist by port forward, not CIDR* —
|
||||
is the load-bearing reason this PRD chooses gvproxy. TSI shows up
|
||||
once more in this doc, under Non-goals, where it is closed off.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
@@ -84,13 +128,25 @@ The feature is **done** when all of the following ship:
|
||||
- Per-bottle Smolfile generation: a runtime-rendered TOML written
|
||||
to the bottle's stage dir, analogous to the compose file the
|
||||
Docker backend writes today. The Smolfile pins `command`,
|
||||
`env`, `--outbound-localhost-only`, and the per-bottle DNS
|
||||
allowlist.
|
||||
`env`, and a virtio-net device backed by a unixgram socket
|
||||
pointed at the per-bottle gvproxy. There is no TSI
|
||||
`--allow-cidr` / `--outbound-localhost-only` / `--allow-host`
|
||||
in the Smolfile — TSI is not used.
|
||||
- Per-bottle gvproxy: one `gvproxy` process per bottle, started
|
||||
before the VM, listening on a unixgram socket the VM's
|
||||
virtio-net device hooks into. The gvproxy config has exactly
|
||||
one `port_forwards` entry — gateway-port to the per-bottle
|
||||
pipelock's host port — and a DNS section that resolves only
|
||||
`proxy.internal`. Every other hostname returns NXDOMAIN; every
|
||||
other destination is unreachable.
|
||||
- Host-side sidecar relocation: pipelock, egress, git-gate, and
|
||||
supervise each run as host processes (one set per bottle),
|
||||
bound to `127.0.0.1` on per-bottle dynamically-allocated ports.
|
||||
The agent's environment carries the resolved URLs (e.g.
|
||||
`HTTPS_PROXY=http://127.0.0.1:<pipelock-port>`).
|
||||
`HTTPS_PROXY=http://proxy.internal:<pipelock-gateway-port>`).
|
||||
Only pipelock is exposed through gvproxy; egress / git-gate /
|
||||
supervise are chained *behind* pipelock on the host side and
|
||||
are not reachable directly from the guest.
|
||||
- The agent guest image is produced from the existing `Dockerfile`
|
||||
(or a thin variant), exported as an OCI archive, and consumed by
|
||||
`smolvm machine create`. The image build step is part of `prepare`,
|
||||
@@ -117,6 +173,15 @@ The feature is **done** when all of the following ship:
|
||||
value of `CLAUDE_BOTTLE_BACKEND`; smolmachines is strictly
|
||||
opt-in until it has been load-bearing on at least one operator's
|
||||
workflow for a release cycle.
|
||||
- **No TSI for network policy.** libkrun's TSI mode is rejected
|
||||
for this backend — it lacks per-port filtering on `127.0.0.0/8`
|
||||
and would expose every host-loopback service to the guest. The
|
||||
Smolfile must select libkrun's virtio-net mode and attach to
|
||||
the per-bottle gvproxy unixgram socket; if that combination is
|
||||
not supported by the pinned smolmachines version (see open
|
||||
question 1), the implementation falls back to driving
|
||||
Virtualization.framework directly via PyObjC and reuses the
|
||||
same gvproxy attachment.
|
||||
- **No host bind mounts.** The smolmachines research note flagged
|
||||
that `-v HOST:GUEST` mounts via virtiofs would defeat the
|
||||
isolation goal. The manifest already has no concept of host
|
||||
@@ -152,12 +217,18 @@ The feature is **done** when all of the following ship:
|
||||
- Per-bottle CA install path: the egress sidecar's CA cert lands
|
||||
inside the microVM via `smolvm machine exec` after start
|
||||
(analogous to the existing `provision_ca` for Docker).
|
||||
- DNS allowlist plumbing: every host in `bottle.egress.allowlist`
|
||||
goes into the Smolfile's DNS filter section (vsock port 6002),
|
||||
so the VMM-layer DNS filter and the bottle's policy stay in
|
||||
sync — agent can't `dig` its way out via raw IP literals (TSI
|
||||
+ CIDR allowlist enforces this; DNS filter denies hostname
|
||||
resolution).
|
||||
- gvproxy lifecycle: per-bottle `gvproxy` started by the backend
|
||||
before VM bringup, torn down after VM teardown, configured with
|
||||
one `port_forwards` entry (gateway → host pipelock port) and a
|
||||
DNS section that resolves only `proxy.internal`. Subnet and
|
||||
gateway IP are derived from the bottle slug so two concurrent
|
||||
bottles don't collide.
|
||||
- DNS policy: the bottle's `egress.allowlist` does *not* go into
|
||||
gvproxy's DNS — the agent resolves only `proxy.internal`, and
|
||||
pipelock on the host enforces the egress allowlist against
|
||||
the actual upstream connect target. This keeps the DNS-exfil
|
||||
attack (PRD 0022 test 4) blocked because gvproxy answers
|
||||
NXDOMAIN for every name except `proxy.internal`.
|
||||
- Preflight `smolvm` check: if the user selects this backend and
|
||||
`smolvm` isn't on `$PATH`, die with an install pointer (brew tap
|
||||
+ version pin TBD in implementation; see open question 3).
|
||||
@@ -200,8 +271,10 @@ claude_bottle/backend/smolmachines/
|
||||
launch.py @contextmanager launch(plan) -> SmolmachinesBottle
|
||||
cleanup.py prepare_cleanup / cleanup / list_active
|
||||
smolfile.py bottle_plan_to_smolfile(...) -> dict + render
|
||||
gvproxy.py per-bottle gvproxy config render + process lifecycle
|
||||
sidecars.py host-side pipelock/egress/git-gate/supervise lifecycle
|
||||
smolvm.py thin subprocess wrapper: machine create/start/exec/stop
|
||||
vfkit_attach.py VZFileHandleNetworkDeviceAttachment + VFKT handshake
|
||||
util.py slugify, port allocation, OCI archive helpers
|
||||
provision/ ca.py, prompt.py, skills.py, git.py, supervise.py
|
||||
```
|
||||
@@ -209,39 +282,65 @@ claude_bottle/backend/smolmachines/
|
||||
### Network + egress topology
|
||||
|
||||
```
|
||||
┌── macOS host ─────────────────────────────────────────────┐
|
||||
│ │
|
||||
│ ┌── per-bottle host sidecars (one set per microVM) ─┐ │
|
||||
│ │ pipelock 127.0.0.1:<p1> │ │
|
||||
│ │ egress 127.0.0.1:<p2> │ │
|
||||
│ │ git-gate 127.0.0.1:<p3> │ │
|
||||
│ │ supervise 127.0.0.1:<p4> │ │
|
||||
│ └───────────────────────────────────────────────────┘ │
|
||||
│ ▲ │
|
||||
│ │ TSI passthrough (localhost) │
|
||||
│ │ │
|
||||
│ ┌── libkrun microVM (per bottle) ───────────────────┐ │
|
||||
│ │ env: HTTPS_PROXY=http://127.0.0.1:<p1> │ │
|
||||
│ │ EGRESS_URL=http://127.0.0.1:<p2> │ │
|
||||
│ │ GIT_GATE_URL=http://127.0.0.1:<p3> │ │
|
||||
│ │ MCP_SUPERVISE_URL=http://127.0.0.1:<p4> │ │
|
||||
│ │ --outbound-localhost-only │ │
|
||||
│ │ DNS filter (vsock:6002) → host allowlist │ │
|
||||
│ └───────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└───────────────────────────────────────────────────────────┘
|
||||
┌── macOS host ─────────────────────────────────────────────────────┐
|
||||
│ │
|
||||
│ ┌── per-bottle sidecar chain (one set per microVM) ────┐ │
|
||||
│ │ agent ──HTTPS_PROXY──► pipelock ──► egress ──► internet │
|
||||
│ │ 127.0.0.1:p1 (DLP) (MITM, │
|
||||
│ │ auth-inject) │
|
||||
│ │ │
|
||||
│ │ git push ──► git-gate ──► upstream │
|
||||
│ │ 127.0.0.1:p3 (gitleaks) │
|
||||
│ │ │
|
||||
│ │ MCP ──► supervise 127.0.0.1:p4 │
|
||||
│ └────────────────────────────────────────────────────────────────┘
|
||||
│ ▲ host TCP, reached via gvproxy port-forward │
|
||||
│ │ │
|
||||
│ ┌── gvproxy (per bottle) ─────────────────────────────┐ │
|
||||
│ │ subnet: 192.168.127.X/24 (X derived from slug) │ │
|
||||
│ │ gateway: 192.168.127.X.1 │ │
|
||||
│ │ port_forwards: │ │
|
||||
│ │ - gateway 8888 → host 127.0.0.1:<p1> │ │
|
||||
│ │ # nothing else │ │
|
||||
│ │ DNS: proxy.internal → gateway IP; * → NXDOMAIN │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ ▲ unixgram socket (VFKT handshake) │
|
||||
│ │ │
|
||||
│ ┌── microVM (per bottle) ─────────────────────────────┐ │
|
||||
│ │ virtio-net device backed by VZFileHandle... │ │
|
||||
│ │ env: HTTPS_PROXY=http://proxy.internal:8888 │ │
|
||||
│ │ GIT_GATE_URL=http://proxy.internal:8889 │ │
|
||||
│ │ MCP_SUPERVISE_URL=http://proxy.internal:8890 │ │
|
||||
│ │ no other host visible │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└───────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
What the guest can reach, exhaustively: **only `proxy.internal`
|
||||
on the gateway-port set we configured.** Everything else —
|
||||
host LAN, host loopback (Postgres, IDE plugins, other bottles'
|
||||
sidecars), public internet directly — is gone, enforced at the
|
||||
gvproxy userspace stack rather than relying on guest cooperation.
|
||||
|
||||
Two changes vs. the Docker backend:
|
||||
|
||||
1. **Sidecars are host processes, not sibling containers.** No
|
||||
internal Docker network; isolation comes from TSI plus the
|
||||
per-bottle loopback port set.
|
||||
2. **The "internal" allowlist becomes localhost-only.** Egress out
|
||||
to the public internet still happens through pipelock + egress
|
||||
— the same scanning + DLP + auth-injection chain — but the
|
||||
agent's first hop is `127.0.0.1:<p1>` reached via TSI, not a
|
||||
sidecar's IP on a Docker-managed bridge.
|
||||
internal Docker network. The isolation primitive is gvproxy's
|
||||
explicit port-forward list, not Docker's `--internal` flag.
|
||||
2. **The agent's first hop is `proxy.internal`, not a sidecar's
|
||||
container hostname.** Egress out to the public internet still
|
||||
happens through pipelock + egress — same scanning + DLP +
|
||||
auth-injection chain — but the first hop crosses a userspace
|
||||
TCP/IP stack we own, not a Docker-managed bridge.
|
||||
|
||||
The chain `agent → pipelock → egress → internet` collapses on
|
||||
the host side: pipelock listens on 127.0.0.1:p1, makes its
|
||||
upstream connect against egress at 127.0.0.1:p2, which makes its
|
||||
upstream connect against the public internet. git-gate and
|
||||
supervise are separate gateway ports if and only if the bottle
|
||||
uses them — otherwise they're omitted from gvproxy's
|
||||
`port_forwards`, narrowing the attack surface further.
|
||||
|
||||
### Lifecycle
|
||||
|
||||
@@ -249,40 +348,56 @@ Two changes vs. the Docker backend:
|
||||
|
||||
1. Cross-backend validation via `BottleBackend._validate` (skills,
|
||||
git identity files).
|
||||
2. Allocate four loopback ports (bind, get free port, release;
|
||||
record on plan).
|
||||
2. Allocate host loopback ports for each sidecar the bottle uses
|
||||
(pipelock always; egress / git-gate / supervise conditional on
|
||||
manifest).
|
||||
3. Resolve the agent OCI archive path (build if missing, cache by
|
||||
Dockerfile + agent-name hash).
|
||||
4. Render the per-bottle Smolfile to `stage_dir/smolfile.toml`,
|
||||
pinning command/env/`--outbound-localhost-only` + DNS allowlist.
|
||||
5. Resolve the in-VM CA paths so launch knows where to copy
|
||||
4. Pick a per-bottle gvproxy subnet (e.g. `192.168.127.X/24` where
|
||||
`X` is derived from the slug) and render
|
||||
`stage_dir/gvproxy.yaml`: one DNS entry for `proxy.internal`,
|
||||
one `port_forwards` entry per active sidecar (gateway port →
|
||||
host loopback port).
|
||||
5. Render the per-bottle Smolfile to `stage_dir/smolfile.toml`,
|
||||
pinning command / env / a virtio-net device backed by the
|
||||
gvproxy unixgram socket path. No TSI flags.
|
||||
6. Resolve the in-VM CA paths so launch knows where to copy
|
||||
pipelock's CA after start.
|
||||
6. Return a `SmolmachinesBottlePlan` carrying the slug, port map,
|
||||
OCI archive path, Smolfile path, and host sidecar specs.
|
||||
7. Return a `SmolmachinesBottlePlan` carrying the slug, port map,
|
||||
OCI archive path, Smolfile path, gvproxy config path, and
|
||||
host sidecar specs.
|
||||
|
||||
`SmolmachinesBottleBackend.launch(plan)`:
|
||||
|
||||
1. Start the four host sidecars in dependency order (pipelock →
|
||||
egress → git-gate → supervise), bound to the plan's allocated
|
||||
ports. Register teardown callbacks in reverse order.
|
||||
2. `smolvm machine create --smolfile <path>` and
|
||||
`smolvm machine start <name>`.
|
||||
3. Provisioning: CA install → prompt → skills → git → supervise
|
||||
1. Start host sidecars in dependency order (egress → pipelock →
|
||||
git-gate → supervise — egress before pipelock so pipelock's
|
||||
upstream resolves; pipelock is the only one exposed through
|
||||
gvproxy). Register teardown callbacks in reverse order.
|
||||
2. Start the per-bottle `gvproxy` against the unixgram socket
|
||||
path the Smolfile references. Wait for the socket to appear
|
||||
(the spike-style poll loop from `agent-vm-isolation.md`).
|
||||
3. `smolvm machine create --smolfile <path>` and
|
||||
`smolvm machine start <name>`. The Smolfile's virtio-net
|
||||
device handshakes (`VFKT` magic) with gvproxy on start.
|
||||
4. Provisioning: CA install → prompt → skills → git → supervise
|
||||
config, each via `smolvm machine exec` (analogous to
|
||||
`docker exec`).
|
||||
4. Yield a `SmolmachinesBottle` whose `exec_claude` / `exec` /
|
||||
5. Yield a `SmolmachinesBottle` whose `exec_claude` / `exec` /
|
||||
`cp_in` all funnel through `smolvm machine exec` /
|
||||
`smolvm machine cp`.
|
||||
5. Teardown: stop and remove the VM, then stop the sidecars (in
|
||||
reverse start order).
|
||||
6. Teardown: stop and remove the VM → stop gvproxy → stop
|
||||
sidecars (in reverse start order).
|
||||
|
||||
### Data model
|
||||
|
||||
No manifest schema change. `bottles[]` continues to carry
|
||||
`egress.allowlist`, `env`, `git`, `skills` references, etc.; the
|
||||
smolmachines backend reads the same fields as the docker backend.
|
||||
The DNS allowlist plumbed into the Smolfile is just
|
||||
`bottle.egress.allowlist` re-encoded as TOML.
|
||||
`egress.allowlist` is enforced by pipelock on the host side
|
||||
(unchanged from the docker backend); gvproxy's DNS resolves only
|
||||
`proxy.internal` regardless of the allowlist's contents, so an
|
||||
agent that bypasses pipelock by raw IP cannot resolve any name
|
||||
gvproxy doesn't know about.
|
||||
|
||||
The `BottleSpec` dataclass and the `Bottle` ABC do not change.
|
||||
|
||||
@@ -308,20 +423,36 @@ The existing "unknown backend" `die()` path stays as-is.
|
||||
the preflight check). Pinned version policy is deferred to the
|
||||
open questions; v1 reads `smolvm --version` and refuses to launch
|
||||
outside a known-good range.
|
||||
- No new Python packages. Subprocess + stdlib `tomllib`/`tomli_w`
|
||||
for Smolfile authoring. (`tomli_w` is the only candidate
|
||||
module; if it's not stdlib in the target Python, render TOML
|
||||
by hand from a `dict[str, Any]` — Smolfile shape is small.)
|
||||
- `gvproxy` binary on `$PATH`
|
||||
(`go install github.com/containers/gvisor-tap-vsock/cmd/gvproxy@latest`,
|
||||
or vendored). Same preflight pattern as `smolvm`.
|
||||
- `pyobjc-framework-Virtualization` *only* if smolmachines does
|
||||
not expose a way to attach virtio-net to a unixgram socket and
|
||||
we fall back to driving Virtualization.framework directly (see
|
||||
open question 1). Default path is "no PyObjC needed."
|
||||
- No new pure-Python packages. Subprocess + stdlib `tomllib` for
|
||||
Smolfile authoring; the gvproxy YAML is small enough to render
|
||||
by hand from a `dict[str, Any]`.
|
||||
|
||||
### Acceptance test plan
|
||||
|
||||
- **Unit:** `tests/unit/test_smolfile.py` verifies the renderer
|
||||
produces the expected TOML for a fixture bottle (allowlist →
|
||||
DNS rules, env → `env =`, command line, outbound-localhost
|
||||
flag).
|
||||
- **Unit (smolfile):** `tests/unit/test_smolfile.py` verifies the
|
||||
renderer produces the expected TOML for a fixture bottle —
|
||||
command line, env entries, virtio-net device referencing the
|
||||
expected unixgram socket path, no TSI flags.
|
||||
- **Unit (gvproxy config):** `tests/unit/test_gvproxy_config.py`
|
||||
verifies the per-bottle YAML has exactly one DNS entry
|
||||
(`proxy.internal`), one `port_forwards` entry per active
|
||||
sidecar pointed at the resolved host loopback port, and a
|
||||
per-bottle subnet/gateway derived from the slug.
|
||||
- **Integration smoke:** `tests/integration/test_smolmachines_smoke.py`
|
||||
with `prepare → launch → exec → teardown`, guarded by a
|
||||
`smolvm` presence check + macOS / KVM platform check.
|
||||
`smolvm` + `gvproxy` presence check + macOS / KVM platform check.
|
||||
- **Localhost-reach probe:** a focused integration test that
|
||||
brings up a bottle, has the host bind a test service on
|
||||
`127.0.0.1:<unused-port>`, and asserts the in-bottle agent
|
||||
cannot connect to it. This is the regression test for the
|
||||
exact gap that motivated choosing gvproxy over TSI.
|
||||
- **PRD 0022 re-run:** with `CLAUDE_BOTTLE_BACKEND=smolmachines`,
|
||||
all five attack categories return sandbox-block markers and the
|
||||
suite passes. The test code does not change beyond the env-var
|
||||
@@ -330,19 +461,24 @@ The existing "unknown backend" `die()` path stays as-is.
|
||||
|
||||
## Sizing — into chunks
|
||||
|
||||
1. **Backend skeleton + selection + Smolfile renderer.** Subpackage
|
||||
layout, `_resolve_plan` stub that emits a TOML file but doesn't
|
||||
launch anything, `_BACKENDS` registration, preflight `smolvm`
|
||||
check. Unit test on the renderer. No VM bringup yet.
|
||||
2. **VM lifecycle + OCI archive build.** `smolvm.py` subprocess
|
||||
wrapper, prepare-time image build (existing Dockerfile → OCI
|
||||
archive), launch path that creates + starts + stops a VM with
|
||||
no sidecars wired. Smoke integration test: `exec("echo hi")`
|
||||
inside a started VM.
|
||||
1. **Backend skeleton + selection + Smolfile + gvproxy renderers.**
|
||||
Subpackage layout, `_resolve_plan` stub that emits both a
|
||||
TOML Smolfile and a gvproxy YAML but doesn't launch anything,
|
||||
`_BACKENDS` registration, preflight `smolvm` + `gvproxy`
|
||||
checks. Unit tests on both renderers. No VM bringup yet.
|
||||
2. **gvproxy + VM lifecycle + OCI archive build.** `smolvm.py`
|
||||
and `gvproxy.py` subprocess wrappers, prepare-time image
|
||||
build (existing Dockerfile → OCI archive), launch path that
|
||||
starts gvproxy, brings up the VM attached to gvproxy's socket
|
||||
via VFKT handshake, exec into the VM, tear everything down.
|
||||
Smoke integration test: `exec("echo hi")` inside a started
|
||||
VM. Includes the localhost-reach probe test from the
|
||||
acceptance plan.
|
||||
3. **Host-side sidecar relocation.** `sidecars.py`: per-bottle
|
||||
pipelock + egress + git-gate + supervise as host processes on
|
||||
loopback. Port allocator. Teardown ordering. No provisioning
|
||||
yet beyond what the sidecars need.
|
||||
loopback, with gvproxy `port_forwards` wired only for the
|
||||
sidecars the bottle actually uses. Port allocator. Teardown
|
||||
ordering. No provisioning yet beyond what the sidecars need.
|
||||
4. **Provisioning parity with Docker.** CA install via
|
||||
`smolvm machine exec`, prompt/skills/.git copy-in, supervise
|
||||
MCP config. End-to-end `start` works for a real agent manifest.
|
||||
@@ -354,63 +490,78 @@ The existing "unknown backend" `die()` path stays as-is.
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **Sidecar locality: host process vs in-VM init.** This PRD
|
||||
defaults to host-process sidecars (proposed design above). The
|
||||
alternative — bake pipelock + egress + git-gate + supervise
|
||||
into the OCI image and start them via init in the same VM —
|
||||
would simplify port plumbing (the agent reaches sidecars over
|
||||
localhost inside the VM, not over TSI) but expands the trust
|
||||
boundary of the agent VM. Default A unless someone identifies
|
||||
a TSI loopback edge case during chunk 3.
|
||||
2. **`smolvm` install policy.** Pin via brew formula version, or
|
||||
build-from-source step, or vendored binary checked into the
|
||||
repo. v1 most likely runs `smolvm --version` at preflight and
|
||||
accepts a documented range; vendoring is heavier but reduces
|
||||
"works on my Mac" drift.
|
||||
1. **VMM choice: smolmachines vs PyObjC + Virtualization.framework.**
|
||||
The network design requires libkrun's virtio-net mode attached
|
||||
to a unixgram socket (so gvproxy is the gateway). The
|
||||
smolmachines research note says libkrun *has* a virtio-net
|
||||
mode but says it "does not support policy" — meaning libkrun
|
||||
itself enforces no allowlist in that mode, which is exactly
|
||||
what we want (gvproxy is the policy). What's unverified is
|
||||
whether the Smolfile surface lets us point virtio-net at a
|
||||
custom unixgram socket. If yes: this is a smolmachines backend
|
||||
verbatim. If no: chunk 2 drops `smolvm` and drives
|
||||
`Virtualization.framework` via PyObjC directly (the recipe in
|
||||
`agent-vm-isolation.md` § "gvisor-tap-vsock + PyObjC +
|
||||
Pipelock"), keeping the backend name "smolmachines" because
|
||||
the operator-facing UX is unchanged. Resolve in chunk 1 via a
|
||||
spike against the pinned smolmachines version.
|
||||
2. **`smolvm` + `gvproxy` install policy.** Pin via brew /
|
||||
`go install` versions, or vendor binaries in the repo. v1
|
||||
likely runs `smolvm --version` / `gvproxy --help` at preflight
|
||||
and accepts a documented range; vendoring is heavier but
|
||||
reduces "works on my Mac" drift.
|
||||
3. **CA install inside the OCI overlay.** Two paths: bake at
|
||||
prepare time (one OCI archive per CA fingerprint, big cache
|
||||
key) vs. inject at start time via `smolvm machine exec` after
|
||||
the VM is up. PRD 0006 chose the runtime path for Docker
|
||||
(docker-cp + `update-ca-certificates`); smolvm has the same
|
||||
shape via `machine exec`. Default to runtime injection unless
|
||||
it conflicts with `--outbound-localhost-only` start order.
|
||||
4. **DNS filter granularity.** smolmachines's vsock-6002 filter
|
||||
accepts an allowlist of hostnames; we want to enforce both
|
||||
"agent can only resolve names on the bottle's allowlist" *and*
|
||||
"agent can only egress via TSI to 127.0.0.1." Confirm
|
||||
empirically (smoke test in chunk 2) that the allowlist applies
|
||||
to *guest-initiated* DNS only and doesn't accidentally NXDOMAIN
|
||||
the host-side pipelock's upstream lookups.
|
||||
it conflicts with VM start order.
|
||||
4. **gvproxy subnet collision.** Two concurrent bottles must not
|
||||
land on the same `192.168.127.X/24` subnet — they'd both want
|
||||
the same gateway IP. Derive the third octet from a hash of
|
||||
the slug (mod 254, skip the docker-default 17), and at launch
|
||||
time confirm the subnet isn't already in use by another
|
||||
bottle's gvproxy. Resolve the hash-collision policy in
|
||||
chunk 2.
|
||||
5. **`bottle.exec(script)` exit-code fidelity.** The PRD 0022 test
|
||||
suite reads `returncode` + stdout + stderr from
|
||||
`ExecResult`. Confirm `smolvm machine exec` propagates exit
|
||||
codes and separated streams — the research note's
|
||||
"external integration is the CLI" implies yes, but the
|
||||
embedded SDK bug it flagged suggests we should verify before
|
||||
coding around it.
|
||||
`ExecResult`. Confirm the VM-exec path (`smolvm machine exec`
|
||||
or its PyObjC equivalent) propagates exit codes and separated
|
||||
streams. The research note's "external integration is the CLI"
|
||||
implies yes, but the embedded SDK bug it flagged suggests we
|
||||
should verify before coding around it.
|
||||
6. **CI gating.** Gitea's act_runner is Linux without nested KVM,
|
||||
so smolmachines integration tests will skip there for the same
|
||||
structural reason the Docker bringup tests do (no real
|
||||
so this backend's integration tests will skip there for the
|
||||
same structural reason the Docker bringup tests do (no real
|
||||
isolation primitive available on the runner). The skip
|
||||
predicate becomes `not (smolvm_available() and
|
||||
(platform.system() == "Darwin" or kvm_available()))`. CI
|
||||
coverage for this backend will come from local runs on the
|
||||
maintainer's macOS host until a Darwin runner is wired up;
|
||||
ack that as a known gap.
|
||||
predicate becomes `not (smolvm_available() and gvproxy_available()
|
||||
and platform.system() == "Darwin")`. CI coverage for this
|
||||
backend will come from local runs on the maintainer's macOS
|
||||
host until a Darwin runner is wired up; ack that as a known
|
||||
gap.
|
||||
7. **Active bottle discovery.** Docker uses container labels to
|
||||
enumerate active bottles (`list_active` queries the daemon).
|
||||
smolmachines's enumeration story is `smolvm machine list`; the
|
||||
plan is to mirror the label scheme via Smolfile metadata
|
||||
The microVM enumeration story is `smolvm machine list`
|
||||
(or the PyObjC backend's own bookkeeping); the plan is to
|
||||
mirror the label scheme via Smolfile metadata
|
||||
(`labels = { "claude-bottle" = "1" }`-style entries, if the
|
||||
format supports it; otherwise via a deterministic name prefix
|
||||
`claude-bottle-<slug>`).
|
||||
`claude-bottle-<slug>` + on-disk metadata under
|
||||
`state/<slug>/`).
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/smolmachines-as-vm-backend.md` — primary research
|
||||
note recommending this adoption; PRD 0023's design hypothesis.
|
||||
- `docs/research/agent-vm-isolation.md` — the broader microVM /
|
||||
gvproxy / pipelock landscape this PRD lands inside of.
|
||||
- `docs/research/agent-vm-isolation.md` — primary reference for
|
||||
the gvproxy + `VZFileHandleNetworkDeviceAttachment` network
|
||||
attachment used here. The "Full Setup: gvisor-tap-vsock +
|
||||
PyObjC + Pipelock" section is the recipe the PyObjC fallback
|
||||
in open question 1 would adopt verbatim.
|
||||
- `docs/research/smolmachines-as-vm-backend.md` — evaluation of
|
||||
smolmachines as the VM lifecycle wrapper. This PRD diverges
|
||||
from its conclusion on the *network* primitive (rejecting TSI
|
||||
in favor of gvproxy) but keeps its VM-lifecycle conclusion
|
||||
conditional on the libkrun-virtio-net spike in open question 1.
|
||||
- `docs/research/agent-sandbox-landscape.md` — identifies
|
||||
`"runtime": "microvm"`-style opt-in as the borrowable idea;
|
||||
smolmachines is the concrete implementation.
|
||||
|
||||
Reference in New Issue
Block a user