bot-bottle/docs/research/apple-container-backend.md

# Supporting Apple's `container` as an alternative backend

Research into the cost and shape of adding Apple's `container` framework
(per-container Linux microVMs on Apple Silicon, announced WWDC 2025) as
an alternative backend alongside Docker. Motivated by the observation
that Apple's tool gives VM-grade isolation "for free" on macOS — no
Firecracker or Kata orchestration to deploy — and that the project's
threat model already cares about the kernel boundary.

## Summary

Realistic effort: roughly two weeks of focused work for one person. The
mechanical 80% (build / run / exec / cp) is a long-but-straightforward
weekend. The remaining 20% is networking: the pipelock egress sidecar
relies on Linux bridge-network multi-attach semantics that Apple's tool
does not model the same way, and either has to be redesigned or
simplified for the `container` path.

The honest framing: a clean port of the easy parts plus a *different*
networking story for the `container` backend (no sidecar, just VM-level
firewall rules), end-to-end in 4–5 days. A faithful port that preserves
pipelock semantics across both backends is closer to two weeks. Pick
which version you want before starting.

## Current Docker surface area

The places bot-bottle shells out to `docker` today:

- `build` — base image plus a per-cwd derived image
  (`bot_bottle/docker.py:67-103`).
- `run` — with `--runtime`, `--env-file`, `-e`, `--name`, `--network`,
  and volume mounts (`bot_bottle/cli/start.py:217-261`).
- `exec -it` / `exec -u 0` — for `claude` itself, file-ownership fixups,
  and SSH provisioning (`bot_bottle/ssh.py`, `bot_bottle/skills.py`,
  `bot_bottle/cli/start.py`).
- `cp` — skills, SSH keys, the prompt file, the workspace `.git`,
  and the pipelock config
  (`bot_bottle/skills.py:73`, `bot_bottle/ssh.py:106`,
  `bot_bottle/cli/start.py:279`, `bot_bottle/pipelock.py:218`).
- `network create` / `connect` / `inspect` / `rm` — bottle network plus
  multi-network attach for the pipelock sidecar
  (`bot_bottle/network.py`, `bot_bottle/pipelock.py:227`).
- `create` / `start` / `rm -f` — pipelock sidecar lifecycle
  (`bot_bottle/pipelock.py:207-258`).
- Misc preflight: `image inspect`, `ps -a -f name=^...$`, `info` for
  registered runtimes (`bot_bottle/docker.py`).

## Mapping to Apple's `container`

| Capability | `container` story |
|---|---|
| build / run / exec / images | Direct equivalents, OCI-compatible |
| `cp` | `container cp` exists, but recursion semantics (trailing `./`) need verifying against the Docker behavior the codebase relies on |
| `--env-file` | Needs verification; may have to translate to repeated `-e` flags |
| `--runtime=runsc` | **Becomes a no-op.** Every container is already in its own VM, so gVisor is redundant. This is a win — `require_runsc` collapses or the manifest unifies the concept (see "Manifest" below). |
| User-defined networks | Limited — fewer knobs than Docker bridge networks |
| **Multi-attach: `network connect` to a running container** | **The hard one.** The pipelock sidecar pattern attaches to two networks. Apple's tool does not model that the same way. |

## Effort breakdown

Roughly two weeks for one person, split as:

1. **Backend abstraction (1–2 days).** `bot_bottle/docker.py` is
   already a partial seam, but `bot_bottle/network.py`,
   `bot_bottle/pipelock.py`, `bot_bottle/ssh.py`,
   `bot_bottle/skills.py`, and `bot_bottle/cli/start.py` all call
   `subprocess.run(["docker", ...])` directly. Define a `Backend`
   protocol — `run`, `exec`, `cp`, `build`, `network_create`,
   `network_connect`, `inspect`, `rm` — route every call through it,
   keep Docker as the default impl. Mostly mechanical.

2. **`container` backend impl (2–3 days).** The easy 80%: run, exec,
   build, image inspect, cp. Plus a `require_container()` analogous to
   `require_docker()`. Verify `container cp` recursion and `--env-file`
   support against actual binary behavior, not docs.

3. **Networking and pipelock (3–5 days, dominant risk).** The egress
   sidecar design assumes Linux bridge-network semantics with multi-
   network attach. On Apple's tool the likely redesign is one of:

   - Run pipelock as a host-side process and have the bottle dial it
     directly via the host loopback. Simpler, but loses the "egress
     proxy is itself isolated" property.
   - Keep pipelock in its own VM and wire the bottle's egress through
     it via a different mechanism (port forwarding, shared network if
     the tool grows that capability). Closer to current semantics, more
     work.

   Either way this is real design work, not a port. Worth a separate
   PRD before code lands.

4. **Manifest spec (½ day).** Collapse `runtime: "runsc"` and "use
   `container` backend" into a single `sandbox: "shared-kernel" | "vm"`
   field. Backend selection follows from the value. Documenting why the
   `runsc` knob disappears on the `container` path matters more than
   the code change.

5. **Tests and docs (2–3 days).** The existing test suite mocks
   `docker`; needs equivalents for `container`. Document which features
   are macOS-only and what the `container` backend trades away
   (currently: pipelock semantics, possibly some network introspection).

## The recommended split

Two distinct paths, each with a clear cost/benefit:

- **Faithful port (~2 weeks).** Both backends offer the same egress
  guarantees. Worth it if pipelock is load-bearing for the threat model
  and the project intends to support `container` as a first-class peer
  to Docker indefinitely.

- **Simplified port (~4–5 days).** The `container` backend uses VM-level
  firewall rules instead of pipelock; documentation calls out the
  difference. Worth it if the VM kernel boundary is judged to make
  pipelock less critical on the `container` path anyway, and the goal
  is to get `container` working as an experimental backend without
  blocking on a redesign.

The simplified path is probably the right starting point. The kernel
boundary that `container` provides was the original motivation for
exploring this in the first place; pipelock's value-add on top of a
real VM is smaller than it is on top of shared-kernel Docker.

## Recommendation

Don't start the implementation before deciding which split is intended,
and don't start any of it before the `Backend` abstraction lands. The
abstraction makes the language choice reversible (per the
`bash-vs-python-vs-go` note) *and* makes adding a second backend
mechanical. Skipping it means rewriting the same call sites twice.