131 lines
6.4 KiB
Markdown
131 lines
6.4 KiB
Markdown
# Supporting Apple's `container` as an alternative backend
|
||
|
||
Research into the cost and shape of adding Apple's `container` framework
|
||
(per-container Linux microVMs on Apple Silicon, announced WWDC 2025) as
|
||
an alternative backend alongside Docker. Motivated by the observation
|
||
that Apple's tool gives VM-grade isolation "for free" on macOS — no
|
||
Firecracker or Kata orchestration to deploy — and that the project's
|
||
threat model already cares about the kernel boundary.
|
||
|
||
## Summary
|
||
|
||
Realistic effort: roughly two weeks of focused work for one person. The
|
||
mechanical 80% (build / run / exec / cp) is a long-but-straightforward
|
||
weekend. The remaining 20% is networking: the pipelock egress sidecar
|
||
relies on Linux bridge-network multi-attach semantics that Apple's tool
|
||
does not model the same way, and either has to be redesigned or
|
||
simplified for the `container` path.
|
||
|
||
The honest framing: a clean port of the easy parts plus a *different*
|
||
networking story for the `container` backend (no sidecar, just VM-level
|
||
firewall rules), end-to-end in 4–5 days. A faithful port that preserves
|
||
pipelock semantics across both backends is closer to two weeks. Pick
|
||
which version you want before starting.
|
||
|
||
## Current Docker surface area
|
||
|
||
The places bot-bottle shells out to `docker` today:
|
||
|
||
- `build` — base image plus a per-cwd derived image
|
||
(`bot_bottle/docker.py:67-103`).
|
||
- `run` — with `--runtime`, `--env-file`, `-e`, `--name`, `--network`,
|
||
and volume mounts (`bot_bottle/cli/start.py:217-261`).
|
||
- `exec -it` / `exec -u 0` — for `claude` itself, file-ownership fixups,
|
||
and SSH provisioning (`bot_bottle/ssh.py`, `bot_bottle/skills.py`,
|
||
`bot_bottle/cli/start.py`).
|
||
- `cp` — skills, SSH keys, the prompt file, the workspace `.git`,
|
||
and the pipelock config
|
||
(`bot_bottle/skills.py:73`, `bot_bottle/ssh.py:106`,
|
||
`bot_bottle/cli/start.py:279`, `bot_bottle/pipelock.py:218`).
|
||
- `network create` / `connect` / `inspect` / `rm` — bottle network plus
|
||
multi-network attach for the pipelock sidecar
|
||
(`bot_bottle/network.py`, `bot_bottle/pipelock.py:227`).
|
||
- `create` / `start` / `rm -f` — pipelock sidecar lifecycle
|
||
(`bot_bottle/pipelock.py:207-258`).
|
||
- Misc preflight: `image inspect`, `ps -a -f name=^...$`, `info` for
|
||
registered runtimes (`bot_bottle/docker.py`).
|
||
|
||
## Mapping to Apple's `container`
|
||
|
||
| Capability | `container` story |
|
||
|---|---|
|
||
| build / run / exec / images | Direct equivalents, OCI-compatible |
|
||
| `cp` | `container cp` exists, but recursion semantics (trailing `./`) need verifying against the Docker behavior the codebase relies on |
|
||
| `--env-file` | Needs verification; may have to translate to repeated `-e` flags |
|
||
| `--runtime=runsc` | **Becomes a no-op.** Every container is already in its own VM, so gVisor is redundant. This is a win — `require_runsc` collapses or the manifest unifies the concept (see "Manifest" below). |
|
||
| User-defined networks | Limited — fewer knobs than Docker bridge networks |
|
||
| **Multi-attach: `network connect` to a running container** | **The hard one.** The pipelock sidecar pattern attaches to two networks. Apple's tool does not model that the same way. |
|
||
|
||
## Effort breakdown
|
||
|
||
Roughly two weeks for one person, split as:
|
||
|
||
1. **Backend abstraction (1–2 days).** `bot_bottle/docker.py` is
|
||
already a partial seam, but `bot_bottle/network.py`,
|
||
`bot_bottle/pipelock.py`, `bot_bottle/ssh.py`,
|
||
`bot_bottle/skills.py`, and `bot_bottle/cli/start.py` all call
|
||
`subprocess.run(["docker", ...])` directly. Define a `Backend`
|
||
protocol — `run`, `exec`, `cp`, `build`, `network_create`,
|
||
`network_connect`, `inspect`, `rm` — route every call through it,
|
||
keep Docker as the default impl. Mostly mechanical.
|
||
|
||
2. **`container` backend impl (2–3 days).** The easy 80%: run, exec,
|
||
build, image inspect, cp. Plus a `require_container()` analogous to
|
||
`require_docker()`. Verify `container cp` recursion and `--env-file`
|
||
support against actual binary behavior, not docs.
|
||
|
||
3. **Networking and pipelock (3–5 days, dominant risk).** The egress
|
||
sidecar design assumes Linux bridge-network semantics with multi-
|
||
network attach. On Apple's tool the likely redesign is one of:
|
||
|
||
- Run pipelock as a host-side process and have the bottle dial it
|
||
directly via the host loopback. Simpler, but loses the "egress
|
||
proxy is itself isolated" property.
|
||
- Keep pipelock in its own VM and wire the bottle's egress through
|
||
it via a different mechanism (port forwarding, shared network if
|
||
the tool grows that capability). Closer to current semantics, more
|
||
work.
|
||
|
||
Either way this is real design work, not a port. Worth a separate
|
||
PRD before code lands.
|
||
|
||
4. **Manifest spec (½ day).** Collapse `runtime: "runsc"` and "use
|
||
`container` backend" into a single `sandbox: "shared-kernel" | "vm"`
|
||
field. Backend selection follows from the value. Documenting why the
|
||
`runsc` knob disappears on the `container` path matters more than
|
||
the code change.
|
||
|
||
5. **Tests and docs (2–3 days).** The existing test suite mocks
|
||
`docker`; needs equivalents for `container`. Document which features
|
||
are macOS-only and what the `container` backend trades away
|
||
(currently: pipelock semantics, possibly some network introspection).
|
||
|
||
## The recommended split
|
||
|
||
Two distinct paths, each with a clear cost/benefit:
|
||
|
||
- **Faithful port (~2 weeks).** Both backends offer the same egress
|
||
guarantees. Worth it if pipelock is load-bearing for the threat model
|
||
and the project intends to support `container` as a first-class peer
|
||
to Docker indefinitely.
|
||
|
||
- **Simplified port (~4–5 days).** The `container` backend uses VM-level
|
||
firewall rules instead of pipelock; documentation calls out the
|
||
difference. Worth it if the VM kernel boundary is judged to make
|
||
pipelock less critical on the `container` path anyway, and the goal
|
||
is to get `container` working as an experimental backend without
|
||
blocking on a redesign.
|
||
|
||
The simplified path is probably the right starting point. The kernel
|
||
boundary that `container` provides was the original motivation for
|
||
exploring this in the first place; pipelock's value-add on top of a
|
||
real VM is smaller than it is on top of shared-kernel Docker.
|
||
|
||
## Recommendation
|
||
|
||
Don't start the implementation before deciding which split is intended,
|
||
and don't start any of it before the `Backend` abstraction lands. The
|
||
abstraction makes the language choice reversible (per the
|
||
`bash-vs-python-vs-go` note) *and* makes adding a second backend
|
||
mechanical. Skipping it means rewriting the same call sites twice.
|