feat(macos): run containers inside a bottle via guest-local podman
Adds the `nested_containers` bottle flag. On the macOS backend it starts a rootless podman service inside the bottle and exposes its Docker-compatible API socket, so the agent still runs `docker` and `docker compose`. No host daemon socket is mounted and the guest gains no capabilities; backends that cannot run a guest-local engine reject the flag in the shared prepare template rather than ignoring it. Podman rather than rootless Docker because Apple Container's capability bounding set omits CAP_SYS_ADMIN, which the kernel requires to write a multi-range uid_map via newuidmap. With no subordinate UID range podman falls back to a single-UID self-mapping an unprivileged process may write itself, so the image build strips /etc/subuid and /etc/subgid entries rather than adding them. That mapping is also why nested containers are not an isolation layer: root inside one is the agent user outside it. They are a build/test convenience; the bottle remains the boundary. Ports the spike branch onto main, renaming docker_access — it implied Docker and granted access to nothing on the host — and drops podman from the derived layer now that every built-in image ships it (#451). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,106 @@
|
||||
# PRD prd-new: Containers inside a bottle
|
||||
|
||||
- **Status:** Draft
|
||||
- **Author:** Claude
|
||||
- **Created:** 2026-07-21
|
||||
- **Issue:** #392
|
||||
|
||||
## Summary
|
||||
|
||||
Let an agent run `docker` and `docker compose` *inside* its bottle by starting
|
||||
a guest-local rootless podman service that exposes a Docker-compatible API
|
||||
socket. Gated per bottle by `nested_containers: true`. No host daemon socket is
|
||||
mounted and no capability is added to the guest, on any backend.
|
||||
|
||||
## Problem
|
||||
|
||||
Agent tasks routinely involve containers: `docker compose up` to verify a
|
||||
scaffolded stack, a throwaway database for a test run, building an image to
|
||||
check a Dockerfile works. Bottles have no way to do any of that, so those tasks
|
||||
either fail or get done outside the bottle — which defeats the point of the
|
||||
bottle.
|
||||
|
||||
## Goals / success criteria
|
||||
|
||||
- A bottle with `nested_containers: true` on the `macos-container` backend can
|
||||
run `docker compose up -d --wait` against a stock compose file, pulling from
|
||||
a routed registry and serving from the workspace.
|
||||
- The agent's habits do not change: `docker` and `docker compose`, not
|
||||
`podman`.
|
||||
- No host Docker socket is mounted, on any backend.
|
||||
- No capability is added to the Apple Container guest.
|
||||
- Backends without a guest-local engine reject the flag with a clear error
|
||||
rather than ignoring it.
|
||||
- Bottles that do not set the flag carry none of its cost.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Nested containers as an isolation layer. The single-UID mapping means `root`
|
||||
inside a nested container is the agent user outside it. This is for build and
|
||||
test workloads; the bottle stays the security boundary.
|
||||
- The `docker` and `firecracker` backends. Both reject the flag for now.
|
||||
- Reaching registries that are not routed through the bottle's egress proxy.
|
||||
|
||||
## Design
|
||||
|
||||
### Why podman, not rootless Docker
|
||||
|
||||
Apple Container's capability bounding set omits `CAP_SYS_ADMIN`, which the
|
||||
kernel requires in order to write a multi-range `uid_map` via `newuidmap`.
|
||||
Rootless Docker has no path that avoids that write, so it cannot run in a
|
||||
bottle without granting `CAP_SYS_ADMIN` — which is close to root in practical
|
||||
terms and gives up most of what the bottle is for.
|
||||
|
||||
Podman does have a path: with **no** subordinate UID range configured it falls
|
||||
back to a single-UID self-mapping, which an unprivileged process may write
|
||||
itself. The image build therefore *removes* the agent user's `/etc/subuid` and
|
||||
`/etc/subgid` entries rather than adding them — their presence is exactly what
|
||||
would send podman down the `newuidmap` path. Full negative result in
|
||||
[`docs/research/rootless-docker-in-apple-container-spike.md`](../research/rootless-docker-in-apple-container-spike.md).
|
||||
|
||||
### The flag, and its name
|
||||
|
||||
The flag is kept because it gates real costs, not because podman needs a
|
||||
privilege grant: ~100MB of derived image, a resident service per bottle, and
|
||||
relaxed modes on `/dev/fuse` and `/dev/net/tun` that the majority of bottles
|
||||
should not get.
|
||||
|
||||
It is named `nested_containers`, not `docker_access`. It implies no Docker and
|
||||
grants access to nothing on the host — naming it after Docker access would
|
||||
describe the one thing the design refuses to do.
|
||||
|
||||
### Shape
|
||||
|
||||
- `bot_bottle/backend/macos_container/nested_containers.py` — derived-image
|
||||
build, guest env, device preparation, service start/readiness.
|
||||
- `bot_bottle/backend/macos_container/nested-containers-init.sh` — the
|
||||
unprivileged bootstrap that runs inside the bottle. Fails closed on every
|
||||
prerequisite it needs (podman, the storage/network helpers, writable device
|
||||
nodes, an *empty* subordinate range) rather than degrading.
|
||||
- The derived image `…-nested-containers` layers the Docker CLI, the compose
|
||||
plugin, `fuse-overlayfs`, `slirp4netns`, and `uidmap` onto the agent image.
|
||||
Podman itself already ships in every built-in image (#451).
|
||||
- `BottleBackend.supports_nested_containers` — false by default, so a backend
|
||||
that cannot honor the flag fails in the shared `prepare` template.
|
||||
|
||||
Guest configuration the single-UID mapping forces:
|
||||
`ignore_chown_errors` (no second UID for layers to be chowned to),
|
||||
`cgroups="disabled"` and `cgroup_manager="cgroupfs"` (no cgroup delegation
|
||||
reaches the guest), `events_logger="file"` (no journald socket).
|
||||
|
||||
### Registry reach
|
||||
|
||||
Image pulls egress through the bottle's proxy like everything else, so each
|
||||
registry needs a route. Docker Hub and GHCR need `preserve_auth: true` — their
|
||||
token dance uses a client-fetched per-scope bearer token that the proxy would
|
||||
otherwise strip. Layer pulls should also set `dlp.outbound_detectors: false`
|
||||
and `dlp.inbound_detectors: false`: body scanning on a multi-hundred-MB layer
|
||||
is what triggers #455, and a registry route's scanned bodies are compressed
|
||||
layer blobs rather than anything a detector can read.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Whether the `docker` and `firecracker` backends want an equivalent, or
|
||||
whether guest-local containers stay macOS-only.
|
||||
- Outbound networking *from* a nested container (DNS, proxy env, CA) beyond
|
||||
what the compat socket gives the bottle itself.
|
||||
Reference in New Issue
Block a user