feat(macos): replace rootless Docker spike with rootless podman
test / stage-firecracker-inputs (pull_request) Successful in 2s
test / integration-docker (pull_request) Successful in 32s
test / unit (pull_request) Successful in 36s
lint / lint (push) Failing after 52s
test / build-infra (pull_request) Successful in 4m5s
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-firecracker (pull_request) Successful in 2m2s
test / coverage (pull_request) Successful in 2m7s
test / publish-infra (pull_request) Has been skipped
test / stage-firecracker-inputs (pull_request) Successful in 2s
test / integration-docker (pull_request) Successful in 32s
test / unit (pull_request) Successful in 36s
lint / lint (push) Failing after 52s
test / build-infra (pull_request) Successful in 4m5s
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-firecracker (pull_request) Successful in 2m2s
test / coverage (pull_request) Successful in 2m7s
test / publish-infra (pull_request) Has been skipped
Podman avoids the CAP_SYS_ADMIN requirement that makes rootless Docker impossible here: with no subordinate UID range configured it falls back to a single-UID self-mapping, which an unprivileged process may write itself, so newuidmap is never invoked. The image build therefore strips /etc/subuid and /etc/subgid entries rather than adding them. The agent-facing surface is unchanged — docker and docker compose talk to podman's Docker-compatible API socket. Two device nodes need relaxing as root inside the bottle (0666 on /dev/fuse and /dev/net/tun); both already exist and neither needs a capability the bottle lacks, unlike CAP_SYS_ADMIN. Verified live on macOS 26 / Apple Container 1.0.0: service starts with zero added capabilities, compose pulls and serves from the workspace on a published port, and a nested container cannot reach the network outside the egress path. Known limitation, not yet addressed: the agent base image is Debian bookworm, whose podman 4.3.1 swallows container exit codes through the compat API (docker run returns 0 regardless). podman 5.4.2 on trixie fixes it; the base bump is a separate PR. The acceptance test asserts on an in-band marker rather than an exit code so it cannot false-pass in the meantime. Nested containers give no isolation from the agent itself — root inside one is the agent user outside it. They are a build/test convenience; the bottle remains the security boundary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GnMp8SUGH57hZX7192Rv2F
This commit is contained in:
@@ -0,0 +1,132 @@
|
||||
"""Experimental rootless podman bootstrap for Apple-container bottles.
|
||||
|
||||
The service and every nested container remain inside the existing per-bottle
|
||||
VM. This module refuses to compensate for missing prerequisites with outer
|
||||
capabilities, a privileged container, or a host Docker socket.
|
||||
|
||||
Podman is used rather than rootless Docker for one specific reason: Apple
|
||||
Container's capability bounding set omits `CAP_SYS_ADMIN`, which the kernel
|
||||
requires to write a multi-range `uid_map` via `newuidmap`. Rootless Docker
|
||||
has no path that avoids that write. Podman does — with no subordinate UID
|
||||
range configured it falls back to a single-UID self-mapping, which an
|
||||
unprivileged process may write itself. See
|
||||
`docs/research/rootless-docker-in-apple-container-spike.md`.
|
||||
|
||||
That fallback is why `build_image` *removes* the agent user's `/etc/subuid`
|
||||
and `/etc/subgid` entries instead of adding them: their presence is precisely
|
||||
what would send podman down the `newuidmap` path that cannot work here.
|
||||
|
||||
The agent still talks to `docker` and `docker compose`; those speak to
|
||||
podman's Docker-compatible API socket, so nothing in the agent's habits
|
||||
changes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
|
||||
from ...log import die, info
|
||||
|
||||
_INIT = "/usr/local/libexec/bot-bottle/rootless-podman-init"
|
||||
_RUNTIME_DIR = "/tmp/bot-bottle-podman-run"
|
||||
_SOCKET = f"{_RUNTIME_DIR}/podman.sock"
|
||||
_LOG = "/tmp/bot-bottle-rootless-podman.log"
|
||||
READY_RETRIES = 30
|
||||
|
||||
# Apple Container creates both device nodes 0600 root:root, so the agent user
|
||||
# cannot open them: /dev/fuse blocks the fuse-overlayfs storage driver and
|
||||
# /dev/net/tun blocks slirp4netns, which rootless podman uses for the default
|
||||
# bridge network that stock compose files expect. Relaxing the modes needs no
|
||||
# capability the bottle does not already hold — unlike CAP_SYS_ADMIN, which is
|
||||
# what killed the rootless-Docker approach.
|
||||
_GUEST_DEVICES = ("/dev/fuse", "/dev/net/tun")
|
||||
|
||||
|
||||
def build_image(
|
||||
base_image: str,
|
||||
build: Callable[..., None],
|
||||
) -> str:
|
||||
"""Layer spike-only tooling on an already-built provider image."""
|
||||
image = f"{base_image}-rootless-podman"
|
||||
init_script = Path(__file__).with_name("rootless-podman-init.sh")
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-rootless-podman.") as tmp:
|
||||
context = Path(tmp)
|
||||
shutil.copy2(init_script, context / "rootless-podman-init.sh")
|
||||
(context / "Dockerfile").write_text(
|
||||
"FROM docker:28-cli AS docker_cli\n"
|
||||
f"FROM {base_image}\n"
|
||||
"USER root\n"
|
||||
"COPY --from=docker_cli /usr/local/bin/docker /usr/local/bin/docker\n"
|
||||
"COPY --from=docker_cli /usr/local/libexec/docker/cli-plugins/"
|
||||
"docker-compose /usr/local/libexec/docker/cli-plugins/docker-compose\n"
|
||||
"RUN apt-get update \\\n"
|
||||
" && apt-get install -y --no-install-recommends podman "
|
||||
"fuse-overlayfs slirp4netns uidmap \\\n"
|
||||
" && rm -rf /var/lib/apt/lists/* \\\n"
|
||||
# Deliberate: an empty subordinate range keeps podman on the
|
||||
# single-UID mapping that needs no CAP_SYS_ADMIN. Adding ranges
|
||||
# here would reintroduce the newuidmap failure this spike exists
|
||||
# to route around.
|
||||
" && sed -i '/^node:/d' /etc/subuid /etc/subgid\n"
|
||||
"COPY rootless-podman-init.sh "
|
||||
"/usr/local/libexec/bot-bottle/rootless-podman-init\n"
|
||||
"RUN chmod 0755 /usr/local/libexec/bot-bottle/rootless-podman-init\n"
|
||||
"USER node\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
build(image, str(context), dockerfile=str(context / "Dockerfile"))
|
||||
return image
|
||||
|
||||
|
||||
def guest_env(enabled: bool) -> dict[str, str]:
|
||||
"""Environment consumed by the Docker CLI inside an enabled bottle."""
|
||||
if not enabled:
|
||||
return {}
|
||||
return {
|
||||
"DOCKER_HOST": f"unix://{_SOCKET}",
|
||||
"XDG_RUNTIME_DIR": _RUNTIME_DIR,
|
||||
}
|
||||
|
||||
|
||||
def prepare_guest_devices(container_name: str, exec_as_root: Callable[..., None]) -> None:
|
||||
"""Make /dev/fuse and /dev/net/tun openable by the agent user.
|
||||
|
||||
Runs as root inside the bottle because the agent must not be able to
|
||||
re-mode device nodes itself. No outer capability is involved.
|
||||
"""
|
||||
exec_as_root(
|
||||
container_name,
|
||||
["sh", "-c", f"chmod 0666 {' '.join(_GUEST_DEVICES)}"],
|
||||
)
|
||||
|
||||
|
||||
def start(bottle: object) -> None:
|
||||
"""Start and verify the unprivileged service through the bottle exec API."""
|
||||
info("starting experimental rootless podman service")
|
||||
result = bottle.exec(shlex.quote(_INIT)) # type: ignore[attr-defined]
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout or "").strip()
|
||||
die(f"rootless podman bootstrap failed: {detail or '<no output>'}")
|
||||
|
||||
for _ in range(READY_RETRIES):
|
||||
result = bottle.exec("docker info >/dev/null 2>&1") # type: ignore[attr-defined]
|
||||
if result.returncode == 0:
|
||||
info("rootless podman service is ready")
|
||||
return
|
||||
time.sleep(0.2)
|
||||
|
||||
logs = bottle.exec( # type: ignore[attr-defined]
|
||||
f"tail -n 80 {_LOG} 2>/dev/null || true"
|
||||
)
|
||||
die(
|
||||
"rootless podman did not become ready without additional outer "
|
||||
f"privileges:\n{(logs.stdout or logs.stderr or '<no log>').strip()}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["build_image", "guest_env", "prepare_guest_devices", "start"]
|
||||
Reference in New Issue
Block a user