- Replace test_all_install_podman with test_none_install_podman: agent
images intentionally no longer include podman; the nested-containers
derived layer is where it lives (see build_image in nested_containers.py).
- Add podman to the package assertions in test_macos_nested_containers so
its presence in the derived Dockerfile is explicitly verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Drop the module-level paragraph emphasizing the single-UID security
boundary (#issuecomment-5010): redundant given the safe environment.
- Move `podman` from each agent image (claude, codex, pi) into the
`build_image` derived layer so bottles without `nested_containers`
pay no image-size cost (#issuecomment-5013).
- Update `build_image` docstring and inline comments to reflect the move.
- Add TODO(#394) noting the planned docker-layer abstraction refactor.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The example previously left DLP enabled on the blob CDNs, which is
exactly backwards: those hosts carry the multi-hundred-MB layers whose
buffering triggers #455, while the registry hosts they hang off carry
little. It also listed one of quay's three CDNs and omitted the CDN
requirement from the prose, so a reader following it would authenticate
fine and then 403 partway through a pull.
Adds the by-design DNS behaviour and the BusyBox wget caveat, both of
which cost real time during the #392 verification: nslookup failing
inside a nested container looks like breakage but is expected, and
BusyBox wget reports "error getting response" for requests the proxy
served successfully.
Verified by feeding the snippet through parse_frontmatter +
ManifestBottle.from_dict: all nine routes parse with the intended
preserve_auth and DLP settings.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documents the proxy-layer precedence that took four attempts to get
right, the blob-CDN routes every registry needs alongside its own, the
podman 5 networking packages and how each one fails when absent, and
the BusyBox wget caveat. Closes the DNS open question: public
resolution inside a nested container fails by design; reaching the
proxy was the real problem.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This is the layer that actually decides it, and it was in the script
from the start: ~/.docker/config.json's proxies block is what the Docker
CLI copies into every container it starts. Being client-side it beats
the podman service outright, which is why containers.conf `env`,
http_proxy=false, and the service environment each looked correct on
disk and changed nothing a nested container saw.
Also corrects the comment on the service-env substitution, which
claimed to be the one place the address sticks. It is not; it covers
podman's own pulls and non-CLI API clients.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third attempt at the same symptom, and the first at the right layer.
podman's Docker-compatible API injects proxy settings from the *daemon*
environment after containers.conf is applied, so neither the env list
(892bfc16) nor http_proxy=false (21c144ae) could override it — the same
compat-path blind spot already seen with hosts_file. Nested containers
kept reporting `wget: bad address 'bot-bottle-gateway:9099'` through
both.
Substituting the resolved address into the service's own environment,
before `podman system service` starts, is the one place it sticks.
NO_PROXY keeps the name, which is matched against what a client asks
for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The address-bearing proxy URLs from 892bfc16 were written correctly but
never took effect: podman copies the host's proxy environment into every
container by default, and that copy overrides containers.conf `env`,
restoring the `bot-bottle-gateway` name a nested container cannot
resolve. Verified on the macOS host — containers still reported
`wget: bad address 'bot-bottle-gateway:9099'` with the addresses in
place.
Setting http_proxy=false disables only podman's own passthrough; the
proxy vars still reach containers, by address, from the env list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman's containers.conf `hosts_file` turns out to apply only to native
`podman run`; the Docker-compatible API ignores it, and the agent types
`docker`. (`base_hosts_file`, used before this, is a no-op in both
paths on podman 5.4.2.) So no config key can put the gateway name in a
nested container's /etc/hosts.
The name is therefore resolved in the bottle, where /etc/hosts does
carry it, and the address goes into the proxy URL handed to nested
containers. NO_PROXY keeps the name, since that is matched against what
a client asks for.
Verified on the macOS host: with the gateway reachable, a nested
container gets 200 from an allowlisted host and 403 from one that is
not — the egress boundary applies inside nested containers too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The flag from #453 was parsed off the manifest and honored by the
addon, but `_route_to_yaml_fields` never wrote it, so it was dropped on
the way to the proxy and has never worked end to end. Every registry
route silently kept the unconditional Authorization strip.
That is the whole of the Docker Hub / GHCR "unauthorized" blocker in
#392: podman fetched a valid 2658-byte bearer from auth.docker.io, the
proxy stripped it, and registry-1.docker.io answered the resulting
anonymous request with a fresh WWW-Authenticate challenge — which reads
exactly like a credential problem rather than a serializer gap.
Found by diffing a bottle's rendered routes.yaml against its manifest:
preserve_auth was in the manifest and absent from the output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Answers the last open question in #392: DNS from inside a nested
container was not a phantom, but it was also not a DNS problem. Public
resolution fails there by design — everything egresses through the
proxy — and the actual breakage was two missing pieces:
- The proxy URL names `bot-bottle-gateway`, which resolves only through
the bottle's own /etc/hosts. Containers got their own hosts file, so
they failed at "Could not resolve proxy", which reads like broken DNS.
base_hosts_file propagates the bottle's entries.
- The gateway TLS-intercepts, so a container that does not trust the
bottle's CA bundle fails with "unable to get local issuer
certificate". The bundle is now mounted read-only and the usual env
vars point at it, which covers curl, wget, python, and node without
distro-specific trust commands.
Verified on the macOS host by reproducing each failure and confirming
that these three conditions applied by hand produced HTTP 200 from
inside a nested container. podman already forwards the proxy env, so
that needed nothing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
netavark shells out to `nft` for the bridge network every compose file
expects, and aardvark-dns serves name resolution inside nested
containers. Neither arrives with the base image's
`--no-install-recommends` podman, and each fails at a different and
misleading layer: without nft containers are created but never start;
without aardvark-dns DNS inside a container fails while the engine, the
pulls, and the bridge all look healthy. That last one is the likeliest
explanation for the "DNS inside nested containers fails entirely" note
in #392 — a missing package, not a design gap.
Verified on the macOS host: with passt alone, `docker run` reached
netavark and died on "unable to execute nft".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman 5 uses pasta as the default rootless network helper; podman 4
used slirp4netns. The spike was written against bookworm's podman 4.3.1,
so its package list never included passt — and the trixie bump (#451)
changed the default out from under it. Every nested container failed to
start with "could not find pasta", while image pulls and the service
itself worked, which disguised a missing dependency as a compat-API bug.
The bootstrap now checks for pasta up front so this fails with a clear
message instead of surfacing at the first `docker run`.
Note the sun_path fix in a12d2191 did NOT cause or cure this; that path
was over the 108-byte limit independently and would have bitten as soon
as attach was reached.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman derives conmon's attach socket as
`$XDG_RUNTIME_DIR/libpod/tmp/socket/<64-hex-id>/attach`. With the
descriptive `/tmp/bot-bottle-podman-run` that path is 116 bytes, past
the 108-byte sun_path limit, so every attached `docker run` failed with
"unable to upgrade to tcp, received 500" — while image pulls and the
service itself worked, which is what made it look like a compat-API bug
rather than a path-length one.
Found on the macOS host: quay.io pulled fine and then no container
could be started. Test pins the budget so a more readable name cannot
silently reintroduce it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
merge_bottles_runtime sees resolved bottles, where a bottle that never
mentioned the key is indistinguishable from one that set it false. With
"later replaces" semantics, `--bottle with-containers --bottle claude-dev`
silently dropped the capability — the second bottle's default clobbered
it. OR instead: composing bottles adds capabilities, it does not remove
them. The file-based extends path still sees raw keys, so an explicit
`nested_containers: false` in a child continues to turn it back off.
Also surface the flag in the launch summary, where its absence was the
only reason the clobber went unnoticed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the `nested_containers` bottle flag. On the macOS backend it starts
a rootless podman service inside the bottle and exposes its
Docker-compatible API socket, so the agent still runs `docker` and
`docker compose`. No host daemon socket is mounted and the guest gains
no capabilities; backends that cannot run a guest-local engine reject
the flag in the shared prepare template rather than ignoring it.
Podman rather than rootless Docker because Apple Container's capability
bounding set omits CAP_SYS_ADMIN, which the kernel requires to write a
multi-range uid_map via newuidmap. With no subordinate UID range podman
falls back to a single-UID self-mapping an unprivileged process may
write itself, so the image build strips /etc/subuid and /etc/subgid
entries rather than adding them.
That mapping is also why nested containers are not an isolation layer:
root inside one is the agent user outside it. They are a build/test
convenience; the bottle remains the boundary.
Ports the spike branch onto main, renaming docker_access — it implied
Docker and granted access to nothing on the host — and drops podman
from the derived layer now that every built-in image ships it (#451).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Investigates storage candidates, retention policy design, and
image-size reduction techniques in response to the PR #459 comment
on a secure remote store for committed Firecracker snapshots.
`cleanup()` enumerated *every* firecracker process under the run root and
*every* run dir, so `./cli.py cleanup` would kill running bottles and
delete their rootfs — despite being described as orphan cleanup. The
backend's `enumerate_active` registry is still a stub (#354), so there
was no live/dead signal.
Use the running firecracker processes themselves as that signal: a run
dir with a live VM is protected (never killed, never removed); only run
dirs with no live process (leaked by a hard-killed launch) are reaped,
plus firecracker pids whose run dir is already gone (lingering VMMs).
This makes cleanup safe to run alongside live bottles and lets it back a
periodic GC for the run-dir leak the launch.py teardown fix closes on the
clean-exit path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each Firecracker bottle created its run dir (holding an ~1G
rootfs.ext4) but never removed it — launch teardown only terminated
the VM. Every run leaked its rootfs, filling the CI runner's disk
(187 stale sandbox-tester dirs / 141G in one incident).
Register a run-dir rmtree callback on the ExitStack before the
vm.terminate callback so, LIFO, the VM is gone before its rootfs is
removed. ignore_errors keeps teardown resilient.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a per-route boolean field preserve_auth (default false) that skips
the gateway's Authorization header stripping for that host. Intended for
registry endpoints like Docker Hub (registry-1.docker.io) and GHCR
(ghcr.io) where the agent must supply its own per-scope bearer token.
Threaded through ManifestEgressRoute → EgressRoute → Route, serialized
in route_to_yaml_dict, and parsed in parse_routes. The strip at
egress_addon.py now checks route.preserve_auth before popping the header.
Closes#392
The shared gateway self-generates a mitmproxy CA that every bottle installs
to trust its TLS interception. It was persisted on a Docker named volume,
which survives `docker rm` but is silently wiped by `docker volume prune` /
`docker system prune --volumes` during routine host maintenance. When that
happens the gateway mints a fresh CA on restart, and every already-running
bottle fails the TLS handshake even after it re-resolves and reconnects to
the moved gateway — a re-attachment blocker distinct from #443/#445.
Move CA persistence to a host bind-mount under the app-data root
(`bot_bottle_root()/gateway-ca`, via `host_gateway_ca_dir()`), mirroring how
the shared DB and control-plane token already live on the host. Docker never
prunes a path under the root, and it stays inspectable + rotatable from the
host. mitmproxy already adopts an existing CA and generates one only on first
run, so the bind-mount gives adopt-existing/generate-on-first-run for free.
Add an explicit rollover path: `rotate_gateway_ca()` clears the persisted CA
so the next start remints it, and `python -m bot_bottle.orchestrator.rotate_ca`
wires that together with dropping the running gateway container (whose
mitmproxy still holds the old CA in memory). Rotation stays an operator action
— it doesn't auto-re-provision running bottles, which re-attach to pick up the
new anchor.
Scope: the Docker infra/gateway path (the "infra container" in the report).
The macOS (`container`-only volume) and Firecracker (VM-attached ext4) backends
persist the CA differently and are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Approval now renders over the hosts page at /hosts/authorize?code=…
rather than on a standalone page. The console keeps /authorize as a
redirect, so this is cosmetic for older consoles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`python -m bot_bottle.cli` failed with "is a package and cannot be
directly executed" — the package had a `if __name__ == "__main__"` block
in `__init__.py`, which never fires for a package and made the invocation
look supported when it wasn't. Add a real `__main__.py` and drop the dead
block.
Matters for `bb login`, whose docstring documents a `bb` entry point that
nothing installs; `-m` is the closest thing to it until there's a
console-script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces broad `except Exception` with specific types that reflect the
actual failure modes:
- _save_credentials: `except OSError` (IO-only path; re-raises for cleanup)
- _post error handler: `except (OSError, ValueError)` (network + bad JSON)
- poll-loop: `except (OSError, ValueError)` (network + bad JSON)
Also:
- Simplify _fake_get to use next(..., default) — removes uncovered
StopIteration branch
- Add encoding="utf-8" to open() in test_approved_flow_returns_0
- Add test_cleanup_on_write_failure to cover the except OSError block
in _save_credentials
All files score 10.00/10 on pylint (fail-under=10) and 0 errors on
pyright strict.
Write credentials via a 0600 temp file + os.replace() so the token file
never appears at its final path with world-readable permissions, even if
the process is interrupted between write and chmod.
Parse poll_interval from the authorization response (clamped to 1–60 s,
falling back to _POLL_SLEEP) so aggressive polling can't trigger console
rate limits.
Tests: add atomicity spy asserting the temp file is 0600 before replace;
patch time.sleep instead of _POLL_SLEEP; add explicit interval-passthrough
assertion.
Starts a device-authorization flow against a bot-bottle console, polls
until the operator approves, then writes access + refresh tokens to
$BOT_BOTTLE_ROOT/console.json. Console URL is read from --console-url
flag or BB_CONSOLE_URL env var.
Part of didericis/bot-bottle-platform#1
Two issues introduced by the rebase conflict resolution:
- test_cached_lookup_requires_ready_marker used _dockerfile_hash as the
cache-dir key, but cached_agent_rootfs_dir now uses _rootfs_digest (which
also folds in the guest init). Updated the test to match.
- test_pyproject_toml_change_bumps_version appeared twice in
test_infra_artifact.py (once from main, once from the PR commit that
added pyproject.toml support). Removed the duplicate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dockerfile.gateway COPYs pyproject.toml into /src and runs pip install
/src, so it is a real input to the baked rootfs. A dependency-only change
previously reused stale artifact versions, potentially booting a rootfs
whose installed packages differed from the current checkout.
Also adds _fake_repo fixture support and a regression test so this input
can't silently drop out of the hash again.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FROM-scratch images (commit_container output) and registry images built
for reproducibility often omit the created field entirely. Both backends
were calling die() in that case, crashing the cached-image quickstart
before any container started.
image_created_at now returns datetime | None — None when the field is
absent or unparseable, die() only on real inspect failures (non-zero
exit, malformed JSON). stale_checks in both backends skips the staleness
check when None is returned.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- config_store.py imported host_db_path from supervise_types (wrong);
it lives in paths.py, matching all other stores (audit, queue, etc.)
- test_stale_checks: two tests expected check_stale called twice
(agent + sidecar) — consolidated arch has no sidecar, so once is
correct; update assertions and remove unused Path import
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pyright strict reportUnusedImport flagged BottleImages in
test_docker_launch_committed_image.py and test_macos_container_launch.py;
neither file references the type by name (they only use the returned
value's attributes).
`from __future__ import annotations` already defers all annotation
evaluation, so quoting `str | Path`, `BottleImages` inside the same
module was redundant and tripped pyright strict mode.
didericis-claude
didericis
didericis-codex
Quality Badge Bot