Add unit tests for the three new public symbols introduced by this PR:
- TestIsBackendAvailable — delegates to has_backend; returns True/False
correctly for available and unavailable backends
- TestIsBackendReady — unknown names return False; known names return
True/False based on status() return code; quiet flag is forwarded
- TestBackendStatusQuiet — verifies the quiet=True branch in
DockerBottleBackend, FirecrackerBottleBackend, and
MacosContainerBottleBackend: return code is propagated and
diagnostic stderr is suppressed
Together these bring diff-coverage for the PR's changed lines to 100%.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
tests/_backend.py imported is_backend_available but never called it,
causing a pyright reportUnusedImport error. Remove the unused import.
test_backend_skip_guards.py was patching tests._backend.has_backend but
_backend.py calls is_backend_ready — the mock never intercepted the real
call, producing AttributeError at test runtime. Update all patch targets
to is_backend_ready and add quiet=False to the assert_called_once_with
assertion to match the actual call signature.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a module-level BackendStatus(IntEnum) with READY=0 to replace magic
0 comparisons. Extend BottleBackend.status() with a quiet parameter:
quiet=False (default) prints diagnostics; quiet=True returns the code
silently for programmatic checks.
Add is_backend_available() (cheap PATH check, alias for has_backend) and
is_backend_ready(name, *, quiet=False) (full status() check) to the
backend package for callers that need to distinguish availability from
readiness.
Update tests/_backend.py guards to use is_backend_ready(quiet=False) so
diagnostic output is printed during test discovery, giving the operator a
concrete reason for each skip rather than a bare "prerequisites
unavailable" message.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Review feedback: the hand-rolled `/dev/kvm` + docker-daemon capability
probes duplicated logic the CLI already owns. Delegate instead.
- tests/_backend.py skip guards now gate on `bot_bottle.backend.has_backend`
(each backend's `is_available()` classmethod — the same probe behind
`./cli.py backend status`), dropping the bespoke `Capability` probes.
- Remove tests/backend_preflight.py; the docker integration job runs
`./cli.py backend status --backend=docker` as its preflight (clear
per-check summary, non-zero exit when unready), matching the firecracker
job. The firecracker preflight reverts to its original binary/KVM checks
(backend status doesn't cover those).
- Unit test + docs updated to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Integration tests now select their backend from BOT_BOTTLE_BACKEND and
skip on the capability that backend actually needs, instead of gating
every backend on unrelated Docker availability.
Task 1 — backend-agnostic guards (tests/_backend.py):
- Capability probes: docker_capability() (reachable daemon) and
firecracker_capability() (accessible /dev/kvm + firecracker on PATH,
Docker-independent). backend_capability()/selected_backend() resolve
the target from BOT_BOTTLE_BACKEND (default docker).
- skip_unless_selected_backend_available() for backend-agnostic tests
(test_sandbox_escape) — runs through whichever backend is selected and
checks that backend's real capability.
- skip_unless_backend("docker") for Docker-implementation tests
(DockerBroker, DockerGateway, backend.docker.*) — they no-op under a
non-Docker run rather than testing internals that run doesn't target.
- Retires tests/_docker.py; the KVM job no longer needs SKIP_DOCKER_TESTS
to steer Docker-only classes.
Task 2 — explicit per-backend skip visibility:
- tests/backend_preflight.py prints a clear PASS/FAIL capability line and
exits non-zero when the selected backend is missing.
- Both integration jobs run it as a preflight, so absent infrastructure
is surfaced at the job level instead of hidden among unittest.skip
lines. The docker job replaces its soft "Show environment" step; the
firecracker job keeps its richer backend-status check.
Docs (tests/README.md, docs/ci.md) updated; unit coverage for the probes,
guards, and preflight in test_backend_skip_guards.py.
Closes#414
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two complementary fixes for the egress gateway OOM:
1. gateway_init: auto-restart any daemon that dies unexpectedly. The
supervisor already had restart_daemon()/request_restart() logic; this
wires it into tick() so an OOM-killed mitmdump is respawned without
operator intervention. Implements the "eventual" failure policy
described in the original module docstring.
2. egress_addon: cap response body bytes passed to the DLP inbound scan
at EGRESS_INBOUND_SCAN_LIMIT_BYTES (default 1 MiB). mitmproxy buffers
the full response before the hook fires; capping at scan time limits
the additional amplification from decoded text and regex strings. Bodies
over the limit emit an egress_scan_truncated log event. Set to 0 to
disable the cap.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The PTY check added in e719022 calls sys.stdin.fileno() which raises
io.UnsupportedOperation under pytest's stdin capture. Guard the fileno()
call with a try/except so the check degrades cleanly to "not a tty"
instead of crashing, then stub os.isatty in the test setUp so headless
unit tests aren't blocked on a real TTY.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
merge_bottles_runtime used .get(n, base_repos_by_name[n]) which eagerly
evaluates the default, crashing when a repo exists only in the override
bottle. Replaced with a conditional expression so the base lookup only
runs when needed.
Adds three regression tests covering override-only, base-only, and
name-collision cases.
Closes#457
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add unit coverage for the encrypted at-rest egress secrets: the
secret_store round-trip (new_env_var_secret/encrypt_value/decrypt_value)
and the registry/service wiring that injects ENV_VAR_SECRET.
Recovered from the bot-bottle-claude-agent-1 VM after the CI runner's
disk filled and forced its rootfs read-only; committed work was already
on origin, these were the agent's uncommitted changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Routes with `dlp: false` skip all DLP scanning (including CRLF injection,
which cannot be disabled via `dlp.outbound_detectors: false`) and tunnel
HTTPS connections without TLS interception so the client sees the server's
real certificate. Fixes Docker image pulls, which fail when the proxy MITM's
the TLS handshake and the container doesn't trust the per-bottle CA.
Closes#462
The runtime merge (merge_bottles_runtime / _fold_two_bottles) was using
OR semantics for nested_containers, which meant `nested_containers: false`
in a later bottle could never override an earlier `true`.
Fix: add `nested_containers_explicit: bool` to ManifestBottle, set to
True only when the key is present in the source dict. The runtime merge
functions now use the override's value only when it was explicitly
supplied; otherwise the base's value is preserved. This matches the
documented last-wins / presence-aware semantics used by supervise and
agent_provider, and aligns with _merge_bottles (the file-based extends
path) which already did presence checks via `"nested_containers" in
child_raw`.
Tests: rename existing test to clarify the preserved-when-omitted case,
add explicit-false-overrides-true and explicit-true-overrides-false cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace test_all_install_podman with test_none_install_podman: agent
images intentionally no longer include podman; the nested-containers
derived layer is where it lives (see build_image in nested_containers.py).
- Add podman to the package assertions in test_macos_nested_containers so
its presence in the derived Dockerfile is explicitly verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This is the layer that actually decides it, and it was in the script
from the start: ~/.docker/config.json's proxies block is what the Docker
CLI copies into every container it starts. Being client-side it beats
the podman service outright, which is why containers.conf `env`,
http_proxy=false, and the service environment each looked correct on
disk and changed nothing a nested container saw.
Also corrects the comment on the service-env substitution, which
claimed to be the one place the address sticks. It is not; it covers
podman's own pulls and non-CLI API clients.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third attempt at the same symptom, and the first at the right layer.
podman's Docker-compatible API injects proxy settings from the *daemon*
environment after containers.conf is applied, so neither the env list
(892bfc16) nor http_proxy=false (21c144ae) could override it — the same
compat-path blind spot already seen with hosts_file. Nested containers
kept reporting `wget: bad address 'bot-bottle-gateway:9099'` through
both.
Substituting the resolved address into the service's own environment,
before `podman system service` starts, is the one place it sticks.
NO_PROXY keeps the name, which is matched against what a client asks
for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The address-bearing proxy URLs from 892bfc16 were written correctly but
never took effect: podman copies the host's proxy environment into every
container by default, and that copy overrides containers.conf `env`,
restoring the `bot-bottle-gateway` name a nested container cannot
resolve. Verified on the macOS host — containers still reported
`wget: bad address 'bot-bottle-gateway:9099'` with the addresses in
place.
Setting http_proxy=false disables only podman's own passthrough; the
proxy vars still reach containers, by address, from the env list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman's containers.conf `hosts_file` turns out to apply only to native
`podman run`; the Docker-compatible API ignores it, and the agent types
`docker`. (`base_hosts_file`, used before this, is a no-op in both
paths on podman 5.4.2.) So no config key can put the gateway name in a
nested container's /etc/hosts.
The name is therefore resolved in the bottle, where /etc/hosts does
carry it, and the address goes into the proxy URL handed to nested
containers. NO_PROXY keeps the name, since that is matched against what
a client asks for.
Verified on the macOS host: with the gateway reachable, a nested
container gets 200 from an allowlisted host and 403 from one that is
not — the egress boundary applies inside nested containers too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The flag from #453 was parsed off the manifest and honored by the
addon, but `_route_to_yaml_fields` never wrote it, so it was dropped on
the way to the proxy and has never worked end to end. Every registry
route silently kept the unconditional Authorization strip.
That is the whole of the Docker Hub / GHCR "unauthorized" blocker in
#392: podman fetched a valid 2658-byte bearer from auth.docker.io, the
proxy stripped it, and registry-1.docker.io answered the resulting
anonymous request with a fresh WWW-Authenticate challenge — which reads
exactly like a credential problem rather than a serializer gap.
Found by diffing a bottle's rendered routes.yaml against its manifest:
preserve_auth was in the manifest and absent from the output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Answers the last open question in #392: DNS from inside a nested
container was not a phantom, but it was also not a DNS problem. Public
resolution fails there by design — everything egresses through the
proxy — and the actual breakage was two missing pieces:
- The proxy URL names `bot-bottle-gateway`, which resolves only through
the bottle's own /etc/hosts. Containers got their own hosts file, so
they failed at "Could not resolve proxy", which reads like broken DNS.
base_hosts_file propagates the bottle's entries.
- The gateway TLS-intercepts, so a container that does not trust the
bottle's CA bundle fails with "unable to get local issuer
certificate". The bundle is now mounted read-only and the usual env
vars point at it, which covers curl, wget, python, and node without
distro-specific trust commands.
Verified on the macOS host by reproducing each failure and confirming
that these three conditions applied by hand produced HTTP 200 from
inside a nested container. podman already forwards the proxy env, so
that needed nothing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
netavark shells out to `nft` for the bridge network every compose file
expects, and aardvark-dns serves name resolution inside nested
containers. Neither arrives with the base image's
`--no-install-recommends` podman, and each fails at a different and
misleading layer: without nft containers are created but never start;
without aardvark-dns DNS inside a container fails while the engine, the
pulls, and the bridge all look healthy. That last one is the likeliest
explanation for the "DNS inside nested containers fails entirely" note
in #392 — a missing package, not a design gap.
Verified on the macOS host: with passt alone, `docker run` reached
netavark and died on "unable to execute nft".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman 5 uses pasta as the default rootless network helper; podman 4
used slirp4netns. The spike was written against bookworm's podman 4.3.1,
so its package list never included passt — and the trixie bump (#451)
changed the default out from under it. Every nested container failed to
start with "could not find pasta", while image pulls and the service
itself worked, which disguised a missing dependency as a compat-API bug.
The bootstrap now checks for pasta up front so this fails with a clear
message instead of surfacing at the first `docker run`.
Note the sun_path fix in a12d2191 did NOT cause or cure this; that path
was over the 108-byte limit independently and would have bitten as soon
as attach was reached.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman derives conmon's attach socket as
`$XDG_RUNTIME_DIR/libpod/tmp/socket/<64-hex-id>/attach`. With the
descriptive `/tmp/bot-bottle-podman-run` that path is 116 bytes, past
the 108-byte sun_path limit, so every attached `docker run` failed with
"unable to upgrade to tcp, received 500" — while image pulls and the
service itself worked, which is what made it look like a compat-API bug
rather than a path-length one.
Found on the macOS host: quay.io pulled fine and then no container
could be started. Test pins the budget so a more readable name cannot
silently reintroduce it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
merge_bottles_runtime sees resolved bottles, where a bottle that never
mentioned the key is indistinguishable from one that set it false. With
"later replaces" semantics, `--bottle with-containers --bottle claude-dev`
silently dropped the capability — the second bottle's default clobbered
it. OR instead: composing bottles adds capabilities, it does not remove
them. The file-based extends path still sees raw keys, so an explicit
`nested_containers: false` in a child continues to turn it back off.
Also surface the flag in the launch summary, where its absence was the
only reason the clobber went unnoticed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the `nested_containers` bottle flag. On the macOS backend it starts
a rootless podman service inside the bottle and exposes its
Docker-compatible API socket, so the agent still runs `docker` and
`docker compose`. No host daemon socket is mounted and the guest gains
no capabilities; backends that cannot run a guest-local engine reject
the flag in the shared prepare template rather than ignoring it.
Podman rather than rootless Docker because Apple Container's capability
bounding set omits CAP_SYS_ADMIN, which the kernel requires to write a
multi-range uid_map via newuidmap. With no subordinate UID range podman
falls back to a single-UID self-mapping an unprivileged process may
write itself, so the image build strips /etc/subuid and /etc/subgid
entries rather than adding them.
That mapping is also why nested containers are not an isolation layer:
root inside one is the agent user outside it. They are a build/test
convenience; the bottle remains the boundary.
Ports the spike branch onto main, renaming docker_access — it implied
Docker and granted access to nothing on the host — and drops podman
from the derived layer now that every built-in image ships it (#451).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`cleanup()` enumerated *every* firecracker process under the run root and
*every* run dir, so `./cli.py cleanup` would kill running bottles and
delete their rootfs — despite being described as orphan cleanup. The
backend's `enumerate_active` registry is still a stub (#354), so there
was no live/dead signal.
Use the running firecracker processes themselves as that signal: a run
dir with a live VM is protected (never killed, never removed); only run
dirs with no live process (leaked by a hard-killed launch) are reaped,
plus firecracker pids whose run dir is already gone (lingering VMMs).
This makes cleanup safe to run alongside live bottles and lets it back a
periodic GC for the run-dir leak the launch.py teardown fix closes on the
clean-exit path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a per-route boolean field preserve_auth (default false) that skips
the gateway's Authorization header stripping for that host. Intended for
registry endpoints like Docker Hub (registry-1.docker.io) and GHCR
(ghcr.io) where the agent must supply its own per-scope bearer token.
Threaded through ManifestEgressRoute → EgressRoute → Route, serialized
in route_to_yaml_dict, and parsed in parse_routes. The strip at
egress_addon.py now checks route.preserve_auth before popping the header.
Closes#392
The shared gateway self-generates a mitmproxy CA that every bottle installs
to trust its TLS interception. It was persisted on a Docker named volume,
which survives `docker rm` but is silently wiped by `docker volume prune` /
`docker system prune --volumes` during routine host maintenance. When that
happens the gateway mints a fresh CA on restart, and every already-running
bottle fails the TLS handshake even after it re-resolves and reconnects to
the moved gateway — a re-attachment blocker distinct from #443/#445.
Move CA persistence to a host bind-mount under the app-data root
(`bot_bottle_root()/gateway-ca`, via `host_gateway_ca_dir()`), mirroring how
the shared DB and control-plane token already live on the host. Docker never
prunes a path under the root, and it stays inspectable + rotatable from the
host. mitmproxy already adopts an existing CA and generates one only on first
run, so the bind-mount gives adopt-existing/generate-on-first-run for free.
Add an explicit rollover path: `rotate_gateway_ca()` clears the persisted CA
so the next start remints it, and `python -m bot_bottle.orchestrator.rotate_ca`
wires that together with dropping the running gateway container (whose
mitmproxy still holds the old CA in memory). Rotation stays an operator action
— it doesn't auto-re-provision running bottles, which re-attach to pick up the
new anchor.
Scope: the Docker infra/gateway path (the "infra container" in the report).
The macOS (`container`-only volume) and Firecracker (VM-attached ext4) backends
persist the CA differently and are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`python -m bot_bottle.cli` failed with "is a package and cannot be
directly executed" — the package had a `if __name__ == "__main__"` block
in `__init__.py`, which never fires for a package and made the invocation
look supported when it wasn't. Add a real `__main__.py` and drop the dead
block.
Matters for `bb login`, whose docstring documents a `bb` entry point that
nothing installs; `-m` is the closest thing to it until there's a
console-script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces broad `except Exception` with specific types that reflect the
actual failure modes:
- _save_credentials: `except OSError` (IO-only path; re-raises for cleanup)
- _post error handler: `except (OSError, ValueError)` (network + bad JSON)
- poll-loop: `except (OSError, ValueError)` (network + bad JSON)
Also:
- Simplify _fake_get to use next(..., default) — removes uncovered
StopIteration branch
- Add encoding="utf-8" to open() in test_approved_flow_returns_0
- Add test_cleanup_on_write_failure to cover the except OSError block
in _save_credentials
All files score 10.00/10 on pylint (fail-under=10) and 0 errors on
pyright strict.
Write credentials via a 0600 temp file + os.replace() so the token file
never appears at its final path with world-readable permissions, even if
the process is interrupted between write and chmod.
Parse poll_interval from the authorization response (clamped to 1–60 s,
falling back to _POLL_SLEEP) so aggressive polling can't trigger console
rate limits.
Tests: add atomicity spy asserting the temp file is 0600 before replace;
patch time.sleep instead of _POLL_SLEEP; add explicit interval-passthrough
assertion.
Starts a device-authorization flow against a bot-bottle console, polls
until the operator approves, then writes access + refresh tokens to
$BOT_BOTTLE_ROOT/console.json. Console URL is read from --console-url
flag or BB_CONSOLE_URL env var.
Part of didericis/bot-bottle-platform#1
didericis-claude
didericis-codex
didericis