Move the eleven per-command modules (backend, cleanup, commit, edit,
info, init, list, login, resume, start, supervise) into a new
bot_bottle/cli/commands/ package, leaving the dispatcher (__init__),
entrypoint (__main__), and shared helpers (_common, tui) at the cli
root. Makes the command surface obvious at a glance and separates
handlers from the plumbing that registers them.
Updated the dispatcher's COMMANDS imports to .commands.*, bumped the
moved files' relative-import depths (.._common, .. import tui, sibling
.start unchanged), and repointed test references to
bot_bottle.cli.commands.*. Full unit suite green (2243); CLI dispatch
verified via `python -m bot_bottle.cli --help`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
egress_entrypoint.sh is the mitmdump launcher for the egress data plane,
so it belongs with the gateway egress service, not at the package root.
Moved to bot_bottle/gateway/egress/entrypoint.sh (prefix stripped now
that the package namespaces it).
Only the Dockerfile.gateway COPY source changes; the runtime target
(/app/egress-entrypoint.sh, referenced by bootstrap's egress DaemonSpec)
is unchanged, and the infra images inherit it via FROM gateway. Updated
the test that resolves the script path and the infra-artifact docstring
example. Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PID-1 process manager in gateway/bootstrap.py was named _Supervisor,
which collided conceptually with the supervise permissions service
(Supervisor / gateway.supervisor). It manages the _DaemonSpec child set,
so _DaemonManager names what it does without the "supervise" echo.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Group the gateway's data-plane modules into three service sub-packages
mirroring the host-side trio (bot_bottle.egress / .supervisor / .git_gate):
gateway/egress/ addon_core, addon, dlp_config, dlp_detectors
gateway/supervisor/ server (was supervise_server)
gateway/git_gate/ render, http_backend
Prefix-stripped filenames now that the package namespaces them; each
sub-package has a thin docstring __init__ (no eager imports, cheap leaf
loads). The two cross-cutting files stay at the gateway root:
policy_resolver (shared per-client lookup) and gateway_init, renamed to
bootstrap now that gateway/ already namespaces it.
Updated all importers (bot_bottle + tests), the in-VM/container `-m`
launch strings, the Dockerfile.gateway addon shim + ENTRYPOINT, and the
five gateway entries in scripts/critical-modules.txt. Full unit suite
green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Split the flat egress.py into an egress/ package: neutral DTOs
(EgressRoute, EgressPlan) in plan.py, and the concrete Egress service
class plus rendering/env helpers in service.py, behind a thin
__getattr__ facade so leaf imports stay cheap.
Egress drops ABC and becomes a concrete service the backends call:
resolve_token_values / agent_env_entries / prepare are now methods.
Backend launch paths (docker, firecracker, macos_container) call
Egress().<method>(...) instead of the module-level functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn the git_gate module into a package with GitGate as a concrete service class
(dropping the ABC), mirroring the Supervisor shape. The host-side git-gate
operations are now methods the backend drives:
git_gate/
__init__.py — thin __getattr__ facade (keeps `from bot_bottle.git_gate
import …` working; render/hook names lazily forwarded to
gateway.git_gate_render)
plan.py — GitGatePlan (the launch DTO the backend contract references)
service.py — GitGate: prepare / provision_dynamic_keys /
revoke_provisioned_keys / preflight_host_keys
provision.py — deploy-key lifecycle (was git_gate_provision.py)
host_key.py — host-key preflight (was git_gate_host_key.py)
The backend launch + base.py preflight now call GitGate() methods instead of the
free functions. The runtime rendering / in-gateway hook execution stay in
gateway/git_gate_render.py (data plane) — only the host-side service moved.
Because the facade forwards names lazily, the ~25 `from bot_bottle.git_gate
import GitGatePlan` call-sites are unchanged, and importing git_gate.plan (the
contract's dependency) no longer drags in the provisioning / forge-API code.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
backend/__init__.py eagerly imported the whole framework (manifest, egress,
git-gate, env, workspace, agent-provider), so importing ANY backend.* submodule
paid ~32 modules just to run the package init. Split it:
- backend/base.py — the abstract contract (BottleSpec, BottlePlan,
BottleCleanupPlan, ExecResult, ActiveAgent, Bottle,
BottleImages, BottleBackend). Carries the framework
imports; loaded only when a caller needs the contract.
- backend/selection.py — backend registry / selection / enumeration
(get_bottle_backend, _get_backends, _auto_select_backend,
has_backend, known_backend_names, enumerate_active_agents).
Concrete backends still imported lazily inside.
- backend/__init__.py — thin: a __getattr__ that resolves the public names
from those submodules on first access (+ a TYPE_CHECKING
block for checkers). Existing `from bot_bottle.backend
import X` and patch.object call-sites keep working.
`import bot_bottle.backend` drops from 32 -> 2 bot_bottle modules. Repointed the
backend-selection test's patch targets to the `selection` module (the functions
reference each other there now; the FirecrackerBottleBackend class-method
patches are unchanged — same class object).
Note: backend.docker.util is still heavy because backend/docker/__init__ eagerly
imports DockerBottleBackend — slimming the three sub-package inits is the
follow-up that makes the leaves cheap.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OrchestratorService wasn't the orchestrator — it's the host-side lifecycle of
the docker infra *container* (the one that runs the Orchestrator). It read as
"the orchestrator as a service" and lived in orchestrator/lifecycle.py, while
its siblings (macOS MacosInfraService, Firecracker infra_vm) live under their
backend package. Rename it DockerInfraService and move it to
backend/docker/infra.py alongside the docker backend, with its docker-only
constants (INFRA_*/ORCHESTRATOR_* image + container names, daemon list, mount
paths).
orchestrator/lifecycle.py keeps only the backend-neutral pieces the other infra
services share — DEFAULT_PORT, DEFAULT_STARTUP_TIMEOUT_SECONDS,
OrchestratorStartError, source_hash — which macOS / firecracker / client still
import from there. backend/docker/infra.py imports those (backend -> orchestrator
is an allowed direction). Renamed the unit test to test_docker_infra.py.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generalize render_diff so it isn't supervise-flavored: the "(current)" /
"(proposed)" side titles are no longer baked in — callers pass `before_title`
and `after_label` (required). Move it from orchestrator/supervisor/util.py to
the base bot_bottle.util alongside sha256_hex, and delete the now-empty
supervisor/util.py.
The supervise callsite (Orchestrator.supervise_respond) assigns
before_title="current", after_label="proposed"; the facade no longer re-exports
render_diff (callers import from bot_bottle.util). Behavior at the supervise
callsite is unchanged.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move sha256_hex out of orchestrator/supervisor/util.py to the root
bot_bottle.util (it's a generic string hash, not supervise-specific), and have
Proposal.new take the proposed file contents and compute current_file_hash
itself instead of every caller passing sha256_hex(proposed_file).
Every call site set current_file_hash to the hash of the proposed file, and
nothing reads the field except the DB round-trip (schema + _row_to_proposal +
from_dict, all untouched), so folding the hash into the factory is
behavior-preserving and removes the boilerplate. Callers now pass just the
file; the orchestrator no longer imports sha256_hex at all.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn the loose queue/audit functions in orchestrator/supervisor/queue.py into
methods on a concrete Supervisor service. The Orchestrator now owns an
injectable `self._supervisor` and calls `self._supervisor.write_proposal(...)`
etc., instead of module-level free functions — the supervise dependency is
explicit and mockable, and a Supervisor can be scoped to a `db_path` (defaults
to the host DB) so tests can point it at a temp database.
- Supervisor (in the package __init__) drops the ABC and gains write_proposal,
read_proposal, list_pending_proposals, list_all_pending_proposals,
write_response, read_response, archive_all_proposals, write_audit_entry,
read_audit_entries, plus the launch-time prepare().
- render_diff / sha256_hex are pure and stateless, so they move to
orchestrator/supervisor/util.py (re-exported from the facade) rather than
becoming methods.
- queue.py is deleted; service.py + the supervise tests call through a
Supervisor instance.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the supervise queue helpers that no live path uses anymore — they're
mechanisms the PRD-0070 / idempotent-poll migration superseded:
- archive_proposal (+ QueueStore.archive_proposal): the poll stopped archiving
on read (#469 review), so single-proposal archiving has no caller; decided
proposals drop off the pending list via their response row and are reaped in
bulk by archive_all_proposals on teardown/reconcile.
- wait_for_response: the data plane polls the queue over the control-plane RPC
now, so nothing block-waits on it.
- audit_dir / audit_log_path: file-based audit paths, replaced by the SQLite
AuditStore.
Also drops the tests that covered those obsolete helpers. Kept the prod-unused
read accessors list_pending_proposals and read_audit_entries: those aren't dead
mechanisms — they're the read side of live write paths that many tests use to
verify real queue/audit behavior.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the `Supervise` lifecycle out of its own `orchestrator/supervisor/
supervise.py` and into the package `__init__`, renaming the class to
`Supervisor`. Callers now import it from `bot_bottle.orchestrator.supervisor`
alongside the queue surface it belongs with.
Remove the dead `try/except ImportError` flat-import fallbacks from the
package-only store modules (db_store, audit_store, config_store, queue_store)
and image_cache. Those fallbacks existed for when the store files were
flat-copied into the gateway; post-PRD-0070 the data plane never opens the DB,
so these modules are only ever imported as part of the package. The two gateway
data-plane files that may still be loaded flat (egress_addon_core,
git_gate_render) keep their fallbacks.
Full unit suite green (2251).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Give each service its own store package + manager, and cut the supervise module
along the control/data-plane boundary so nothing in the shared layer reaches up
into the orchestrator.
Stores, by owner:
- bot_bottle/store/ keeps only the shared base (DbStore, migrations) and the
concrete stores that aren't service-owned (audit_store, config_store).
- bot_bottle/orchestrator/store/ now houses the orchestrator-owned stores —
queue_store (supervise queue), secret_store, config_store — plus a new
orchestrator store_manager that migrates them (composing audit/config
downward from the base). The old shared store_manager is gone.
Supervise plane, by tier:
- bot_bottle/supervisor/ (NEUTRAL, importable by every tier including the
gateway): types.py (the Proposal/Response/AuditEntry wire types + the tool/
status/poll constants + the shared daemon constants moved out of
supervise.py) and plan.py (SupervisePlan, a pure DTO).
- bot_bottle/orchestrator/supervisor/ (orchestrator-only): queue.py (the
queue/audit I/O wrappers + render_diff + sha256_hex) and supervise.py (the
Supervise lifecycle that stages the DB via the store manager). Its __init__
re-exports the neutral vocabulary so orchestrator-side callers import from
one place.
The gateway now imports only bot_bottle.supervisor.types (never
bot_bottle.supervise), so the data plane holds no code dependency on the
orchestrator — it reaches the queue over the control-plane RPC. This removes
the circular import that moving queue_store under orchestrator introduced
(supervise -> orchestrator -> service -> supervise).
supervise_types.py -> supervisor/types.py; supervise.py deleted (split). Full
unit suite green (2251).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Separate the gateway (data plane) from the orchestrator (control plane) at the
module level. The gateway runtime files move out of the package root — and the
backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of
orchestrator/ — into a new bot_bottle/gateway/ package:
gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts
+ rotate_gateway_ca)
gateway/gateway_init.py (the PID-1 daemon supervisor)
gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py,
dlp_detectors.py (the egress mitmproxy daemon)
gateway/git_http_backend.py (the git-http daemon)
gateway/git_gate_render.py (the git-gate pre-receive rendering)
gateway/supervise_server.py (the supervise MCP daemon)
gateway/policy_resolver.py (the data-plane control-plane RPC client)
orchestrator/ now holds only control-plane files. The shared plan/types/auth
layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py,
supervise_types.py, control_auth.py) and the launch-time git-gate provisioning
helpers stay at root, so orchestrator/ and backend/ still own them.
Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat
by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than
Python imports: the `-m` invocations (firecracker/macOS infra scripts), the
Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module
paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*.
No behavior change; full unit suite green (2251).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the nine root-level manifest modules into a bot_bottle/manifest/ package,
dropping the redundant manifest_ prefix: the facade (manifest.py) becomes the
package __init__ and keeps re-exporting Manifest / ManifestIndex and the piece
types, while manifest_<x>.py become manifest/<x>.py (agent, bottle, egress,
extends, git, loader, schema, util).
Because the facade stays the package __init__, the ~13 callers that import
`from bot_bottle.manifest import ...` are unchanged; only direct-piece imports
move to `bot_bottle.manifest.<name>`. Inside the package, intra-manifest
imports drop the prefix and the non-manifest siblings (log, yaml_subset,
agent_provider) move to `..`.
No behavior change; full unit suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the DbStore base (db_store), the shared schema-migration base (migrations /
TableMigrations), the concrete stores (audit_store, config_store, queue_store),
and StoreManager out of the package root into a bot_bottle/store/ package, so
persistence lives in one place rather than scattered across the root.
Callers use direct submodule imports (from bot_bottle.store.store_manager
import StoreManager). Inside the moved modules the relative imports point back
up to their non-store siblings (..paths, ..supervise_types) while co-moved
siblings stay package-local; the flat-import fallbacks are untouched. The
orchestrator's own stores (config_store, secret_store, registry) stay in
orchestrator/ and repoint to the moved DbStore / TableMigrations.
No behavior change; full unit suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The `DockerGateway` container-lifecycle impl now lives in
`backend/docker/gateway.py` (the shape backend gateway classes will share);
`orchestrator/gateway.py` keeps only the backend-neutral pieces (the `Gateway`
ABC, constants, `rotate_gateway_ca`).
Delete the standalone-gateway path it was the only consumer of. `--gateway` on
`python -m bot_bottle.orchestrator` was invoked nowhere — the production docker
flow runs the gateway data plane inside the combined `bot-bottle-infra`
container via `OrchestratorService`, never this class. Removing it takes with it
`Orchestrator.ensure_gateway()` and the `gateway` ctor arg; `gateway_status()`
becomes a stub reporting `configured: false` so the documented `GET /gateway`
control-plane route keeps its contract.
Fix a latent bug the move surfaced: `launch.py` read the shared gateway CA via
`docker exec bot-bottle-orch-gateway`, a container the consolidated flow never
creates — so the agent CA install was reaching a nonexistent name. Read it from
`bot-bottle-infra` (INFRA_NAME) instead.
Repoint the affected tests to the new module; drop the unit test + fake that
covered the deleted standalone wiring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Firecracker infra VM started the control plane with no signing key, so it
ran open and granted every unauthenticated caller the `cli` role. The nft
boundary only fences off the separate agent VM — the egress / supervise /
git-http daemons run in this same VM and reach the orchestrator over 127.0.0.1,
so a compromised data-plane daemon could still drive the operator routes
(approve its own supervise proposals, rewrite policy, read injected tokens).
Generate the signing key on the persistent volume, hand it only to the
orchestrator process, mint a `gateway` JWT for the data-plane daemons, and
mirror the key back to the host token file so the CLI signs `cli` tokens the VM
verifies — the same per-process scoping the docker / macOS launchers use.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The control-plane supervise poll archived the proposal on the same call
that returned the decision, so a dropped connection lost the operator's
decision (the retry saw `unknown`). Poll no longer archives — a re-poll
returns the same decision. Decided proposals still drop off the operator's
pending list (a response row exists); their rows are reaped when the
bottle is torn down or reconciled.
Also raise the grace-window poll interval from 50ms to 250ms now that each
poll is an HTTP round trip rather than a local file read.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review follow-up on #469: the data plane held the same control-plane secret
that authorizes every route, so a compromised egress/git-gate could queue a
supervise proposal AND approve it (or rewrite policy, read injected tokens) —
the (source_ip, identity_token) checks attribute the *bottle*, not the caller.
Replace the single shared bearer secret with role-scoped, HMAC-signed tokens
(compact HS256 JWTs, stdlib-only — no new dependency):
* new `control_auth` mints/verifies `{role}` tokens; roles are `gateway`
(data plane) and `cli` (host operator/launcher).
* the orchestrator holds only the signing *key* and verifies; `dispatch`
gates each route by role — `gateway` reaches /resolve + /supervise/
{propose,poll}, everything else is `cli`-only (401 unauthenticated,
403 wrong role).
* the gateway is handed a pre-minted `gateway` token it cannot rewrite into
`cli`; the host CLI mints its own `cli` token from the host key.
* `gateway_init` scopes the signing key to the orchestrator process and the
gateway token to the data-plane daemons, so even in the combined infra
container a compromised data-plane daemon never sees the key.
Launchers (docker gateway + infra, macOS infra) inject the minted token(s);
Firecracker stays open behind its nft boundary. Open mode (no key) still grants
full `cli` access — the fail-visible fallback for tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add pyright-strict parameter/return annotations to the fake resolvers and
test helpers, and cover the new control-plane validation/403 branches
(/supervise/propose + /supervise/poll) plus the supervise-server and egress
poll-error fail-closed paths, so the diff-coverage gate stays above 90%.
No production behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PRD 0070's rule — only the orchestrator opens bot-bottle.db; the data
plane reaches state through the control-plane RPC — was not in force.
Three data-plane daemons held a direct read-write handle on the shared
SQLite file: the supervise MCP server, the egress DLP addon (the most
attack-exposed process, TLS-bumping hostile traffic), and the git-gate
pre-receive hook. An RCE in any of them could read every bottle's
plaintext identity_token and forge attribution fleet-wide (issue #469).
Add the agent half of the supervise flow to the control plane:
POST /supervise/propose -> queue a proposal, 201 {proposal_id}
POST /supervise/poll -> non-blocking decision poll, 200 {status,...}
Both attribute the caller by (source_ip, identity_token) exactly like
/resolve — never a caller-supplied slug — so a bottle can only ever queue
or read its own proposals even if the data plane is compromised. A decided
poll archives server-side, preserving the archive-after-read contract.
Data plane: the supervise server, egress addon, and git-gate hook now
queue/poll through PolicyResolver.propose_supervise / poll_supervise
instead of opening the DB. supervise_server keeps its ~30s grace window
by polling the RPC; egress keeps its safelist keyed by resolved bottle;
the git-gate hook gets (source_ip, identity_token) from the CGI env.
Packaging: drop the DB bind-mount and SUPERVISE_DB_PATH from the
data-plane containers/VMs (docker gateway + infra, macOS infra,
firecracker infra). The orchestrator remains the sole opener of the one
file via BOT_BOTTLE_ROOT / host_db_path().
Update PRD 0070: the rule is now in force; remove the transitional caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two complementary fixes for the egress gateway OOM:
1. gateway_init: auto-restart any daemon that dies unexpectedly. The
supervisor already had restart_daemon()/request_restart() logic; this
wires it into tick() so an OOM-killed mitmdump is respawned without
operator intervention. Implements the "eventual" failure policy
described in the original module docstring.
2. egress_addon: cap response body bytes passed to the DLP inbound scan
at EGRESS_INBOUND_SCAN_LIMIT_BYTES (default 1 MiB). mitmproxy buffers
the full response before the hook fires; capping at scan time limits
the additional amplification from decoded text and regex strings. Bodies
over the limit emit an egress_scan_truncated log event. Set to 0 to
disable the cap.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The PTY check added in e719022 calls sys.stdin.fileno() which raises
io.UnsupportedOperation under pytest's stdin capture. Guard the fileno()
call with a try/except so the check degrades cleanly to "not a tty"
instead of crashing, then stub os.isatty in the test setUp so headless
unit tests aren't blocked on a real TTY.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
merge_bottles_runtime used .get(n, base_repos_by_name[n]) which eagerly
evaluates the default, crashing when a repo exists only in the override
bottle. Replaced with a conditional expression so the base lookup only
runs when needed.
Adds three regression tests covering override-only, base-only, and
name-collision cases.
Closes#457
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add unit coverage for the encrypted at-rest egress secrets: the
secret_store round-trip (new_env_var_secret/encrypt_value/decrypt_value)
and the registry/service wiring that injects ENV_VAR_SECRET.
Recovered from the bot-bottle-claude-agent-1 VM after the CI runner's
disk filled and forced its rootfs read-only; committed work was already
on origin, these were the agent's uncommitted changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Routes with `dlp: false` skip all DLP scanning (including CRLF injection,
which cannot be disabled via `dlp.outbound_detectors: false`) and tunnel
HTTPS connections without TLS interception so the client sees the server's
real certificate. Fixes Docker image pulls, which fail when the proxy MITM's
the TLS handshake and the container doesn't trust the per-bottle CA.
Closes#462
The runtime merge (merge_bottles_runtime / _fold_two_bottles) was using
OR semantics for nested_containers, which meant `nested_containers: false`
in a later bottle could never override an earlier `true`.
Fix: add `nested_containers_explicit: bool` to ManifestBottle, set to
True only when the key is present in the source dict. The runtime merge
functions now use the override's value only when it was explicitly
supplied; otherwise the base's value is preserved. This matches the
documented last-wins / presence-aware semantics used by supervise and
agent_provider, and aligns with _merge_bottles (the file-based extends
path) which already did presence checks via `"nested_containers" in
child_raw`.
Tests: rename existing test to clarify the preserved-when-omitted case,
add explicit-false-overrides-true and explicit-true-overrides-false cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace test_all_install_podman with test_none_install_podman: agent
images intentionally no longer include podman; the nested-containers
derived layer is where it lives (see build_image in nested_containers.py).
- Add podman to the package assertions in test_macos_nested_containers so
its presence in the derived Dockerfile is explicitly verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This is the layer that actually decides it, and it was in the script
from the start: ~/.docker/config.json's proxies block is what the Docker
CLI copies into every container it starts. Being client-side it beats
the podman service outright, which is why containers.conf `env`,
http_proxy=false, and the service environment each looked correct on
disk and changed nothing a nested container saw.
Also corrects the comment on the service-env substitution, which
claimed to be the one place the address sticks. It is not; it covers
podman's own pulls and non-CLI API clients.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Third attempt at the same symptom, and the first at the right layer.
podman's Docker-compatible API injects proxy settings from the *daemon*
environment after containers.conf is applied, so neither the env list
(892bfc16) nor http_proxy=false (21c144ae) could override it — the same
compat-path blind spot already seen with hosts_file. Nested containers
kept reporting `wget: bad address 'bot-bottle-gateway:9099'` through
both.
Substituting the resolved address into the service's own environment,
before `podman system service` starts, is the one place it sticks.
NO_PROXY keeps the name, which is matched against what a client asks
for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The address-bearing proxy URLs from 892bfc16 were written correctly but
never took effect: podman copies the host's proxy environment into every
container by default, and that copy overrides containers.conf `env`,
restoring the `bot-bottle-gateway` name a nested container cannot
resolve. Verified on the macOS host — containers still reported
`wget: bad address 'bot-bottle-gateway:9099'` with the addresses in
place.
Setting http_proxy=false disables only podman's own passthrough; the
proxy vars still reach containers, by address, from the env list.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman's containers.conf `hosts_file` turns out to apply only to native
`podman run`; the Docker-compatible API ignores it, and the agent types
`docker`. (`base_hosts_file`, used before this, is a no-op in both
paths on podman 5.4.2.) So no config key can put the gateway name in a
nested container's /etc/hosts.
The name is therefore resolved in the bottle, where /etc/hosts does
carry it, and the address goes into the proxy URL handed to nested
containers. NO_PROXY keeps the name, since that is matched against what
a client asks for.
Verified on the macOS host: with the gateway reachable, a nested
container gets 200 from an allowlisted host and 403 from one that is
not — the egress boundary applies inside nested containers too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The flag from #453 was parsed off the manifest and honored by the
addon, but `_route_to_yaml_fields` never wrote it, so it was dropped on
the way to the proxy and has never worked end to end. Every registry
route silently kept the unconditional Authorization strip.
That is the whole of the Docker Hub / GHCR "unauthorized" blocker in
#392: podman fetched a valid 2658-byte bearer from auth.docker.io, the
proxy stripped it, and registry-1.docker.io answered the resulting
anonymous request with a fresh WWW-Authenticate challenge — which reads
exactly like a credential problem rather than a serializer gap.
Found by diffing a bottle's rendered routes.yaml against its manifest:
preserve_auth was in the manifest and absent from the output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Answers the last open question in #392: DNS from inside a nested
container was not a phantom, but it was also not a DNS problem. Public
resolution fails there by design — everything egresses through the
proxy — and the actual breakage was two missing pieces:
- The proxy URL names `bot-bottle-gateway`, which resolves only through
the bottle's own /etc/hosts. Containers got their own hosts file, so
they failed at "Could not resolve proxy", which reads like broken DNS.
base_hosts_file propagates the bottle's entries.
- The gateway TLS-intercepts, so a container that does not trust the
bottle's CA bundle fails with "unable to get local issuer
certificate". The bundle is now mounted read-only and the usual env
vars point at it, which covers curl, wget, python, and node without
distro-specific trust commands.
Verified on the macOS host by reproducing each failure and confirming
that these three conditions applied by hand produced HTTP 200 from
inside a nested container. podman already forwards the proxy env, so
that needed nothing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
netavark shells out to `nft` for the bridge network every compose file
expects, and aardvark-dns serves name resolution inside nested
containers. Neither arrives with the base image's
`--no-install-recommends` podman, and each fails at a different and
misleading layer: without nft containers are created but never start;
without aardvark-dns DNS inside a container fails while the engine, the
pulls, and the bridge all look healthy. That last one is the likeliest
explanation for the "DNS inside nested containers fails entirely" note
in #392 — a missing package, not a design gap.
Verified on the macOS host: with passt alone, `docker run` reached
netavark and died on "unable to execute nft".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman 5 uses pasta as the default rootless network helper; podman 4
used slirp4netns. The spike was written against bookworm's podman 4.3.1,
so its package list never included passt — and the trixie bump (#451)
changed the default out from under it. Every nested container failed to
start with "could not find pasta", while image pulls and the service
itself worked, which disguised a missing dependency as a compat-API bug.
The bootstrap now checks for pasta up front so this fails with a clear
message instead of surfacing at the first `docker run`.
Note the sun_path fix in a12d2191 did NOT cause or cure this; that path
was over the 108-byte limit independently and would have bitten as soon
as attach was reached.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
podman derives conmon's attach socket as
`$XDG_RUNTIME_DIR/libpod/tmp/socket/<64-hex-id>/attach`. With the
descriptive `/tmp/bot-bottle-podman-run` that path is 116 bytes, past
the 108-byte sun_path limit, so every attached `docker run` failed with
"unable to upgrade to tcp, received 500" — while image pulls and the
service itself worked, which is what made it look like a compat-API bug
rather than a path-length one.
Found on the macOS host: quay.io pulled fine and then no container
could be started. Test pins the budget so a more readable name cannot
silently reintroduce it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
merge_bottles_runtime sees resolved bottles, where a bottle that never
mentioned the key is indistinguishable from one that set it false. With
"later replaces" semantics, `--bottle with-containers --bottle claude-dev`
silently dropped the capability — the second bottle's default clobbered
it. OR instead: composing bottles adds capabilities, it does not remove
them. The file-based extends path still sees raw keys, so an explicit
`nested_containers: false` in a child continues to turn it back off.
Also surface the flag in the launch summary, where its absence was the
only reason the clobber went unnoticed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the `nested_containers` bottle flag. On the macOS backend it starts
a rootless podman service inside the bottle and exposes its
Docker-compatible API socket, so the agent still runs `docker` and
`docker compose`. No host daemon socket is mounted and the guest gains
no capabilities; backends that cannot run a guest-local engine reject
the flag in the shared prepare template rather than ignoring it.
Podman rather than rootless Docker because Apple Container's capability
bounding set omits CAP_SYS_ADMIN, which the kernel requires to write a
multi-range uid_map via newuidmap. With no subordinate UID range podman
falls back to a single-UID self-mapping an unprivileged process may
write itself, so the image build strips /etc/subuid and /etc/subgid
entries rather than adding them.
That mapping is also why nested containers are not an isolation layer:
root inside one is the agent user outside it. They are a build/test
convenience; the bottle remains the boundary.
Ports the spike branch onto main, renaming docker_access — it implied
Docker and granted access to nothing on the host — and drops podman
from the derived layer now that every built-in image ships it (#451).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
didericis
didericis-claude
didericis-codex