Implements the interim secret-provider design (PRD prd-new-secret-provider):
each agent receives a random ENV_VAR_SECRET injected into its container env
at launch. The host uses this key to encrypt each egress auth token value
(HMAC-SHA256 CTR mode, stdlib-only) and store it in a new
bottled_agent_secrets table (one row per env var, key column plaintext for
auditing). The key never touches the DB.
On infra container restart the in-memory token map is lost. launch_consolidated
now calls _reprovision_running_bottles after ensure_running: for each
registered bottle still alive on the gateway network it execs
`printenv ENV_VAR_SECRET` into the agent container and posts the result to the
new POST /bottles/<id>/reprovision_gateway control-plane endpoint, which
decrypts the stored rows and restores _tokens — no manual intervention needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The orchestrator holds each bottle's egress auth tokens in process memory
only, so recreating the infra container strips every already-running
bottle of its upstream credentials. The registry row and the gateway CA
both survive; the tokens do not, so /resolve serves an intact policy with
an empty token map and the addon fails closed on `token_env unset`.
Drafts the interim slice of #355: persist the tokens encrypted so they
survive a restart, without regressing to plaintext at rest and without
foreclosing the per-request minting end state. Design section is left for
the author to fill in.
Adds a per-route boolean field preserve_auth (default false) that skips
the gateway's Authorization header stripping for that host. Intended for
registry endpoints like Docker Hub (registry-1.docker.io) and GHCR
(ghcr.io) where the agent must supply its own per-scope bearer token.
Threaded through ManifestEgressRoute → EgressRoute → Route, serialized
in route_to_yaml_dict, and parsed in parse_routes. The strip at
egress_addon.py now checks route.preserve_auth before popping the header.
Closes#392
The shared gateway self-generates a mitmproxy CA that every bottle installs
to trust its TLS interception. It was persisted on a Docker named volume,
which survives `docker rm` but is silently wiped by `docker volume prune` /
`docker system prune --volumes` during routine host maintenance. When that
happens the gateway mints a fresh CA on restart, and every already-running
bottle fails the TLS handshake even after it re-resolves and reconnects to
the moved gateway — a re-attachment blocker distinct from #443/#445.
Move CA persistence to a host bind-mount under the app-data root
(`bot_bottle_root()/gateway-ca`, via `host_gateway_ca_dir()`), mirroring how
the shared DB and control-plane token already live on the host. Docker never
prunes a path under the root, and it stays inspectable + rotatable from the
host. mitmproxy already adopts an existing CA and generates one only on first
run, so the bind-mount gives adopt-existing/generate-on-first-run for free.
Add an explicit rollover path: `rotate_gateway_ca()` clears the persisted CA
so the next start remints it, and `python -m bot_bottle.orchestrator.rotate_ca`
wires that together with dropping the running gateway container (whose
mitmproxy still holds the old CA in memory). Rotation stays an operator action
— it doesn't auto-re-provision running bottles, which re-attach to pick up the
new anchor.
Scope: the Docker infra/gateway path (the "infra container" in the report).
The macOS (`container`-only volume) and Firecracker (VM-attached ext4) backends
persist the CA differently and are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Approval now renders over the hosts page at /hosts/authorize?code=…
rather than on a standalone page. The console keeps /authorize as a
redirect, so this is cosmetic for older consoles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`python -m bot_bottle.cli` failed with "is a package and cannot be
directly executed" — the package had a `if __name__ == "__main__"` block
in `__init__.py`, which never fires for a package and made the invocation
look supported when it wasn't. Add a real `__main__.py` and drop the dead
block.
Matters for `bb login`, whose docstring documents a `bb` entry point that
nothing installs; `-m` is the closest thing to it until there's a
console-script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces broad `except Exception` with specific types that reflect the
actual failure modes:
- _save_credentials: `except OSError` (IO-only path; re-raises for cleanup)
- _post error handler: `except (OSError, ValueError)` (network + bad JSON)
- poll-loop: `except (OSError, ValueError)` (network + bad JSON)
Also:
- Simplify _fake_get to use next(..., default) — removes uncovered
StopIteration branch
- Add encoding="utf-8" to open() in test_approved_flow_returns_0
- Add test_cleanup_on_write_failure to cover the except OSError block
in _save_credentials
All files score 10.00/10 on pylint (fail-under=10) and 0 errors on
pyright strict.
Write credentials via a 0600 temp file + os.replace() so the token file
never appears at its final path with world-readable permissions, even if
the process is interrupted between write and chmod.
Parse poll_interval from the authorization response (clamped to 1–60 s,
falling back to _POLL_SLEEP) so aggressive polling can't trigger console
rate limits.
Tests: add atomicity spy asserting the temp file is 0600 before replace;
patch time.sleep instead of _POLL_SLEEP; add explicit interval-passthrough
assertion.
Starts a device-authorization flow against a bot-bottle console, polls
until the operator approves, then writes access + refresh tokens to
$BOT_BOTTLE_ROOT/console.json. Console URL is read from --console-url
flag or BB_CONSOLE_URL env var.
Part of didericis/bot-bottle-platform#1
Two issues introduced by the rebase conflict resolution:
- test_cached_lookup_requires_ready_marker used _dockerfile_hash as the
cache-dir key, but cached_agent_rootfs_dir now uses _rootfs_digest (which
also folds in the guest init). Updated the test to match.
- test_pyproject_toml_change_bumps_version appeared twice in
test_infra_artifact.py (once from main, once from the PR commit that
added pyproject.toml support). Removed the duplicate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dockerfile.gateway COPYs pyproject.toml into /src and runs pip install
/src, so it is a real input to the baked rootfs. A dependency-only change
previously reused stale artifact versions, potentially booting a rootfs
whose installed packages differed from the current checkout.
Also adds _fake_repo fixture support and a regression test so this input
can't silently drop out of the hash again.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FROM-scratch images (commit_container output) and registry images built
for reproducibility often omit the created field entirely. Both backends
were calling die() in that case, crashing the cached-image quickstart
before any container started.
image_created_at now returns datetime | None — None when the field is
absent or unparseable, die() only on real inspect failures (non-zero
exit, malformed JSON). stale_checks in both backends skips the staleness
check when None is returned.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- config_store.py imported host_db_path from supervise_types (wrong);
it lives in paths.py, matching all other stores (audit, queue, etc.)
- test_stale_checks: two tests expected check_stale called twice
(agent + sidecar) — consolidated arch has no sidecar, so once is
correct; update assertions and remove unused Path import
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pyright strict reportUnusedImport flagged BottleImages in
test_docker_launch_committed_image.py and test_macos_container_launch.py;
neither file references the type by name (they only use the returned
value's attributes).
`from __future__ import annotations` already defers all annotation
evaluation, so quoting `str | Path`, `BottleImages` inside the same
module was redundant and tripped pyright strict mode.
Addresses review comments 3098, 3099, 3100 on PR #336:
- Add BottleImages(agent, sidecar) dataclass to backend/__init__.py.
Docker/macOS backends use str image refs; smolmachines uses Path
artifacts. Replaces the singular `image` variable from the canonical
pattern in comment 3100.
- Replace _image_stale_checks/skip_stale with public prelaunch_checks().
CLI now calls backend.prelaunch_checks(plan) before backend.launch(plan);
if StaleImageError is raised and the operator confirms, launch proceeds
without re-checking. Removes the while-True/skip_stale retry loop.
- Add abstract _build_or_load_images(plan) -> BottleImages to
BottleBackend. launch() calls it then passes images to _launch_impl.
Each backend implements both methods.
- Fix comment 3098 (macos-container): _build_images is removed.
build_or_load_images() has separate fresh/cached code paths — the
cached path never calls a build helper.
- Update _start_bundle (smolmachines) to accept sidecar_artifact: Path
directly. Sidecar artifact resolution moves to _sidecar_from_path(),
called by build_or_load_images alongside _agent_from_path().
- image_cache: StaleImageError exception + check_stale/check_stale_path (raise instead of warn)
- BottleBackend.launch: template method (skip_stale flag) that calls _image_stale_checks then _launch_impl
- Each backend: _image_stale_checks delegates to a stale_checks() function in its launch module; _launch_impl replaces launch override
- macos_container: adds image_created_at to util, cached-image support in _build_images, stale_checks
- cli/start.py: catches StaleImageError, prompts interactively, retries with skip_stale=True; headless mode dies on it
Reads the host's Claude OAuth session key from ~/.claude.json at launch
and forwards it only to the egress sidecar (never to the agent), placing
a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent env so Claude Code
starts without seeing the real credential.
Mirrors the existing Codex forward_host_credentials flow (PRD 0029).
Adds claude_auth.py to extract and validate the sessionKey, a
CLAUDE_HOST_CREDENTIAL_TOKEN_REF constant in egress.py, and updates
manifest_agent.py to allow the flag for both 'codex' and 'claude'
templates. Also adds a mutual-exclusion check that rejects setting
both auth_token and forward_host_credentials together.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(claude): read credentials from ~/.claude/.credentials.json
The actual OAuth token is in ~/.claude/.credentials.json under
claudeAiOauth.accessToken, not in ~/.claude.json.
~/.claude.json holds only UI state and profile metadata (oauthAccount
has no token fields). expiresAt in the credentials file is milliseconds,
not seconds.
Discovered after testing against Claude Code 2.1.198.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(claude): fall back to macOS Keychain for credentials
On macOS, Claude Code stores credentials in the Keychain under
service "Claude Code-credentials" rather than in a file. When
~/.claude/.credentials.json is absent, shell out to:
security find-generic-password -s "Claude Code-credentials" -w
and parse the result as the same JSON schema.
~/.claude.json holds only profile/UI metadata (oauthAccount has
no token fields). expiresAt in the credentials is milliseconds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docs(prd): fix credential path references (~/.claude/.credentials.json)
fix(test): suppress gitleaks false positives on synthetic Claude tokens
upload-artifact@v3's glob silently skips hidden files, so uploading a
bare `.coverage.unit` logged "No files were found. No artifacts will be
uploaded" and registered nothing — the coverage job's download then 404'd
("List Artifacts failed: 404"). The coverage report step read the same
file fine, confirming it existed; only the leading dot broke the upload.
The old pipeline's cross-job artifacts (infra-candidate/, firecracker-
inputs) worked precisely because they were non-dotfiles.
Each test job now copies its .coverage.<suffix> to a non-dot
coverage-<suffix>.dat before upload (the cp also fails loudly if coverage
never wrote the file), and the coverage job renames them back to
.coverage.* before `coverage combine`.
coverage run's --data-file flag can be overridden or ignored in some
runner environments (Nix Python, older act-based runners). Switching to
the COVERAGE_FILE env var with an absolute ${{ github.workspace }} path
ensures coverage.py writes to a known location in every runner context,
so upload-artifact can find the file.
Also adds --reuse-published to the infra build step: if the artifact for
this content hash already exists in the registry, download it instead of
running the full docker build → mke2fs → gzip pipeline.
The delphi-ci runner resolves relative paths in upload-artifact and
download-artifact from a different CWD than run: shell steps, so
'.coverage.unit' etc. were never found. Using ${{ github.workspace }}
gives an absolute path that does not depend on the JS action's CWD.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each test job now runs once under coverage and uploads a small .coverage.*
artifact. The coverage job combines them on ubuntu-latest — no test reruns,
no KVM dependency. The infra candidate is built directly on the KVM runner,
eliminating the build-infra job and the ~70 s upload + ~83 s combined
download. For PRs, no rootfs artifact is transferred at all. Main-branch
pushes upload the tested rootfs and matching dropbear so publish-infra
publishes the byte-identical artifact. relative_files = True in .coveragerc
lets coverage files from different runners combine without path remapping.
Closes#446
Add tests for the three uncovered paths introduced by the poll_ca_cert
extraction: the timeout + sleep branches in backend/util, and the
TimeoutError → GatewayError and TimeoutError → die() conversions in the
macOS and Firecracker callers.
Extract the shared CA cert polling loop into `backend/util.poll_ca_cert`
(firecracker and macos backends were duplicating deadline/sleep/raise logic).
Each caller now wraps a fetch lambda and converts TimeoutError to its own
error type. Also corrects the PRD port publication line from {port}:{port}
to {host_port}:8099.
- `_build_images()` now builds Dockerfile.gateway → Dockerfile.orchestrator
→ Dockerfile.infra in order; Dockerfile.infra starts FROM bot-bottle-gateway
so the base must exist on clean hosts.
- Publish mapping corrected from `self.port:self.port` to `self.port:DEFAULT_PORT`
(8099) — gateway_init hardcodes the orchestrator on port 8099 inside the
container, so the host-side published port must map to that fixed internal port.
- BOT_BOTTLE_ORCHESTRATOR_URL inside the container now always points to
127.0.0.1:8099, not self.port, since gateway daemons reach the orchestrator
over loopback at the fixed internal port.
- Update test_ensure_running_builds_both_images → _all_images for the new
three-step build sequence; add test_publish_maps_host_port_to_fixed_internal_port
to lock in the port-mapping fix.
Addresses the P1 findings from the didericis-codex review on PR #432.
Add three new tests:
- noop when healthy but docker inspect fails (returns True → don't churn)
- build failure raises GatewayError
- _ensure_network creates the network when it doesn't exist
Also update the integration test to use new OrchestratorService API
(infra_name/image instead of orchestrator_name/gateway_name/gateway_image).
Brings diff-coverage from 86% to 90.3% against origin/main.
- Remove unused INFRA_IMAGE import from test_orchestrator_lifecycle
- Update integration test to use new single-container OrchestratorService
API (infra_name/image replaces orchestrator_name/gateway_name/gateway_image)
- Move type: ignore to the lambda line in gateway_init SIGHUP handler
- Break two long lines in test_orchestrator_lifecycle
- Extract _sigkill_all() to cut nesting depth below the 5-block limit
- Add pylint: disable=consider-using-with on Popen (process must outlive caller)
- Break long SIGHUP signal line to stay within 100 chars
Collapses the two-container Docker model (gateway + orchestrator) into one
bot-bottle-infra container, matching the macOS and Firecracker backends.
- Dockerfile.infra: now a shared gateway+orchestrator base (COPY bot_bottle
from orchestrator build, no CMD override)
- Dockerfile.infra.fc: new Firecracker-specific layer (buildah/crun/netavark)
- gateway_init: adds orchestrator daemon with _OPT_IN_DAEMONS gating so it
only starts when BOT_BOTTLE_GATEWAY_DAEMONS explicitly includes it
- orchestrator/lifecycle: OrchestratorService manages one infra container;
builds orchestrator (intermediate) then infra; live source bind-mounted at
/bot-bottle-src with PYTHONPATH so the subprocess uses the checkout
- backend/consolidated_util: extracts provision_bottle + teardown_consolidated
shared across all three backends; removes duplication in docker/fc/macos
consolidated_launch modules
- firecracker/infra_vm: builds four images (orchestrator→gateway→infra→infra.fc)
- All unit tests updated and passing (1878 tests)
- PRD status: Draft → Active
Adds PRD for collapsing the Docker backend from two containers
(gateway + orchestrator) to a single bot-bottle-infra container,
restructuring Dockerfile.infra as the shared base, and extracting
duplicated CA polling / teardown / provision helpers into a shared
backend utility module.
Closes#431
didericis-claude
didericis
didericis-codex