Compare commits

..

158 Commits

Author SHA1 Message Date
didericis-claude 182a28d724 docs(research): firecracker image remote store
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 40s
tracker-policy-pr / check-pr (pull_request) Successful in 6s
test / integration-firecracker (pull_request) Successful in 3m33s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
test / integration-docker (push) Successful in 35s
test / unit (push) Successful in 39s
Update Quality Badges / update-badges (push) Successful in 47s
lint / lint (push) Successful in 51s
test / integration-firecracker (push) Successful in 5m8s
test / coverage (push) Successful in 44s
test / publish-infra (push) Successful in 1m53s
Investigates storage candidates, retention policy design, and
image-size reduction techniques in response to the PR #459 comment
on a secure remote store for committed Firecracker snapshots.
2026-07-21 22:47:44 -04:00
didericis cfb2284b99 fix(firecracker): make cleanup reap orphans only, never live VMs
`cleanup()` enumerated *every* firecracker process under the run root and
*every* run dir, so `./cli.py cleanup` would kill running bottles and
delete their rootfs — despite being described as orphan cleanup. The
backend's `enumerate_active` registry is still a stub (#354), so there
was no live/dead signal.

Use the running firecracker processes themselves as that signal: a run
dir with a live VM is protected (never killed, never removed); only run
dirs with no live process (leaked by a hard-killed launch) are reaped,
plus firecracker pids whose run dir is already gone (lingering VMMs).

This makes cleanup safe to run alongside live bottles and lets it back a
periodic GC for the run-dir leak the launch.py teardown fix closes on the
clean-exit path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 22:47:44 -04:00
didericis 2cd06814e6 fix(firecracker): remove per-bottle run dir on teardown
Each Firecracker bottle created its run dir (holding an ~1G
rootfs.ext4) but never removed it — launch teardown only terminated
the VM. Every run leaked its rootfs, filling the CI runner's disk
(187 stale sandbox-tester dirs / 141G in one incident).

Register a run-dir rmtree callback on the ExitStack before the
vm.terminate callback so, LIFO, the VM is gone before its rootfs is
removed. ignore_errors keeps teardown resilient.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 22:47:44 -04:00
didericis-codex 8fed02fd1e chore(ci): gate tests on relevant runtime inputs 2026-07-22 01:24:02 +00:00
Quality Badge Bot a7b2befc06 chore: update quality badges
- Coverage: 83%
- Core coverage: 94%

[skip ci]
2026-07-22 01:19:07 +00:00
didericis-codex 6fdf090469 fix(ci): authenticate badge pushes with PAT
test / unit (push) Failing after 2s
test / integration-docker (push) Successful in 10s
test / integration-firecracker (push) Successful in 2m59s
test / coverage (push) Has been skipped
test / publish-infra (push) Has been skipped
2026-07-22 01:09:42 +00:00
didericis-codex d32e9cc3c3 fix(ci): use Gitea token for badge updates
test / integration-docker (push) Successful in 11s
test / unit (push) Successful in 36s
test / integration-firecracker (push) Successful in 3m13s
test / coverage (push) Successful in 43s
test / publish-infra (push) Successful in 44s
2026-07-22 00:53:32 +00:00
didericis-codex ef89ed084f fix(ci): allow badge workflow to push updates
test / integration-docker (push) Successful in 21s
test / unit (push) Successful in 47s
lint / lint (push) Successful in 3m3s
test / integration-firecracker (push) Successful in 3m39s
test / coverage (push) Successful in 29s
Update Quality Badges / update-badges (push) Failing after 49s
test / publish-infra (push) Successful in 1m17s
2026-07-21 19:19:43 +00:00
didericis-claude ccd987a501 feat(egress): add preserve_auth flag to pass agent Authorization through
test / integration-docker (push) Successful in 11s
test / unit (push) Successful in 37s
lint / lint (push) Successful in 55s
Update Quality Badges / update-badges (push) Failing after 37s
test / integration-firecracker (push) Successful in 4m49s
test / coverage (push) Successful in 14s
test / publish-infra (push) Successful in 1m42s
Adds a per-route boolean field preserve_auth (default false) that skips
the gateway's Authorization header stripping for that host. Intended for
registry endpoints like Docker Hub (registry-1.docker.io) and GHCR
(ghcr.io) where the agent must supply its own per-scope bearer token.

Threaded through ManifestEgressRoute → EgressRoute → Route, serialized
in route_to_yaml_dict, and parsed in parse_routes. The strip at
egress_addon.py now checks route.preserve_auth before popping the header.

Closes #392
2026-07-21 15:01:59 -04:00
didericis-codex 2cd44cf79a fix(macos): persist gateway CA on host
test / integration-docker (push) Successful in 19s
Update Quality Badges / update-badges (push) Failing after 42s
lint / lint (push) Successful in 52s
test / unit (push) Successful in 1m43s
test / integration-firecracker (push) Successful in 4m56s
test / coverage (push) Successful in 17s
test / publish-infra (push) Successful in 1m49s
2026-07-21 15:01:38 -04:00
didericis-claude 8a1b833aaa fix(gateway): persist mitmproxy CA on the host, not a named volume (#450)
The shared gateway self-generates a mitmproxy CA that every bottle installs
to trust its TLS interception. It was persisted on a Docker named volume,
which survives `docker rm` but is silently wiped by `docker volume prune` /
`docker system prune --volumes` during routine host maintenance. When that
happens the gateway mints a fresh CA on restart, and every already-running
bottle fails the TLS handshake even after it re-resolves and reconnects to
the moved gateway — a re-attachment blocker distinct from #443/#445.

Move CA persistence to a host bind-mount under the app-data root
(`bot_bottle_root()/gateway-ca`, via `host_gateway_ca_dir()`), mirroring how
the shared DB and control-plane token already live on the host. Docker never
prunes a path under the root, and it stays inspectable + rotatable from the
host. mitmproxy already adopts an existing CA and generates one only on first
run, so the bind-mount gives adopt-existing/generate-on-first-run for free.

Add an explicit rollover path: `rotate_gateway_ca()` clears the persisted CA
so the next start remints it, and `python -m bot_bottle.orchestrator.rotate_ca`
wires that together with dropping the running gateway container (whose
mitmproxy still holds the old CA in memory). Rotation stays an operator action
— it doesn't auto-re-provision running bottles, which re-attach to pick up the
new anchor.

Scope: the Docker infra/gateway path (the "infra container" in the report).
The macOS (`container`-only volume) and Firecracker (VM-attached ext4) backends
persist the CA differently and are unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 15:01:38 -04:00
didericis-codex 3b5c55bc8e test(git): cover provisioning failures
prd-number / assign-numbers (push) Failing after 17s
test / integration-docker (push) Successful in 19s
Update Quality Badges / update-badges (push) Failing after 44s
lint / lint (push) Successful in 52s
test / unit (push) Successful in 1m44s
test / integration-firecracker (push) Successful in 4m40s
test / coverage (push) Successful in 15s
test / publish-infra (push) Successful in 1m34s
2026-07-21 14:52:44 -04:00
didericis-codex 95220b4808 fix(firecracker): restore agent home ownership at boot 2026-07-21 14:52:44 -04:00
didericis-codex b032562d74 fix(firecracker): pin the provisioned Git config 2026-07-21 14:52:44 -04:00
didericis-codex 1d925172ec fix(firecracker): repair runtime Git config ownership 2026-07-21 14:52:44 -04:00
didericis-codex f6ae485b68 Revert "refactor(agent-images): use explicit Debian base"
This reverts commit 51b82f80d1.
2026-07-21 14:52:44 -04:00
didericis-codex e3258d0683 refactor(agent-images): use explicit Debian base 2026-07-21 14:52:44 -04:00
didericis-codex 0ff11d8ed7 fix(agent-images): own Git config directory 2026-07-21 14:52:44 -04:00
didericis-codex a970f974a2 fix(agent-images): retain SSH client 2026-07-21 14:52:44 -04:00
didericis-codex c845d3fed4 feat(agent-images): update Debian and add Podman 2026-07-21 14:52:44 -04:00
didericis c6a9419b95 feat(login): point the printed URL at the hosts-page modal
test / integration-docker (push) Successful in 13s
test / unit (push) Successful in 37s
Update Quality Badges / update-badges (push) Failing after 41s
lint / lint (push) Successful in 55s
test / integration-firecracker (push) Successful in 4m44s
test / coverage (push) Successful in 15s
test / publish-infra (push) Successful in 1m29s
Approval now renders over the hosts page at /hosts/authorize?code=…
rather than on a standalone page. The console keeps /authorize as a
redirect, so this is cosmetic for older consoles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 14:25:38 -04:00
didericis 36fb019007 feat(cli): make bot_bottle.cli runnable with python -m
`python -m bot_bottle.cli` failed with "is a package and cannot be
directly executed" — the package had a `if __name__ == "__main__"` block
in `__init__.py`, which never fires for a package and made the invocation
look supported when it wasn't. Add a real `__main__.py` and drop the dead
block.

Matters for `bb login`, whose docstring documents a `bb` entry point that
nothing installs; `-m` is the closest thing to it until there's a
console-script.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 14:25:38 -04:00
didericis-claude adc033a902 fix(login): narrow exception types and cover cleanup path
Replaces broad `except Exception` with specific types that reflect the
actual failure modes:
- _save_credentials: `except OSError` (IO-only path; re-raises for cleanup)
- _post error handler: `except (OSError, ValueError)` (network + bad JSON)
- poll-loop: `except (OSError, ValueError)` (network + bad JSON)

Also:
- Simplify _fake_get to use next(..., default) — removes uncovered
  StopIteration branch
- Add encoding="utf-8" to open() in test_approved_flow_returns_0
- Add test_cleanup_on_write_failure to cover the except OSError block
  in _save_credentials

All files score 10.00/10 on pylint (fail-under=10) and 0 errors on
pyright strict.
2026-07-21 14:25:38 -04:00
didericis-codex 21b253c7eb test(login): satisfy type and coverage gates 2026-07-21 14:25:38 -04:00
didericis-claude d4e2bc5f93 fix(login): atomic credential write and respect server poll_interval
Write credentials via a 0600 temp file + os.replace() so the token file
never appears at its final path with world-readable permissions, even if
the process is interrupted between write and chmod.

Parse poll_interval from the authorization response (clamped to 1–60 s,
falling back to _POLL_SLEEP) so aggressive polling can't trigger console
rate limits.

Tests: add atomicity spy asserting the temp file is 0600 before replace;
patch time.sleep instead of _POLL_SLEEP; add explicit interval-passthrough
assertion.
2026-07-21 14:25:38 -04:00
didericis-claude 4998a5ec6a feat: add bb login command for console host registration
Starts a device-authorization flow against a bot-bottle console, polls
until the operator approves, then writes access + refresh tokens to
$BOT_BOTTLE_ROOT/console.json. Console URL is read from --console-url
flag or BB_CONSOLE_URL env var.

Part of didericis/bot-bottle-platform#1
2026-07-21 14:25:38 -04:00
didericis-codex efd413c1ba test(macos): cover cached image selection
tracker-policy-pr / check-pr (pull_request) Successful in 6s
test / integration-docker (pull_request) Successful in 13s
test / unit (pull_request) Successful in 38s
test / integration-firecracker (pull_request) Successful in 3m19s
test / coverage (pull_request) Successful in 39s
test / publish-infra (pull_request) Has been skipped
test / integration-docker (push) Successful in 13s
test / unit (push) Successful in 38s
lint / lint (push) Successful in 54s
Update Quality Badges / update-badges (push) Failing after 1m41s
test / integration-firecracker (push) Successful in 4m45s
test / coverage (push) Successful in 19s
test / publish-infra (push) Successful in 1m36s
2026-07-21 13:56:29 -04:00
didericis-claude d3d468532f fix(rebase): correct test_cached_lookup key and drop duplicate test
Two issues introduced by the rebase conflict resolution:

- test_cached_lookup_requires_ready_marker used _dockerfile_hash as the
  cache-dir key, but cached_agent_rootfs_dir now uses _rootfs_digest (which
  also folds in the guest init). Updated the test to match.
- test_pyproject_toml_change_bumps_version appeared twice in
  test_infra_artifact.py (once from main, once from the PR commit that
  added pyproject.toml support). Removed the duplicate.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-21 13:56:29 -04:00
didericis-claude ad2927b3b1 fix(infra-artifact): include pyproject.toml in rootfs version digest
Dockerfile.gateway COPYs pyproject.toml into /src and runs pip install
/src, so it is a real input to the baked rootfs. A dependency-only change
previously reused stale artifact versions, potentially booting a rootfs
whose installed packages differed from the current checkout.

Also adds _fake_repo fixture support and a regression test so this input
can't silently drop out of the hash again.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-21 13:56:29 -04:00
didericis-claude 2582373490 fix(pyright): assert not None before accessing datetime attributes in tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-21 13:56:29 -04:00
didericis-claude 17ac1be93b fix: return None from image_created_at when timestamp is absent
FROM-scratch images (commit_container output) and registry images built
for reproducibility often omit the created field entirely. Both backends
were calling die() in that case, crashing the cached-image quickstart
before any container started.

image_created_at now returns datetime | None — None when the field is
absent or unparseable, die() only on real inspect failures (non-zero
exit, malformed JSON). stale_checks in both backends skips the staleness
check when None is returned.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-21 13:56:29 -04:00
didericis-codex c53254e9d5 fix(firecracker): honor cached image policy 2026-07-21 13:56:29 -04:00
didericis-claude 58ecd8cb90 fix: correct host_db_path import and stale-check test expectations
- config_store.py imported host_db_path from supervise_types (wrong);
  it lives in paths.py, matching all other stores (audit, queue, etc.)
- test_stale_checks: two tests expected check_stale called twice
  (agent + sidecar) — consolidated arch has no sidecar, so once is
  correct; update assertions and remove unused Path import

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-21 13:56:29 -04:00
didericis-claude 310b36196d fix(lint): remove unused BottleImages imports from two test files
pyright strict reportUnusedImport flagged BottleImages in
test_docker_launch_committed_image.py and test_macos_container_launch.py;
neither file references the type by name (they only use the returned
value's attributes).
2026-07-21 13:56:29 -04:00
didericis-claude c473e5e5d8 fix: remove redundant quoted annotations in BottleImages and abstract methods
`from __future__ import annotations` already defers all annotation
evaluation, so quoting `str | Path`, `BottleImages` inside the same
module was redundant and tripped pyright strict mode.
2026-07-21 13:56:29 -04:00
didericis-claude 137426d9ac refactor: BottleImages dataclass, prelaunch_checks, build_or_load_images
Addresses review comments 3098, 3099, 3100 on PR #336:

- Add BottleImages(agent, sidecar) dataclass to backend/__init__.py.
  Docker/macOS backends use str image refs; smolmachines uses Path
  artifacts. Replaces the singular `image` variable from the canonical
  pattern in comment 3100.

- Replace _image_stale_checks/skip_stale with public prelaunch_checks().
  CLI now calls backend.prelaunch_checks(plan) before backend.launch(plan);
  if StaleImageError is raised and the operator confirms, launch proceeds
  without re-checking. Removes the while-True/skip_stale retry loop.

- Add abstract _build_or_load_images(plan) -> BottleImages to
  BottleBackend. launch() calls it then passes images to _launch_impl.
  Each backend implements both methods.

- Fix comment 3098 (macos-container): _build_images is removed.
  build_or_load_images() has separate fresh/cached code paths — the
  cached path never calls a build helper.

- Update _start_bundle (smolmachines) to accept sidecar_artifact: Path
  directly. Sidecar artifact resolution moves to _sidecar_from_path(),
  called by build_or_load_images alongside _agent_from_path().
2026-07-21 13:56:29 -04:00
didericis-claude 2a3a7dfb5c fix(tests): resolve pyright errors in stale-check test files
Remove unused imports, add missing type annotations, fix Die()
constructor calls (int code, not str), replace fake backend
subclass with patch.object approach to avoid reportMissingParameterType
and override-incompatibility errors in strict pyright mode.
2026-07-21 13:56:29 -04:00
didericis-claude 16c12177d2 test: add unit tests for stale-image checks to reach ≥90% diff-coverage
Covers StaleImageError / check_stale / check_stale_path, the
BottleBackend.launch template method (skip_stale flag), stale_checks
functions across docker / smolmachines / macos-container backends,
_build_images cached paths in the macOS backend, image_created_at edge
cases in both docker and container util modules, the CLI stale loop
(headless die, interactive decline, interactive confirm + retry), and
ConfigStore.cached_image_stale_warning_days fallback paths.

Diff-coverage: 488/507 changed lines covered (96.3%).
2026-07-21 13:56:29 -04:00
didericis-claude 0f1734b823 refactor(image-cache): replace warn_if_stale with StaleImageError; add launch template
- image_cache: StaleImageError exception + check_stale/check_stale_path (raise instead of warn)
- BottleBackend.launch: template method (skip_stale flag) that calls _image_stale_checks then _launch_impl
- Each backend: _image_stale_checks delegates to a stale_checks() function in its launch module; _launch_impl replaces launch override
- macos_container: adds image_created_at to util, cached-image support in _build_images, stale_checks
- cli/start.py: catches StaleImageError, prompts interactively, retries with skip_stale=True; headless mode dies on it
2026-07-21 13:56:29 -04:00
didericis-codex 2bf28e03f4 fix: make config store schema explicit 2026-07-21 13:56:29 -04:00
didericis-codex d3428b8c14 Add cached image quickstart 2026-07-21 13:56:29 -04:00
didericis-claude 4199de5e3e feat(claude): add forward_host_credentials support
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 1m41s
test / integration-firecracker (pull_request) Successful in 3m17s
test / coverage (pull_request) Successful in 21s
test / publish-infra (pull_request) Has been skipped
test / integration-docker (push) Successful in 19s
prd-number / assign-numbers (push) Failing after 23s
test / unit (push) Successful in 42s
Update Quality Badges / update-badges (push) Failing after 43s
lint / lint (push) Successful in 51s
test / integration-firecracker (push) Successful in 4m46s
test / coverage (push) Successful in 15s
test / publish-infra (push) Successful in 1m35s
Reads the host's Claude OAuth session key from ~/.claude.json at launch
and forwards it only to the egress sidecar (never to the agent), placing
a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent env so Claude Code
starts without seeing the real credential.

Mirrors the existing Codex forward_host_credentials flow (PRD 0029).
Adds claude_auth.py to extract and validate the sessionKey, a
CLAUDE_HOST_CREDENTIAL_TOKEN_REF constant in egress.py, and updates
manifest_agent.py to allow the flag for both 'codex' and 'claude'
templates. Also adds a mutual-exclusion check that rejects setting
both auth_token and forward_host_credentials together.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix(claude): read credentials from ~/.claude/.credentials.json

The actual OAuth token is in ~/.claude/.credentials.json under
claudeAiOauth.accessToken, not in ~/.claude.json.
~/.claude.json holds only UI state and profile metadata (oauthAccount
has no token fields). expiresAt in the credentials file is milliseconds,
not seconds.

Discovered after testing against Claude Code 2.1.198.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix(claude): fall back to macOS Keychain for credentials

On macOS, Claude Code stores credentials in the Keychain under
service "Claude Code-credentials" rather than in a file. When
~/.claude/.credentials.json is absent, shell out to:
  security find-generic-password -s "Claude Code-credentials" -w
and parse the result as the same JSON schema.

~/.claude.json holds only profile/UI metadata (oauthAccount has
no token fields). expiresAt in the credentials is milliseconds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs(prd): fix credential path references (~/.claude/.credentials.json)

fix(test): suppress gitleaks false positives on synthetic Claude tokens
2026-07-21 13:15:05 -04:00
didericis-codex 8348714e3e test(firecracker): cover published artifact reuse
tracker-policy-pr / check-pr (pull_request) Successful in 9s
test / integration-docker (pull_request) Successful in 16s
test / unit (pull_request) Successful in 1m37s
test / integration-firecracker (pull_request) Successful in 3m30s
test / coverage (pull_request) Successful in 15s
test / publish-infra (pull_request) Has been skipped
prd-number / assign-numbers (push) Failing after 23s
test / integration-docker (push) Successful in 15s
lint / lint (push) Successful in 58s
test / unit (push) Successful in 38s
Update Quality Badges / update-badges (push) Failing after 1m40s
test / integration-firecracker (push) Successful in 5m9s
test / coverage (push) Successful in 44s
test / publish-infra (push) Successful in 2m7s
2026-07-21 16:55:06 +00:00
didericis-claude 26002b75ca fix(ci): stage coverage data under non-dot names so upload-artifact uploads them
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 18s
lint / lint (push) Successful in 52s
test / unit (pull_request) Successful in 1m59s
test / integration-firecracker (pull_request) Successful in 3m23s
test / coverage (pull_request) Failing after 40s
test / publish-infra (pull_request) Has been skipped
upload-artifact@v3's glob silently skips hidden files, so uploading a
bare `.coverage.unit` logged "No files were found. No artifacts will be
uploaded" and registered nothing — the coverage job's download then 404'd
("List Artifacts failed: 404"). The coverage report step read the same
file fine, confirming it existed; only the leading dot broke the upload.
The old pipeline's cross-job artifacts (infra-candidate/, firecracker-
inputs) worked precisely because they were non-dotfiles.

Each test job now copies its .coverage.<suffix> to a non-dot
coverage-<suffix>.dat before upload (the cp also fails loudly if coverage
never wrote the file), and the coverage job renames them back to
.coverage.* before `coverage combine`.
2026-07-21 12:43:26 -04:00
didericis-claude 0c91c75a05 fix(ci): use COVERAGE_FILE env var for reliable artifact paths; add --reuse-published to infra build
coverage run's --data-file flag can be overridden or ignored in some
runner environments (Nix Python, older act-based runners). Switching to
the COVERAGE_FILE env var with an absolute ${{ github.workspace }} path
ensures coverage.py writes to a known location in every runner context,
so upload-artifact can find the file.

Also adds --reuse-published to the infra build step: if the artifact for
this content hash already exists in the registry, download it instead of
running the full docker build → mke2fs → gzip pipeline.
2026-07-21 12:43:26 -04:00
didericis-claude 8ce8a8cc62 fix(ci): use absolute github.workspace paths for coverage artifact upload/download
The delphi-ci runner resolves relative paths in upload-artifact and
download-artifact from a different CWD than run: shell steps, so
'.coverage.unit' etc. were never found. Using ${{ github.workspace }}
gives an absolute path that does not depend on the JS action's CWD.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-21 12:43:26 -04:00
didericis-claude 3fba385513 ci: artifact-based coverage and local Firecracker candidate flow
Each test job now runs once under coverage and uploads a small .coverage.*
artifact. The coverage job combines them on ubuntu-latest — no test reruns,
no KVM dependency. The infra candidate is built directly on the KVM runner,
eliminating the build-infra job and the ~70 s upload + ~83 s combined
download. For PRs, no rootfs artifact is transferred at all. Main-branch
pushes upload the tested rootfs and matching dropbear so publish-infra
publishes the byte-identical artifact. relative_files = True in .coveragerc
lets coverage files from different runners combine without path remapping.

Closes #446
2026-07-21 12:43:26 -04:00
didericis-claude b25cd72fc3 test(backend): cover poll_ca_cert timeout paths for diff-coverage gate
test / integration-docker (pull_request) Successful in 12s
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / unit (pull_request) Successful in 35s
test / stage-firecracker-inputs (pull_request) Successful in 2s
test / build-infra (pull_request) Successful in 3m37s
test / integration-firecracker (pull_request) Successful in 1m30s
test / coverage (pull_request) Successful in 1m32s
test / publish-infra (pull_request) Has been skipped
test / integration-docker (push) Successful in 13s
prd-number / assign-numbers (push) Failing after 19s
lint / lint (push) Successful in 43s
Update Quality Badges / update-badges (push) Failing after 41s
test / unit (push) Successful in 1m42s
test / stage-firecracker-inputs (push) Successful in 2s
test / build-infra (push) Failing after 7s
test / integration-firecracker (push) Has been skipped
test / coverage (push) Has been skipped
test / publish-infra (push) Has been skipped
Add tests for the three uncovered paths introduced by the poll_ca_cert
extraction: the timeout + sleep branches in backend/util, and the
TimeoutError → GatewayError and TimeoutError → die() conversions in the
macOS and Firecracker callers.
2026-07-21 04:54:40 +00:00
didericis-claude 8e43c26ab4 fix(backend): extract poll_ca_cert helper and fix PRD port docs
Extract the shared CA cert polling loop into `backend/util.poll_ca_cert`
(firecracker and macos backends were duplicating deadline/sleep/raise logic).
Each caller now wraps a fetch lambda and converts TimeoutError to its own
error type. Also corrects the PRD port publication line from {port}:{port}
to {host_port}:8099.
2026-07-21 04:54:40 +00:00
didericis-claude 14ff4fe186 fix(lifecycle): build gateway before infra and fix orchestrator port mapping
- `_build_images()` now builds Dockerfile.gateway → Dockerfile.orchestrator
  → Dockerfile.infra in order; Dockerfile.infra starts FROM bot-bottle-gateway
  so the base must exist on clean hosts.
- Publish mapping corrected from `self.port:self.port` to `self.port:DEFAULT_PORT`
  (8099) — gateway_init hardcodes the orchestrator on port 8099 inside the
  container, so the host-side published port must map to that fixed internal port.
- BOT_BOTTLE_ORCHESTRATOR_URL inside the container now always points to
  127.0.0.1:8099, not self.port, since gateway daemons reach the orchestrator
  over loopback at the fixed internal port.
- Update test_ensure_running_builds_both_images → _all_images for the new
  three-step build sequence; add test_publish_maps_host_port_to_fixed_internal_port
  to lock in the port-mapping fix.

Addresses the P1 findings from the didericis-codex review on PR #432.
2026-07-21 04:54:40 +00:00
didericis-claude cae1215f63 test(lifecycle): cover edge paths to satisfy diff-coverage gate
Add three new tests:
- noop when healthy but docker inspect fails (returns True → don't churn)
- build failure raises GatewayError
- _ensure_network creates the network when it doesn't exist

Also update the integration test to use new OrchestratorService API
(infra_name/image instead of orchestrator_name/gateway_name/gateway_image).

Brings diff-coverage from 86% to 90.3% against origin/main.
2026-07-21 04:54:40 +00:00
didericis-claude 28766d7733 fix(pyright): resolve type errors introduced by lifecycle refactor
- Remove unused INFRA_IMAGE import from test_orchestrator_lifecycle
- Update integration test to use new single-container OrchestratorService
  API (infra_name/image replaces orchestrator_name/gateway_name/gateway_image)
- Move type: ignore to the lambda line in gateway_init SIGHUP handler
- Break two long lines in test_orchestrator_lifecycle
2026-07-21 04:54:40 +00:00
didericis-claude 819f967844 fix(lint): resolve pylint findings in gateway_init
- Extract _sigkill_all() to cut nesting depth below the 5-block limit
- Add pylint: disable=consider-using-with on Popen (process must outlive caller)
- Break long SIGHUP signal line to stay within 100 chars
2026-07-21 04:54:40 +00:00
didericis-claude 2f45f5afec feat(docker): consolidate to single infra container under gateway_init supervise tree
Collapses the two-container Docker model (gateway + orchestrator) into one
bot-bottle-infra container, matching the macOS and Firecracker backends.

- Dockerfile.infra: now a shared gateway+orchestrator base (COPY bot_bottle
  from orchestrator build, no CMD override)
- Dockerfile.infra.fc: new Firecracker-specific layer (buildah/crun/netavark)
- gateway_init: adds orchestrator daemon with _OPT_IN_DAEMONS gating so it
  only starts when BOT_BOTTLE_GATEWAY_DAEMONS explicitly includes it
- orchestrator/lifecycle: OrchestratorService manages one infra container;
  builds orchestrator (intermediate) then infra; live source bind-mounted at
  /bot-bottle-src with PYTHONPATH so the subprocess uses the checkout
- backend/consolidated_util: extracts provision_bottle + teardown_consolidated
  shared across all three backends; removes duplication in docker/fc/macos
  consolidated_launch modules
- firecracker/infra_vm: builds four images (orchestrator→gateway→infra→infra.fc)
- All unit tests updated and passing (1878 tests)
- PRD status: Draft → Active
2026-07-21 04:54:40 +00:00
didericis-claude 31a5ec2fc8 docs(prd): consolidate infra backend for docker
Adds PRD for collapsing the Docker backend from two containers
(gateway + orchestrator) to a single bot-bottle-infra container,
restructuring Dockerfile.infra as the shared base, and extracting
duplicated CA polling / teardown / provision helpers into a shared
backend utility module.

Closes #431
2026-07-21 04:49:18 +00:00
didericis-claude 1f192d785a refactor(config_store): replace generic key/value table with explicit schema
test / stage-firecracker-inputs (push) Successful in 2s
test / integration-docker (push) Successful in 17s
Update Quality Badges / update-badges (push) Failing after 42s
lint / lint (push) Successful in 47s
test / unit (push) Successful in 1m41s
test / build-infra (push) Successful in 3m43s
test / integration-firecracker (push) Successful in 1m28s
test / coverage (push) Successful in 1m29s
test / publish-infra (push) Successful in 1m49s
Switch orchestrator_config from a generic key/text store to a typed
single-row table with a dedicated teardown_timeout_seconds REAL column.
The CHECK (id = 1) constraint enforces at most one config row.

Remove TEARDOWN_TIMEOUT_KEY constant; update OrchestratorConfigStore to
expose get/set/delete_teardown_timeout_seconds() instead of the generic
get/set/delete(key). Resolving the DB value in resolve_teardown_timeout()
no longer needs a float() conversion since the column is already REAL.
2026-07-21 00:40:59 -04:00
didericis-codex 853b6d1678 test(orchestrator): satisfy config store type checking 2026-07-21 00:40:59 -04:00
didericis-claude cf9a53d582 fix: resolve teardown timeout before registration to prevent orphaned state
If resolve_teardown_timeout() raised after launch_consolidated()/register_agent()
returned, the teardown callback was never registered, leaving the bottle
registered with no cleanup path. Resolve the timeout into a local variable
before the registration call so any failure aborts before the bottle exists.

Add tests covering OrchestratorConfigStore, resolve_teardown_timeout priority
ordering (env > db > default), and the source-order invariant that the resolver
runs before registration in all three backends.
2026-07-21 00:40:59 -04:00
didericis-claude 0b36c3eb48 fix: make orchestrator teardown timeout configurable (closes #435)
Resolves via ENV VAR -> orchestrator DB config -> default (30 s, up from 5 s):
  BOT_BOTTLE_ORCHESTRATOR_TEARDOWN_TIMEOUT_SECONDS
  teardown_timeout_seconds key in new orchestrator_config table (bot-bottle.db)

New OrchestratorConfigStore (same DbStore/TableMigrations pattern as the
registry) stores the DB-level setting. resolve_teardown_timeout() implements
the priority chain and is called at stack.callback registration time in all
three backends (macos_container, docker, firecracker).
2026-07-21 00:40:59 -04:00
didericis-claude 28fcc3f2d2 test: cover inspect_container_network_ip parser directly
test / stage-firecracker-inputs (pull_request) Successful in 5s
test / unit (pull_request) Successful in 35s
test / integration-docker (pull_request) Successful in 9s
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / build-infra (pull_request) Successful in 3m48s
test / integration-firecracker (pull_request) Successful in 1m43s
test / coverage (pull_request) Successful in 1m53s
test / publish-infra (pull_request) Has been skipped
test / integration-docker (push) Successful in 11s
test / unit (push) Successful in 35s
test / stage-firecracker-inputs (push) Successful in 2s
Update Quality Badges / update-badges (push) Failing after 40s
lint / lint (push) Successful in 2m43s
test / build-infra (push) Successful in 3m25s
test / integration-firecracker (push) Successful in 1m45s
test / coverage (push) Successful in 1m50s
test / publish-infra (push) Successful in 2m7s
The None/empty-string distinction is the invariant that prevents
live_source_ips from treating a failed inspect as a legitimately
no-address container. Add TestInspectContainerNetworkIp to util tests
covering: IP found, CIDR stripping, no address yet, absent network
list, non-zero exit, malformed JSON, and unexpected JSON shape.
2026-07-21 03:49:19 +00:00
didericis-claude 571030b8e8 fix: require authoritative container snapshot before reconciling
test / stage-firecracker-inputs (pull_request) Successful in 5s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 33s
test / unit (pull_request) Successful in 39s
lint / lint (push) Successful in 49s
test / build-infra (pull_request) Successful in 3m23s
test / integration-firecracker (pull_request) Successful in 1m42s
test / coverage (pull_request) Failing after 1m26s
test / publish-infra (pull_request) Has been skipped
A failed container list or a partial per-container inspect were
previously indistinguishable from a legitimately empty/partial live
set, so reconciliation could silently unregister healthy bottles.

- enumerate_active() now raises EnumerationError instead of returning []
  when `container list` fails
- Add inspect_container_network_ip() to util, which returns None on
  inspect failure (vs "" for "no DHCP address yet"), so live_source_ips
  can tell the two apart
- live_source_ips() raises EnumerationError on either failure mode;
  register_agent() catches it and skips reconciliation, same as
  OrchestratorClientError
- Update tests: rename test_empty_when_the_cli_fails to
  test_raises_when_the_cli_fails, update patching to the new function,
  add coverage for list failure, per-container inspect failure, and the
  launch-not-blocked path
2026-07-21 03:39:18 +00:00
didericis-codex 288b205a44 test(macos): isolate registration tests from container discovery
test / stage-firecracker-inputs (pull_request) Successful in 2s
test / integration-docker (pull_request) Successful in 6s
tracker-policy-pr / check-pr (pull_request) Successful in 5s
test / unit (pull_request) Successful in 30s
lint / lint (push) Successful in 42s
test / build-infra (pull_request) Successful in 3m18s
test / integration-firecracker (pull_request) Successful in 1m27s
test / coverage (pull_request) Successful in 1m51s
test / publish-infra (pull_request) Has been skipped
2026-07-21 03:27:16 +00:00
didericis 0c1d27b605 fix(egress): the unattributed message must name the token-mismatch cause too
`/resolve` fail-closes on a missing/ambiguous registry row *and* on a
request whose identity token doesn't match. The message named only the
first, so a bottle that was registered correctly but sent no token read as
"not registered" and sent the reader looking for a deregistered bottle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 03:27:16 +00:00
didericis 69361114d1 fix(egress): name the real fault when a deny-all is not an allowlist miss
An unattributed bottle, an unreachable orchestrator, and an unparseable
policy all become a deny-all Config, and a deny-all is indistinguishable
from "policy loaded, host not allowed" at the decision point — both are just
"no matching route". So every one of them reported `host X is not in the
bottle's egress.routes allowlist`, which reads as a config problem and sends
the operator hunting for a route that was never missing. Diagnosing a
bricked registration cost hours for exactly this reason.

Carry the structural reason on Config and prefer it in decide(). A genuine
allowlist miss — a policy that loaded and simply lacks the host — keeps the
original wording, so the message now tells the two cases apart.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 03:27:16 +00:00
didericis e4d53fd360 fix(orchestrator): reap registry rows whose bottle is no longer running
A registry row only ever left the registry two ways: an explicit
teardown_bottle (the launcher's cleanup callback) or the same-IP supersede
sweep in register(). Neither runs when the launching CLI dies hard, so the
row outlives its container.

That orphan is not inert. Source IPs are recycled by the backend's DHCP and
by_source_ip fail-closes on ambiguity, so a leftover row at a reused address
resolves no policy at all for the next bottle that lands there — and a
bottle with no policy denies every host, which surfaces to the agent as
"host X is not in the allowlist" for hosts that were never the problem.

Add reap_absent/reconcile and call it from the macOS launch path before
registering, so each launch self-heals the registry. Restores the invariant
the data plane needs: at most one active row per live address, and none for
a dead one. The second half matters as much as the first — when several rows
claim a *live* address the newest wins and the rest are swept, otherwise a
recycled address stays ambiguous, which is exactly the bricked state.

The host supplies the live set because the orchestrator runs inside the
infra container and cannot see the backend. A grace window exempts rows
younger than it, so reconciliation cannot race a bottle still coming up, and
a reconcile failure is logged rather than blocking an otherwise-fine launch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 03:27:16 +00:00
didericis-codex 5e01c28016 test(macos): cover root container exec helper
test / integration-docker (pull_request) Successful in 11s
tracker-policy-pr / check-pr (pull_request) Successful in 20s
test / unit (pull_request) Successful in 32s
test / stage-firecracker-inputs (pull_request) Successful in 3s
test / build-infra (pull_request) Successful in 8m31s
test / integration-firecracker (pull_request) Successful in 1m32s
test / coverage (pull_request) Successful in 1m27s
test / publish-infra (pull_request) Has been skipped
test / stage-firecracker-inputs (push) Successful in 1s
test / integration-docker (push) Successful in 7s
test / unit (push) Successful in 29s
Update Quality Badges / update-badges (push) Failing after 33s
lint / lint (push) Successful in 42s
test / build-infra (push) Successful in 8m37s
test / integration-firecracker (push) Successful in 1m27s
test / coverage (push) Successful in 1m27s
test / publish-infra (push) Successful in 1m48s
2026-07-21 03:12:17 +00:00
didericis 2f8539c2c7 fix(macos): name the gateway instead of addressing it, so bottles survive it moving
test / integration-docker (pull_request) Successful in 14s
test / unit (pull_request) Successful in 38s
tracker-policy-pr / check-pr (pull_request) Successful in 25s
lint / lint (push) Successful in 49s
test / stage-firecracker-inputs (pull_request) Successful in 5s
test / build-infra (pull_request) Successful in 3m31s
test / integration-firecracker (pull_request) Successful in 1m51s
test / coverage (pull_request) Failing after 1m33s
test / publish-infra (pull_request) Has been skipped
The shared gateway's address is DHCP-assigned and changes whenever the infra
container is recreated — a source-hash bump, an image upgrade, a crash. Every
agent-facing URL embedded that address, and the proxy URL reaches the agent
as process environment at `container exec` time. A running process's environ
cannot be rewritten from outside, so a moved gateway stranded every running
bottle permanently: not degraded, unreachable, until relaunched and its agent
session thrown away.

Give the agent a stable name instead. `GATEWAY_HOSTNAME` replaces the address
in the egress proxy URL, NO_PROXY, git-http, and supervise URLs, and resolves
through the bottle's own /etc/hosts. Unlike environ that is a file, so it can
be rewritten inside a container that is already running — which is the whole
point: a gateway that returns at a new address is picked up by live bottles.

Launch writes the entry before anything execs (every agent URL names the
gateway, so it must resolve for the first connection), and re-points every
running bottle once the gateway is up, so one stranded by an earlier restart
re-attaches instead of needing a relaunch.

The write needs root and the agent runs as `node`: the host can repoint a
bottle's gateway name, the agent cannot repoint its own. Keep that asymmetry.

Apple Container 1.0 has no container-name DNS on a user network and
`container run` has no --add-host, so the entry is written by exec after
start rather than declared at run.

Does not address the other half of #443: per-bottle egress auth tokens are
held in memory by the orchestrator and are still lost across a restart, so a
re-attached bottle resolves its policy but not its injected credentials.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 23:08:59 -04:00
didericis ad100b8a84 fix(macos): supply proxy env only at exec, so Codex keeps its identity token
test / stage-firecracker-inputs (push) Successful in 4s
test / integration-docker (push) Successful in 17s
test / unit (push) Successful in 34s
lint / lint (push) Successful in 46s
Update Quality Badges / update-badges (push) Failing after 35s
test / build-infra (push) Successful in 9m18s
test / integration-firecracker (push) Successful in 1m46s
test / coverage (push) Successful in 2m3s
test / publish-infra (push) Successful in 2m31s
`container exec --env` does not override the run-time environment on Apple
Container — it appends. The launch path baked a token-less `*_PROXY` into
`container run` and relied on the exec-time, token-bearing value superseding
it, so the agent's `environ` ended up with two `HTTPS_PROXY` entries,
token-less first.

Which entry a runtime reads is then luck. Node reads the last, so Claude
bottles picked up the token and worked. Rust's `std::env::var` reads the
first, so Codex proxied with no identity token at all; `/resolve` requires a
matching (source_ip, identity_token) pair and fail-closes, so every request
from a Codex bottle was denied — its model calls, its `wss://` reconnects,
and its MCP servers alike. The registry row was correct the whole time,
which is what made this read as a registration bug.

Drop the run-time proxy vars entirely. A token-less proxy URL has no
legitimate consumer: the init process is `sleep` and everything that
egresses arrives by exec. Its only benefit was a tidy 403 for unattributed
callers, which is not worth silently dropping attribution for — and a
process that egresses before the exec-time env still fails closed, since the
agent network is host-only and the gateway is the only route off it.

Also corrects the two comments that asserted the false "exec --env wins"
invariant, so the next reader doesn't rebuild the same assumption.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 22:54:52 -04:00
didericis-claude c7375051fd fix(lint): mark read_tty_line as an explicit re-export in _common.py
test / coverage (push) Successful in 2m6s
test / publish-infra (push) Successful in 2m38s
test / build-infra (push) Successful in 3m49s
test / integration-firecracker (push) Successful in 2m14s
test / stage-firecracker-inputs (pull_request) Successful in 2s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 27s
test / unit (pull_request) Successful in 34s
test / build-infra (pull_request) Successful in 3m39s
test / integration-firecracker (pull_request) Successful in 1m33s
test / coverage (pull_request) Successful in 2m2s
test / publish-infra (pull_request) Has been skipped
test / stage-firecracker-inputs (push) Successful in 3s
test / unit (push) Successful in 34s
test / integration-docker (push) Successful in 32s
lint / lint (push) Successful in 43s
Update Quality Badges / update-badges (push) Failing after 39s
Pyright reportUnusedImport fired because the bare import is not used
within _common.py itself. The X as X idiom signals an intentional
public re-export without requiring __all__.
2026-07-20 20:11:04 +00:00
didericis-claude d9e685e860 fix(backend): don't prompt in headless/non-interactive backend selection
lint / lint (push) Failing after 45s
test / unit (pull_request) Successful in 1m34s
test / integration-docker (pull_request) Successful in 12s
tracker-policy-pr / check-pr (pull_request) Successful in 20s
test / stage-firecracker-inputs (pull_request) Successful in 3s
test / build-infra (pull_request) Successful in 3m21s
test / integration-firecracker (pull_request) Successful in 1m30s
test / coverage (pull_request) Successful in 1m30s
test / publish-infra (pull_request) Has been skipped
_auto_select_backend gains a prompt parameter (default True). When
prompt=False the docker-fallback [i/d/q] menu is skipped and the call
dies immediately with an actionable message ("set
BOT_BOTTLE_BACKEND=docker or install a VM backend"), preventing hangs
in CI, webhook dispatch, and orchestrator launches.

prepare_with_preflight passes prompt=not spec.headless so the headless
start path can never block waiting for TTY input it cannot receive.
2026-07-20 19:58:14 +00:00
didericis-claude b4b73a8acc refactor: consolidate read_tty_line into bot_bottle/util.py
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / integration-docker (pull_request) Successful in 26s
test / unit (pull_request) Successful in 36s
lint / lint (push) Failing after 42s
test / stage-firecracker-inputs (pull_request) Successful in 2s
test / build-infra (pull_request) Successful in 3m57s
test / integration-firecracker (pull_request) Successful in 1m55s
test / coverage (pull_request) Successful in 2m14s
test / publish-infra (pull_request) Has been skipped
Remove the private _read_tty_line duplicate from backend/__init__.py and
the local definition from cli/_common.py. Both now import from the
shared bot_bottle.util module.
2026-07-20 19:44:32 +00:00
didericis-claude b1ebc6f1b8 test: cover _read_tty_line, macos install branch, and reply==i path
test / stage-firecracker-inputs (pull_request) Successful in 3s
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 14s
lint / lint (push) Successful in 51s
test / unit (pull_request) Successful in 1m29s
test / build-infra (pull_request) Successful in 3m30s
test / integration-firecracker (pull_request) Successful in 1m34s
test / coverage (pull_request) Successful in 1m30s
test / publish-infra (pull_request) Has been skipped
2026-07-20 19:26:00 +00:00
didericis-claude 8b5b5730ae fix: remove CLI backend assumptions and add docker fallback prompt
test / stage-firecracker-inputs (pull_request) Successful in 3s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
lint / lint (push) Successful in 42s
test / integration-docker (pull_request) Successful in 33s
test / unit (pull_request) Successful in 37s
test / build-infra (pull_request) Successful in 3m47s
test / integration-firecracker (pull_request) Successful in 1m39s
test / coverage (pull_request) Failing after 1m27s
test / publish-infra (pull_request) Has been skipped
- Remove hardcoded --backend=macos-container flag reference in
  firecracker/util.py require_firecracker() error message
- Remove --backend flag from cli.py start; backend selection now
  driven exclusively by BOT_BOTTLE_BACKEND env var or auto-selection
- Skip unavailable backends in cli.py cleanup (fixes crash on Linux
  when macos-container.prepare_cleanup calls require_container())
- Add two-tier auto-selection: VM backend first (macos-container on
  macOS, firecracker on Linux+KVM); fall back to docker with a
  security warning and interactive i/d/q prompt; exit if docker
  also unavailable and print VM install instructions

Closes #344
2026-07-20 19:13:08 +00:00
didericis 44479f328e fix(macos-container): launch gateway as a module, not /app/gateway_init.py
test / stage-firecracker-inputs (pull_request) Successful in 5s
test / integration-docker (pull_request) Successful in 8s
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / unit (pull_request) Successful in 30s
test / build-infra (pull_request) Successful in 3m56s
test / integration-firecracker (pull_request) Successful in 1m36s
test / coverage (pull_request) Successful in 2m33s
test / publish-infra (pull_request) Has been skipped
test / stage-firecracker-inputs (push) Successful in 2s
test / integration-docker (push) Successful in 33s
test / unit (push) Successful in 34s
lint / lint (push) Successful in 48s
Update Quality Badges / update-badges (push) Failing after 44s
test / build-infra (push) Successful in 3m48s
test / integration-firecracker (push) Successful in 1m44s
test / coverage (push) Successful in 1m36s
test / publish-infra (push) Successful in 2m33s
The macOS infra container's init script still spawned the supervisor as
`python3 /app/gateway_init.py`. 5ad3449 moved bot_bottle from flat files
under /app into a pip-installed package, so that file no longer exists:
the supervisor never started, mitmdump never generated its CA, and the
launch failed downstream with a misleading "gateway CA not available"
error pointing at TLS rather than at the supervisor.

This is the same defect 127ba49 fixed for the firecracker backend. It
survived on macOS because CI has no Apple Container runner — the KVM
runner added in c193b04 covers firecracker, and ubuntu-latest covers
docker, but nothing exercises macos_container.

test_macos_infra asserted the substring "gateway_init.py", which the
broken path satisfied; it now asserts the module form, matching the
assertion 127ba49 already made in test_firecracker_infra_vm.

Verified end to end on the macos-container backend: image builds, CA is
generated, agent registers, and a headless bottle runs a prompt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 14:37:09 -04:00
didericis 2de223a33b test(gateway-image): probe package imports, not pre-refactor flat modules
test / stage-firecracker-inputs (pull_request) Successful in 3s
test / integration-docker (pull_request) Successful in 10s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / unit (pull_request) Successful in 1m31s
test / build-infra (pull_request) Successful in 3m45s
test / integration-firecracker (pull_request) Successful in 1m34s
test / coverage (pull_request) Successful in 1m59s
test / publish-infra (pull_request) Has been skipped
test / stage-firecracker-inputs (push) Successful in 3s
test / integration-docker (push) Successful in 16s
Update Quality Badges / update-badges (push) Failing after 42s
lint / lint (push) Successful in 53s
test / unit (push) Successful in 1m35s
test / build-infra (push) Successful in 3m30s
test / integration-firecracker (push) Successful in 1m46s
test / coverage (push) Successful in 2m1s
test / publish-infra (push) Successful in 2m14s
test_python_imports_supervise_module probed the image with
`import supervise; import supervise_server`, the flat sibling modules that
5ad3449 replaced. They now live in the installed package, so the test fails
with ModuleNotFoundError when actually run.

It went unnoticed because the whole class carries a GITEA_ACTIONS skip — it
is interactive-only, so CI never executes it.

All 4 tests in the class now pass against a freshly built gateway image.

Closes #429.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 13:40:14 -04:00
didericis-codex af1690ab22 docs(research): consolidate agent sandbox landscape 2026-07-20 14:36:40 +00:00
didericis-claude 09debcf4f0 docs(research): add Agent Safehouse HN launch and Apple Container 1.0 stable
- agent-sandbox-landscape.md: update agent-safehouse star count (~1.4k →
  ~1.8k), add HN thread #47301085 link and key discussion notes (creator
  framing, Simon Willison observation, top community quote on sandboxing
  being THE major challenge), note Apple Container 1.0 stable (Jun 9 2026)
- hn-agent-safety-discourse-july-2026.md: add Agent Safehouse March 12
  launch to the sandboxing boom section as context that prefigures the
  June–July shift in community tone

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-20 13:51:52 +00:00
didericis fa11ad9a4a fix(ci,infra): repair post-merge publish + harden candidate handling
test / unit (push) Successful in 32s
test / stage-firecracker-inputs (pull_request) Successful in 2s
test / integration-docker (pull_request) Successful in 14s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / unit (pull_request) Successful in 34s
test / build-infra (pull_request) Successful in 3m35s
test / integration-firecracker (pull_request) Successful in 1m33s
test / coverage (pull_request) Successful in 1m39s
test / publish-infra (pull_request) Has been skipped
test / stage-firecracker-inputs (push) Successful in 2s
test / integration-docker (push) Successful in 32s
lint / lint (push) Successful in 43s
test / build-infra (push) Successful in 3m37s
test / integration-firecracker (push) Successful in 1m34s
test / coverage (push) Successful in 1m33s
test / publish-infra (push) Failing after 39s
Update Quality Badges / update-badges (push) Failing after 1m33s
Review follow-ups for the infra candidate-artifact flow:

1. publish-infra would fail on the first merge to main. The rootfs version now
   hashes the dropbear bytes, and build-infra sets BOT_BOTTLE_FC_DROPBEAR to the
   staged dropbear — but publish-infra (ubuntu-latest) set none, so _publish_
   bundle re-derived the version with a "<missing>" dropbear and rejected the
   candidate as "does not match checkout". Download the same firecracker-inputs
   dropbear and export BOT_BOTTLE_FC_DROPBEAR in publish-infra (and add
   stage-firecracker-inputs to its needs, since it now consumes that artifact).

2. Guard stage-firecracker-inputs with the same fork-PR check as the other
   KVM-runner jobs, so a fork PR can't spin the privileged runner (it only
   copies a static binary, but keep the posture consistent; it gates the whole
   Firecracker chain via needs).

3. ensure_artifact_gz / _publish_bundle read version.txt before checking it
   exists — a bundle missing it raised a raw FileNotFoundError instead of the
   intended "bundle is incomplete" die. Guard the read (kept before the gz/sha
   completeness check so a wrong-version bundle still reports the version
   mismatch, not "incomplete").

4. test_infra_artifact.setUp didn't isolate BOT_BOTTLE_INFRA_ARTIFACT_DIR, so
   an ambient candidate dir (the coverage job exports one) would send the
   registry-pull tests down the local-bundle path. Pin it off in setUp; the
   candidate-path cases set it explicitly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-20 02:07:08 -04:00
didericis-codex ad6471af12 test(infra): cover candidate release failure paths
test / stage-firecracker-inputs (pull_request) Successful in 3s
test / integration-docker (pull_request) Successful in 12s
tracker-policy-pr / check-pr (pull_request) Successful in 25s
test / unit (pull_request) Successful in 32s
lint / lint (push) Successful in 45s
test / build-infra (pull_request) Successful in 3m59s
test / integration-firecracker (pull_request) Successful in 1m36s
test / coverage (pull_request) Successful in 1m45s
test / publish-infra (pull_request) Has been skipped
2026-07-19 22:42:54 +00:00
didericis-codex 137df6f853 fix(coverage): scope infra candidate to integration tests
test / stage-firecracker-inputs (pull_request) Successful in 4s
test / integration-docker (pull_request) Successful in 12s
test / unit (pull_request) Successful in 34s
test / build-infra (pull_request) Successful in 3m34s
test / integration-firecracker (pull_request) Successful in 1m46s
test / coverage (pull_request) Failing after 1m46s
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 11m25s
2026-07-19 22:33:38 +00:00
didericis-codex 4252ca3562 fix(ci): stage complete reproducible infra inputs
test / stage-firecracker-inputs (pull_request) Successful in 14s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 22s
test / unit (pull_request) Successful in 34s
lint / lint (push) Successful in 43s
test / build-infra (pull_request) Successful in 3m32s
test / integration-firecracker (pull_request) Successful in 1m41s
test / coverage (pull_request) Failing after 57s
test / publish-infra (pull_request) Has been skipped
2026-07-19 22:25:45 +00:00
didericis-codex 701f5bf5e3 ci(infra): test and publish one candidate artifact
tracker-policy-pr / check-pr (pull_request) Successful in 20s
test / integration-docker (pull_request) Successful in 22s
lint / lint (push) Failing after 46s
test / build-infra (pull_request) Failing after 49s
test / integration-firecracker (pull_request) Has been skipped
test / coverage (pull_request) Has been skipped
test / unit (pull_request) Successful in 1m28s
test / publish-infra (pull_request) Has been skipped
2026-07-19 22:22:42 +00:00
didericis d589c08d9d test(firecracker): pin INFRA_BUILD off in the default-pull test
test / integration-docker (pull_request) Successful in 17s
tracker-policy-pr / check-pr (pull_request) Successful in 15s
test / integration-firecracker (pull_request) Successful in 42s
lint / lint (push) Successful in 51s
test / coverage (pull_request) Successful in 1m7s
test / unit (pull_request) Successful in 1m31s
The coverage CI job exports BOT_BOTTLE_INFRA_BUILD=local so the integration
suite builds infra/agent images from source. But that job also runs the unit
suite, and the ambient =local leaked into test_default_pulls_artifact_without_
docker — it drove ensure_built() down the local Docker-build branch, so the
"build_image not called / artifact pulled" assertion failed (build_image
called 3 times).

Pin the var off in that test so it exercises the default pull path regardless
of the runner's ambient env (mirrors test_local_mode_builds_deps_before_infra,
which pins it on). Full unit suite verified green under BOT_BOTTLE_INFRA_BUILD
=local.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 18:03:11 -04:00
didericis-claude 559dc03bb5 ci(infra): build images locally in test jobs; publish post-merge
tracker-policy-pr / check-pr (pull_request) Successful in 16s
test / integration-docker (pull_request) Successful in 33s
test / coverage (pull_request) Failing after 34s
test / unit (pull_request) Successful in 37s
test / integration-firecracker (pull_request) Successful in 44s
The test jobs were pulling a pre-published infra rootfs artifact from the
registry, which made CI dependent on an out-of-band publish step. Test suites
should be self-contained.

- Set BOT_BOTTLE_INFRA_BUILD=local in integration-firecracker and coverage
  so both jobs build agent and infra images from the checked-out source.
- Add publish-infra.yml: a separate workflow that fires after the test
  workflow succeeds on main, builds the same rootfs, and uploads it to the
  Gitea generic-package registry for operators who want the pull path.
- Registry credentials (BOT_BOTTLE_INFRA_ARTIFACT_TOKEN) are only needed
  by the post-merge publish job, never by test jobs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 21:43:39 +00:00
didericis-claude 9172bf3a42 fix(infra-artifact): include pyproject.toml in rootfs version digest
test / integration-firecracker (pull_request) Failing after 13s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / integration-docker (pull_request) Successful in 17s
test / coverage (pull_request) Failing after 34s
lint / lint (push) Successful in 43s
test / unit (pull_request) Successful in 1m28s
Dockerfile.gateway COPYs pyproject.toml into /src and runs pip install
/src, so it is a real input to the baked rootfs. A dependency-only change
previously reused stale artifact versions, potentially booting a rootfs
whose installed packages differed from the current checkout.

Also adds _fake_repo fixture support and a regression test so this input
can't silently drop out of the hash again.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 16:58:13 +00:00
didericis 0adbf25977 test(firecracker): satisfy pyright on the new infra-VM tests
tracker-policy-pr / check-pr (pull_request) Successful in 18s
test / integration-docker (pull_request) Successful in 30s
test / unit (pull_request) Successful in 42s
lint / lint (push) Successful in 45s
test / integration-firecracker (pull_request) Successful in 44s
test / coverage (pull_request) Successful in 1m8s
Two lint fixes, both test-only (no effect on the infra artifact version):
- annotate the TestAdoptable / TestKillInfraFirecrackers helper params
  (reportMissingParameterType).
- test_docker_test_helpers: read __unittest_skip__ via getattr on the
  dynamically-built Case type, matching the sibling assertion — pyright can't
  see the attribute the skip decorator adds (reportAttributeAccessIssue).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 01:31:06 -04:00
didericis d1aec706e3 fix(firecracker): fold guest init into the agent-rootfs cache key
tracker-policy-pr / check-pr (pull_request) Successful in 18s
test / integration-docker (pull_request) Successful in 35s
test / unit (pull_request) Successful in 40s
lint / lint (push) Failing after 45s
test / integration-firecracker (pull_request) Successful in 2m50s
test / coverage (pull_request) Successful in 2m52s
build_agent_rootfs_dir cached the built rootfs by Dockerfile content alone,
but util.inject_guest_boot then writes util._GUEST_INIT into it. So a fix to
the init — making /tmp world-writable (1777) so the agent can create scratch
dirs / git worktrees there — did NOT bust the cache: the KVM runner kept
reusing a stale agent-<dockerfilehash> rootfs built with the old init, and the
sandbox-escape README-push test kept failing at `git init /tmp/...` with
"Permission denied".

Key the cache on Dockerfile content AND the injected init (_rootfs_digest), so
an init change rebuilds. Self-busting: the new key yields a fresh cache dir, so
no manual cache clear on the runner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 01:21:23 -04:00
didericis a589604aa0 test(firecracker): derive netpool names from config, not hardcoded defaults
The KVM CI runner now sets BOT_BOTTLE_FC_* for its isolated pool (distinct
iface prefix / orch iface / nft table / IP base), and that env leaks into the
coverage job's test process. Five netpool tests hardcoded the default names
(bbfc*, bot_bottle_fc) and so failed there with e.g. ['bbfc1'] != ['bbci1'].

Assert against netpool's env-driven config instead — slot(i).iface / the
configured prefix — so the tests check the LOGIC regardless of which pool the
host is configured for. The single-source test now compares the parsed
defaults (netpool._DEFAULTS, env-independent) for the module constants, since
IFACE_PREFIX/NFT_TABLE legitimately layer an env override on top.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 01:21:23 -04:00
didericis 4c01e31e96 fix(firecracker): version-aware infra-VM adoption + robust teardown
test / integration-docker (pull_request) Successful in 15s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / coverage (pull_request) Failing after 35s
test / unit (pull_request) Successful in 40s
test / integration-firecracker (pull_request) Failing after 1m46s
lint / lint (push) Failing after 2m33s
The infra VM is a per-host singleton that outlives short-lived launchers, so
`ensure_running` adopts it when its control plane is healthy. But it adopted
ANY healthy VM regardless of the code that built it — so after an infra-code
change the old VM kept being adopted and the new code never booted. The only
way to dislodge it was an out-of-band `kill`, which then raced whatever
launched next. On CI this meant every infra change needed a manual VM kill.

Make adoption version-aware:

  * boot records the infra-artifact version it booted from in a `booted-version`
    marker beside the singleton; `stop` clears it.
  * `ensure_running` adopts only when the marker matches the current version
    (`_adoptable`); a missing/mismatched marker falls through to stop + reboot.
    So a stale VM is replaced automatically on the next launch — no manual kill,
    and it's concurrency-safe (the reboot happens under the singleton flock).

Also harden teardown: the PID file drifts after crashes / out-of-band kills,
so `stop` now also reaps any orphaned firecracker still bound to the infra
config path (scoped to that path, so interactive-pool VMs are untouched) —
otherwise a survivor holds the orchestrator TAP and the fresh boot dies with
"tap … Resource busy".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 01:03:30 -04:00
didericis 6f885af4b4 fix(firecracker): make guest /tmp world-writable so agents can use it
test / integration-docker (pull_request) Successful in 16s
test / coverage (pull_request) Failing after 36s
test / integration-firecracker (pull_request) Failing after 43s
lint / lint (push) Failing after 56s
test / unit (pull_request) Successful in 1m37s
tracker-policy-pr / check-pr (pull_request) Failing after 13m24s
The rootless agent rootfs build can land /tmp as 0755/root-owned, so the
agent (uid 1000 node) can't create scratch dirs there. The sandbox-escape
suite's README-push test does `cd /tmp && git init sandbox-escape-repo` and
died with "cannot mkdir sandbox-escape-repo: Permission denied" — before the
git-gate gitleaks hook could run — so the test read it as a missing hook.
On docker the agent inherits node:22-slim's 1777 /tmp, which is why only the
Firecracker path was affected (and only now that the suite runs end-to-end).

Set /tmp to 1777 in the guest PID-1 init, so every agent VM boots with a
usable /tmp regardless of rootfs perm drift.

Also update the infra-init unit test to assert the gateway launches via the
`bot_bottle.gateway_init` module (matching the prior fix), not a file path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 00:49:00 -04:00
didericis 127ba49372 fix(firecracker): launch infra-VM gateway as a module, not /app/gateway_init.py
test / integration-docker (pull_request) Successful in 18s
tracker-policy-pr / check-pr (pull_request) Successful in 19s
test / unit (pull_request) Failing after 35s
lint / lint (push) Failing after 46s
test / coverage (pull_request) Failing after 1m37s
test / integration-firecracker (pull_request) Failing after 1m50s
The infra VM's init boots the orchestrator control plane AND the gateway data
plane (egress/git-http/supervise). Since 5ad3449 moved the daemons into the
installed `bot_bottle` package, there is no `/app/gateway_init.py` file — the
gateway image's entrypoint is `python3 -m bot_bottle.gateway_init`. But the
firecracker init still spawned the old file path, so the guest logged:

    python3: can't open file '/app/gateway_init.py': No such file or directory

The control plane came up (it already used `python3 -m bot_bottle.orchestrator`)
but the gateway never started, so mitmproxy never generated its CA and launch
died with "gateway CA not available after 30s". Mirror the orchestrator line:
run the gateway as `python3 -m bot_bottle.gateway_init` (bot_bottle resolves
from the /app CWD, same as the control plane; egress-entrypoint.sh /
egress_addon.py are present from the gateway base image).

Third build/runtime bug from 5ad3449's package refactor that the Firecracker
integration suite never exercised (the artifact publish was broken, so the
job 404'd before boot). Changes the init, so the infra artifact version moves;
the matching rootfs has been rebuilt and published.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-19 00:35:19 -04:00
didericis 0d696674e3 fix(infra-build): repair gateway image build so infra artifact can publish
tracker-policy-pr / check-pr (pull_request) Successful in 16s
test / integration-docker (pull_request) Successful in 31s
test / unit (pull_request) Successful in 40s
test / integration-firecracker (pull_request) Failing after 2m36s
test / coverage (pull_request) Failing after 3m34s
The Firecracker integration + coverage jobs pull a prebuilt infra rootfs
artifact (PRD 0069 Stage 2) versioned by a content hash of the Dockerfiles,
bot_bottle/, and the guest init. Building that artifact (publish_infra ->
docker build Dockerfile.gateway) has been broken since 5ad3449, so the
artifact was never published and the KVM runner's integration test 404'd on
the pull — the failure this branch surfaced once it stopped falsely skipping.

Two build-time bugs, both from 5ad3449, neither exercised since:

- pyproject.toml declared build-backend "setuptools.backends.legacy:build",
  which is not an importable module; `pip install /src/` failed with
  BackendUnavailable. Use the real backend, "setuptools.build_meta"
  (the project has proper [project] metadata + flat-layout autodiscovery).
  Not part of the artifact hash, so this alone doesn't move the version.

- Dockerfile.gateway wrote /app/egress_addon.py before /app existed (the
  mkdir/WORKDIR came later), so the RUN redirect died with exit 2. Move
  WORKDIR /app above the shim write (WORKDIR creates it) and drop the now
  redundant later WORKDIR. This changes the gateway Dockerfile, so the infra
  artifact version moves 3c9e7b23260992db -> 01e6aaa714756fce; the matching
  artifact has been built and published to the generic package registry.

Also add Dockerfile* and pyproject.toml to test.yml's path filters: these
inputs determine what the firecracker jobs build/pull, so a change to them
must re-run the suite (and lets this push trigger a pull_request run).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-18 22:59:49 -04:00
didericis-codex 626f07efa6 fix(tests): run sandbox integration on firecracker
test / integration-firecracker (pull_request) Failing after 5s
test / integration-docker (pull_request) Successful in 9s
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / unit (pull_request) Successful in 33s
test / coverage (pull_request) Failing after 27s
lint / lint (push) Failing after 42s
2026-07-19 02:34:53 +00:00
didericis-claude d117460192 test: cover _daemon_reachable timeout path and DbStore.is_migrated
test / integration-firecracker (pull_request) Successful in 13s
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 14s
test / coverage (pull_request) Successful in 34s
lint / lint (push) Successful in 50s
test / unit (pull_request) Successful in 1m31s
Two diff-coverage gaps on the ci-kvm-runner branch:

1. bot_bottle/backend/docker/setup.py: the try/except TimeoutExpired
   block added in a prior commit had no tests reaching the subprocess
   path. Add two tests to TestDockerSetupStatus: one for the success
   path (subprocess returns 0) and one for the TimeoutExpired fallback.

2. bot_bottle/db_store.py: the _connection() context manager change in
   is_migrated() was never exercised by unit tests (all callers mock
   is_migrated() directly). Add test_db_store.py covering the absent-DB,
   missing-schema-table, migrated, and behind-schema cases.
2026-07-19 02:18:07 +00:00
didericis-claude e72ec71047 fix(tests): mock name_color_modal in test_cli_start_selector setUp
On a self-hosted KVM runner the process has a real controlling terminal
so name_color_modal successfully opens /dev/tty and enters a curses
loop waiting for keyboard input, hanging the test indefinitely.

Docker containers (ubuntu-latest runners) don't have a real /dev/tty,
causing an OSError that triggers the existing fallback — this is why
the hang was invisible in ubuntu-latest CI.

Also add timeout=5 to _daemon_reachable() to match the same defensive
fix already applied to docker_available() in tests/_docker.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 02:16:10 +00:00
didericis-claude 7aff69fbe0 fix(coverage): skip docker integration tests on the KVM runner
Docker integration tests are already covered by the integration-docker
job on ubuntu-latest. On the KVM runner the Firecracker TAP/nftables
pool conflicts with Docker networking, causing those tests to hang
and the coverage job to never complete.

Add SKIP_DOCKER_TESTS env-var support to docker_available() and set
it for the integration phase of coverage.sh so only Firecracker
integration tests run there.
2026-07-19 02:16:10 +00:00
didericis-claude 1d91db3e31 ci: fix tracker-policy-pr trigger — synchronize not synchronized
Gitea fires the pull_request push event as 'synchronize' (GitHub spec),
not 'synchronized'. The typo meant the workflow only ran on opened/
edited/reopened, leaving the required check yellow with no details link
after every commit push.
2026-07-19 02:16:10 +00:00
didericis-claude 686ca0d74b fix(tests): add 5-second timeout to docker_available() to prevent hang on KVM runner
On the self-hosted KVM runner Docker is on PATH but the daemon socket
is unreachable (firewalled/dropped). subprocess.run(["docker", "info"])
with no timeout hangs indefinitely on a dropped connection, stalling the
coverage job for hours — one hang per @skip_unless_docker()-decorated
class, ~8 per integration suite run.

Add timeout=5 with a TimeoutExpired → False fallback so the check
resolves quickly to "unreachable" rather than blocking.
2026-07-19 02:16:10 +00:00
didericis-codex 6d44a1be0a ci: scope firecracker backend to integration coverage 2026-07-19 02:16:10 +00:00
didericis-claude 32e85de16f fix(db): close SQLite connections explicitly to suppress ResourceWarning on Python 3.13
`sqlite3.Connection.__exit__` only commits/rolls back a transaction — it
does not close the connection. Python 3.13 (the Nix env on the KVM
runner) emits `ResourceWarning: unclosed database` for every connection
GC'd without an explicit close, producing noisy output in the coverage job.

Add `DbStore._connection()`, a `contextmanager` that calls `self._connect()`,
wraps it in the existing transaction context manager, and closes the
connection in a `finally` block. Change all `with self._connect() as conn:`
call sites in `db_store.py`, `audit_store.py`, `queue_store.py`, and
`orchestrator/registry.py` to `with self._connection() as conn:`.
`_connect()` remains as the per-subclass hook (RegistryStore overrides
it to set `busy_timeout`); `_connection()` delegates to `self._connect()` so
the override is respected.
2026-07-19 02:16:10 +00:00
didericis a1d2c4a500 ci: drop dev-requirements pip install on the self-hosted KVM runner
The self-hosted runner's Nix python env has no `pip` module, so
`python3 -m pip install -r requirements-dev.txt` failed with "No module
named pip" in both firecracker jobs. Neither job needs that install:

- integration-firecracker runs the stdlib `unittest` suite (no deps);
- coverage needs only `coverage`, which the runner's Nix python env
  already ships (7.12.0) — verified `coverage run`/`coverage json` work.

pylint/pyright are lint.yml's concern, not test.yml's. The ubuntu-latest
`unit` job keeps its `--break-system-packages` install unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1qRZTJC6qgBsUSjNrBdkX
2026-07-19 02:16:10 +00:00
didericis-claude a6fe31a424 ci(test): split integration into per-backend jobs
Add separate `integration-docker` and `integration-firecracker` jobs,
each with an explicit BOT_BOTTLE_BACKEND env var, so the backend used
is visible in CI output and skipped backends surface as a distinct job
rather than silent unittest.skip lines.

- integration-docker: ubuntu-latest, BOT_BOTTLE_BACKEND=docker
- integration-firecracker: [self-hosted, kvm], BOT_BOTTLE_BACKEND=firecracker,
  same-repo PRs + push + workflow_dispatch only (untrusted fork PRs do
  not execute on the privileged KVM runner)
- coverage: same same-repo restriction; refs #414 for the planned
  follow-up that moves coverage to ubuntu-latest via artifact combination

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 02:16:10 +00:00
didericis-claude 41b2b24b36 test(integration): lift GITEA_ACTIONS skip for Firecracker backend
The sandbox-escape test was unconditionally skipped when GITEA_ACTIONS=true,
which prevented Firecracker orchestration coverage from being measured even
when BOT_BOTTLE_BACKEND=firecracker is set on the KVM runner.

Narrow the skip to: GITEA_ACTIONS=true AND BOT_BOTTLE_BACKEND != firecracker.
When BOT_BOTTLE_BACKEND=firecracker the test is explicitly opted in to run on
the self-hosted KVM runner where the required /dev/kvm + TAP pool exist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 02:16:10 +00:00
didericis-claude 37045ca147 ci(coverage): address review findings from PR #349
- Finding 1: set BOT_BOTTLE_BACKEND=firecracker on the coverage step so
  the integration suite actually exercises the Firecracker orchestration
  paths rather than defaulting to Docker
- Finding 2: restrict the coverage job to push+workflow_dispatch only;
  PR-controlled code no longer executes on the privileged KVM runner
  automatically — maintainers trigger workflow_dispatch for trusted PRs
- Finding 3: expand path filters to include workflow files, scripts, and
  README so changes to CI configuration trigger the workflow itself

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 02:16:10 +00:00
didericis-claude 9b54cfa854 ci(coverage): install dev requirements on the KVM runner
The self-hosted KVM runner is a persistent machine, so
--break-system-packages is inappropriate. Use --user instead so
coverage (and pyright/pylint for future jobs) land in ~/.local
and survive between runs without touching the system Python.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-19 02:16:10 +00:00
didericis c193b04338 ci(coverage): run the diff-coverage gate on a self-hosted KVM runner
Re-land the coverage gate deferred from #343. The Firecracker VM/SSH
orchestration (~230 lines) is only exercised by the integration suite,
which needs /dev/kvm + the provisioned TAP/nft pool — a container runner
skips it and those lines read uncovered, so the 90% diff gate can't pass
on ubuntu-latest. Move the `coverage` job to a self-hosted `kvm` runner
with a firecracker-readiness preflight (binary + /dev/kvm + `backend
status`) so the integration test actually runs. Unit/lint stay on
ubuntu-latest. README documents the runner prerequisites.

Depends on a registered self-hosted runner labelled `kvm`; until one is
provisioned this gate will not run. See PRD 0069 / #348.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-19 02:16:10 +00:00
didericis c7ab3e0957 fix(tests): resolve sleep from PATH so subprocess tests run on NixOS
test / integration (push) Successful in 8s
test / unit (push) Successful in 33s
lint / lint (push) Successful in 44s
test / coverage (push) Successful in 36s
Update Quality Badges / update-badges (push) Successful in 34s
NixOS doesn't populate /bin (no /bin/sleep), so the gateway-init
end-to-end tests that spawn a real `sleep` errored with
FileNotFoundError. Add tests/_bin.py with a PATH-resolved SLEEP
constant (falling back to /bin/sleep on FHS hosts) and import it in
test_gateway_init.py instead of hardcoding the path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
2026-07-18 22:09:05 -04:00
didericis 034f774529 feat(supervise): non-blocking MCP — pending carries proposal id + check-proposal poll tool
test / integration (pull_request) Successful in 10s
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / coverage (pull_request) Successful in 39s
test / unit (pull_request) Successful in 1m30s
prd-number / assign-numbers (push) Failing after 10s
test / integration (push) Successful in 7s
test / unit (push) Successful in 30s
lint / lint (push) Successful in 42s
test / coverage (push) Successful in 35s
Update Quality Badges / update-badges (push) Successful in 34s
Closes #412.

The supervise MCP server blocked the agent's tool call polling for the
operator's decision, and on timeout returned `status: pending` with no
proposal id and no way to poll a specific proposal — so the only way to
learn a late decision was to re-propose (a duplicate).

- `handle_tools_call` pending timeout now returns the `proposal_id` and
  points the agent at `check-proposal`.
- New `check-proposal` MCP tool: non-blocking status lookup by proposal id
  (pending | approved | modified | rejected | unknown). Reuses the queue's
  FileNotFoundError semantics; archives a decided proposal exactly like the
  synchronous path, so a pending proposal stays visible to the operator
  until it's both decided and polled.
- `TOOL_CHECK_PROPOSAL` constant, re-exported from supervise; kept out of
  TOOLS since it never becomes a Proposal.tool.

Enforcement is unchanged — the tools only propose policy; the egress proxy
and git-gate still enforce — so returning early opens no hole. Follow-ups
(git-gate reject-requeue, backpressure, notifications, web console) are in
the PRD.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 17:06:04 -04:00
didericis 5b359fe8d2 fix(git-gate): scan $new --not --all for every push, not $old..$new
test / unit (push) Successful in 32s
test / integration (push) Successful in 31s
lint / lint (push) Successful in 43s
Update Quality Badges / update-badges (push) Successful in 38s
test / coverage (push) Successful in 40s
The pre-receive hook scanned existing-branch updates with the delta range
$old..$new. On a rebase / non-fast-forward force-push onto an advanced main,
$old is no longer an ancestor of $new, so $old..$new expands to all of main's
new history — including the deliberate sandbox-escape gitleaks fixtures — and
the push is rejected on commits that belong to main, not the branch.

Unify the range on `$new --not --all` for every non-delete push (this is the
deferred open question from PRD 0028, which already applied it to new refs
for #106). It scans only the commits the push introduces and is
security-equivalent: the bare repo's refs come only from trusted upstream
mirror-fetch and gitleaks-gated pushes, so an excluded commit is
already-upstream or already-scanned. It is also more correct for
non-fast-forward pushes, where $old..$new can skip commits off the direct path.

Fixes #421

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1qRZTJC6qgBsUSjNrBdkX
2026-07-18 16:51:31 -04:00
didericis 015ff52eda fix(cli): exempt backend command from DB migration gate
test / integration (pull_request) Successful in 8s
tracker-policy-pr / check-pr (pull_request) Successful in 9s
test / unit (pull_request) Successful in 31s
test / coverage (pull_request) Successful in 36s
test / integration (push) Successful in 12s
test / unit (push) Successful in 36s
test / coverage (push) Successful in 39s
Update Quality Badges / update-badges (push) Successful in 37s
lint / lint (push) Successful in 2m38s
`backend setup/status/teardown` manage host prerequisites only and never
open the store, but the CLI dispatcher ran the schema-migration gate before
every command. On a non-TTY runner the gate's `Migrate now? [y/N]` prompt
reads EOF and refuses, so `backend status --backend=firecracker` exits 1 —
breaking the Firecracker CI preflight on any host without a pre-migrated DB.

Exempt `backend` from the gate; store-touching commands stay gated.

Fixes #419

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1qRZTJC6qgBsUSjNrBdkX
2026-07-18 15:27:29 -04:00
didericis-claude 4302678f3e docs(research): expand sandbox landscape with 6 new tools; add agent-tailored policy axis
tracker-policy-pr / check-pr (pull_request) Failing after 6s
Isolation tools added: Cleanroom (Buildkite), container-use (Dagger),
Docker sbx, Anthropic srt.

Governance/pre-action layers added as a separate section: Microsoft
Agent Governance Toolkit (per-agent DID + YAML policy + trust score),
Open Agent Passport (declarative policy + cryptographic audit).

Comparison table: 14 → 14 columns; new Agent-tailored policy row added.
Second addendum covers competitive position on role-tailoring, Docker
sbx as new DX-class competitor, and borrowable ideas (trust-score decay,
live network TUI, cryptographic audit chain).

Discourse note: adds Per-agent role tailoring to "What it covers well"
with competitive comparison table across 9 tools.
2026-07-18 19:11:14 +00:00
didericis-claude 3a6fbad057 ci: split tracker-policy into separate issue and PR workflows
tracker-policy-pr / check-pr (pull_request) Successful in 4s
Gitea Actions reports skipped jobs as a non-success status, which caused
label-issue to block PRs even though its if-condition correctly excluded it.
Two dedicated workflows eliminate the skipped-job problem entirely.

After merge, update the branch-protection required status context from
`tracker-policy / check-pr` to `tracker-policy-pr / check-pr`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-18 18:37:44 +00:00
didericis-codex a800a417d9 ci: enforce pylint score instead of warning exit bits
test / integration (pull_request) Successful in 9s
test / coverage (pull_request) Successful in 39s
test / unit (pull_request) Successful in 1m22s
test / integration (push) Successful in 25s
test / unit (push) Successful in 34s
lint / lint (push) Successful in 42s
Update Quality Badges / update-badges (push) Successful in 39s
test / coverage (push) Successful in 42s
2026-07-18 14:18:39 -04:00
didericis-codex 293218035d ci: enforce canonical issue metadata policy 2026-07-18 14:18:39 -04:00
didericis-claude 727eafe0f9 docs(research): clarify prompt injection framing and blast-radius risks
Collapse "trusted-channel data injection" into prompt injection
throughout — the trusted channel is a delivery vector, not a distinct
attack class. Add explicit inbound/outbound orthogonality framing.
Replace the two redundant "weaker" bullets with a single prompt
injection section and a new blast-radius breakdown covering work
product corruption, malicious commits past gitleaks, exfiltration
through allowlisted channels, and dependency-install injection.
2026-07-18 18:13:33 +00:00
didericis-claude 1ec114b6d7 docs(research): survey HN agent safety discourse June-July 2026
Covers the CVE cascade (DuneSlide, CVE-2026-39861, MCP STDIO injection),
Agentjacking and README-injection attack classes, community opinion
clusters, and a frank assessment of where bot-bottle covers or falls
short against each issue.
2026-07-18 18:08:40 +00:00
didericis aa44feea02 docs(research): note on malicious-commit scanning at the git-gate + paid-feature analysis
Adds a research note on whether/how to scan for malicious code (not just
secrets) in commits pushed through the git-gate, and whether the semantic
(LLM) layer is a defensible paid feature.

Verdict: no scanner reliably detects malicious code (undecidable +
adversarial), so the frame is raise-cost + cover-the-obvious + human-gate
the dangerous. Ranked layers: dependency/supply-chain scanning (Socket/OSV/
GuardDog) > heuristic/obfuscation (Semgrep-on-diff) > risk-based human
gating via the existing supervise plane > best-effort LLM diff-review.
Fast scanners inline in the synchronous pre-receive; heavy analysis async.

Monetization: the paid unit is the governed git-egress review bundle
(managed semantic review + web-console human-review flow + RBAC + audit +
cross-run policy), not the raw scanner — which stays OSS like gitleaks.
Extends the egress audit+custody wedge to code artifacts; the supervise
console generalizes across all proposal types (egress, gitleaks, commit
review). Sell the workflow, not the detector's accuracy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 07:00:22 -04:00
didericis f2e2572a40 docs(research): add DX axis — "run Claude yolo-style" — to the sandbox landscape
Adds a "DX: run Claude yolo-style" row to the comparison table plus a note
framing developer experience as a differentiator. The field splits into
wrappers-around-the-agent (bot-bottle, agent-safehouse — one command, the
agent just runs, `--dangerously-skip-permissions` on by default with the
sandbox as the guardrail) vs libraries/services (boxlite, microsandbox,
CubeSandbox, E2B — you wire the agent in via SDK/cluster). agent-safehouse
is the only DX peer, but it's macOS-only Seatbelt with no egress story.
"As easy as native yolo, but actually sandboxed" is the defensible line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:41:26 -04:00
didericis 7069fa225d docs(research): add long-running posture axis to the sandbox landscape
Adds a "Long-running posture" row to the comparison table and an addendum
note contrasting the two models: E2B and CubeSandbox are ephemeral-per-task
(5-min default timeout, tier-capped continuous runtime, duration via
pause/resume + reconnect-by-id), while bot-bottle bottles are persistent,
named, and supervised by default. For agents that run for hours/days this
posture difference matters more than the isolation primitive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:28:26 -04:00
didericis aa224c4381 docs(research): add CubeSandbox to the sandbox landscape; fix stale bot-bottle self-description
Adds CubeSandbox (Tencent Cloud, Apache 2.0, RustVMM/KVM microVM) to the
agent-sandbox landscape survey: per-project note, comparison-table column,
and a dated addendum on what it means for positioning. CubeSandbox is the
first surveyed project to bundle a connection-level egress allowlist +
audit + in-flight credential custody, but it does NOT do content DLP on
authorized channels — that plus the orchestration layer is where
bot-bottle stays distinctive.

Also corrects two stale self-descriptions the survey (2026-05-11) baked
in and I'd propagated:
- Default isolation is now a VM per bottle (Firecracker microVM on KVM
  Linux, Apple Container on macOS); Docker is only the legacy fallback,
  per _default_backend_name(). Was described as Docker-by-default.
- Outbound DLP is bot-bottle's own mitmproxy egress scanner + gitleaks on
  git push, not pipelock (removed). All references updated; a note
  records the change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YBCHap11yGAKuKfsehNPaD
2026-07-18 06:28:12 -04:00
Quality Badge Bot aed686d85d chore: update quality badges
- Coverage: 81%
- Core coverage: 94%

[skip ci]
2026-07-18 09:15:12 +00:00
didericis-claude 410c19aaaf fix(backend): fix pyright errors in lazy-load implementation
test / integration (push) Successful in 10s
test / coverage (push) Successful in 38s
Update Quality Badges / update-badges (push) Successful in 37s
lint / lint (push) Successful in 42s
test / unit (push) Successful in 1m21s
- Rename _BACKENDS → _backends: pyright treats uppercase module-level
  names as constants and flags the reassignment in _get_backends() as
  reportConstantRedefinition; lowercase avoids this.
- Add TYPE_CHECKING guard importing CommitCancelled/Freezer/get_freezer
  from .freeze: pyright cannot see module-level __getattr__ bindings, so
  reportUnsupportedDunderAll fired for those three __all__ entries; the
  guard makes them visible to the type checker without running at import
  time.
- Update test_backend_selection.py to patch _backends (lowercase).
2026-07-18 05:14:27 -04:00
didericis-claude f0ba399f17 fix(backend): silence pylint false positives from lazy-load pattern
`undefined-all-variable` fires on CommitCancelled / Freezer / get_freezer
in __all__ because pylint can't see module-level __getattr__ bindings;
`global-statement` fires on the _BACKENDS singleton setter. Both are
intentional patterns — add inline disables rather than suppress globally.
2026-07-18 05:14:27 -04:00
didericis-claude 8b442b8718 perf: lazy-load backend modules and consolidate docker subprocess helpers
Importing backend.docker.util previously triggered eager loading of all
three backend packages (~76 modules) because backend/__init__.py imported
DockerBottleBackend, FirecrackerBottleBackend, and MacosContainerBottleBackend
at module scope. This made the module prohibitively expensive to import
from the orchestrator layer and elsewhere.

The three backend imports are now deferred into _get_backends(), which
loads all three on first call and caches the result in the module-level
_BACKENDS variable (initially None). Module-level __getattr__ exposes
backend classes and freeze symbols lazily for existing import/patch sites.

backend/docker/util.py raw subprocess.run(["docker", ...]) calls are
replaced with the shared run_docker primitive from docker_cmd, eliminating
the duplication between the backend and orchestrator implementations.
_silent_run() is removed; image_exists() is inlined directly onto
run_docker. The commit_container test is updated to patch run_docker
instead of subprocess.run.
2026-07-18 05:14:27 -04:00
didericis 5eb6c8d99b ci: drop actions/setup-python from the remaining workflows
Applies the same fix as test/lint to every workflow that still used
actions/setup-python, which the old act_runner engine mishandles:

- update-badges: was broken identically to lint — setup-python + pip
  install hit the image's externally-managed system Python. Drop
  setup-python, install with --break-system-packages, and use `python3`
  (not bare `python`) for the coverage steps.
- canaries, prd-number: no pip install, so not failing, but they carried
  the same fragile (and network-heavy) setup-python for stdlib-only work.
  Removed — the image's system Python 3.12 runs them directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-18 05:11:48 -04:00
didericis 4f10b810d4 ci(lint): drop actions/setup-python; install into the container's system Python
lint / lint (push) Successful in 2m16s
Same fix as the test workflow: the old act_runner engine mishandles
actions/setup-python's PATH, so `pip install` hit the image's
externally-managed system Python and failed with
"externally-managed-environment" on the "Install dev dependencies" step.

The runner image already ships Python 3.12 and the job container is
ephemeral, so drop setup-python and install straight into system Python
with --break-system-packages. pylint/pyright console scripts land on
/usr/local/bin (on PATH), so the lint steps still resolve. Also drops the
now-pointless `pip install --upgrade pip`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-18 05:08:18 -04:00
didericis d3c4fc0fd4 ci(test): drop actions/setup-python; install into the container's system Python
test / integration (pull_request) Successful in 7s
test / unit (pull_request) Successful in 44s
test / coverage (pull_request) Successful in 36s
test / integration (push) Successful in 7s
Update Quality Badges / update-badges (push) Failing after 11s
test / unit (push) Successful in 29s
test / coverage (push) Successful in 35s
lint / lint (push) Successful in 2m24s
The old act_runner engine (v0.2.13 on the delphi-ci runner) mishandles
actions/setup-python's PATH injection: pip installs coverage into the
toolcache interpreter while `python3` in later steps resolves back to the
image's system Python, so unit/coverage jobs failed with "No module named
coverage". Newer runners (TrueNAS's v0.6.1) don't, which is why it only
broke on delphi.

The runner-images/act container already ships Python 3.12, and the job
container is ephemeral, so drop setup-python entirely and install straight
into the system Python with --break-system-packages. Every step now uses
one interpreter consistently, on any runner version. Also removes the
redundant setup-python step from the integration job (stdlib-only).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-18 04:58:38 -04:00
didericis-claude 232dfdf37a refactor(gateway): replace egress_addon.py copy with a one-line shim
test / integration (pull_request) Successful in 1m15s
test / coverage (pull_request) Failing after 1m13s
test / unit (pull_request) Successful in 1m19s
mitmdump -s requires a file path, not a module. Instead of copying the
full egress_addon.py to /app/, write a one-line shim at image build time
that re-exports addons from the installed package. mitmdump finds the
addons list in the shim's namespace; all real addon code stays in
bot_bottle/egress_addon.py.
2026-07-18 07:59:57 +00:00
didericis-claude 9a0dd821ef refactor(gateway): invoke daemons via python3 -m instead of /app/ file copies
lint / lint (push) Successful in 2m20s
test / unit (pull_request) Successful in 1m12s
test / integration (pull_request) Successful in 26s
test / coverage (pull_request) Successful in 1m29s
supervise_server, git_http_backend, and gateway_init all have __main__
guards, so python3 -m bot_bottle.X replaces the individual COPY lines
to /app/. egress_addon.py stays as a file copy because mitmdump -s
requires a file path rather than a module reference.
2026-07-18 07:55:38 +00:00
didericis-claude 5ad3449e3b refactor(gateway): replace flat-file import shims with installed package
lint / lint (push) Successful in 2m22s
test / unit (pull_request) Successful in 1m12s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m23s
Install bot_bottle via pip in Dockerfile.gateway instead of COPYing
individual .py files flat under /app/. This eliminates the try/except
import shims in egress_addon_core, dlp_detectors, egress_addon,
supervise, supervise_server, and git_http_backend that existed only
to support the flat-bundle layout.

Adds bot_bottle/constants.py as a single source of truth for
IDENTITY_HEADER and GIT_GATE_TIMEOUT_SECS, removing the duplicated
literal definitions in egress_addon.py, supervise_server.py,
git_http_backend.py, and git_gate_render.py.

Test files updated to match: test_supervise_server.py drops the
sys.path.insert hack in favour of direct package imports; the
egress_addon test shims no longer pre-populate sys.modules with a
bare egress_addon_core alias.
2026-07-18 03:01:41 +00:00
didericis d8e3947bd3 test(git-http): wire the resolver into the access-hook-503 regression test
test / unit (pull_request) Successful in 1m9s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m21s
lint / lint (push) Successful in 2m16s
test / unit (push) Successful in 1m9s
test / integration (push) Successful in 31s
test / coverage (push) Successful in 1m33s
Update Quality Badges / update-badges (push) Successful in 1m21s
b1850be's fail-closed-503 test (rebased in from main) built a git-http
server with a flat repo.git and no policy_resolver. The resolver-only
data plane on this branch denies an unattributed request with 404 before
it reaches the access-hook path the test exercises, so it saw 404 != 503.

Nest the bare repo under <root>/<_BID>/ and set _FixedResolver(_BID) on
the server, matching every other test in this module, so the request is
attributed and reaches the access-hook (mocked to raise PermissionError)
that the 503 fail-closed behavior guards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 22:34:28 -04:00
didericis d0b7de119f test(egress): set orchestrator URL for entrypoint tests; cover fail-closed guard
The egress_entrypoint.sh fail-closed guard (this branch) exits 1 when
BOT_BOTTLE_ORCHESTRATOR_URL is unset, which broke the argv-construction
tests that ran the script without it. Set the URL in the shared
_run_entrypoint helper (a precondition for reaching mitmdump now, like
PATH) and add a test asserting the guard fails closed when it's absent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 22:34:28 -04:00
didericis ea1fbeeaa0 refactor(gateway): fail closed without an orchestrator URL; drop stale single-tenant comments
Follow-ups from the #402 review of the single-tenant data-plane teardown.

- egress_entrypoint.sh: refuse to launch mitmdump when
  BOT_BOTTLE_ORCHESTRATOR_URL is unset, so the fail-closed guarantee no
  longer rests solely on mitmproxy's errorcheck addon exiting on the
  addon's load-time raise. A misconfigured gateway can never come up as
  a bare TLS-bumping open proxy with no policy.
- orchestrator/gateway.py: ensure_running() raises GatewayError on an
  empty orchestrator URL — a URL-less launch would only crash-loop the
  now-resolver-only daemons (egress raises, git-http exits 1, supervise
  exits 2). The env-injection branch is now unconditional.
- Drop stale "single-tenant" / "reads routes.yaml" comments in
  gateway.py and egress_entrypoint.sh, and the /etc/egress/routes.yaml
  layout line in Dockerfile.gateway.
- Tests: gateway fixtures supply an orchestrator URL; add a
  refuse-without-URL test and assert the URL env is injected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 22:34:28 -04:00
didericis b601b663e2 refactor(gateway): remove the single-tenant data-plane paths (audit #400 finding 3)
All three backends (docker, firecracker, macos-container) now launch through
the consolidated orchestrator, and every production gateway sets
BOT_BOTTLE_ORCHESTRATOR_URL — so the legacy single-tenant (`resolver is None`)
branches in the shared gateway's data plane were unreachable dead code, a second
security-relevant path to keep correct in parallel with the live one. Make the
orchestrator resolver mandatory and delete the single-tenant paths from the
three data-plane modules.

egress_addon.py: drop the static routes file entirely — EGRESS_ROUTES, _reload,
the SIGHUP handler, self.config, and the SUPERVISE_BOTTLE_SLUG env slug. The
per-request /resolve is the only policy source; __init__ fail-closes if
BOT_BOTTLE_ORCHESTRATOR_URL is unset. Introspection (`_egress.local/allowlist`)
now reports the calling bottle's *resolved* routes. The block/redact log gates
and _req_ctx redaction now read the per-flow config/env from the request-time
stash, so they use each bottle's log level and token set (they silently used the
empty static config before). Nothing sends `docker kill --signal HUP` to the
gateway in the consolidated model (the egress applicators fail closed), so
removing the SIGHUP reload is safe.

git_http_backend.py: resolver mandatory; no flat-root fallback. main() refuses
to start without an orchestrator URL; a request whose source resolves to no
bottle 404s.

supervise_server.py: resolver mandatory; every proposal is attributed to the
source-IP-resolved bottle. Remove handle_list_egress_routes (the proxy-fetch
introspection that only worked when the proxy carried one bottle's identity) —
list-egress-routes is answered from the resolved policy. main() refuses to start
without an orchestrator URL.

Tests: a host-side fake resolver serves each test's Config through the real
parse path (a small YAML-subset emitter round-trips route_to_yaml_dict); the
response/websocket hooks stash it as request() would. Deletes the tests for the
removed static-config, SIGHUP-reload, and single-tenant-passthrough paths; adds
fail-closed-without-orchestrator coverage.

Follow-up: gateway_init still forwards SIGHUP to the egress child (now dormant —
no one sends it); the README still describes the docker backend's per-bottle
topology. Both are outside the data-plane teardown.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 22:34:28 -04:00
didericis 2aec30e501 fix(git-gate): install the gitleaks binary for the build's target arch
The gateway Dockerfile hardcoded the linux_x64 gitleaks download, so an
image built on/for aarch64 (Apple Silicon) baked in an x86_64 binary.
It sat quiet until the git-gate pre-receive hook first invoked it, where
the kernel refused the foreign-arch exec — `gitleaks: Exec format error`
— failing every push through that gateway.

Pick the asset + pinned SHA from the build's target architecture:
TARGETARCH (auto-populated by BuildKit) with a `dpkg --print-architecture`
fallback for a legacy builder, and hard-fail on any unsupported arch.
Each arch keeps its own SHA256 verification, so no supply-chain regression.

The existing integration test test_gateway_image.py::
test_gitleaks_binary_present_and_versioned execs `gitleaks version` in the
built image, so it now passes on arm64 instead of hitting the same error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 22:14:47 -04:00
didericis b1850be5d1 fix(git-gate): make the gateway access-hook executable regardless of copy transport
test / unit (pull_request) Successful in 1m19s
test / integration (pull_request) Successful in 30s
test / coverage (pull_request) Successful in 1m35s
lint / lint (push) Successful in 2m35s
test / unit (push) Successful in 1m21s
test / integration (push) Successful in 29s
test / coverage (push) Successful in 1m31s
Update Quality Badges / update-badges (push) Successful in 1m21s
Cloning/fetching from the git-gate on the Apple-container backend failed with
"empty reply from server" (curl exit 52). Root cause: the git-http handler
crashed on every upload-pack with

    PermissionError: [Errno 13] Permission denied: '/etc/git-gate/access-hook'

The access-hook is exec'd directly, so it needs the x bit. prepare() stages it
0o700 and trusted the gateway copy to carry that mode. `docker cp` does; the
Apple `container cp` (AppleGatewayTransport) does not, landing the hook 0o644 →
EACCES. The unhandled exception killed the handler thread, closing the socket
with no HTTP response — which the client sees as the opaque empty reply.

- provision_git_gate now `chmod +x`es the access-hook on the gateway side after
  the copy, so it's executable under every transport (docker/apple/firecracker).
- git-http handler wraps the access-hook subprocess.run: an OSError /
  SubprocessError (un-execable, timed out) now fails closed with a 503 instead
  of crashing the thread into an empty reply — a gate that can't run its hook
  should deny, visibly.
- Updates the now-misleading "docker cp preserves source mode" comment in
  git_gate.prepare().

Regression tests: provisioning applies +x to the access-hook; the handler
returns 503 (not an empty reply) when the hook can't be exec'd.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 21:49:30 -04:00
didericis fd86e7fa99 fix(egress): redact the per-bottle token in the now-active multi-tenant response log
test / unit (pull_request) Successful in 1m9s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Successful in 1m24s
lint / lint (push) Successful in 2m15s
test / unit (push) Successful in 1m8s
test / integration (push) Successful in 24s
test / coverage (push) Successful in 1m20s
Update Quality Badges / update-badges (push) Successful in 1m18s
Self-review of this PR: making `response()` run in the consolidated gateway
also activates its `LOG_FULL` `_log_response` call there — previously
unreachable, since the empty static config made `response()` return early. That
logger redacted with `os.environ`, which in multi-tenant mode does NOT hold the
bottle's per-request `/resolve` tokens (only the resolved `env` overlay does),
so a non-token-shaped provisioned secret appearing in a response could be logged
in the clear.

Thread the resolved per-flow `env` into `_log_request` / `_log_response` so the
LOG_FULL redaction scrubs the calling bottle's secrets. Adds a regression test
(a non-token-shaped `/resolve` secret, absent from os.environ, must not appear
in the response log) and updates the redaction-test helpers for the new arg.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 17:14:52 -04:00
didericis 3bbd839917 fix(egress): scan response + websocket DLP against the resolved per-flow config
In the consolidated (multi-tenant) gateway the addon's static `self.config`
is empty — each request's real policy comes from the per-request `/resolve`.
`response()` and `websocket_message()` still matched routes against that empty
config, so inbound prompt-injection DLP and WebSocket credential/injection DLP
silently skipped every scan (fail-open) whenever the gateway ran multi-tenant.
This is backend-agnostic: the gateway image (and this addon) is shared by the
Firecracker, macOS, and docker consolidated backends.

Resolve the per-flow (config, slug, env) once in `request()`, stash it on
`flow.metadata`, and have both hooks read it back — falling back to the static
single-tenant values for a flow that never passed through `request()`. Reusing
the request's one `/resolve` avoids a round-trip per response and per WebSocket
frame.

Adds multi-tenant regression tests for both hooks that fail against the old
fall-open behaviour.

Refs: audit issue #400 (finding #2)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 17:14:52 -04:00
didericis 5c526860bc fix(test): resolve the remaining review findings on the control-plane auth test
test / unit (pull_request) Successful in 1m17s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m21s
lint / lint (push) Successful in 2m23s
test / unit (push) Successful in 1m24s
test / integration (push) Successful in 30s
test / coverage (push) Successful in 1m26s
Update Quality Badges / update-badges (push) Successful in 1m24s
Addresses the 5 lower-priority findings left as follow-up in the earlier
review, now that each has a concrete answer:

- Add gateway_name: str = GATEWAY_NAME to OrchestratorService.__init__
  (mirrors the existing orchestrator_name param) and thread it through
  _gateway(). Deletes the test's _IsolatedOrchestratorService subclass,
  which existed only to override a private method for this one kwarg —
  any caller needing gateway-name isolation can now use the public
  constructor. Backward compatible: every existing caller constructs
  OrchestratorService with keyword args and a sensible default is kept.

- Give the test its own fixed image tags (bot-bottle-orchestrator:itest,
  bot-bottle-gateway:itest) instead of the production :latest ones.
  _running_image_is_current() keys gateway staleness off the image tag's
  ID, not per-instance identity, so rebuilding the shared :latest tag from
  whatever's on disk during a test run could make a real host's running
  production gateway look stale and get force-recreated. Fixed tags (not
  per-run-suffixed, so they don't accumulate) fully decouple the two.

- setUp -> setUpClass/tearDownClass: all 5 tests are read-only checks
  against the same running control plane, so one shared container
  lifecycle replaces 5 (each of which paid its own container-start +
  image-build + health-poll cycle). Cuts the file's wall-clock roughly
  4x (11.5s -> 2.9-4.3s) and, combined with the network-rm cleanup from
  the previous commit, means one cleanup instead of five.

- Reuse OrchestratorClient (bot_bottle/orchestrator/client.py) instead of
  a hand-rolled urllib helper — the test now exercises the same
  request/response code path the real host CLI uses, rather than a
  private copy that could silently drift from it.

- Add the chown workaround test_multitenant_isolation.py already needed
  for this exact bind-mount: the orchestrator container has no USER
  directive, so it writes the registry DB as root into the throwaway
  host_root; chown it back before tempdir cleanup so that doesn't raise
  PermissionError on native Linux Docker (no UID remap, unlike Docker
  Desktop's macOS VM).

Verified: ran the suite twice in a row (idempotency — fixed image tags
don't accumulate, 5/5 pass both times, 2.99-4.33s each), the real
~/.bot-bottle/control-plane-token is untouched, zero leaked networks or
containers after either run, exactly 2 :itest images (not growing), the
full orchestrator unit suite (93 tests) and the sibling docker
gateway/broker integration tests still pass. pyright clean, pylint
10.00/10 on both changed files.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 17:00:31 -04:00
didericis 492669e620 fix(test): skip the control-plane auth test under act_runner (CI)
lint / lint (push) Successful in 2m23s
test / unit (pull_request) Successful in 1m17s
test / integration (pull_request) Successful in 30s
test / coverage (pull_request) Successful in 1m29s
Pushing the previous two fixes surfaced a real, currently-failing CI run
(gitea actions run 2164, jobs "integration" and "coverage"): every one of
the 5 new tests errored in setUp with

  docker: Error response from daemon: error while creating mount source
  path '/workspace/didericis/bot-bottle': mkdir /workspace: read-only
  file system

This is finding #4 from the review of the previous commit, now confirmed
live rather than just plausible: act_runner's job-container topology
can't satisfy the orchestrator's host-path bind mount, the same
constraint test_multitenant_isolation.py, test_gateway_image.py, and
test_sandbox_escape.py already skip around. Add the identical
skip_unless_docker + GITEA_ACTIONS guard so this test degrades the same
way its siblings do instead of failing the job.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 16:45:50 -04:00
didericis d32f36bb2b fix(test): actually isolate the docker control-plane auth test's root + network
Two bugs surfaced by a review of the previous commit:

- host_control_plane_token() resolves its path via the ambient
  BOT_BOTTLE_ROOT env var, not the host_root kwarg passed to
  OrchestratorService (that kwarg only controls the DB bind-mount
  destination). The test's isolation claim was false as a result: running
  it read/wrote the developer's real ~/.bot-bottle/control-plane-token
  instead of the throwaway temp dir — confirmed directly on disk. Fixed
  by pointing the env var at the same temp dir for the test's duration
  and restoring it via addCleanup.

- ensure_running() creates a per-bottle Docker network but stop() only
  ever removes containers, never the network — every run of this test
  leaked one bridge network permanently (found and removed 5 from prior
  runs via `docker network ls`). Fixed with an explicit `docker network
  rm` in addCleanup.

Verified: re-ran the suite twice: 5/5 pass both times, the real
~/.bot-bottle/control-plane-token timestamp is unchanged across both runs
(proving isolation), and `docker network ls` shows zero leaked
bot-bottle-net-itest-* networks afterward. pyright clean, pylint 10.00/10.

Remaining findings from the same review (missing GITEA_ACTIONS skip
guard, root-owned bind-mount cleanup on native Linux, no setUpClass,
reinvented OrchestratorClient, gateway_name should be a constructor
param rather than a subclassed private-method override) are left for a
follow-up — each is a real, separate design/scope call, not a
mechanical fix like these two.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 16:43:07 -04:00
didericis 4f36b919b2 test(orchestrator): live Docker coverage for the control-plane auth fix (#400)
lint / lint (push) Successful in 2m30s
test / unit (pull_request) Successful in 1m24s
test / integration (pull_request) Failing after 3m8s
test / coverage (pull_request) Failing after 1m59s
ca91fc4 fixed the control plane's missing caller authentication but only
verified it end-to-end on Apple Container; Docker had the same code change
(docker_cmd.py's env injection) backed only by the in-process dispatch()
unit tests, which never exercise the real HTTP server or a real container.

Ran it for real first: brought up the actual orchestrator + gateway via
Docker, hit the published control-plane port directly. Confirms /health
stays open, /bottles and /resolve both 401 with no token or a wrong one
(the enumeration and credential-lift vectors from #400), and /bottles
succeeds with the real per-host secret.

Added as a proper integration test so this stays covered: a subclass of
OrchestratorService giving the gateway its own unique name too (the base
class only parameterizes the orchestrator's), plus a throwaway
BOT_BOTTLE_ROOT and a random port, so a run can never collide with a real
host's orchestrator or gateway. Gated on a reachable Docker daemon, same
as the existing docker gateway/broker integration tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 16:17:42 -04:00
didericis ca91fc4d91 fix(security): authenticate the orchestrator control plane (issue #400 Critical)
lint / lint (push) Successful in 2m34s
test / unit (pull_request) Successful in 1m21s
test / integration (pull_request) Successful in 24s
test / coverage (pull_request) Successful in 1m26s
The control plane had no caller authentication — dispatch() routed every
request straight through, and the only guards anywhere were the
(source_ip, identity_token) checks inside /resolve and /attribute, which
authenticate the *bottle* a request is about, not the *caller*. On the
consolidated backends the agent shares a network with the control-plane port
(macOS: the infra container runs 8099 on the agent's --internal network;
docker: the orchestrator container sits on the shared gateway network), so a
prompt-injected agent could GET /bottles (enumerate siblings + policies),
PUT /bottles/<id>/policy (rewrite its own egress allowlist to allow-all),
POST /resolve (read the injected upstream credentials it's never meant to
see), and POST /supervise/respond (self-approve its own proposals). Every
README guarantee collapsed to the agent's discretion.

Fix: a per-host control-plane secret required on every route but GET /health,
compared with hmac.compare_digest. It is held only by the trusted callers and
never handed to an agent:
- minted + persisted 0600 at <root>/control-plane-token (paths.host_control_plane_token);
- injected as $BOT_BOTTLE_CONTROL_PLANE_TOKEN into the orchestrator + gateway
  containers via bare `--env NAME` (value inherited from the launch process,
  so it never lands on argv or in `container/docker inspect`);
- presented by the gateway's PolicyResolver (reads the env) on /resolve, and by
  the host CLI's OrchestratorClient (reads the host file) on every call.

The agent container is never given the env var or the host file, so from a
bottle every /bottles*, /resolve, /attribute, and /supervise/* call now
returns 401 — closing the enumeration, allowlist-rewrite, credential-lift, and
self-approval. The existing (source_ip, identity_token) checks stay as
defense-in-depth.

Enforced when configured: macOS + docker inject the secret (→ enforced). With
no secret set the server runs open and warns loudly at startup — a
fail-visible fallback for the unit suite and for Firecracker, whose
port-scoped nft already blocks agents from 8099 (wiring the secret into its
infra-VM init is a clean fast-follow, left out here to avoid churning the
prebuilt-artifact hash).

Verified end-to-end on real Apple Container: infra comes up healthy, the host
CLI (with the secret) lists bottles while an unauthenticated GET /bottles gets
401, all five issue-#400 attacks from inside the agent get 401, and egress
policy still works (200 allowed / 403 denied) — proving the gateway
authenticates to /resolve with the secret. 1829 unit tests pass, pyright
clean, pylint 9.91.

Refs #400.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 05:15:16 -04:00
didericis 4a607ad098 refactor(macos): one infra container (control plane + gateway), fixes shared-DB races
lint / lint (push) Successful in 2m15s
test / unit (pull_request) Successful in 1m16s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m17s
Adopts the firecracker infra-VM pattern for macOS: the orchestrator control
plane and the gateway data plane now run in a SINGLE Apple container instead of
two. Apple Containers are lightweight VMs with separate kernels, so the prior
two-container design had both guests writing one bot-bottle.db over virtiofs,
where fcntl locks are not coherent across kernels — concurrent writes (the
orchestrator's registry vs the gateway supervise daemon's queue) could corrupt
it. One container = one kernel = coherent locking.

The DB moves onto a container-only Apple volume (bot-bottle-mac-db), never
bind-mounted from the host, so no host process opens the live file either. The
host CLI already reaches registry + supervise state over the control-plane HTTP
surface (cli/supervise.py uses OrchestratorClient), exactly as firecracker's
VM-only DB requires.

Two simplifications fall out of the single container:
- No DNS dance: the control plane and gateway daemons reach each other over
  127.0.0.1, so the orchestrator-before-gateway ordering (a workaround for
  Apple having no container DNS) is gone, along with the moved-IP recreate
  logic it needed.
- Net -243 lines.

Mechanics: the infra container runs from the gateway image with the
control-plane source bind-mounted read-only (like the docker orchestrator, so a
code change needs no rebuild) and a small sh -c init that starts both processes
(mirrors firecracker's _infra_init). Also implements the macOS backend's
ensure_orchestrator() and adds it to discover_orchestrator_url, so operator
tools (supervise) can bring up / find the control plane on demand — previously
the macOS backend died with "no orchestrator control plane".

Verified end-to-end on real Apple Container 1.0.0: the single infra container
comes up healthy (one address for control plane + gateway), both processes run,
the DB is written on the container-only volume, host-side supervise works over
HTTP, and a registered agent gets 200 for an allowed host / 403 for a denied
one. 1824 unit tests pass with `container` absent (CI parity), pyright clean,
pylint 9.89.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 04:14:14 -04:00
didericis e24b62b6b9 fix(macos): review fixes — token on plan, self-heal, symmetric digest, DHCP poll
lint / lint (push) Successful in 2m18s
test / unit (pull_request) Successful in 1m6s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m18s
Addresses findings from a high-effort review of the PRD 0070 macOS backend.

Correctness:
- Stamp identity_token onto MacosContainerBottlePlan after registration. git's
  gitconfig extraHeader and the supervise MCP --header read
  getattr(plan,"identity_token","") at provision time, and both reach the
  gateway on NO_PROXY (bypassing the egress proxy that carries the token). The
  plan never carried it, so /resolve fail-closed and every git fetch/push and
  supervise call from a macOS bottle would have been denied. Registration
  precedes provision(), so — unlike the run-time env — the plan can carry it.
- Self-heal the orchestrator: recreate when it is not (source-current AND
  answering /health), not on the source-hash label alone. A container running
  current code but with a wedged HTTP server was left alone and polled to
  death, failing every launch until manual deletion.
- image_digest and container_image_digest now read the same descriptor.digest
  field; dropped image_digest's id/tag fallback that could yield a value the
  container side can't produce — a permanent mismatch would have recreated the
  shared gateway on every launch (severing every live bottle's egress, since
  the replacement gets a new DHCP address).
- Poll for the agent's and gateway's DHCP address instead of a fatal read
  right after `container run` (there is no --ip; the address can lag start).

Cleanup:
- One _inspect_first + _descriptor_digest behind the four inspect readers.
- Shared bind_mount_spec (util) and host_db_dir (paths) replace per-module
  copies; _GIT_HTTP_PORT now imports git_http_backend.DEFAULT_PORT.
- Drop the dead _url cache / url property and the write-only agent_proxy_url.

Deferred (noted on the PR, not fixed here): the gateway image rebuilding on
every launch (needs source-hash-labeled build), SQLite shared across VM
guests, and the sh -lc profile-override edge — each is design-level or
behavior-risk beyond a review fix.

Verified: real Apple Container bring-up is green and idempotent; 1826 unit
tests pass with `container` absent (CI parity), pyright clean, pylint 9.86.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 03:26:33 -04:00
didericis a5910696a5 fix(test): stop the macOS unit tests shelling out to the container CLI
lint / lint (push) Successful in 2m26s
test / unit (pull_request) Successful in 1m8s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Successful in 1m20s
CI's unit + coverage jobs failed with `FileNotFoundError: 'container'`: three
tests reached the real Apple CLI, which exists on a macOS dev host but not on
the Linux runner. They passed locally for that reason alone — and two of them
were quietly creating real Apple networks on the dev host as a side effect.

- `test_enumerate_active_is_empty_while_disabled` asserted the disabled-era
  stub and called `enumerate_active()` unmocked. The backend launches bottles
  again, so it now covers the real enumeration: slug parsing, exclusion of the
  shared gateway/orchestrator singletons, and the CLI-failure path.
- The two orchestrator tests patched `orchestrator_service.container_mod`, but
  `_run_orchestrator_container` reaches the CLI through `ensure_networks`,
  which is imported from the gateway module and resolves `container_mod` in
  *its* namespace. Patch the imported name instead.

Adds a test that the networks exist before the orchestrator runs — the
ordering the escaped call was hiding.

Verified by reproducing the CI environment locally (`PATH` without the
`container` binary): 3 failures before, 1818 passing after.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 01:27:40 -04:00
didericis c69642e568 feat(macos): consolidated per-host gateway for the Apple backend (PRD 0070)
Re-enables the macos-container backend on the shared per-host orchestrator +
gateway, replacing the per-bottle companion container removed in #385. This is
the last backend in PRD 0070's roadmap.

Apple Container 1.0.0 forced three departures from the docker shape, each
verified against the live CLI (findings recorded in the networking spike):

- No `--ip`. The address is DHCP-assigned and knowable only once the container
  runs, so the order inverts: gateway up -> run agent -> read its address ->
  register. The identity token is minted by registration and therefore cannot
  be in the agent's run-time env; it rides the proxy URL applied at
  `container exec` time (bare `--env` names keep it off argv).
- No container DNS. The gateway can only be handed the control plane's IP, so
  the orchestrator starts first and the gateway is pointed at its address.
- No `network connect`. Networks are fixed at run time, so the shared host-only
  network is created up front; per-bottle networks would restart the gateway
  on every launch and defeat the consolidation.

The agent runs with `--cap-drop CAP_NET_RAW`: Apple grants NET_RAW by default,
which would let an agent forge a neighbour's source address on the shared
segment. NET_ADMIN is already absent, so this closes the source-address half of
PRD 0070's attribution invariant.

Verified end-to-end on real Apple Container 1.0.0: both images build, the
control plane comes up healthy, the gateway reaches it by IP, and a registered
agent gets 200 for a host in its routes and 403 for one outside them. Bring-up
is idempotent — a second launch does not churn the singletons.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 01:27:40 -04:00
didericis dfc693e0b6 fix(firecracker): keep the snapshot partial private even if one was left behind
test / unit (pull_request) Successful in 1m9s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Successful in 1m14s
lint / lint (push) Successful in 2m21s
test / unit (push) Successful in 1m18s
test / integration (push) Successful in 30s
test / coverage (push) Successful in 1m27s
Update Quality Badges / update-badges (push) Successful in 1m17s
Follow-up to the codex review on #398. os.open's mode arg only applies on
creation, so a committed-rootfs.tar.partial left 0644 by an interrupted run
would be opened/truncated (not re-moded) and stay world-readable for the
whole SSH stream. Unlink any leftover and exclusively recreate it
(O_EXCL|O_NOFOLLOW), then fchmod 0600 immediately so umask can't loosen it.

Test pre-creates a 0644 partial and asserts the fd is 0600 mid-stream.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 01:02:19 -04:00
didericis 39d47b8108 fix(firecracker): harden committed-snapshot resume against guest-controlled data
lint / lint (push) Successful in 2m14s
test / unit (pull_request) Successful in 1m7s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m17s
Address the codex review on #398:

- P1: inject_guest_boot no longer follows a symlink at bb-init/bb-dropbear.
  A committed snapshot is guest-controlled and could plant those paths as
  symlinks aimed at a host file (e.g. bb-init -> ~/.bashrc); write_text /
  copy2 would then overwrite the target as the host user during resume.
  Replace any pre-existing entry and create the files with
  O_EXCL|O_NOFOLLOW so the write stays inside the staging tree.

- P2: write the snapshot tar owner-only (0600). It can contain the bottle's
  private workspace; it was being created world-readable (0644).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 00:44:06 -04:00
didericis d0a0ce8d60 test(firecracker): satisfy pyright strict + line length in committed-rootfs tests
lint / lint (push) Successful in 2m22s
test / unit (pull_request) Successful in 1m20s
test / integration (pull_request) Successful in 28s
test / coverage (pull_request) Successful in 1m20s
Annotate the counting_run subprocess.run wrapper (reportMissingParameterType)
and wrap an over-long patch target line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 00:36:43 -04:00
didericis 5c08701983 feat(firecracker): port freeze/migrate off host Docker (PRD 0069 / #397)
lint / lint (push) Failing after 2m9s
test / unit (pull_request) Successful in 1m7s
test / integration (pull_request) Successful in 24s
test / coverage (pull_request) Successful in 1m22s
The last host-Docker dependency in the Firecracker launch path. Freeze
and resume no longer touch the docker daemon, so the backend needs
firecracker + KVM only — completing #348.

Freeze: stream the guest rootfs over SSH straight into a persistent
committed-rootfs.tar (the resumable/migratable artifact) instead of
round-tripping through `docker build` from a scratch image. Written to
a .partial sibling and atomically renamed so a failed freeze leaves no
truncated artifact.

Resume: extract the snapshot tar into a cached base dir and feed it to
the existing rootless `mke2fs -d` pipeline, replacing the
`docker create` + `docker export | tar` path. Recreate the
proc/sys/dev/run mount points the freezer excludes so the guest init
can mount them.

`util.build_base_rootfs_dir` / `docker_image_id` stay — they still back
the opt-in BOT_BOTTLE_INFRA_BUILD=local dev path and off-host
publish_infra, which are out of scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 00:31:00 -04:00
didericis e3e195f866 chore(firecracker): name the artifact package bot-bottle-firecracker-infra
test / unit (pull_request) Successful in 1m19s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m18s
lint / lint (push) Successful in 2m17s
test / unit (push) Successful in 1m18s
test / integration (push) Successful in 32s
test / coverage (push) Successful in 1m21s
Update Quality Badges / update-badges (push) Successful in 1m17s
Rename the Gitea generic package from bot-bottle-infra to
bot-bottle-firecracker-infra so it's self-evident in the package list which
backend it serves (and leaves room for other artifacts, e.g. a shipped
kernel). The version slot stays the content hash — "firecracker" belongs in
the package name, not the version. Docker image / VM names are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-17 00:04:48 -04:00
didericis e3d24b7e41 chore(firecracker): ship an about.txt description with the infra artifact
lint / lint (push) Successful in 2m55s
test / unit (pull_request) Successful in 1m47s
test / integration (pull_request) Successful in 38s
test / coverage (pull_request) Successful in 1m28s
Generic packages have no description field, so publish_infra now uploads a
short about.txt alongside the rootfs on every publish — it's what identifies
the package as the Firecracker backend's infra rootfs on the package page.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-16 23:59:38 -04:00
didericis f2891a1634 fix(firecracker): hash all baked-in files + stream the artifact upload
lint / lint (push) Successful in 2m26s
test / unit (pull_request) Successful in 1m44s
test / integration (pull_request) Successful in 40s
test / coverage (pull_request) Successful in 1m32s
Address PR #395 review (two P1s):

- Version hash covered only `bot_bottle/**.py`, but the image `COPY`s the
  whole package — non-Python inputs baked in (egress_entrypoint.sh,
  netpool.defaults.env) didn't change the version, so a launch host could
  boot a stale rootfs whose code differs from its checkout. Hash every
  regular file under bot_bottle/ (excluding __pycache__/.pyc). Regression
  tests: a shell-script change bumps the version; .pyc/__pycache__ don't.

- publish_infra `_put` read the whole (hundreds-of-MB) gz into memory via
  read_bytes(). Stream it from disk with an explicit Content-Length; the
  tiny .sha256 stays in-memory. Test asserts the body is the file object,
  not bytes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-16 23:46:46 -04:00
didericis 8d8a88aeeb fix(firecracker): read only BOT_BOTTLE_INFRA_ARTIFACT_TOKEN for the artifact
lint / lint (push) Successful in 2m37s
test / unit (pull_request) Successful in 1m25s
test / integration (pull_request) Successful in 31s
test / coverage (pull_request) Successful in 1m31s
Drop the fallback to the general-purpose BOT_BOTTLE_CLAUDE_GITEA_TOKEN so
the artifact pull/publish uses a dedicated, package-scoped token that can
be granted (or revoked) independently of the general Gitea token.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-16 23:41:53 -04:00
didericis 18f190b7e3 feat(firecracker): pull the infra rootfs as a prebuilt artifact (PRD 0069 Stage 2)
Stage 2 of the docker-free Firecracker backend (#348): stop building the
fixed infra image on the launch host. The infra VM's rootfs is host- and
bottle-agnostic (authorized_keys + guest IP ride the kernel cmdline, not the
rootfs), so it's built once off-host and published as a versioned, ready-to-
boot ext4; the launch host downloads + verifies + boots it — no Docker, no
image tooling, just HTTP + gunzip.

- infra_artifact.py: version = content hash of the rootfs inputs (the shipped
  bot_bottle package + the three Dockerfiles + the init), so a launch host
  pulls the artifact matching its code and a content change can't silently
  boot a stale rootfs. Pull + sha256-verify (fail-closed) + gunzip from a
  Gitea generic package; base/owner/token configurable, default this Gitea.
- infra_vm.ensure_built/boot default to the pull path; BOT_BOTTLE_INFRA_BUILD=
  local keeps the docker build-from-source path for iterating on Dockerfiles.
- publish_infra.py: the off-host half — builds the images with Docker, mke2fs
  the rootfs (with buildah slack), gzips, and PUTs it to the generic package.

Rollout note: default=pull means a launch 404s until an artifact is published;
until the Gitea packages endpoint is enabled + an artifact published, use
BOT_BOTTLE_INFRA_BUILD=local. Freeze/migrate's remaining docker use is a
separate PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
2026-07-16 23:41:53 -04:00
didericis-claude bbb8913382 refactor(git-gate): centralize hostname qualification in globalize_slug
test / unit (pull_request) Successful in 1m19s
test / integration (pull_request) Successful in 24s
test / coverage (pull_request) Successful in 1m24s
lint / lint (push) Successful in 2m37s
test / unit (push) Successful in 1m27s
test / integration (push) Successful in 35s
test / coverage (push) Successful in 1m36s
Update Quality Badges / update-badges (push) Successful in 1m29s
Adds globalize_slug(slug) to bottle_state alongside bottle_identity.
git_gate_provision now calls globalize_slug(slug) instead of inlining
socket.gethostname(), so the hostname-qualification logic has a single,
named home. Assumes slug is a mint_slug output.

Title format changes from bot-bottle:{host}:{slug}:{name}
to bot-bottle:{host}-{slug}:{name} to match the globalize_slug contract.
2026-07-16 23:07:12 -04:00
didericis-claude 59be808ab1 feat(git-gate): include hostname in deploy key title
Closes #388 (part 1 of 3). Deploy key titles now carry the machine
hostname so keys provisioned on different hosts don't collide with
each other on the forge when a prior bottle was never torn down.

Title format: bot-bottle:<hostname>:<slug>:<repo-name>
2026-07-16 23:07:12 -04:00
181 changed files with 12488 additions and 1965 deletions
+4
View File
@@ -1,6 +1,10 @@
[run] [run]
branch = True branch = True
source = . source = .
# Store paths relative to the project root so .coverage.* files produced on
# different runners (ubuntu-latest vs self-hosted KVM) can be combined by the
# coverage job without a [paths] remapping section.
relative_files = True
[report] [report]
# Coverage policy: see docs/decisions/0004-coverage-policy.md. # Coverage policy: see docs/decisions/0004-coverage-policy.md.
+2 -5
View File
@@ -22,10 +22,7 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python # No actions/setup-python: canaries are stdlib unittest on the image's
uses: actions/setup-python@v5 # system Python 3.12 (older act_runner mishandles setup-python's PATH).
with:
python-version: "3.12"
- name: Run canaries - name: Run canaries
run: python3 -m unittest discover -t . -s tests/canaries -v run: python3 -m unittest discover -t . -s tests/canaries -v
+20 -10
View File
@@ -13,20 +13,30 @@ jobs:
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python # No actions/setup-python: the runner image already ships Python 3.12,
uses: actions/setup-python@v4 # and older act_runner engines mishandle setup-python's PATH. Install
with: # into the ephemeral job container's system Python — the pylint/pyright
python-version: "3.12" # console scripts land on /usr/local/bin (on PATH) so the steps below
# still resolve. --break-system-packages is safe: the container is
# disposable.
- name: Install dev dependencies - name: Install dev dependencies
run: | run: python3 -m pip install --break-system-packages -r requirements-dev.txt
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Run pylint - name: Run pylint
run: | run: |
# Run pylint on all Python files in the repo # Pylint's normal exit code is nonzero for any emitted finding,
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0 # regardless of --fail-under. Preserve the full report but enforce
# the aggregate score this workflow promises.
set +e
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' \
| xargs pylint --fail-under=8.0 \
| tee /tmp/pylint-output.txt
set -e
SCORE=$(sed -n \
's/^Your code has been rated at \([-0-9.]*\)\/10.*/\1/p' \
/tmp/pylint-output.txt | tail -1)
test -n "$SCORE"
awk -v score="$SCORE" 'BEGIN { exit !(score >= 8.0) }'
- name: Run pyright - name: Run pyright
run: | run: |
+2 -5
View File
@@ -37,11 +37,8 @@ jobs:
fetch-depth: 0 fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Python # No actions/setup-python: the inline script is stdlib-only on the
uses: actions/setup-python@v5 # image's system Python 3.12 (older act_runner mishandles its PATH).
with:
python-version: "3.12"
- name: Configure git - name: Configure git
run: | run: |
git config user.name "github-actions[bot]" git config user.name "github-actions[bot]"
+243 -39
View File
@@ -1,19 +1,21 @@
# Run the project's test suite on every PR push and on push to main. # Run the project's test suite when package or runtime inputs change on a PR
# or on push to main.
# #
# The suite uses stdlib `unittest` discovery — no external Python # The suite uses stdlib `unittest` discovery — no external Python
# dependencies are required to execute it. Tests are split by directory: # dependencies are required to execute it. Tests are split by directory:
# #
# tests/unit/ — pure unit tests; always run # tests/unit/ — pure unit tests; always run
# tests/integration/ — need a reachable Docker daemon; skip cleanly # tests/integration/ — need a reachable backend; skip cleanly when
# (via tests/_docker.py:skip_unless_docker) when # the backend isn't available on the runner
# Docker isn't available on the runner
# tests/canaries/ — upstream regression canaries; run on a separate # tests/canaries/ — upstream regression canaries; run on a separate
# schedule (see canaries.yml), not here # schedule (see canaries.yml), not here
# #
# This workflow assumes the Gitea Actions runner exposes the host Docker # Each test job runs once under coverage and uploads a small .coverage.*
# socket to the job container so `docker` commands inside the job can # artifact. The `coverage` job combines them — no test reruns, no KVM
# reach the daemon. If that's not yet configured on the runner the # dependency on that job. For main-branch pushes only, the tested rootfs
# integration tests will skip rather than fail. # and matching dropbear are uploaded so `publish-infra` can publish the
# byte-identical artifact that was tested. PRs avoid the ~194 MB rootfs
# transfer entirely.
name: test name: test
@@ -22,10 +24,35 @@ on:
branches: branches:
- main - main
paths: paths:
- '**.py' - 'bot_bottle/**'
- 'tests/**/*.py'
- 'cli.py'
- 'scripts/coverage.sh'
- 'scripts/critical-modules.txt'
- 'scripts/diff_coverage.py'
- 'scripts/tracker_policy.py'
- 'scripts/firecracker-netpool.sh'
- 'Dockerfile*'
- 'pyproject.toml'
- 'requirements-dev.txt'
- '.coveragerc'
- '.dockerignore'
pull_request: pull_request:
paths: paths:
- '**.py' - 'bot_bottle/**'
- 'tests/**/*.py'
- 'cli.py'
- 'scripts/coverage.sh'
- 'scripts/critical-modules.txt'
- 'scripts/diff_coverage.py'
- 'scripts/tracker_policy.py'
- 'scripts/firecracker-netpool.sh'
- 'Dockerfile*'
- 'pyproject.toml'
- 'requirements-dev.txt'
- '.coveragerc'
- '.dockerignore'
workflow_dispatch:
jobs: jobs:
unit: unit:
@@ -34,30 +61,47 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python # No actions/setup-python: the runner image already ships Python 3.12,
uses: actions/setup-python@v5 # and older act_runner engines mishandle setup-python's PATH (coverage
with: # lands in one interpreter, `python3` resolves to another). Install
python-version: "3.12" # straight into the ephemeral job container's system Python —
# --break-system-packages is safe because the container is disposable.
- name: Install dev requirements - name: Install dev requirements
run: python3 -m pip install -r requirements-dev.txt run: python3 -m pip install --break-system-packages -r requirements-dev.txt
- name: Run unit tests - name: Run unit tests with coverage
env:
COVERAGE_FILE: ${{ github.workspace }}/.coverage.unit
run: python3 -m coverage run -m unittest discover -t . -s tests/unit -v run: python3 -m coverage run -m unittest discover -t . -s tests/unit -v
- name: Report unit coverage - name: Report unit coverage
env:
COVERAGE_FILE: ${{ github.workspace }}/.coverage.unit
run: python3 -m coverage report -m run: python3 -m coverage report -m
integration: # upload-artifact@v3's glob skips dotfiles, so a bare `.coverage.unit`
# silently uploads nothing ("No files were found"). Stage it under a
# non-dot name; the coverage job renames it back before `coverage
# combine`. `cp` also fails loudly if coverage never wrote the file.
- name: Stage unit coverage for upload
run: cp .coverage.unit coverage-unit.dat
- name: Upload unit coverage artifact
uses: actions/upload-artifact@v3
with:
name: coverage-unit
path: coverage-unit.dat
integration-docker:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python # No actions/setup-python (see the note in the `unit` job); the
uses: actions/setup-python@v5 # container's system Python 3.12 runs the stdlib test suite directly.
with: - name: Install coverage
python-version: "3.12" run: python3 -m pip install --break-system-packages coverage
- name: Show environment - name: Show environment
run: | run: |
@@ -68,33 +112,193 @@ jobs:
echo "docker not on PATH — integration tests will skip" echo "docker not on PATH — integration tests will skip"
fi fi
- name: Run integration tests - name: Run integration tests (docker) with coverage
run: python3 -m unittest discover -t . -s tests/integration -v env:
BOT_BOTTLE_BACKEND: docker
COVERAGE_FILE: ${{ github.workspace }}/.coverage.docker
run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
# Combined unit+integration coverage report (informational). See # Non-dot name so upload-artifact's dotfile-skipping glob picks it up.
# docs/decisions/0004-coverage-policy.md. - name: Stage docker coverage for upload
run: cp .coverage.docker coverage-docker.dat
- name: Upload docker coverage artifact
uses: actions/upload-artifact@v3
with:
name: coverage-docker
path: coverage-docker.dat
# Integration tests against the Firecracker backend. Runs on a self-hosted
# KVM runner (label `kvm`) where /dev/kvm and the TAP/nft pool are available.
# #
# The hard diff-coverage gate (changed lines >= 90%) is DEFERRED: the # Restricted to same-repo PRs, push to main, and workflow_dispatch — fork
# Firecracker backend's VM/SSH orchestration is covered by the integration # PRs don't execute untrusted code on the privileged runner.
# suite, which needs /dev/kvm + the provisioned TAP/nft pool — a #
# container-based runner skips it and those lines read uncovered, so the # Runner prerequisites (provision once; see README "Firecracker on Linux"):
# gate can't pass here. Re-enabling it on a self-hosted KVM runner is # `firecracker` on PATH, `/dev/kvm` accessible, cached kernel +
# tracked separately (see PRD 0069 / #348 and the ci-runner branch). # static dropbear at /var/cache/bot-bottle-fc/dropbear, and the pool as a
# persistent systemd unit.
#
# The infra candidate is built here directly (no artifact download) to
# eliminate the ~70 s ubuntu-latest upload + ~83 s combined download that
# the old build-infra → integration-firecracker + coverage chain incurred.
# For main-branch pushes the tested rootfs and matching dropbear are
# uploaded so publish-infra can publish the byte-identical artifact; PRs
# skip those uploads entirely.
integration-firecracker:
runs-on: [self-hosted, kvm]
if: >-
github.event_name == 'push' ||
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository)
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Preflight — Firecracker host is ready
run: |
command -v firecracker >/dev/null || {
echo "firecracker not on PATH — provision the runner (README: Firecracker on Linux)"; exit 1; }
test -e /dev/kvm || { echo "/dev/kvm missing — KVM not available on this runner"; exit 1; }
# `backend status` exits non-zero unless the TAP pool is up + no
# range overlap; it prints the exact `backend setup` fix.
python3 cli.py backend status --backend=firecracker
- name: Build infra candidate from this checkout
env:
BOT_BOTTLE_FC_DROPBEAR: /var/cache/bot-bottle-fc/dropbear
run: python3 -m bot_bottle.backend.firecracker.publish_infra --output infra-candidate --reuse-published
- name: Replace the persistent infra VM with the candidate
run: python3 -c 'from bot_bottle.backend.firecracker import infra_vm; infra_vm.stop()'
# No dev-requirements install: `coverage` is already provided by the
# self-hosted runner's Nix python env, and that env has no `pip`
# module to install into anyway.
- name: Run integration tests (firecracker) with coverage
env:
BOT_BOTTLE_BACKEND: firecracker
BOT_BOTTLE_INFRA_ARTIFACT_DIR: ${{ github.workspace }}/infra-candidate
COVERAGE_FILE: ${{ github.workspace }}/.coverage.firecracker
run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
# Non-dot name so upload-artifact's dotfile-skipping glob picks it up.
- name: Stage firecracker coverage for upload
run: cp .coverage.firecracker coverage-firecracker.dat
- name: Upload firecracker coverage artifact
uses: actions/upload-artifact@v3
with:
name: coverage-firecracker
path: coverage-firecracker.dat
# Only upload the large rootfs artifact on main-branch pushes;
# PRs avoid the ~194 MB transfer. publish-infra only runs on main
# and downloads these to publish the byte-identical tested rootfs.
- name: Upload tested rootfs (main branch only)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: actions/upload-artifact@v3
with:
name: infra-candidate
path: infra-candidate/
- name: Upload dropbear for publish verification (main branch only)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: actions/upload-artifact@v3
with:
name: firecracker-inputs
path: /var/cache/bot-bottle-fc/dropbear
# Combined coverage gate: aggregates .coverage.* artifacts uploaded by each
# test job, then runs the diff-coverage gate (new/changed lines >= 90%).
#
# Runs on ubuntu-latest — no KVM needed, no test reruns. Coverage files use
# relative_files = True (.coveragerc) so they combine cleanly across runners.
# Each test job sets COVERAGE_FILE to an absolute path so coverage.py writes
# to a known location that upload-artifact can find regardless of runner env.
#
# Restricted to the same events as integration-firecracker: it depends on
# that job's coverage artifact and skips for fork PRs alongside it.
coverage: coverage:
needs: [unit, integration-docker, integration-firecracker]
timeout-minutes: 15
runs-on: ubuntu-latest runs-on: ubuntu-latest
if: >-
github.event_name == 'push' ||
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository)
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Set up Python - name: Install coverage
uses: actions/setup-python@v5 run: python3 -m pip install --break-system-packages coverage
- name: Download unit coverage artifact
uses: actions/download-artifact@v3
with: with:
python-version: "3.12" name: coverage-unit
path: ${{ github.workspace }}
- name: Install dev requirements - name: Download docker coverage artifact
run: python3 -m pip install -r requirements-dev.txt uses: actions/download-artifact@v3
with:
name: coverage-docker
path: ${{ github.workspace }}
- name: Combined coverage report (unit + integration) - name: Download firecracker coverage artifact
run: PYTHON=python3 bash scripts/coverage.sh critical uses: actions/download-artifact@v3
with:
name: coverage-firecracker
path: ${{ github.workspace }}
# Rename the non-dot upload names back to the .coverage.* files that
# `coverage combine` discovers (see the staging steps in each test job).
- name: Reassemble coverage data files
run: |
mv coverage-unit.dat .coverage.unit
mv coverage-docker.dat .coverage.docker
mv coverage-firecracker.dat .coverage.firecracker
- name: Combined coverage (unit + integration, incl. firecracker)
run: PYTHON=python3 bash scripts/coverage.sh aggregate critical
- name: Diff-coverage gate (changed lines >= 90%)
run: |
git fetch --no-tags origin main:refs/remotes/origin/main
python3 scripts/diff_coverage.py --base origin/main --min 90
publish-infra:
needs: [unit, integration-docker, integration-firecracker, coverage]
runs-on: ubuntu-latest
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps:
- name: Checkout the tested revision
uses: actions/checkout@v4
- name: Download the tested rootfs
uses: actions/download-artifact@v3
with:
name: infra-candidate
path: infra-candidate
# publish_infra re-derives the version from the checkout to confirm the
# bundle matches before uploading, and the version hashes the dropbear
# bytes. Download the SAME dropbear integration-firecracker used, or
# the recheck computes a "<missing>"-dropbear version and rejects the
# candidate.
- name: Download the staged dropbear (matches build's version)
uses: actions/download-artifact@v3
with:
name: firecracker-inputs
path: firecracker-inputs
- name: Publish the tested candidate
env:
BOT_BOTTLE_INFRA_ARTIFACT_TOKEN: ${{ secrets.BOT_BOTTLE_INFRA_ARTIFACT_TOKEN }}
BOT_BOTTLE_FC_DROPBEAR: ${{ github.workspace }}/firecracker-inputs/dropbear
run: python3 -m bot_bottle.backend.firecracker.publish_infra --publish-dir infra-candidate
@@ -0,0 +1,17 @@
name: tracker-policy-issues
on:
issues:
types: [opened, unlabeled]
jobs:
label-issue:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/checkout@v4
- name: Ensure the issue has a label
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 scripts/tracker_policy.py label-issue
+18
View File
@@ -0,0 +1,18 @@
name: tracker-policy-pr
on:
pull_request:
types: [opened, edited, reopened, synchronize, labeled, unlabeled]
jobs:
check-pr:
runs-on: ubuntu-latest
permissions:
issues: read
pull-requests: read
steps:
- uses: actions/checkout@v4
- name: Require an unlabeled PR linked to an issue
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 scripts/tracker_policy.py check-pr
+12 -12
View File
@@ -14,27 +14,27 @@ on:
jobs: jobs:
update-badges: update-badges:
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
contents: write
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
with: with:
fetch-depth: 0 fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.BADGE_PUSH_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'
# No actions/setup-python: the runner image ships Python 3.12 and older
# act_runner engines mishandle setup-python's PATH. Install into the
# ephemeral job container's system Python (--break-system-packages is
# safe because the container is disposable).
- name: Install dev dependencies - name: Install dev dependencies
run: | run: python3 -m pip install --break-system-packages -r requirements-dev.txt
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Run coverage and extract percentage - name: Run coverage and extract percentage
id: coverage id: coverage
run: | run: |
python -m coverage run -m unittest discover -t . -s tests/unit > /dev/null 2>&1 || true python3 -m coverage run -m unittest discover -t . -s tests/unit > /dev/null 2>&1 || true
PERCENT=$(python -m coverage report 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1) PERCENT=$(python3 -m coverage report 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
echo "percent=$PERCENT" >> $GITHUB_OUTPUT echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Coverage: $PERCENT%" echo "Coverage: $PERCENT%"
@@ -45,7 +45,7 @@ jobs:
# the single source of truth in scripts/critical-modules.txt; every # the single source of truth in scripts/critical-modules.txt; every
# core module is unit-tested, so the unit-only run is accurate for it. # core module is unit-tested, so the unit-only run is accurate for it.
INCLUDE=$(grep -vE '^[[:space:]]*(#|$)' scripts/critical-modules.txt | paste -sd, -) INCLUDE=$(grep -vE '^[[:space:]]*(#|$)' scripts/critical-modules.txt | paste -sd, -)
PERCENT=$(python -m coverage report --include="$INCLUDE" 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1) PERCENT=$(python3 -m coverage report --include="$INCLUDE" 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
echo "percent=$PERCENT" >> $GITHUB_OUTPUT echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Core coverage: $PERCENT%" echo "Core coverage: $PERCENT%"
+35 -32
View File
@@ -16,10 +16,11 @@
# Layout: # Layout:
# #
# /usr/bin/gitleaks gitleaks binary # /usr/bin/gitleaks gitleaks binary
# /app/egress_addon.py + siblings mitmproxy addon (egress) # /app/egress_addon.py mitmproxy addon entry point
# /app/egress-entrypoint.sh mitmdump launcher # /app/egress-entrypoint.sh mitmdump launcher
# /app/supervise_server.py + .py supervise MCP server # /usr/local/lib/python*/bot_bottle/ installed package (all daemons + shared modules)
# /app/gateway_init.py PID 1 supervisor # /app/egress_addon.py one-line shim: re-exports addons from package
# (mitmdump -s requires a file path, not a module)
# /etc/egress/routes.yaml bind-mounted at run time # /etc/egress/routes.yaml bind-mounted at run time
# /etc/git-gate/pre-receive docker-cp'd at start time # /etc/git-gate/pre-receive docker-cp'd at start time
# /git-gate-entrypoint.sh docker-cp'd at start time # /git-gate-entrypoint.sh docker-cp'd at start time
@@ -66,35 +67,41 @@ RUN pip install --no-cache-dir mitmproxy==11.1.3
# would pin us to that image's cadence). python (already present) does the # would pin us to that image's cadence). python (already present) does the
# download so we add no curl/wget. trixie apt also ships gitleaks, but an # download so we add no curl/wget. trixie apt also ships gitleaks, but an
# older 8.16; the pinned download keeps the verified 8.30.1. # older 8.16; the pinned download keeps the verified 8.30.1.
#
# Arch-aware: the asset + SHA are picked from the build's target
# architecture so an arm64 host (Apple Silicon) gets the arm64 binary
# rather than an x86_64 one that dies with "Exec format error" the first
# time the pre-receive hook runs it. TARGETARCH is auto-populated by
# BuildKit; the dpkg fallback keeps it correct under a legacy builder.
ARG GITLEAKS_VERSION=8.30.1 ARG GITLEAKS_VERSION=8.30.1
ARG GITLEAKS_SHA256=551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb ARG GITLEAKS_SHA256_AMD64=551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb
RUN url="https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz" \ ARG GITLEAKS_SHA256_ARM64=e4a487ee7ccd7d3a7f7ec08657610aa3606637dab924210b3aee62570fb4b080
ARG TARGETARCH
RUN arch="${TARGETARCH:-$(dpkg --print-architecture)}" \
&& case "$arch" in \
amd64) asset="linux_x64"; sha="${GITLEAKS_SHA256_AMD64}" ;; \
arm64) asset="linux_arm64"; sha="${GITLEAKS_SHA256_ARM64}" ;; \
*) echo "unsupported gitleaks target arch: $arch" >&2; exit 1 ;; \
esac \
&& url="https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_${asset}.tar.gz" \
&& python3 -c "import sys,urllib.request; urllib.request.urlretrieve(sys.argv[1], '/tmp/gitleaks.tar.gz')" "$url" \ && python3 -c "import sys,urllib.request; urllib.request.urlretrieve(sys.argv[1], '/tmp/gitleaks.tar.gz')" "$url" \
&& echo "${GITLEAKS_SHA256} /tmp/gitleaks.tar.gz" | sha256sum -c - \ && echo "${sha} /tmp/gitleaks.tar.gz" | sha256sum -c - \
&& tar -xzf /tmp/gitleaks.tar.gz -C /usr/bin gitleaks \ && tar -xzf /tmp/gitleaks.tar.gz -C /usr/bin gitleaks \
&& rm /tmp/gitleaks.tar.gz && rm /tmp/gitleaks.tar.gz
# Project Python: addon + server modules + the init supervisor. # Install bot_bottle as a proper package so entry-point scripts can use
# Kept flat under /app/ so mitmdump's loader resolves them as # `from bot_bottle.X import Y` absolute imports. A rename or a missing
# top-level siblings (absolute imports), matching the prior # module is caught at pip-install time — not at container runtime.
# Dockerfile.egress / Dockerfile.supervise layout. COPY pyproject.toml /src/
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py COPY bot_bottle/ /src/bot_bottle/
COPY bot_bottle/egress_dlp_config.py /app/egress_dlp_config.py RUN pip install --no-cache-dir /src/
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/policy_resolver.py /app/policy_resolver.py # mitmdump -s requires a file path, not a module. Write a one-line shim that
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py # re-exports `addons` from the installed package; mitmdump finds it there.
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py # WORKDIR here also creates /app so the shim + COPYs below can write into it
COPY bot_bottle/paths.py /app/paths.py # (nothing created /app before this point).
COPY bot_bottle/migrations.py /app/migrations.py WORKDIR /app
COPY bot_bottle/db_store.py /app/db_store.py RUN printf 'from bot_bottle.egress_addon import addons\n' > /app/egress_addon.py
COPY bot_bottle/supervise_types.py /app/supervise_types.py
COPY bot_bottle/queue_store.py /app/queue_store.py
COPY bot_bottle/audit_store.py /app/audit_store.py
COPY bot_bottle/store_manager.py /app/store_manager.py
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/gateway_init.py /app/gateway_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
RUN chmod +x /app/egress-entrypoint.sh RUN chmod +x /app/egress-entrypoint.sh
@@ -113,10 +120,6 @@ RUN mkdir -p \
# subset the bottle uses. # subset the bottle uses.
EXPOSE 8888 9099 9418 9420 9100 EXPOSE 8888 9099 9418 9420 9100
# WORKDIR matches Dockerfile.supervise's prior layout so the
# in-app same-dir import in supervise_server.py stays deterministic.
WORKDIR /app
# PID 1 is the supervisor. It owns signal handling and exit-code # PID 1 is the supervisor. It owns signal handling and exit-code
# propagation; no `exec` chain in the entrypoint itself. # propagation; no `exec` chain in the entrypoint itself.
ENTRYPOINT ["python3", "/app/gateway_init.py"] ENTRYPOINT ["python3", "-m", "bot_bottle.gateway_init"]
+13 -36
View File
@@ -1,45 +1,22 @@
# Firecracker single infra-VM image (PRD 0070 Stage B). # Shared infra image: gateway data plane + orchestrator control plane.
# #
# The per-host infra VM runs the orchestrator control plane, the gateway # Used directly by the Docker backend (run as one `bot-bottle-infra`
# data plane, AND builds agent images (buildah) — all in one microVM (see # container, replacing the prior two-container split). The Firecracker
# backend/firecracker/infra_vm.py). It composes: # backend extends this via Dockerfile.infra.fc, adding buildah/crun/
# * FROM the gateway image (mitmproxy / git / gitleaks / supervise + the # netavark for in-VM agent-image building.
# flat daemon modules) — now trixie-based, so buildah 1.39 is available; #
# * `COPY --from` the orchestrator image's content (the single definition # Dockerfile.orchestrator is the single definition of the orchestrator
# of the control-plane payload — see Dockerfile.orchestrator), so this # content (the lean `bot_bottle` package on python:3.12-slim). Both this
# VM and the docker backend share one orchestrator definition; and # image and Dockerfile.infra.fc pull it in via `COPY --from`.
# * buildah, installed HERE only (the docker orchestrator/gateway images
# never carry it).
# #
# multi-`FROM` can't union two bases (that's multi-stage, not multiple # multi-`FROM` can't union two bases (that's multi-stage, not multiple
# inheritance), so the orchestrator content is pulled in via `COPY --from` # inheritance), so the orchestrator content is pulled in via `COPY --from`
# rather than a second base. Both images share the trixie `python:3.12-slim` # rather than a second base. Both images share the trixie `python:3.12-slim`
# base, so the copy is clean (same python; future installed deps copy too). # base, so the copy is clean (same python; future installed deps copy too).
#
# The docker backend keeps orchestrator + gateway as separate images; this
# combined image exists only for the Firecracker single-VM cut. Splitting a
# service back into its own VM later is a routing change, not a repackaging
# (PRD 0070's "secret concentration"; a disposable builder can boot from
# this same image on its own TAP).
FROM bot-bottle-gateway:latest FROM bot-bottle-gateway:latest
# --- in-VM agent-image builder (PRD 0069 Stage 3) ------------------- # The orchestrator content, from its single definition. The gateway image
# The Firecracker backend builds users' agent Dockerfiles *inside this VM* # already has the flat daemon modules under /app; this adds the full
# with buildah (rootless, daemonless) instead of on the host — no host # `bot_bottle` package so `python3 -m bot_bottle.orchestrator` resolves —
# Docker daemon, no root-equivalent `docker` group. `crun` is the OCI # used by gateway_init when BOT_BOTTLE_GATEWAY_DAEMONS includes `orchestrator`.
# runtime; `netavark` + `aardvark-dns` are the network backend for `FROM`
# pulls + `RUN` egress. Requires the trixie base (buildah 1.39: bookworm's
# 1.28 can't parse Dockerfile heredocs that agent images use).
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
buildah crun netavark aardvark-dns \
&& rm -rf /var/lib/apt/lists/*
# vfs + chroot: buildah works as root in the bare microVM (no
# fuse-overlayfs / overlay module / subuid maps). Matches image_builder.
ENV STORAGE_DRIVER=vfs \
BUILDAH_ISOLATION=chroot
# The orchestrator content, pulled from its single definition. The gateway
# image already has the flat daemon modules under /app; this adds the full
# `bot_bottle` package so `python3 -m bot_bottle.orchestrator` resolves.
COPY --from=bot-bottle-orchestrator:latest /app/bot_bottle /app/bot_bottle COPY --from=bot-bottle-orchestrator:latest /app/bot_bottle /app/bot_bottle
+23
View File
@@ -0,0 +1,23 @@
# Firecracker infra VM image (PRD 0070 Stage B).
#
# Extends the shared infra base (Dockerfile.infra: gateway + orchestrator
# control plane) with the in-VM agent-image builder. The Firecracker backend
# builds users' agent Dockerfiles *inside this VM* with buildah (rootless,
# daemonless) instead of on the host — no host Docker daemon, no
# root-equivalent `docker` group.
#
# Requires the trixie base from bot-bottle-gateway (buildah 1.39: bookworm's
# 1.28 can't parse Dockerfile heredocs that agent images use).
#
# `crun` is the OCI runtime; `netavark` + `aardvark-dns` are the network
# backend for `FROM` pulls + `RUN` egress. `vfs` + `chroot`: buildah works
# as root in the bare microVM (no fuse-overlayfs / overlay module / subuid
# maps). Matches image_builder.
FROM bot-bottle-infra:latest
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
buildah crun netavark aardvark-dns \
&& rm -rf /var/lib/apt/lists/*
ENV STORAGE_DRIVER=vfs \
BUILDAH_ISOLATION=chroot
+13 -2
View File
@@ -5,8 +5,8 @@
# bot-bottle # bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml) [![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-82%25-brightgreen)](https://coverage.readthedocs.io/) [![coverage](https://img.shields.io/badge/coverage-83%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-95%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md) [![core coverage](https://img.shields.io/badge/core%20coverage-94%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data. **Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -90,6 +90,8 @@ BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`. > **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
> **CI:** the coverage gate (`.gitea/workflows/test.yml` → `coverage` job) runs on a self-hosted runner labelled `kvm`, because the Firecracker backend's VM/SSH orchestration is exercised only by the integration suite, which needs `/dev/kvm` + the provisioned pool (a container runner would skip it and read as uncovered). Provision that runner exactly like a normal Firecracker host — `firecracker` on `PATH`, `/dev/kvm`, the cached guest kernel + static dropbear, and the pool installed as the persistent systemd unit — then register it with the `kvm` label. A Docker-capable hosted job builds the candidate once; KVM tests boot those exact bytes, and a successful main run publishes them. The unit/lint jobs still run on `ubuntu-latest`.
```sh ```sh
./cli.py start <agent> # builds the image on first run, drops you into claude ./cli.py start <agent> # builds the image on first run, drops you into claude
``` ```
@@ -171,6 +173,15 @@ When an outbound DLP detector matches a token, the route's `dlp.outbound_on_matc
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`. More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Tracker policy
Issues are the canonical work items and own all tracker labels; every issue
must have at least one. Pull requests stay unlabeled and deliberately reference
an issue with `Closes #…`, `Part of #…`, or another form defined in
[`ADR 0005`](docs/decisions/0005-issues-own-tracker-metadata.md). Gitea Actions
enforces the convention for new work from 2026-07-18 onward. Earlier closed
PRs are grandfathered rather than given artificial retrospective issues.
## Trademarks ## Trademarks
bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox. bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox.
+43 -2
View File
@@ -45,6 +45,10 @@ PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_PI})
# forward_host_credentials is enabled. Pipelock must pass these through # forward_host_credentials is enabled. Pipelock must pass these through
# (no TLS MITM) or its header DLP blocks the injected JWT. # (no TLS MITM) or its header DLP blocks the injected JWT.
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com") CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
# Host that egress injects the host Claude bearer on when Claude
# forward_host_credentials is enabled.
CLAUDE_HOST_CREDENTIAL_HOSTS = ("api.anthropic.com",)
PromptMode = Literal[ PromptMode = Literal[
"append_file", "append_file",
"read_prompt_file", "read_prompt_file",
@@ -257,7 +261,28 @@ class AgentProvider(ABC):
Default: Debian/node — writes the git-gate insteadOf gitconfig Default: Debian/node — writes the git-gate insteadOf gitconfig
and sets user.name/email as node. Workspace copy runs through and sets user.name/email as node. Workspace copy runs through
BottleBackend.provision_workspace against the running bottle.""" BottleBackend.provision_workspace against the running bottle."""
from .log import info from .log import die, info
# Firecracker exports image rootfs files through an unprivileged host
# tar extraction, so image-time ownership of XDG directories is not
# preserved. Git consults ~/.config/git even when the actual config
# is ~/.gitconfig; an unreadable directory there can prevent the
# git-gate insteadOf rules below from taking effect. Repair this at
# runtime, after every backend's copy/export path has completed.
git_xdg_dir = f"{plan.guest_home}/.config/git"
repair = bottle.exec(
f"chown node:node {shlex.quote(plan.guest_home)} && "
f"chmod 755 {shlex.quote(plan.guest_home)} && "
f"mkdir -p {shlex.quote(git_xdg_dir)} && "
f"chown -R node:node {shlex.quote(f'{plan.guest_home}/.config')} && "
f"chmod -R u+rwX,go+rX {shlex.quote(f'{plan.guest_home}/.config')}",
user="root",
)
if repair.returncode != 0:
die(
"git provisioning: could not make the runtime Git config "
f"directory readable: {(repair.stderr or repair.stdout).strip()}"
)
manifest_bottle = plan.manifest.bottle manifest_bottle = plan.manifest.bottle
if manifest_bottle.git: if manifest_bottle.git:
@@ -280,11 +305,27 @@ class AgentProvider(ABC):
f"{len(manifest_bottle.git)} insteadOf rule(s)" f"{len(manifest_bottle.git)} insteadOf rule(s)"
) )
bottle.cp_in(str(config_file), guest_gitconfig) bottle.cp_in(str(config_file), guest_gitconfig)
bottle.exec( permissions = bottle.exec(
f"chown node:node {shlex.quote(guest_gitconfig)} && " f"chown node:node {shlex.quote(guest_gitconfig)} && "
f"chmod 644 {shlex.quote(guest_gitconfig)}", f"chmod 644 {shlex.quote(guest_gitconfig)}",
user="root", user="root",
) )
if permissions.returncode != 0:
die(
"git provisioning: could not set ownership on "
f"{guest_gitconfig}: "
f"{(permissions.stderr or permissions.stdout).strip()}"
)
configured = bottle.exec(
"git config --global --get-regexp '^url\\..*\\.insteadof$'",
user="node",
)
if configured.returncode != 0:
die(
"git provisioning: the runtime user cannot read the "
f"git-gate insteadOf rules from {guest_gitconfig}: "
f"{(configured.stderr or configured.stdout).strip()}"
)
gu = manifest_bottle.git_user gu = manifest_bottle.git_user
if not gu.is_empty(): if not gu.is_empty():
+2 -2
View File
@@ -42,7 +42,7 @@ class AuditStore(DbStore):
super().__init__(db_path or host_db_path(), migrations) super().__init__(db_path or host_db_path(), migrations)
def write_audit_entry(self, entry: AuditEntry) -> Path: def write_audit_entry(self, entry: AuditEntry) -> Path:
with self._connect() as conn: with self._connection() as conn:
conn.execute( conn.execute(
""" """
INSERT INTO supervise_audit_entries ( INSERT INTO supervise_audit_entries (
@@ -66,7 +66,7 @@ class AuditStore(DbStore):
def read_audit_entries(self, component: str, slug: str) -> list[AuditEntry]: def read_audit_entries(self, component: str, slug: str) -> list[AuditEntry]:
if not self.db_path.is_file(): if not self.db_path.is_file():
return [] return []
with self._connect() as conn: with self._connection() as conn:
rows = conn.execute( rows = conn.execute(
""" """
SELECT * FROM supervise_audit_entries SELECT * FROM supervise_audit_entries
+178 -37
View File
@@ -37,15 +37,16 @@ import os
import shlex import shlex
import sys import sys
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from contextlib import AbstractContextManager from contextlib import AbstractContextManager, contextmanager
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any, Generic, Sequence, TypeVar from typing import TYPE_CHECKING, Any, Generator, Generic, Sequence, TypeVar
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
from ..egress import EgressPlan from ..egress import EgressPlan
from ..git_gate import GitGatePlan from ..git_gate import GitGatePlan
from ..log import die, info from ..log import die, info, warn
from ..util import read_tty_line
from ..manifest import Manifest, ManifestIndex from ..manifest import Manifest, ManifestIndex
from ..supervise import SupervisePlan from ..supervise import SupervisePlan
from ..util import expand_tilde from ..util import expand_tilde
@@ -54,6 +55,9 @@ from ..workspace import WorkspacePlan, workspace_plan
from .print_util import print_multi, visible_agent_env_names from .print_util import print_multi, visible_agent_env_names
from .util import host_skill_dir from .util import host_skill_dir
if TYPE_CHECKING:
from .freeze import CommitCancelled, Freezer, get_freezer
@dataclass(frozen=True) @dataclass(frozen=True)
class BottleSpec: class BottleSpec:
@@ -79,6 +83,9 @@ class BottleSpec:
# True when launched via --headless (no TTY, no interactive prompts). # True when launched via --headless (no TTY, no interactive prompts).
# The git-gate host-key preflight uses this to error rather than prompt. # The git-gate host-key preflight uses this to error rather than prompt.
headless: bool = False headless: bool = False
# Image startup policy. "fresh" preserves the normal build path;
# "cached" reuses the current local image/artifact without rebuilding.
image_policy: str = "fresh"
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -274,6 +281,18 @@ PlanT = TypeVar("PlanT", bound=BottlePlan)
CleanupT = TypeVar("CleanupT", bound=BottleCleanupPlan) CleanupT = TypeVar("CleanupT", bound=BottleCleanupPlan)
@dataclass(frozen=True)
class BottleImages:
"""Resolved image references (or artifact paths) for a bottle launch.
For Docker/macOS-container backends, `agent` and `sidecar` are string
image refs. For the smolmachines backend they are Path objects pointing
to pre-built `.smolmachine` artifacts."""
agent: str | Path
sidecar: str | Path = ""
class BottleBackend(ABC, Generic[PlanT, CleanupT]): class BottleBackend(ABC, Generic[PlanT, CleanupT]):
"""Abstract base for selectable bottle backends. Concrete subclasses """Abstract base for selectable bottle backends. Concrete subclasses
(e.g. DockerBottleBackend) own their own prepare/launch impls. (e.g. DockerBottleBackend) own their own prepare/launch impls.
@@ -433,9 +452,27 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
prompt file, Dockerfile path, and guest home all live on prompt file, Dockerfile path, and guest home all live on
`agent_provision_plan` — the source of truth.""" `agent_provision_plan` — the source of truth."""
def prelaunch_checks(self, plan: PlanT) -> None:
"""Raise StaleImageError if any cached image used by this plan is stale.
No-op default; backends override to call the shared check_stale*
helpers on their image/artifact timestamps. Called by the CLI before
launch so the operator can be prompted outside the launch context."""
@contextmanager
def launch(self, plan: PlanT) -> Generator[Bottle, None, None]:
"""Template: build or load images, then delegate to _launch_impl."""
images = self._build_or_load_images(plan)
with self._launch_impl(plan, images) as bottle:
yield bottle
@abstractmethod @abstractmethod
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]: def _build_or_load_images(self, plan: PlanT) -> BottleImages:
"""Build/run the bottle and yield a handle; tear down on exit.""" """Return the agent and sidecar image references (or artifact paths)
for this plan, building fresh images when the policy requires it."""
@abstractmethod
def _launch_impl(self, plan: PlanT, images: BottleImages) -> AbstractContextManager[Bottle]:
"""Bring up the bottle using pre-resolved images; yield a handle; tear down on exit."""
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None: def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
"""Copy host-side files (CA cert, prompt, skills, .git) into """Copy host-side files (CA cert, prompt, skills, .git) into
@@ -584,68 +621,170 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
Not called by the launch path or the test suite.""" Not called by the launch path or the test suite."""
# Import concrete backend classes AFTER the base types are defined, so # _backends is None until the first call to _get_backends(), at which
# each backend module can pull BottleSpec / BottlePlan / BottleBackend # point all three concrete backend classes are imported and instantiated.
# via `from . import ...` without hitting a partially-initialized module. # Keeping the imports out of module scope means that importing any
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position # backend sub-module (e.g. `backend.docker.util`) no longer drags the
from .firecracker import FirecrackerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position # firecracker and macos-container implementations into memory.
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position #
# Tests may replace _backends with a {name: fake} dict via patch.object;
# Freezer is imported after the backend classes for the same reason: # _get_backends() returns the current module-level value as-is when it
# Freezer.commit_slug constructs ActiveAgent, which must be fully # is not None, so test fakes take effect without triggering real imports.
# defined first. _backends: dict[str, BottleBackend[Any, Any]] | None = None
from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylint: disable=wrong-import-position
# The dict is heterogeneous: each value is a BottleBackend specialized def _get_backends() -> dict[str, BottleBackend[Any, Any]]:
# over its own plan type. Concrete plan types are erased here because """Return the registry of all backend instances, loading lazily on first call."""
# the registry is selected at runtime and the CLI only needs the global _backends # pylint: disable=global-statement
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.). if _backends is None:
_BACKENDS: dict[str, BottleBackend[Any, Any]] = { from .docker import DockerBottleBackend
from .firecracker import FirecrackerBottleBackend
from .macos_container import MacosContainerBottleBackend
_backends = {
"docker": DockerBottleBackend(), "docker": DockerBottleBackend(),
"firecracker": FirecrackerBottleBackend(), "firecracker": FirecrackerBottleBackend(),
"macos-container": MacosContainerBottleBackend(), "macos-container": MacosContainerBottleBackend(),
} }
return _backends
def __getattr__(name: str) -> Any:
"""Lazily surface concrete backend classes and freeze symbols at the
package level so existing `from bot_bottle.backend import X` and
`patch.object(backend_mod, X, ...)` call-sites keep working without
forcing an import of every backend at module-init time."""
if name == "DockerBottleBackend":
from .docker import DockerBottleBackend
globals()[name] = DockerBottleBackend
return DockerBottleBackend
if name == "FirecrackerBottleBackend":
from .firecracker import FirecrackerBottleBackend
globals()[name] = FirecrackerBottleBackend
return FirecrackerBottleBackend
if name == "MacosContainerBottleBackend":
from .macos_container import MacosContainerBottleBackend
globals()[name] = MacosContainerBottleBackend
return MacosContainerBottleBackend
if name == "CommitCancelled":
from .freeze import CommitCancelled
globals()[name] = CommitCancelled
return CommitCancelled
if name == "Freezer":
from .freeze import Freezer
globals()[name] = Freezer
return Freezer
if name == "get_freezer":
from .freeze import get_freezer
globals()[name] = get_freezer
return get_freezer
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
def get_bottle_backend( def get_bottle_backend(
name: str | None = None, name: str | None = None,
*,
prompt: bool = True,
) -> BottleBackend[Any, Any]: ) -> BottleBackend[Any, Any]:
"""Resolve the bottle backend. """Resolve the bottle backend.
`name` precedence: `name` precedence:
1. explicit arg (CLI `--backend=<name>` passes through here) 1. explicit arg (e.g. resume passes the recorded backend name)
2. BOT_BOTTLE_BACKEND env var 2. BOT_BOTTLE_BACKEND env var
3. `macos-container` on compatible macOS hosts 3. auto-selection: VM backend first, docker fallback with prompt
4. `firecracker` on KVM-capable Linux hosts
5. default `docker` `prompt` controls whether auto-selection may block on an interactive
[i/d/q] prompt when falling back to docker. Pass `prompt=False` in
non-interactive contexts (headless launches, CI) so the call dies
with an actionable message instead of hanging.
Dies with a pointer at the known backends if the chosen name Dies with a pointer at the known backends if the chosen name
isn't implemented.""" isn't implemented."""
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or _default_backend_name() resolved = name or os.environ.get("BOT_BOTTLE_BACKEND")
if resolved not in _BACKENDS: if resolved is None:
known = ", ".join(sorted(_BACKENDS)) resolved = _auto_select_backend(prompt=prompt)
backends = _get_backends()
if resolved not in backends:
known = ", ".join(sorted(backends))
die(f"unknown backend {resolved!r}; known backends: {known}") die(f"unknown backend {resolved!r}; known backends: {known}")
return _BACKENDS[resolved] return backends[resolved]
def _default_backend_name() -> str: def _platform_vm_suggestion() -> str:
"""Platform-appropriate VM backend name for install suggestions."""
return "macos-container" if sys.platform == "darwin" else "firecracker"
def _print_vm_install_instructions() -> None:
"""Print platform-appropriate VM backend install instructions to stderr."""
vm = _platform_vm_suggestion()
if vm == "macos-container":
info("Install Apple Container: https://github.com/apple/container/releases")
info("Then start the service: container system start")
else:
info("Install Firecracker: https://github.com/firecracker-microvm/firecracker/releases")
info("Configure the host: ./cli.py backend setup")
def _auto_select_backend(prompt: bool = True) -> str:
"""Tier-1 / tier-2 backend auto-selection.
Tier 1: VM backend — macos-container on macOS when Apple Container is
installed; firecracker on KVM-capable Linux even before the binary is
present (its preflight prints an install pointer).
Tier 2: docker, with a security warning and an interactive prompt.
When `prompt=False` (headless / CI), dies with an actionable message
instead of blocking on a TTY read. When docker is also absent, prints
VM install instructions and exits.
"""
# --- Tier 1: VM backend -----------------------------------------
if has_backend("macos-container"): if has_backend("macos-container"):
return "macos-container" return "macos-container"
# A KVM-capable Linux host defaults to firecracker even when the # A KVM-capable Linux host defaults to firecracker even when the
# `firecracker` binary isn't installed yet: selecting it here routes # `firecracker` binary isn't installed yet: selecting it here routes
# start through firecracker's preflight, which prints an install # start through firecracker's preflight, which prints an install
# pointer, instead of silently falling back to docker. # pointer, instead of silently falling back to docker.
from .firecracker import FirecrackerBottleBackend
if FirecrackerBottleBackend.is_host_capable(): if FirecrackerBottleBackend.is_host_capable():
return "firecracker" return "firecracker"
# --- Tier 2: docker fallback ------------------------------------
if not has_backend("docker"):
info("No backend available on this host.")
_print_vm_install_instructions()
die("no backend available; install a VM backend and re-run")
vm = _platform_vm_suggestion()
warn(
"docker is less secure than VM backends — "
"containers share the host kernel."
)
if not prompt:
die(
f"no VM backend available; set BOT_BOTTLE_BACKEND=docker to proceed "
f"with docker, or install the {vm!r} backend."
)
sys.stderr.write(
f"bot-bottle: For better isolation, install the {vm!r} backend.\n"
f" [i] show {vm} install instructions and exit\n"
" [d] use docker anyway\n"
" [q] quit\n"
"bot-bottle: choice [i/d/q]: "
)
sys.stderr.flush()
reply = read_tty_line().strip().lower()
if reply == "d":
return "docker" return "docker"
if reply == "i":
_print_vm_install_instructions()
die("not proceeding with docker; install a VM backend or set BOT_BOTTLE_BACKEND=docker")
def known_backend_names() -> tuple[str, ...]: def known_backend_names() -> tuple[str, ...]:
"""Sorted tuple of all backend keys in `_BACKENDS`. Used by """Sorted tuple of all backend keys in `_get_backends()`. Used by
argparse (`--backend` choices) and the dashboard's backend argparse (`--backend` choices) and the dashboard's backend
picker.""" picker."""
return tuple(sorted(_BACKENDS)) return tuple(sorted(_get_backends()))
def has_backend(name: str) -> bool: def has_backend(name: str) -> bool:
@@ -657,9 +796,10 @@ def has_backend(name: str) -> bool:
Returns False for unknown names so callers can pass Returns False for unknown names so callers can pass
arbitrary input without separate validation.""" arbitrary input without separate validation."""
if name not in _BACKENDS: backends = _get_backends()
if name not in backends:
return False return False
return _BACKENDS[name].is_available() return backends[name].is_available()
def enumerate_active_agents() -> list[ActiveAgent]: def enumerate_active_agents() -> list[ActiveAgent]:
@@ -675,10 +815,11 @@ def enumerate_active_agents() -> list[ActiveAgent]:
deterministic tiebreaker. Agents with missing metadata deterministic tiebreaker. Agents with missing metadata
(`started_at == ""`) sort first.""" (`started_at == ""`) sort first."""
out: list[ActiveAgent] = [] out: list[ActiveAgent] = []
for name in known_backend_names(): backends = _get_backends()
if not has_backend(name): for name in sorted(backends):
if not backends[name].is_available():
continue continue
out.extend(_BACKENDS[name].enumerate_active()) out.extend(backends[name].enumerate_active())
out.sort(key=lambda a: (a.started_at, a.slug)) out.sort(key=lambda a: (a.started_at, a.slug))
return out return out
+60
View File
@@ -0,0 +1,60 @@
"""Shared helpers for the consolidated launch sequence (PRD 0070).
Logic that was duplicated across the docker, macos_container, and
firecracker consolidated_launch modules — extracted so each backend
imports it rather than re-implementing it.
"""
from __future__ import annotations
from ..egress import EgressPlan
from ..git_gate import GitGatePlan
from ..orchestrator.client import OrchestratorClient
from ..orchestrator.registration import registration_inputs
from .docker.gateway_provision import GatewayTransport, deprovision_git_gate, provision_git_gate
def provision_bottle(
client: OrchestratorClient,
source_ip: str,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
transport: GatewayTransport,
*,
image_ref: str = "",
tokens: dict[str, str] | None = None,
):
"""Register the bottle and provision its git-gate state. Rolls back the
registration if provisioning fails so no orphan is left. Returns the
`RegisteredBottle` from the orchestrator."""
inputs = registration_inputs(egress_plan)
reg = client.register_bottle(
source_ip, image_ref=image_ref, policy=inputs.policy,
metadata=inputs.metadata, tokens=tokens,
)
try:
provision_git_gate(transport, reg.bottle_id, git_gate_plan)
except Exception:
client.teardown_bottle(reg.bottle_id)
raise
return reg
def teardown_consolidated(
bottle_id: str,
transport: GatewayTransport,
*,
orchestrator_url: str,
timeout: float | None = None,
) -> None:
"""Deregister the bottle and remove its git-gate state. Both steps are
idempotent so this is safe from a cleanup trap."""
from ..orchestrator.config_store import DEFAULT_TEARDOWN_TIMEOUT_SECONDS
OrchestratorClient(
orchestrator_url,
timeout=timeout if timeout is not None else DEFAULT_TEARDOWN_TIMEOUT_SECONDS,
).teardown_bottle(bottle_id)
deprovision_git_gate(transport, bottle_id)
__all__ = ["provision_bottle", "teardown_consolidated"]
+9 -3
View File
@@ -31,7 +31,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan from ...supervise import SupervisePlan
from ...manifest import Manifest from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup from . import cleanup as _cleanup
from . import enumerate as _enumerate from . import enumerate as _enumerate
from . import launch as _launch from . import launch as _launch
@@ -100,9 +100,15 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
stage_dir=stage_dir, stage_dir=stage_dir,
) )
def prelaunch_checks(self, plan: DockerBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: DockerBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager @contextmanager
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]: def _launch_impl(self, plan: DockerBottlePlan, images: BottleImages) -> Generator[DockerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle: with _launch.launch(plan, images, provision=self.provision) as bottle:
yield bottle yield bottle
def ensure_orchestrator(self) -> str: def ensure_orchestrator(self) -> str:
@@ -1,19 +1,13 @@
"""Consolidated bottle launch sequence for the docker backend (PRD 0070). """Consolidated bottle launch sequence for the docker backend (PRD 0070).
Composes the orchestrator primitives into the register/teardown sequence that Composes the orchestrator primitives into the register/teardown sequence:
replaces the per-bottle gateway:
1. ensure the orchestrator control plane + shared gateway are up; 1. ensure the single infra container (control plane + gateway) is up;
2. allocate the bottle a pinned source IP on the gateway network (the 2. allocate the bottle a pinned source IP on the gateway network;
attribution key), skipping the gateway's own address + live bottles; 3. register it and provision its git-gate repos/creds into the gateway.
3. register it (egress policy blob + slug metadata) → bottle id + identity
token;
4. provision its git-gate repos/creds into the running gateway.
It returns a `LaunchContext` with everything the agent container needs to Returns a `LaunchContext` with everything the agent container needs to
attach — network, pinned IP, the gateway's address (its proxy target), the attach. The agent `docker run` itself is the backend's job; this owns the
orchestrator URL, and the identity token. The agent `docker run` itself is
the backend's job (it owns provider provisioning); this owns the
orchestrator-facing wiring so that sequence stays testable in isolation. orchestrator-facing wiring so that sequence stays testable in isolation.
""" """
@@ -25,15 +19,12 @@ from ...docker_cmd import run_docker
from ...egress import EgressPlan from ...egress import EgressPlan
from ...git_gate import GitGatePlan from ...git_gate import GitGatePlan
from ...orchestrator.client import OrchestratorClient from ...orchestrator.client import OrchestratorClient
from ...orchestrator.gateway import GATEWAY_NAME, GATEWAY_NETWORK from ...orchestrator.gateway import GATEWAY_NETWORK
from ...orchestrator.lifecycle import OrchestratorService from ...orchestrator.lifecycle import INFRA_NAME, OrchestratorService
from ...orchestrator.registration import registration_inputs from ..consolidated_util import provision_bottle
from ..consolidated_util import teardown_consolidated as _teardown_util
from .gateway_provision import DockerGatewayTransport
from .gateway_net import next_free_ip from .gateway_net import next_free_ip
from .gateway_provision import (
DockerGatewayTransport,
deprovision_git_gate,
provision_git_gate,
)
class ConsolidatedLaunchError(RuntimeError): class ConsolidatedLaunchError(RuntimeError):
@@ -75,24 +66,21 @@ def _container_ip(name: str, network: str) -> str:
ip = proc.stdout.strip() ip = proc.stdout.strip()
if proc.returncode != 0 or not ip: if proc.returncode != 0 or not ip:
raise ConsolidatedLaunchError( raise ConsolidatedLaunchError(
f"gateway {name} has no address on {network}: {proc.stderr.strip()}" f"container {name} has no address on {network}: {proc.stderr.strip()}"
) )
return ip return ip
def _network_container_ips(network: str) -> list[str]: def _network_container_ips(network: str) -> list[str]:
"""Every address currently assigned on the gateway network — the ground """Every address currently assigned on the gateway network — the ground
truth for "in use": the gateway + orchestrator infrastructure containers truth for "in use": the infra container and every live agent. Read from
and every live agent. Read from the network so a new bottle can't collide the network so a new bottle can't collide with anything actually attached."""
with anything actually attached (a registry-only view would miss the
orchestrator/gateway containers)."""
proc = run_docker([ proc = run_docker([
"docker", "network", "inspect", "--format", "docker", "network", "inspect", "--format",
"{{range .Containers}}{{.IPv4Address}} {{end}}", network, "{{range .Containers}}{{.IPv4Address}} {{end}}", network,
]) ])
ips: list[str] = [] ips: list[str] = []
for entry in proc.stdout.split(): for entry in proc.stdout.split():
# entries look like "172.20.0.2/16" — keep the address.
ips.append(entry.split("/", 1)[0]) ips.append(entry.split("/", 1)[0])
return ips return ips
@@ -104,33 +92,24 @@ def launch_consolidated(
image_ref: str = "", image_ref: str = "",
tokens: dict[str, str] | None = None, tokens: dict[str, str] | None = None,
service: OrchestratorService | None = None, service: OrchestratorService | None = None,
gateway_name: str = GATEWAY_NAME, infra_name: str = INFRA_NAME,
network: str = GATEWAY_NETWORK, network: str = GATEWAY_NETWORK,
) -> LaunchContext: ) -> LaunchContext:
"""Ensure the orchestrator + gateway are up, allocate + register the """Ensure the infra container is up, allocate + register the bottle, and
bottle, and provision its git-gate state. Returns the agent's attach provision its git-gate state. Returns the agent's attach context."""
context. Raises `ConsolidatedLaunchError` (or the primitives' own errors)
if any step fails — the caller tears down on failure."""
service = service or OrchestratorService() service = service or OrchestratorService()
url = service.ensure_running() url = service.ensure_running()
client = OrchestratorClient(url) client = OrchestratorClient(url)
cidr = _network_cidr(network) cidr = _network_cidr(network)
gateway_ip = _container_ip(gateway_name, network) gateway_ip = _container_ip(infra_name, network)
source_ip = next_free_ip(cidr, _network_container_ips(network)) source_ip = next_free_ip(cidr, _network_container_ips(network))
inputs = registration_inputs(egress_plan) transport = DockerGatewayTransport(infra_name)
reg = client.register_bottle( reg = provision_bottle(
source_ip, image_ref=image_ref, policy=inputs.policy, client, source_ip, egress_plan, git_gate_plan, transport,
metadata=inputs.metadata, tokens=tokens, image_ref=image_ref, tokens=tokens,
) )
try:
provision_git_gate(
DockerGatewayTransport(gateway_name), reg.bottle_id, git_gate_plan)
except Exception:
# Roll the registration back so a provisioning failure leaves no orphan.
client.teardown_bottle(reg.bottle_id)
raise
return LaunchContext( return LaunchContext(
bottle_id=reg.bottle_id, bottle_id=reg.bottle_id,
identity_token=reg.identity_token, identity_token=reg.identity_token,
@@ -142,12 +121,12 @@ def launch_consolidated(
def teardown_consolidated( def teardown_consolidated(
bottle_id: str, *, orchestrator_url: str, gateway_name: str = GATEWAY_NAME, bottle_id: str, *, orchestrator_url: str, infra_name: str = INFRA_NAME,
timeout: float | None = None,
) -> None: ) -> None:
"""Deregister the bottle and remove its git-gate state from the gateway. """Deregister the bottle and remove its git-gate state. Idempotent."""
Both steps are idempotent so this is safe from a cleanup trap.""" _teardown_util(bottle_id, DockerGatewayTransport(infra_name),
OrchestratorClient(orchestrator_url).teardown_bottle(bottle_id) orchestrator_url=orchestrator_url, timeout=timeout)
deprovision_git_gate(DockerGatewayTransport(gateway_name), bottle_id)
__all__ = [ __all__ = [
@@ -94,6 +94,12 @@ def provision_git_gate(
transport.exec(["mkdir", "-p", "/etc/git-gate"]) transport.exec(["mkdir", "-p", "/etc/git-gate"])
transport.cp_into(str(plan.hook_script), "/etc/git-gate/pre-receive") transport.cp_into(str(plan.hook_script), "/etc/git-gate/pre-receive")
transport.cp_into(str(plan.access_hook_script), "/etc/git-gate/access-hook") transport.cp_into(str(plan.access_hook_script), "/etc/git-gate/access-hook")
# The access-hook is exec'd directly (not via `sh`), so it needs the x bit.
# Set it here rather than trusting the copy to carry the staged 0o700:
# `docker cp` preserves source mode, but the Apple `container cp` does not,
# landing the hook 0o644 → EACCES when the git-http handler tries to exec it.
# chmod on the gateway side is backend-neutral and fixes every transport.
transport.exec(["chmod", "+x", "/etc/git-gate/access-hook"])
creds = _creds_dir(bottle_id) creds = _creds_dir(bottle_id)
transport.exec(["mkdir", "-p", creds]) transport.exec(["mkdir", "-p", creds])
for u in plan.upstreams: for u in plan.upstreams:
+59 -23
View File
@@ -42,7 +42,9 @@ from ...git_gate import (
provision_git_gate_dynamic_keys, provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys, revoke_git_gate_provisioned_keys,
) )
from ...log import info, warn from ...image_cache import check_stale
from ...log import die, info, warn
from .. import BottleImages
from . import util as docker_mod from . import util as docker_mod
from .bottle import DockerBottle from .bottle import DockerBottle
from .bottle_plan import DockerBottlePlan from .bottle_plan import DockerBottlePlan
@@ -62,6 +64,7 @@ from .compose import (
write_compose_file, write_compose_file,
) )
from .consolidated_compose import consolidated_agent_compose from .consolidated_compose import consolidated_agent_compose
from ...orchestrator.config_store import resolve_teardown_timeout
from .consolidated_launch import launch_consolidated, teardown_consolidated from .consolidated_launch import launch_consolidated, teardown_consolidated
from ...orchestrator.gateway import DockerGateway from ...orchestrator.gateway import DockerGateway
@@ -70,16 +73,47 @@ from ...orchestrator.gateway import DockerGateway
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent) _REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def build_or_load_images(plan: DockerBottlePlan) -> BottleImages:
"""Resolve the agent image ref for this plan.
Returns the committed snapshot if one exists, the cached image when the
policy is 'cached', or builds a fresh image and returns that."""
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return BottleImages(agent=committed)
if plan.spec.image_policy == "cached":
if not docker_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
return BottleImages(agent=plan.image)
docker_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
docker_mod.verify_agent_image(
plan.image, runtime_for(plan.agent_provider_template).smoke_test,
)
return BottleImages(agent=plan.image)
@contextmanager @contextmanager
def launch( def launch(
plan: DockerBottlePlan, plan: DockerBottlePlan,
images: BottleImages,
*, *,
provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None], provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None],
) -> Generator[DockerBottle, None, None]: ) -> Generator[DockerBottle, None, None]:
"""Build, launch, and provision a Docker bottle via compose. """Launch and provision a Docker bottle via compose. Teardown on exit."""
Teardown on exit."""
stack = ExitStack() stack = ExitStack()
# Stamp the resolved agent image ref into the plan so compose rendering
# picks up the right image (may be a committed snapshot or cached ref).
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=str(images.agent)),
)
_bottle_for_revoke = plan.manifest.bottle _bottle_for_revoke = plan.manifest.bottle
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug) _git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
@@ -96,25 +130,6 @@ def launch(
) )
try: try:
# Step 1: agent image. Use a committed snapshot when one exists
# and is present in the local daemon; otherwise build from the
# Dockerfile. (The gateway image is built by the orchestrator.)
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
else:
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
docker_mod.verify_agent_image(
plan.image, runtime_for(plan.agent_provider_template).smoke_test,
)
# Step 2: mint the git-gate dynamic (gitea) deploy keys, if any, before # Step 2: mint the git-gate dynamic (gitea) deploy keys, if any, before
# provisioning the bottle's repos into the shared gateway. # provisioning the bottle's repos into the shared gateway.
git_gate_plan = plan.git_gate_plan git_gate_plan = plan.git_gate_plan
@@ -133,11 +148,14 @@ def launch(
token_values = egress_resolve_token_values( token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env, plan.egress_plan.token_env_map, effective_env,
) )
teardown_timeout = resolve_teardown_timeout()
ctx = launch_consolidated( ctx = launch_consolidated(
plan.egress_plan, git_gate_plan, image_ref=plan.image, tokens=token_values, plan.egress_plan, git_gate_plan, image_ref=plan.image, tokens=token_values,
) )
stack.callback( stack.callback(
teardown_consolidated, ctx.bottle_id, orchestrator_url=ctx.orchestrator_url, teardown_consolidated, ctx.bottle_id,
orchestrator_url=ctx.orchestrator_url,
timeout=teardown_timeout,
) )
# Step 4: install the SHARED gateway CA into the agent (replaces the # Step 4: install the SHARED gateway CA into the agent (replaces the
@@ -207,3 +225,21 @@ def launch(
yield bottle yield bottle
finally: finally:
teardown() teardown()
def stale_checks(plan: DockerBottlePlan) -> None:
"""Raise StaleImageError if a cached image is older than the configured
threshold. Only runs when image_policy is 'cached'. Called by the backend
class's _image_stale_checks before _launch_impl starts any resources."""
if plan.spec.image_policy != "cached":
return
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
ts = docker_mod.image_created_at(committed)
if ts is not None:
check_stale(f"agent image {committed!r}", ts)
return
if docker_mod.image_exists(plan.image):
ts = docker_mod.image_created_at(plan.image)
if ts is not None:
check_stale(f"agent image {plan.image!r}", ts)
+5 -1
View File
@@ -27,10 +27,14 @@ def _docker_on_path() -> bool:
def _daemon_reachable() -> bool: def _daemon_reachable() -> bool:
if not _docker_on_path(): if not _docker_on_path():
return False return False
try:
return subprocess.run( return subprocess.run(
["docker", "info"], ["docker", "info"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False, timeout=5,
).returncode == 0 ).returncode == 0
except subprocess.TimeoutExpired:
return False
def _print_install_pointer() -> None: def _print_install_pointer() -> None:
+48 -31
View File
@@ -5,10 +5,11 @@ existence, and building images."""
from __future__ import annotations from __future__ import annotations
import os import os
from datetime import datetime, timezone
import re import re
import shutil import shutil
import subprocess import subprocess
from typing import Iterable, Iterator from typing import Iterator
from ...docker_cmd import run_docker from ...docker_cmd import run_docker
from ...log import die, info from ...log import die, info
@@ -32,12 +33,7 @@ def container_name_candidates(base: str) -> Iterator[str]:
def runsc_available() -> bool: def runsc_available() -> bool:
"""Return True if the Docker daemon has the gVisor (`runsc`) runtime """Return True if the Docker daemon has the gVisor (`runsc`) runtime
registered. Called once per prepare; the result lives on the plan.""" registered. Called once per prepare; the result lives on the plan."""
r = subprocess.run( r = run_docker(["docker", "info", "--format", "{{json .Runtimes}}"])
["docker", "info", "--format", "{{json .Runtimes}}"],
capture_output=True,
text=True,
check=False,
)
return r.returncode == 0 and "runsc" in r.stdout return r.returncode == 0 and "runsc" in r.stdout
@@ -51,20 +47,15 @@ def require_docker() -> None:
def image_exists(ref: str) -> bool: def image_exists(ref: str) -> bool:
return _silent_run(["docker", "image", "inspect", ref]) == 0 return run_docker(["docker", "image", "inspect", ref]).returncode == 0
def container_exists(name: str) -> bool: def container_exists(name: str) -> bool:
"""Returns True if a container (running or stopped) with the given """Returns True if a container (running or stopped) with the given
name exists. Uses `docker ps -a -q -f name=^<name>$` so substring name exists. Uses `docker ps -a -q -f name=^<name>$` so substring
matches don't false-positive.""" matches don't false-positive."""
result = subprocess.run( result = run_docker(["docker", "ps", "-a", "-q", "-f", f"name=^{name}$"])
["docker", "ps", "-a", "-q", "-f", f"name=^{name}$"], return result.returncode == 0 and bool(result.stdout.strip())
capture_output=True,
text=True,
check=True,
)
return bool(result.stdout.strip())
def force_remove_container(name: str) -> None: def force_remove_container(name: str) -> None:
@@ -72,12 +63,7 @@ def force_remove_container(name: str) -> None:
doesn't — and the rm itself is best-effort (errors swallowed) so doesn't — and the rm itself is best-effort (errors swallowed) so
this is safe to register as a teardown callback.""" this is safe to register as a teardown callback."""
if container_exists(name): if container_exists(name):
subprocess.run( run_docker(["docker", "rm", "-f", name])
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
def docker_exec_root(container: str, argv: list[str]) -> None: def docker_exec_root(container: str, argv: list[str]) -> None:
@@ -205,10 +191,7 @@ def verify_agent_image(image: str, argv: tuple[str, ...]) -> None:
def commit_container(container_name: str, image_tag: str) -> None: def commit_container(container_name: str, image_tag: str) -> None:
"""Run `docker commit <container_name> <image_tag>` to snapshot the """Run `docker commit <container_name> <image_tag>` to snapshot the
running container's filesystem state as a local Docker image.""" running container's filesystem state as a local Docker image."""
result = subprocess.run( result = run_docker(["docker", "commit", container_name, image_tag])
["docker", "commit", container_name, image_tag],
capture_output=True, text=True, check=False,
)
if result.returncode != 0: if result.returncode != 0:
die( die(
f"docker commit {container_name!r}{image_tag!r} failed: " f"docker commit {container_name!r}{image_tag!r} failed: "
@@ -217,10 +200,44 @@ def commit_container(container_name: str, image_tag: str) -> None:
info(f"committed {container_name!r}{image_tag!r}") info(f"committed {container_name!r}{image_tag!r}")
def _silent_run(cmd: Iterable[str]) -> int: def image_created_at(ref: str) -> datetime | None:
return subprocess.run( """Return Docker's image Created timestamp as an aware UTC datetime, or
list(cmd), None when the field is absent or unparseable. Callers should skip the
stdout=subprocess.DEVNULL, stale check when None is returned."""
stderr=subprocess.DEVNULL, r = subprocess.run(
["docker", "image", "inspect", "--format", "{{.Created}}", ref],
capture_output=True,
text=True,
check=False, check=False,
).returncode )
if r.returncode != 0:
die(
f"docker image inspect for {ref!r} failed: "
f"{(r.stderr or '').strip() or '<no stderr>'}"
)
raw = r.stdout.strip()
if not raw:
return None
try:
return _parse_docker_timestamp(raw)
except ValueError:
return None
def _parse_docker_timestamp(raw: str) -> datetime:
text = raw.strip()
if text.endswith("Z"):
text = text[:-1] + "+00:00"
dot = text.find(".")
if dot != -1:
tz_plus = text.find("+", dot)
tz_minus = text.find("-", dot)
tz_candidates = [pos for pos in (tz_plus, tz_minus) if pos != -1]
if tz_candidates:
tz_pos = min(tz_candidates)
frac = text[dot + 1:tz_pos]
text = text[:dot + 1] + frac[:6].ljust(6, "0") + text[tz_pos:]
dt = datetime.fromisoformat(text)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
+11 -4
View File
@@ -18,7 +18,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan from ...git_gate import GitGatePlan
from ...manifest import Manifest from ...manifest import Manifest
from ...supervise import SupervisePlan from ...supervise import SupervisePlan
from .. import ActiveAgent, BottleBackend, BottleSpec from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup from . import cleanup as _cleanup
from . import enumerate as _enumerate from . import enumerate as _enumerate
from . import launch as _launch from . import launch as _launch
@@ -92,11 +92,18 @@ class FirecrackerBottleBackend(
stage_dir=stage_dir, stage_dir=stage_dir,
) )
def _build_or_load_images(self, plan: FirecrackerBottlePlan) -> BottleImages:
return BottleImages(agent=_launch.build_or_load_agent_base(plan))
def prelaunch_checks(self, plan: FirecrackerBottlePlan) -> None:
_launch.stale_checks(plan)
@contextmanager @contextmanager
def launch( def _launch_impl(
self, plan: FirecrackerBottlePlan self, plan: FirecrackerBottlePlan, images: BottleImages,
) -> Generator[FirecrackerBottle, None, None]: ) -> Generator[FirecrackerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle: assert isinstance(images.agent, Path)
with _launch.launch(plan, images.agent, provision=self.provision) as bottle:
yield bottle yield bottle
def prepare_cleanup(self) -> FirecrackerBottleCleanupPlan: def prepare_cleanup(self) -> FirecrackerBottleCleanupPlan:
+66 -16
View File
@@ -1,8 +1,23 @@
"""Cleanup for the Firecracker backend. """Cleanup for the Firecracker backend.
Orphans are: firecracker VMM processes whose config lives under our run Reaps *orphans* only — resources with no live VM behind them:
dir, and the per-bottle run dirs. TAP slots free themselves (the flock
drops when the launcher exits), so there is nothing to reclaim there. * orphan run dirs: a per-bottle run dir (holding the ~1G rootfs.ext4)
whose firecracker process has exited. These leak when a launch is
hard-killed before its teardown runs (host OOM/crash, a cancelled CI
job, `kill -9`); the clean-exit path already removes its own dir in
launch.py.
* orphan VM pids: a firecracker process whose run dir is already gone
— a VMM left lingering after its dir was removed.
A run dir with a *live* firecracker process is a running bottle and is
left strictly alone: it is neither killed nor removed. (The backend's
`enumerate_active` registry is still a stub — #354 — so a live process
is the only reliable "this bottle is in use" signal we have. Once the
registry lands, registry-orphaned-but-running VMs can be reaped too.)
TAP slots free themselves (the flock drops when the launcher exits), so
there is nothing to reclaim there.
""" """
from __future__ import annotations from __future__ import annotations
@@ -22,38 +37,73 @@ def _run_root() -> Path:
return util.cache_dir() / "run" return util.cache_dir() / "run"
def _orphan_vm_pids() -> list[int]: def _run_dir_of(cmd: str, run_root: Path) -> Path | None:
"""firecracker processes whose --config-file is under our run dir.""" """The bottle run dir a firecracker cmdline belongs to, or None.
run_root = str(_run_root())
A bottle VM is launched with `--config-file <run_root>/<slug>/config.json`,
so the run dir is the config file's parent when it sits directly under
the run root. Anything else (a builder VM, the infra VM elsewhere) is
not ours to reap here.
"""
toks = cmd.split()
for i, tok in enumerate(toks):
if tok == "--config-file" and i + 1 < len(toks):
parent = Path(toks[i + 1]).parent
if parent.parent == run_root:
return parent
return None
def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
"""Inspect running firecracker VMs under ``run_root``.
Returns ``(live_run_dirs, orphan_pids)``:
* ``live_run_dirs`` — run dirs backed by a running VM (never reaped);
* ``orphan_pids`` — firecracker pids whose run dir no longer exists
(a lingering VMM to kill).
"""
result = subprocess.run( result = subprocess.run(
["pgrep", "-a", "firecracker"], ["pgrep", "-a", "firecracker"],
capture_output=True, text=True, check=False, capture_output=True, text=True, check=False,
) )
if result.returncode != 0: if result.returncode != 0:
return [] return set(), []
pids: list[int] = [] live: set[str] = set()
orphan_pids: list[int] = []
for line in result.stdout.splitlines(): for line in result.stdout.splitlines():
parts = line.split(None, 1) parts = line.split(None, 1)
if len(parts) != 2 or run_root not in parts[1]: if len(parts) != 2:
continue continue
try: try:
pids.append(int(parts[0])) pid = int(parts[0])
except ValueError: except ValueError:
continue continue
return pids run_dir = _run_dir_of(parts[1], run_root)
if run_dir is None:
continue
if run_dir.is_dir():
live.add(str(run_dir))
else:
orphan_pids.append(pid)
return live, orphan_pids
def _run_dirs() -> list[str]: def _orphan_run_dirs(run_root: Path, live: set[str]) -> list[str]:
run_root = _run_root() """Run dirs with no live VM behind them — the leaked ones to remove."""
if not run_root.is_dir(): if not run_root.is_dir():
return [] return []
return sorted(str(p) for p in run_root.iterdir() if p.is_dir()) return sorted(
str(p) for p in run_root.iterdir()
if p.is_dir() and str(p) not in live
)
def prepare_cleanup() -> FirecrackerBottleCleanupPlan: def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
run_root = _run_root()
live, orphan_pids = _scan_processes(run_root)
return FirecrackerBottleCleanupPlan( return FirecrackerBottleCleanupPlan(
vm_pids=tuple(_orphan_vm_pids()), vm_pids=tuple(orphan_pids),
run_dirs=tuple(_run_dirs()), run_dirs=tuple(_orphan_run_dirs(run_root, live)),
) )
@@ -33,8 +33,7 @@ from ...orchestrator.client import OrchestratorClient
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
OrchestratorStartError, # re-exported so callers can catch it OrchestratorStartError, # re-exported so callers can catch it
) )
from ...orchestrator.registration import registration_inputs from ..consolidated_util import provision_bottle, teardown_consolidated as _teardown_util
from ..docker.gateway_provision import deprovision_git_gate, provision_git_gate
from . import infra_vm from . import infra_vm
@@ -68,18 +67,11 @@ def launch_consolidated(
url = infra.control_plane_url url = infra.control_plane_url
client = OrchestratorClient(url) client = OrchestratorClient(url)
inputs = registration_inputs(egress_plan) transport = infra_vm.gateway_transport()
reg = client.register_bottle( reg = provision_bottle(
guest_ip, image_ref=image_ref, policy=inputs.policy, client, guest_ip, egress_plan, git_gate_plan, transport,
metadata=inputs.metadata, tokens=tokens, image_ref=image_ref, tokens=tokens,
) )
try:
provision_git_gate(
infra_vm.gateway_transport(), reg.bottle_id, git_gate_plan)
except Exception:
client.teardown_bottle(reg.bottle_id)
raise
# The shared gateway CA every agent on this host trusts for TLS # The shared gateway CA every agent on this host trusts for TLS
# interception — fetched from the infra VM over SSH. # interception — fetched from the infra VM over SSH.
return LaunchContext( return LaunchContext(
@@ -91,13 +83,15 @@ def launch_consolidated(
) )
def teardown_consolidated(bottle_id: str, *, orchestrator_url: str) -> None: def teardown_consolidated(
bottle_id: str, *, orchestrator_url: str, timeout: float | None = None,
) -> None:
"""Deregister the bottle and remove its git-gate state from the gateway """Deregister the bottle and remove its git-gate state from the gateway
VM. Both steps are idempotent so this is safe from a cleanup trap. Does VM. Both steps are idempotent so this is safe from a cleanup trap. Does
NOT stop the infra VM — it's a persistent per-host singleton shared by NOT stop the infra VM — it's a persistent per-host singleton shared by
every bottle.""" every bottle."""
OrchestratorClient(orchestrator_url).teardown_bottle(bottle_id) _teardown_util(bottle_id, infra_vm.gateway_transport(),
deprovision_git_gate(infra_vm.gateway_transport(), bottle_id) orchestrator_url=orchestrator_url, timeout=timeout)
__all__ = [ __all__ = [
+35 -21
View File
@@ -1,9 +1,12 @@
"""FirecrackerFreezer — snapshot a running microVM to a Docker image. """FirecrackerFreezer — snapshot a running microVM to a rootfs tar.
The VM is live and can't be block-copied safely, so — like the macOS The VM is live and can't be block-copied safely, so — like the macOS
backend — we stream the guest root filesystem out over the control backend — we stream the guest root filesystem out over the control
channel (SSH here) and rebuild an image from it. The bottle keeps channel (SSH here). Unlike the other backends this needs no Docker: the
running after the snapshot. tar *is* the resumable artifact. `resume` extracts it and rebuilds a
fresh per-bottle ext4 with `mke2fs -d` (see `util.build_committed_rootfs_dir`
and `launch.build_or_load_agent_base`). The bottle keeps running after the
snapshot.
""" """
from __future__ import annotations from __future__ import annotations
@@ -11,9 +14,9 @@ from __future__ import annotations
import json import json
import os import os
import subprocess import subprocess
import tempfile
from pathlib import Path from pathlib import Path
from ...bottle_state import committed_rootfs_path
from ...log import die, info from ...log import die, info
from .. import ActiveAgent from .. import ActiveAgent
from ..freeze import Freezer from ..freeze import Freezer
@@ -30,14 +33,13 @@ class FirecrackerFreezer(Freezer):
if not private_key.is_file() or not guest_ip: if not private_key.is_file() or not guest_ip:
die(f"cannot freeze {agent.slug}: run dir {run_dir} is missing the " die(f"cannot freeze {agent.slug}: run dir {run_dir} is missing the "
f"SSH key or VM config (is the bottle still running?)") f"SSH key or VM config (is the bottle still running?)")
image_tag = f"bot-bottle-committed-{agent.slug}:latest" tar_path = committed_rootfs_path(agent.slug)
_commit_via_ssh(private_key, guest_ip, image_tag) _commit_rootfs_via_ssh(private_key, guest_ip, tar_path)
info(f"committed {agent.slug} -> {image_tag!r}") info(f"committed {agent.slug} -> {tar_path}")
return image_tag return str(tar_path)
def _export_hint(self, slug: str, image_ref: str) -> None: def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: docker image save {image_ref} " info(f"to export for migration: cp {image_ref} {slug}.tar")
f"-o {slug}.tar")
def _guest_ip_from_config(config_path: Path) -> str: def _guest_ip_from_config(config_path: Path) -> str:
@@ -53,11 +55,27 @@ def _guest_ip_from_config(config_path: Path) -> str:
return "" return ""
def _commit_via_ssh(private_key: Path, guest_ip: str, image_tag: str) -> None: def _commit_rootfs_via_ssh(private_key: Path, guest_ip: str, tar_path: Path) -> None:
with tempfile.TemporaryDirectory(prefix="bot-bottle-fc-commit.") as tmp: """Stream the guest rootfs out over SSH into `tar_path`. Excludes the
rootfs_tar = os.path.join(tmp, "rootfs.tar") virtual/live mounts (proc/sys/dev/run) — resume recreates those empty
mount points. Written to a `.partial` sibling and renamed on success so
a failed freeze never leaves a truncated artifact in its place."""
tar_path.parent.mkdir(parents=True, exist_ok=True)
partial = tar_path.with_name(tar_path.name + ".partial")
ssh = util.ssh_base_argv(private_key, guest_ip) ssh = util.ssh_base_argv(private_key, guest_ip)
with open(rootfs_tar, "wb") as tar_out: # The snapshot can contain the bottle's private workspace, so keep it
# owner-only (0600) for the whole stream. The `os.open` mode only applies
# on *creation*, so unlink any leftover partial (a prior interrupted run
# could have left it world-readable, or something could swap in a symlink
# at this predictable name) and exclusively recreate it — O_EXCL|O_NOFOLLOW
# — then fchmod immediately so umask can't loosen it. Re-assert after the
# rename too (os.replace carries the source mode, but be explicit).
partial.unlink(missing_ok=True)
fd = os.open(
partial, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600
)
os.fchmod(fd, 0o600)
with os.fdopen(fd, "wb") as tar_out:
result = subprocess.run( result = subprocess.run(
[*ssh, "--", "tar", "--create", "--one-file-system", [*ssh, "--", "tar", "--create", "--one-file-system",
"--exclude=./proc", "--exclude=./sys", "--exclude=./dev", "--exclude=./proc", "--exclude=./sys", "--exclude=./dev",
@@ -65,12 +83,8 @@ def _commit_via_ssh(private_key: Path, guest_ip: str, image_tag: str) -> None:
stdout=tar_out, stderr=subprocess.PIPE, check=False, stdout=tar_out, stderr=subprocess.PIPE, check=False,
) )
if result.returncode != 0: if result.returncode != 0:
partial.unlink(missing_ok=True)
die(f"ssh tar for {guest_ip} failed: " die(f"ssh tar for {guest_ip} failed: "
f"{(result.stderr or b'').decode().strip() or '<no stderr>'}") f"{(result.stderr or b'').decode().strip() or '<no stderr>'}")
with open(os.path.join(tmp, "Dockerfile"), "w", encoding="utf-8") as f: os.replace(partial, tar_path)
f.write("FROM scratch\nADD rootfs.tar /\nUSER node\nWORKDIR /home/node\n") os.chmod(tar_path, 0o600)
build = subprocess.run(
["docker", "build", "-t", image_tag, tmp], check=False,
)
if build.returncode != 0:
die(f"docker build for {image_tag!r} failed")
@@ -38,27 +38,47 @@ _BUILD_TIMEOUT_SECONDS = 900.0
def _dockerfile_hash(dockerfile: Path) -> str: def _dockerfile_hash(dockerfile: Path) -> str:
"""Cache key: the Dockerfile's content. The shipped agent Dockerfiles """The Dockerfile's content hash. The shipped agent Dockerfiles COPY
COPY nothing from the build context (see .dockerignore), so their content nothing from the build context (see .dockerignore), so their content fully
fully determines the image; a Dockerfile that adds COPY will want the determines the built image; a Dockerfile that adds COPY will want the
context folded in here too.""" context folded in here too."""
return hashlib.sha256(dockerfile.read_bytes()).hexdigest()[:16] return hashlib.sha256(dockerfile.read_bytes()).hexdigest()[:16]
def _rootfs_digest(dockerfile: Path) -> str:
"""Cache key for the built AND boot-injected agent rootfs. Two inputs
determine the on-disk rootfs: the Dockerfile (the image) and the guest init
injected into it (`util._GUEST_INIT`). Folding the init in means a fix to
it — e.g. making /tmp world-writable — busts the cache instead of silently
reusing a stale rootfs built with the old init."""
h = hashlib.sha256()
h.update(_dockerfile_hash(dockerfile).encode())
h.update(b"\0")
h.update(util._GUEST_INIT.encode())
return h.hexdigest()[:16]
def cached_agent_rootfs_dir(dockerfile: Path) -> Path | None:
"""Return the ready cached rootfs for ``dockerfile``, if one exists."""
base = util.cache_dir() / "rootfs" / f"agent-{_rootfs_digest(dockerfile)}"
return base if (base / ".bb-ready").is_file() else None
def build_agent_rootfs_dir( def build_agent_rootfs_dir(
dockerfile: Path, *, image_tag: str, smoke_test: tuple[str, ...] = (), dockerfile: Path, *, image_tag: str, smoke_test: tuple[str, ...] = (),
) -> Path: ) -> Path:
"""Build `dockerfile` in the infra VM (buildah, no host docker), export its """Build `dockerfile` in the infra VM (buildah, no host docker), export its
rootfs, inject the guest boot bits, and return the cached base dir — the rootfs, inject the guest boot bits, and return the cached base dir — the
same shape `util.build_rootfs_ext4` consumes. Cached by Dockerfile content, same shape `util.build_rootfs_ext4` consumes. Cached by Dockerfile content
so a repeat launch skips the rebuild. + injected guest init, so a repeat launch skips the rebuild but an init or
Dockerfile change rebuilds.
`smoke_test` (the provider's declared argv, e.g. `("claude","--version")`) `smoke_test` (the provider's declared argv, e.g. `("claude","--version")`)
is run in the freshly built image before export, catching an npm is run in the freshly built image before export, catching an npm
silent-failure image at build time rather than at first agent use.""" silent-failure image at build time rather than at first agent use."""
digest = _dockerfile_hash(dockerfile) digest = _rootfs_digest(dockerfile)
base = util.cache_dir() / "rootfs" / f"agent-{digest}" base = util.cache_dir() / "rootfs" / f"agent-{digest}"
if (base / ".bb-ready").is_file(): if cached_agent_rootfs_dir(dockerfile) is not None:
info(f"using cached agent rootfs {base.name}") info(f"using cached agent rootfs {base.name}")
return base return base
@@ -41,11 +41,11 @@ from . import util
_ARTIFACT_FORMAT = "1" _ARTIFACT_FORMAT = "1"
_REPO_ROOT = Path(__file__).resolve().parents[3] _REPO_ROOT = Path(__file__).resolve().parents[3]
_DOCKERFILES = ("Dockerfile.orchestrator", "Dockerfile.gateway", "Dockerfile.infra") _DOCKERFILES = ("Dockerfile.orchestrator", "Dockerfile.gateway", "Dockerfile.infra", "Dockerfile.infra.fc")
_DEFAULT_BASE = "https://gitea.dideric.is" _DEFAULT_BASE = "https://gitea.dideric.is"
_DEFAULT_OWNER = "didericis" _DEFAULT_OWNER = "didericis"
_PACKAGE = "bot-bottle-infra" _PACKAGE = "bot-bottle-firecracker-infra"
# Streaming copy chunk for the (hundreds-of-MB) download. # Streaming copy chunk for the (hundreds-of-MB) download.
_CHUNK = 1 << 20 _CHUNK = 1 << 20
@@ -57,24 +57,39 @@ def local_build_requested() -> bool:
return os.environ.get("BOT_BOTTLE_INFRA_BUILD", "").strip().lower() == "local" return os.environ.get("BOT_BOTTLE_INFRA_BUILD", "").strip().lower() == "local"
def infra_artifact_version(init_script: str) -> str: def infra_artifact_version(init_script: str, *, repo_root: Path = _REPO_ROOT) -> str:
"""Content hash (16 hex) of everything baked into the infra rootfs: the """Content hash (16 hex) of everything baked into the infra rootfs: the
whole shipped `bot_bottle` package (the infra image `COPY`s it wholesale), whole shipped `bot_bottle` package, the three fixed Dockerfiles, and the
the three fixed Dockerfiles, and the guest init. Deterministic across the guest init. Deterministic across the publish host and the launch host when
publish host and the launch host when both run the same checkout, so the both run the same checkout, so the tag the launch host pulls is exactly the
tag the launch host pulls is exactly the tag publish produced.""" tag publish produced.
The package is `COPY bot_bottle /app/bot_bottle`'d wholesale into the image,
so hash *every* regular file under it — not just `*.py`. Non-Python inputs
(e.g. `egress_entrypoint.sh`, `netpool.defaults.env`) are baked in too, and
a change to one must bump the version or a launch host could boot a stale
rootfs whose code differs from its checkout. `__pycache__`/`.pyc` are the
only exclusions — build artifacts, never copied."""
h = hashlib.sha256() h = hashlib.sha256()
h.update(f"format={_ARTIFACT_FORMAT}\n".encode()) h.update(f"format={_ARTIFACT_FORMAT}\n".encode())
pkg = _REPO_ROOT / "bot_bottle" pkg = repo_root / "bot_bottle"
for path in sorted(pkg.rglob("*.py")): for path in sorted(pkg.rglob("*")):
if "__pycache__" in path.parts: if not path.is_file():
continue continue
h.update(str(path.relative_to(_REPO_ROOT)).encode()) if "__pycache__" in path.parts or path.suffix == ".pyc":
continue
h.update(str(path.relative_to(repo_root)).encode())
h.update(b"\0")
h.update(path.read_bytes()) h.update(path.read_bytes())
for name in _DOCKERFILES: for name in _DOCKERFILES:
p = _REPO_ROOT / name
h.update(name.encode()) h.update(name.encode())
h.update(p.read_bytes()) h.update(b"\0")
h.update((repo_root / name).read_bytes())
h.update(b"pyproject.toml\0")
h.update((repo_root / "pyproject.toml").read_bytes())
h.update(b"dropbear\0")
dropbear = util.dropbear_path()
h.update(dropbear.read_bytes() if dropbear.is_file() else b"<missing>")
h.update(b"init\0") h.update(b"init\0")
h.update(init_script.encode()) h.update(init_script.encode())
return h.hexdigest()[:16] return h.hexdigest()[:16]
@@ -101,6 +116,7 @@ def artifact_url(version: str, filename: str) -> str:
_GZ_NAME = "rootfs.ext4.gz" _GZ_NAME = "rootfs.ext4.gz"
_SHA_NAME = "rootfs.ext4.gz.sha256" _SHA_NAME = "rootfs.ext4.gz.sha256"
_CANDIDATE_DIR_ENV = "BOT_BOTTLE_INFRA_ARTIFACT_DIR"
def _cache_root(version: str) -> Path: def _cache_root(version: str) -> Path:
@@ -150,6 +166,33 @@ def ensure_artifact_gz(version: str) -> Path:
"""The verified, cached `rootfs.ext4.gz` for `version` — downloading it (and """The verified, cached `rootfs.ext4.gz` for `version` — downloading it (and
its `.sha256`) once, then reusing it. Fail-closed on a checksum mismatch: its `.sha256`) once, then reusing it. Fail-closed on a checksum mismatch:
the partial is removed and we die rather than boot an unverified rootfs.""" the partial is removed and we die rather than boot an unverified rootfs."""
candidate_dir = os.environ.get(_CANDIDATE_DIR_ENV, "").strip()
if candidate_dir:
root = Path(candidate_dir)
version_file = root / "version.txt"
# Guard the read so a missing version.txt is a clean error, not a raw
# FileNotFoundError.
if not version_file.is_file():
die(f"infra candidate bundle is incomplete: {root}")
declared = version_file.read_text(encoding="utf-8").strip()
if declared != version:
die(
f"infra candidate version mismatch: expected {version}, "
f"bundle contains {declared or '<empty>'}"
)
gz = root / _GZ_NAME
sha = root / _SHA_NAME
if not gz.is_file() or not sha.is_file():
die(f"infra candidate bundle is incomplete: {root}")
expected = sha.read_text().split()[0].strip().lower()
actual = _sha256_file(gz)
if actual != expected:
die(
f"infra candidate checksum mismatch for {version}:\n"
f" expected {expected}\n actual {actual}"
)
return gz
root = _cache_root(version) root = _cache_root(version)
root.mkdir(parents=True, exist_ok=True) root.mkdir(parents=True, exist_ok=True)
gz = root / _GZ_NAME gz = root / _GZ_NAME
+82 -18
View File
@@ -33,6 +33,7 @@ from pathlib import Path
from typing import Generator from typing import Generator
from ...log import die, info from ...log import die, info
from .. import util as backend_util
from ..docker import util as docker_mod from ..docker import util as docker_mod
from ..docker.gateway_provision import GatewayProvisionError from ..docker.gateway_provision import GatewayProvisionError
from . import firecracker_vm, infra_artifact, netpool, util from . import firecracker_vm, infra_artifact, netpool, util
@@ -93,19 +94,18 @@ class InfraVm:
"""The gateway's mitmproxy CA (PEM) that agents install to trust its """The gateway's mitmproxy CA (PEM) that agents install to trust its
TLS interception. Generated a moment after boot, so this polls over TLS interception. Generated a moment after boot, so this polls over
SSH until it appears (mirrors DockerGateway.ca_cert_pem).""" SSH until it appears (mirrors DockerGateway.ca_cert_pem)."""
deadline = time.monotonic() + timeout def _fetch() -> str | None:
while True:
proc = subprocess.run( proc = subprocess.run(
util.ssh_base_argv(self.private_key, self.guest_ip) util.ssh_base_argv(self.private_key, self.guest_ip)
+ [f"cat {_GATEWAY_CA_PATH}"], + [f"cat {_GATEWAY_CA_PATH}"],
capture_output=True, text=True, timeout=15, check=False, capture_output=True, text=True, timeout=15, check=False,
) )
if proc.returncode == 0 and "BEGIN CERTIFICATE" in proc.stdout: ok = proc.returncode == 0 and "BEGIN CERTIFICATE" in proc.stdout
return proc.stdout return proc.stdout if ok else None
if time.monotonic() >= deadline: try:
die(f"gateway CA not available after {timeout:g}s: " return backend_util.poll_ca_cert(_fetch, timeout=timeout)
f"{proc.stderr.strip() or 'empty'}") except TimeoutError as exc:
time.sleep(_HEALTH_POLL_SECONDS) die(str(exc))
def ensure_built() -> None: def ensure_built() -> None:
@@ -125,16 +125,19 @@ def ensure_built() -> None:
def build_infra_images_with_docker() -> None: def build_infra_images_with_docker() -> None:
"""Build the three fixed images from source with host Docker: orchestrator, """Build the four fixed images from source with host Docker: orchestrator,
gateway, then the combined infra image (`COPY --from` orchestrator, `FROM` gateway, the shared infra base (Dockerfile.infra), then the Firecracker
gateway). The launch host uses this only in `BOT_BOTTLE_INFRA_BUILD=local` infra image (Dockerfile.infra.fc: FROM infra + buildah). The launch host
mode; `publish_infra` uses it off-host to produce the published artifact.""" uses this only in `BOT_BOTTLE_INFRA_BUILD=local` mode; `publish_infra`
uses it off-host to produce the published artifact."""
docker_mod.build_image( docker_mod.build_image(
_ORCHESTRATOR_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.orchestrator") _ORCHESTRATOR_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.orchestrator")
docker_mod.build_image( docker_mod.build_image(
_GATEWAY_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.gateway") _GATEWAY_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.gateway")
docker_mod.build_image( docker_mod.build_image(
_INFRA_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.infra") "bot-bottle-infra:latest", str(_REPO_ROOT), dockerfile="Dockerfile.infra")
docker_mod.build_image(
_INFRA_IMAGE, str(_REPO_ROOT), dockerfile="Dockerfile.infra.fc")
def build_infra_rootfs_dir() -> Path: def build_infra_rootfs_dir() -> Path:
@@ -161,20 +164,23 @@ def ensure_running() -> InfraVm:
slot = netpool.orch_slot() slot = netpool.orch_slot()
url = f"http://{slot.guest_ip}:{CONTROL_PLANE_PORT}" url = f"http://{slot.guest_ip}:{CONTROL_PLANE_PORT}"
key = _infra_dir() / "id_ed25519" key = _infra_dir() / "id_ed25519"
if key.exists() and _health_ok(url): want = _expected_version()
if _adoptable(key, url, want):
info(f"adopting running infra VM at {url}") info(f"adopting running infra VM at {url}")
return InfraVm(guest_ip=slot.guest_ip, private_key=key) return InfraVm(guest_ip=slot.guest_ip, private_key=key)
with _singleton_lock(): with _singleton_lock():
# Re-check under the lock: another launcher may have booted it while # Re-check under the lock: another launcher may have booted it while
# we waited for the lock (double-checked, so we adopt not re-boot). # we waited for the lock (double-checked, so we adopt not re-boot).
if key.exists() and _health_ok(url): if _adoptable(key, url, want):
info(f"adopting running infra VM at {url}") info(f"adopting running infra VM at {url}")
return InfraVm(guest_ip=slot.guest_ip, private_key=key) return InfraVm(guest_ip=slot.guest_ip, private_key=key)
stop() # clear a stale/hung VM holding the link before booting fresh # Clear a stale/hung/OUTDATED VM holding the link before booting fresh.
stop()
ensure_built() ensure_built()
infra = boot() infra = boot()
wait_for_health(infra) wait_for_health(infra)
_record_booted_version(want)
return infra return infra
@@ -193,9 +199,15 @@ def _singleton_lock() -> Generator[None, None, None]:
def stop() -> None: def stop() -> None:
"""Stop the infra VM singleton (idempotent — absent is success).""" """Stop the infra VM singleton (idempotent — absent is success). Reaps the
recorded VMM AND any orphaned firecracker still bound to the infra config —
the PID file drifts after crashes / out-of-band kills, and a survivor would
hold the orchestrator TAP so the next boot dies with "tap … Resource busy".
Drops the version marker so a stopped VM is never treated as adoptable."""
_kill_pidfile() _kill_pidfile()
_kill_infra_firecrackers()
_pid_file().unlink(missing_ok=True) _pid_file().unlink(missing_ok=True)
_version_file().unlink(missing_ok=True)
def boot() -> InfraVm: def boot() -> InfraVm:
@@ -238,6 +250,36 @@ def _pid_file() -> Path:
return _infra_dir() / "vm.pid" return _infra_dir() / "vm.pid"
def _version_file() -> Path:
"""Records the infra-artifact version the *running* VM booted from, so a
later launcher can tell whether the singleton it found is the current code.
Without it, a healthy VM built from an older image gets adopted forever and
the new code never boots — every infra change would need an out-of-band
kill to dislodge the stale VM (and races whatever launched next)."""
return _infra_dir() / "booted-version"
def _expected_version() -> str:
return infra_artifact.infra_artifact_version(_infra_init())
def _adoptable(key: Path, url: str, want: str) -> bool:
"""Adopt a running infra VM only if it booted from the CURRENT version and
its control plane is healthy. A missing/mismatched marker means a prior
launcher booted an older infra image — reboot rather than reuse stale code."""
if not key.exists():
return False
try:
booted = _version_file().read_text(encoding="utf-8").strip()
except OSError:
return False
return booted == want and _health_ok(url)
def _record_booted_version(version: str) -> None:
_version_file().write_text(version + "\n", encoding="utf-8")
# The registry "volume": a host-side ext4 file attached to the infra VM as a # The registry "volume": a host-side ext4 file attached to the infra VM as a
# second virtio-block device (guest /dev/vdb), mounted at the control plane's # second virtio-block device (guest /dev/vdb), mounted at the control plane's
# DB dir. It outlives the ephemeral rootfs, so the bottle registry survives an # DB dir. It outlives the ephemeral rootfs, so the bottle registry survives an
@@ -309,6 +351,28 @@ def _kill_pidfile() -> None:
pass pass
def _kill_infra_firecrackers(proc_root: Path = Path("/proc")) -> None:
"""SIGKILL any firecracker VMM whose `--config-file` is this host's infra
config, independent of the PID file — reaps orphans it lost track of so the
orchestrator TAP is free to rebind. Scoped to the infra config path, so the
interactive pool's agent/infra VMs (other config paths) are untouched."""
cfg = str(_infra_dir() / "config.json")
for entry in proc_root.iterdir():
if not entry.name.isdigit():
continue
try:
if (entry / "comm").read_text().strip() != "firecracker":
continue
args = (entry / "cmdline").read_bytes().split(b"\0")
except OSError:
continue # process vanished / not ours
if any(a.decode("utf-8", "replace") == cfg for a in args):
try:
os.kill(int(entry.name), signal.SIGKILL)
except (OSError, ValueError):
pass
def _health_ok(url: str) -> bool: def _health_ok(url: str) -> bool:
try: try:
with urllib.request.urlopen(f"{url}/health", timeout=1.0) as resp: with urllib.request.urlopen(f"{url}/health", timeout=1.0) as resp:
@@ -433,7 +497,7 @@ BOT_BOTTLE_ROOT=/var/lib/bot-bottle python3 -m bot_bottle.orchestrator \\
BOT_BOTTLE_GATEWAY_DAEMONS=egress,git-http,supervise \\ BOT_BOTTLE_GATEWAY_DAEMONS=egress,git-http,supervise \\
BOT_BOTTLE_ORCHESTRATOR_URL=http://127.0.0.1:{CONTROL_PLANE_PORT} \\ BOT_BOTTLE_ORCHESTRATOR_URL=http://127.0.0.1:{CONTROL_PLANE_PORT} \\
SUPERVISE_DB_PATH=/var/lib/bot-bottle/db/bot-bottle.db \\ SUPERVISE_DB_PATH=/var/lib/bot-bottle/db/bot-bottle.db \\
python3 /app/gateway_init.py & python3 -m bot_bottle.gateway_init &
# Reap as PID 1; children are backgrounded, so `wait` blocks. # Reap as PID 1; children are backgrounded, so `wait` blocks.
while : ; do wait ; done while : ; do wait ; done
+54 -22
View File
@@ -1,7 +1,8 @@
"""Launch flow for the Firecracker backend (PRD 0070, consolidated). """Launch flow for the Firecracker backend (PRD 0070, consolidated).
Per bottle: Per bottle:
1. build the agent image (docker), export it to a cached ext4 rootfs; 1. build the agent rootfs in a builder VM (buildah, no host docker), or
resume a frozen bottle from its committed rootfs tar; cache the ext4;
2. ensure the per-host orchestrator + shared gateway are up; 2. ensure the per-host orchestrator + shared gateway are up;
3. claim a free TAP pool slot (rootless flock); 3. claim a free TAP pool slot (rootless flock);
4. register the bottle on the orchestrator by the VM's guest IP (the 4. register the bottle on the orchestrator by the VM's guest IP (the
@@ -25,12 +26,14 @@ from __future__ import annotations
import dataclasses import dataclasses
import os import os
import shutil
from contextlib import ExitStack, contextmanager from contextlib import ExitStack, contextmanager
from pathlib import Path from pathlib import Path
from typing import Callable, Generator from typing import Callable, Generator
from ...agent_provider import runtime_for from ...agent_provider import runtime_for
from ...bottle_state import ( from ...bottle_state import (
committed_rootfs_path,
egress_state_dir, egress_state_dir,
git_gate_state_dir, git_gate_state_dir,
read_committed_image, read_committed_image,
@@ -43,14 +46,15 @@ from ...git_gate import (
provision_git_gate_dynamic_keys, provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys, revoke_git_gate_provisioned_keys,
) )
from ...log import info, warn from ...image_cache import check_stale_path
from ...log import die, info, warn
from ...supervise import SUPERVISE_PORT from ...supervise import SUPERVISE_PORT
from ..docker import util as docker_mod
from ..docker.egress import EGRESS_PORT from ..docker.egress import EGRESS_PORT
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import firecracker_vm, image_builder, isolation_probe, netpool, util from . import firecracker_vm, image_builder, isolation_probe, netpool, util
from .bottle import FirecrackerBottle from .bottle import FirecrackerBottle
from .bottle_plan import FirecrackerBottlePlan from .bottle_plan import FirecrackerBottlePlan
from ...orchestrator.config_store import resolve_teardown_timeout
from .consolidated_launch import ( from .consolidated_launch import (
launch_consolidated, launch_consolidated,
teardown_consolidated, teardown_consolidated,
@@ -63,6 +67,7 @@ _GIT_HTTP_PORT = 9420
@contextmanager @contextmanager
def launch( def launch(
plan: FirecrackerBottlePlan, plan: FirecrackerBottlePlan,
agent_base: Path,
*, *,
provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None], provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None],
) -> Generator[FirecrackerBottle, None, None]: ) -> Generator[FirecrackerBottle, None, None]:
@@ -84,11 +89,9 @@ def launch(
raise teardown_exc raise teardown_exc
try: try:
# Step 1: agent rootfs. Built from the Dockerfile inside a Firecracker # Step 1 (rootfs resolution/build) runs in BottleBackend.launch before
# builder VM (buildah, no host docker); a committed snapshot is reused # this context starts resources. ``agent_base`` is the selected cache,
# when present. Returns the base dir the per-bottle ext4 is made from. # fresh build, or committed snapshot.
plan, agent_base = _build_agent_base(plan)
# Step 2: mint the git-gate dynamic (gitea) deploy keys, if any. # Step 2: mint the git-gate dynamic (gitea) deploy keys, if any.
git_gate_plan = plan.git_gate_plan git_gate_plan = plan.git_gate_plan
if git_gate_plan.upstreams: if git_gate_plan.upstreams:
@@ -111,6 +114,7 @@ def launch(
token_values = egress_resolve_token_values( token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env, plan.egress_plan.token_env_map, effective_env,
) )
teardown_timeout = resolve_teardown_timeout()
ctx = launch_consolidated( ctx = launch_consolidated(
plan.egress_plan, git_gate_plan, plan.egress_plan, git_gate_plan,
guest_ip=slot.guest_ip, guest_ip=slot.guest_ip,
@@ -120,6 +124,7 @@ def launch(
stack.callback( stack.callback(
teardown_consolidated, ctx.bottle_id, teardown_consolidated, ctx.bottle_id,
orchestrator_url=ctx.orchestrator_url, orchestrator_url=ctx.orchestrator_url,
timeout=teardown_timeout,
) )
# Step 5: install the SHARED gateway CA (replaces the per-bottle CA). # Step 5: install the SHARED gateway CA (replaces the per-bottle CA).
@@ -163,6 +168,10 @@ def launch(
# Step 6: build the per-bottle rootfs + SSH key, then boot. # Step 6: build the per-bottle rootfs + SSH key, then boot.
run_dir = util.cache_dir() / "run" / plan.slug run_dir = util.cache_dir() / "run" / plan.slug
run_dir.mkdir(parents=True, exist_ok=True) run_dir.mkdir(parents=True, exist_ok=True)
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
# doesn't leak. Registered before vm.terminate below so it runs *after*
# it (ExitStack is LIFO): the VM is gone before we rm its rootfs.
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
rootfs = run_dir / "rootfs.ext4" rootfs = run_dir / "rootfs.ext4"
util.build_rootfs_ext4(agent_base, rootfs) util.build_rootfs_ext4(agent_base, rootfs)
private_key, pubkey = util.generate_keypair(run_dir) private_key, pubkey = util.generate_keypair(run_dir)
@@ -205,27 +214,45 @@ def launch(
teardown() teardown()
def _build_agent_base( def build_or_load_agent_base(plan: FirecrackerBottlePlan) -> Path:
plan: FirecrackerBottlePlan,
) -> tuple[FirecrackerBottlePlan, Path]:
"""Produce the agent's base rootfs dir. Primary path: build the Dockerfile """Produce the agent's base rootfs dir. Primary path: build the Dockerfile
inside a Firecracker builder VM (buildah, no host docker), smoke-testing inside a Firecracker builder VM (buildah, no host docker), smoke-testing
the image before export. A committed snapshot (freeze/migrate) is still the image before export. A committed snapshot (freeze/migrate) is resumed
exported via the host docker path until that is ported too.""" directly from the rootfs tar the freezer wrote no host docker either."""
committed = read_committed_image(plan.slug) committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed): committed_tar = committed_rootfs_path(plan.slug)
info(f"using committed image {committed!r}") if committed and committed_tar.is_file():
plan = dataclasses.replace( info(f"resuming from committed rootfs {committed_tar}")
plan, return util.build_committed_rootfs_dir(committed_tar)
agent_provision=dataclasses.replace(plan.agent_provision, image=committed), dockerfile = Path(plan.dockerfile_path)
if plan.spec.image_policy == "cached":
cached = image_builder.cached_agent_rootfs_dir(dockerfile)
if cached is None:
die(
f"cached agent rootfs for {plan.image!r} not found; "
"run without --cached-images to build it"
) )
return plan, util.build_base_rootfs_dir(committed) info(f"using cached agent rootfs {cached.name}")
base = image_builder.build_agent_rootfs_dir( return cached
Path(plan.dockerfile_path), return image_builder.build_agent_rootfs_dir(
dockerfile,
image_tag=plan.image, image_tag=plan.image,
smoke_test=runtime_for(plan.agent_provider_template).smoke_test, smoke_test=runtime_for(plan.agent_provider_template).smoke_test,
) )
return plan, base
def stale_checks(plan: FirecrackerBottlePlan) -> None:
"""Raise when the cached rootfs selected by this plan is stale."""
if plan.spec.image_policy != "cached":
return
committed = read_committed_image(plan.slug)
committed_tar = committed_rootfs_path(plan.slug)
if committed and committed_tar.is_file():
check_stale_path(f"agent rootfs {committed_tar}", committed_tar)
return
cached = image_builder.cached_agent_rootfs_dir(Path(plan.dockerfile_path))
if cached is not None:
check_stale_path(f"agent rootfs {cached}", cached / ".bb-ready")
# --- agent guest env ------------------------------------------------- # --- agent guest env -------------------------------------------------
@@ -241,6 +268,11 @@ def _agent_guest_env(plan: FirecrackerBottlePlan, host_ip: str) -> dict[str, str
"HTTPS_PROXY": proxy_url, "HTTP_PROXY": proxy_url, "HTTPS_PROXY": proxy_url, "HTTP_PROXY": proxy_url,
"https_proxy": proxy_url, "http_proxy": proxy_url, "https_proxy": proxy_url, "http_proxy": proxy_url,
"NO_PROXY": no_proxy, "no_proxy": no_proxy, "NO_PROXY": no_proxy, "no_proxy": no_proxy,
# Rootfs export can leave Git's implicit XDG paths unreadable even
# after the runtime repair. Bypass that discovery and name the
# provisioned global config explicitly so insteadOf can never fall
# through to the credential-bearing upstream URL.
"GIT_CONFIG_GLOBAL": f"{plan.guest_home}/.gitconfig",
"NODE_EXTRA_CA_CERTS": AGENT_CA_PATH, "NODE_EXTRA_CA_CERTS": AGENT_CA_PATH,
"SSL_CERT_FILE": AGENT_CA_BUNDLE, "SSL_CERT_FILE": AGENT_CA_BUNDLE,
"REQUESTS_CA_BUNDLE": AGENT_CA_BUNDLE, "REQUESTS_CA_BUNDLE": AGENT_CA_BUNDLE,
+129 -21
View File
@@ -5,13 +5,14 @@ never on the launch host. It runs the same pipeline the launch host used to run
locally `docker build` the three fixed images, export to a rootfs dir, inject locally `docker build` the three fixed images, export to a rootfs dir, inject
the guest boot, `mke2fs` to an ext4 with the buildah build slack then gzips the guest boot, `mke2fs` to an ext4 with the buildah build slack then gzips
the ext4 and PUTs it (plus a `.sha256`) to the ext4 and PUTs it (plus a `.sha256`) to
`/api/packages/<owner>/generic/bot-bottle-infra/<version>/`. `/api/packages/<owner>/generic/bot-bottle-firecracker-infra/<version>/`.
The `<version>` is `infra_artifact.infra_artifact_version(...)`, the content The `<version>` is `infra_artifact.infra_artifact_version(...)`, the content
hash of the rootfs inputs, so a launch host at the same code checkout resolves hash of the rootfs inputs, so a launch host at the same code checkout resolves
the exact artifact this produced. the exact artifact this produced.
python3 -m bot_bottle.backend.firecracker.publish_infra [--dry-run] [--force] python3 -m bot_bottle.backend.firecracker.publish_infra --output DIR
python3 -m bot_bottle.backend.firecracker.publish_infra --publish-dir DIR
Auth: a token with `write:package` on the target owner, from Auth: a token with `write:package` on the target owner, from
`BOT_BOTTLE_INFRA_ARTIFACT_TOKEN`. `BOT_BOTTLE_INFRA_ARTIFACT_TOKEN`.
@@ -24,7 +25,6 @@ import gzip
import hashlib import hashlib
import shutil import shutil
import sys import sys
import tempfile
import urllib.error import urllib.error
import urllib.request import urllib.request
from pathlib import Path from pathlib import Path
@@ -33,6 +33,18 @@ from . import infra_artifact, infra_vm, util
_CHUNK = 1 << 20 _CHUNK = 1 << 20
# A human-readable description shipped alongside the artifact — generic packages
# have no description field, so this file *is* the description on the package
# page. Uploaded on every publish so it never goes stale.
_ABOUT_NAME = "about.txt"
_ABOUT_TEXT = (
"bot-bottle infra rootfs for the Firecracker backend (PRD 0069 Stage 2, "
"#348): the per-host infra VM (orchestrator control plane + gateway + "
"buildah). Prebuilt off-host, gzip ext4; the launch host downloads + "
"sha256-verifies + boots it, no host Docker. The version tag is a content "
"hash of the rootfs inputs. Files: rootfs.ext4.gz + rootfs.ext4.gz.sha256.\n"
)
def _gzip(src: Path, dest: Path) -> None: def _gzip(src: Path, dest: Path) -> None:
with open(src, "rb") as fh, gzip.open(dest, "wb") as out: with open(src, "rb") as fh, gzip.open(dest, "wb") as out:
@@ -47,8 +59,22 @@ def _sha256(path: Path) -> str:
return h.hexdigest() return h.hexdigest()
def _put(url: str, body: bytes, token: str) -> None: def _put(url: str, body: "bytes | Path", token: str) -> None:
req = urllib.request.Request(url, data=body, method="PUT") """PUT `body` (raw bytes, or a Path streamed from disk) to `url`. The rootfs
is hundreds of MB, so it is passed as a Path and streamed `urlopen` reads
the open file in blocks rather than materializing it in memory (with an
explicit Content-Length, which Gitea requires and which also stops urllib
from `len()`-ing a non-bytes body)."""
handle = None
if isinstance(body, Path):
length = body.stat().st_size
handle = open(body, "rb")
data: object = handle
else:
length = len(body)
data = body
req = urllib.request.Request(url, data=data, method="PUT") # type: ignore[arg-type]
req.add_header("Content-Length", str(length))
if token: if token:
req.add_header("Authorization", f"token {token}") req.add_header("Authorization", f"token {token}")
req.add_header("Content-Type", "application/octet-stream") req.add_header("Content-Type", "application/octet-stream")
@@ -64,6 +90,9 @@ def _put(url: str, body: bytes, token: str) -> None:
raise SystemExit(f"upload failed (HTTP {e.code}): {url}\n{e.read().decode(errors='replace')}") raise SystemExit(f"upload failed (HTTP {e.code}): {url}\n{e.read().decode(errors='replace')}")
except urllib.error.URLError as e: except urllib.error.URLError as e:
raise SystemExit(f"registry unreachable: {url} ({e.reason})") raise SystemExit(f"registry unreachable: {url} ({e.reason})")
finally:
if handle is not None:
handle.close()
def _delete(url: str, token: str) -> None: def _delete(url: str, token: str) -> None:
@@ -102,33 +131,112 @@ def build_artifact(out_dir: Path) -> tuple[str, Path, Path]:
return version, gz, sha return version, gz, sha
def _try_download_published(out_dir: Path) -> tuple[str, Path, Path] | None:
"""If this version's artifact is already in the registry, download the gz
and sha to out_dir and return (version, gz_path, sha_path). Returns None
when not yet published."""
version = infra_artifact.infra_artifact_version(infra_vm._infra_init())
sha_url = infra_artifact.artifact_url(version, "rootfs.ext4.gz.sha256")
try:
with urllib.request.urlopen(infra_artifact._open(sha_url)):
pass
except urllib.error.HTTPError as e:
if e.code == 404:
return None
raise SystemExit(f"registry check failed (HTTP {e.code}): {sha_url}")
except urllib.error.URLError as e:
raise SystemExit(f"registry unreachable: {sha_url} ({e.reason})")
print(f"infra rootfs {version} already published — downloading instead of building")
gz = out_dir / "rootfs.ext4.gz"
sha = out_dir / "rootfs.ext4.gz.sha256"
infra_artifact._download(infra_artifact.artifact_url(version, "rootfs.ext4.gz"), gz)
infra_artifact._download(infra_artifact.artifact_url(version, "rootfs.ext4.gz.sha256"), sha)
return version, gz, sha
def _publish_bundle(root: Path, token: str) -> str:
version_file = root / "version.txt"
# Guard the read so a missing version.txt is a clean error, not a raw
# FileNotFoundError.
if not version_file.is_file():
raise SystemExit(f"incomplete artifact bundle: {root}")
version = version_file.read_text(encoding="utf-8").strip()
expected = infra_artifact.infra_artifact_version(infra_vm._infra_init())
if version != expected:
raise SystemExit(
f"artifact bundle version {version!r} does not match checkout {expected!r}"
)
gz = root / "rootfs.ext4.gz"
sha = root / "rootfs.ext4.gz.sha256"
if not gz.is_file() or not sha.is_file():
raise SystemExit(f"incomplete artifact bundle: {root}")
expected_sha = sha.read_text().split()[0].strip().lower()
if _sha256(gz) != expected_sha:
raise SystemExit("artifact bundle checksum mismatch")
gz_url = infra_artifact.artifact_url(version, gz.name)
sha_url = infra_artifact.artifact_url(version, sha.name)
about_url = infra_artifact.artifact_url(version, _ABOUT_NAME)
# Publishing is idempotent. If this exact complete artifact is already
# present, a test-only main commit is a no-op. Otherwise clear any partial
# upload left by an interrupted prior attempt and upload the complete set.
try:
with urllib.request.urlopen(infra_artifact._open(sha_url)) as resp:
remote_sha = resp.read().decode("utf-8").split()[0].strip().lower()
except urllib.error.HTTPError as e:
if e.code != 404:
raise SystemExit(f"checking existing artifact failed (HTTP {e.code})")
remote_sha = ""
except urllib.error.URLError as e:
raise SystemExit(f"registry unreachable: {sha_url} ({e.reason})")
if remote_sha == expected_sha:
print(f"infra rootfs {version} already published")
return version
for url in (gz_url, sha_url, about_url):
_delete(url, token)
_put(gz_url, gz, token)
_put(sha_url, sha.read_bytes(), token)
_put(about_url, _ABOUT_TEXT.encode(), token)
return version
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
prog="publish_infra", description="Build + publish the infra rootfs artifact.") prog="publish_infra", description="Build + publish the infra rootfs artifact.")
parser.add_argument("--dry-run", action="store_true", mode = parser.add_mutually_exclusive_group(required=True)
help="build the artifact but do not upload") mode.add_argument("--output", type=Path,
parser.add_argument("--force", action="store_true", help="build a candidate bundle in DIR without publishing")
help="overwrite an already-published artifact of this version") mode.add_argument("--publish-dir", type=Path,
help="publish an already-built and tested candidate bundle")
parser.add_argument("--reuse-published", action="store_true",
help="with --output: download from registry if already published instead of building")
args = parser.parse_args(argv) args = parser.parse_args(argv)
_, _, token = infra_artifact._config() _, _, token = infra_artifact._config()
if not args.dry_run and not token: if args.publish_dir is not None and not token:
raise SystemExit( raise SystemExit(
"no publish token: set BOT_BOTTLE_INFRA_ARTIFACT_TOKEN to a token " "no publish token: set BOT_BOTTLE_INFRA_ARTIFACT_TOKEN to a token "
"with write:package") "with write:package")
with tempfile.TemporaryDirectory(prefix="bb-publish-infra.") as tmp: if args.output is not None:
version, gz, sha = build_artifact(Path(tmp)) args.output.mkdir(parents=True, exist_ok=True)
gz_url = infra_artifact.artifact_url(version, gz.name) reused = None
sha_url = infra_artifact.artifact_url(version, sha.name) if args.reuse_published:
if args.dry_run: reused = _try_download_published(args.output)
print(f"dry-run: would upload -> {gz_url}") if reused is not None:
version, _, _ = reused
(args.output / "version.txt").write_text(version + "\n", encoding="utf-8")
print(f"reused published infra rootfs candidate {version}")
return 0 return 0
if args.force: version, _gz, _sha = build_artifact(args.output)
_delete(gz_url, token) (args.output / "version.txt").write_text(version + "\n", encoding="utf-8")
_delete(sha_url, token) print(f"built infra rootfs candidate {version}")
_put(gz_url, gz.read_bytes(), token) return 0
_put(sha_url, sha.read_bytes(), token)
assert args.publish_dir is not None
version = _publish_bundle(args.publish_dir, token)
print(f"published infra rootfs {version}") print(f"published infra rootfs {version}")
return 0 return 0
+84 -7
View File
@@ -14,6 +14,7 @@ and `./cli.py backend setup --backend=firecracker`.
from __future__ import annotations from __future__ import annotations
import hashlib
import os import os
import platform import platform
import shutil import shutil
@@ -87,7 +88,7 @@ def require_firecracker() -> None:
booting a VM without it.""" booting a VM without it."""
if not is_linux(): if not is_linux():
die("firecracker backend is only supported on Linux (KVM). " die("firecracker backend is only supported on Linux (KVM). "
"On macOS use --backend=macos-container.") "On macOS use the macos-container backend.")
if shutil.which("firecracker") is None: if shutil.which("firecracker") is None:
info("Firecracker is required but was not found on PATH.") info("Firecracker is required but was not found on PATH.")
info("Install: https://github.com/firecracker-microvm/firecracker/releases") info("Install: https://github.com/firecracker-microvm/firecracker/releases")
@@ -212,15 +213,80 @@ def build_base_rootfs_dir(
return base return base
def build_committed_rootfs_dir(tar_path: Path) -> Path:
"""Prepare a base rootfs dir from a frozen-bottle snapshot tar (the
freeze/resume path no Docker). Extracts the snapshot, recreates the
virtual mount points the freezer excluded, and injects the guest init +
static dropbear, mirroring `build_base_rootfs_dir` but sourced from a tar
we control rather than a Docker image.
Cached under the rootfs cache, keyed by the tar's size+mtime so a
re-freeze re-extracts but repeated resumes of the same snapshot don't.
Returns the prepared directory (read as the `mke2fs -d` source)."""
st = tar_path.stat()
fingerprint = hashlib.sha256(
f"{tar_path}:{st.st_size}:{st.st_mtime_ns}".encode()
).hexdigest()[:16]
base = cache_dir() / "rootfs" / f"committed-{fingerprint}"
ready = base / ".bb-ready"
if ready.is_file():
return base
if base.exists():
shutil.rmtree(base, ignore_errors=True)
base.mkdir(parents=True)
info(f"extracting committed rootfs {tar_path} -> {base}")
result = subprocess.run(
["tar", "-x", "-f", str(tar_path), "-C", str(base)],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"extracting committed rootfs {tar_path} failed: "
f"{result.stderr.strip() or '<no stderr>'}")
# The freezer excludes the live/virtual filesystems from the snapshot;
# recreate them as empty mount points so the guest init can mount
# proc/sys/dev and dropbear has a writable /run.
for mount_point in ("proc", "sys", "dev", "run"):
(base / mount_point).mkdir(mode=0o755, exist_ok=True)
inject_guest_boot(base)
ready.write_text("ok\n")
return base
def inject_guest_boot(rootfs: Path, init_script: str | None = None) -> None: def inject_guest_boot(rootfs: Path, init_script: str | None = None) -> None:
"""Drop the static dropbear and the PID-1 init into the rootfs. """Drop the static dropbear and the PID-1 init into the rootfs.
`init_script` defaults to the SSH-only agent init; the infra VM `init_script` defaults to the SSH-only agent init; the infra VM
passes its own (control plane + gateway) init.""" passes its own (control plane + gateway) init.
shutil.copy2(dropbear_path(), rootfs / "bb-dropbear")
os.chmod(rootfs / "bb-dropbear", 0o755) A committed snapshot is guest-controlled, so `bb-dropbear`/`bb-init`
init = rootfs / "bb-init" may already exist as symlinks aimed at a host file (e.g. bb-init ->
init.write_text(init_script or _GUEST_INIT) ~/.bashrc). Replace whatever is there and create the files with
os.chmod(init, 0o755) O_EXCL|O_NOFOLLOW so the write always lands a fresh regular file in
the staging tree and never follows a planted symlink out of it."""
_write_staged_file(rootfs / "bb-dropbear", dropbear_path().read_bytes())
_write_staged_file(rootfs / "bb-init", (init_script or _GUEST_INIT).encode())
def _write_staged_file(path: Path, data: bytes) -> None:
"""Write `data` to `path` (mode 0755) as a fresh regular file inside a
staging rootfs, replacing any pre-existing entry without following a
symlink at `path`. Fails closed on anything unexpected there."""
if path.is_symlink() or path.exists():
if path.is_dir() and not path.is_symlink():
shutil.rmtree(path)
else:
path.unlink()
fd = os.open(
path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o755
)
try:
os.write(fd, data)
finally:
os.close(fd)
os.chmod(path, 0o755)
def build_rootfs_ext4(base_dir: Path, out_path: Path, *, slack_mib: int = 1024) -> None: def build_rootfs_ext4(base_dir: Path, out_path: Path, *, slack_mib: int = 1024) -> None:
@@ -302,6 +368,17 @@ mount -t devtmpfs dev /dev 2>/dev/null
mkdir -p /dev/pts && mount -t devpts devpts /dev/pts 2>/dev/null mkdir -p /dev/pts && mount -t devpts devpts /dev/pts 2>/dev/null
mount -o remount,rw / 2>/dev/null mount -o remount,rw / 2>/dev/null
# /tmp must be world-writable + sticky. The rootless rootfs build can land
# it 0755/root-owned, leaving the agent (uid 1000 node) unable to create
# scratch dirs there — git worktrees, build temp, `git init /tmp/...`, etc.
mkdir -p /tmp && chmod 1777 /tmp
# Rootfs export also maps the image's original owners to the unprivileged
# host build uid. That uid is not guaranteed to be node's uid in the guest;
# restore the home-directory boundary before any SSH provisioning runs.
chown node:node /home/node 2>/dev/null || true
chmod 755 /home/node 2>/dev/null || true
# Install the per-bottle SSH pubkey from the kernel cmdline. # Install the per-bottle SSH pubkey from the kernel cmdline.
KEY=$(sed -n 's/.*bb_pubkey=\([^ ]*\).*/\1/p' /proc/cmdline | base64 -d 2>/dev/null) KEY=$(sed -n 's/.*bb_pubkey=\([^ ]*\).*/\1/p' /proc/cmdline | base64 -d 2>/dev/null)
if [ -n "$KEY" ]; then if [ -n "$KEY" ]; then
+18 -4
View File
@@ -12,7 +12,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan from ...supervise import SupervisePlan
from ...manifest import Manifest from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup from . import cleanup as _cleanup
from . import enumerate as _enumerate from . import enumerate as _enumerate
from . import launch as _launch from . import launch as _launch
@@ -82,13 +82,27 @@ class MacosContainerBottleBackend(
stage_dir=stage_dir, stage_dir=stage_dir,
) )
def prelaunch_checks(self, plan: MacosContainerBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: MacosContainerBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager @contextmanager
def launch( def _launch_impl(
self, plan: MacosContainerBottlePlan self, plan: MacosContainerBottlePlan, images: BottleImages
) -> Generator[MacosContainerBottle, None, None]: ) -> Generator[MacosContainerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle: with _launch.launch(plan, images, provision=self.provision) as bottle:
yield bottle yield bottle
def ensure_orchestrator(self) -> str:
"""Bring up the per-host infra container (control plane + gateway) and
return its control-plane URL the on-demand entry point operator tools
(`supervise`) call when no control plane is running yet. Mirrors
firecracker's infra-VM bring-up."""
from .infra import MacosInfraService
return MacosInfraService().ensure_running().control_plane_url
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan: def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
return _cleanup.prepare_cleanup() return _cleanup.prepare_cleanup()
+37 -4
View File
@@ -52,6 +52,7 @@ class MacosContainerBottle(Bottle):
terminal_title: str = "", terminal_title: str = "",
terminal_color: str = "", terminal_color: str = "",
agent_workdir: str = "/home/node", agent_workdir: str = "/home/node",
exec_env: dict[str, str] | None = None,
): ):
self.name = container self.name = container
self._teardown = teardown self._teardown = teardown
@@ -62,6 +63,20 @@ class MacosContainerBottle(Bottle):
self.terminal_color = terminal_color self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir self.agent_workdir = agent_workdir
# Env applied to the agent process at `container exec` time, on top of
# what the container was run with. This is how the identity token
# reaches the agent (PRD 0070): registration mints it *after* the
# container exists — its source IP is the registration key and Apple
# Container assigns that by DHCP — so it cannot be in the run-time env
# the way docker's compose spec does it.
#
# `container exec --env` does NOT override a run-time value — it
# appends, leaving duplicate entries in the agent's `environ` whose
# resolution is runtime-specific (Node last-wins, Rust first-wins). So
# nothing here may rely on superseding: the proxy vars are supplied
# *only* at exec time and are deliberately absent from the run-time
# env. See `launch._agent_env_entries`.
self._exec_env = dict(exec_env or {})
self._closed = False self._closed = False
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]: def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
@@ -74,6 +89,12 @@ class MacosContainerBottle(Bottle):
) )
) )
container_exec = ["container", "exec"] container_exec = ["container", "exec"]
# Bare env names, same rule as the terminal hints below: the value
# stays in the child env `exec_agent` builds and never reaches argv —
# the proxy URL here carries the identity token, which `ps` would
# otherwise expose to every process on the host.
for name in sorted(self._exec_env):
container_exec.extend(["--env", name])
if tty: if tty:
container_exec.extend(["--interactive", "--tty"]) container_exec.extend(["--interactive", "--tty"])
# Forward terminal capability hints so TUIs can enable modified-key # Forward terminal capability hints so TUIs can enable modified-key
@@ -94,21 +115,33 @@ class MacosContainerBottle(Bottle):
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int: def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
agent_argv = self.agent_argv(argv, tty=tty) agent_argv = self.agent_argv(argv, tty=tty)
# The values behind the bare `--env` names in `agent_argv`. `sh -lc`
# below is in this process tree, so the child env reaches `container
# exec` either way.
env = {**os.environ, **self._exec_env} if self._exec_env else None
script = ( script = (
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
if tty else None if tty else None
) )
if script is None: if script is None:
return subprocess.run(agent_argv, check=False).returncode return subprocess.run(agent_argv, env=env, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode return subprocess.run(["sh", "-lc", script], env=env, check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult: def exec(self, script: str, *, user: str = "node") -> ExecResult:
# Carry the same exec env the agent gets: provisioning steps run
# through here, and a provider whose provision step fetches anything
# would egress without the identity token and be denied by /resolve.
# Bare `--env NAME` again, so the token stays off argv.
argv = ["container", "exec", "--user", user, "--interactive"]
for name in sorted(self._exec_env):
argv.extend(["--env", name])
argv.extend([self.name, "sh", "-s"])
result = subprocess.run( result = subprocess.run(
["container", "exec", "--user", user, "--interactive", argv,
self.name, "sh", "-s"],
input=script, input=script,
capture_output=True, capture_output=True,
text=True, text=True,
env={**os.environ, **self._exec_env} if self._exec_env else None,
check=False, check=False,
) )
return ExecResult( return ExecResult(
@@ -13,9 +13,13 @@ from .. import BottlePlan
class MacosContainerBottlePlan(BottlePlan): class MacosContainerBottlePlan(BottlePlan):
slug: str slug: str
forwarded_env: dict[str, str] = field(repr=False) forwarded_env: dict[str, str] = field(repr=False)
agent_proxy_url: str = ""
agent_git_gate_url: str = "" agent_git_gate_url: str = ""
agent_supervise_url: str = "" agent_supervise_url: str = ""
# Read by provision-time consumers (git extraHeader, supervise MCP header)
# via getattr(plan, "identity_token", ""); stamped in launch after the
# bottle is registered. See launch.py's stamp for why it lives here and not
# only in the exec-time proxy env.
identity_token: str = ""
@property @property
def container_name(self) -> str: def container_name(self) -> str:
@@ -0,0 +1,178 @@
"""Consolidated bottle launch sequence for the macOS backend (PRD 0070).
The docker backend allocates a free address, pins the agent to it with
`--ip`, registers it, *then* starts the agent registration precedes launch
because the pinned address is known up front.
**Apple Container 1.0.0 has no `--ip`.** The `--network` flag takes only
`<name>[,mac=][,mtu=]`; the address is assigned by vmnet's DHCP and is
knowable only once the container is running. So the macOS order inverts:
ensure_gateway() -> caller starts the agent -> register_agent(source_ip)
That is why this module exposes two functions where docker has one the
caller has to start the agent in between. `ensure_gateway` runs first because
the agent's proxy env needs the gateway's address at `container run` time; the
agent's *own* address (the attribution key) only exists afterwards.
The control plane and the gateway are one **infra container** here (see
`infra`), so `gateway_ip` and the control-plane host are the same address.
The consequence for the identity token: it is minted by registration, i.e.
*after* the agent container exists, so it cannot be baked into the run-time
env the way docker's compose spec does. It is delivered at `container exec`
time instead see `bottle.MacosContainerBottle`.
That delivery is load-bearing, not a nicety: `/resolve` requires a matching
`(source_ip, identity_token)` pair and fail-closes with no source-IP-only
fallback (#366). So egress that does not carry the token is denied — which is
the safe direction, and is why the agent's init process is a bare `sleep` and
every real command arrives through `container exec`.
"""
from __future__ import annotations
from dataclasses import dataclass
from ...egress import EgressPlan
from ...git_gate import GitGatePlan
from ...log import info
from ...orchestrator.client import OrchestratorClient, OrchestratorClientError
from ..consolidated_util import provision_bottle, teardown_consolidated as _teardown_util
from . import util as container_mod
from .enumerate import CONTAINER_NAME_PREFIX, EnumerationError, enumerate_active
from .gateway import GATEWAY_NETWORK
from .gateway_provision import AppleGatewayTransport
from .infra import MacosInfraService, OrchestratorStartError
class ConsolidatedLaunchError(RuntimeError):
"""The consolidated register/provision sequence could not complete."""
@dataclass(frozen=True)
class GatewayEndpoint:
"""What the agent `container run` needs to reach the shared gateway (the
infra container). `gateway_ip` is that container's host-only address, the
same host the control-plane URL points at."""
orchestrator_url: str
gateway_ip: str # the gateway's address — the agent's proxy target
gateway_ca_pem: str # the shared CA the provisioner installs
network: str # the shared host-only network to attach to
@dataclass(frozen=True)
class LaunchContext:
"""What the running agent needs once it has been registered."""
bottle_id: str
identity_token: str
source_ip: str # the agent's DHCP-assigned address (attribution key)
gateway_ip: str
network: str
orchestrator_url: str
def ensure_gateway(
*, service: MacosInfraService | None = None,
) -> GatewayEndpoint:
"""Ensure the per-host infra container (control plane + gateway) is up and
report how to reach it. Idempotent one singleton, so N bottle launches
share it. Call before starting the agent container: the agent's proxy env
needs `gateway_ip` at run time."""
service = service or MacosInfraService()
infra = service.ensure_running()
return GatewayEndpoint(
orchestrator_url=infra.control_plane_url,
gateway_ip=infra.gateway_ip,
gateway_ca_pem=service.ca_cert_pem(),
network=service.network,
)
def live_source_ips(network: str) -> list[str]:
"""Every running agent container's address on `network`.
The reconciliation input: the orchestrator lives inside the infra
container and cannot enumerate the host's containers, so the host has to
tell it which bottles are actually up. Containers that have not been
assigned an address yet contribute nothing the reap's grace window, not
this list, is what protects an in-flight launch.
Raises `EnumerationError` when the live set cannot be determined
authoritatively: either the container listing fails or any individual
inspect fails. Callers must skip reconciliation in that case to avoid
unregistering healthy bottles."""
ips: list[str] = []
for agent in enumerate_active():
name = f"{CONTAINER_NAME_PREFIX}{agent.slug}"
ip = container_mod.inspect_container_network_ip(name, network)
if ip is None:
raise EnumerationError(
f"container inspect {name!r} failed; live set is not authoritative"
)
if ip:
ips.append(ip)
return ips
def register_agent(
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
*,
source_ip: str,
endpoint: GatewayEndpoint,
image_ref: str = "",
tokens: dict[str, str] | None = None,
) -> LaunchContext:
"""Register the (already running) agent by its address and provision its
git-gate state into the gateway. `source_ip` must be read from the live
container it is the attribution key the gateway resolves policy by.
Raises on failure; the caller tears down."""
client = OrchestratorClient(endpoint.orchestrator_url)
# Self-heal before registering: a launcher that died hard (SIGKILL, closed
# terminal, host sleep) never ran its teardown callback, leaving an active
# row with no container. vmnet recycles addresses, so such a row can
# collide with this bottle's — and `by_source_ip` fail-closes on ambiguity,
# which would resolve no policy at all and deny every host. Best-effort: a
# reconciliation failure must not block an otherwise-fine launch.
try:
client.reconcile(live_source_ips(endpoint.network))
except (OrchestratorClientError, EnumerationError) as e:
info(f"registry reconciliation skipped: {e}")
reg = provision_bottle(
client, source_ip, egress_plan, git_gate_plan, AppleGatewayTransport(),
image_ref=image_ref, tokens=tokens,
)
return LaunchContext(
bottle_id=reg.bottle_id,
identity_token=reg.identity_token,
source_ip=source_ip,
gateway_ip=endpoint.gateway_ip,
network=endpoint.network,
orchestrator_url=endpoint.orchestrator_url,
)
def teardown_consolidated(
bottle_id: str, *, orchestrator_url: str, timeout: float | None = None,
) -> None:
"""Deregister the bottle and remove its git-gate state from the gateway.
Both steps are idempotent so this is safe from a cleanup trap. Does NOT
stop the gateway it's a persistent per-host singleton."""
_teardown_util(bottle_id, AppleGatewayTransport(),
orchestrator_url=orchestrator_url, timeout=timeout)
__all__ = [
"GatewayEndpoint",
"LaunchContext",
"ensure_gateway",
"live_source_ips",
"register_agent",
"teardown_consolidated",
"ConsolidatedLaunchError",
"OrchestratorStartError",
"GATEWAY_NETWORK",
]
@@ -1,9 +1,11 @@
"""Host-side egress route-apply for the macos-container backend. """Host-side egress route-apply for the macos-container backend.
The per-bottle companion container this used to signal (`container kill The per-bottle companion container this used to signal (`container kill
--signal HUP <container>`) was removed in the companion-container removal (#385), --signal HUP <container>`) was removed in the companion-container removal
along with the disabled macOS launch path. Fails closed until the macOS (#385). In the consolidated model the shared gateway resolves egress policy
backend grows the consolidated gateway. per-request against the orchestrator rather than reloading a per-bottle routes
file, so the live per-bottle reload is not supported here and fails closed
until the gateway-side apply lands same posture as the docker backend.
""" """
from __future__ import annotations from __future__ import annotations
@@ -16,8 +18,8 @@ class MacOSContainerEgressApplicator(EgressApplicator):
del slug del slug
raise EgressApplyError( raise EgressApplyError(
"live egress route-apply was removed with the per-bottle " "live egress route-apply was removed with the per-bottle "
"companion container (#385); the macos-container backend is " "companion container (#385); route changes will flow through "
"disabled until it uses the consolidated gateway." "the consolidated gateway in a follow-up."
) )
@@ -1,14 +1,53 @@
"""Active-agent enumeration for the macOS Apple Container backend. """Active-agent enumeration for the macOS Apple Container backend."""
The backend is disabled during the companion-container removal (#385) — it can't
launch bottles, so there are none to enumerate. Enumeration returns when
the backend grows the consolidated gateway.
"""
from __future__ import annotations from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent from .. import ActiveAgent
from .infra import INFRA_NAME
# The name every agent container carries: `bot-bottle-<slug>`. Exported
# because callers that act on a running bottle (gateway-host rewrites,
# registry reconciliation) have to map an enumerated slug back to a
# container name.
CONTAINER_NAME_PREFIX = "bot-bottle-"
# The shared per-host infra container carries the same prefix as agent
# containers but is infrastructure, not a bottle — one control plane + gateway
# serves every agent, so listing it as an agent would invent one per host.
_INFRA_NAMES = frozenset({INFRA_NAME})
class EnumerationError(RuntimeError):
"""container list failed; the resulting live set is not authoritative."""
def enumerate_active() -> list[ActiveAgent]: def enumerate_active() -> list[ActiveAgent]:
return [] result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
raise EnumerationError(
f"container list failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
out: list[ActiveAgent] = []
for name in sorted(line.strip() for line in result.stdout.splitlines()):
if not name.startswith(CONTAINER_NAME_PREFIX) or name in _INFRA_NAMES:
continue
slug = name[len(CONTAINER_NAME_PREFIX):]
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="macos-container",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=(),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
@@ -0,0 +1,46 @@
"""Shared network/image constants for the macOS consolidated infra container.
The gateway data plane no longer runs as its own Apple container it shares a
single per-host **infra container** with the control plane (see `infra`),
because two Apple-Container guests writing one `bot-bottle.db` over virtiofs
would race incoherent `fcntl` locks. This module holds the pieces both the
infra service and the launch/provision glue need: the network names, the
gateway image, and the network-creation helper.
"""
from __future__ import annotations
import os
from ...orchestrator.gateway import GatewayError
from . import util as container_mod
# The shared host-only network the infra container and every agent bottle sit
# on. The agent's address here is the attribution key. Distinct from the docker
# names so both backends can coexist on one host.
GATEWAY_NETWORK = "bot-bottle-mac-gateway"
# The NAT network that gives the infra container (and only it) a route out.
GATEWAY_EGRESS_NETWORK = "bot-bottle-mac-egress"
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_GATEWAY_IMAGE", "bot-bottle-gateway:latest")
DEFAULT_CA_TIMEOUT_SECONDS = 30.0
def ensure_networks(
network: str = GATEWAY_NETWORK, egress_network: str = GATEWAY_EGRESS_NETWORK,
) -> None:
"""Create the shared host-only network + the NAT egress network. Idempotent
`create_network` tolerates 'already exists'."""
container_mod.create_network(egress_network)
container_mod.create_network(network, internal=True)
__all__ = [
"GATEWAY_NETWORK",
"GATEWAY_EGRESS_NETWORK",
"GATEWAY_IMAGE",
"GatewayError",
"DEFAULT_CA_TIMEOUT_SECONDS",
"ensure_networks",
]
@@ -0,0 +1,96 @@
"""Stable gateway name for macOS agents, via each bottle's `/etc/hosts`.
The shared gateway's address is assigned by vmnet's DHCP and changes whenever
the infra container is recreated a source-hash bump, an image upgrade, a
crash. Every agent-facing URL (egress proxy, git-http, supervise) embeds that
address, and the proxy URL reaches the agent as **process environment** at
`container exec` time. A running process's `environ` cannot be rewritten from
outside, so a moved gateway used to strand every running bottle permanently:
not degraded, unreachable, until the bottle was relaunched and its session
thrown away.
So the agent never learns the address. It is given a stable *name*
(`GATEWAY_HOSTNAME`) in every URL, resolved through its own `/etc/hosts`.
Unlike `environ`, that is a file it can be rewritten inside a container that
is already running, so a gateway that comes back at a new address is picked up
by live bottles instead of orphaning them.
Apple Container 1.0 offers no container-name DNS on a user network (the only
nameserver an agent sees is vmnet's, which does not know container names) and
`container run` has no `--add-host`, so the entry is written by exec after the
container starts.
Writing it needs root, and the agent runs as `node`: the agent therefore
cannot repoint its own gateway name, while the host (which drives `container
exec --user root`) can. That asymmetry is deliberate keep it.
"""
from __future__ import annotations
from ...log import warn
from . import util as container_mod
from .enumerate import CONTAINER_NAME_PREFIX, enumerate_active
# The name every agent-facing gateway URL uses. Must not collide with a real
# DNS name the agent might resolve; it is bottle-local by construction.
GATEWAY_HOSTNAME = "bot-bottle-gateway"
# Marker so the rewrite is idempotent and only ever touches our own line —
# the rest of /etc/hosts (localhost, the container's own name) is preserved.
_MARKER = "# bot-bottle gateway"
def _rewrite_script(gateway_ip: str) -> str:
"""A shell one-liner that replaces our managed line in `/etc/hosts`.
Rewrites in place via a temp file + `cat` rather than `mv`, so the file
keeps its original inode, ownership, and mode a bind-mounted or
pre-created `/etc/hosts` must not be replaced by a root-owned 0644 copy
that the runtime then refuses to update.
"""
return (
"set -e; "
f"grep -v '{_MARKER}' /etc/hosts > /tmp/.bb-hosts || true; "
f"printf '%s %s %s\\n' '{gateway_ip}' '{GATEWAY_HOSTNAME}' "
f"'{_MARKER}' >> /tmp/.bb-hosts; "
"cat /tmp/.bb-hosts > /etc/hosts; "
"rm -f /tmp/.bb-hosts"
)
def set_gateway_host(container_name: str, gateway_ip: str) -> None:
"""Point `GATEWAY_HOSTNAME` at `gateway_ip` inside one running container.
Must run before the agent is exec'd: the agent's proxy URL names the
gateway, so the entry has to exist for its first connection. Idempotent
re-running with the same address is a no-op in effect.
"""
container_mod.exec_container_as_root(
container_name, ["sh", "-c", _rewrite_script(gateway_ip)],
)
def refresh_gateway_host(gateway_ip: str) -> list[str]:
"""Re-point every running bottle at the current gateway address.
Called once the shared gateway is known to be up, so a bottle stranded by
an earlier gateway restart re-attaches instead of needing a relaunch.
Returns the containers updated.
Best-effort per bottle: one container that refuses the write (already
exiting, say) must not stop the others from being repaired, and must not
fail the launch that triggered the sweep.
"""
updated: list[str] = []
for agent in enumerate_active():
name = f"{CONTAINER_NAME_PREFIX}{agent.slug}"
try:
set_gateway_host(name, gateway_ip)
updated.append(name)
# One bad bottle must not stop the sweep, so this is deliberately broad.
except Exception as e: # noqa: BLE001 # pylint: disable=broad-exception-caught
warn(f"could not re-point {name} at the gateway: {e}")
return updated
__all__ = ["GATEWAY_HOSTNAME", "set_gateway_host", "refresh_gateway_host"]
@@ -0,0 +1,44 @@
"""`GatewayTransport` for the Apple infra container (PRD 0070).
The provisioning *logic* (per-bottle creds dirs, namespaced repo init) is
backend-neutral and lives in `backend.docker.gateway_provision`; this is only
the transport how files and commands reach the running gateway. Docker uses
`docker exec`/`docker cp` and Firecracker uses SSH; Apple uses the `container`
CLI's equivalents against the infra container that hosts the gateway daemons.
"""
from __future__ import annotations
from ..docker.gateway_provision import GatewayProvisionError
from . import util as container_mod
from .infra import INFRA_NAME
class AppleGatewayTransport:
"""`GatewayTransport` for the gateway daemons in the Apple infra container."""
def __init__(self, gateway: str = INFRA_NAME) -> None:
self.gateway = gateway
def exec(self, argv: list[str]) -> None:
result = container_mod.run_container_argv(
["container", "exec", self.gateway, *argv]
)
if result.returncode != 0:
raise GatewayProvisionError(
f"gateway exec {argv!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def cp_into(self, src: str, dest: str) -> None:
result = container_mod.run_container_argv(
["container", "cp", src, f"{self.gateway}:{dest}"]
)
if result.returncode != 0:
raise GatewayProvisionError(
f"gateway cp {src} -> {dest} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
__all__ = ["AppleGatewayTransport", "GatewayProvisionError"]
+313
View File
@@ -0,0 +1,313 @@
"""The per-host infra container for the macOS backend (PRD 0070).
A single persistent Apple container that runs BOTH the orchestrator control
plane and the gateway data plane the macOS analogue of the Firecracker infra
VM (`backend/firecracker/infra_vm.py`), not the docker backend's two separate
containers.
Why one container, not two: Apple Containers are lightweight VMs, each with its
own kernel. The docker backend runs the orchestrator and gateway as two
containers safely because they share the host kernel, so their concurrent
writes to the one `bot-bottle.db` (the orchestrator's registry + the gateway
supervise daemon's queue) are serialized by coherent `fcntl` locks. Across two
*guest* kernels sharing a virtiofs-mounted DB those locks are not coherent, and
concurrent writers can corrupt the file. Firecracker solved this by putting
both services in one guest with the DB on a device only that guest mounts; this
does the same with Apple primitives.
Two consequences fall out of the single container, both simplifications:
- **No DNS dance.** The control plane and the gateway daemons reach each other
over `127.0.0.1`, so nothing depends on Apple's (absent) container DNS and
there is no orchestrator-before-gateway ordering to get right.
- **The DB is never host-shared.** It lives on a container-only volume, so no
host process opens the live file. The host CLI reaches registry + supervise
state through the control-plane HTTP surface (`cli/supervise.py` already uses
`OrchestratorClient`), exactly as it does for firecracker.
The control-plane source is bind-mounted (like the docker orchestrator), so a
code change takes effect on the next launch without an image rebuild; the
gateway daemons are baked in the gateway image and rebuild through its own
digest check.
"""
from __future__ import annotations
import os
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
from pathlib import Path
from ... import log
from ...orchestrator.gateway import GATEWAY_CA_CERT, MITMPROXY_HOME
from ...orchestrator.lifecycle import (
DEFAULT_PORT,
DEFAULT_STARTUP_TIMEOUT_SECONDS,
OrchestratorStartError,
source_hash,
)
from ...paths import (
CONTROL_PLANE_TOKEN_ENV,
HOST_DB_FILENAME,
host_control_plane_token,
host_gateway_ca_dir,
)
from .. import util as backend_util
from . import util as container_mod
from .gateway import (
DEFAULT_CA_TIMEOUT_SECONDS,
GATEWAY_EGRESS_NETWORK,
GATEWAY_IMAGE,
GATEWAY_NETWORK,
GatewayError,
ensure_networks,
)
# The one per-host infra container: control plane + gateway data plane.
INFRA_NAME = "bot-bottle-mac-infra"
INFRA_LABEL = "bot-bottle-mac-infra=1"
# Container-only volume holding bot-bottle.db. No host bind-mount, so the DB is
# written by exactly one kernel (this container's). Survives recreation.
INFRA_DB_VOLUME = "bot-bottle-mac-db"
# BOT_BOTTLE_ROOT inside the container; host_db_path() resolves the DB to
# <root>/db/<filename> and the supervise daemon writes the same file.
_DB_ROOT_IN_CONTAINER = "/var/lib/bot-bottle"
_DB_PATH_IN_CONTAINER = f"{_DB_ROOT_IN_CONTAINER}/db/{HOST_DB_FILENAME}"
_SRC_IN_CONTAINER = "/bot-bottle-src"
_REPO_ROOT = Path(__file__).resolve().parents[3]
_HEALTH_POLL_SECONDS = 0.25
_HEALTH_REQUEST_TIMEOUT_SECONDS = 1.0
_CA_POLL_SECONDS = 0.5
# The gateway subset the consolidated model runs (no per-bottle git:// daemon).
_GATEWAY_DAEMONS = "egress,git-http,supervise"
def _init_script(port: int) -> str:
"""PID-1 init: start the control plane and the gateway daemons, both in
this container, reaching each other over loopback. Backgrounded so `wait`
reaps as PID 1. No `set -e` a transient daemon failure must not kill the
whole container (gateway_init applies the same 'stay up' policy)."""
return (
"export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\n"
f"mkdir -p $(dirname {_DB_PATH_IN_CONTAINER})\n"
# Control plane, from the bind-mounted source (stdlib-only package).
f"( cd {_SRC_IN_CONTAINER} && BOT_BOTTLE_ROOT={_DB_ROOT_IN_CONTAINER} "
f"python3 -m bot_bottle.orchestrator --host 0.0.0.0 --port {port} "
"--broker stub ) &\n"
# Gateway data plane, multi-tenant against the local control plane.
f"( cd /app && BOT_BOTTLE_GATEWAY_DAEMONS={_GATEWAY_DAEMONS} "
f"BOT_BOTTLE_ORCHESTRATOR_URL=http://127.0.0.1:{port} "
f"SUPERVISE_DB_PATH={_DB_PATH_IN_CONTAINER} python3 -m bot_bottle.gateway_init ) &\n"
"while : ; do wait ; done\n"
)
@dataclass(frozen=True)
class InfraEndpoint:
"""How to reach the running infra container. The control plane and the
gateway are the same container, so one address serves both."""
control_plane_url: str # http://<infra ip>:8099 — host CLI + registration
gateway_ip: str # same container; agents' proxy / git-http / MCP target
class MacosInfraService:
"""Manages the single per-host infra container. Callers use
`ensure_running()` (returns the endpoint) and `ca_cert_pem()`."""
def __init__(
self,
*,
port: int = DEFAULT_PORT,
network: str = GATEWAY_NETWORK,
egress_network: str = GATEWAY_EGRESS_NETWORK,
image: str = GATEWAY_IMAGE,
repo_root: Path = _REPO_ROOT,
name: str = INFRA_NAME,
db_volume: str = INFRA_DB_VOLUME,
) -> None:
self.port = port
self.network = network
self.egress_network = egress_network
self.image = image
self._repo_root = repo_root
self._name = name
self._db_volume = db_volume
def _resolve_url(self) -> str:
"""The control-plane URL, or "" while the container has no address."""
ip = container_mod.try_container_ipv4_on_network(self._name, self.network)
return f"http://{ip}:{self.port}" if ip else ""
def is_healthy(
self, url: str, *, timeout: float = _HEALTH_REQUEST_TIMEOUT_SECONDS,
) -> bool:
if not url:
return False
try:
with urllib.request.urlopen(f"{url}/health", timeout=timeout) as resp:
return resp.status == 200
except (urllib.error.URLError, TimeoutError, OSError):
return False
def _source_current(self, current_hash: str) -> bool:
"""True iff the running infra container was created from the current
bind-mounted control-plane source. The control-plane process loads that
code at startup and won't reload it, so a stale container keeps serving
OLD code."""
if not container_mod.container_is_running(self._name):
return False
env = container_mod.container_env(self._name)
if not env:
return True # can't compare → don't churn a working container
return env.get("BOT_BOTTLE_SOURCE_HASH") == current_hash
def _running_healthy_endpoint(self, current_hash: str) -> InfraEndpoint | None:
"""The endpoint if the running container is BOTH source-current and
answering /health, else None ( recreate). Health, not just the source
label, is what lets a wedged-but-current container self-heal instead of
being polled to death forever."""
if not self._source_current(current_hash):
return None
url = self._resolve_url()
if url and self.is_healthy(url):
return InfraEndpoint(control_plane_url=url, gateway_ip=_ip_of(url))
return None
def ensure_built(self) -> None:
"""Ensure the gateway data-plane image exists. The control-plane source
is bind-mounted, not baked, so only the gateway image needs building."""
container_mod.build_image(
self.image, str(self._repo_root), dockerfile="Dockerfile.gateway",
)
def ensure_running(
self, *, startup_timeout: float = DEFAULT_STARTUP_TIMEOUT_SECONDS,
) -> InfraEndpoint:
"""Ensure the single infra container is up; return how to reach it.
Idempotent per-host singleton a healthy container on current source
is left untouched, so N launches share the one control plane + gateway.
Raises `OrchestratorStartError` on startup timeout."""
current_hash = source_hash(self._repo_root)
endpoint = self._running_healthy_endpoint(current_hash)
if endpoint is not None:
return endpoint
self.ensure_built()
log.info("starting infra container", context={"name": self._name})
self._run_container(current_hash)
return self._wait_healthy(startup_timeout)
def _run_container(self, current_hash: str) -> None:
ensure_networks(self.network, self.egress_network)
container_mod.force_remove_container(self._name)
argv = [
"container", "run", "--detach",
"--name", self._name,
"--label", "bot-bottle.backend=macos-container",
"--label", INFRA_LABEL,
# NAT network FIRST so the gateway's egress has a default route;
# the host-only network is where agents (and the host CLI) reach it.
"--network", self.egress_network,
"--network", self.network,
"--dns", container_mod.dns_server(),
# Container-only DB volume: one kernel writes bot-bottle.db, never
# shared with the host or another guest.
"--volume", f"{self._db_volume}:{_DB_ROOT_IN_CONTAINER}",
# The DB needs a container-only ext4 volume for coherent SQLite
# locking, but the CA has no such constraint. Keep it in the host
# app-data root so infra-container recreation and Apple Container
# volume pruning cannot silently rotate every bottle's trust
# anchor (issue #450).
"--mount",
container_mod.bind_mount_spec(
str(host_gateway_ca_dir()), MITMPROXY_HOME),
# Bind-mount the control-plane source (read-only); a code change
# takes effect on relaunch with no image rebuild.
"--mount",
container_mod.bind_mount_spec(
str(self._repo_root), _SRC_IN_CONTAINER, readonly=True),
# Baked onto the container so `_source_current` can detect a real
# control-plane code change and recreate.
"--env", f"BOT_BOTTLE_SOURCE_HASH={current_hash}",
# The control-plane secret, for BOTH the control plane (to require
# it) and the gateway's PolicyResolver (to present it) — they share
# this one container. Bare `--env NAME` inherits the value from the
# run process below, so the secret never lands on argv or in
# `container inspect`'s command line. The agent runs in a SEPARATE
# container that is never given this var, which is the whole point.
"--env", CONTROL_PLANE_TOKEN_ENV,
"--entrypoint", "sh",
self.image,
"-c", _init_script(self.port),
]
run_env = {**os.environ, CONTROL_PLANE_TOKEN_ENV: host_control_plane_token()}
result = container_mod.run_container_argv(argv, env=run_env)
if result.returncode != 0:
raise OrchestratorStartError(
f"infra container failed to start: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _wait_healthy(self, startup_timeout: float) -> InfraEndpoint:
deadline = time.monotonic() + startup_timeout
while True:
url = self._resolve_url()
if url and self.is_healthy(url):
log.info("infra container healthy", context={"url": url})
return InfraEndpoint(control_plane_url=url, gateway_ip=_ip_of(url))
if time.monotonic() >= deadline:
raise OrchestratorStartError(
f"infra container did not become healthy within "
f"{startup_timeout:g}s"
)
time.sleep(_HEALTH_POLL_SECONDS)
def ca_cert_pem(self, *, timeout: float = DEFAULT_CA_TIMEOUT_SECONDS) -> str:
"""The gateway's mitmproxy CA (PEM) agents install to trust its TLS
interception. Read through the container path backed by the persistent
host CA directory; polls because mitmproxy writes it a beat after
start."""
def _fetch() -> str | None:
result = container_mod.run_container_argv(
["container", "exec", self._name, "cat", GATEWAY_CA_CERT])
return result.stdout if result.returncode == 0 and result.stdout.strip() else None
try:
return backend_util.poll_ca_cert(_fetch, timeout=timeout)
except TimeoutError as exc:
raise GatewayError(
f"gateway CA not available in {self._name} after {timeout:g}s"
) from exc
def stop(self) -> None:
"""Remove the infra container (idempotent). The DB volume persists."""
container_mod.force_remove_container(self._name)
def _ip_of(url: str) -> str:
"""The host from an http://host:port URL."""
return url.split("://", 1)[-1].rsplit(":", 1)[0]
def probe_control_plane_url(port: int = DEFAULT_PORT) -> str:
"""The running infra container's control-plane URL, or "" if it isn't up.
Used by host-side control-plane discovery (`discover_orchestrator_url`);
safe to call on any host returns "" when the container or the `container`
CLI isn't present."""
ip = container_mod.try_container_ipv4_on_network(INFRA_NAME, GATEWAY_NETWORK)
return f"http://{ip}:{port}" if ip else ""
__all__ = [
"MacosInfraService",
"InfraEndpoint",
"OrchestratorStartError",
"GatewayError",
"INFRA_NAME",
"INFRA_DB_VOLUME",
]
+400 -20
View File
@@ -1,39 +1,419 @@
"""Launch flow for the macOS Apple Container backend — disabled (#385). """Launch flow for the macOS Apple Container backend (PRD 0070).
This backend launched a per-bottle companion container (the egress / The agent container attaches to the **shared host-only gateway network** and
git-gate / supervise data plane) alongside the agent container, with the proxies egress through the one per-host gateway, replacing the per-bottle
agent's proxy env pointed at the companion's host-only IP. That companion container removed in #385.
per-bottle-companion architecture was removed in the companion-container removal;
the macOS backend will be re-enabled once it grows the consolidated
per-host gateway the docker backend already uses.
Until then, launching a macOS bottle fails closed. `prepare` / `status` The order differs from docker's, forced by Apple Container 1.0.0 having no
/ cleanup still work. `--ip` (see `consolidated_launch`): the agent is started *before* it is
registered, because its DHCP-assigned address the attribution key does not
exist until then.
gateway up -> run agent -> read its IP -> register it -> provision
Two things follow from that inversion:
- The **identity token** is minted by registration and so cannot be in the
agent's run-time env; it rides the proxy URL applied at `container exec`
time (`bottle.MacosContainerBottle`). `/resolve` requires it (#366), so
egress without it is denied hence the bare `sleep` init: every real agent
command goes through exec and therefore carries the token.
- The agent is run with `--cap-drop CAP_NET_RAW`. Apple Container grants
NET_RAW by default, which would let an agent open a raw socket and forge a
neighbour's source address on the shared segment. NET_ADMIN is already
absent (the agent cannot change its own address or route), so dropping
NET_RAW is what closes the source-address half of PRD 0070's invariant:
"a packet's source address, as seen by the orchestrator, provably identifies
the originating bottle." The identity token is the other half — an attacker
would need to forge the address *and* steal the token but the invariant is
a stated precondition of consolidation, so it is enforced on its own terms
rather than left to the token.
""" """
from __future__ import annotations from __future__ import annotations
from contextlib import contextmanager import dataclasses
import os
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator from typing import Callable, Generator
from ...log import die from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
egress_agent_env_entries,
egress_resolve_token_values,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...git_http_backend import DEFAULT_PORT as _GIT_HTTP_PORT
from ...image_cache import check_stale
from ...log import die, info, warn
from .. import BottleImages
from ...supervise import SUPERVISE_PORT
from ..docker.egress import EGRESS_PORT
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import util as container_mod
from .bottle import MacosContainerBottle from .bottle import MacosContainerBottle
from .gateway_hosts import (
GATEWAY_HOSTNAME,
refresh_gateway_host,
set_gateway_host,
)
from .bottle_plan import MacosContainerBottlePlan from .bottle_plan import MacosContainerBottlePlan
from ...orchestrator.config_store import resolve_teardown_timeout
from .consolidated_launch import (
GatewayEndpoint,
ensure_gateway,
register_agent,
teardown_consolidated,
)
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_AGENT_SLEEP_SECONDS = "2147483647"
def build_or_load_images(plan: MacosContainerBottlePlan) -> BottleImages:
"""Resolve the agent image ref for this plan. The gateway's own image is
built by `ensure_gateway` it belongs to the shared singleton."""
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return BottleImages(agent=committed)
if plan.spec.image_policy == "cached":
if not container_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
return BottleImages(agent=plan.image)
container_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return BottleImages(agent=plan.image)
@contextmanager @contextmanager
def launch( def launch(
plan: MacosContainerBottlePlan, plan: MacosContainerBottlePlan,
images: BottleImages,
*, *,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None], provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]: ) -> Generator[MacosContainerBottle, None, None]:
"""Fail closed: the macOS backend is disabled until it grows the """Run, register, provision, and yield an Apple Container bottle on the
consolidated per-host gateway (the companion-container path it used shared per-host gateway."""
was removed in #385).""" stack = ExitStack()
del plan, provision bottle_for_revoke = plan.manifest.bottle
die( git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
"the macos-container backend is temporarily disabled during the "
"companion-container removal (#385); it will return once it uses " plan = dataclasses.replace(
"the consolidated gateway. Use --backend=docker for now." plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=str(images.agent)),
) )
yield # unreachable — `die` raises; keeps this a generator/contextmanager
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"macos-container teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
# Step 1: the per-host singletons. Must precede the agent run — its
# proxy env needs the gateway's address at `container run` time.
endpoint = ensure_gateway()
# The gateway's address may have changed since these bottles launched
# (any infra recreate re-runs DHCP). They name the gateway rather than
# address it, so re-pointing /etc/hosts re-attaches them in place
# instead of leaving them stranded until relaunch.
refresh_gateway_host(endpoint.gateway_ip)
# Step 2: mint this bottle's deploy keys, then point it at the SHARED
# gateway's CA + git-http/supervise ports.
plan = _provision_git_gate_keys(plan)
plan = _install_gateway_ca(plan, endpoint)
plan = _stamp_agent_urls(plan, endpoint)
# Step 3: run the agent. It has no identity token yet — registration
# needs the address this run assigns.
container_mod.force_remove_container(plan.container_name)
_start_agent(plan, endpoint)
stack.callback(container_mod.force_remove_container, plan.container_name)
# Step 4: read the assigned address and register by it. This is the
# attribution key; `--cap-drop CAP_NET_RAW` at run is what makes it
# unforgeable. Poll: `container run --detach` can return before vmnet's
# DHCP has assigned the address.
# Resolve the gateway name before anything execs: every agent-facing
# URL uses it, so the entry must exist for the first connection.
set_gateway_host(plan.container_name, endpoint.gateway_ip)
source_ip = container_mod.wait_container_ipv4_on_network(
plan.container_name, endpoint.network,
)
if not source_ip:
die(
f"agent {plan.container_name} never got an address on "
f"{endpoint.network}"
)
effective_env = {**os.environ, **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
teardown_timeout = resolve_teardown_timeout()
ctx = register_agent(
plan.egress_plan,
plan.git_gate_plan,
source_ip=source_ip,
endpoint=endpoint,
image_ref=plan.image,
tokens=token_values,
)
stack.callback(
teardown_consolidated, ctx.bottle_id,
orchestrator_url=ctx.orchestrator_url,
timeout=teardown_timeout,
)
info(
f"agent {plan.container_name} registered "
f"(gateway {endpoint.gateway_ip}, ip {source_ip})"
)
# Stamp the token onto the plan so provision-time consumers can read it,
# not only the exec-time egress proxy. git-gate's gitconfig extraHeader
# and the supervise MCP --header both reach the gateway on NO_PROXY (they
# bypass the egress proxy that carries the token), so without this the
# gateway's /resolve fail-closes and every git fetch/push and supervise
# call from the bottle is denied. Registration already produced the
# token above, so — unlike the run-time env — the plan CAN carry it.
plan = dataclasses.replace(plan, identity_token=ctx.identity_token)
bottle = MacosContainerBottle(
plan.container_name,
teardown,
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=(
f"{plan.spec.label} ({plan.spec.agent_name})"
if plan.spec.label else plan.spec.agent_name
),
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
exec_env=_identity_proxy_env(endpoint, ctx.identity_token),
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def stale_checks(plan: MacosContainerBottlePlan) -> None:
"""Raise StaleImageError if a cached image is older than the configured
threshold. Only runs when image_policy is 'cached'. Called by the backend
class's _image_stale_checks before _launch_impl starts any resources."""
if plan.spec.image_policy != "cached":
return
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
ts = container_mod.image_created_at(committed)
if ts is not None:
check_stale(f"agent image {committed!r}", ts)
return
if container_mod.image_exists(plan.image):
ts = container_mod.image_created_at(plan.image)
if ts is not None:
check_stale(f"agent image {plan.image!r}", ts)
def _provision_git_gate_keys(
plan: MacosContainerBottlePlan,
) -> MacosContainerBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _install_gateway_ca(
plan: MacosContainerBottlePlan, endpoint: GatewayEndpoint,
) -> MacosContainerBottlePlan:
"""Stage the SHARED gateway's CA for the provisioner to install, replacing
the per-bottle CA the companion container used to mint. Every bottle on
this host trusts this one CA."""
ca_dir = egress_state_dir(plan.slug) / "gateway-ca"
ca_dir.mkdir(parents=True, exist_ok=True)
ca_file = ca_dir / "gateway-ca.pem"
ca_file.write_text(endpoint.gateway_ca_pem)
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=ca_file,
mitmproxy_ca_cert_only_host_path=ca_file,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _stamp_agent_urls(
plan: MacosContainerBottlePlan, endpoint: GatewayEndpoint,
) -> MacosContainerBottlePlan:
"""Point the agent's git-gate insteadOf rewrites + supervise MCP at the
shared gateway's ports. Both bypass the egress proxy (NO_PROXY covers the
gateway name).
Addressed by `GATEWAY_HOSTNAME`, never by IP: these URLs are baked into
the agent's gitconfig and MCP config at provision time, so an address here
would strand the bottle the moment the gateway moved. The name is resolved
per connection through `/etc/hosts`, which stays rewritable while the
bottle runs."""
del endpoint # addressed by name; the address reaches the bottle via /etc/hosts
git_gate_url = (
f"http://{GATEWAY_HOSTNAME}:{_GIT_HTTP_PORT}"
if plan.git_gate_plan.upstreams else ""
)
supervise_url = (
f"http://{GATEWAY_HOSTNAME}:{SUPERVISE_PORT}/"
if plan.supervise_plan is not None else ""
)
return dataclasses.replace(
plan,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
def _proxy_url(identity_token: str = "") -> str:
"""The agent's egress proxy URL. The identity token rides as proxy
credentials the gateway reads Proxy-Authorization, resolves the
(source_ip, token) pair against the control plane, and strips it before
upstream. Without a valid pair `/resolve` denies the request (#366).
Names the gateway rather than addressing it: this URL reaches the agent as
process environment, which cannot be rewritten once the agent is running,
so an address baked here is unfixable if the gateway moves."""
cred = f"bottle:{identity_token}@" if identity_token else ""
return f"http://{cred}{GATEWAY_HOSTNAME}:{EGRESS_PORT}"
def _no_proxy() -> str:
# git-http + supervise live on the gateway and must NOT go through the
# egress proxy — the agent reaches them directly by name. Deliberately
# address-free: NO_PROXY is baked into the run-time env and is therefore
# just as unfixable as the proxy URL if the gateway moves.
return f"localhost,127.0.0.1,{GATEWAY_HOSTNAME}"
def _identity_proxy_env(
endpoint: GatewayEndpoint, identity_token: str,
) -> dict[str, str]:
"""The token-bearing proxy env applied at `container exec` — the only way
to get the token in, since it does not exist until after the container
runs (registration keys on the DHCP-assigned address).
This is the *sole* source of `*_PROXY` for the agent. It deliberately does
not rely on overriding a run-time value: `container exec --env` appends
rather than replaces, so a run-time `HTTPS_PROXY` would survive alongside
this one and first-wins runtimes would read the wrong entry. See
`_agent_env_entries`."""
if not identity_token:
return {}
del endpoint # the gateway is named, not addressed
url = _proxy_url(identity_token)
return {
"HTTPS_PROXY": url, "HTTP_PROXY": url,
"https_proxy": url, "http_proxy": url,
}
def _start_agent(plan: MacosContainerBottlePlan, endpoint: GatewayEndpoint) -> None:
argv = _agent_run_argv(plan, endpoint)
env = {**os.environ, **plan.forwarded_env}
info(f"container run agent {plan.container_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for agent {plan.container_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _agent_run_argv(
plan: MacosContainerBottlePlan, endpoint: GatewayEndpoint,
) -> list[str]:
argv = [
"container", "run",
"--name", plan.container_name,
"--detach",
"--label", "bot-bottle.backend=macos-container",
"--network", endpoint.network,
# The attribution invariant: without NET_RAW the agent cannot open a
# raw socket, so it cannot source-IP-spoof its neighbours on the shared
# segment. NET_ADMIN is not granted by default, so its address and
# route are already fixed. See the module docstring.
"--cap-drop", "CAP_NET_RAW",
]
for entry in _agent_env_entries(plan, endpoint):
argv += ["--env", entry]
# The init process is a no-op: every agent command arrives via
# `container exec`, which is also how the identity token gets in.
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
return argv
def _agent_env_entries(
plan: MacosContainerBottlePlan, endpoint: GatewayEndpoint,
) -> tuple[str, ...]:
# No `*_PROXY` here on purpose. The token-bearing URL is applied at
# `container exec` (`_identity_proxy_env`), and Apple's `container exec
# --env` **appends** to the run-time environment rather than replacing it:
# setting a token-less value here leaves two `HTTPS_PROXY` entries in the
# agent's `environ`, token-less first. Which one a runtime reads is then
# pure luck — Node takes the last (and worked), Rust's `std::env::var`
# takes the first, so Codex proxied without its identity token and
# `/resolve` fail-closed on every request.
#
# A token-less proxy URL has no legitimate consumer anyway: the init
# process is `sleep` and everything that egresses arrives via exec. Its
# only value was a tidy 403 for unattributed callers, which is not worth
# silently dropping attribution for. Without it a process that egresses
# before the exec-time env still fails closed — the agent network is
# host-only, so there is no route off it except the gateway.
no_proxy = _no_proxy()
env = [
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
if plan.agent_git_gate_url:
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
if plan.agent_supervise_url:
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
# Forwarded vars: bare name → inherits from the `container run` process env
# so the secret value never lands on argv.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
return tuple(env)
__all__ = ["launch"]
+216 -9
View File
@@ -10,6 +10,7 @@ import shutil
import subprocess import subprocess
import tempfile import tempfile
import time import time
from datetime import datetime, timezone
from typing import Iterable from typing import Iterable
from ...log import die, info from ...log import die, info
@@ -360,6 +361,21 @@ def exec_container(name: str, argv: list[str]) -> None:
) )
def exec_container_as_root(name: str, argv: list[str]) -> None:
"""`exec_container`, but as uid 0 inside the container.
For host-driven maintenance the agent itself must not be able to perform
rewriting `/etc/hosts` to point the gateway name at an address. The agent
runs as `node`, so it cannot repoint its own gateway; the host can.
"""
result = _run_container_op([_CONTAINER, "exec", "--user", "root", name, *argv])
if result.returncode != 0:
die(
f"container exec (root) in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]: def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]:
result = subprocess.run( result = subprocess.run(
cmd, cmd,
@@ -437,22 +453,180 @@ def inspect_container(name: str) -> dict[str, object]:
def container_ipv4_on_network(name: str, network: str) -> str: def container_ipv4_on_network(name: str, network: str) -> str:
data = inspect_container(name) """The container's IPv4 address on `network`. Fatal if absent — callers
that can tolerate "not yet" want `try_container_ipv4_on_network`."""
ip = try_container_ipv4_on_network(name, network)
if not ip:
die(f"container {name} has no IPv4 address on {network}")
return ip
def run_container_argv(
argv: list[str], *, env: dict[str, str] | None = None,
) -> subprocess.CompletedProcess[str]:
"""Run a `container` command, returning the result for the caller to
interpret. Unlike the `die`-on-failure helpers above, this lets callers
that raise their own typed errors (the gateway / orchestrator lifecycle)
keep control of the failure path.
`env` sets the child process environment used to hand a secret to a bare
`--env NAME` flag (Apple's "just key → inherit from host" form) so the
value is inherited from this process, never written onto argv or into
`container inspect`'s recorded command line."""
return subprocess.run(
argv, capture_output=True, text=True, check=False, env=env)
def bind_mount_spec(source: str, target: str, *, readonly: bool = False) -> str:
"""A `container run --mount` bind spec. One definition so the gateway and
orchestrator emit an identical string a divergence here would silently
break one backend's mounts while the other kept working."""
spec = f"type=bind,source={source},target={target}"
if readonly:
spec += ",readonly"
return spec
def _normalize_digest(value: str) -> str:
return value.split(":", 1)[1] if ":" in value else value
def _inspect_first(argv: list[str]) -> dict[str, object]:
"""Run an inspect command and return its first JSON object, or {} on any
failure (non-zero exit, malformed JSON, unexpected shape). {} is the shared
'don't know' signal all the non-fatal inspect readers below build on — a
caller comparing against it treats it as 'leave the working container
alone', never as a mismatch."""
result = run_container_argv(argv)
if result.returncode != 0:
return {}
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError:
return {}
if isinstance(data, list):
data = data[0] if data else {}
return data if isinstance(data, dict) else {}
def _descriptor_digest(node: object) -> str:
"""The normalized digest under a `{... "descriptor": {"digest": ...}}`
node, or "". Both the image and container inspect shapes nest the image's
identity this way, so the digest readers stay symmetric a difference
between them is what would spuriously recreate a container."""
if not isinstance(node, dict):
return ""
descriptor = node.get("descriptor")
if isinstance(descriptor, dict) and descriptor.get("digest"):
return _normalize_digest(str(descriptor["digest"]))
return ""
def image_digest(ref: str) -> str:
"""The digest of image `ref`, or "" if it can't be read. Reads exactly the
field `container_image_digest` reads (`configuration.descriptor.digest`) so
the two are comparable; "" means 'don't know' → callers don't churn."""
data = _inspect_first([_CONTAINER, "image", "inspect", ref])
return _descriptor_digest(data.get("configuration"))
def container_image_digest(name: str) -> str:
"""The digest of the image container `name` was created from, or "" if it
can't be read. Compare with `image_digest(ref)` to tell whether a running
container predates an image rebuild."""
config = _inspect_first([_CONTAINER, "inspect", name]).get("configuration")
image = config.get("image") if isinstance(config, dict) else None
return _descriptor_digest(image)
def container_env(name: str) -> dict[str, str]:
"""The env container `name` was started with, or {} if unreadable. Lets a
caller tell whether a running container's baked-in configuration still
matches what it would pass today."""
config = _inspect_first([_CONTAINER, "inspect", name]).get("configuration")
init = config.get("initProcess") if isinstance(config, dict) else None
entries = init.get("environment") if isinstance(init, dict) else None
if not isinstance(entries, list):
return {}
env: dict[str, str] = {}
for entry in entries:
if isinstance(entry, str) and "=" in entry:
key, value = entry.split("=", 1)
env[key] = value
return env
def try_container_ipv4_on_network(name: str, network: str) -> str:
"""`container_ipv4_on_network` without the fatal exit: "" when the address
isn't readable yet. For pollers — a container is created before it has an
address, so "not yet" is an expected state there, not an error."""
status = _inspect_first([_CONTAINER, "inspect", name]).get("status")
networks = status.get("networks") if isinstance(status, dict) else None
if not isinstance(networks, list):
return ""
for entry in networks:
if not isinstance(entry, dict) or entry.get("network") != network:
continue
raw = entry.get("ipv4Address")
if isinstance(raw, str) and raw:
return raw.split("/", 1)[0]
return ""
def inspect_container_network_ip(name: str, network: str) -> str | None:
"""IP of `name` on `network`, distinguishing inspect failure from "not yet".
Returns:
- the IP string when the container has one on `network`
- "" when inspect succeeds but no address is assigned yet (in-flight DHCP)
- None when the inspect command itself fails (authoritative list impossible)
"""
result = subprocess.run(
[_CONTAINER, "inspect", name],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return None
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError:
return None
if isinstance(data, list):
data = data[0] if data else {}
if not isinstance(data, dict):
return None
status = data.get("status") status = data.get("status")
networks = status.get("networks") if isinstance(status, dict) else None networks = status.get("networks") if isinstance(status, dict) else None
if not isinstance(networks, list): if not isinstance(networks, list):
die(f"container inspect {name} did not include status.networks") return ""
for entry in networks: for entry in networks:
if not isinstance(entry, dict): if not isinstance(entry, dict) or entry.get("network") != network:
continue
if entry.get("network") != network:
continue continue
raw = entry.get("ipv4Address") raw = entry.get("ipv4Address")
if not isinstance(raw, str) or not raw: if isinstance(raw, str) and raw:
die(f"container {name} has no IPv4 address on {network}")
return raw.split("/", 1)[0] return raw.split("/", 1)[0]
die(f"container {name} is not attached to network {network}") return ""
raise AssertionError("unreachable")
def wait_container_ipv4_on_network(
name: str, network: str, *, timeout: float = 15.0, poll: float = 0.25,
) -> str:
"""Poll for the container's DHCP-assigned address on `network`, returning
it once available or "" on timeout.
Apple Container has no `--ip`: `container run --detach` can return before
vmnet's DHCP has populated `status.networks[].ipv4Address`, so a bare read
right after start races the assignment. Callers that need the address (the
attribution key, the gateway's proxy target) poll through here instead of
the fatal `container_ipv4_on_network`."""
deadline = time.monotonic() + timeout
while True:
ip = try_container_ipv4_on_network(name, network)
if ip:
return ip
if time.monotonic() >= deadline:
return ""
time.sleep(poll)
def image_id(ref: str) -> str: def image_id(ref: str) -> str:
@@ -488,6 +662,39 @@ def image_id(ref: str) -> str:
raise AssertionError("unreachable") raise AssertionError("unreachable")
def image_created_at(ref: str) -> datetime | None:
"""Return the image creation timestamp as an aware UTC datetime, or None
when the field is absent or unparseable (e.g. FROM-scratch images, images
pulled from registries that omit the field). Callers should skip the stale
check when None is returned rather than treating it as an error."""
result = subprocess.run(
[_CONTAINER, "image", "inspect", ref],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container image inspect for {ref!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "{}")
except json.JSONDecodeError as exc:
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
if isinstance(data, list) and data:
data = data[0]
if isinstance(data, dict):
value = data.get("created") or data.get("Created")
if isinstance(value, str) and value:
try:
ts = value.rstrip("Z")
return datetime.fromisoformat(ts).replace(tzinfo=timezone.utc)
except ValueError:
pass
return None
def save(ref: str, output: str) -> None: def save(ref: str, output: str) -> None:
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True) subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
+20
View File
@@ -7,6 +7,8 @@ from __future__ import annotations
import hashlib import hashlib
import os import os
import ssl import ssl
import time
from collections.abc import Callable
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
@@ -15,6 +17,24 @@ from ..log import die, info
if TYPE_CHECKING: if TYPE_CHECKING:
from ..egress import EgressPlan from ..egress import EgressPlan
_CA_POLL_INTERVAL = 0.5
def poll_ca_cert(fetch: Callable[[], str | None], *, timeout: float) -> str:
"""Poll `fetch` until it returns a non-empty PEM string or `timeout` expires.
`fetch` should return the PEM on success and `None` (or empty string) when
the cert is not yet available. Raises `TimeoutError` if the cert never
appears within `timeout` seconds."""
deadline = time.monotonic() + timeout
while True:
result = fetch()
if result:
return result
if time.monotonic() >= deadline:
raise TimeoutError(f"CA cert not available after {timeout:g}s")
time.sleep(_CA_POLL_INTERVAL)
# Debian-family CA layout, shared by every backend (all guest images # Debian-family CA layout, shared by every backend (all guest images
# are Debian-family). AGENT_CA_PATH is the source path that # are Debian-family). AGENT_CA_PATH is the source path that
+21
View File
@@ -31,6 +31,7 @@ from __future__ import annotations
import dataclasses import dataclasses
import json import json
import secrets import secrets
import socket
import string import string
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
@@ -43,6 +44,7 @@ from .paths import bot_bottle_root
_STATE_SUBDIR = "state" _STATE_SUBDIR = "state"
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile" _PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
_COMMITTED_IMAGE_NAME = "committed-image" _COMMITTED_IMAGE_NAME = "committed-image"
_COMMITTED_ROOTFS_NAME = "committed-rootfs.tar"
_TRANSCRIPT_SUBDIR = "transcript" _TRANSCRIPT_SUBDIR = "transcript"
# Per-daemon scratch subdirs. PRD 0018 chunk 2: bind-mount sources # Per-daemon scratch subdirs. PRD 0018 chunk 2: bind-mount sources
# live here so chunk 3's `docker compose up` can find them at stable # live here so chunk 3's `docker compose up` can find them at stable
@@ -87,6 +89,14 @@ def bottle_identity(agent_name: str) -> str:
return f"{slug}-{suffix}" return f"{slug}-{suffix}"
def globalize_slug(slug: str) -> str:
"""Return a globally-unique slug qualified with the current hostname.
Assumes slug is a value returned from mint_slug. Use wherever a slug
must be unique across hosts (e.g. deploy-key titles)."""
return f"{socket.gethostname()}-{slug}"
@dataclass(frozen=True) @dataclass(frozen=True)
class BottleMetadata: class BottleMetadata:
"""Persistent record of how a bottle was launched, written at """Persistent record of how a bottle was launched, written at
@@ -191,6 +201,15 @@ def committed_image_path(identity: str) -> Path:
return bottle_state_dir(identity) / _COMMITTED_IMAGE_NAME return bottle_state_dir(identity) / _COMMITTED_IMAGE_NAME
def committed_rootfs_path(identity: str) -> Path:
"""Where the Firecracker freezer stores a snapshot of the bottle's
guest rootfs (a plain tar). This is the resumable/migratable artifact
the Firecracker backend boots from no Docker image involved. The
matching `committed-image` state file records that a snapshot exists
(and its path); `resume` boots from this tar when both are present."""
return bottle_state_dir(identity) / _COMMITTED_ROOTFS_NAME
def write_committed_image(identity: str, image_tag: str) -> Path: def write_committed_image(identity: str, image_tag: str) -> Path:
"""Persist the committed image tag for `identity`. The next """Persist the committed image tag for `identity`. The next
`cli.py resume <identity>` will boot from this image instead of `cli.py resume <identity>` will boot from this image instead of
@@ -340,10 +359,12 @@ __all__ = [
"BottleMetadata", "BottleMetadata",
"agent_state_dir", "agent_state_dir",
"bottle_identity", "bottle_identity",
"globalize_slug",
"bottle_state_dir", "bottle_state_dir",
"cleanup_state", "cleanup_state",
"clear_preserve_marker", "clear_preserve_marker",
"committed_image_path", "committed_image_path",
"committed_rootfs_path",
"egress_state_dir", "egress_state_dir",
"git_gate_state_dir", "git_gate_state_dir",
"is_preserved", "is_preserved",
+11 -5
View File
@@ -19,6 +19,7 @@ from .commit import cmd_commit
from .edit import cmd_edit from .edit import cmd_edit
from .info import cmd_info from .info import cmd_info
from .init import cmd_init from .init import cmd_init
from .login import cmd_login
from .resume import cmd_resume from .resume import cmd_resume
from .start import cmd_start from .start import cmd_start
from .supervise import cmd_supervise from .supervise import cmd_supervise
@@ -33,11 +34,19 @@ COMMANDS = {
"info": cmd_info, "info": cmd_info,
"init": cmd_init, "init": cmd_init,
"list": cmd_list, "list": cmd_list,
"login": cmd_login,
"resume": cmd_resume, "resume": cmd_resume,
"start": cmd_start, "start": cmd_start,
"supervise": cmd_supervise, "supervise": cmd_supervise,
} }
# Commands that manage host prerequisites (or are otherwise store-free) and
# must run before — or without — a migrated DB. `backend` provisions/probes
# the host (TAP pool, /dev/kvm, firecracker) and never opens the store, so
# gating it on the schema breaks preflight on a fresh CI runner where stdin
# isn't a TTY and the migration prompt can't be answered.
NO_MIGRATION_COMMANDS = frozenset({"backend", "login"})
def usage() -> None: def usage() -> None:
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n") sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
@@ -49,6 +58,7 @@ def usage() -> None:
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n") sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n") sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
sys.stderr.write(" list list available agents or active containers\n") sys.stderr.write(" list list available agents or active containers\n")
sys.stderr.write(" login register this host with a bot-bottle console\n")
sys.stderr.write( sys.stderr.write(
" resume re-launch a bottle by its identity " " resume re-launch a bottle by its identity "
"(continues state from PRD 0016)\n" "(continues state from PRD 0016)\n"
@@ -80,7 +90,7 @@ def main(argv: list[str] | None = None) -> int:
usage() usage()
die(f"unknown command: {command}") die(f"unknown command: {command}")
mgr = StoreManager.instance() mgr = StoreManager.instance()
if not mgr.is_migrated(): if command not in NO_MIGRATION_COMMANDS and not mgr.is_migrated():
sys.stderr.write("bot-bottle: database schema is out of date\n") sys.stderr.write("bot-bottle: database schema is out of date\n")
sys.stderr.write("Migrate now? [y/N] ") sys.stderr.write("Migrate now? [y/N] ")
sys.stderr.flush() sys.stderr.flush()
@@ -104,7 +114,3 @@ def main(argv: list[str] | None = None) -> int:
return e.code if isinstance(e.code, int) else 1 return e.code if isinstance(e.code, int) else 1
except KeyboardInterrupt: except KeyboardInterrupt:
return 130 return 130
if __name__ == "__main__":
sys.exit(main())
+15
View File
@@ -0,0 +1,15 @@
"""Entry point for `python -m bot_bottle.cli`.
`cli.py` at the repo root is the usual way in; this makes the package
runnable too, so the CLI works from an installed copy where there is no
`cli.py` on disk to point at.
"""
from __future__ import annotations
import sys
from . import main
if __name__ == "__main__":
sys.exit(main())
+2 -10
View File
@@ -3,18 +3,10 @@
from __future__ import annotations from __future__ import annotations
import os import os
import sys
from pathlib import Path from pathlib import Path
from ..util import read_tty_line as read_tty_line
PROG = "cli.py" PROG = "cli.py"
USER_CWD = os.getcwd() USER_CWD = os.getcwd()
REPO_DIR = str(Path(__file__).resolve().parent.parent.parent) REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
def read_tty_line() -> str:
"""Mirror `IFS= read -r REPLY </dev/tty`. Falls back to stdin."""
try:
with open("/dev/tty", "r", encoding="utf-8") as tty:
return tty.readline().rstrip("\n")
except OSError:
return sys.stdin.readline().rstrip("\n")
+7 -3
View File
@@ -21,16 +21,20 @@ from __future__ import annotations
import sys import sys
from ..backend import get_bottle_backend, known_backend_names from ..backend import get_bottle_backend, has_backend, known_backend_names
from ..log import info from ..log import info
from ._common import read_tty_line from ._common import read_tty_line
def cmd_cleanup(_argv: list[str]) -> int: def cmd_cleanup(_argv: list[str]) -> int:
# Order: stable backend iteration so the y/N output is # Order: stable backend iteration so the y/N output is
# deterministic across runs. # deterministic across runs. Skip backends whose runtime
# isn't available on this host so e.g. macos-container
# doesn't error on Linux.
plans = [ plans = [
(name, get_bottle_backend(name)) for name in known_backend_names() (name, get_bottle_backend(name))
for name in known_backend_names()
if has_backend(name)
] ]
prepared = [(name, b, b.prepare_cleanup()) for name, b in plans] prepared = [(name, b, b.prepare_cleanup()) for name, b in plans]
+168
View File
@@ -0,0 +1,168 @@
"""bb login — register this host with a bot-bottle console.
Opens a device-authorization flow against the target console, waits for the
operator to approve, then writes access and refresh tokens to
~/.bot-bottle/console.json (or $BOT_BOTTLE_ROOT/console.json).
Usage:
bb login [--console-url URL] [--label LABEL]
Flags:
--console-url URL Target console URL (overrides BB_CONSOLE_URL env var)
--label LABEL Host label shown in the console (default: hostname)
"""
from __future__ import annotations
import json
import os
import socket
import sys
import tempfile
import time
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any
from ..paths import bot_bottle_root
_CONSOLE_URL_ENV = "BB_CONSOLE_URL"
_POLL_SLEEP = 2 # seconds between polls; matches console's poll_interval default
def _usage() -> None:
sys.stderr.write(
"usage: bb login [--console-url URL] [--label LABEL]\n"
"\n"
"Options:\n"
" --console-url URL Console base URL (or BB_CONSOLE_URL env var)\n"
" --label LABEL Host label shown in the console (default: hostname)\n"
)
def _flag(argv: list[str], name: str) -> str | None:
for i, arg in enumerate(argv):
if arg == name and i + 1 < len(argv):
return argv[i + 1]
if arg.startswith(f"{name}="):
return arg[len(name) + 1:]
return None
def _post(url: str, payload: dict[str, Any]) -> dict[str, Any]:
data = json.dumps(payload).encode()
req = urllib.request.Request(
url, data=data, headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read())
def _get(url: str) -> tuple[int, dict[str, Any]]:
req = urllib.request.Request(url)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
return resp.status, json.loads(resp.read())
except urllib.error.HTTPError as e:
return e.code, {}
def _save_credentials(
console_url: str, host_id: str, access_token: str, refresh_token: str
) -> Path:
path = bot_bottle_root() / "console.json"
path.parent.mkdir(parents=True, exist_ok=True)
content = (
json.dumps(
{
"url": console_url,
"host_id": host_id,
"access_token": access_token,
"refresh_token": refresh_token,
},
indent=2,
)
+ "\n"
)
fd, tmp_path_str = tempfile.mkstemp(dir=path.parent, prefix=".console-")
tmp = Path(tmp_path_str)
try:
tmp.chmod(0o600)
with os.fdopen(fd, "w") as f:
f.write(content)
os.replace(tmp, path)
except OSError:
try:
tmp.unlink()
except OSError:
pass
raise
return path
def cmd_login(argv: list[str]) -> int:
if "--help" in argv or "-h" in argv:
_usage()
return 0
console_url = _flag(argv, "--console-url") or os.environ.get(_CONSOLE_URL_ENV)
if not console_url:
sys.stderr.write(
"bb login: --console-url or BB_CONSOLE_URL is required\n"
)
return 1
console_url = console_url.rstrip("/")
label = _flag(argv, "--label") or socket.gethostname()
try:
resp = _post(f"{console_url}/api/v1/hosts/authorize", {"label": label})
except (OSError, ValueError) as exc:
sys.stderr.write(f"bb login: failed to start authorization: {exc}\n")
return 1
device_code = resp["device_code"]
user_code = resp["user_code"]
expires_in = resp.get("expires_in", 300)
poll_sleep = max(1, min(int(resp.get("poll_interval", _POLL_SLEEP)), 60))
sys.stderr.write(
f"\nOpen this URL in your browser to authorize this host:\n\n"
f" {console_url}/hosts/authorize?code={user_code}\n\n"
f"Waiting for approval"
)
deadline = time.monotonic() + expires_in
while time.monotonic() < deadline:
sys.stderr.write(".")
sys.stderr.flush()
time.sleep(poll_sleep)
try:
code, result = _get(
f"{console_url}/api/v1/hosts/authorize/{device_code}"
)
except (OSError, ValueError):
continue
if code == 410:
break
st = result.get("status")
if st == "approved":
sys.stderr.write("\n\nApproved.\n")
path = _save_credentials(
console_url,
result["host_id"],
result["access_token"],
result["refresh_token"],
)
sys.stderr.write(f"Credentials saved to {path}\n")
return 0
if st == "denied":
sys.stderr.write("\n\nDenied by operator.\n")
return 1
sys.stderr.write("\n\nAuthorization timed out.\n")
return 1
+40 -22
View File
@@ -27,7 +27,6 @@ from ..backend import (
BottleSpec, BottleSpec,
enumerate_active_agents, enumerate_active_agents,
get_bottle_backend, get_bottle_backend,
known_backend_names,
) )
from ..backend.docker import util as docker_mod from ..backend.docker import util as docker_mod
from ..backend.docker.bottle_plan import DockerBottlePlan from ..backend.docker.bottle_plan import DockerBottlePlan
@@ -36,6 +35,7 @@ from ..bottle_state import (
is_preserved, is_preserved,
mark_preserved, mark_preserved,
) )
from ..image_cache import StaleImageError
from ..log import info, die from ..log import info, die
from ..manifest import Manifest, ManifestIndex from ..manifest import Manifest, ManifestIndex
from ._common import PROG, USER_CWD, read_tty_line from ._common import PROG, USER_CWD, read_tty_line
@@ -57,15 +57,6 @@ def cmd_start(argv: list[str]) -> int:
"into a cached layer." "into a cached layer."
), ),
) )
parser.add_argument(
"--backend",
choices=known_backend_names(),
default=None,
help=(
"backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND "
"or host auto-selection). Overrides the env var when set."
),
)
parser.add_argument( parser.add_argument(
"--headless", "--headless",
action="store_true", action="store_true",
@@ -74,6 +65,14 @@ def cmd_start(argv: list[str]) -> int:
"skip all prompts. For orchestrators, CI, and webhooks." "skip all prompts. For orchestrators, CI, and webhooks."
), ),
) )
parser.add_argument(
"--cached-images",
action="store_true",
help=(
"quickstart with existing local agent and sidecar images; "
"only valid with --headless"
),
)
parser.add_argument( parser.add_argument(
"--bottle", "--bottle",
action="append", action="append",
@@ -106,6 +105,8 @@ def cmd_start(argv: list[str]) -> int:
help="agent name defined in bot-bottle.json (omit to pick interactively)", help="agent name defined in bot-bottle.json (omit to pick interactively)",
) )
args = parser.parse_args(argv) args = parser.parse_args(argv)
if args.cached_images and not args.headless:
die("--cached-images is only supported with --headless")
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1" dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
if args.no_cache or os.environ.get("BOT_BOTTLE_NO_CACHE") == "1": if args.no_cache or os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
@@ -115,11 +116,10 @@ def cmd_start(argv: list[str]) -> int:
os.environ["BOT_BOTTLE_NO_CACHE"] = "1" os.environ["BOT_BOTTLE_NO_CACHE"] = "1"
manifest = ManifestIndex.resolve(USER_CWD) manifest = ManifestIndex.resolve(USER_CWD)
backend_name: str | None = args.backend
if args.headless: if args.headless:
return _start_headless( return _start_headless(
manifest, args, dry_run=dry_run, backend_name=backend_name manifest, args, dry_run=dry_run
) )
agent_name: str | None = args.name agent_name: str | None = args.name
@@ -158,6 +158,10 @@ def cmd_start(argv: list[str]) -> int:
label, color = tui.name_color_modal(default_label=agent_name) label, color = tui.name_color_modal(default_label=agent_name)
label, color = _resolve_unique_label(label, color) label, color = _resolve_unique_label(label, color)
image_policy = _select_image_policy()
if image_policy is None:
return 0
spec = BottleSpec( spec = BottleSpec(
manifest=manifest, manifest=manifest,
agent_name=agent_name, agent_name=agent_name,
@@ -166,11 +170,11 @@ def cmd_start(argv: list[str]) -> int:
label=label, label=label,
color=color, color=color,
bottle_names=bottle_names, bottle_names=bottle_names,
image_policy=image_policy,
) )
return _launch_bottle( return _launch_bottle(
spec, spec,
dry_run=dry_run, dry_run=dry_run,
backend_name=backend_name,
) )
@@ -182,7 +186,6 @@ def _start_headless(
args: argparse.Namespace, args: argparse.Namespace,
*, *,
dry_run: bool, dry_run: bool,
backend_name: str | None,
) -> int: ) -> int:
"""Non-interactive launch path for orchestrators / CI / webhooks. """Non-interactive launch path for orchestrators / CI / webhooks.
@@ -226,11 +229,11 @@ def _start_headless(
color=args.color or "", color=args.color or "",
bottle_names=bottle_names, bottle_names=bottle_names,
headless=True, headless=True,
image_policy="cached" if args.cached_images else "fresh",
) )
return _launch_bottle( return _launch_bottle(
spec, spec,
dry_run=dry_run, dry_run=dry_run,
backend_name=backend_name,
assume_yes=True, assume_yes=True,
headless_prompt_text=prompt, headless_prompt_text=prompt,
) )
@@ -268,15 +271,18 @@ def prepare_with_preflight(
injected callable, prompt y/N via the injected callable. injected callable, prompt y/N via the injected callable.
`backend_name` selects which backend prepares the plan `backend_name` selects which backend prepares the plan
(`None` `$BOT_BOTTLE_BACKEND` host auto-selection). The CLI (`None` `$BOT_BOTTLE_BACKEND` host auto-selection).
passes whatever `--backend` resolved to.
When `spec.headless` is True the docker-fallback prompt is suppressed:
auto-selection dies with an actionable message rather than blocking
on a TTY read (which would hang CI, webhook dispatch, and orchestrators).
Returns `(plan, identity)`. `plan` is None on dry-run or Returns `(plan, identity)`. `plan` is None on dry-run or
operator-N, but `identity` is set as soon as `backend.prepare` operator-N, but `identity` is set as soon as `backend.prepare`
returns so callers can reap the prepare-time state dir via returns so callers can reap the prepare-time state dir via
`settle_state(identity)` in their finally exactly the existing `settle_state(identity)` in their finally exactly the existing
semantics.""" semantics."""
backend = get_bottle_backend(backend_name) backend = get_bottle_backend(backend_name, prompt=not spec.headless)
plan = backend.prepare(spec, stage_dir=stage_dir) plan = backend.prepare(spec, stage_dir=stage_dir)
identity = _identity_from_plan(plan) identity = _identity_from_plan(plan)
@@ -406,6 +412,13 @@ def _text_prompt_yes() -> bool:
return reply in ("y", "Y", "yes", "YES") return reply in ("y", "Y", "yes", "YES")
def _select_image_policy() -> str | None:
return tui.filter_select(
["fresh", "cached"],
title="Select image startup mode",
)
def _text_render_preflight(): def _text_render_preflight():
def _render(plan: DockerBottlePlan, backend_name: str) -> None: def _render(plan: DockerBottlePlan, backend_name: str) -> None:
print(file=sys.stderr) print(file=sys.stderr)
@@ -548,6 +561,15 @@ def _launch_bottle(
return 0 return 0
backend = get_bottle_backend(backend_name) backend = get_bottle_backend(backend_name)
try:
backend.prelaunch_checks(plan)
except StaleImageError as exc:
if assume_yes:
die(str(exc))
sys.stderr.write(f"bot-bottle: {exc}\nLaunch anyway? [y/N] ")
sys.stderr.flush()
if read_tty_line() not in ("y", "Y", "yes", "YES"):
return 0
with backend.launch(plan) as bottle: with backend.launch(plan) as bottle:
agent_provider_template = getattr(plan, "agent_provider_template", "claude") agent_provider_template = getattr(plan, "agent_provider_template", "claude")
extra_args: tuple[str, ...] = () extra_args: tuple[str, ...] = ()
@@ -566,10 +588,6 @@ def _launch_bottle(
f"session ended (exit {exit_code}); " f"session ended (exit {exit_code}); "
f"container {bottle.name} will be removed" f"container {bottle.name} will be removed"
) )
# While the container is still alive: always snapshot the
# transcript and — if the agent exited non-zero — mark
# the state for preservation. This picks up crashes /
# Ctrl-Cs / OOM kills before cleanup removes the state dir.
if agent_provider_template == "claude": if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code) capture_claude_session_state(identity, exit_code)
return 0 return 0
+71
View File
@@ -0,0 +1,71 @@
"""SQLite-backed bot-bottle configuration store."""
from __future__ import annotations
from pathlib import Path
try:
from .db_store import DbStore
from .migrations import TableMigrations
from .paths import host_db_path
except ImportError:
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS = 1
class ConfigStore(DbStore):
"""SQLite configuration for host-side bot-bottle settings."""
def __init__(self, db_path: Path | None = None) -> None:
migrations = TableMigrations("config_store", [
# v1 — host-side bot-bottle settings
"""
CREATE TABLE IF NOT EXISTS bot_bottle_config (
id INTEGER PRIMARY KEY CHECK (id = 1),
cached_image_stale_warning_days INTEGER NOT NULL DEFAULT 1
)
""",
])
super().__init__(db_path or host_db_path(), migrations)
def cached_image_stale_warning_days(self) -> int:
if not self.db_path.is_file():
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
with self._connect() as conn:
row = conn.execute(
"""
SELECT cached_image_stale_warning_days
FROM bot_bottle_config
WHERE id = 1
""",
).fetchone()
if row is None:
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
try:
return int(row["cached_image_stale_warning_days"])
except (TypeError, ValueError):
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
def set_cached_image_stale_warning_days(self, days: int) -> Path:
with self._connect() as conn:
conn.execute(
"""
INSERT INTO bot_bottle_config (id, cached_image_stale_warning_days)
VALUES (1, ?)
ON CONFLICT(id) DO UPDATE SET
cached_image_stale_warning_days = excluded.cached_image_stale_warning_days
""",
(days,),
)
self._chmod()
return self.db_path
__all__ = [
"DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS",
"ConfigStore",
]
+17
View File
@@ -0,0 +1,17 @@
"""Shared wire-protocol constants for gateway-bundled modules.
Single source of truth for values that appear across the egress addon,
git-http backend, supervise server, and git-gate renderer. Importing
from this module instead of duplicating the literals means a rename is
a one-line change and is caught by the type checker at the import site."""
# App-layer identity token header. Delivered as proxy credentials
# (HTTPS_PROXY=http://<bottle_id>:<token>@gw) by launch; the egress
# addon reads and strips it, the supervise server and git-http backend
# read it for attribution, and none of them forward it upstream.
IDENTITY_HEADER = "x-bot-bottle-identity"
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
# git daemon (--timeout/--init-timeout), the access-hook subprocess in
# git_http_backend, and the git http-backend CGI subprocess.
GIT_GATE_TIMEOUT_SECS = 15
+15 -2
View File
@@ -10,7 +10,7 @@
# Current Node LTS; slim variant keeps the image small while still # Current Node LTS; slim variant keeps the image small while still
# providing apt-get for any future additions. # providing apt-get for any future additions.
FROM node:22-slim FROM node:22-trixie-slim
# Install runtime system deps. claude-code shells out to git for several # Install runtime system deps. claude-code shells out to git for several
# features (status checks, commits, PR creation) — without git in the # features (status checks, commits, PR creation) — without git in the
@@ -21,7 +21,15 @@ FROM node:22-slim
# to it) works against egress's bumped TLS without the agent needing # to it) works against egress's bumped TLS without the agent needing
# local DNS. # local DNS.
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep iproute2 dnsutils \ && apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
openssh-client \
podman \
ripgrep \
iproute2 \
dnsutils \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by claude-code itself # App-specific deps. Python isn't required by claude-code itself
@@ -39,6 +47,11 @@ RUN apt-get update \
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \ RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
&& npm cache clean --force && npm cache clean --force
# Git reads both ~/.gitconfig and ~/.config/git/config. Keep its XDG config
# path traversable by the non-root runtime user so permission errors do not
# suppress bot-bottle's git-gate insteadOf rules.
RUN install -d -o node -g node -m 755 /home/node/.config /home/node/.config/git
# Run as a non-root user. The node image already provides a `node` user # Run as a non-root user. The node image already provides a `node` user
# (uid 1000) with a home directory, which is where claude-code will write # (uid 1000) with a home directory, which is where claude-code will write
# its session state. # its session state.
+17 -5
View File
@@ -23,8 +23,9 @@ from ...agent_provider import (
provider_startup_args, provider_startup_args,
) )
from ...backend.docker import util as docker_mod from ...backend.docker import util as docker_mod
from ...egress import EgressRoute from ...egress import CLAUDE_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
from ...log import die, info, warn from ...log import die, info, warn
from .claude_auth import claude_host_access_token
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -118,7 +119,6 @@ class ClaudeAgentProvider(AgentProvider):
color: str = "", color: str = "",
provider_settings: dict[str, object] | None = None, provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan: ) -> AgentProvisionPlan:
del forward_host_credentials, host_env
resolved_guest_env = dict(guest_env or {}) resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings) startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home guest_home = self.guest_home
@@ -180,13 +180,24 @@ class ClaudeAgentProvider(AgentProvider):
claude_settings, claude_settings,
f"{guest_home}/.claude/settings.json", f"{guest_home}/.claude/settings.json",
)) ))
provisioned_env: dict[str, str] = {}
if forward_host_credentials:
_host_env = host_env or dict(os.environ)
provisioned_env[CLAUDE_HOST_CREDENTIAL_TOKEN_REF] = (
claude_host_access_token(_host_env)
)
cred_token_ref = (
CLAUDE_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials
else auth_token
)
egress_routes = (EgressRoute( egress_routes = (EgressRoute(
host="api.anthropic.com", host="api.anthropic.com",
auth_scheme="Bearer" if auth_token else "", auth_scheme="Bearer" if (auth_token or forward_host_credentials) else "",
token_ref=auth_token, token_ref=cred_token_ref,
),) ),)
hidden_env_names: frozenset[str] = frozenset() hidden_env_names: frozenset[str] = frozenset()
if auth_token: if auth_token or forward_host_credentials:
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder" env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"}) hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
@@ -208,6 +219,7 @@ class ClaudeAgentProvider(AgentProvider):
files=tuple(files), files=tuple(files),
egress_routes=egress_routes, egress_routes=egress_routes,
hidden_env_names=hidden_env_names, hidden_env_names=hidden_env_names,
provisioned_env=provisioned_env,
) )
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None: def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
+114
View File
@@ -0,0 +1,114 @@
"""Host Claude auth helpers.
Reads the host's Claude Code credentials and returns only the access
token needed by egress. Does not expose refresh tokens or raw payloads.
Credential storage by platform:
Linux ~/.claude/.credentials.json
macOS macOS Keychain, service "Claude Code-credentials"
(file path is tried first; Keychain is the fallback)
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path
from ...log import die
_KEYCHAIN_SERVICE = "Claude Code-credentials"
def claude_auth_path(host_env: dict[str, str] | None = None) -> Path:
env = os.environ if host_env is None else host_env
home = env.get("HOME")
if home:
return Path(home) / ".claude" / ".credentials.json"
return Path.home() / ".claude" / ".credentials.json"
def _read_keychain() -> dict[str, object] | None:
"""Try the macOS Keychain. Returns parsed JSON dict or None."""
if sys.platform != "darwin":
return None
try:
result = subprocess.run(
["security", "find-generic-password", "-s", _KEYCHAIN_SERVICE, "-w"],
capture_output=True,
text=True,
timeout=10,
)
except (FileNotFoundError, subprocess.TimeoutExpired):
return None
if result.returncode != 0 or not result.stdout.strip():
return None
try:
raw = json.loads(result.stdout.strip())
except json.JSONDecodeError:
return None
return raw if isinstance(raw, dict) else None
def claude_host_access_token(
host_env: dict[str, str] | None = None,
*,
now: datetime | None = None,
) -> str:
path = claude_auth_path(host_env)
raw: dict[str, object] | None = None
if path.is_file():
try:
raw = json.loads(path.read_text())
except (OSError, json.JSONDecodeError) as e:
die(f"claude host credentials: could not read valid JSON at {path}: {e}")
if not isinstance(raw, dict):
die(f"claude host credentials: {path} must contain a JSON object")
else:
raw = _read_keychain()
if raw is None:
die(
f"claude host credentials: auth file missing at {path} and "
f"macOS Keychain lookup for '{_KEYCHAIN_SERVICE}' failed. "
"Run `claude login` on the host or disable "
"agent_provider.forward_host_credentials."
)
oauth = raw.get("claudeAiOauth")
if not isinstance(oauth, dict):
die(
"claude host credentials: claudeAiOauth is missing from credentials. "
"Run `claude login` on the host or disable "
"agent_provider.forward_host_credentials."
)
access_token = oauth.get("accessToken")
if not isinstance(access_token, str) or not access_token:
die(
"claude host credentials: claudeAiOauth.accessToken is missing or empty. "
"Run `claude login` on the host and restart the bottle."
)
# expiresAt is in milliseconds
expires_at = oauth.get("expiresAt")
if isinstance(expires_at, (int, float)):
check_now = now or datetime.now(timezone.utc)
exp_dt = datetime.fromtimestamp(float(expires_at) / 1000.0, timezone.utc)
if exp_dt <= check_now:
die(
"claude host credentials: host Claude access token is expired. "
"Run `claude login` on the host and restart the bottle."
)
return access_token
__all__ = [
"claude_auth_path",
"claude_host_access_token",
]
+11 -2
View File
@@ -3,10 +3,17 @@
# Mirrors the default Claude image shape: Node LTS, git/network tooling, # Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed for that user. # non-root node user, and the provider CLI installed for that user.
FROM node:22-slim FROM node:22-trixie-slim
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl procps ripgrep \ && apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
openssh-client \
podman \
procps \
ripgrep \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by codex itself # App-specific deps. Python isn't required by codex itself
@@ -17,6 +24,8 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \ && apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
RUN install -d -o node -g node -m 755 /home/node/.config /home/node/.config/git
USER node USER node
WORKDIR /home/node WORKDIR /home/node
+5 -2
View File
@@ -2,7 +2,7 @@
# #
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally. # Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
FROM node:22-slim FROM node:22-trixie-slim
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends \ && apt-get install -y --no-install-recommends \
@@ -10,6 +10,8 @@ RUN apt-get update \
ca-certificates \ ca-certificates \
curl \ curl \
fd-find \ fd-find \
openssh-client \
podman \
ripgrep \ ripgrep \
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \ && ln -s /usr/bin/fdfind /usr/local/bin/fd \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
@@ -21,7 +23,8 @@ RUN apt-get update \
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \ RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
&& npm cache clean --force && npm cache clean --force
RUN mkdir -p /home/node/.pi/agent \ RUN install -d -o node -g node -m 755 /home/node/.config /home/node/.config/git \
&& mkdir -p /home/node/.pi/agent \
/home/node/.pi/context-mode/sessions \ /home/node/.pi/context-mode/sessions \
/tmp/pi-subagents-uid-1000 \ /tmp/pi-subagents-uid-1000 \
&& chown -R node:node /home/node/.pi /tmp \ && chown -R node:node /home/node/.pi /tmp \
+12 -2
View File
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
import sqlite3 import sqlite3
from contextlib import contextmanager
from pathlib import Path from pathlib import Path
try: try:
@@ -28,12 +29,21 @@ class DbStore:
conn.row_factory = sqlite3.Row conn.row_factory = sqlite3.Row
return conn return conn
@contextmanager
def _connection(self):
conn = self._connect()
try:
with conn:
yield conn
finally:
conn.close()
def is_migrated(self) -> bool: def is_migrated(self) -> bool:
"""Return True if the DB is fully up-to-date, False if migration is needed.""" """Return True if the DB is fully up-to-date, False if migration is needed."""
if not self.db_path.exists(): if not self.db_path.exists():
return False return False
try: try:
with self._connect() as conn: with self._connection() as conn:
row = conn.execute( row = conn.execute(
"SELECT version FROM schema_versions WHERE module = ?", "SELECT version FROM schema_versions WHERE module = ?",
(self._migrations.schema_key,), (self._migrations.schema_key,),
@@ -45,7 +55,7 @@ class DbStore:
def migrate(self) -> None: def migrate(self) -> None:
"""Apply any pending migrations and set permissions on the DB file.""" """Apply any pending migrations and set permissions on the DB file."""
with self._connect() as conn: with self._connection() as conn:
self._migrations.apply(conn) self._migrations.apply(conn)
self._chmod() self._chmod()
+2 -6
View File
@@ -3,9 +3,8 @@
Pure Python, no mitmproxy dependency. Each detector is a module-level Pure Python, no mitmproxy dependency. Each detector is a module-level
function returning `ScanResult | None`. function returning `ScanResult | None`.
Ships flat into the gateway image alongside Available in the gateway via the installed `bot_bottle` package
`egress_addon_core.py` both this file and the package source use (see `Dockerfile.gateway`).
the same try/except import shim pattern.
""" """
from __future__ import annotations from __future__ import annotations
@@ -20,9 +19,6 @@ from math import log2
from collections import Counter from collections import Counter
from urllib.parse import quote as url_quote from urllib.parse import quote as url_quote
try:
from egress_addon_core import ScanResult # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from .egress_addon_core import ScanResult from .egress_addon_core import ScanResult
+10 -3
View File
@@ -14,13 +14,20 @@ from __future__ import annotations
import subprocess import subprocess
def run_docker(argv: list[str]) -> subprocess.CompletedProcess[str]: def run_docker(
argv: list[str], *, env: dict[str, str] | None = None,
) -> subprocess.CompletedProcess[str]:
"""Run a `docker` command, capturing stdout/stderr as text. Never raises """Run a `docker` command, capturing stdout/stderr as text. Never raises
on a non-zero exit callers inspect `returncode` / `stderr` so they can on a non-zero exit callers inspect `returncode` / `stderr` so they can
stay fail-closed or tolerate idempotent no-ops (e.g. removing an stay fail-closed or tolerate idempotent no-ops (e.g. removing an
already-absent container).""" already-absent container).
`env` sets the child process environment used to hand a secret to a bare
`--env NAME` flag (docker inherits its value from this process) so the
value never lands on argv or in `docker inspect`'s recorded command line."""
return subprocess.run( return subprocess.run(
argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False, argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
check=False, env=env,
) )
+3
View File
@@ -30,6 +30,7 @@ if TYPE_CHECKING:
from .manifest import ManifestBottle from .manifest import ManifestBottle
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN" CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
CLAUDE_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CLAUDE_HOST_ACCESS_TOKEN"
EGRESS_HOSTNAME = "egress" EGRESS_HOSTNAME = "egress"
@@ -145,6 +146,7 @@ def egress_manifest_routes(
outbound_detectors=r.OutboundDetectors, outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors, inbound_detectors=r.InboundDetectors,
outbound_on_match=r.OutboundOnMatch, outbound_on_match=r.OutboundOnMatch,
preserve_auth=r.PreserveAuth,
)) ))
return tuple(out) return tuple(out)
@@ -400,6 +402,7 @@ class Egress(ABC):
) )
__all__ = [ __all__ = [
"CLAUDE_HOST_CREDENTIAL_TOKEN_REF",
"CODEX_HOST_CREDENTIAL_TOKEN_REF", "CODEX_HOST_CREDENTIAL_TOKEN_REF",
"EGRESS_HOSTNAME", "EGRESS_HOSTNAME",
"EGRESS_ROUTES_FILENAME", "EGRESS_ROUTES_FILENAME",
+137 -119
View File
@@ -10,14 +10,14 @@ import base64
import binascii import binascii
import json import json
import os import os
import signal
import sys import sys
import typing import typing
from pathlib import Path
from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error from bot_bottle.constants import IDENTITY_HEADER
from bot_bottle.dlp_detectors import redact_tokens, strip_crlf
from bot_bottle.egress_addon_core import (
LOG_BLOCKS, LOG_BLOCKS,
LOG_FULL, LOG_FULL,
DEFAULT_OUTBOUND_ON_MATCH, DEFAULT_OUTBOUND_ON_MATCH,
@@ -33,7 +33,6 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
decide_git_fetch, decide_git_fetch,
is_git_fetch_request, is_git_fetch_request,
is_git_push_request, is_git_push_request,
load_config,
match_route, match_route,
resolve_client_context, resolve_client_context,
outbound_scan_headers, outbound_scan_headers,
@@ -41,42 +40,25 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
scan_inbound, scan_inbound,
scan_outbound, scan_outbound,
) )
from bot_bottle import supervise as _sv
try:
from dlp_detectors import redact_tokens, strip_crlf # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.dlp_detectors import ( # type: ignore[import-not-found]
redact_tokens,
strip_crlf,
)
try:
import supervise as _sv # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
try:
from policy_resolver import PolicyResolver # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.policy_resolver import PolicyResolver from bot_bottle.policy_resolver import PolicyResolver
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
INTROSPECT_HOST = "_egress.local" INTROSPECT_HOST = "_egress.local"
# Consolidated (multi-tenant) mode: when this points at the per-host # The per-host orchestrator control plane the addon resolves every request's
# orchestrator's control plane, the addon resolves each client's Config by # Config against, by source IP (PRD 0070). Mandatory: the consolidated gateway
# source IP per request instead of using a single static routes file. Unset # is the only topology now — there is no static per-bottle routes file to fall
# → legacy per-bottle single-tenant mode (unchanged). # back to — so an unset value is a fatal misconfiguration (see __init__).
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL" ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
# App-layer identity token. Delivered as proxy credentials # Per-flow key under which `request()` stashes the resolved (Config, supervise
# (`HTTPS_PROXY=http://<bottle_id>:<token>@gw`): clients honor it as part of # slug, env) so the later `response()` and `websocket_message()` hooks scan
# the proxy protocol without app changes, and the addon reads + strips it so # against the *calling bottle's* policy — the same one the request was decided
# it never leaks upstream. The legacy `x-bot-bottle-identity` request header # on — without a second `/resolve` per response or per WebSocket frame. A hook
# is still stripped defensively (git-http uses that header on its own port). # on a flow that never resolved (no stash) fails closed to deny-all, so it's a
IDENTITY_HEADER = "x-bot-bottle-identity" # safe no-op rather than an unscanned pass.
_FLOW_CTX_KEY = "bot_bottle_egress_ctx"
def _token_from_proxy_auth(header: str) -> str: def _token_from_proxy_auth(header: str) -> str:
@@ -109,21 +91,30 @@ _TOKEN_ALLOW_JUSTIFICATION = (
class EgressAddon: class EgressAddon:
# Class default so addons built via __new__ (e.g. in tests) default to # Bare annotations (no class value): __init__ sets a live PolicyResolver for
# single-tenant; __init__ sets the instance attribute for real runs. # real runs, and every host-side test builds an addon via __new__ and sets a
_resolver: "PolicyResolver | None" = None # fake resolver. Egress is resolver-only now — the per-request policy always
# comes from the orchestrator's /resolve (PRD 0070); there is no static
# per-bottle routes file, SIGHUP reload, or single-tenant fallback.
_resolver: "PolicyResolver"
# Class default so __new__-built addons have it (real runs get a fresh # Class default so __new__-built addons have it (real runs get a fresh
# per-instance dict in __init__; only http_connect mutates it, which the # per-instance dict in __init__; only http_connect mutates it, which the
# request-flow tests don't exercise). # request-flow tests don't exercise).
_conn_tokens: "dict[str, str]" = {} _conn_tokens: "dict[str, str]" = {}
def __init__(self) -> None: def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH) # Resolver-only: the gateway is always multi-tenant, resolving each
self.config: Config = Config(routes=()) # request's policy by source IP against the orchestrator control plane
# Consolidated mode: resolve per-client Config from the orchestrator. # (PRD 0070). The URL is mandatory — without a policy source the gateway
# Absent → single-tenant (static routes file); behaviour unchanged. # must not come up (fail-closed), rather than silently allowing nothing.
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip() orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
self._resolver = PolicyResolver(orch_url) if orch_url else None if not orch_url:
raise RuntimeError(
f"{ORCHESTRATOR_URL_ENV} is required: the egress gateway "
"resolves every request's policy from the orchestrator and has "
"no static routes file to fall back to."
)
self._resolver = PolicyResolver(orch_url)
# Tokens the operator has approved this session (PRD 0062), keyed by # Tokens the operator has approved this session (PRD 0062), keyed by
# bottle so the shared gateway keeps each bottle's safelist separate — # bottle so the shared gateway keeps each bottle's safelist separate —
# a global set would let bottle A's approved secret pass bottle B's DLP # a global set would let bottle A's approved secret pass bottle B's DLP
@@ -134,16 +125,13 @@ class EgressAddon:
# `Proxy-Authorization` (HTTPS tunnels don't repeat it on the bumped # `Proxy-Authorization` (HTTPS tunnels don't repeat it on the bumped
# inner requests). Keyed by client_conn.id; cleared on disconnect. # inner requests). Keyed by client_conn.id; cleared on disconnect.
self._conn_tokens: dict[str, str] = {} self._conn_tokens: dict[str, str] = {}
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ) self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True)
self._install_sighup()
@staticmethod @staticmethod
def _supervise_available(slug: str) -> bool: def _supervise_available(slug: str) -> bool:
"""Supervise is reachable for this request iff we resolved a bottle to """Supervise is reachable for this request iff we resolved a bottle to
attribute its proposals to (single-tenant env slug, or a source-IP attribute its proposals to (the source-IP-attributed bottle id). Empty
-attributed bottle id). Empty fail closed (no queue to write to).""" fail closed (no queue to write to)."""
return bool(slug) return bool(slug)
def _safe_tokens_for(self, slug: str) -> set[str]: def _safe_tokens_for(self, slug: str) -> set[str]:
@@ -152,40 +140,15 @@ class EgressAddon:
bottle's approved token into another's scan.""" bottle's approved token into another's scan."""
return self._safe_tokens.setdefault(slug, set()) return self._safe_tokens.setdefault(slug, set())
def _reload(self, *, initial: bool = False) -> None: def _serve_introspection(
try: self, flow: http.HTTPFlow, path: str, config: Config,
text = Path(self.routes_path).read_text(encoding="utf-8") ) -> None:
new_config = load_config(text) """Serve the calling bottle's own allowlist. `config` is this flow's
except (OSError, ValueError) as e: resolved policy (the same one every hook uses), so the agent sees the
tag = "boot" if initial else "SIGHUP" routes that actually apply to it."""
sys.stderr.write(
f"egress: {tag} load failed: {e}\n"
)
if initial:
self.config = Config(routes=())
return
self.config = new_config
log_label = ("off", "blocks", "full")[self.config.log]
sys.stderr.write(
f"egress: loaded {len(self.config.routes)} route(s): "
f"{', '.join(r.host for r in self.config.routes)}"
f" [log={log_label}]\n"
)
def _install_sighup(self) -> None:
if not hasattr(signal, "SIGHUP"):
return
def handler(signum: int, frame: object) -> None:
del signum, frame
self._reload()
signal.signal(signal.SIGHUP, handler)
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
if path == "/allowlist": if path == "/allowlist":
payload = json.dumps( payload = json.dumps(
{"routes": [route_to_yaml_dict(r) for r in self.config.routes]}, {"routes": [route_to_yaml_dict(r) for r in config.routes]},
indent=2, indent=2,
).encode("utf-8") ).encode("utf-8")
flow.response = http.Response.make( flow.response = http.Response.make(
@@ -199,11 +162,21 @@ class EgressAddon:
{"Content-Type": "text/plain; charset=utf-8"}, {"Content-Type": "text/plain; charset=utf-8"},
) )
def _flow_log(self, flow: http.HTTPFlow) -> int:
"""This flow's log level, from the policy `request()` resolved and
stashed. The block/redact log gates were a single global in the static-
config world; they are per bottle now, so they read it from the flow."""
return self._flow_ctx(flow)[0].log
def _req_ctx(self, flow: http.HTTPFlow) -> dict[str, object]: def _req_ctx(self, flow: http.HTTPFlow) -> dict[str, object]:
# Redact with this flow's resolved env overlay (process env + the
# bottle's /resolve tokens), so the ctx scrubs the calling bottle's
# provisioned secrets, not just os.environ's.
env = self._flow_ctx(flow)[2]
return { return {
"host": redact_tokens(flow.request.pretty_host, env=os.environ), "host": redact_tokens(flow.request.pretty_host, env=env),
"method": flow.request.method, "method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ), "path": redact_tokens(flow.request.path, env=env),
} }
def _block( def _block(
@@ -212,7 +185,7 @@ class EgressAddon:
reason: str, reason: str,
ctx: dict[str, object] | None = None, ctx: dict[str, object] | None = None,
) -> None: ) -> None:
if self.config.log >= LOG_BLOCKS: if self._flow_log(flow) >= LOG_BLOCKS:
entry: dict[str, object] = {"event": "egress_block", "reason": reason} entry: dict[str, object] = {"event": "egress_block", "reason": reason}
if ctx: if ctx:
entry.update(ctx) entry.update(ctx)
@@ -223,31 +196,39 @@ class EgressAddon:
{"Content-Type": "text/plain; charset=utf-8"}, {"Content-Type": "text/plain; charset=utf-8"},
) )
def _log_request(self, flow: http.HTTPFlow) -> None: def _log_request(
self, flow: http.HTTPFlow, env: "typing.Mapping[str, str]",
) -> None:
# `env` is the per-flow resolved overlay (process env + this bottle's
# /resolve tokens), so the log redaction scrubs the calling bottle's
# provisioned secrets — not just the process-level ones in os.environ.
headers = { headers = {
k: redact_tokens(v, env=os.environ) k: redact_tokens(v, env=env)
for k, v in flow.request.headers.items() for k, v in flow.request.headers.items()
if k.lower() != "authorization" if k.lower() != "authorization"
} }
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ) body = redact_tokens(flow.request.get_text(strict=False) or "", env=env)
sys.stderr.write( sys.stderr.write(
json.dumps({ json.dumps({
"event": "egress_request", "event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ), "host": redact_tokens(flow.request.pretty_host, env=env),
"method": flow.request.method, "method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ), "path": redact_tokens(flow.request.path, env=env),
"headers": headers, "headers": headers,
"body": body, "body": body,
}) })
+ "\n" + "\n"
) )
def _log_response(self, flow: http.HTTPFlow) -> None: def _log_response(
self, flow: http.HTTPFlow, env: "typing.Mapping[str, str]",
) -> None:
# Per-flow env overlay (see _log_request): redact this bottle's tokens.
headers = { headers = {
k: redact_tokens(v, env=os.environ) k: redact_tokens(v, env=env)
for k, v in flow.response.headers.items() for k, v in flow.response.headers.items()
} }
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ) body = redact_tokens(flow.response.get_text(strict=False) or "", env=env)
sys.stderr.write( sys.stderr.write(
json.dumps({ json.dumps({
"event": "egress_response", "event": "egress_response",
@@ -262,17 +243,12 @@ class EgressAddon:
def _resolve_flow( def _resolve_flow(
self, flow: http.HTTPFlow, self, flow: http.HTTPFlow,
) -> "tuple[Config, str, typing.Mapping[str, str]]": ) -> "tuple[Config, str, typing.Mapping[str, str]]":
"""The `(Config, supervise slug, env)` to apply to this request. """The calling bottle's `(Config, supervise slug, env)`, resolved by
Single-tenant the static `self.config`, the env slug, and the process source IP in one round-trip against the orchestrator fail-closed to
env. Consolidated the calling bottle's Config + bottle id + auth deny-all + empty slug if unattributed. `env` is the process env overlaid
tokens, resolved by source IP in one round-trip (fail-closed to deny-all with the bottle's `/resolve` tokens, so upstream-auth injection (and DLP)
+ empty slug if unattributed); `env` is the process env overlaid with use *this* bottle's credentials. The identity token, if the agent
the bottle's tokens, so upstream-auth injection (and DLP) use *this* injected one, is read then stripped so it never leaks upstream."""
bottle's credentials — exactly what the per-bottle gateway daemon's env did.
The identity token, if the agent injected one, is read then stripped so
it never leaks upstream."""
if self._resolver is None:
return self.config, self._supervise_slug, os.environ
conn = flow.client_conn conn = flow.client_conn
client_ip = conn.peername[0] if conn and conn.peername else "" client_ip = conn.peername[0] if conn and conn.peername else ""
token = self._request_token(flow) token = self._request_token(flow)
@@ -280,6 +256,36 @@ class EgressAddon:
env = {**os.environ, **tokens} if tokens else os.environ env = {**os.environ, **tokens} if tokens else os.environ
return config, slug, env return config, slug, env
def _stash_flow_ctx(
self,
flow: http.HTTPFlow,
config: Config,
slug: str,
env: "typing.Mapping[str, str]",
) -> None:
"""Remember the per-flow context `request()` resolved, so the later
`response()` / `websocket_message()` hooks reuse it scanning against
the same bottle's policy the request was decided on, with one `/resolve`
per flow rather than one per frame."""
meta = getattr(flow, "metadata", None)
if isinstance(meta, dict):
meta[_FLOW_CTX_KEY] = (config, slug, env)
def _flow_ctx(
self, flow: http.HTTPFlow,
) -> "tuple[Config, str, typing.Mapping[str, str]]":
"""The `(Config, supervise slug, env)` `request()` resolved for this
flow, so a later hook scans against the calling bottle's policy. Falls
back to deny-all (empty routes, empty slug) for a flow that never passed
through `request()` (or a flow object without metadata) fail-closed, so
a DLP hook on such a flow is a safe no-op rather than an unscanned pass."""
meta = getattr(flow, "metadata", None)
if isinstance(meta, dict):
ctx = meta.get(_FLOW_CTX_KEY)
if ctx is not None:
return ctx
return Config(routes=()), "", os.environ
def _request_token(self, flow: http.HTTPFlow) -> str: def _request_token(self, flow: http.HTTPFlow) -> str:
"""The per-bottle identity token for this request, from the proxy """The per-bottle identity token for this request, from the proxy
credentials the delivery mechanism (`HTTPS_PROXY=http://id:token@gw`) credentials the delivery mechanism (`HTTPS_PROXY=http://id:token@gw`)
@@ -314,11 +320,17 @@ class EgressAddon:
async def request(self, flow: http.HTTPFlow) -> None: async def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?") request_path, _, query = flow.request.path.partition("?")
if flow.request.pretty_host == INTROSPECT_HOST:
self._serve_introspection(flow, request_path)
return
config, slug, env = self._resolve_flow(flow) config, slug, env = self._resolve_flow(flow)
# Stash for the response / websocket hooks so their DLP scans reuse this
# bottle's resolved policy (one /resolve per flow — see _flow_ctx).
self._stash_flow_ctx(flow, config, slug, env)
# Introspection ("_egress.local/allowlist") reports the calling bottle's
# own resolved routes — served after resolution so it reflects this
# bottle's policy, not a stale global.
if flow.request.pretty_host == INTROSPECT_HOST:
self._serve_introspection(flow, request_path, config)
return
# DLP outbound scan BEFORE stripping auth — catches tokens the # DLP outbound scan BEFORE stripping auth — catches tokens the
# agent tried to smuggle in any header, path, query param, or body. # agent tried to smuggle in any header, path, query param, or body.
@@ -355,6 +367,9 @@ class EgressAddon:
# Strip agent-set Authorization after DLP scan so smuggled tokens # Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject gateway-owned auth below. # are caught above; the route may inject gateway-owned auth below.
# Routes with preserve_auth=True pass the header through as-is so the
# agent's own credentials (e.g. registry bearer tokens) reach the upstream.
if route is None or not route.preserve_auth:
flow.request.headers.pop("authorization", None) flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation # Build headers mapping for match evaluation
@@ -367,6 +382,7 @@ class EgressAddon:
env, env,
request_method=flow.request.method, request_method=flow.request.method,
request_headers=req_headers, request_headers=req_headers,
deny_reason=config.deny_reason,
) )
if decision.action == "block": if decision.action == "block":
@@ -377,7 +393,7 @@ class EgressAddon:
flow.request.headers["authorization"] = decision.inject_authorization flow.request.headers["authorization"] = decision.inject_authorization
if config.log >= LOG_FULL: if config.log >= LOG_FULL:
self._log_request(flow) self._log_request(flow, env)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None: def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
ctx = self._req_ctx(flow) ctx = self._req_ctx(flow)
@@ -424,7 +440,7 @@ class EgressAddon:
# forwards; it fails closed only if a match survives the scrub. # forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT: if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route, env): if self._redact_outbound(flow, route, env):
if self.config.log >= LOG_BLOCKS: if self._flow_log(flow) >= LOG_BLOCKS:
sys.stderr.write(json.dumps({ sys.stderr.write(json.dumps({
"event": "egress_redacted", "event": "egress_redacted",
"reason": f"egress DLP: {result.reason}", "reason": f"egress DLP: {result.reason}",
@@ -546,7 +562,7 @@ class EgressAddon:
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED, _sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
): ):
self._safe_tokens_for(slug).add(result.matched) self._safe_tokens_for(slug).add(result.matched)
if self.config.log >= LOG_BLOCKS: if self._flow_log(flow) >= LOG_BLOCKS:
sys.stderr.write(json.dumps({ sys.stderr.write(json.dumps({
"event": "egress_token_allowed", "event": "egress_token_allowed",
"reason": f"egress DLP: {result.reason}", "reason": f"egress DLP: {result.reason}",
@@ -586,14 +602,16 @@ class EgressAddon:
await asyncio.sleep(TOKEN_ALLOW_POLL_INTERVAL_SECONDS) await asyncio.sleep(TOKEN_ALLOW_POLL_INTERVAL_SECONDS)
def response(self, flow: http.HTTPFlow) -> None: def response(self, flow: http.HTTPFlow) -> None:
"""DLP inbound scan on response headers and body.""" """DLP inbound scan on response headers and body, against the calling
route = match_route(self.config.routes, flow.request.pretty_host) bottle's resolved config (`request()` stashed it — see `_flow_ctx`)."""
config, _slug, env = self._flow_ctx(flow)
route = match_route(config.routes, flow.request.pretty_host)
if route is None: if route is None:
return return
if flow.response is None: if flow.response is None:
return return
if self.config.log >= LOG_FULL: if config.log >= LOG_FULL:
self._log_response(flow) self._log_response(flow, env)
resp_headers = {k.lower(): v for k, v in flow.response.headers.items()} resp_headers = {k.lower(): v for k, v in flow.response.headers.items()}
body = flow.response.get_text(strict=False) or "" body = flow.response.get_text(strict=False) or ""
scan_text = build_inbound_scan_text(resp_headers, body) scan_text = build_inbound_scan_text(resp_headers, body)
@@ -610,7 +628,7 @@ class EgressAddon:
resp_ctx = {**resp_ctx, "context": result.context} resp_ctx = {**resp_ctx, "context": result.context}
if result.severity == "block": if result.severity == "block":
self._block(flow, f"egress DLP: {result.reason}", ctx=resp_ctx) self._block(flow, f"egress DLP: {result.reason}", ctx=resp_ctx)
elif result.severity == "warn" and self.config.log >= LOG_BLOCKS: elif result.severity == "warn" and config.log >= LOG_BLOCKS:
sys.stderr.write( sys.stderr.write(
json.dumps({ json.dumps({
"event": "egress_warn", "event": "egress_warn",
@@ -621,7 +639,9 @@ class EgressAddon:
) )
def websocket_message(self, flow: http.HTTPFlow) -> None: def websocket_message(self, flow: http.HTTPFlow) -> None:
"""DLP scan on WebSocket frames. """DLP scan on WebSocket frames, against the calling bottle's resolved
config (see `_flow_ctx`). `request()` resolves and stashes the per-flow
(config, slug, env) at the upgrade, and every frame reuses it.
Outbound frames (from_client) are scanned for credential leakage; Outbound frames (from_client) are scanned for credential leakage;
inbound frames are scanned for prompt injection. On a block the inbound frames are scanned for prompt injection. On a block the
@@ -630,10 +650,8 @@ class EgressAddon:
""" """
if flow.websocket is None: # type: ignore[union-attr] if flow.websocket is None: # type: ignore[union-attr]
return return
# WebSocket DLP runs against the static config only (single-tenant); in config, slug, env = self._flow_ctx(flow)
# the consolidated gateway self.config has no routes, so this is inert route = match_route(config.routes, flow.request.pretty_host)
# until websocket routing is made source-IP-aware (a separate slice).
route = match_route(self.config.routes, flow.request.pretty_host)
if route is None: if route is None:
return return
message = flow.websocket.messages[-1] # type: ignore[union-attr] message = flow.websocket.messages[-1] # type: ignore[union-attr]
@@ -642,8 +660,8 @@ class EgressAddon:
# A WebSocket data frame is not an HTTP request line, so CRLF is # A WebSocket data frame is not an HTTP request line, so CRLF is
# not an injection vector here — scan only for credential leakage. # not an injection vector here — scan only for credential leakage.
result = scan_outbound( result = scan_outbound(
route, content, os.environ, route, content, env,
safe_tokens=self._safe_tokens_for(self._supervise_slug), crlf_text="", safe_tokens=self._safe_tokens_for(slug), crlf_text="",
) )
if result is not None and result.severity == "block": if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n") sys.stderr.write(f"egress DLP: {result.reason}\n")
+63 -29
View File
@@ -6,9 +6,9 @@ exercise the parse + decision functions without depending on the
`mitmproxy.http.HTTPFlow` API and is loaded inside the gateway `mitmproxy.http.HTTPFlow` API and is loaded inside the gateway
container. container.
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and Imports: stdlib + sibling package modules (`yaml_subset`,
ships flat into the gateway image alongside this file `egress_dlp_config`). Available in the gateway via the installed
see `Dockerfile.gateway`).""" `bot_bottle` package (see `Dockerfile.gateway`)."""
from __future__ import annotations from __future__ import annotations
@@ -16,26 +16,10 @@ import re
import typing import typing
from dataclasses import dataclass from dataclasses import dataclass
try:
from yaml_subset import YamlSubsetError, parse_yaml_subset # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset from .yaml_subset import YamlSubsetError, parse_yaml_subset
# DLP detector-config parsing lives in a sibling module (also flat-bundled # DLP detector-config parsing lives in a sibling module. Re-exported below
# into the gateway — see Dockerfile.gateway). Re-exported below so existing # so existing `from egress_addon_core import ON_MATCH_*` callers keep working.
# `from egress_addon_core import ON_MATCH_*` callers keep working.
try:
from egress_dlp_config import ( # type: ignore[import-not-found]
DEFAULT_OUTBOUND_ON_MATCH,
INBOUND_DETECTOR_NAMES,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
ON_MATCH_SUPERVISE,
OUTBOUND_DETECTOR_NAMES,
OUTBOUND_ON_MATCH_VALUES,
parse_dlp_block,
)
except ImportError: # pragma: no cover - host-side path
from .egress_dlp_config import ( from .egress_dlp_config import (
DEFAULT_OUTBOUND_ON_MATCH, DEFAULT_OUTBOUND_ON_MATCH,
INBOUND_DETECTOR_NAMES, INBOUND_DETECTOR_NAMES,
@@ -94,6 +78,7 @@ class Route:
inbound_detectors: tuple[str, ...] | None = None inbound_detectors: tuple[str, ...] | None = None
# "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES. # "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES.
outbound_on_match: str = "" outbound_on_match: str = ""
preserve_auth: bool = False
LOG_OFF = 0 # no logging LOG_OFF = 0 # no logging
@@ -105,6 +90,14 @@ LOG_FULL = 2 # log block/warn events + full request and response bodies
class Config: class Config:
routes: tuple[Route, ...] routes: tuple[Route, ...]
log: int = LOG_OFF log: int = LOG_OFF
# Why this Config is a deny-all, when it is one for a reason *other* than
# the bottle's own policy genuinely not listing the host. A deny-all is
# indistinguishable from "policy loaded, host not allowed" at the decision
# point — both are simply "no matching route" — so without this the
# operator sees `host X is not in the allowlist` and goes hunting for a
# missing route that was never the problem. Empty for a normally-parsed
# policy; `decide` prefers it over the allowlist wording when set.
deny_reason: str = ""
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -316,11 +309,18 @@ def _parse_one(idx: int, raw: object) -> Route:
idx, host, raw_dict, idx, host, raw_dict,
) )
preserve_auth_raw = raw_dict.get("preserve_auth", False)
if preserve_auth_raw is not True and preserve_auth_raw is not False:
raise ValueError(
f"{label} ({host}): 'preserve_auth' must be a boolean"
)
preserve_auth: bool = preserve_auth_raw
for k in raw_dict: for k in raw_dict:
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git"): if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git", "preserve_auth"):
raise ValueError( raise ValueError(
f"{label} ({host}): unknown key {k!r}; accepted keys " f"{label} ({host}): unknown key {k!r}; accepted keys "
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git'" f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git', 'preserve_auth'"
) )
return Route( return Route(
@@ -332,6 +332,7 @@ def _parse_one(idx: int, raw: object) -> Route:
outbound_detectors=outbound_detectors, outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors, inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match, outbound_on_match=outbound_on_match,
preserve_auth=preserve_auth,
) )
@@ -384,6 +385,8 @@ def route_to_yaml_dict(r: Route) -> dict[str, object]:
dlp["outbound_on_match"] = r.outbound_on_match dlp["outbound_on_match"] = r.outbound_on_match
if dlp: if dlp:
d["dlp"] = dlp d["dlp"] = dlp
if r.preserve_auth:
d["preserve_auth"] = True
return d return d
@@ -421,16 +424,40 @@ class PolicyResolverLike(typing.Protocol):
... ...
# Deny-all explanations. Each names the *actual* failure so an operator isn't
# sent looking for a missing egress route when the bottle never had a policy
# to begin with — the failure mode that made a bricked registration read like
# a misconfigured allowlist.
DENY_UNATTRIBUTED = (
"egress: this request was not attributed to any bottle, so no egress "
"policy applies and every host is denied. Either the bottle's registry "
"row is missing/ambiguous (torn down, or another bottle claimed its "
"source IP), or the request carried no matching identity token — check "
"that the caller's proxy URL includes it. This is not an allowlist problem."
)
DENY_UNPARSEABLE = (
"egress: this bottle's egress policy could not be parsed, so it is being "
"treated as deny-all. Fix the bottle's egress.routes; every host is denied "
"until it loads."
)
DENY_RESOLVER_ERROR = (
"egress: the orchestrator could not be reached to resolve this bottle's "
"egress policy, so every host is denied (fail-closed). Check that the "
"control plane is up; this is not an allowlist problem."
)
def _config_from_policy(policy: "str | None") -> "Config": def _config_from_policy(policy: "str | None") -> "Config":
"""Parse a resolved policy blob into a Config, fail-closed: None / empty / """Parse a resolved policy blob into a Config, fail-closed: None / empty /
unparseable all become a deny-all Config (no routes every request unparseable all become a deny-all Config (no routes every request
blocked).""" blocked). Each deny-all carries the reason it is one, so the block message
names the real fault instead of blaming the allowlist."""
if not policy: if not policy:
return Config(routes=()) # unattributed or empty → deny-all return Config(routes=(), deny_reason=DENY_UNATTRIBUTED)
try: try:
return load_config(policy) return load_config(policy)
except ValueError: except ValueError:
return Config(routes=()) # unparseable policy → deny return Config(routes=(), deny_reason=DENY_UNPARSEABLE)
def resolve_client_config( def resolve_client_config(
@@ -444,7 +471,7 @@ def resolve_client_config(
try: try:
policy = resolver.resolve(client_ip, identity_token) policy = resolver.resolve(client_ip, identity_token)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()) # orchestrator unreachable/errored → deny return Config(routes=(), deny_reason=DENY_RESOLVER_ERROR)
return _config_from_policy(policy) return _config_from_policy(policy)
@@ -473,7 +500,7 @@ def resolve_client_context(
client_ip, identity_token, client_ip, identity_token,
) )
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()), "", {} # orchestrator unreachable/errored → deny return Config(routes=(), deny_reason=DENY_RESOLVER_ERROR), "", {}
return _config_from_policy(policy), (bottle_id or ""), tokens return _config_from_policy(policy), (bottle_id or ""), tokens
@@ -588,12 +615,16 @@ def decide(
*, *,
request_method: str = "GET", request_method: str = "GET",
request_headers: typing.Mapping[str, str] | None = None, request_headers: typing.Mapping[str, str] | None = None,
deny_reason: str = "",
) -> Decision: ) -> Decision:
"""`deny_reason` is `Config.deny_reason`: when the deny-all came from a
missing/unparseable policy rather than the bottle's own allowlist, report
that instead of implying a route is merely absent."""
route = match_route(routes, request_host) route = match_route(routes, request_host)
if route is None: if route is None:
return Decision( return Decision(
action="block", action="block",
reason=( reason=deny_reason or (
f"egress: host {request_host!r} is not in the " f"egress: host {request_host!r} is not in the "
f"bottle's egress.routes allowlist. Declare a " f"bottle's egress.routes allowlist. Declare a "
f"route for it or remove the request." f"route for it or remove the request."
@@ -868,6 +899,9 @@ __all__ = [
"is_git_push_request", "is_git_push_request",
"is_git_fetch_request", "is_git_fetch_request",
"load_config", "load_config",
"DENY_UNATTRIBUTED",
"DENY_UNPARSEABLE",
"DENY_RESOLVER_ERROR",
"resolve_client_config", "resolve_client_config",
"resolve_client_context", "resolve_client_context",
"PolicyResolverLike", "PolicyResolverLike",
+14 -2
View File
@@ -15,11 +15,23 @@
# mitmproxy at it. The option REPLACES mitmproxy's default # mitmproxy at it. The option REPLACES mitmproxy's default
# trust store, so passing the upstream CA alone would break # trust store, so passing the upstream CA alone would break
# non-chained hosts. # non-chained hosts.
# * `-s /app/egress_addon.py` loads the addon that reads # * `-s /app/egress_addon.py` loads the addon that resolves each
# /etc/egress/routes.yaml. # request's policy from the orchestrator control plane by source
# IP (PRD 0070). There is no static routes file.
set -e set -e
# Fail closed on a missing policy source. The addon itself raises at
# load when BOT_BOTTLE_ORCHESTRATOR_URL is unset (so mitmdump exits via
# its errorcheck addon), but that leaves the fail-closed guarantee at the
# mercy of a mitmproxy version keeping that behavior. Refuse here too, so
# a misconfigured gateway can never come up as a bare TLS-bumping open
# proxy with no policy — independent of mitmproxy's startup-error handling.
if [ -z "$BOT_BOTTLE_ORCHESTRATOR_URL" ]; then
echo "egress: BOT_BOTTLE_ORCHESTRATOR_URL is required (no static routes fallback)" >&2
exit 1
fi
# Pin mitmproxy's config dir to the bind-mount location of its CA # Pin mitmproxy's config dir to the bind-mount location of its CA
# regardless of which user mitmdump runs as. In the legacy # regardless of which user mitmdump runs as. In the legacy
# four-daemon setup (Dockerfile.egress, USER mitmproxy) this # four-daemon setup (Dockerfile.egress, USER mitmproxy) this
+37 -17
View File
@@ -61,6 +61,11 @@ class _DaemonSpec:
_EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",) _EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",)
_READY_GATED_DAEMONS: tuple[str, ...] = ("git-gate", "git-http") _READY_GATED_DAEMONS: tuple[str, ...] = ("git-gate", "git-http")
# Daemons that must be requested explicitly via BOT_BOTTLE_GATEWAY_DAEMONS
# and are NOT started in the default (env-var-unset) case. The orchestrator
# only runs in the combined infra container, never in a standalone gateway.
_OPT_IN_DAEMONS: frozenset[str] = frozenset({"orchestrator"})
def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]: def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]:
"""Egress sees the full bundle env. Everyone else gets a copy """Egress sees the full bundle env. Everyone else gets a copy
@@ -75,11 +80,18 @@ def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]:
} }
# The orchestrator is listed first so it starts before the gateway daemons,
# giving the control plane a head start to accept /resolve calls. The gateway
# daemons tolerate early /resolve failures and retry per-request.
_DAEMONS: tuple[_DaemonSpec, ...] = ( _DAEMONS: tuple[_DaemonSpec, ...] = (
_DaemonSpec("orchestrator", (
"python3", "-m", "bot_bottle.orchestrator",
"--host", "0.0.0.0", "--port", "8099", "--broker", "stub",
)),
_DaemonSpec("egress", ("/bin/sh", "/app/egress-entrypoint.sh")), _DaemonSpec("egress", ("/bin/sh", "/app/egress-entrypoint.sh")),
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")), _DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
_DaemonSpec("git-http", ("python3", "/app/git_http_backend.py")), _DaemonSpec("git-http", ("python3", "-m", "bot_bottle.git_http_backend")),
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")), _DaemonSpec("supervise", ("python3", "-m", "bot_bottle.supervise_server")),
) )
@@ -103,18 +115,20 @@ def _selected_daemons(
env: dict[str, str], env: dict[str, str],
all_daemons: Sequence[_DaemonSpec] | None = None, all_daemons: Sequence[_DaemonSpec] | None = None,
) -> tuple[_DaemonSpec, ...]: ) -> tuple[_DaemonSpec, ...]:
"""Filter the daemon set by the BOT_BOTTLE_GATEWAY_DAEMONS env """Filter the daemon set by the BOT_BOTTLE_GATEWAY_DAEMONS env var.
var. Unknown names in the list are ignored the caller is the
source of truth for which daemons are wired.
`all_daemons` defaults to `_DAEMONS` resolved at call time (not When the var is unset/empty, return all non-opt-in daemons (the
at definition time), so tests can monkey-patch the module-level standard gateway subset). Opt-in daemons (e.g. `orchestrator`) only
`_DAEMONS` and have the new value take effect.""" run when explicitly named they never start in a plain gateway
container that doesn't set the env var. Unknown names are ignored.
`all_daemons` defaults to `_DAEMONS` resolved at call time (not at
definition time), so tests can pass a custom list."""
if all_daemons is None: if all_daemons is None:
all_daemons = _DAEMONS all_daemons = _DAEMONS
raw = env.get("BOT_BOTTLE_GATEWAY_DAEMONS", "").strip() raw = env.get("BOT_BOTTLE_GATEWAY_DAEMONS", "").strip()
if not raw: if not raw:
return tuple(all_daemons) return tuple(d for d in all_daemons if d.name not in _OPT_IN_DAEMONS)
wanted = {n.strip() for n in raw.split(",") if n.strip()} wanted = {n.strip() for n in raw.split(",") if n.strip()}
return tuple(d for d in all_daemons if d.name in wanted) return tuple(d for d in all_daemons if d.name in wanted)
@@ -136,7 +150,7 @@ def _pump(name: str, stream: IO[bytes]) -> None:
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]: def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
env = _env_for_daemon(spec.name, dict(os.environ)) env = _env_for_daemon(spec.name, dict(os.environ))
proc = subprocess.Popen( proc = subprocess.Popen( # pylint: disable=consider-using-with
_argv_for_daemon(spec.name, spec.argv, env), _argv_for_daemon(spec.name, spec.argv, env),
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, stderr=subprocess.STDOUT,
@@ -183,6 +197,14 @@ class _Supervisor:
except ProcessLookupError: except ProcessLookupError:
pass pass
def _sigkill_all(self) -> None:
for _, p in self.procs:
if p.poll() is None:
try:
p.kill()
except ProcessLookupError:
pass
def request_restart(self, daemon_name: str) -> bool: def request_restart(self, daemon_name: str) -> bool:
"""Queue a daemon restart for the main loop to process. """Queue a daemon restart for the main loop to process.
@@ -235,12 +257,7 @@ class _Supervisor:
f"grace ({_GRACE_SECONDS:.0f}s) elapsed; SIGKILL on " f"grace ({_GRACE_SECONDS:.0f}s) elapsed; SIGKILL on "
f"{', '.join(still_running)}" f"{', '.join(still_running)}"
) )
for _, p in self.procs: self._sigkill_all()
if p.poll() is None:
try:
p.kill()
except ProcessLookupError:
pass
done = all(p.poll() is not None for _, p in self.procs) done = all(p.poll() is not None for _, p in self.procs)
if done: if done:
@@ -361,7 +378,10 @@ def main(argv: Sequence[str] | None = None) -> int:
# --signal HUP <bundle>` after writing routes.yaml. The kernel # --signal HUP <bundle>` after writing routes.yaml. The kernel
# delivers SIGHUP to PID 1 (this supervisor); forward it to # delivers SIGHUP to PID 1 (this supervisor); forward it to
# mitmdump so it reloads its addon. # mitmdump so it reloads its addon.
signal.signal(signal.SIGHUP, lambda *_: sup.forward_signal(signal.SIGHUP, "egress")) # type: ignore signal.signal(
signal.SIGHUP,
lambda *_: sup.forward_signal(signal.SIGHUP, "egress"), # type: ignore[misc]
)
while not sup.tick(): while not sup.tick():
time.sleep(_POLL_INTERVAL) time.sleep(_POLL_INTERVAL)
+4 -2
View File
@@ -112,8 +112,10 @@ class GitGate(ABC):
access_hook = stage_dir / "git_gate_access_hook.sh" access_hook = stage_dir / "git_gate_access_hook.sh"
access_hook.write_text(git_gate_render_access_hook()) access_hook.write_text(git_gate_render_access_hook())
# 0o700 (not 0o600): git daemon execs --access-hook directly, # 0o700 (not 0o600): git daemon execs --access-hook directly,
# not via `sh`, so the script needs the x bit. docker cp # not via `sh`, so the script needs the x bit. The gateway copy
# preserves source mode into the container. # does not necessarily preserve this mode (`docker cp` does, the
# Apple `container cp` does not), so provision_git_gate re-applies
# +x on the gateway side — see backend/docker/gateway_provision.py.
access_hook.chmod(0o700) access_hook.chmod(0o700)
upstreams_with_files: list[GitGateUpstream] = [] upstreams_with_files: list[GitGateUpstream] = []
for u in upstreams: for u in upstreams:
+2 -1
View File
@@ -13,6 +13,7 @@ import dataclasses
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from .bottle_state import globalize_slug
from .errors import MissingEnvVarError from .errors import MissingEnvVarError
from .log import info from .log import info
from .manifest import ManifestBottle, ManifestGitEntry from .manifest import ManifestBottle, ManifestGitEntry
@@ -46,7 +47,7 @@ def _provision_dynamic_key(
owner_repo = entry.UpstreamPath owner_repo = entry.UpstreamPath
if owner_repo.endswith(".git"): if owner_repo.endswith(".git"):
owner_repo = owner_repo[:-4] owner_repo = owner_repo[:-4]
title = f"bot-bottle:{slug}:{entry.Name}" title = f"bot-bottle:{globalize_slug(slug)}:{entry.Name}"
info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]") info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]")
key_id, private_key_bytes = provisioner.create(owner_repo, title) key_id, private_key_bytes = provisioner.create(owner_repo, title)
+18 -18
View File
@@ -14,18 +14,12 @@ import shlex
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from .constants import GIT_GATE_TIMEOUT_SECS, IDENTITY_HEADER
from .manifest import ManifestBottle, ManifestGitEntry from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the gateway. The # Short network alias for git-gate inside the gateway. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name. # agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate" GIT_GATE_HOSTNAME = "git-gate"
# App-layer identity token header the agent's git sends to git-http and the
# gateway validates (mirrors egress_addon / git_http_backend IDENTITY_HEADER).
IDENTITY_HEADER = "x-bot-bottle-identity"
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
# git daemon (--timeout/--init-timeout), the access-hook subprocess in
# git_http_backend, and the git http-backend CGI subprocess.
GIT_GATE_TIMEOUT_SECS = 15
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -425,18 +419,24 @@ PY
while IFS=' ' read -r old new ref; do while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue [ -z "$ref" ] && continue
[ "$new" = "$zero" ] && continue [ "$new" = "$zero" ] && continue
if [ "$old" = "$zero" ]; then # Scan only the commits this push introduces — those reachable from
# New ref: scan only the commits this push introduces — those # $new but not from any ref the gate already has. Everything already
# reachable from $new but not from any ref the gate already has. # on the gate arrived via upstream mirror-fetch or a previously
# Everything already on the gate arrived via upstream mirror-fetch # gitleaks-scanned push, so it's already-upstream or already-scanned;
# or a previously gitleaks-scanned push, so it's already-upstream # re-scanning it only resurfaces historical fixture findings.
# or already-scanned; re-scanning it (the old `$new` full-ancestry #
# range) only resurfaces historical findings and blocks every new # Applies to both new refs and updates. The old existing-branch range
# branch. See PRD 0028 / issue #106. # `$old..$new` walks commits reachable from the new tip but not the
# *old branch tip*: on a rebase/force-push onto a freshly-advanced
# main that pulls in all of main's new history (incl. the deliberate
# sandbox-escape gitleaks fixtures), blocking the push. `--not --all`
# excludes anything already on the gate regardless of ancestry, so it
# is also correct for non-fast-forward pushes (a rebase can skip
# commits off the direct path). Security-equivalent per PRD 0028's
# analysis: the bare repo's refs come only from trusted upstream
# mirror-fetch or gitleaks-gated pushes.
# See PRD 0028 (open question) / issues #106, #346.
log_opts="$new --not --all" log_opts="$new --not --all"
else
log_opts="$old..$new"
fi
echo "git-gate: gitleaks scanning $ref ($log_opts)" >&2 echo "git-gate: gitleaks scanning $ref ($log_opts)" >&2
if ! gitleaks git --log-opts="$log_opts" --no-banner --redact 1>&2; then if ! gitleaks git --log-opts="$log_opts" --no-banner --redact 1>&2; then
echo "git-gate: gitleaks rejected push to $ref" >&2 echo "git-gate: gitleaks rejected push to $ref" >&2
+52 -70
View File
@@ -7,14 +7,13 @@ wrapper serves the same `/git/*.git` bare repos through
`git http-backend`, so pre-receive and upstream forwarding remain the `git http-backend`, so pre-receive and upstream forwarding remain the
git-gate enforcement point. git-gate enforcement point.
Consolidated (PRD 0070): when `BOT_BOTTLE_ORCHESTRATOR_URL` is set, one One shared gateway serves every bottle (PRD 0070): each request is served
shared gateway serves every bottle, and each request is served from the from the calling bottle's repo namespace (`<root>/<bottle_id>`), attributed
calling bottle's repo namespace (`<root>/<bottle_id>`), attributed from from the unspoofable source IP via the orchestrator. Per-repo credentials +
the unspoofable source IP via the orchestrator. Per-repo credentials +
hooks scope by repo directory, so isolating the *root* per bottle isolates hooks scope by repo directory, so isolating the *root* per bottle isolates
its creds too. Unattributed clients fail closed (404). Unset the legacy its creds too. Unattributed clients and a missing/unreachable orchestrator
per-bottle single-tenant flat root, unchanged a transitional path that fail closed (404). `BOT_BOTTLE_ORCHESTRATOR_URL` is mandatory: there is no
gets stripped out once every backend runs the consolidated gateway. single-tenant flat-root fallback.
""" """
from __future__ import annotations from __future__ import annotations
@@ -27,36 +26,19 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path from pathlib import Path
from urllib.parse import urlsplit from urllib.parse import urlsplit
# policy_resolver ships flat alongside this file in the gateway from bot_bottle.constants import GIT_GATE_TIMEOUT_SECS, IDENTITY_HEADER
# image (see Dockerfile.gateway); the bot_bottle.* fallback is the
# host-side / test path. Mirrors egress_addon's import shape.
try:
from policy_resolver import ( # type: ignore[import-not-found]
PolicyResolveError,
PolicyResolver,
)
except ImportError: # pragma: no cover - host-side path
from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver
DEFAULT_PORT = 9420 DEFAULT_PORT = 9420
# Consolidated (multi-tenant) mode: when this points at the per-host # The per-host orchestrator control plane the backend attributes each request
# orchestrator's control plane, the backend serves each request from the # to, serving from the *calling* bottle's repo namespace selected by source IP.
# *calling* bottle's repo namespace, selected by source IP, instead of a # Mandatory — the same env the egress addon requires; there is no single flat
# single flat repo root. Unset → legacy per-bottle single-tenant mode # repo-root fallback.
# (unchanged). Same env the egress addon reads, so one orchestrator setting
# flips the whole shared gateway multi-tenant.
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL" ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
# App-layer identity token (defense-in-depth over the source-IP invariant); # The base under which each bottle's `<bottle_id>` repo namespace is nested.
# the agent injects it, the backend reads it for attribution and never
# forwards it to `git http-backend`. Mirrors egress_addon.IDENTITY_HEADER
# (duplicated, not imported: egress_addon pulls in mitmproxy).
IDENTITY_HEADER = "x-bot-bottle-identity"
# Default flat repo root (single-tenant, and the base under which
# consolidated mode nests each sandbox's namespace).
DEFAULT_REPO_ROOT = "/git" DEFAULT_REPO_ROOT = "/git"
@@ -71,25 +53,16 @@ class ResolverLike(typing.Protocol):
def resolve_sandbox_root( def resolve_sandbox_root(
resolver: "ResolverLike | None", resolver: "ResolverLike",
base_root: Path, base_root: Path,
source_ip: str, source_ip: str,
identity_token: str = "", identity_token: str = "",
) -> Path | None: ) -> Path | None:
"""The per-sandbox repo root to serve this request from, or None to """The per-sandbox repo root to serve this request from — `base_root/
deny (404). <bottle_id>`, where the sandbox is attributed from the source IP via the
orchestrator or None to deny (404). Fail-closed: an unattributed client, a
Single-tenant (`resolver is None`): the flat `base_root`, unchanged. resolver error, or a namespace that would escape `base_root` all deny, so one
NOTE: this legacy per-bottle single-tenant path is transitional it sandbox can never reach another's repos."""
will be stripped out once every backend runs the consolidated gateway
(PRD 0070), leaving only the source-IP-attributed path below.
Consolidated: `base_root/<bottle_id>`, where the sandbox is attributed
from the source IP via the orchestrator. Fail-closed an unattributed
client, a resolver error, or a namespace that would escape `base_root`
all deny, so one sandbox can never reach another's repos."""
if resolver is None:
return base_root
try: try:
bottle_id = resolver.resolve_bottle_id(source_ip, identity_token) bottle_id = resolver.resolve_bottle_id(source_ip, identity_token)
except PolicyResolveError: except PolicyResolveError:
@@ -102,13 +75,6 @@ def resolve_sandbox_root(
return None # bottle_id tried to escape the root → deny return None # bottle_id tried to escape the root → deny
return namespace return namespace
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
# imported: this module ships as a flat top-level sibling in the gateway
# bundle image (see Dockerfile.gateway), not as part of the bot_bottle
# package, so `bot_bottle.git_gate` and its dependency chain aren't
# available at runtime.
GIT_GATE_TIMEOUT_SECS = 15
# Bound memory use while still allowing ordinary git push packfiles. # Bound memory use while still allowing ordinary git push packfiles.
MAX_BODY_BYTES = 100 * 1024 * 1024 MAX_BODY_BYTES = 100 * 1024 * 1024
@@ -123,12 +89,13 @@ class GitHttpHandler(BaseHTTPRequestHandler):
self._run_backend() self._run_backend()
def _sandbox_root(self) -> Path | None: def _sandbox_root(self) -> Path | None:
"""This request's per-sandbox repo root, or None to deny. Single-tenant """This request's per-sandbox repo root (the calling bottle's source-IP-
unless the server was started with a resolver (consolidated mode), in selected `<base>/<bottle_id>` namespace), or None to deny. `GIT_PROJECT_
which case the root is the calling sandbox's source-IP-selected ROOT` keeps git's own env-var name."""
namespace. `GIT_PROJECT_ROOT` keeps git's own env-var name."""
base = Path(os.environ.get("GIT_PROJECT_ROOT", DEFAULT_REPO_ROOT)) base = Path(os.environ.get("GIT_PROJECT_ROOT", DEFAULT_REPO_ROOT))
resolver = getattr(self.server, "policy_resolver", None) resolver = getattr(self.server, "policy_resolver", None)
if resolver is None:
return None # server started without a resolver (misconfig) → deny
token = self.headers.get(IDENTITY_HEADER, "") token = self.headers.get(IDENTITY_HEADER, "")
return resolve_sandbox_root(resolver, base, self.client_address[0], token) return resolve_sandbox_root(resolver, base, self.client_address[0], token)
@@ -148,12 +115,24 @@ class GitHttpHandler(BaseHTTPRequestHandler):
"GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook", "GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook",
) )
peer = self.client_address[0] peer = self.client_address[0]
try:
hook = subprocess.run( hook = subprocess.run(
[hook_path, "upload-pack", str(repo_dir), peer, peer], [hook_path, "upload-pack", str(repo_dir), peer, peer],
capture_output=True, capture_output=True,
check=False, check=False,
timeout=GIT_GATE_TIMEOUT_SECS, timeout=GIT_GATE_TIMEOUT_SECS,
) )
except (OSError, subprocess.SubprocessError) as exc:
# The access-hook couldn't be run (missing, not executable,
# timed out, …). Fail closed with a real HTTP error rather
# than letting the exception kill the handler thread — an
# unhandled exception closes the socket with no response, which
# the client sees as an opaque "empty reply from server".
self.log_message(
"access-hook could not run for %s: %s", parsed.path, exc,
)
self.send_error(503, "git-gate access-hook unavailable")
return
if hook.returncode != 0: if hook.returncode != 0:
detail = (hook.stderr or hook.stdout).decode( detail = (hook.stderr or hook.stdout).decode(
"utf-8", errors="replace", "utf-8", errors="replace",
@@ -188,13 +167,10 @@ class GitHttpHandler(BaseHTTPRequestHandler):
"SERVER_PORT": str(self.server.server_port), # type: ignore "SERVER_PORT": str(self.server.server_port), # type: ignore
"SERVER_PROTOCOL": self.request_version, "SERVER_PROTOCOL": self.request_version,
}) })
# Consolidated mode: attribute the gitleaks-allow supervise proposal # Attribute the gitleaks-allow supervise proposal (written by
# (written by receive-pack's pre-receive hook, a child of the CGI we # receive-pack's pre-receive hook, a child of the CGI we spawn below) to
# spawn below) to the calling bottle. The namespaced root is # the calling bottle. The namespaced root is `<base>/<bottle_id>`, so its
# `<base>/<bottle_id>`, so its final component is the bottle id — the # final component is the bottle id — the same per-bottle key egress uses.
# same per-bottle key egress uses. Single-tenant leaves the hook's
# container-stamped SUPERVISE_BOTTLE_SLUG untouched.
if getattr(self.server, "policy_resolver", None) is not None:
env["SUPERVISE_BOTTLE_SLUG"] = sandbox_root.name env["SUPERVISE_BOTTLE_SLUG"] = sandbox_root.name
for header, variable in ( for header, variable in (
("accept", "HTTP_ACCEPT"), ("accept", "HTTP_ACCEPT"),
@@ -285,14 +261,20 @@ class GitHttpHandler(BaseHTTPRequestHandler):
def main() -> int: def main() -> int:
port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT))) port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT)))
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip() orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
# Consolidated mode: resolve each request's sandbox namespace by source if not orch_url:
# IP. Absent → single-tenant (flat repo root); behaviour unchanged. # Resolver-only: without an orchestrator the backend can't attribute a
resolver = PolicyResolver(orch_url) if orch_url else None # request to a bottle namespace, so it must not serve (fail-closed).
server.policy_resolver = resolver # type: ignore[attr-defined] sys.stderr.write(
mode = "multi-tenant" if orch_url else "single-tenant" f"git-http: {ORCHESTRATOR_URL_ENV} is required "
sys.stdout.write(f"git-http listening on 0.0.0.0:{port} ({mode})\n") "(no single-tenant flat-root fallback)\n"
)
return 1
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
# Resolve each request's sandbox namespace by source IP against the
# orchestrator control plane.
server.policy_resolver = PolicyResolver(orch_url) # type: ignore[attr-defined]
sys.stdout.write(f"git-http listening on 0.0.0.0:{port} (multi-tenant)\n")
sys.stdout.flush() sys.stdout.flush()
server.serve_forever() server.serve_forever()
return 0 return 0
+42
View File
@@ -0,0 +1,42 @@
"""Shared helpers for cached-image quickstart stale checks."""
from __future__ import annotations
from datetime import datetime, timezone
from pathlib import Path
try:
from .config_store import ConfigStore
except ImportError:
from config_store import ConfigStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
class StaleImageError(Exception):
"""Raised when a cached image or artifact exceeds the configured staleness
threshold. Callers can catch this to prompt interactively; headless paths
let it propagate as a fatal error."""
def check_stale(label: str, created_at: datetime) -> None:
"""Raise StaleImageError if `created_at` is older than the configured
stale-warning threshold. Negative threshold disables the check."""
threshold_days = ConfigStore().cached_image_stale_warning_days()
if threshold_days < 0:
return
now = datetime.now(timezone.utc)
created = created_at.astimezone(timezone.utc)
age = now - created
if age.total_seconds() <= threshold_days * 86400:
return
raise StaleImageError(
f"cached {label} is {age.days} day(s) old; "
"quickstart does not verify it matches the current Dockerfile/context"
)
def check_stale_path(label: str, path: Path) -> None:
"""Raise StaleImageError if `path`'s mtime exceeds the staleness threshold."""
check_stale(label, datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc))
__all__ = ["StaleImageError", "check_stale", "check_stale_path"]
+10 -4
View File
@@ -25,8 +25,9 @@ class ManifestAgentProvider:
header, and sets a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent header, and sets a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent
so the Claude Code CLI starts. so the Claude Code CLI starts.
`forward_host_credentials` forwards the host Codex auth token into `forward_host_credentials` forwards the host provider auth token into
the egress daemon (Codex only). the egress sidecar (Codex and Claude). For Codex this reads
`~/.codex/auth.json`; for Claude it reads `~/.claude/.credentials.json`.
""" """
template: str = "claude" template: str = "claude"
@@ -92,10 +93,15 @@ class ManifestAgentProvider:
f"is only supported for built-in templates " f"is only supported for built-in templates "
f"({', '.join(sorted(PROVIDER_TEMPLATES))})" f"({', '.join(sorted(PROVIDER_TEMPLATES))})"
) )
if forward_host_credentials and template != "codex": if forward_host_credentials and template not in {"codex", "claude"}:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' agent_provider.forward_host_credentials " f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
"is currently only supported for template 'codex'" "is only supported for templates 'codex' and 'claude'"
)
if forward_host_credentials and auth_token:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
"and auth_token both set; use one or the other"
) )
settings = _parse_provider_settings(bottle_name, template, d.get("settings")) settings = _parse_provider_settings(bottle_name, template, d.get("settings"))
return cls( return cls(
+15 -2
View File
@@ -71,6 +71,7 @@ class ManifestEgressRoute:
OutboundDetectors: tuple[str, ...] | None = None OutboundDetectors: tuple[str, ...] | None = None
InboundDetectors: tuple[str, ...] | None = None InboundDetectors: tuple[str, ...] | None = None
OutboundOnMatch: str = "" OutboundOnMatch: str = ""
PreserveAuth: bool = False
@classmethod @classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute": def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
@@ -190,11 +191,22 @@ class ManifestEgressRoute:
f"only 'fetch' is accepted" f"only 'fetch' is accepted"
) )
# --- preserve_auth ---
preserve_auth = False
if "preserve_auth" in d:
raw_preserve_auth = d.get("preserve_auth")
if not isinstance(raw_preserve_auth, bool):
raise ManifestError(
f"{label} preserve_auth must be a boolean "
f"(was {type(raw_preserve_auth).__name__})"
)
preserve_auth = raw_preserve_auth
for k in d: for k in d:
if k not in ("host", "matches", "auth", "role", "dlp", "git"): if k not in ("host", "matches", "auth", "role", "dlp", "git", "preserve_auth"):
raise ManifestError( raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are " f"{label} has unknown key {k!r}; accepted keys are "
f"'host', 'matches', 'auth', 'role', 'dlp', 'git'" f"'host', 'matches', 'auth', 'role', 'dlp', 'git', 'preserve_auth'"
) )
return cls( return cls(
@@ -207,6 +219,7 @@ class ManifestEgressRoute:
OutboundDetectors=outbound_detectors, OutboundDetectors=outbound_detectors,
InboundDetectors=inbound_detectors, InboundDetectors=inbound_detectors,
OutboundOnMatch=outbound_on_match, OutboundOnMatch=outbound_on_match,
PreserveAuth=preserve_auth,
) )
+52 -2
View File
@@ -15,11 +15,25 @@ from __future__ import annotations
import json import json
import urllib.error import urllib.error
import urllib.request import urllib.request
from collections.abc import Iterable
from dataclasses import dataclass from dataclasses import dataclass
from ..paths import host_control_plane_token
from .control_plane import CONTROL_AUTH_HEADER
DEFAULT_TIMEOUT_SECONDS = 5.0 DEFAULT_TIMEOUT_SECONDS = 5.0
def _host_auth_token() -> str:
"""The per-host control-plane secret, or "" if it can't be read. "" means
'send no auth header' correct against an open (unconfigured) control
plane, and harmlessly rejected by a secured one."""
try:
return host_control_plane_token()
except OSError:
return ""
class OrchestratorClientError(RuntimeError): class OrchestratorClientError(RuntimeError):
"""A control-plane call failed (unreachable, or an unexpected status).""" """A control-plane call failed (unreachable, or an unexpected status)."""
@@ -34,11 +48,24 @@ class RegisteredBottle:
class OrchestratorClient: class OrchestratorClient:
"""Trusted host-side client for the orchestrator control plane.""" """Trusted host-side client for the orchestrator control plane.
def __init__(self, base_url: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS) -> None: Presents the per-host control-plane secret on every call (the header the
control plane requires on all routes but `/health`). The secret is read
from the host file this client only ever runs host-side (CLI, launcher,
discovery), so it can read what an agent can't. `auth_token` is overridable
for tests; the default reads the host file, minting it on first use."""
def __init__(
self,
base_url: str,
*,
timeout: float = DEFAULT_TIMEOUT_SECONDS,
auth_token: str | None = None,
) -> None:
self._base = base_url.rstrip("/") self._base = base_url.rstrip("/")
self._timeout = timeout self._timeout = timeout
self._auth_token = auth_token if auth_token is not None else _host_auth_token()
def _request( def _request(
self, method: str, path: str, body: dict[str, object] | None = None, self, method: str, path: str, body: dict[str, object] | None = None,
@@ -49,6 +76,8 @@ class OrchestratorClient:
callers can treat 404 as a meaningful "no such bottle".""" callers can treat 404 as a meaningful "no such bottle"."""
data = json.dumps(body).encode() if body is not None else None data = json.dumps(body).encode() if body is not None else None
headers = {"Content-Type": "application/json"} if data is not None else {} headers = {"Content-Type": "application/json"} if data is not None else {}
if self._auth_token:
headers[CONTROL_AUTH_HEADER] = self._auth_token
req = urllib.request.Request( req = urllib.request.Request(
f"{self._base}{path}", data=data, method=method, headers=headers, f"{self._base}{path}", data=data, method=method, headers=headers,
) )
@@ -119,6 +148,20 @@ class OrchestratorClient:
raise OrchestratorClientError(f"teardown {bottle_id}: HTTP {status}") raise OrchestratorClientError(f"teardown {bottle_id}: HTTP {status}")
return True return True
def reconcile(
self, live_source_ips: Iterable[str], *, grace_seconds: float | None = None,
) -> list[str]:
"""Drop registry rows for bottles that are no longer running
(`POST /reconcile`), returning the reaped bottle ids. `live_source_ips`
is the caller's enumeration of its live bottles — the orchestrator
can't see the backend from inside the infra container."""
body: dict[str, object] = {"live_source_ips": list(live_source_ips)}
if grace_seconds is not None:
body["grace_seconds"] = grace_seconds
payload = self._ok("POST", "/reconcile", body)
reaped = payload.get("reaped")
return [r for r in reaped if isinstance(r, str)] if isinstance(reaped, list) else []
def set_policy(self, bottle_id: str, policy: str) -> bool: def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Live-reload a bottle's policy (`PUT /bottles/<id>/policy`). False on """Live-reload a bottle's policy (`PUT /bottles/<id>/policy`). False on
404 (unknown bottle).""" 404 (unknown bottle)."""
@@ -186,6 +229,13 @@ def discover_orchestrator_url(*, timeout: float = 2.0) -> str:
f"http://{netpool.orch_slot().guest_ip}:{CONTROL_PLANE_PORT}") f"http://{netpool.orch_slot().guest_ip}:{CONTROL_PLANE_PORT}")
except Exception: # noqa: BLE001 — backend optional / not firecracker except Exception: # noqa: BLE001 — backend optional / not firecracker
pass pass
try: # macOS: infra container control plane on its host-only address
from ..backend.macos_container.infra import probe_control_plane_url
url = probe_control_plane_url()
if url:
candidates.append(url)
except Exception: # noqa: BLE001 — backend optional / not macOS
pass
for url in candidates: for url in candidates:
if OrchestratorClient(url, timeout=timeout).health(): if OrchestratorClient(url, timeout=timeout).health():
return url return url
+107
View File
@@ -0,0 +1,107 @@
"""Per-host orchestrator configuration store (settings in bot-bottle.db).
Co-tenants the shared `bot-bottle.db` via the `DbStore` framework. Settings
are readable by the host launch path directly (no HTTP round-trip to the
orchestrator), so they take effect even before the orchestrator is reachable.
"""
from __future__ import annotations
import os
import sqlite3
from pathlib import Path
from ..db_store import DbStore
from ..migrations import TableMigrations
from ..paths import host_db_path
TEARDOWN_TIMEOUT_ENV = "BOT_BOTTLE_ORCHESTRATOR_TEARDOWN_TIMEOUT_SECONDS"
DEFAULT_TEARDOWN_TIMEOUT_SECONDS = 30.0
_MIGRATIONS = TableMigrations(
"orchestrator_config",
[
"""
CREATE TABLE IF NOT EXISTS orchestrator_config (
id INTEGER PRIMARY KEY CHECK (id = 1),
teardown_timeout_seconds REAL
)
""",
],
)
class OrchestratorConfigStore(DbStore):
"""Orchestrator settings in the shared host DB."""
def __init__(self, db_path: Path | None = None) -> None:
super().__init__(db_path or host_db_path(), _MIGRATIONS)
def _connect(self) -> sqlite3.Connection:
conn = super()._connect()
conn.execute("PRAGMA busy_timeout=5000")
return conn
def get_teardown_timeout_seconds(self) -> float | None:
"""Return the configured teardown timeout, or None if not set."""
try:
with self._connection() as conn:
row = conn.execute(
"SELECT teardown_timeout_seconds FROM orchestrator_config WHERE id = 1"
).fetchone()
except sqlite3.OperationalError:
return None
return row["teardown_timeout_seconds"] if row else None
def set_teardown_timeout_seconds(self, value: float) -> None:
"""Persist the teardown timeout."""
with self._connection() as conn:
conn.execute(
"INSERT OR REPLACE INTO orchestrator_config"
" (id, teardown_timeout_seconds) VALUES (1, ?)",
(value,),
)
self._chmod()
def delete_teardown_timeout_seconds(self) -> bool:
"""Clear the stored teardown timeout. Returns True if a value existed."""
with self._connection() as conn:
cur = conn.execute(
"UPDATE orchestrator_config SET teardown_timeout_seconds = NULL"
" WHERE id = 1 AND teardown_timeout_seconds IS NOT NULL"
)
return cur.rowcount > 0
def resolve_teardown_timeout(db_path: Path | None = None) -> float:
"""Return the teardown timeout to use, in priority order:
1. ``BOT_BOTTLE_ORCHESTRATOR_TEARDOWN_TIMEOUT_SECONDS`` env var
2. ``teardown_timeout_seconds`` in the orchestrator config DB
3. ``DEFAULT_TEARDOWN_TIMEOUT_SECONDS`` (30 s)
"""
raw = os.environ.get(TEARDOWN_TIMEOUT_ENV, "").strip()
if raw:
try:
value = float(raw)
if value > 0:
return value
except ValueError:
pass
store = OrchestratorConfigStore(db_path)
if not store.is_migrated():
store.migrate()
db_value = store.get_teardown_timeout_seconds()
if db_value is not None and db_value > 0:
return db_value
return DEFAULT_TEARDOWN_TIMEOUT_SECONDS
__all__ = [
"OrchestratorConfigStore",
"resolve_teardown_timeout",
"TEARDOWN_TIMEOUT_ENV",
"DEFAULT_TEARDOWN_TIMEOUT_SECONDS",
]
+83 -5
View File
@@ -13,6 +13,9 @@ vsock / unix-socket portability caveats):
PUT /bottles/<bottle_id>/policy -> 200 {"updated": true} | 404 (live reload) PUT /bottles/<bottle_id>/policy -> 200 {"updated": true} | 404 (live reload)
body: {"policy"} body: {"policy"}
DELETE /bottles/<bottle_id> -> 200 {"torn_down": true} | 404 (teardown) DELETE /bottles/<bottle_id> -> 200 {"torn_down": true} | 404 (teardown)
POST /reconcile -> 200 {"reaped": [bottle_id, ...]}
body: {"live_source_ips": [...],
["grace_seconds"]}
POST /attribute -> 200 {"bottle_id"} | 403 POST /attribute -> 200 {"bottle_id"} | 403
POST /resolve -> 200 {"bottle_id","policy"} | 403 POST /resolve -> 200 {"bottle_id","policy"} | 403
body: {"source_ip","identity_token"} body: {"source_ip","identity_token"}
@@ -34,6 +37,7 @@ returned only once, to the caller that launches the bottle.
from __future__ import annotations from __future__ import annotations
import hmac
import http.server import http.server
import json import json
import os import os
@@ -42,11 +46,20 @@ import sys
import typing import typing
from urllib.parse import urlsplit from urllib.parse import urlsplit
from ..paths import CONTROL_PLANE_TOKEN_ENV
from .service import Orchestrator from .service import Orchestrator
# JSON body payload type (parsed request / rendered response). # JSON body payload type (parsed request / rendered response).
Json = dict[str, object] Json = dict[str, object]
# The request header carrying the per-host control-plane secret. Every route
# except `GET /health` requires it (see `dispatch`). The trusted callers hold
# the secret (the gateway's PolicyResolver, the host CLI's OrchestratorClient);
# an agent that can merely *reach* the port cannot present it, so it can't
# enumerate bottles, rewrite policy, read injected upstream tokens, or approve
# its own supervise proposals.
CONTROL_AUTH_HEADER = "x-bot-bottle-control-auth"
def _parse_json_object(body: bytes) -> Json: def _parse_json_object(body: bytes) -> Json:
"""Parse a JSON object body. Raises ValueError for non-objects / bad JSON.""" """Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
@@ -59,15 +72,29 @@ def _parse_json_object(body: bytes) -> Json:
def dispatch( # pylint: disable=too-many-return-statements,too-many-branches def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
orch: Orchestrator, method: str, path: str, body: bytes orch: Orchestrator, method: str, path: str, body: bytes, *, authorized: bool = True,
) -> tuple[int, Json]: ) -> tuple[int, Json]:
"""Route one control-plane request to a (status, payload) pair. Pure — """Route one control-plane request to a (status, payload) pair. Pure —
no I/O beyond the orchestrator so it is fully testable without a socket.""" no I/O beyond the orchestrator so it is fully testable without a socket.
`authorized` is whether the request presented the control-plane secret (or
no secret is configured see `ControlPlaneServer`). Every route except
`GET /health` requires it: the source-IP + identity-token checks inside
`/resolve` and `/attribute` authenticate the *bottle* a request is about,
not the *caller*, so without this gate any agent that can reach the port
could rewrite another bottle's policy, read the injected upstream tokens,
or approve its own supervise proposals. Defaults True so unit tests of the
routing logic don't have to thread it through."""
route = urlsplit(path).path.rstrip("/") or "/" route = urlsplit(path).path.rstrip("/") or "/"
if method == "GET" and route == "/health": if method == "GET" and route == "/health":
return 200, {"status": "ok"} return 200, {"status": "ok"}
if not authorized:
# Everything below is a trusted-caller operation. Deny before touching
# the registry / broker / supervise store.
return 401, {"error": "control-plane authentication required"}
if method == "GET" and route == "/gateway": if method == "GET" and route == "/gateway":
return 200, orch.gateway_status() return 200, orch.gateway_status()
@@ -117,6 +144,27 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
return 200, {"torn_down": True} return 200, {"torn_down": True}
return 404, {"error": "no such bottle"} return 404, {"error": "no such bottle"}
if method == "POST" and route == "/reconcile":
# Host-driven self-heal: the caller enumerates its live bottles (only
# the host can see the backend) and the orchestrator drops rows for
# every other active bottle. Trusted-caller only — an agent that could
# reach this would be able to unregister its neighbours.
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
raw_ips = data.get("live_source_ips")
if not isinstance(raw_ips, list):
return 400, {"error": "live_source_ips (list of strings) is required"}
live = [ip for ip in raw_ips if isinstance(ip, str) and ip]
grace = data.get("grace_seconds")
kwargs = (
{"grace_seconds": float(grace)}
if isinstance(grace, (int, float)) and not isinstance(grace, bool)
else {}
)
return 200, {"reaped": orch.reconcile(live, **kwargs)}
if method == "POST" and route == "/attribute": if method == "POST" and route == "/attribute":
try: try:
data = _parse_json_object(body) data = _parse_json_object(body)
@@ -209,8 +257,10 @@ class Handler(http.server.BaseHTTPRequestHandler):
assert isinstance(server, ControlPlaneServer) assert isinstance(server, ControlPlaneServer)
length = int(self.headers.get("Content-Length") or 0) length = int(self.headers.get("Content-Length") or 0)
body = self.rfile.read(length) if length > 0 else b"" body = self.rfile.read(length) if length > 0 else b""
authorized = server.is_authorized(self.headers.get(CONTROL_AUTH_HEADER, ""))
try: try:
status, payload = dispatch(server.orchestrator, method, self.path, body) status, payload = dispatch(
server.orchestrator, method, self.path, body, authorized=authorized)
except Exception as e: # noqa: BLE001 — the control plane must stay up except Exception as e: # noqa: BLE001 — the control plane must stay up
sys.stderr.write(f"orchestrator: {method} {self.path} failed: {e!r}\n") sys.stderr.write(f"orchestrator: {method} {self.path} failed: {e!r}\n")
sys.stderr.flush() sys.stderr.flush()
@@ -236,15 +286,40 @@ class Handler(http.server.BaseHTTPRequestHandler):
class ControlPlaneServer(socketserver.ThreadingMixIn, http.server.HTTPServer): class ControlPlaneServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
"""Threading HTTP server that carries the orchestrator for its handlers.""" """Threading HTTP server that carries the orchestrator for its handlers.
Holds the per-host control-plane secret (from `$BOT_BOTTLE_CONTROL_PLANE_TOKEN`,
injected by the launcher into this container only). When a secret is set,
every route but `/health` requires it; when it is unset the server runs
**open** and says so loudly at startup a fail-visible fallback for tests
and any backend that hasn't wired the secret yet (e.g. Firecracker, whose
nft boundary already blocks agents from the control-plane port)."""
daemon_threads = True daemon_threads = True
allow_reuse_address = True allow_reuse_address = True
def __init__(self, address: tuple[str, int], orchestrator: Orchestrator) -> None: def __init__(self, address: tuple[str, int], orchestrator: Orchestrator) -> None:
self.orchestrator = orchestrator self.orchestrator = orchestrator
self._auth_token = os.environ.get(CONTROL_PLANE_TOKEN_ENV, "").strip()
if not self._auth_token:
sys.stderr.write(
"orchestrator: WARNING — no control-plane secret "
f"(${CONTROL_PLANE_TOKEN_ENV}); running WITHOUT caller "
"authentication. Any client that can reach this port can drive "
"it. Backends that put the control plane on an agent-reachable "
"network MUST set this.\n"
)
sys.stderr.flush()
super().__init__(address, Handler) super().__init__(address, Handler)
def is_authorized(self, presented: str) -> bool:
"""True iff the request may proceed past `/health`: either no secret is
configured (open mode) or the presented header matches it. Constant-time
compare so a wrong token leaks nothing timing-wise."""
if not self._auth_token:
return True
return hmac.compare_digest(presented, self._auth_token)
def make_server( def make_server(
orchestrator: Orchestrator, host: str = "127.0.0.1", port: int = 0 orchestrator: Orchestrator, host: str = "127.0.0.1", port: int = 0
@@ -254,4 +329,7 @@ def make_server(
return ControlPlaneServer((host, port), orchestrator) return ControlPlaneServer((host, port), orchestrator)
__all__ = ["dispatch", "Handler", "ControlPlaneServer", "make_server", "Json"] __all__ = [
"dispatch", "Handler", "ControlPlaneServer", "make_server", "Json",
"CONTROL_AUTH_HEADER",
]
+70 -16
View File
@@ -23,7 +23,12 @@ import time
from pathlib import Path from pathlib import Path
from ..docker_cmd import run_docker from ..docker_cmd import run_docker
from ..paths import host_db_path from ..paths import (
CONTROL_PLANE_TOKEN_ENV,
host_control_plane_token,
host_db_path,
host_gateway_ca_dir,
)
from ..supervise import DB_PATH_IN_CONTAINER from ..supervise import DB_PATH_IN_CONTAINER
# The host DB dir is bind-mounted here so the gateway's supervise daemon # The host DB dir is bind-mounted here so the gateway's supervise daemon
@@ -44,14 +49,23 @@ GATEWAY_LABEL = "bot-bottle-orch-gateway=1"
# the source IP the gateway attributes by is the address on this network. # the source IP the gateway attributes by is the address on this network.
GATEWAY_NETWORK = "bot-bottle-gateway" GATEWAY_NETWORK = "bot-bottle-gateway"
# mitmproxy's CA dir in the bundle. A persistent named volume here keeps the # mitmproxy's CA dir in the bundle. The host's gateway-CA dir (see
# gateway's self-generated CA STABLE across container recreation — every agent # `host_gateway_ca_dir`) is bind-mounted here so the gateway's self-generated
# installs this one CA to trust the shared gateway's TLS interception, so it # CA stays STABLE across container recreation — every agent installs this one
# must not rotate when the gateway restarts. # CA to trust the shared gateway's TLS interception, so it must not rotate when
# the gateway restarts. A host bind-mount rather than a named volume: a named
# volume is silently wiped by `docker volume prune`, minting a fresh CA that
# breaks every running bottle (issue #450).
MITMPROXY_HOME = "/home/mitmproxy/.mitmproxy" MITMPROXY_HOME = "/home/mitmproxy/.mitmproxy"
GATEWAY_CA_VOLUME = "bot-bottle-gateway-mitmproxy"
GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem" GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem"
# The CA material mitmproxy writes into its confdir. mitmproxy reuses these on
# startup when present and generates them only on first run, so persisting them
# is what makes the CA stable; deleting them (see `rotate_gateway_ca`) forces a
# fresh CA on the next start. `mitmproxy-ca.pem` (cert + private key) is the
# signing identity; the rest are derived encodings agents/clients consume.
GATEWAY_CA_GLOB = "mitmproxy-ca*"
# The gateway data-plane image + its Dockerfile. Kept as a local constant # The gateway data-plane image + its Dockerfile. Kept as a local constant
# rather than imported from the backend layer, which would drag # rather than imported from the backend layer, which would drag
# the whole backend layer into the lean orchestrator (see #359); unify when # the whole backend layer into the lean orchestrator (see #359); unify when
@@ -69,6 +83,26 @@ def _host_db_dir() -> str:
return str(db_dir) return str(db_dir)
def rotate_gateway_ca(ca_dir: Path | None = None) -> list[Path]:
"""Delete the persisted mitmproxy CA so the next gateway start mints a
fresh one the explicit, deliberate CA-rollover path (issue #450).
Persistence keeps the CA stable across restarts precisely because mitmproxy
reuses the on-disk CA; rotation is therefore just removing that material.
Returns the files removed (empty when there was no CA yet); idempotent.
This only clears the on-disk CA. It does NOT stop the running gateway (whose
mitmproxy still holds the old CA in memory) or re-provision agents the
caller recreates the gateway to mint the new CA and re-attaches bottles.
`rotate-ca` on the orchestrator CLI wires those steps together."""
ca_dir = ca_dir if ca_dir is not None else host_gateway_ca_dir()
removed: list[Path] = []
for path in sorted(ca_dir.glob(GATEWAY_CA_GLOB)):
path.unlink()
removed.append(path)
return removed
class GatewayError(Exception): class GatewayError(Exception):
"""The shared gateway failed to build/start/stop (non-zero `docker` exit).""" """The shared gateway failed to build/start/stop (non-zero `docker` exit)."""
@@ -122,7 +156,10 @@ class DockerGateway(Gateway):
self.network = network self.network = network
# The control-plane URL the gateway's data plane resolves per bottle # The control-plane URL the gateway's data plane resolves per bottle
# against — reached by container name over docker DNS on the shared # against — reached by container name over docker DNS on the shared
# network (container↔container, no host firewall). Empty → single-tenant. # network (container↔container, no host firewall). Mandatory to *run*
# the gateway (see `ensure_running`); empty is tolerated only for the
# construct-then-read-CA path (`ca_cert_pem` on an already-running
# container), which never launches a container.
self._orchestrator_url = orchestrator_url self._orchestrator_url = orchestrator_url
self._build_context = build_context or _REPO_ROOT self._build_context = build_context or _REPO_ROOT
self._dockerfile = dockerfile self._dockerfile = dockerfile
@@ -189,6 +226,16 @@ class DockerGateway(Gateway):
) )
def ensure_running(self) -> None: def ensure_running(self) -> None:
# Fail closed on a missing policy source. The data-plane daemons are
# resolver-only now (PRD 0070) — without an orchestrator URL egress
# raises, git-http exits 1, and supervise exits 2 — so launching a
# gateway without one would only crash-loop its daemons. Refuse here so
# the misconfiguration surfaces as a clear error, not a broken container.
if not self._orchestrator_url:
raise GatewayError(
"gateway requires an orchestrator URL to run "
"(resolver-only data plane; no single-tenant fallback)"
)
# Recreate when the running container's image is stale (a rebuild), # Recreate when the running container's image is stale (a rebuild),
# so source changes to the gateway's flat daemons take effect — not # so source changes to the gateway's flat daemons take effect — not
# just when the container is absent. # just when the container is absent.
@@ -204,9 +251,10 @@ class DockerGateway(Gateway):
"--name", self.name, "--name", self.name,
"--label", GATEWAY_LABEL, "--label", GATEWAY_LABEL,
"--network", self.network, "--network", self.network,
# Persist the self-generated CA so it survives restarts (agents # Persist the self-generated CA on the host so it survives both
# trust it) — see GATEWAY_CA_VOLUME. # container recreation AND docker volume pruning (agents trust it)
"--volume", f"{GATEWAY_CA_VOLUME}:{MITMPROXY_HOME}", # — see host_gateway_ca_dir / issue #450.
"--volume", f"{host_gateway_ca_dir()}:{MITMPROXY_HOME}",
# Share the one host DB: the supervise daemon queues proposals # Share the one host DB: the supervise daemon queues proposals
# into the same file the orchestrator (and the operator, over # into the same file the orchestrator (and the operator, over
# HTTP) reads — no second, disconnected DB in the container. # HTTP) reads — no second, disconnected DB in the container.
@@ -215,12 +263,18 @@ class DockerGateway(Gateway):
] ]
for port in self._host_port_bindings: for port in self._host_port_bindings:
argv += ["--publish", f"0.0.0.0:{port}:{port}"] argv += ["--publish", f"0.0.0.0:{port}:{port}"]
if self._orchestrator_url: run_env = dict(os.environ)
# Makes the gateway's egress / git / supervise daemons multi-tenant: # The gateway's egress / git / supervise daemons resolve source-IP ->
# each request resolves source-IP -> policy against the control plane. # policy against the control plane per request (guaranteed non-empty by
# the check above).
argv += ["--env", f"BOT_BOTTLE_ORCHESTRATOR_URL={self._orchestrator_url}"] argv += ["--env", f"BOT_BOTTLE_ORCHESTRATOR_URL={self._orchestrator_url}"]
# ...and present the control-plane secret on those /resolve calls (the
# control plane requires it). Bare `--env NAME` keeps the value off argv
# / `docker inspect`; only the gateway (not the agent) is given it.
argv += ["--env", CONTROL_PLANE_TOKEN_ENV]
run_env[CONTROL_PLANE_TOKEN_ENV] = host_control_plane_token()
argv.append(self.image_ref) argv.append(self.image_ref)
proc = run_docker(argv) proc = run_docker(argv, env=run_env)
if proc.returncode != 0: if proc.returncode != 0:
raise GatewayError(f"gateway failed to start: {proc.stderr.strip()}") raise GatewayError(f"gateway failed to start: {proc.stderr.strip()}")
@@ -249,7 +303,7 @@ class DockerGateway(Gateway):
__all__ = [ __all__ = [
"Gateway", "DockerGateway", "GatewayError", "Gateway", "DockerGateway", "GatewayError", "rotate_gateway_ca",
"GATEWAY_NAME", "GATEWAY_LABEL", "GATEWAY_IMAGE", "GATEWAY_NETWORK", "GATEWAY_NAME", "GATEWAY_LABEL", "GATEWAY_IMAGE", "GATEWAY_NETWORK",
"GATEWAY_CA_VOLUME", "GATEWAY_CA_CERT", "GATEWAY_CA_CERT", "GATEWAY_CA_GLOB",
] ]
+159 -148
View File
@@ -1,17 +1,15 @@
"""Orchestrator + gateway lifecycle (PRD 0070, docker slice). """Orchestrator + gateway lifecycle (PRD 0070, docker slice).
Runs the orchestrator control plane **as a container** on the shared gateway Runs both the orchestrator control plane and the gateway data plane inside
network, alongside the gateway container. This is the PRD's "virtualize the a single `bot-bottle-infra` container on the shared gateway network
orchestrator": container↔container between the gateway and the orchestrator matching the structure already used by the macOS and Firecracker backends.
avoids the host firewall (which drops containerhost traffic), and the gateway `gateway_init` is PID 1 and supervises both; the infra container is an
reaches the control plane by container name over docker DNS. The host CLI idempotent per-host singleton.
reaches it via a published loopback port.
The orchestrator runs with the **register-only broker** the *backend* The combined container replaces the prior two-container split
launches agent containers (compose), so the orchestrator needs no docker (bot-bottle-orchestrator + bot-bottle-orch-gateway). The host CLI reaches
socket. That keeps this control-plane container unprivileged; the host manages the control plane via a published loopback port; gateway daemons reach it
both containers. `ensure_running` is an idempotent singleton (fixed container over 127.0.0.1 (same container).
names + the published port).
""" """
from __future__ import annotations from __future__ import annotations
@@ -25,50 +23,75 @@ from pathlib import Path
from .. import log from .. import log
from ..docker_cmd import run_docker from ..docker_cmd import run_docker
from ..paths import bot_bottle_root from ..paths import (
from .gateway import GATEWAY_IMAGE, GATEWAY_NETWORK, DockerGateway, GatewayError CONTROL_PLANE_TOKEN_ENV,
bot_bottle_root,
host_control_plane_token,
host_gateway_ca_dir,
)
from ..supervise import DB_PATH_IN_CONTAINER
from .gateway import (
GATEWAY_DOCKERFILE,
GATEWAY_IMAGE,
GATEWAY_NETWORK,
GatewayError,
MITMPROXY_HOME,
_host_db_dir,
)
DEFAULT_PORT = 8099 DEFAULT_PORT = 8099
ORCHESTRATOR_NAME = "bot-bottle-orchestrator" DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
ORCHESTRATOR_LABEL = "bot-bottle-orchestrator=1"
# The control-plane's own runtime image — lean (python + the stdlib-only INFRA_NAME = "bot-bottle-infra"
# `bot_bottle` package, bind-mounted at run time), distinct from the heavy INFRA_LABEL = "bot-bottle-infra=1"
# gateway data-plane image it used to borrow (#384). Env override for # The combined infra image: gateway data plane + orchestrator content.
# operators pinning a published build. # Built from Dockerfile.infra (FROM gateway + COPY --from orchestrator).
INFRA_IMAGE = os.environ.get("BOT_BOTTLE_INFRA_IMAGE", "bot-bottle-infra:latest")
INFRA_DOCKERFILE = "Dockerfile.infra"
# Baked as a container label so `ensure_running` can detect whether the
# running container is executing the current bind-mounted source.
INFRA_SOURCE_HASH_LABEL = "bot-bottle-infra-source-hash"
# Orchestrator image: the single canonical definition of the control-plane
# content (lean: python:3.12-slim + bot_bottle package, no mitmproxy/git).
# Used as a build intermediate: `Dockerfile.infra` COPY --from this image.
ORCHESTRATOR_IMAGE = os.environ.get( ORCHESTRATOR_IMAGE = os.environ.get(
"BOT_BOTTLE_ORCHESTRATOR_IMAGE", "bot-bottle-orchestrator:latest" "BOT_BOTTLE_ORCHESTRATOR_IMAGE", "bot-bottle-orchestrator:latest"
) )
ORCHESTRATOR_DOCKERFILE = "Dockerfile.orchestrator" ORCHESTRATOR_DOCKERFILE = "Dockerfile.orchestrator"
# Baked onto the container as a label so `ensure_running` can tell whether the
# running process is executing the *current* bind-mounted source — see
# `_source_hash`.
ORCHESTRATOR_SOURCE_HASH_LABEL = "bot-bottle-orchestrator-source-hash"
# The repo root is bind-mounted into the control-plane container so # The gateway daemons + orchestrator the infra container runs.
# `python -m bot_bottle.orchestrator` resolves the package (the orchestrator # BOT_BOTTLE_GATEWAY_DAEMONS listing `orchestrator` opts it in to
# is stdlib-only, so the lean orchestrator image's python is enough). # gateway_init's supervise tree (see gateway_init._OPT_IN_DAEMONS).
_REPO_ROOT = Path(__file__).resolve().parents[2] _INFRA_DAEMONS = "egress,git-http,supervise,orchestrator"
_APP_DIR = "/app"
# The bind-mount path for the live control-plane source inside the
# container. Separate from /app so the gateway's baked scripts
# (egress_addon.py, egress-entrypoint.sh) are not overlaid.
_SRC_IN_CONTAINER = "/bot-bottle-src"
# Bot-bottle host-root bind-mount inside the container (DB + state).
_ROOT_IN_CONTAINER = "/bot-bottle-root" _ROOT_IN_CONTAINER = "/bot-bottle-root"
# The supervise daemon writes proposals into the host DB directory.
_SUPERVISE_DB_DIR_IN_CONTAINER = os.path.dirname(DB_PATH_IN_CONTAINER)
_HEALTH_POLL_SECONDS = 0.25 _HEALTH_POLL_SECONDS = 0.25
DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
_HEALTH_REQUEST_TIMEOUT_SECONDS = 1.0 _HEALTH_REQUEST_TIMEOUT_SECONDS = 1.0
_REPO_ROOT = Path(__file__).resolve().parents[2]
class OrchestratorStartError(RuntimeError): class OrchestratorStartError(RuntimeError):
"""The orchestrator container did not become healthy within the timeout.""" """The infra container did not become healthy within the timeout."""
def _source_hash(repo_root: Path) -> str: def source_hash(repo_root: Path) -> str:
"""Content hash of the orchestrator's bind-mounted Python source (the """Content hash of the orchestrator's bind-mounted Python source (the
`bot_bottle` package the control-plane process imports). This only `bot_bottle` package the control-plane process imports). Changes only
changes when the code that would actually run inside the container when the code that would actually run changes `ensure_running`
changes `ensure_running` recreates the container on a mismatch and recreates the container on a mismatch so a code change takes effect,
otherwise leaves a healthy one alone, so a bottle launch that isn't but leaves a healthy up-to-date container alone to preserve in-memory
accompanied by a code change doesn't restart the process and drop every egress tokens."""
*other* active bottle's in-memory egress tokens (`Orchestrator._tokens`
in `service.py`, never persisted to disk by design)."""
h = hashlib.sha256() h = hashlib.sha256()
for path in sorted((repo_root / "bot_bottle").rglob("*.py")): for path in sorted((repo_root / "bot_bottle").rglob("*.py")):
h.update(str(path.relative_to(repo_root)).encode()) h.update(str(path.relative_to(repo_root)).encode())
@@ -77,52 +100,37 @@ def _source_hash(repo_root: Path) -> str:
class OrchestratorService: class OrchestratorService:
"""Manages the orchestrator control-plane container + the shared gateway. """Manages the single per-host infra container (control plane + gateway).
Callers only need `ensure_running()` + `url`. Callers only need `ensure_running()` + `url`.
`orchestrator_name` / `orchestrator_label` let backends run independent `infra_name` / `infra_label` let backends run independent infra containers
orchestrators on the same host without name collisions (e.g. the on the same host without name collisions (e.g. isolated integration tests
Firecracker backend uses `bot-bottle-fc-orchestrator` alongside the Docker that can't share the production INFRA_NAME singleton)."""
backend's `bot-bottle-orchestrator`). Subclass and override `_gateway()`
to supply a backend-specific gateway variant."""
def __init__( def __init__(
self, self,
*, *,
port: int = DEFAULT_PORT, port: int = DEFAULT_PORT,
network: str = GATEWAY_NETWORK, network: str = GATEWAY_NETWORK,
image: str = ORCHESTRATOR_IMAGE, image: str = INFRA_IMAGE,
gateway_image: str = GATEWAY_IMAGE,
repo_root: Path = _REPO_ROOT, repo_root: Path = _REPO_ROOT,
host_root: Path | None = None, host_root: Path | None = None,
orchestrator_name: str = ORCHESTRATOR_NAME, infra_name: str = INFRA_NAME,
orchestrator_label: str = ORCHESTRATOR_LABEL, infra_label: str = INFRA_LABEL,
) -> None: ) -> None:
self.port = port self.port = port
self.network = network self.network = network
# Two distinct images (#384): `image` is the lean control-plane
# runtime this container runs; `_gateway_image` is the heavy egress /
# git-gate / supervise data plane the gateway container runs. They
# were one conflated image before the split.
self.image = image self.image = image
self._gateway_image = gateway_image
self._repo_root = repo_root self._repo_root = repo_root
self._host_root = host_root or bot_bottle_root() self._host_root = host_root or bot_bottle_root()
self._orchestrator_name = orchestrator_name self._infra_name = infra_name
self._orchestrator_label = orchestrator_label self._infra_label = infra_label
@property @property
def url(self) -> str: def url(self) -> str:
"""Host-side control-plane URL (published loopback port).""" """Host-side control-plane URL (published loopback port)."""
return f"http://127.0.0.1:{self.port}" return f"http://127.0.0.1:{self.port}"
@property
def internal_url(self) -> str:
"""Control-plane URL as the gateway container reaches it — by name over
docker DNS on the shared network. This is the gateway's
BOT_BOTTLE_ORCHESTRATOR_URL."""
return f"http://{self._orchestrator_name}:{self.port}"
def is_healthy(self, *, timeout: float = _HEALTH_REQUEST_TIMEOUT_SECONDS) -> bool: def is_healthy(self, *, timeout: float = _HEALTH_REQUEST_TIMEOUT_SECONDS) -> bool:
try: try:
with urllib.request.urlopen(f"{self.url}/health", timeout=timeout) as resp: with urllib.request.urlopen(f"{self.url}/health", timeout=timeout) as resp:
@@ -134,128 +142,131 @@ class OrchestratorService:
proc = run_docker(["docker", "ps", "--filter", f"name=^/{name}$", "--format", "{{.Names}}"]) proc = run_docker(["docker", "ps", "--filter", f"name=^/{name}$", "--format", "{{.Names}}"])
return name in proc.stdout.split() return name in proc.stdout.split()
def _run_orchestrator_container(self, source_hash: str) -> None: def _infra_source_current(self, current_hash: str) -> bool:
"""Start the control-plane container (idempotent: clears a stale """True iff the running infra container was started from the current
fixed-name container first). Register-only broker no docker socket. bind-mounted source. Mirrors the macOS backend's `_source_current`."""
Labels the container with `source_hash` so a later `ensure_running` if not self._container_running(self._infra_name):
can detect a real code change (see `_source_hash`).""" return False
run_docker(["docker", "rm", "--force", self._orchestrator_name])
proc = run_docker([ proc = run_docker([
"docker", "run", "--detach", "docker", "inspect", "--format",
"--name", self._orchestrator_name, "{{ index .Config.Labels \"" + INFRA_SOURCE_HASH_LABEL + "\" }}",
"--label", self._orchestrator_label, self._infra_name,
"--label", f"{ORCHESTRATOR_SOURCE_HASH_LABEL}={source_hash}",
"--network", self.network,
# Host CLI reaches the control plane here; bound to loopback so it
# is not exposed on the host's external interfaces.
"--publish", f"127.0.0.1:{self.port}:{self.port}",
"--volume", f"{self._repo_root}:{_APP_DIR}:ro",
"--workdir", _APP_DIR,
# Persist the registry DB on the host (sole-owner: only the
# orchestrator opens bot-bottle.db).
"--volume", f"{self._host_root}:{_ROOT_IN_CONTAINER}",
"--env", f"BOT_BOTTLE_ROOT={_ROOT_IN_CONTAINER}",
"--entrypoint", "python3",
self.image,
"-m", "bot_bottle.orchestrator",
"--host", "0.0.0.0", "--port", str(self.port), "--broker", "stub",
]) ])
if proc.returncode != 0: if proc.returncode != 0:
raise OrchestratorStartError( return True # can't compare → don't churn a working container
f"orchestrator container failed to start: {proc.stderr.strip()}" return proc.stdout.strip() == current_hash
)
def _gateway(self) -> DockerGateway: def _ensure_network(self) -> None:
return DockerGateway( if run_docker(["docker", "network", "inspect", self.network]).returncode == 0:
self._gateway_image, network=self.network, orchestrator_url=self.internal_url
)
def _ensure_orchestrator_image(self) -> None:
"""Build the lean control-plane image from `Dockerfile.orchestrator`
when it's missing (#384). Cheap — a `FROM python:*-slim` base with no
deps to install, so the layer cache makes rebuilds a no-op. Unlike the
gateway image this is build-if-missing, not build-every-time: the
control plane bind-mounts its source, so a code change is caught by the
source-hash recreate (below), not by an image rebuild."""
if run_docker(["docker", "image", "inspect", self.image]).returncode == 0:
return return
argv = ["docker", "build", "-t", self.image, proc = run_docker(["docker", "network", "create", self.network])
"-f", str(self._repo_root / ORCHESTRATOR_DOCKERFILE), if proc.returncode != 0 and "already exists" not in proc.stderr:
raise GatewayError(
f"gateway network {self.network} failed to create: {proc.stderr.strip()}"
)
def _build_images(self) -> None:
"""Build the gateway base, the orchestrator intermediate, then the
infra image. All are cache-aware: a no-op when nothing changed."""
for tag, dockerfile in (
(GATEWAY_IMAGE, GATEWAY_DOCKERFILE),
(ORCHESTRATOR_IMAGE, ORCHESTRATOR_DOCKERFILE),
(self.image, INFRA_DOCKERFILE),
):
argv = ["docker", "build", "-t", tag,
"-f", str(self._repo_root / dockerfile),
str(self._repo_root)] str(self._repo_root)]
if os.environ.get("BOT_BOTTLE_NO_CACHE"): if os.environ.get("BOT_BOTTLE_NO_CACHE"):
argv.insert(2, "--no-cache") argv.insert(2, "--no-cache")
proc = run_docker(argv) proc = run_docker(argv)
if proc.returncode != 0: if proc.returncode != 0:
raise GatewayError( raise GatewayError(f"{dockerfile} build failed: {proc.stderr.strip()}")
f"orchestrator image build failed: {proc.stderr.strip()}"
)
def _orchestrator_source_current(self, current_hash: str) -> bool: def _run_infra_container(self, current_hash: str) -> None:
"""True iff the running orchestrator container was created from the """Start the combined infra container (idempotent: clears a stale
*current* bind-mounted source. Mirrors `DockerGateway`'s fixed-name container first). Labels the container with `current_hash`
image-staleness check, but by content hash rather than image id since so a later `ensure_running` can detect a real code change."""
the orchestrator runs bind-mounted source, not a built image.""" self._ensure_network()
if not self._container_running(self._orchestrator_name): run_docker(["docker", "rm", "--force", self._infra_name])
return False
proc = run_docker([ proc = run_docker([
"docker", "inspect", "--format", "docker", "run", "--detach",
"{{ index .Config.Labels \"" + ORCHESTRATOR_SOURCE_HASH_LABEL + "\" }}", "--name", self._infra_name,
self._orchestrator_name, "--label", self._infra_label,
]) "--label", f"{INFRA_SOURCE_HASH_LABEL}={current_hash}",
"--network", self.network,
# Host CLI reaches the control plane here (loopback only).
# gateway_init always starts the orchestrator on DEFAULT_PORT (8099)
# inside the container; self.port is the host-side published port.
"--publish", f"127.0.0.1:{self.port}:{DEFAULT_PORT}",
# Persist the mitmproxy CA on the host so it survives container
# recreation AND docker volume pruning (issue #450): every agent
# trusts this one CA, so a fresh one would break all running bottles.
"--volume", f"{host_gateway_ca_dir()}:{MITMPROXY_HOME}",
# Shared supervise DB (same file the operator reads over HTTP).
"--volume", f"{_host_db_dir()}:{_SUPERVISE_DB_DIR_IN_CONTAINER}",
"--env", f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
# Live control-plane source, mounted to a path that does not
# overlay the gateway's baked /app scripts.
"--volume", f"{self._repo_root}:{_SRC_IN_CONTAINER}:ro",
# PYTHONPATH lets the orchestrator (and other Python daemons)
# import the live source ahead of the installed package.
"--env", f"PYTHONPATH={_SRC_IN_CONTAINER}",
# Orchestrator registry DB on the host (sole writer: control plane).
"--volume", f"{self._host_root}:{_ROOT_IN_CONTAINER}",
"--env", f"BOT_BOTTLE_ROOT={_ROOT_IN_CONTAINER}",
# Control-plane secret: required by the orchestrator (to enforce)
# and by the gateway daemons (to present on /resolve calls).
"--env", CONTROL_PLANE_TOKEN_ENV,
# Gateway daemons reach the orchestrator over loopback at its
# fixed internal port (DEFAULT_PORT), independent of self.port.
"--env", f"BOT_BOTTLE_ORCHESTRATOR_URL=http://127.0.0.1:{DEFAULT_PORT}",
# Opt the orchestrator into gateway_init's supervise tree.
"--env", f"BOT_BOTTLE_GATEWAY_DAEMONS={_INFRA_DAEMONS}",
self.image,
], env={**os.environ, CONTROL_PLANE_TOKEN_ENV: host_control_plane_token()})
if proc.returncode != 0: if proc.returncode != 0:
return True # can't compare -> don't churn a working container raise OrchestratorStartError(
return proc.stdout.strip() == current_hash f"infra container failed to start: {proc.stderr.strip()}"
)
def ensure_running( def ensure_running(
self, *, startup_timeout: float = DEFAULT_STARTUP_TIMEOUT_SECONDS, self, *, startup_timeout: float = DEFAULT_STARTUP_TIMEOUT_SECONDS,
) -> str: ) -> str:
"""Ensure the control plane + shared gateway are up; return the host """Ensure the infra container (control plane + gateway) is up; return
control-plane URL. Idempotent a healthy control plane running the host control-plane URL. Idempotent a healthy container on current
current code and a running gateway are left untouched. Raises source is left untouched. Raises `OrchestratorStartError` on timeout."""
`OrchestratorStartError` on timeout.""" self._build_images()
gateway = self._gateway()
gateway.ensure_built() # rebuild the bundle image on a source change
gateway.ensure_running() # creates the shared network + (re)starts gateway
# Recreate the orchestrator container only when its bind-mounted current_hash = source_hash(self._repo_root)
# source has actually changed since it started — its Python process if self.is_healthy() and self._infra_source_current(current_hash):
# loaded that code at startup and won't reload, so a stale container
# would keep running OLD control-plane code. Recreating on *every*
# launch (the prior behaviour) would drop every other active
# bottle's in-memory egress tokens each time a new bottle starts,
# since the orchestrator process holds them only in memory (#381).
current_hash = _source_hash(self._repo_root)
if self.is_healthy() and self._orchestrator_source_current(current_hash):
return self.url return self.url
self._ensure_orchestrator_image() log.info("starting infra container", context={"name": self._infra_name})
log.info( self._run_infra_container(current_hash)
"starting orchestrator container",
context={"name": self._orchestrator_name},
)
self._run_orchestrator_container(current_hash)
deadline = time.monotonic() + startup_timeout deadline = time.monotonic() + startup_timeout
while time.monotonic() < deadline: while time.monotonic() < deadline:
if self.is_healthy(): if self.is_healthy():
log.info("orchestrator healthy", context={"url": self.url}) log.info("infra container healthy", context={"url": self.url})
return self.url return self.url
time.sleep(_HEALTH_POLL_SECONDS) time.sleep(_HEALTH_POLL_SECONDS)
raise OrchestratorStartError( raise OrchestratorStartError(
f"orchestrator at {self.url} did not become healthy within {startup_timeout:g}s" f"infra container at {self.url} did not become healthy within {startup_timeout:g}s"
) )
def stop(self) -> None: def stop(self) -> None:
"""Remove the orchestrator + gateway containers (idempotent).""" """Remove the infra container (idempotent)."""
run_docker(["docker", "rm", "--force", self._orchestrator_name]) run_docker(["docker", "rm", "--force", self._infra_name])
self._gateway().stop()
__all__ = [ __all__ = [
"OrchestratorService", "OrchestratorService",
"OrchestratorStartError", "OrchestratorStartError",
"ORCHESTRATOR_NAME", "INFRA_NAME",
"INFRA_IMAGE",
"INFRA_SOURCE_HASH_LABEL",
"ORCHESTRATOR_IMAGE", "ORCHESTRATOR_IMAGE",
"DEFAULT_PORT", "DEFAULT_PORT",
"DEFAULT_STARTUP_TIMEOUT_SECONDS", "DEFAULT_STARTUP_TIMEOUT_SECONDS",
"source_hash",
] ]
+78 -6
View File
@@ -32,6 +32,7 @@ import hmac
import secrets import secrets
import sqlite3 import sqlite3
import time import time
from collections.abc import Iterable
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
@@ -42,6 +43,12 @@ from ..paths import host_db_path
# 256 bits of urandom, URL-safe — unguessable per-bottle identity token. # 256 bits of urandom, URL-safe — unguessable per-bottle identity token.
IDENTITY_TOKEN_BYTES = 32 IDENTITY_TOKEN_BYTES = 32
# How recently a row must have been registered to be exempt from
# `reap_absent`. Covers the window between `container run` and the address
# becoming visible to another launch's enumeration, so reconciliation never
# reaps a bottle that is still coming up.
DEFAULT_REAP_GRACE_SECONDS = 120.0
def new_identity_token() -> str: def new_identity_token() -> str:
"""A fresh per-bottle identity token (PRD 0070 attribution defence).""" """A fresh per-bottle identity token (PRD 0070 attribution defence)."""
@@ -167,7 +174,7 @@ class RegistryStore(DbStore):
metadata=metadata, metadata=metadata,
policy=policy, policy=policy,
) )
with self._connect() as conn: with self._connection() as conn:
conn.execute( conn.execute(
"DELETE FROM orchestrator_bottles " "DELETE FROM orchestrator_bottles "
"WHERE source_ip = ? AND state = 'active' AND bottle_id != ?", "WHERE source_ip = ? AND state = 'active' AND bottle_id != ?",
@@ -193,7 +200,7 @@ class RegistryStore(DbStore):
def set_policy(self, bottle_id: str, policy: str) -> bool: def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's policy in place (live reload). Returns True if """Update a bottle's policy in place (live reload). Returns True if
the bottle exists.""" the bottle exists."""
with self._connect() as conn: with self._connection() as conn:
cur = conn.execute( cur = conn.execute(
"UPDATE orchestrator_bottles SET policy = ? WHERE bottle_id = ?", "UPDATE orchestrator_bottles SET policy = ? WHERE bottle_id = ?",
(policy, bottle_id), (policy, bottle_id),
@@ -203,7 +210,7 @@ class RegistryStore(DbStore):
def deregister(self, bottle_id: str) -> bool: def deregister(self, bottle_id: str) -> bool:
"""Remove a bottle. Returns True if a row was deleted.""" """Remove a bottle. Returns True if a row was deleted."""
with self._connect() as conn: with self._connection() as conn:
cur = conn.execute( cur = conn.execute(
"DELETE FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,) "DELETE FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,)
) )
@@ -211,7 +218,7 @@ class RegistryStore(DbStore):
def get(self, bottle_id: str) -> BottleRecord | None: def get(self, bottle_id: str) -> BottleRecord | None:
"""Return the bottle by id, or None if absent.""" """Return the bottle by id, or None if absent."""
with self._connect() as conn: with self._connection() as conn:
row = conn.execute( row = conn.execute(
"SELECT * FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,) "SELECT * FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,)
).fetchone() ).fetchone()
@@ -219,12 +226,76 @@ class RegistryStore(DbStore):
def all(self) -> list[BottleRecord]: def all(self) -> list[BottleRecord]:
"""Every registered bottle, oldest first.""" """Every registered bottle, oldest first."""
with self._connect() as conn: with self._connection() as conn:
rows = conn.execute( rows = conn.execute(
"SELECT * FROM orchestrator_bottles ORDER BY created_at" "SELECT * FROM orchestrator_bottles ORDER BY created_at"
).fetchall() ).fetchall()
return [_row_to_record(r) for r in rows] return [_row_to_record(r) for r in rows]
def reap_absent(
self,
live_source_ips: Iterable[str],
*,
grace_seconds: float = DEFAULT_REAP_GRACE_SECONDS,
now: float | None = None,
) -> list[BottleRecord]:
"""Delete active rows whose source IP is not held by a live bottle.
A row only ever leaves the registry two ways: an explicit
`teardown_bottle` (the launcher's cleanup callback) or the supersede
sweep in `register`. Neither runs when the launching CLI dies hard
SIGKILL, a closed terminal, a host sleep/crash so the row outlives
its container. That orphan is not inert: source IPs are recycled by
the backend's DHCP, and `by_source_ip` fail-closes on ambiguity, so a
leftover row at a reused address can brick the *next* bottle that
lands on it (no policy resolved -> every host denied, reported to the
agent as "not in the allowlist"). Reconciling against the live set at
launch keeps the registry from accumulating those landmines.
Restores the invariant the data plane needs: **at most one active row
per live address, and none at all for a dead one.** Two cases, because
a dead bottle's address may already have been handed to a live one:
* no live bottle holds the address every row there is an orphan;
* a live bottle holds it but several rows claim it the newest
registration is authoritative and the rest are orphans, the same
rule `register`'s same-IP supersede sweep applies. Without this
second case a recycled address stays ambiguous, which is exactly
the state that resolves no policy.
`grace_seconds` protects an in-flight launch: registration happens
moments after `container run`, and a concurrent launch's address may
not be visible to the caller's enumeration yet. Rows younger than the
grace window are never reaped, so reconciliation can't race a bottle
that is still coming up. Returns the deleted records."""
live = {ip for ip in live_source_ips if ip}
cutoff = (time.time() if now is None else now) - grace_seconds
with self._connection() as conn:
rows = conn.execute(
"SELECT * FROM orchestrator_bottles WHERE state = 'active'",
).fetchall()
by_ip: dict[str, list[BottleRecord]] = {}
for row in rows:
rec = _row_to_record(row)
by_ip.setdefault(rec.source_ip, []).append(rec)
candidates: list[BottleRecord] = []
for ip, recs in by_ip.items():
if ip not in live:
candidates.extend(recs)
continue
# Keep the newest claim on a live address; supersede the rest.
recs.sort(key=lambda r: r.created_at)
candidates.extend(recs[:-1])
doomed = [r for r in candidates if r.created_at <= cutoff]
for rec in doomed:
conn.execute(
"DELETE FROM orchestrator_bottles WHERE bottle_id = ?",
(rec.bottle_id,),
)
if doomed:
self._chmod()
return doomed
def by_source_ip(self, source_ip: str) -> BottleRecord | None: def by_source_ip(self, source_ip: str) -> BottleRecord | None:
"""Network-layer attribution: the single active bottle at this source """Network-layer attribution: the single active bottle at this source
IP, or None if unknown or ambiguous (more than one a IP, or None if unknown or ambiguous (more than one a
@@ -232,7 +303,7 @@ class RegistryStore(DbStore):
source IP is unspoofable (Firecracker `/31` + nft) and the control source IP is unspoofable (Firecracker `/31` + nft) and the control
plane is reachable only by the trusted gateway; pair with the plane is reachable only by the trusted gateway; pair with the
identity token (`attribute`) elsewhere.""" identity token (`attribute`) elsewhere."""
with self._connect() as conn: with self._connection() as conn:
rows = conn.execute( rows = conn.execute(
"SELECT * FROM orchestrator_bottles " "SELECT * FROM orchestrator_bottles "
"WHERE source_ip = ? AND state = 'active'", "WHERE source_ip = ? AND state = 'active'",
@@ -262,4 +333,5 @@ __all__ = [
"new_identity_token", "new_identity_token",
"default_db_path", "default_db_path",
"IDENTITY_TOKEN_BYTES", "IDENTITY_TOKEN_BYTES",
"DEFAULT_REAP_GRACE_SECONDS",
] ]
+62
View File
@@ -0,0 +1,62 @@
"""Rotate the shared gateway's mitmproxy CA (issue #450).
python -m bot_bottle.orchestrator.rotate_ca
A deliberate CA rollover has two halves: drop the *persisted* CA so a fresh one
is minted, and drop the *running* gateway so its mitmproxy (which holds the old
CA in memory) is replaced. This one-shot command does both:
1. Delete the persisted CA under the host gateway-CA dir the next gateway
start generates a new one (mitmproxy reuses an existing CA, generates only
when absent).
2. Force-remove the infra / standalone-gateway containers so the stale
in-memory CA is gone; the next bottle launch's idempotent `ensure_running`
brings the gateway back up and mints the fresh CA.
It does NOT re-provision the new CA into already-running bottles those must be
re-attached so they install the new trust anchor. Rotation is thus an explicit,
operator-driven action with a brief egress interruption, not an automatic one.
"""
from __future__ import annotations
import sys
from pathlib import Path
from ..docker_cmd import run_docker
from ..paths import host_gateway_ca_dir
from .gateway import GATEWAY_NAME, rotate_gateway_ca
from .lifecycle import INFRA_NAME
# The containers whose mitmproxy would still be serving the old CA from memory:
# the consolidated infra container and the standalone per-host gateway.
_GATEWAY_CONTAINERS = (INFRA_NAME, GATEWAY_NAME)
def _out(msg: str) -> None:
sys.stdout.write(f"rotate-ca: {msg}\n")
def main(argv: list[str] | None = None) -> int:
del argv # no flags — a single deliberate action
ca_dir: Path = host_gateway_ca_dir()
removed = rotate_gateway_ca(ca_dir)
if removed:
_out(f"removed {len(removed)} CA file(s) from {ca_dir}")
else:
_out(f"no persisted CA under {ca_dir}; a fresh one is minted on next start")
# Drop any running gateway so its in-memory (now-stale) CA is replaced on
# the next launch. `rm --force` on an absent name is a tolerated no-op.
for name in _GATEWAY_CONTAINERS:
proc = run_docker(["docker", "rm", "--force", name])
if proc.returncode == 0 and proc.stdout.strip():
_out(f"removed running container {name}")
_out("done — the next bottle launch remints the CA; re-attach bottles to "
"install the new trust anchor")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+30 -1
View File
@@ -13,15 +13,20 @@ Launch lifecycle:
and returns the record. If the broker rejects/fails, the registry entry and returns the record. If the broker rejects/fails, the registry entry
is rolled back so a failed launch leaves no orphan. is rolled back so a failed launch leaves no orphan.
* `teardown_bottle` sends a signed teardown request, then deregisters. * `teardown_bottle` sends a signed teardown request, then deregisters.
* `reconcile` sweeps rows whose bottle is no longer running the
self-heal for the teardown paths that never got to run (a hard-killed
launcher), since an orphan row at a recycled source IP bricks the next
bottle that lands on it.
""" """
from __future__ import annotations from __future__ import annotations
import json import json
from collections.abc import Iterable
from datetime import datetime, timezone from datetime import datetime, timezone
from .broker import LaunchBroker, LaunchRequest, sign_request from .broker import LaunchBroker, LaunchRequest, sign_request
from .registry import BottleRecord, RegistryStore from .registry import DEFAULT_REAP_GRACE_SECONDS, BottleRecord, RegistryStore
from .gateway import Gateway from .gateway import Gateway
from ..supervise import ( from ..supervise import (
AuditEntry, AuditEntry,
@@ -117,6 +122,30 @@ class Orchestrator:
self._tokens.pop(bottle_id, None) self._tokens.pop(bottle_id, None)
return True return True
def reconcile(
self,
live_source_ips: Iterable[str],
*,
grace_seconds: float = DEFAULT_REAP_GRACE_SECONDS,
) -> list[str]:
"""Drop registry rows for bottles that are no longer running, and
forget their in-memory egress tokens. Returns the reaped bottle ids.
The caller supplies the live set because only the host can enumerate
its own containers the orchestrator runs *inside* the infra
container and has no view of the backend. Deliberately does not
broker a teardown: the container is already gone, so there is nothing
to stop, and a broker error must not stop the sweep from clearing
the row that would otherwise brick the next bottle at that address.
See `RegistryStore.reap_absent` for why orphans accumulate and why
they are harmful rather than merely untidy."""
reaped = self.registry.reap_absent(
live_source_ips, grace_seconds=grace_seconds)
for rec in reaped:
self._tokens.pop(rec.bottle_id, None)
return [rec.bottle_id for rec in reaped]
def tokens_for(self, bottle_id: str) -> dict[str, str]: def tokens_for(self, bottle_id: str) -> dict[str, str]:
"""The bottle's in-memory egress auth tokens (env_name -> value), or """The bottle's in-memory egress auth tokens (env_name -> value), or
empty. The gateway injects these per request; they are never empty. The gateway injects these per request; they are never
+85 -1
View File
@@ -16,6 +16,8 @@ layer (and to COPY flat into the gateway).
from __future__ import annotations from __future__ import annotations
import os import os
import secrets
import stat
from pathlib import Path from pathlib import Path
# The single shared host state DB. All bot-bottle SQLite stores (supervise # The single shared host state DB. All bot-bottle SQLite stores (supervise
@@ -23,6 +25,21 @@ from pathlib import Path
# TableMigrations schema_key namespaces each store's tables. # TableMigrations schema_key namespaces each store's tables.
HOST_DB_FILENAME = "bot-bottle.db" HOST_DB_FILENAME = "bot-bottle.db"
# The per-host control-plane secret file, and the env var the launchers inject
# its value into. The control plane requires this secret on every mutating /
# reading route (see orchestrator/control_plane.py); it is held only by the
# trusted callers (control plane, gateway, host CLI) and never handed to an
# agent, so an agent that can reach the control-plane port still can't drive it.
CONTROL_PLANE_TOKEN_FILENAME = "control-plane-token"
CONTROL_PLANE_TOKEN_ENV = "BOT_BOTTLE_CONTROL_PLANE_TOKEN"
# The host directory holding the gateway's persistent mitmproxy CA. Bind-mounted
# into the infra/gateway container at mitmproxy's confdir so the self-generated
# CA survives container recreation — every agent installs this one CA to trust
# the shared gateway's TLS interception, so it must not rotate on restart. See
# host_gateway_ca_dir() for why this is a host bind-mount, not a named volume.
GATEWAY_CA_DIRNAME = "gateway-ca"
def bot_bottle_root() -> Path: def bot_bottle_root() -> Path:
"""The app data root — `$BOT_BOTTLE_ROOT` if set, else `~/.bot-bottle`.""" """The app data root — `$BOT_BOTTLE_ROOT` if set, else `~/.bot-bottle`."""
@@ -40,4 +57,71 @@ def host_db_path() -> Path:
return bot_bottle_root() / "db" / HOST_DB_FILENAME return bot_bottle_root() / "db" / HOST_DB_FILENAME
__all__ = ["HOST_DB_FILENAME", "bot_bottle_root", "host_db_path"] def host_db_dir() -> Path:
"""The directory holding the shared host state DB, created if missing.
Backends bind-mount this into their gateway so the supervise daemon writes
to the one DB the orchestrator (and the operator over HTTP) reads."""
db_dir = host_db_path().parent
db_dir.mkdir(parents=True, exist_ok=True)
return db_dir
def host_gateway_ca_dir() -> Path:
"""The directory holding the gateway's persistent mitmproxy CA, created if
missing. Backends bind-mount this into the infra/gateway container at
mitmproxy's confdir so the CA persists across container recreation.
A host bind-mount under the app-data root deliberately NOT a Docker
named volume. A named volume survives `docker rm` but is silently wiped by
`docker volume prune` / `docker system prune --volumes` during routine host
maintenance; the gateway then mints a fresh CA that every already-running
bottle distrusts, failing the TLS handshake even after it reconnects to the
moved gateway (issue #450). A path under the root docker never prunes it,
and it stays directly inspectable + rotatable from the host."""
ca_dir = bot_bottle_root() / GATEWAY_CA_DIRNAME
ca_dir.mkdir(parents=True, exist_ok=True)
return ca_dir
def host_control_plane_token() -> str:
"""The per-host control-plane secret, minted (256-bit, url-safe) and
persisted 0600 on first use, then reused.
This is the shared secret the launchers inject into the control-plane and
gateway containers and that the host CLI presents on every call. It is a
*host* artifact the file lives under the root the agent never mounts, and
the env var is set only on the trusted containers so reading it here is
safe on the host launch path but the value never reaches a bottle."""
path = bot_bottle_root() / CONTROL_PLANE_TOKEN_FILENAME
try:
existing = path.read_text().strip()
if existing:
return existing
except OSError:
pass
path.parent.mkdir(parents=True, exist_ok=True)
token = secrets.token_urlsafe(32)
# Create 0600 up front (O_EXCL loses a concurrent race harmlessly — we
# re-read the winner's token below) so the secret is never briefly world-
# readable between write and chmod.
try:
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
except FileExistsError:
return path.read_text().strip()
with os.fdopen(fd, "w") as f:
f.write(token)
os.chmod(path, stat.S_IRUSR | stat.S_IWUSR)
return token
__all__ = [
"HOST_DB_FILENAME",
"CONTROL_PLANE_TOKEN_FILENAME",
"CONTROL_PLANE_TOKEN_ENV",
"GATEWAY_CA_DIRNAME",
"bot_bottle_root",
"host_db_path",
"host_db_dir",
"host_gateway_ca_dir",
"host_control_plane_token",
]
+20 -3
View File
@@ -22,18 +22,35 @@ closed too rather than silently serving stale or empty policy.
The resolved value is the policy blob the orchestrator stores verbatim; the The resolved value is the policy blob the orchestrator stores verbatim; the
consumer parses it (e.g. the egress addon's `load_config`). This module is consumer parses it (e.g. the egress addon's `load_config`). This module is
stdlib-only and free of bot-bottle imports so it can be COPYed flat into stdlib-only and free of bot-bottle imports.
the gateway.
""" """
from __future__ import annotations from __future__ import annotations
import json import json
import os
import urllib.error import urllib.error
import urllib.request import urllib.request
DEFAULT_TIMEOUT_SECONDS = 2.0 DEFAULT_TIMEOUT_SECONDS = 2.0
# The control-plane secret this gateway presents on every /resolve call, read
# from the env the launcher injects into the gateway container. The control
# plane requires it (orchestrator/control_plane.py). Constant + env-var name are
# duplicated here rather than imported because this module is COPYed flat into
# the gateway image, free of bot-bottle imports — same rationale as
# IDENTITY_HEADER in egress_addon / git_http_backend.
CONTROL_AUTH_HEADER = "x-bot-bottle-control-auth"
CONTROL_PLANE_TOKEN_ENV = "BOT_BOTTLE_CONTROL_PLANE_TOKEN"
def _control_auth_headers() -> dict[str, str]:
"""The auth header to send, or {} when no secret is configured (an open
control plane, e.g. Firecracker behind its nft boundary sending nothing
is correct there and harmlessly ignored)."""
token = os.environ.get(CONTROL_PLANE_TOKEN_ENV, "").strip()
return {CONTROL_AUTH_HEADER: token} if token else {}
class PolicyResolveError(RuntimeError): class PolicyResolveError(RuntimeError):
"""The orchestrator was unreachable or returned an unexpected status — """The orchestrator was unreachable or returned an unexpected status —
@@ -57,7 +74,7 @@ class PolicyResolver:
).encode() ).encode()
req = urllib.request.Request( req = urllib.request.Request(
f"{self._base}/resolve", data=body, method="POST", f"{self._base}/resolve", data=body, method="POST",
headers={"Content-Type": "application/json"}, headers={"Content-Type": "application/json", **_control_auth_headers()},
) )
try: try:
with urllib.request.urlopen(req, timeout=self._timeout) as resp: with urllib.request.urlopen(req, timeout=self._timeout) as resp:
+7 -7
View File
@@ -66,7 +66,7 @@ class QueueStore(DbStore):
super().__init__(resolved, migrations) super().__init__(resolved, migrations)
def write_proposal(self, proposal: Proposal) -> Path: def write_proposal(self, proposal: Proposal) -> Path:
with self._connect() as conn: with self._connection() as conn:
conn.execute( conn.execute(
""" """
INSERT OR REPLACE INTO supervise_proposals ( INSERT OR REPLACE INTO supervise_proposals (
@@ -89,7 +89,7 @@ class QueueStore(DbStore):
return self.db_path return self.db_path
def read_proposal(self, proposal_id: str) -> Proposal: def read_proposal(self, proposal_id: str) -> Proposal:
with self._connect() as conn: with self._connection() as conn:
row = conn.execute( row = conn.execute(
""" """
SELECT * FROM supervise_proposals SELECT * FROM supervise_proposals
@@ -104,7 +104,7 @@ class QueueStore(DbStore):
def list_pending_proposals(self) -> list[Proposal]: def list_pending_proposals(self) -> list[Proposal]:
if not self.db_path.is_file(): if not self.db_path.is_file():
return [] return []
with self._connect() as conn: with self._connection() as conn:
rows = conn.execute( rows = conn.execute(
""" """
SELECT p.* FROM supervise_proposals p SELECT p.* FROM supervise_proposals p
@@ -125,7 +125,7 @@ class QueueStore(DbStore):
def list_all_pending_proposals(self) -> list[Proposal]: def list_all_pending_proposals(self) -> list[Proposal]:
if not self.db_path.is_file(): if not self.db_path.is_file():
return [] return []
with self._connect() as conn: with self._connection() as conn:
rows = conn.execute( rows = conn.execute(
""" """
SELECT p.* FROM supervise_proposals p SELECT p.* FROM supervise_proposals p
@@ -142,7 +142,7 @@ class QueueStore(DbStore):
return [self._row_to_proposal(row) for row in rows] return [self._row_to_proposal(row) for row in rows]
def write_response(self, response: Response) -> Path: def write_response(self, response: Response) -> Path:
with self._connect() as conn: with self._connection() as conn:
conn.execute( conn.execute(
""" """
INSERT OR REPLACE INTO supervise_responses ( INSERT OR REPLACE INTO supervise_responses (
@@ -161,7 +161,7 @@ class QueueStore(DbStore):
return self.db_path return self.db_path
def read_response(self, proposal_id: str) -> Response: def read_response(self, proposal_id: str) -> Response:
with self._connect() as conn: with self._connection() as conn:
row = conn.execute( row = conn.execute(
""" """
SELECT * FROM supervise_responses SELECT * FROM supervise_responses
@@ -176,7 +176,7 @@ class QueueStore(DbStore):
def archive_proposal(self, proposal_id: str) -> None: def archive_proposal(self, proposal_id: str) -> None:
if not self.db_path.is_file(): if not self.db_path.is_file():
return return
with self._connect() as conn: with self._connection() as conn:
conn.execute( conn.execute(
""" """
UPDATE supervise_proposals SET archived = 1 UPDATE supervise_proposals SET archived = 1
+4
View File
@@ -6,9 +6,11 @@ from pathlib import Path
try: try:
from .audit_store import AuditStore from .audit_store import AuditStore
from .config_store import ConfigStore
from .queue_store import QueueStore from .queue_store import QueueStore
except ImportError: except ImportError:
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from config_store import ConfigStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
_instance: StoreManager | None = None _instance: StoreManager | None = None
@@ -47,11 +49,13 @@ class StoreManager:
return ( return (
QueueStore("", self.db_path).is_migrated() QueueStore("", self.db_path).is_migrated()
and AuditStore(self.db_path).is_migrated() and AuditStore(self.db_path).is_migrated()
and ConfigStore(self.db_path).is_migrated()
) )
def migrate(self) -> None: def migrate(self) -> None:
QueueStore("", self.db_path).migrate() QueueStore("", self.db_path).migrate()
AuditStore(self.db_path).migrate() AuditStore(self.db_path).migrate()
ConfigStore(self.db_path).migrate()
__all__ = ["StoreManager"] __all__ = ["StoreManager"]
+2 -18
View File
@@ -37,7 +37,6 @@ from abc import ABC
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
try:
from .supervise_types import ( from .supervise_types import (
ACTION_OPERATOR_EDIT, ACTION_OPERATOR_EDIT,
AuditEntry, AuditEntry,
@@ -48,23 +47,7 @@ try:
STATUS_MODIFIED, STATUS_MODIFIED,
STATUS_REJECTED, STATUS_REJECTED,
TOOLS, TOOLS,
TOOL_EGRESS_ALLOW, TOOL_CHECK_PROPOSAL,
TOOL_EGRESS_BLOCK,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
except ImportError:
from supervise_types import ( # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
ACTION_OPERATOR_EDIT,
AuditEntry,
Proposal,
Response,
STATUSES,
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOLS,
TOOL_EGRESS_ALLOW, TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK, TOOL_EGRESS_BLOCK,
TOOL_EGRESS_TOKEN_ALLOW, TOOL_EGRESS_TOKEN_ALLOW,
@@ -281,6 +264,7 @@ __all__ = [
"TOOLS", "TOOLS",
"EGRESS_FORWARD_PROXY", "EGRESS_FORWARD_PROXY",
"EGRESS_INTROSPECT_URL", "EGRESS_INTROSPECT_URL",
"TOOL_CHECK_PROPOSAL",
"TOOL_EGRESS_ALLOW", "TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK", "TOOL_EGRESS_BLOCK",
"TOOL_GITLEAKS_ALLOW", "TOOL_GITLEAKS_ALLOW",
+183 -130
View File
@@ -2,37 +2,44 @@
Per-bottle MCP server exposing tools the agent calls to propose egress Per-bottle MCP server exposing tools the agent calls to propose egress
config changes when stuck. The tools are `egress-allow`, config changes when stuck. The tools are `egress-allow`,
`egress-block`, and `list-egress-routes`. `egress-block`, `list-egress-routes`, and `check-proposal`.
Each queued tool call: Each queued proposal tool call:
1. Validates the proposed file syntactically. 1. Validates the proposed file syntactically.
2. Writes a Proposal to the host SQLite database. 2. Writes a Proposal to the host SQLite database.
3. Blocks polling for a matching Response row. 3. Blocks polling for a matching Response row, up to a short grace
4. Returns the operator's `{status, notes}` to the agent. window (`SUPERVISE_RESPONSE_TIMEOUT_SECONDS`, default 30s).
4. On a decision within the window, returns the operator's
`{status, notes}`. On timeout, returns `status: pending` **with the
proposal id** and leaves the proposal queued the flow is
non-blocking past the grace window (PRD prd-new / issue #412).
The bottle slug arrives via SUPERVISE_BOTTLE_SLUG env (stamped at `check-proposal` is the non-blocking companion: given a `proposal_id`
container creation by the backend's start step). SUPERVISE_DB_PATH returned by a `pending` response, it reports the current decision
(`pending` | `approved` | `modified` | `rejected`) without re-proposing,
so an approval made out-of-band (e.g. a web review console) can be resumed
without holding an HTTP request open.
One shared server fronts every bottle (PRD 0070) and attributes each
proposal to the calling bottle by source IP, resolved from the orchestrator
an unattributed or unreachable source fails closed. BOT_BOTTLE_ORCHESTRATOR_URL
is mandatory: there is no fixed-slug single-tenant fallback. SUPERVISE_DB_PATH
points at the bind-mounted host database. points at the bind-mounted host database.
Consolidated (PRD 0070): when BOT_BOTTLE_ORCHESTRATOR_URL is set, one
shared server fronts every bottle and attributes each proposal to the
calling bottle by source IP (resolved from the orchestrator) instead of a
fixed slug an unattributed source fails closed. Unset the legacy
per-bottle single-tenant server, unchanged.
Speaks MCP over HTTP+JSON-RPC. Methods handled: Speaks MCP over HTTP+JSON-RPC. Methods handled:
* `initialize` handshake; returns server info + caps. * `initialize` handshake; returns server info + caps.
* `notifications/initialized` ack-only. * `notifications/initialized` ack-only.
* `tools/list` returns the tool definitions. * `tools/list` returns the tool definitions.
* `tools/call` validates, queues, blocks, returns. * `tools/call` validates, queues, waits out the grace
window, returns (pending past it); or, for
`check-proposal`, a non-blocking status poll.
Everything else returns JSON-RPC error -32601 (method not found). Everything else returns JSON-RPC error -32601 (method not found).
Stdlib-only. The Dockerfile copies this file + bot_bottle/supervise.py The Dockerfile copies this script to /app/supervise_server.py and installs
into the image; the server imports `supervise` for the queue / Proposal the bot_bottle package so its `from bot_bottle.*` imports resolve.
plumbing.
""" """
from __future__ import annotations from __future__ import annotations
@@ -44,33 +51,20 @@ import socketserver
import sys import sys
import time import time
import typing import typing
import urllib.error
import urllib.request
from dataclasses import dataclass, replace from dataclasses import dataclass, replace
try: from bot_bottle.constants import IDENTITY_HEADER
# Same-directory imports inside the bundle container; these files are from bot_bottle.egress_addon_core import (
# COPYed flat under /app by Dockerfile.gateway.
from egress_addon_core import (
LOG_OFF, load_config, resolve_client_context, route_to_yaml_dict, LOG_OFF, load_config, resolve_client_context, route_to_yaml_dict,
) )
from policy_resolver import PolicyResolveError, PolicyResolver from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver
import supervise as _sv from bot_bottle import supervise as _sv
except ModuleNotFoundError:
# Package imports for host-side tests and tooling.
from .egress_addon_core import (
LOG_OFF, load_config, resolve_client_context, route_to_yaml_dict,
)
from .policy_resolver import PolicyResolveError, PolicyResolver
from . import supervise as _sv
# --- JSON-RPC / MCP plumbing ---------------------------------------------- # --- JSON-RPC / MCP plumbing ----------------------------------------------
MCP_PROTOCOL_VERSION = "2024-11-05" MCP_PROTOCOL_VERSION = "2024-11-05"
# App-layer identity token header (mirrors egress_addon / git_http_backend).
IDENTITY_HEADER = "x-bot-bottle-identity"
SERVER_NAME = "bot-bottle-supervise" SERVER_NAME = "bot-bottle-supervise"
SERVER_VERSION = "0.1.0" SERVER_VERSION = "0.1.0"
@@ -85,12 +79,10 @@ ERR_INTERNAL = -32603
DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0 DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05 MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
EGRESS_LIST_TIMEOUT_SECONDS = 5.0
# Consolidated (multi-tenant) mode: when set, one shared supervise server # The per-host orchestrator control plane the shared supervise server attributes
# fronts every bottle and attributes each proposal to the calling bottle by # each proposal to, by source IP. Mandatory — there is no single-tenant
# source IP (resolved from the orchestrator), instead of a single # SUPERVISE_BOTTLE_SLUG fallback.
# SUPERVISE_BOTTLE_SLUG env. Unset → legacy per-bottle single-tenant.
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL" ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
@@ -252,6 +244,31 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
), ),
"inputSchema": _proposal_input_schema(), "inputSchema": _proposal_input_schema(),
}, },
{
"name": _sv.TOOL_CHECK_PROPOSAL,
"description": (
"Poll a previously queued proposal for the operator's decision "
"WITHOUT blocking or re-proposing. Pass the `proposal_id` you "
"got back when an `egress-allow`/`egress-block` call returned "
"`status: pending`. Returns the current status: `pending` (no "
"decision yet — poll again later), `approved`, `modified`, "
"`rejected`, or `unknown` (no such queued proposal — wrong id, "
"or it was already resolved and read)."
),
"inputSchema": {
"type": "object",
"properties": {
"proposal_id": {
"type": "string",
"description": (
"The proposal id from a `pending` response."
),
},
},
"required": ["proposal_id"],
"additionalProperties": False,
},
},
] ]
@@ -310,42 +327,6 @@ def handle_tools_list(_params: dict[str, object]) -> dict[str, object]:
return {"tools": TOOL_DEFINITIONS} return {"tools": TOOL_DEFINITIONS}
def handle_list_egress_routes(
_params: dict[str, object],
_config: ServerConfig,
) -> dict[str, object]:
"""Fetch the live egress route table via its
`_egress.local/allowlist` introspection endpoint. The
request goes through egress as a forward proxy; the
addon recognises the magic host and synthesizes a response
no real upstream connection, no allowlist enforcement
against the magic host. Returns the JSON payload as the
tool's text content."""
proxy_handler = urllib.request.ProxyHandler({
"http": _sv.EGRESS_FORWARD_PROXY,
})
opener = urllib.request.build_opener(proxy_handler)
try:
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=EGRESS_LIST_TIMEOUT_SECONDS) as resp:
body = resp.read().decode("utf-8")
except (urllib.error.URLError, OSError) as e:
return {
"content": [{
"type": "text",
"text": (
f"list-egress-routes: could not reach "
f"{_sv.EGRESS_INTROSPECT_URL!r} via "
f"{_sv.EGRESS_FORWARD_PROXY!r}: {e}"
),
}],
"isError": True,
}
return {
"content": [{"type": "text", "text": body}],
"isError": False,
}
def handle_tools_call( def handle_tools_call(
params: dict[str, object], params: dict[str, object],
config: ServerConfig, config: ServerConfig,
@@ -353,14 +334,13 @@ def handle_tools_call(
"""Validates the proposal, writes it to the queue, blocks waiting """Validates the proposal, writes it to the queue, blocks waiting
for a Response, returns the result wrapped in MCP `content`. for a Response, returns the result wrapped in MCP `content`.
Side-effect-free `list-*` tools short-circuit before the queue/ `list-egress-routes` never reaches here the handler answers it from
blocking machinery they're read-only introspection that the calling bottle's resolved policy before dispatching (see
doesn't need operator approval.""" `MCPHandler._dispatch`); this path is the queued, operator-approved
`egress-allow` / `egress-block` tools."""
name = params.get("name") name = params.get("name")
if not isinstance(name, str): if not isinstance(name, str):
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call missing 'name'") raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
if name == _sv.TOOL_LIST_EGRESS_ROUTES:
return handle_list_egress_routes(typing.cast(dict[str, object], params.get("arguments", {})), config)
args_raw = params.get("arguments", {}) args_raw = params.get("arguments", {})
if not isinstance(args_raw, dict): if not isinstance(args_raw, dict):
@@ -410,7 +390,7 @@ def handle_tools_call(
deadline=deadline, deadline=deadline,
) )
except TimeoutError: except TimeoutError:
text = format_pending_response_text(config.response_timeout_seconds) text = format_pending_response_text(proposal.id, config.response_timeout_seconds)
return { return {
"content": [{"type": "text", "text": text}], "content": [{"type": "text", "text": text}],
"isError": False, "isError": False,
@@ -427,6 +407,54 @@ def handle_tools_call(
} }
def handle_check_proposal(
params: dict[str, object],
config: ServerConfig,
) -> dict[str, object]:
"""Non-blocking poll of a queued proposal's decision, by id.
Never creates a Proposal (so `check-proposal` isn't in `TOOLS`); it only
reads the queue. Resolution order mirrors the synchronous path's terminal
step a decided proposal is archived here exactly as `handle_tools_call`
archives it after `wait_for_response`, so `pending` proposals stay visible
to the operator until they're both decided *and* polled."""
args_raw = params.get("arguments", {})
if not isinstance(args_raw, dict):
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
proposal_id = args_raw.get("proposal_id")
if not isinstance(proposal_id, str) or not proposal_id.strip():
raise _RpcClientError(
ERR_INVALID_PARAMS,
"check-proposal: 'proposal_id' is required and must be a non-empty string",
)
proposal_id = proposal_id.strip()
try:
response = _sv.read_response(config.bottle_slug, proposal_id)
except FileNotFoundError:
# No decision yet — distinguish "still queued" from "unknown id".
try:
_sv.read_proposal(config.bottle_slug, proposal_id)
except FileNotFoundError:
return {
"content": [{"type": "text", "text": format_unknown_proposal_text(proposal_id)}],
"isError": True,
}
return {
"content": [{"type": "text", "text": format_still_pending_text(proposal_id)}],
"isError": False,
}
try:
_sv.archive_proposal(config.bottle_slug, proposal_id)
except OSError as e:
raise _RpcInternalError(f"failed to archive proposal: {e}") from e
return {
"content": [{"type": "text", "text": format_response_text(response)}],
"isError": response.status == _sv.STATUS_REJECTED,
}
def format_response_text(response: "_sv.Response") -> str: def format_response_text(response: "_sv.Response") -> str:
"""Pretty-print a Response for the tool's text content. The agent """Pretty-print a Response for the tool's text content. The agent
reads the text and decides whether to retry / give up / surface.""" reads the text and decides whether to retry / give up / surface."""
@@ -439,12 +467,35 @@ def format_response_text(response: "_sv.Response") -> str:
return "\n".join(lines) return "\n".join(lines)
def format_pending_response_text(timeout_seconds: float) -> str: def format_pending_response_text(proposal_id: str, timeout_seconds: float) -> str:
"""Grace-window timeout: the proposal stays queued, and the agent is
told the id so it can `check-proposal` instead of re-proposing."""
return "\n".join([ return "\n".join([
"status: pending", "status: pending",
f"proposal_id: {proposal_id}",
( (
"notes: operator response timed out after " f"notes: no operator decision within {timeout_seconds:g}s; the "
f"{timeout_seconds:g}s; proposal remains queued" "proposal remains queued. Poll it (do not re-propose) by calling "
f"`check-proposal` with proposal_id={proposal_id!r}."
),
])
def format_still_pending_text(proposal_id: str) -> str:
return "\n".join([
"status: pending",
f"proposal_id: {proposal_id}",
"notes: still queued; no operator decision yet. Call `check-proposal` again later.",
])
def format_unknown_proposal_text(proposal_id: str) -> str:
return "\n".join([
"status: unknown",
f"proposal_id: {proposal_id}",
(
"notes: no queued proposal with this id for this bottle — the id "
"may be wrong, or the proposal was already resolved and read."
), ),
]) ])
@@ -531,36 +582,40 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
if method == "tools/list": if method == "tools/list":
return handle_tools_list(req.params) return handle_tools_list(req.params)
if method == "tools/call": if method == "tools/call":
# `list-egress-routes` is read-only introspection. In consolidated # `list-egress-routes` is read-only introspection. The shared gateway
# mode the gateway's *static* route table is empty (routes are # has no static route table (routes are resolved per request by
# resolved per request by source IP), so answer it from the calling # source IP), so answer it from the calling bottle's resolved policy.
# bottle's resolved policy. Otherwise the agent sees an empty # Otherwise the agent sees an empty allowlist and composes an egress
# allowlist and composes an egress proposal that *replaces* the live # proposal that *replaces* the live routes instead of extending them
# routes instead of extending them — silently dropping base routes # — silently dropping base routes like api.anthropic.com on approval.
# like api.anthropic.com when the operator approves it.
if req.params.get("name") == _sv.TOOL_LIST_EGRESS_ROUTES: if req.params.get("name") == _sv.TOOL_LIST_EGRESS_ROUTES:
resolved = self._resolved_routes_payload() return self._resolved_routes_payload()
if resolved is not None: # `check-proposal` is a non-blocking read of the calling bottle's
return resolved # own queue — attributed by source IP like a proposal, but it
# Attribute the proposal to the calling bottle. Single-tenant → the # never queues or blocks.
# env slug on `config`; consolidated → the source-IP-resolved if req.params.get("name") == _sv.TOOL_CHECK_PROPOSAL:
# bottle id, so one shared server queues each bottle's proposal return handle_check_proposal(req.params, self._attributed_config(config))
# under its own slug. # Attribute the proposal to the source-IP-resolved bottle, so the one
# shared server queues each bottle's proposal under its own slug.
return handle_tools_call(req.params, self._attributed_config(config)) return handle_tools_call(req.params, self._attributed_config(config))
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}") raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def _resolved_routes_payload(self) -> dict[str, object] | None: def _resolver_or_fail(self) -> "PolicyResolver":
"""The calling bottle's live egress routes as the `list-egress-routes` """This server's policy resolver. A server started without one is a
JSON payload, resolved by (source_ip, identity token) the same shape misconfiguration, not a tenancy mode fail closed rather than
the single-tenant introspection endpoint returns. None when there is no attribute (or list) anything."""
resolver (single-tenant), so the caller falls back to that endpoint.
Fail-closed like `_attributed_config`: an unattributed source or an
unreachable orchestrator yields an empty route list (never another
bottle's), courtesy of `resolve_client_context`."""
resolver = getattr(self.server, "policy_resolver", None) resolver = getattr(self.server, "policy_resolver", None)
if resolver is None: if resolver is None:
return None raise _RpcInternalError("supervise server has no policy resolver")
return resolver
def _resolved_routes_payload(self) -> dict[str, object]:
"""The calling bottle's live egress routes as the `list-egress-routes`
JSON payload, resolved by (source_ip, identity token). Fail-closed like
`_attributed_config`: an unattributed source or an unreachable
orchestrator yields an empty route list (never another bottle's),
courtesy of `resolve_client_context`."""
resolver = self._resolver_or_fail()
headers = getattr(self, "headers", None) headers = getattr(self, "headers", None)
token = headers.get(IDENTITY_HEADER, "") if headers is not None else "" token = headers.get(IDENTITY_HEADER, "") if headers is not None else ""
conf, _slug, _tokens = resolve_client_context( conf, _slug, _tokens = resolve_client_context(
@@ -572,14 +627,11 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
return {"content": [{"type": "text", "text": body}], "isError": False} return {"content": [{"type": "text", "text": body}], "isError": False}
def _attributed_config(self, config: ServerConfig) -> ServerConfig: def _attributed_config(self, config: ServerConfig) -> ServerConfig:
"""The ServerConfig with `bottle_slug` bound to *this request's* bottle. """The ServerConfig with `bottle_slug` bound to *this request's* bottle:
Single-tenant (no resolver): unchanged. Consolidated: the bottle id the bottle id attributed from the source IP **fail-closed**, an
attributed from the source IP **fail-closed**, an unattributed or unattributed or unreachable source raises so no proposal is queued under
unreachable source raises so no proposal is queued under the wrong (or the wrong (or empty) slug."""
empty) slug.""" resolver = self._resolver_or_fail()
resolver = getattr(self.server, "policy_resolver", None)
if resolver is None:
return config
# The agent's MCP client sends the identity token as a request header # The agent's MCP client sends the identity token as a request header
# (provisioned via `mcp add --header`); the orchestrator requires the # (provisioned via `mcp add --header`); the orchestrator requires the
# (source_ip, token) pair, so a missing/wrong token fail-closes below. # (source_ip, token) pair, so a missing/wrong token fail-closes below.
@@ -616,8 +668,9 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
allow_reuse_address = True allow_reuse_address = True
daemon_threads = True daemon_threads = True
config: ServerConfig = ServerConfig(bottle_slug="") config: ServerConfig = ServerConfig(bottle_slug="")
# None → single-tenant (proposals use config.bottle_slug); set → consolidated # Set by `serve`; every proposal is attributed to the source-IP-resolved
# (each proposal attributed to the source-IP-resolved bottle). # bottle. The class default is a placeholder — a server without a resolver
# fails closed per request (see `_resolver_or_fail`).
policy_resolver: "PolicyResolver | None" = None policy_resolver: "PolicyResolver | None" = None
@@ -626,21 +679,21 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
def serve( def serve(
*, *,
bottle_slug: str, resolver: "PolicyResolver",
port: int = _sv.SUPERVISE_PORT, port: int = _sv.SUPERVISE_PORT,
bind: str = "0.0.0.0", bind: str = "0.0.0.0",
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS, response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
resolver: "PolicyResolver | None" = None,
) -> typing.NoReturn: ) -> typing.NoReturn:
server = MCPServer((bind, port), MCPHandler) server = MCPServer((bind, port), MCPHandler)
# bottle_slug is a placeholder: every request's proposal is attributed to
# the source-IP-resolved bottle (see MCPHandler._attributed_config).
server.config = ServerConfig( server.config = ServerConfig(
bottle_slug=bottle_slug, bottle_slug="",
response_timeout_seconds=response_timeout_seconds, response_timeout_seconds=response_timeout_seconds,
) )
server.policy_resolver = resolver server.policy_resolver = resolver
mode = "multi-tenant" if resolver else f"slug={bottle_slug!r}"
sys.stderr.write( sys.stderr.write(
f"supervise listening on {bind}:{port}; {mode}; " f"supervise listening on {bind}:{port}; multi-tenant; "
f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type] f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type]
) )
sys.stderr.flush() sys.stderr.flush()
@@ -656,12 +709,13 @@ def serve(
def main(argv: list[str]) -> int: def main(argv: list[str]) -> int:
del argv # config is env-only, no CLI flags del argv # config is env-only, no CLI flags
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip() orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
resolver = PolicyResolver(orch_url) if orch_url else None if not orch_url:
bottle_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "") # Resolver-only: without an orchestrator the server can't attribute a
# Consolidated mode resolves the slug per request, so the env slug is # proposal to a bottle, so it must not serve (fail-closed).
# optional there; single-tenant still requires it. sys.stderr.write(
if not bottle_slug and resolver is None: f"supervise: {ORCHESTRATOR_URL_ENV} is required "
sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n") "(no single-tenant SUPERVISE_BOTTLE_SLUG fallback)\n"
)
return 2 return 2
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT))) port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0") bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
@@ -671,11 +725,10 @@ def main(argv: list[str]) -> int:
sys.stderr.write(f"supervise: {e}\n") sys.stderr.write(f"supervise: {e}\n")
return 2 return 2
serve( serve(
bottle_slug=bottle_slug, resolver=PolicyResolver(orch_url),
port=port, port=port,
bind=bind, bind=bind,
response_timeout_seconds=response_timeout_seconds, response_timeout_seconds=response_timeout_seconds,
resolver=resolver,
) )
return 0 # serve() does not return return 0 # serve() does not return
+5
View File
@@ -20,6 +20,10 @@ TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow" TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow" TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes" TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
# Read-only agent tool: poll a queued proposal for the operator's decision
# without blocking or re-proposing. It never becomes a `Proposal.tool` (no
# queue record is created for it), so it is intentionally NOT in `TOOLS`.
TOOL_CHECK_PROPOSAL = "check-proposal"
TOOLS: tuple[str, ...] = ( TOOLS: tuple[str, ...] = (
TOOL_EGRESS_ALLOW, TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK, TOOL_EGRESS_BLOCK,
@@ -156,6 +160,7 @@ __all__ = [
"TOOLS", "TOOLS",
"TOOL_EGRESS_ALLOW", "TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK", "TOOL_EGRESS_BLOCK",
"TOOL_CHECK_PROPOSAL",
"TOOL_EGRESS_TOKEN_ALLOW", "TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_GITLEAKS_ALLOW", "TOOL_GITLEAKS_ALLOW",
"TOOL_LIST_EGRESS_ROUTES", "TOOL_LIST_EGRESS_ROUTES",
+10
View File
@@ -7,6 +7,7 @@ from __future__ import annotations
import ipaddress import ipaddress
import os import os
import sys
def is_ip_literal(value: str) -> bool: def is_ip_literal(value: str) -> bool:
@@ -17,6 +18,15 @@ def is_ip_literal(value: str) -> bool:
return True return True
def read_tty_line() -> str:
"""Mirror `IFS= read -r REPLY </dev/tty`. Falls back to stdin."""
try:
with open("/dev/tty", "r", encoding="utf-8") as tty:
return tty.readline().rstrip("\n")
except OSError:
return sys.stdin.readline().rstrip("\n")
def expand_tilde(path: str) -> str: def expand_tilde(path: str) -> str:
"""Expand a leading '~' to $HOME. Leaves paths without a leading """Expand a leading '~' to $HOME. Leaves paths without a leading
tilde unchanged. Falls back to the empty string if $HOME is unset tilde unchanged. Falls back to the empty string if $HOME is unset
@@ -0,0 +1,57 @@
# ADR 0005: Keep tracker metadata on issues
- **Status:** Accepted
- **Date:** 2026-07-18
- **Deciders:** didericis
## Context
Gitea exposes labels on both issues and pull requests. Applying the same labels
to both copies planning metadata, creates a synchronization obligation, and
makes disagreements between the two records possible. At the same time,
unlabelled objects look accidental unless the repository states which object
owns the metadata.
The repository already uses issues as work items and PRs as implementations of
those work items. At this decision's cutoff, all open PRs reference issues, but
121 of 219 historically merged PRs do not. Manufacturing retrospective issues
for that history would create records that never participated in planning and
would make the issue history less truthful.
## Decision
Issues are the canonical tracker records and own labels. Every issue has at
least one label. An issue opened or left without labels receives
`Status/Needs Triage` automatically until it is classified.
Pull requests carry no labels. Every new PR deliberately references at least
one existing issue in its title or description with one of these forms:
- `Closes #123`, `Fixes #123`, or `Resolves #123` when merging completes it.
- `Part of #123`, `Related to #123`, `Refs #123`, or `References #123` when it
contributes without completing it.
Gitea Actions enforces both PR rules as a status check and repairs the empty
issue-label state. Branch protection makes the PR policy check required.
The policy applies from 2026-07-18 onward. Existing issues may be labelled as
they are encountered, but closed PRs are grandfathered: no retrospective
issues or PR labels are created solely to make history conform.
## Consequences
- Classification, priority, and workflow metadata have one source of truth.
- A PR's issue link is the navigation path to its planning metadata.
- Multi-PR issues do not require copied or synchronized labels.
- `Status/Needs Triage` is an intentional fallback, not a final
classification.
- Direct issue creation remains convenient; automation repairs a missing label
immediately after creation because Gitea has no native required-label rule.
- The required check must be configured in branch protection after this
workflow lands.
## Links
- Issue #405.
- `.gitea/workflows/tracker-policy.yml`.
- `scripts/tracker_policy.py`.
+1 -1
View File
@@ -1,6 +1,6 @@
# PRD 0023: smolmachines bottle backend # PRD 0023: smolmachines bottle backend
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis - **Author:** didericis
@@ -1,6 +1,6 @@
# PRD 0032: Decompose smolmachines launch and harden bringup sequencing # PRD 0032: Decompose smolmachines launch and harden bringup sequencing
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-claude - **Author:** didericis-claude
+1 -1
View File
@@ -1,6 +1,6 @@
# PRD 0038: smolmachines Env Contract and Secret-Safe Injection # PRD 0038: smolmachines Env Contract and Secret-Safe Injection
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-codex - **Author:** didericis-codex
@@ -1,6 +1,6 @@
# PRD 0039: smolmachines Capability-Block Remediation # PRD 0039: smolmachines Capability-Block Remediation
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-codex - **Author:** didericis-codex
+1 -1
View File
@@ -1,6 +1,6 @@
# PRD 0042: smolmachines Cross-Backend Parity Tests # PRD 0042: smolmachines Cross-Backend Parity Tests
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-codex - **Author:** didericis-codex
+1 -1
View File
@@ -1,6 +1,6 @@
# PRD 0057: Promote smolmachines to default backend; convert Docker to example-only # PRD 0057: Promote smolmachines to default backend; convert Docker to example-only
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis - **Author:** didericis
+1 -1
View File
@@ -1,6 +1,6 @@
# PRD 0068: smolmachines backend on Linux # PRD 0068: smolmachines backend on Linux
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`. > **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/agent-sandbox-landscape.md`.
- **Status:** Superseded (2026-07-11) — was Active - **Status:** Superseded (2026-07-11) — was Active
- **Author:** Claude - **Author:** Claude
@@ -112,12 +112,12 @@ bottle-agnostic**: the per-boot bits (authorized_keys, guest IP) arrive on the
published ext4 boots on any launch host. published ext4 boots on any launch host.
- **Artifact.** `rootfs.ext4` + a `rootfs.ext4.sha256`, published as a Gitea - **Artifact.** `rootfs.ext4` + a `rootfs.ext4.sha256`, published as a Gitea
**generic package** (`bot-bottle-infra/<tag>`) — generic packages take **generic package** (`bot-bottle-firecracker-infra/<tag>`) — generic packages take
arbitrary large binaries (no attachment size cap / file-type allowlist that arbitrary large binaries (no attachment size cap / file-type allowlist that
release attachments impose). The matching `vmlinux` kernel can ship the same release attachments impose). The matching `vmlinux` kernel can ship the same
way, so the whole VM is fetchable. way, so the whole VM is fetchable.
- **Pull.** The launch host `GET`s - **Pull.** The launch host `GET`s
`…/api/packages/<owner>/generic/bot-bottle-infra/<tag>/rootfs.ext4` (+ `…/api/packages/<owner>/generic/bot-bottle-firecracker-infra/<tag>/rootfs.ext4` (+
`.sha256`) for its pinned tag, verifies the checksum, caches it under the `.sha256`) for its pinned tag, verifies the checksum, caches it under the
tag, and attaches it as the infra VM's root disk. Host prerequisite is an tag, and attaches it as the infra VM's root disk. Host prerequisite is an
HTTP client — nothing else. Public packages need no auth to pull; a token HTTP client — nothing else. Public packages need no auth to pull; a token
+38
View File
@@ -312,6 +312,44 @@ reaches over the RPC rather than a shared mount into the VM. WAL on the
shared DB is therefore a deliberate, tested future change — not enabled ad shared DB is therefore a deliberate, tested future change — not enabled ad
hoc. `sqlite3` itself is stdlib, so "the host needs SQLite" is a non-cost. hoc. `sqlite3` itself is stdlib, so "the host needs SQLite" is a non-cost.
### Gateway CA: host-resident, like the DB
The shared gateway bumps TLS with a self-generated mitmproxy CA, and **every
bottle installs that CA** into its trust store to accept the bumped leaves. So
the CA is durable per-host state with the same rule as the DB: it must outlive
any single gateway container, or a restart mints a fresh CA that every
already-running bottle distrusts — the TLS handshake then fails even after the
bottle re-resolves and reconnects to the moved gateway (issue #450, a
re-attachment blocker distinct from #443/#445).
The CA lives on the **host filesystem** at `bot_bottle_root()/gateway-ca`
(`host_gateway_ca_dir()`), bind-mounted into the container at mitmproxy's
confdir. This is deliberately a host bind-mount, **not a container-runtime
named volume**: a named volume survives ordinary container removal but can be
silently wiped by Docker's or Apple Container's volume-prune commands during
routine host maintenance, which is exactly how the ephemeral-CA symptom shows
up in practice. A path under the app-data root is not managed or pruned by the
container runtime, and stays directly inspectable and rotatable from the host.
mitmproxy reuses an existing CA and generates one only on first run, so the
bind-mount alone gives
"adopt-existing, generate-on-first-run" for free.
The macOS backend uses the same host-resident CA directory and bind-mounts it
into the consolidated Apple infra container. Its `bot-bottle-mac-db` named
volume remains container-only because that prevents incoherent cross-kernel
SQLite locking, but the CA is deliberately not stored there: Apple Container
also has a `container volume prune` operation, and the named volume is
temporarily unreferenced while the infra container is recreated. Keeping the
CA on the host makes both ordinary recreation and volume pruning safe.
**Deliberate rollover** is the explicit inverse: `rotate_gateway_ca()` removes
the persisted CA material so the next start remints it, and the
`python -m bot_bottle.orchestrator.rotate_ca` one-shot wires that together with
dropping the running gateway container (whose mitmproxy still holds the old CA
in memory). Rotation does not auto-re-provision the new CA into running bottles
— those re-attach to install the new anchor — so it is an operator action with
a brief egress interruption, never an implicit one.
## Sequencing ## Sequencing
Jump straight to the **virtualized** end state (not a host-daemon stepping Jump straight to the **virtualized** end state (not a host-daemon stepping

Some files were not shown because too many files have changed in this diff Show More