Compare commits

..

117 Commits

Author SHA1 Message Date
didericis-claude dee0121e8d feat(prd-0081): reprovision CA, git-gate, and egress tokens on gateway bring-up
prd-number-check / require-numbered-prds (pull_request) Successful in 13s
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / image-input-builds (pull_request) Successful in 41s
test / unit (pull_request) Successful in 51s
test / integration-docker (pull_request) Successful in 58s
test / coverage (pull_request) Successful in 41s
test / image-input-builds (push) Successful in 48s
test / unit (push) Successful in 56s
lint / lint (push) Successful in 1m2s
Update Quality Badges / update-badges (push) Successful in 1m4s
test / integration-docker (push) Failing after 2m53s
test / coverage (push) Has been skipped
On a Firecracker gateway cold boot, reconcile every live agent VM against
the fresh gateway: push the new mitmproxy CA into each agent's trust store,
re-provision git-gate repos/creds from the persisted upstreams snapshot,
and restore egress tokens. Per-bottle failures are logged and skipped so
one unreachable VM does not block the rest.

New modules / changes:
- backend/firecracker/reconcile.py: attach_bottled_agents_to_gateway,
  _push_ca, _reprovision_git_gate, _guest_ip_from_config
- git_gate/provision.py: write upstreams.json after key provisioning so
  the bring-up reconcile can reconstruct the upstream table without the
  manifest
- backend/firecracker/infra.py: call attach_bottled_agents_to_gateway in
  the cold-boot branch of ensure_running()
- backend/base.py: no-op default on BottleBackend
- backend/firecracker/consolidated_launch.py: remove superseded
  _reprovision_running_bottles / _guest_ip_from_config
- orchestrator: OrchestratorCore.update_agent_secret + POST
  /bottles/<id>/secret + client.update_agent_secret (single-secret
  in-place update, the reusable primitive from the 2026-07-26 hotfix)
- tests: 15 new tests in test_firecracker_reconcile.py; updated
  test_firecracker_infra.py cold-boot cases; stale FC reprovision tests
  removed from test_backend_secret_reprovision.py

Closes #516.
2026-07-28 01:50:15 +00:00
didericis-codex 6f997bf118 test(release): cover publication failure paths
test / unit (push) Successful in 1m0s
test / image-input-builds (push) Successful in 58s
test / integration-docker (push) Successful in 1m3s
Update Quality Badges / update-badges (push) Successful in 1m6s
test / coverage (push) Successful in 14s
lint / lint (push) Successful in 3m47s
2026-07-27 17:00:50 -04:00
didericis-codex 2fb08fc976 ci: rerun release flow checks 2026-07-27 17:00:50 -04:00
didericis-codex 509eb73497 docs(prd): finalize complete release flow 2026-07-27 17:00:50 -04:00
didericis-codex df9c4ca59b feat(release): automate artifact and tag flows 2026-07-27 17:00:50 -04:00
didericis-codex 6ffa8d8843 feat(firecracker): pull pinned agent images 2026-07-27 17:00:50 -04:00
didericis-codex e3d17dbc07 feat(release): publish qualified channel pointers 2026-07-27 17:00:50 -04:00
didericis-codex 1b86847f0d feat(installer): select immutable release bundles 2026-07-27 17:00:50 -04:00
didericis-codex 9dee92406e feat(release): publish immutable commit bundles 2026-07-27 17:00:50 -04:00
didericis-codex 309ec34f29 feat(release): pin first-party agent images 2026-07-27 17:00:50 -04:00
didericis-codex 9eab5be2c4 feat(release): define commit-addressed bundle index 2026-07-27 17:00:50 -04:00
didericis-codex b3834ebf66 docs(prd): define complete release flow 2026-07-27 17:00:50 -04:00
didericis-codex e2857f1eeb fix(release): publish only verified wheel 2026-07-27 17:00:50 -04:00
didericis-codex fd529a98b5 test(infra): cover malformed release manifests 2026-07-27 17:00:50 -04:00
didericis-codex 49b407437c docs(prd): activate packaged infrastructure artifacts 2026-07-27 17:00:50 -04:00
didericis-codex 8315e4192a feat(infra): pull artifacts pinned by package release 2026-07-27 17:00:50 -04:00
didericis-codex 938df8513f docs(prd): define packaged infrastructure artifacts 2026-07-27 17:00:50 -04:00
didericis-codex ec38458c94 fix(firecracker): include copied context directories
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
tracker-policy-pr / check-pr (pull_request) Successful in 9s
test / image-input-builds (pull_request) Successful in 39s
test / unit (pull_request) Successful in 50s
test / integration-docker (pull_request) Successful in 1m0s
test / coverage (pull_request) Successful in 17s
test / image-input-builds (push) Successful in 43s
lint / lint (push) Successful in 1m0s
test / integration-docker (push) Successful in 1m4s
test / unit (push) Successful in 3m4s
test / coverage (push) Successful in 18s
Update Quality Badges / update-badges (push) Failing after 10m3s
2026-07-27 17:53:48 +00:00
didericis f6df85a8cd fix(fc): ship agent-image build context files to the infra VM
test / integration-docker (pull_request) Has been cancelled
prd-number-check / require-numbered-prds (pull_request) Successful in 11s
lint / lint (push) Successful in 59s
test / unit (pull_request) Successful in 1m0s
test / coverage (pull_request) Has been skipped
test / image-input-builds (pull_request) Successful in 1m1s
tracker-policy-pr / check-pr (pull_request) Successful in 17s
Agent images build with buildah inside the infra VM, and
`_build_in_infra` shipped only the Dockerfile into an empty `{ctx}/ctx`
context — on the documented assumption that agent Dockerfiles COPY
nothing from the context. The "pin & verify image inputs" work broke
that: the codex Dockerfile COPYs `codex-package_SHA256SUMS` and the
claude/pi Dockerfiles COPY their npm `package.json`/`package-lock.json`,
so any fresh agent image build fails with `COPY ...: no such file or
directory` (existing agents only survive on a rootfs cached from before
the change).

Parse each Dockerfile's context COPY sources, tar those committed files
from the build root into the VM-side `{ctx}/ctx`, and fold their content
into the rootfs cache digest so a repinned input rebuilds instead of
reusing stale bytes. `COPY --from=<stage>` and absolute/traversing
sources are excluded.

Closes #538.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 13:18:15 -04:00
didericis 9bf2961d13 fix: name an absolute path in the sudo re-run hint
test / unit (push) Successful in 57s
test / image-input-builds (push) Successful in 1m2s
Update Quality Badges / update-badges (push) Successful in 1m8s
test / integration-docker (push) Successful in 58s
test / coverage (push) Successful in 15s
lint / lint (push) Failing after 10m9s
The firecracker setup message told users to run

    sudo bot-bottle backend setup --backend=firecracker

which fails for exactly the users who followed the documented install. sudo
replaces PATH with sudoers' secure_path — /usr/local/sbin:/usr/local/bin:
/usr/sbin:/usr/bin:/sbin:/bin on Debian and Ubuntu — which deliberately
excludes user-writable directories. Both supported install paths land in one:
pipx uses ~/.local/bin, and install.sh's venv fallback symlinks there. So the
hint works for anyone who installed system-wide and breaks with "command not
found" for everyone else, which is how it survives a read-through.

Add bot_bottle/invocation.py: self_path() resolves the running entry point to
an absolute path, and sudo_command() builds the copy-pasteable form. The one
sudo recommendation in the tree now uses it. Non-sudo hints keep the bare
`bot-bottle`, which is correct — the user reached them by running it.

self_path() falls back to the bare name when argv[0] cannot be resolved (a
`python -m` style invocation), because a slightly wrong hint beats a traceback
raised while reporting some unrelated problem.

Tested behaviourally rather than by scanning source: the first version of the
test grepped the module and failed on the comment explaining why the bare form
is wrong. It now drives _setup_systemd() as non-root and asserts what is
actually printed. Verified the guard bites by restoring the bare form and
watching it fail.

Not verified end to end: this message only prints on the systemd path, so it
is unreachable on macOS, where the rest of this work was tested.
2026-07-27 12:35:51 -04:00
didericis 6385752040 feat: get Alpine and NixOS cells to green
Completes the matrix: all five distros now pass both `test` and `test-ready`,
validated end-to-end on the delphi KVM host (QEMU 11.0.2).

Alpine — the generic_ cloud image silently ignores a NoCloud seed, so cloud-init
never ran. Fixes, all confirmed by booting:
- switch to the nocloud_ image variant (built for a local seed),
- start sshd from a runcmd (OpenRC doesn't auto-start it after key injection),
- give the account a throwaway password — Alpine's non-PAM sshd refuses pubkey
  auth for a cloud-init-locked ('!') account, unlike the UsePAM=yes distros,
- install sudo via cloud-init packages: (the minimal image has none, so the
  test-ready `sudo apk add` prereq failed).
Also add instance-id to the NoCloud meta-data, which the nocloud image requires.

NixOS — publishes no downloadable cloud qcow2 (Hydra builds AMIs), so the harness
now BUILDS one with nixos-generators (new scripts/linux-install-test-nixos.nix:
cloud-init for the key, flakes enabled, deliberately no python/git/pipx).
ensure_base_image branches to build_nixos_image for the nixos distro. The
test-ready prereq uses `nix profile install` pinned to nixpkgs/nixos-24.11,
because the guest's default unstable registry builds pipx from source and its
test suite currently fails to build.

Validation: 10/10 cells pass, clean teardown, no leaked VMs or run dirs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 496608fc25 fix: correct harness defects found running the matrix
First full run on the KVM host surfaced three harness bugs and two stale
image URLs; all fixed here.

Harness bugs:
- Liveness probe used `bot-bottle --version`, which the CLI does not implement
  (unknown args die non-zero) — so every SUCCESSFUL install was misreported as
  "no runnable entry point". Switched to `bot-bottle --help`, which exits 0
  before any DB/migration/network work.
- cmd_down never removed serial.log, so its rmdir failed and every run left an
  orphan scratch dir behind. Added serial.log to the cleanup.
- The teardown trap was armed AFTER cmd_up, but cmd_up's wait_for_ssh can fail
  with QEMU already running (a guest that never opens SSH) — leaking the VM.
  Arm the trap before cmd_up.

Stale image URLs:
- Fedora 41 is EOL and 404s; bumped to Fedora 44 (44-1.7).
- Alpine bumped to 3.21.7; its cloud images ship a .sha512 only (this verifier
  is sha256), so SUM_URL is now empty (skip) with a note.

Validated on delphi (QEMU 11.0.2): ubuntu/fedora/arch pass BOTH test and
test-ready. Alpine (cloud-init seed not applied → no SSH) and NixOS (no
upstream cloud qcow2) remain blocked on image provisioning, documented in the
research note as follow-ups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 218f29cb05 refactor: adopt macOS test/test-ready convention for variants
Replaces the BB_TEST_PREREQS env toggle with the two subcommands the macOS
harness established, so the two repos read the same:

  test        A bare host — prerequisites NOT set up (the default state of a
              stock cloud image). Exercises install.sh's own prerequisite-guard
              logic. SOUND (PASS) when install.sh either installs cleanly or
              declines with one of its own recognized, actionable errors; a
              crash or unrecognized failure fails.
  test-ready  Prerequisites satisfied — the harness installs python3/git/pipx
              first, then runs install.sh, which must actually land: entry point
              runnable, doctor reporting a usable python and config.

Structure now mirrors scripts/macos-install-test.sh: a shared _test_cycle
driving up -> (prereqs) -> run -> verdict -> down, thin cmd_test / cmd_test_ready
wrappers setting _STEPS / _PASS_CLAIM / _REQUIRE_INSTALL, a standalone `prereqs`
subcommand, and a _test_teardown that prints the PASS/FAIL claim.

The doctor check is now the macOS-style classifier rather than a bare exit-code
read: a Traceback is an install defect (fail), a missing `ok: python:` /
`ok: config:` line is a fail, and a not-ready backend is reported, not fatal
(BB_TEST_REQUIRE_BACKEND=1 makes it fatal for a nested-virt host). This is where
Linux diverges from macOS test-ready — a plain VM has no nested KVM for
Firecracker and the harness doesn't provision the Docker backend, so backend
readiness is never the Linux criterion.

test-all now runs every distro × {test, test-ready}. Research note updated.

Validated with `bash -n` and `shellcheck`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 1518f73de5 feat: add bare-host install variant to the Linux harness
Splits the Linux clean-install harness into two variants, selected with
BB_TEST_PREREQS:

  with    (default) — install python3 + git + pipx first, then install.sh.
                      The ready-host happy path; PASS = install.sh exits 0 and
                      leaves a runnable bot-bottle entry point.
  without           — run install.sh on the BARE cloud image. Exercises
                      install.sh's own prerequisite-guard logic (python gate,
                      git-for-git-specs gate, pipx/pip PEP-668 handling). PASS =
                      install.sh either fully succeeds OR declines with one of
                      its own recognized, actionable prerequisite errors; a
                      crash or unrecognized failure is a FAIL.

cmd_run now captures install.sh's exit code + full output (install.rc /
install.log) instead of aborting on non-zero, so the verdict step applies the
variant's criterion. The old assert_installed is replaced by a quiet
entry_point_runnable helper plus classify_outcome, which matches the bare-host
declines against the exact die() messages install.sh prints. doctor still runs
for visibility, but only when an entry point exists.

test-all now runs the full matrix -- every distro x both variants -- each cell
in its own throwaway VM, and can be pinned to one variant via BB_TEST_PREREQS.
Research note updated to document both variants and their pass criteria.

Validated with `bash -n` and `shellcheck`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis 9d82535390 feat: add Linux clean-install test harness
Adds scripts/linux-install-test.sh, a throwaway-VM harness that exercises
install.sh the way a brand-new user would across a Linux distro matrix
(Ubuntu, Fedora, Arch, Alpine, NixOS), plus the research note
docs/research/testing-clean-install-on-linux.md that motivates the approach.

This is the Linux counterpart to the macOS clean-install harness. On Linux
the boundary of choice flips from a throwaway user account to a disposable
KVM VM: it's a genuine kernel + userland + package-manager boundary that
wipes to nothing on teardown, and a single harness can swap distro cloud
images to cover the several package-management regimes Linux fragments into.
The host already needs KVM for the Firecracker backend, so a per-run,
copy-on-write VM (qemu-img create -b base) is cheap here -- the equivalent
of `docker run --rm`, for a whole machine.

Per run the harness caches one read-only base image, boots a throwaway
overlay via QEMU/KVM with user-mode networking (no root, no bridge), injects
an ephemeral SSH key + passwordless login through a cloud-init seed ISO,
installs the distro's prerequisites (python3 + git + pipx), pipes THIS
checkout's install.sh into the guest exactly as `curl ... | sh` would, and
asserts the CLI installed cleanly. The overlay is deleted on teardown, so
even the OS-level prerequisites are wiped -- unlike a throwaway user, the
reset is total.

Subcommands mirror the macOS harness (up/run/status/down/test) plus test-all
(the matrix) and ssh (an interactive guest shell). test arms an EXIT/INT/TERM
trap the moment the VM exists, so a failure or Ctrl-C still tears it down.

Scope is installer correctness, not runtime: there is no nested KVM/Docker in
the VM, so `bot-bottle doctor` correctly reports every backend not-ready and
exits non-zero by design. The pass criterion is therefore install.sh exiting
0, the bot-bottle entry point being present on the fresh user's PATH, and
`bot-bottle --version` running -- not a green doctor. doctor's output is still
printed so a real installer regression (broken shim, import error) stays
visible.

Validated with `bash -n` and `shellcheck`. Runtime is host-only (needs
/dev/kvm, qemu, cloud-localds), so like the macOS harness it isn't exercised
by the Linux PR CI. The cloud-image URLs in the DISTRO table are the one
place to bump when a distro cuts a newer build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:24:31 -04:00
didericis c89847b626 fix(macos-container): tag the pinned base from its ref, not its ID
lint / lint (push) Successful in 59s
Update Quality Badges / update-badges (push) Successful in 1m6s
test / coverage (push) Successful in 22s
test / image-input-builds (push) Successful in 58s
test / integration-docker (push) Successful in 58s
test / unit (push) Successful in 2m51s
`container image tag` only accepts image-name[:tag] as its source and
rejects a bare 64-hex image ID ("cannot specify 64 byte hex string as
reference"), so pinning the agent image for the nested-containers layer
died before the build could start. Tag from the ref instead; the
existing post-tag inspection is what keeps the handoff fail-closed if
the ref moves between the two commands.

The unit test mocked subprocess and asserted the ID-as-source call
shape, so it never saw the CLI's rejection. It now pins the ref as the
source, and covers the bare-hex ID that `container image inspect`
actually reports plus the mid-flight tag move.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 10:56:59 -04:00
didericis cd9f023f3d docs: name bot-bottle, not ./cli.py, in every instruction
prd-number-check / require-numbered-prds (pull_request) Successful in 11s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / unit (pull_request) Successful in 59s
test / integration-docker (pull_request) Successful in 1m6s
test / coverage (push) Successful in 21s
test / image-input-builds (push) Successful in 44s
test / image-input-builds (pull_request) Successful in 1m15s
test / unit (push) Successful in 57s
test / coverage (pull_request) Successful in 20s
lint / lint (push) Successful in 1m3s
Update Quality Badges / update-badges (push) Successful in 1m12s
test / integration-docker (push) Successful in 1m0s
`doctor` told users to run `./cli.py backend setup`. cli.py is a four-line
wrapper at the repo root that calls bot_bottle.cli:main, and it does not ship —
[tool.setuptools.packages.find] includes only bot_bottle*, so anyone who
installed rather than cloned has no such file. Confirmed against a real
install: the user's tree contains exactly one executable, `bot-bottle`, and no
cli.py anywhere, while doctor recommended `./cli.py` three times. Invisible in
development, where ./cli.py works fine from a checkout, which is why only a
clean-install test surfaced it.

The two entry points are the same code, so the fix is to name the one that
always exists. 141 replacements across 45 files: the runtime messages that
caused this, plus README, docs, PRDs, research notes and test prose, so nothing
teaches the invocation a user cannot run.

scripts/demo.sh is deliberately untouched — it *executes* ./cli.py from a
checkout, where that is the correct and available path.

Verified end to end: a sandbox install now reports
"Run: bot-bottle backend setup --backend=firecracker", and bot-bottle is on
that user's PATH. Unit suite unchanged against the pre-existing baseline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 10:20:17 -04:00
didericis 5c499d290b docs: record what the passing runs established
test / image-input-builds (pull_request) Successful in 47s
test / unit (pull_request) Successful in 59s
test / integration-docker (pull_request) Failing after 2m50s
test / coverage (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 12m32s
prd-number-check / require-numbered-prds (pull_request) Failing after 12m45s
Both variants now pass, and the with-prerequisites run settled several things
that were guesses before:

* The Apple `container` service is per-user, confirmed directly rather than
  inferred: the throwaway account's appRoot is its own
  (/Users/bbtest/Library/Application Support/com.apple.container/), and
  starting it left the admin's service running and its agents intact.
* Setting up a new account is two steps, not one. The guest kernel lives in
  that same per-user app root, so a fresh account has none and
  `container system start` prompts to download it — and only prompts, since
  the flags default to asking. Headless callers need
  --enable-kernel-install.
* Neither step is performed by `bot-bottle backend setup
  --backend=macos-container`, which checks and then defers to `container
  system start`. So the real path for a new account is
  `container system start --enable-kernel-install`.
* Any of it requires entering the user's launchd domain with `launchctl
  asuser`, because the apiserver is a per-user agent reached over XPC.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 10:12:21 -04:00
didericis 0500ae5f4c fix(harness): make the prereq step non-interactive
prd-number-check / require-numbered-prds (pull_request) Successful in 16s
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / image-input-builds (pull_request) Successful in 42s
test / unit (pull_request) Successful in 53s
test / integration-docker (pull_request) Successful in 1m1s
test / coverage (pull_request) Failing after 15s
The launchd fix worked. `container system start` now gets past "Testing access
to container-apiserver" and "Verifying machine API server is running" — the two
steps it could not reach before — so the XPC domain problem is solved. It then
failed somewhere new:

    No default kernel configured.
    Error: failed to read user input
    Install the recommended default kernel from [...kata-static...]? [Y/n]:

`container system start` PROMPTS for the default Linux kernel when given
neither --enable-kernel-install nor --disable-kernel-install ("default: prompt
user"). A throwaway account always hits that prompt, because the kernel lives
in the per-user app root and a fresh home has none — and with no TTY the
prompt dies immediately.

Pass --enable-kernel-install. That is also the honest choice for what this
variant claims: the backend cannot run a bottle without a guest kernel, so
"prerequisites satisfied" has to include it. The cost is a kernel download per
run, since the previous run's copy went with the deleted home;
BB_TEST_KERNEL_INSTALL=0 switches to --disable-kernel-install when you only
care that the service comes up.

This also means the per-user prerequisite is not one step but two — start the
service, install a kernel — which is worth knowing for anyone setting up a
second account by hand.

The rig's stub now models the prompt: an unflagged start fails the way the real
CLI does. Verified that guard bites by removing the flag and watching the
scenario fail, so a future edit cannot silently reintroduce an interactive
prereq step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 10:00:49 -04:00
didericis 5614a56ccd fix(harness): run as the user inside their launchd domain
prd-number-check / require-numbered-prds (pull_request) Successful in 11s
test / image-input-builds (pull_request) Successful in 44s
test / unit (pull_request) Successful in 57s
test / integration-docker (pull_request) Failing after 3m4s
test / coverage (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 13m12s
`test` passes. `test-ready` got as far as the service start and failed:

    Launching container-apiserver...
    Error: failed to get a response from apiserver: invalidState: "unauthorized request"

which is the limitation this script's own header predicted — "may need a full
launchd user session (`launchctl asuser`)".

`container system start` registers com.apple.container.apiserver as a per-USER
launchd agent and then talks to it over XPC. `launchctl list` confirms the
shape: the apiserver and every container-network/runtime job are agents in the
invoking user's domain. Under plain `sudo -u` the caller stays in root's
bootstrap namespace, so the lookup crosses domains and the apiserver rejects it
as unauthorized — the agent launched fine, the client just could not reach it.

run_as_user now enters the target user's domain with `launchctl asuser <uid>`,
which is also what a real user gets from Terminal, so it is the more faithful
way to run everything here rather than a special case for the service start.

A never-GUI-logged-in account may have no bootstrappable domain at all, so
availability is probed once and cached, and everything falls back to plain
`sudo -u` when it is missing. Without that fallback a missing domain would
break `test` — which passes today and does not need a session — for a reason
unrelated to what it tests. The rig covers that path explicitly.

If the start still fails, the error now names the cause and the way out
(log into the account once to get it a domain) instead of just relaying the
CLI's message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:56:51 -04:00
didericis 71c0f8ed88 feat(harness): add test-ready, the with-prerequisites variant
prd-number-check / require-numbered-prds (pull_request) Successful in 11s
tracker-policy-pr / check-pr (pull_request) Successful in 21s
test / image-input-builds (pull_request) Successful in 48s
test / integration-docker (pull_request) Successful in 1m4s
test / unit (pull_request) Successful in 2m52s
test / coverage (pull_request) Failing after 20s
"Does the installer work" and "can a new user actually run a bottle" are
different questions, and collapsing them is what made the previous run
ambiguous. Split them into two cycles over the same throwaway account:

  test        a macOS system WITHOUT the prerequisites set up for this user,
              which is the default state of every new account since the Apple
              `container` service is per-user. Asserts the install is sound;
              reports backend readiness without failing on it, because
              install.sh provides no backend and cannot regress one.

  test-ready  the same system WITH them. Runs `container system start` for the
              throwaway user, then demands doctor go fully green, backend
              included — the end-to-end claim.

The new `prereqs` step runs `container system start` rather than
`bot-bottle backend setup --backend=macos-container`, because that subcommand
does not start anything: it checks, then tells you to run `container system
start` yourself (backend/macos_container/setup.py, "no privileged host setup
required"). The harness runs what the product actually asks for.

test-ready sets BB_TEST_REQUIRE_BACKEND, so it demands exactly what `test`
merely reports. Both share one cycle function; the step count and the PASS
claim are the only differences beyond the extra step, so neither variant can
drift from the other's teardown or dirty-account guarantees.

Rig covers the distinguishing case directly: identical host state where `test`
passes on install soundness and `test-ready` starts the service and reaches a
green backend, plus the service-start failure and an install failure under
test-ready. 22 scenarios, all passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:51:35 -04:00
didericis 556217ae7b fix(harness): judge the install separately from backend readiness
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
test / image-input-builds (pull_request) Successful in 46s
test / integration-docker (pull_request) Successful in 1m3s
test / unit (pull_request) Successful in 2m58s
test / coverage (pull_request) Failing after 18s
tracker-policy-pr / check-pr (pull_request) Failing after 14m1s
The first run against the branch's own code confirmed the doctor fix — no
traceback, the firecracker probes now report "TAP pool: 0/8" instead of dying
on PermissionError — and then failed for a reason the installer cannot fix.

The Apple `container` service is per-USER. `container system status` reports an
appRoot under ~/Library/Application Support/com.apple.container/, and bbtest
sees "container system service: NOT running" while the creating admin's is
running. A brand-new account therefore has no backend until it runs
`container system start` once, doctor rightly fails, and `test` would be
permanently red for host state that install.sh neither creates nor can regress.

So stop treating doctor's exit code as one verdict. `test` now fails when the
install is broken — no entry point, doctor crashed, or doctor could not report
a usable python and config dir — and reports backend readiness as a note.
BB_TEST_REQUIRE_BACKEND=1 restores the strict behaviour.

A traceback is checked for explicitly rather than inferred from the exit code,
because those are the same value. The PermissionError bug exited non-zero
exactly like a missing prerequisite does; only the traceback distinguishes a
defect from an environment fact, and that distinction is the whole point of
this change.

The research doc's footprint table had the service as a host-wide launchd
service that survives user deletion. It does not. The state lives in the home
and goes with it, so the reset is more complete than claimed — but the
corollary is that a fresh account cannot run a bottle until the service is
started for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:45:57 -04:00
didericis 7461802b62 fix(harness): install the branch under test, and stop mangling snippets
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / unit (pull_request) Successful in 1m3s
test / integration-docker (pull_request) Successful in 1m2s
test / image-input-builds (pull_request) Successful in 1m6s
test / coverage (pull_request) Failing after 20s
lint / lint (push) Successful in 3m30s
Two harness bugs, both of which made the last run lie about what it verified.

The run installed main. install.sh's default spec is the repo's default
branch, so `run` piped *this checkout's* installer into the throwaway user and
then had it install a package that does not contain the branch's changes —
which is why the doctor PermissionError fix appeared not to work: it was never
in the code under test. The clone line said so plainly (commit 420184b, not
this branch's HEAD) and I read past it. The spec now pins to the current
branch, over https because the throwaway user has neither our SSH key nor read
access to this mode-700 checkout. Since what it clones is whatever the remote
has, a dirty tree or an unpushed branch now prints a warning rather than
quietly testing something other than what is on disk.

The doctor probe also died with "sh: -c: line 1: syntax error: unexpected end
of file". `sudo -i` joins its argv into one string for the login shell's -c, so
an argument's quoting is not preserved and a newline terminates the command.
The multi-line snippet added in the previous commit could not survive that.
run_as_user now feeds snippets on stdin to `sh -s`, which is the same trick
install.sh already arrives by, and is immune to the joining. The install spec
moves into the stream as an export for the same reason — as an argument it had
the identical latent quoting bug, unnoticed only because the default spec has
no shell metacharacters in it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:39 -04:00
didericis 5c12c6bf4e fix(doctor): survive a PATH entry the user can't execute
The install itself now works end to end on a fresh macOS account — venv built,
package installed from git, entry point linked — and `doctor` then died with an
unhandled traceback:

    PermissionError: [Errno 13] Permission denied: 'ip'

netpool's probe helpers caught only FileNotFoundError. That is not the only way
a probe binary can be unavailable: when a name on PATH exists but this user
cannot execute it, exec fails with EACCES, and CPython reports that in
preference to the ENOENT from the other PATH entries. So `except
FileNotFoundError` misses it and the crash propagates all the way out of
`doctor`. Reproduced directly: a mode-000 file named `ip` on PATH yields
exactly the error above.

_run_ok's docstring already stated the intent — treat an unavailable binary as
failure rather than crashing — so this widens the catch to OSError to match
what it says. Any OSError means the probe could not run, which for a
fail-closed check is indistinguishable from "not present". The same narrow
catch is fixed in overlapping_routes and in the two docker probes
(compose ls, docker ps), which are the same shape and equally reachable.

Deliberately not touched: the FileNotFoundError catches around file I/O in
bottle_state and orchestrator/service, where the narrow exception is correct.

The harness also required `bot-bottle` on PATH before running doctor, which
could never be true: install.sh prints the PATH line rather than editing a
shell profile, by design, so on a fresh account the entry point is installed
and working but not on PATH. It now looks where the installer actually puts
it (~/.local/bin, then the venv) and notes when it's running by absolute path.
That was the harness failing a run for a reason the installer intends.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:39 -04:00
didericis 4096409495 fix(install): name the profile the user's shell actually reads
The PATH hint hardcoded ~/.zprofile, which is right for macOS's zsh and wrong
for the Linux users this installer also serves. Pick from $SHELL instead.
~/.profile is the default for non-zsh rather than ~/.bash_profile, because
creating the latter would shadow an existing ~/.profile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis 8d0782d1c7 fix(install): install into a private venv when pipx is absent
The harness's second run hit the wall the first one predicted: a fresh account
has no pipx, so install.sh fell to `pip install --user`, and every Python a Mac
offers — Homebrew and python.org alike — is externally managed, so PEP 668
blocked it. That fallback was never a fallback on macOS; it was a dead end that
printed instructions.

Replace it with a venv at ~/.bot-bottle/venv (BOT_BOTTLE_VENV to move it),
with the console script symlinked into ~/.local/bin. PEP 668 does not apply
inside a venv, and venv is stdlib, so unlike pipx there is nothing to bootstrap
first. pipx stays the preferred path when present, so anyone already managing
their Python apps that way is unaffected — and the post-install PATH check now
asks pipx for PIPX_BIN_DIR instead of assuming ~/.local/bin.

Keeping the venv under ~/.bot-bottle rather than ~/.local/share means the whole
footprint stays in one directory, which is what lets the throwaway-account
teardown remain a complete reset.

This removes the PEP 668 pre-flight and the sysconfig user-scheme lookup, both
of which existed only to serve the --user path. Their tests go with them:

* `detects_externally_managed_python` asserted the check that is now moot;
  replaced by one asserting pipx is still preferred when present.
* `checks_pip_usable_before_fallback` pinned a pip probe that no longer runs;
  replaced by one asserting the venv's own pip does the install, since using
  the base interpreter's would install outside the venv.
* `resolves_user_scripts_dir_not_hardcoded` and
  `macos_user_scheme_is_not_dot_local_bin` guarded the ~/Library/Python
  scripts-dir lookup. Nothing installs there now. The surviving "don't
  hardcode" concern is pipx's bin dir, which has its own test.

Five tests are added for the new path: the venv fallback exists, no --user path
survives, the venv is under the config dir, venv creation failure names
python3-venv (Debian ships it separately), and the entry point is exposed
outside the venv.

Verified end to end in a sandbox HOME with a fresh-account PATH and no pipx:
venv built, package installed, symlink created, `doctor` reached and green
(python 3.14.5, macos-container ready), exit 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis 6793a09e2b docs: correct two claims the live harness run disproved
The PATH story was wrong in an instructive way. I said a Homebrew Python is on
PATH only because of a shell-profile line; on this host /etc/paths.d/homebrew
puts /opt/homebrew/bin on every login shell's PATH, fresh accounts included.
The stub still wins, because path_helper appends /etc/paths.d/* *after*
/etc/paths and /usr/bin is in the latter. Ordering, not absence, is what makes
bare `python3` the 3.9.6 stub — which is also why the versioned `python3.14`
candidate is the one that matched during the real run, rather than the
/opt/homebrew/bin/python3 fallback I expected.

The harness header also credited install.sh with writing a PATH line into the
login shell. It does not write one. The reset argument is unaffected (such a
line would live in the deleted home either way), but the claim was untrue.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis af293d7036 fix(install): find a usable python instead of dead-ending on PATH's
The macOS install harness caught this on its first real run: a throwaway
account gets `bot-bottle install: error: python3 3.11 or newer is required`
and stops. A fresh account's PATH is just /etc/paths, which excludes
/opt/homebrew/bin, so `python3` resolves to the Command Line Tools stub —
still 3.9.6 on macOS 26. Homebrew's shellenv line lives in the *installing*
user's ~/.zprofile and is inherited by nobody. The documented `curl … | sh`
path therefore dead-ends for anyone whose profile isn't already set up, which
is every new user, launchd job, and CI runner.

So look past PATH before giving up: try `python3`, then python3.11-3.14, then
/opt/homebrew/bin, /usr/local/bin, ~/.local/bin, and python.org framework
builds, and say which one was picked when it isn't the obvious one. On this
host that turns the failure into a successful install.

The chosen interpreter is now threaded through everything downstream — the pip
probe, the PEP 668 check, the pip --user fallback, and the user-scripts-dir
lookup — which were all still hardcoding `python3` and would otherwise have
run against the 3.9 stub we just rejected. `pipx install` gains `--python`,
since pipx otherwise builds the venv with whichever interpreter pipx itself
was installed with, not the one that passed the version check.

When nothing usable is found the error is now actionable: what was found and
why it's insufficient, where else it looked, a platform-appropriate install
command, and BOT_BOTTLE_PYTHON to point at an interpreter directly. An
explicit BOT_BOTTLE_PYTHON that is too old or unusable is an error rather than
a silent fallback to a different interpreter than the caller asked for.

Three existing tests asserted on incidental literals rather than the behaviour
they describe, and are narrowed to their actual intent:

* `never_uses_sudo` matched the word anywhere, including the new "sudo apt
  install python3.12" remediation *advice*. It now strips string literals and
  comments first, so it still catches sudo as a bare command, in a pipeline,
  and in a command substitution — verified by mutation — while allowing the
  script to print the word.
* `checks_pip_usable_before_fallback` pinned the literal `python3 -m pip`.
* `resolves_user_scripts_dir_not_hardcoded` banned ".local/bin" script-wide;
  it's now scoped to the USER_SCRIPTS assignment it exists to guard, since
  ~/.local/bin is a legitimate place to *find an interpreter*.

README grows the install command and a Requirements section it never had,
leading with the Python floor and why a working `python3` in your own shell
says nothing about what a fresh account sees.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis d415581adc feat: add a one-shot test cycle to the macOS install harness
`test` chains up -> run -> status -> down, which is the loop you actually want
when verifying a clean install. Three things make it more than a convenience
wrapper:

* It refuses to start against an existing account. A reused home is not a clean
  install, so testing one silently would defeat the harness.
* It tears the account down from an EXIT/INT trap armed the moment the account
  exists, so a failed run — or a Ctrl-C mid-install — still leaves the machine
  clean. BB_TEST_KEEP=1 opts out to poke at a failure.
* Its verdict is stricter than the installer's. install.sh exits 0 when it
  finishes but `doctor` reports unmet prerequisites, so "the installer
  succeeded" is not a useful assertion; `test` fails if the install fails, if
  bot-bottle never reached the new user's PATH, or if doctor is unhappy. That
  meant giving cmd_status a real exit status instead of swallowing doctor's.

Also fixes two bugs in `run`'s installer staging, by removing the staging
entirely and feeding install.sh in on stdin:

* `mktemp /tmp/bb-install.XXXXXX.sh` did not do what it looks like. BSD mktemp
  only substitutes trailing Xs, so every run wrote the *same* predictable path,
  as root, mode 644, in a world-writable directory.
* The cleanup only ran on the normal and failure returns, so an interrupted run
  leaked the file.

Piping on stdin sidesteps both: root opens the redirect before sudo drops
privileges, so the mode-700 home that motivated the staging is a non-issue,
there is no file to leak, and `sh -s` matches the documented `curl … | sh`
shape more closely than executing a staged copy did.

The command-level `exit 1`s become `return 1` so the steps compose under the
trap, and the "next, run this" hints are suppressed inside `test`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
2026-07-27 09:38:20 -04:00
didericis-claude 27b9ba5247 feat: add macOS clean-install test harness
Add scripts/macos-install-test.sh, a throwaway-user harness for exercising
install.sh the way a brand-new user would on macOS, plus the research note
that motivates the approach.

The harness has up/run/status/down/deep-reset subcommands. Because install.sh
writes only to the user home (pipx venv, ~/.bot-bottle, a PATH line) and never
installs the backend, deleting the account is a complete, deterministic reset
of the install surface. A disposable macOS VM can't stand in on M1/M2: the
Apple `container` backend needs Virtualization.framework, and running it inside
a guest VM requires nested virtualization (M3+ only), so a throwaway user is
the only way to reach the real host backend from a clean $HOME.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 09:38:20 -04:00
didericis-codex 420184b874 fix: close consolidated quality review findings
test / image-input-builds (push) Successful in 44s
lint / lint (push) Successful in 1m3s
test / unit (push) Successful in 1m4s
test / integration-docker (push) Successful in 1m6s
test / coverage (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 2m57s
2026-07-27 09:26:25 -04:00
didericis-codex 7938b90d19 fix: enforce shared storage permissions 2026-07-27 09:26:25 -04:00
didericis-codex d879258f62 fix: bound heavy gateway operations 2026-07-27 09:26:25 -04:00
didericis-codex 78d3b43061 fix: enforce cleanup and secret integrity 2026-07-27 09:26:25 -04:00
didericis-codex cdc5c8203d fix(cleanup): use authoritative resource identities 2026-07-27 09:26:25 -04:00
didericis-codex d04bbf1454 fix(gateway): contain output pump shutdown races 2026-07-27 09:26:25 -04:00
didericis-codex c00097d998 fix(gateway): bound stdlib HTTP request work 2026-07-27 09:26:25 -04:00
didericis-codex 6630a1f701 fix(cleanup): revalidate destructive backend plans 2026-07-27 09:26:25 -04:00
didericis-codex a7ad82f03a refactor(supervisor): separate MCP dispatch from transport 2026-07-27 09:26:25 -04:00
didericis-codex 9b8f126b8f refactor(egress): extract outbound DLP request stage 2026-07-27 09:26:25 -04:00
didericis-codex d167c03215 refactor(egress): extract request policy stages 2026-07-27 09:26:25 -04:00
didericis-codex 479b93bd0b fix(orchestrator): bound streamed request bodies 2026-07-27 09:26:25 -04:00
didericis-codex 2de61eeb17 fix(orchestrator): keep host client dependency-free 2026-07-27 09:26:25 -04:00
didericis-codex a4bc5d4508 refactor(orchestrator): replace manual HTTP dispatch with FastAPI 2026-07-27 09:26:25 -04:00
didericis-codex 7ebd067d2c build(orchestrator): pin FastAPI runtime dependencies 2026-07-27 09:26:25 -04:00
didericis-codex be50354100 docs(prd): require shared storage permissions 2026-07-27 09:26:25 -04:00
didericis-codex e1163184c9 docs(prd): bound heavy gateway operations 2026-07-27 09:26:25 -04:00
didericis-codex 70f6a9be20 docs(prd): require cleanup execution integrity 2026-07-27 09:26:25 -04:00
didericis-codex aa43fd032f docs(prd): extend authoritative cleanup boundaries 2026-07-27 09:26:25 -04:00
didericis-codex be5c803e8e docs(prd): define authoritative failure boundaries 2026-07-27 09:26:25 -04:00
didericis-codex 38ff4a5e88 fix(orchestrator): satisfy adapter type contracts 2026-07-27 09:26:25 -04:00
didericis-codex 8df76f6126 refactor(egress): split request policy pipeline stages 2026-07-27 09:26:25 -04:00
didericis-codex b63c41db9c feat(firecracker): enumerate running bottles 2026-07-27 09:26:25 -04:00
didericis-codex 6c52686069 fix(orchestrator): bound unauthenticated HTTP requests 2026-07-27 09:26:25 -04:00
didericis-codex a52245af95 fix(security): authenticate persisted egress secrets 2026-07-27 09:26:25 -04:00
didericis-codex d4e48a4169 fix(docker): fail closed on network address scan errors 2026-07-27 09:26:25 -04:00
didericis-codex 53cc3ca492 fix(firecracker): abort cleanup on scan errors 2026-07-27 09:26:25 -04:00
didericis-codex bafb73fdb9 refactor(backend): type enumeration failures 2026-07-27 09:26:25 -04:00
didericis-codex b0f317c499 fix(docker): surface enumeration failures 2026-07-27 09:26:25 -04:00
didericis-codex 57eb4394dc fix(security): prohibit unauthenticated orchestrator 2026-07-27 09:26:25 -04:00
didericis-codex 82e1a353bd docs: analyze Sandbox Agent SDK architecture 2026-07-27 03:53:02 +00:00
didericis-codex a6e1aebda1 docs: update agent sandbox competitor landscape 2026-07-27 03:44:41 +00:00
didericis-claude 74ec9843f0 chore(coverage): relax thresholds to cut low-value churn
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / unit (pull_request) Successful in 49s
test / image-input-builds (pull_request) Successful in 50s
test / integration-docker (pull_request) Successful in 56s
test / coverage (pull_request) Successful in 15s
test / image-input-builds (push) Successful in 46s
Update Quality Badges / update-badges (push) Successful in 56s
test / coverage (push) Successful in 22s
test / integration-docker (push) Successful in 1m3s
test / unit (push) Successful in 48s
lint / lint (push) Successful in 2m59s
Lower the diff-coverage gate from 90% to 80% and the critical-module
target from 90% to 85%. The 90% diff gate forced back-fill tests on
nearly every changed line; 80% keeps new code honest without the churn.
Global coverage stays informational per ADR 0004 (no new gate added).

Updates scripts/diff_coverage.py, scripts/coverage.sh,
scripts/critical-modules.txt, .gitea/workflows/test.yml, and records the
change as a dated revision in ADR 0004.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 21:45:08 -04:00
didericis-claude ed9fc76f97 fix(docker): only heal on the ParseAddr poison signature, not any inspect error
prd-number-check / require-numbered-prds (pull_request) Successful in 5s
test / unit (pull_request) Successful in 50s
tracker-policy-pr / check-pr (pull_request) Successful in 6s
test / image-input-builds (pull_request) Successful in 59s
test / integration-docker (pull_request) Successful in 1m6s
test / coverage (pull_request) Successful in 42s
Update Quality Badges / update-badges (push) Successful in 49s
lint / lint (push) Successful in 1m2s
test / coverage (push) Successful in 24s
test / integration-docker (push) Successful in 1m3s
test / unit (push) Successful in 48s
test / image-input-builds (push) Successful in 2m44s
Address review: the self-heal classified every non-absence `network inspect`
failure as a poisoned network and force-removed the shared gateway. But
inspect also fails on transient daemon/API errors, permission failures,
timeouts, or a bad context — destroying a healthy gateway on that guess would
tear the network out from under every live bottle.

Now the destructive path runs only for the known poison signature (docker's
`ParseAddr` error from the malformed `::1/64` IPv6 gateway). The absent case
(`No such network`) still just creates; any other inspect failure raises a
clear GatewayError without mutating shared state.

Adds a regression test asserting a generic inspect error issues neither
`docker rm --force` nor `docker network rm` and surfaces the error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 01:41:09 +00:00
didericis-claude bd8a146a46 fix(docker): heal a poisoned IPv6 gateway network, not just avoid creating one
test / integration-docker (pull_request) Successful in 1m15s
test / image-input-builds (pull_request) Successful in 1m14s
lint / lint (push) Successful in 3m28s
tracker-policy-pr / check-pr (pull_request) Failing after 10m28s
test / unit (pull_request) Failing after 10m39s
prd-number-check / require-numbered-prds (pull_request) Failing after 10m45s
test / coverage (pull_request) Has been skipped
PR #515 stopped bot-bottle from *creating* a gateway network with a
malformed IPv6 subnet (--ipv6=false), but it can't recover a network that
is *already* poisoned. On a daemon that default-enables IPv6, the fixed-name
`bot-bottle-gateway` network gets an `fdd0::/64` subnet whose `::1/64`
gateway trips docker's own netip.ParseAddr, so `docker network inspect`/`ls`
exit non-zero — poisoning every command that reads networks.

Such a network can survive on a shared runner from a pre-fix or concurrent
launch. `_ensure_network` never healed it: its migrate/recreate branch only
ran when `network inspect` *succeeded*, but a poisoned network makes inspect
*fail*, so the code fell through to `network create`, which no-ops on
"already exists" — leaving the poison in place. The next subnet read then
failed with ConsolidatedLaunchError (test_multitenant_isolation), and a bare
`docker network ls` failed too (test_orphan_cleanup).

Fix, two parts:
- gateway.py: when `network inspect` fails for a reason other than "no such
  network", treat the network as poisoned and force-remove + recreate it
  IPv4-only. Absent-vs-poisoned is distinguished by the inspect stderr.
- test.yml: add an integration-docker preflight that drops the leftover
  gateway network (and its container) before the suite, so direct `network
  ls`/`inspect` calls in tests are clean even on a daemon where the in-code
  heal can't run because inspect itself is what's broken.

Adds unit coverage for the poisoned-heal and the absent-create split.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 01:23:25 +00:00
didericis-claude 85fb6b0c98 fix(docker): disable IPv6 on the gateway network
test / unit (push) Successful in 55s
lint / lint (push) Successful in 1m5s
test / image-input-builds (push) Successful in 1m8s
test / integration-docker (push) Has been cancelled
Update Quality Badges / update-badges (push) Failing after 10m53s
test / coverage (push) Has been cancelled
On a docker daemon that default-enables IPv6 (default-address-pools),
creating the gateway network with only `--subnet` lets the daemon also
attach an fdd0::/64 IPv6 subnet. Its gateway is stored as `::1/64`,
which trips docker's own netip.ParseAddr in `network inspect`/`ls`:

  ParseAddr("fdd0:0:0:6::1/64"): unexpected character, want colon

That poisons every `_network_cidr`/`network ls` read and fails the
docker integration suite intermittently (whichever run the runner's
IPv6 pool index lands on a broken network). bot-bottle attribution
pins IPv4 source IPs and has no IPv6 support, so pass `--ipv6=false`
explicitly at network create to keep the gateway network IPv4-only
regardless of the daemon default.

Note: an already-poisoned runner still needs a one-time
`docker network rm bot-bottle-gateway` (and possibly a daemon
restart) to clear the malformed network.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 18:33:48 -04:00
didericis-codex 902286dbc0 fix(build): key nested images by base content
refresh-image-locks / refresh (push) Successful in 36s
lint / lint (push) Successful in 1m2s
test / image-input-builds (push) Successful in 1m6s
test / integration-docker (push) Successful in 1m12s
Update Quality Badges / update-badges (push) Successful in 56s
test / unit (push) Successful in 2m19s
test / coverage (push) Successful in 15s
2026-07-26 17:30:36 -04:00
didericis-codex 73c566f3ff fix(build): close remaining mutable image inputs 2026-07-26 17:30:36 -04:00
didericis-codex 33b7bcd082 build: centralize pinned base image arguments 2026-07-26 17:30:36 -04:00
didericis-codex 9e83ff1992 test: cover image input policy failure paths 2026-07-26 17:30:36 -04:00
didericis-codex 1d10595e5c build: make pinned image inputs portable 2026-07-26 17:30:36 -04:00
didericis-codex 364cad7e56 ci: isolate gateway lock refresh tooling 2026-07-26 17:30:36 -04:00
didericis-codex ff355f81de test: narrow pinned base match for type checking 2026-07-26 17:30:36 -04:00
didericis-codex cf97a49eac ci: pin compatible gateway lock tooling 2026-07-26 17:30:36 -04:00
didericis-codex e607af73ab test: assert reproducible provider image inputs 2026-07-26 17:30:36 -04:00
didericis-codex 998b7c5dd7 ci: make image input refreshes deterministic 2026-07-26 17:30:36 -04:00
didericis-codex 9a4899d4e1 docs: explain image input refreshes 2026-07-26 17:30:36 -04:00
didericis-codex 5083b45f42 ci: reject mutable image build inputs 2026-07-26 17:30:36 -04:00
didericis-codex 24aeba9676 build: freeze operating system package inputs 2026-07-26 17:30:36 -04:00
Gitea Actions ff36b73ff1 build: refresh image input locks 2026-07-26 17:30:36 -04:00
didericis-codex 10d295eaf6 build: pin and verify image inputs 2026-07-26 17:30:36 -04:00
Gitea Actions 69c4c85cd5 build: refresh image input locks 2026-07-26 17:30:36 -04:00
didericis-codex 52e1249c2f ci: keep feature-branch image locks current 2026-07-26 17:30:36 -04:00
didericis-codex bd4f384038 ci: generate refreshed image input locks 2026-07-26 17:30:36 -04:00
didericis-codex 0500e71383 build: declare reproducible image dependencies 2026-07-26 17:30:36 -04:00
didericis-claude 72165b5db5 fix(git-gate): reject AGit review refs in pre-receive
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / unit (pull_request) Successful in 54s
test / integration-docker (pull_request) Successful in 1m9s
test / coverage (pull_request) Successful in 17s
prd-number-check / require-numbered-prds (pull_request) Successful in 8s
test / unit (push) Successful in 54s
Update Quality Badges / update-badges (push) Successful in 54s
lint / lint (push) Successful in 1m4s
test / integration-docker (push) Successful in 1m9s
test / coverage (push) Successful in 38s
Gitea AGit accepts pushes to refs/for/*, refs/draft/*, and
refs/for-review/* and opens pull requests backed by server-managed
refs/pull/<n>/head refs rather than ordinary refs/heads/* branches.
That breaks the git-gate branch workflow: follow-up commits can't be
pushed back through the branch, and Gitea rejects later direct updates
to the generated review ref, so recovery means recreating the PR.

Add a Phase 0 guard to the shared pre-receive hook that rejects
creation or update of those AGit review refs before any gitleaks scan
or upstream forward, with a message pointing callers at the
branch-backed PR workflow. Deletions (new == zero) stay allowed so
legacy AGit refs can still be cleaned up; normal branches and tags are
untouched.

Closes #506

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 17:59:21 +00:00
didericis-claude 0edd46d56d chore(ci): only run PRD number check for PRs into main
prd-number-check / require-numbered-prds (pull_request) Successful in 5s
tracker-policy-pr / check-pr (pull_request) Successful in 12s
The require-numbered-prds gate previously ran on every pull request.
Scope its trigger to PRs whose base branch is main, so numbering is
only enforced at the point of merging into main.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 17:19:36 +00:00
Quality Badge Bot 4682dd441f chore: update quality badges
- Coverage: 84%
- Core coverage: 94%

[skip ci]
2026-07-26 17:06:23 +00:00
didericis-codex 1827593b89 test(docker): authenticate multitenant egress probes
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
test / unit (pull_request) Successful in 53s
test / integration-docker (pull_request) Successful in 1m7s
test / coverage (pull_request) Successful in 17s
tracker-policy-pr / check-pr (pull_request) Successful in 32s
test / unit (push) Successful in 57s
lint / lint (push) Successful in 1m4s
test / integration-docker (push) Successful in 1m11s
test / coverage (push) Successful in 21s
Update Quality Badges / update-badges (push) Successful in 56s
2026-07-26 08:41:00 +00:00
didericis-codex 73e70e326c test(docker): wait for multitenant probe readiness
prd-number-check / require-numbered-prds (pull_request) Successful in 6s
tracker-policy-pr / check-pr (pull_request) Successful in 26s
test / unit (pull_request) Successful in 49s
lint / lint (push) Successful in 59s
test / integration-docker (pull_request) Failing after 1m1s
test / coverage (pull_request) Has been skipped
2026-07-26 08:38:16 +00:00
didericis-codex d744bec7b1 fix(docker): configure the attributed gateway subnet
tracker-policy-pr / check-pr (pull_request) Successful in 12s
prd-number-check / require-numbered-prds (pull_request) Successful in 42s
lint / lint (push) Successful in 1m4s
test / unit (pull_request) Successful in 57s
test / integration-docker (pull_request) Failing after 1m5s
test / coverage (pull_request) Has been skipped
2026-07-26 08:35:37 +00:00
didericis-codex f3fbfb3cc3 fix(ci): join control planes to the runner network
prd-number-check / require-numbered-prds (pull_request) Successful in 13s
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-docker (pull_request) Failing after 35s
test / unit (pull_request) Successful in 52s
test / coverage (pull_request) Has been skipped
lint / lint (push) Successful in 1m7s
2026-07-26 08:30:57 +00:00
didericis-codex b2245ae1f3 test(orchestrator): make wrong-key coverage deterministic 2026-07-26 08:30:57 +00:00
didericis-codex f0fe33b1d0 fix(ci): bypass proxies for Docker host probes
prd-number-check / require-numbered-prds (pull_request) Successful in 6s
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / unit (pull_request) Failing after 45s
test / integration-docker (pull_request) Failing after 7m1s
test / coverage (pull_request) Has been skipped
2026-07-26 08:24:21 +00:00
didericis-codex d4889663d1 fix(ci): enable the scheduled canary suite
prd-number-check / require-numbered-prds (pull_request) Successful in 9s
tracker-policy-pr / check-pr (pull_request) Successful in 9s
test / unit (pull_request) Successful in 50s
test / integration-docker (pull_request) Failing after 7m45s
test / coverage (pull_request) Has been skipped
2026-07-26 08:15:21 +00:00
didericis-codex 1022247ce5 test(ci): cover assurance gate entry points
prd-number-check / require-numbered-prds (pull_request) Successful in 7s
tracker-policy-pr / check-pr (pull_request) Successful in 8s
lint / lint (push) Successful in 58s
test / unit (pull_request) Successful in 48s
test / integration-docker (pull_request) Failing after 19m28s
test / coverage (pull_request) Has been skipped
2026-07-26 08:14:00 +00:00
didericis-codex 671d91070e docs(ci): document the assurance topology 2026-07-26 08:02:12 +00:00
didericis-codex 1c6d30ffd8 test(canary): verify the pinned gitleaks release 2026-07-26 08:01:07 +00:00
didericis-codex 583ff98b27 ci(docker): require the full integration suite 2026-07-26 08:00:15 +00:00
didericis-codex fafb828bb7 ci(coverage): enforce the critical core contract 2026-07-26 07:50:44 +00:00
Quality Badge Bot f9ad6c85aa chore: update quality badges
- Coverage: 83%
- Core coverage: 93%

[skip ci]
2026-07-26 07:38:37 +00:00
didericis-codex 7488110e71 chore: tighten upkeep boundaries and static checks
prd-number-check / require-numbered-prds (pull_request) Successful in 8s
test / integration-docker (pull_request) Successful in 12s
test / unit (pull_request) Successful in 49s
test / coverage (pull_request) Successful in 15s
tracker-policy-pr / check-pr (pull_request) Successful in 5s
test / integration-docker (push) Successful in 13s
test / unit (push) Successful in 48s
lint / lint (push) Successful in 56s
Update Quality Badges / update-badges (push) Successful in 50s
test / coverage (push) Successful in 13s
2026-07-26 06:55:49 +00:00
didericis-codex 22dde95561 fix(diagnostics): make optional failures observable and safe 2026-07-26 06:55:49 +00:00
didericis-codex e29b79d517 refactor(backend): extract shared bottle preparation planner 2026-07-26 06:55:49 +00:00
didericis-codex 1d85acfd99 refactor(egress): make addon core a compatibility facade 2026-07-26 06:55:49 +00:00
didericis-codex 90defdc9cd refactor(egress): separate matching, DLP, and context concerns 2026-07-26 06:55:49 +00:00
didericis-codex 2039ef635f refactor: validate reconciliation inputs and neutralize CLI helpers 2026-07-26 06:55:49 +00:00
221 changed files with 16777 additions and 3865 deletions
+6 -3
View File
@@ -2,7 +2,7 @@
# digest, etc.) without coupling every dev push to upstream registry
# availability.
#
# Opt-in via CLAUDE_BOTTLE_RUN_CANARIES=1 so the same files can be run
# Opt-in via BOT_BOTTLE_RUN_CANARIES=1 so the same files can be run
# locally with the same gating.
name: canaries
@@ -17,7 +17,7 @@ jobs:
canaries:
runs-on: ubuntu-latest
env:
CLAUDE_BOTTLE_RUN_CANARIES: "1"
BOT_BOTTLE_RUN_CANARIES: "1"
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -25,4 +25,7 @@ jobs:
# No actions/setup-python: canaries are stdlib unittest on the image's
# system Python 3.12 (older act_runner mishandles setup-python's PATH).
- name: Run canaries
run: python3 -m unittest discover -t . -s tests/canaries -v
run: |
python3 -m scripts.unittest_gate \
-t . -s tests/canaries -v \
--minimum-executed 1 --fail-on-skip
+10
View File
@@ -4,6 +4,13 @@ on:
push:
paths:
- "**.py"
- "Dockerfile*"
- "bot_bottle/contrib/*/Dockerfile"
- "bot_bottle/contrib/*/package.json"
- "bot_bottle/contrib/*/package-lock.json"
- "bot_bottle/contrib/codex/codex-package_SHA256SUMS"
- "requirements.gateway.*"
- "image-build-args.json"
- ".pylintrc"
- ".gitea/workflows/lint.yml"
@@ -13,6 +20,9 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Enforce immutable image inputs
run: python3 scripts/check_image_inputs.py
# No actions/setup-python: the runner image already ships Python 3.12,
# and older act_runner engines mishandle setup-python's PATH. Install
# into the ephemeral job container's system Python — the pylint/pyright
+1
View File
@@ -3,6 +3,7 @@ name: prd-number-check
on:
pull_request:
types: [opened, reopened, synchronize]
branches: [main]
jobs:
require-numbered-prds:
+27 -1
View File
@@ -60,6 +60,9 @@ jobs:
integration-docker:
runs-on: ubuntu-latest
concurrency:
group: integration-docker-infra
cancel-in-progress: false
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -84,7 +87,30 @@ jobs:
env:
BOT_BOTTLE_BACKEND: docker
COVERAGE_FILE: ${{ github.workspace }}/.coverage.docker
run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
run: |
set -euo pipefail
DOCKER_CLIENT_NETWORK=$(
docker inspect "$(hostname)" |
python3 -c 'import json,sys; n=json.load(sys.stdin)[0]["NetworkSettings"]["Networks"]; print(next(iter(n)))'
)
test -n "$DOCKER_CLIENT_NETWORK"
RUN_KEY="${GITHUB_RUN_ID:-${GITHUB_RUN_NUMBER:-0}}"
export NO_PROXY="*"
export no_proxy="*"
export BOT_BOTTLE_DOCKER_CLIENT_NETWORK="$DOCKER_CLIENT_NETWORK"
export BOT_BOTTLE_DOCKER_ROOT_MOUNT="bot-bottle-ci-root-$RUN_KEY"
export BOT_BOTTLE_DOCKER_CA_MOUNT="bot-bottle-ci-ca-$RUN_KEY"
python3 -m coverage run -m scripts.unittest_gate \
-t . -s tests/integration -v \
--minimum-executed 22 --fail-on-skip
- name: Clean Docker integration volumes
if: always()
run: |
RUN_KEY="${GITHUB_RUN_ID:-${GITHUB_RUN_NUMBER:-0}}"
docker volume rm --force \
"bot-bottle-ci-root-$RUN_KEY" \
"bot-bottle-ci-ca-$RUN_KEY" 2>/dev/null || true
# Non-dot name so upload-artifact's dotfile-skipping glob picks it up.
- name: Stage docker coverage for upload
+97
View File
@@ -0,0 +1,97 @@
# Manually refresh the committed package locks and the pinned Codex installer
# checksum after deliberately changing a direct version in the source files.
#
# The job uploads the generated files for review; it never commits or pushes.
name: refresh-image-locks
on:
workflow_dispatch:
push:
paths:
- 'requirements.gateway.in'
- 'bot_bottle/contrib/claude/package.json'
- 'bot_bottle/contrib/pi/package.json'
- 'bot_bottle/contrib/codex/Dockerfile'
- 'image-build-args.json'
- '.gitea/workflows/refresh-image-locks.yml'
permissions:
code: read
jobs:
refresh:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Resolve image runtime versions
id: runtimes
run: |
python3 - <<'PY' >> "$GITHUB_OUTPUT"
import json
import re
inputs = json.load(open("image-build-args.json"))
for output, name in (
("python-version", "PYTHON_BASE_IMAGE"),
("node-version", "NODE_BASE_IMAGE"),
):
match = re.search(r":(\d+\.\d+\.\d+)-", inputs[name])
if match is None:
raise SystemExit(f"cannot resolve runtime version from {name}")
print(f"{output}={match.group(1)}")
PY
- name: Use the image Python version
uses: actions/setup-python@v5
with:
python-version: '${{ steps.runtimes.outputs.python-version }}'
- name: Use the image Node version
uses: actions/setup-node@v4
with:
node-version: '${{ steps.runtimes.outputs.node-version }}'
- name: Compile gateway Python lock
run: |
python3 -m venv /tmp/image-lock-tools
/tmp/image-lock-tools/bin/python -m pip install \
pip==25.2 \
pip-tools==7.5.1
/tmp/image-lock-tools/bin/python -m piptools compile \
--generate-hashes \
--output-file requirements.gateway.lock \
requirements.gateway.in
- name: Resolve provider npm locks
run: |
for provider in claude pi; do
(
cd "bot_bottle/contrib/$provider"
npm install --package-lock-only --ignore-scripts --no-audit --no-fund
)
done
python3 scripts/complete_npm_lock_integrity.py \
bot_bottle/contrib/claude/package-lock.json \
bot_bottle/contrib/pi/package-lock.json
- name: Refresh pinned Codex archive checksums
run: |
CODEX_VERSION=$(
sed -n 's/^ARG CODEX_VERSION=//p' bot_bottle/contrib/codex/Dockerfile
)
test -n "$CODEX_VERSION"
curl -fsSL \
"https://github.com/openai/codex/releases/download/rust-v${CODEX_VERSION}/codex-package_SHA256SUMS" \
-o bot_bottle/contrib/codex/codex-package_SHA256SUMS
- name: Upload refreshed inputs
uses: actions/upload-artifact@v3
with:
name: image-input-locks
path: |
requirements.gateway.lock
bot_bottle/contrib/claude/package-lock.json
bot_bottle/contrib/pi/package-lock.json
bot_bottle/contrib/codex/codex-package_SHA256SUMS
+174 -7
View File
@@ -12,33 +12,49 @@ on:
- 'bot_bottle/**'
- 'tests/**/*.py'
- 'cli.py'
- 'install.sh'
- 'setup.py'
- 'MANIFEST.in'
- 'flake.nix'
- 'nix/firecracker-netpool.nix'
- 'scripts/coverage.sh'
- 'scripts/critical-modules.txt'
- 'scripts/diff_coverage.py'
- 'scripts/tracker_policy.py'
- 'scripts/**/*.py'
- 'scripts/firecracker-netpool.sh'
- 'Dockerfile*'
- 'image-build-args.json'
- 'pyproject.toml'
- 'requirements-dev.txt'
- 'requirements.gateway.*'
- '.coveragerc'
- '.dockerignore'
- '.gitea/workflows/test.yml'
- '.gitea/workflows/refresh-image-locks.yml'
- '.gitea/workflows/pre-release-test.yml'
pull_request:
paths:
- 'bot_bottle/**'
- 'tests/**/*.py'
- 'cli.py'
- 'install.sh'
- 'setup.py'
- 'MANIFEST.in'
- 'flake.nix'
- 'nix/firecracker-netpool.nix'
- 'scripts/coverage.sh'
- 'scripts/critical-modules.txt'
- 'scripts/diff_coverage.py'
- 'scripts/tracker_policy.py'
- 'scripts/**/*.py'
- 'scripts/firecracker-netpool.sh'
- 'Dockerfile*'
- 'image-build-args.json'
- 'pyproject.toml'
- 'requirements-dev.txt'
- 'requirements.gateway.*'
- '.coveragerc'
- '.dockerignore'
- '.gitea/workflows/test.yml'
- '.gitea/workflows/refresh-image-locks.yml'
- '.gitea/workflows/pre-release-test.yml'
jobs:
unit:
@@ -71,6 +87,9 @@ jobs:
integration-docker:
runs-on: ubuntu-latest
concurrency:
group: integration-docker-infra
cancel-in-progress: false
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -83,11 +102,51 @@ jobs:
python3 --version
python3 cli.py backend status --backend=docker
- name: Preflight — clear any leftover poisoned gateway network
run: |
# The gateway network has a fixed name and persists across jobs on
# this shared runner. A pre-fix or concurrent launch can leave it with
# a malformed IPv6 subnet that trips docker's own ParseAddr in
# `network inspect` (see PR #515); the code now self-heals it, but the
# heal can't run if `network inspect` is what's broken on some daemon
# versions. Drop the network here so this run recreates it IPv4-only.
# Remove the attached gateway container first (else `network rm` fails
# on active endpoints); both are recreated by ensure_running. Harmless
# when absent.
docker rm --force bot-bottle-orch-gateway 2>/dev/null || true
docker network rm bot-bottle-gateway 2>/dev/null || true
- name: Run integration tests (docker) with coverage
env:
BOT_BOTTLE_BACKEND: docker
COVERAGE_FILE: ${{ github.workspace }}/.coverage.docker
run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
run: |
set -euo pipefail
# act_runner executes this job in a container while sharing the host
# Docker socket. Attach control-plane siblings to the job's network,
# and use named volumes for state the host daemon must mount.
DOCKER_CLIENT_NETWORK=$(
docker inspect "$(hostname)" |
python3 -c 'import json,sys; n=json.load(sys.stdin)[0]["NetworkSettings"]["Networks"]; print(next(iter(n)))'
)
test -n "$DOCKER_CLIENT_NETWORK"
RUN_KEY="${GITHUB_RUN_ID:-${GITHUB_RUN_NUMBER:-0}}"
export NO_PROXY="*"
export no_proxy="*"
export BOT_BOTTLE_DOCKER_CLIENT_NETWORK="$DOCKER_CLIENT_NETWORK"
export BOT_BOTTLE_DOCKER_ROOT_MOUNT="bot-bottle-ci-root-$RUN_KEY"
export BOT_BOTTLE_DOCKER_CA_MOUNT="bot-bottle-ci-ca-$RUN_KEY"
python3 -m coverage run -m scripts.unittest_gate \
-t . -s tests/integration -v \
--minimum-executed 22 --fail-on-skip
- name: Clean Docker integration volumes
if: always()
run: |
RUN_KEY="${GITHUB_RUN_ID:-${GITHUB_RUN_NUMBER:-0}}"
docker volume rm --force \
"bot-bottle-ci-root-$RUN_KEY" \
"bot-bottle-ci-ca-$RUN_KEY" 2>/dev/null || true
- name: Stage docker coverage for upload
run: cp .coverage.docker coverage-docker.dat
@@ -98,6 +157,114 @@ jobs:
name: coverage-docker
path: coverage-docker.dat
image-input-builds:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Verify shared bases cover supported architectures
run: |
set -euo pipefail
python_ref=$(python3 -c \
'import json; print(json.load(open("image-build-args.json"))["PYTHON_BASE_IMAGE"])')
node_ref=$(python3 -c \
'import json; print(json.load(open("image-build-args.json"))["NODE_BASE_IMAGE"])')
docker_cli_ref=$(python3 -c \
'import json; print(json.load(open("image-build-args.json"))["DOCKER_CLI_BASE_IMAGE"])')
test -n "$python_ref"
test -n "$node_ref"
test -n "$docker_cli_ref"
for ref in "$python_ref" "$node_ref" "$docker_cli_ref"; do
docker buildx imagetools inspect --raw "$ref" |
python3 -c '
import json
import sys
manifest = json.load(sys.stdin)
platforms = {
(item["platform"]["os"], item["platform"]["architecture"])
for item in manifest["manifests"]
if item.get("platform", {}).get("os") != "unknown"
}
required = {("linux", "amd64"), ("linux", "arm64")}
missing = required - platforms
if missing:
raise SystemExit(f"base manifest lacks supported platforms: {missing}")
'
done
- name: Verify Codex archives for all supported architectures
run: |
set -euo pipefail
codex_version=$(
sed -n 's/^ARG CODEX_VERSION=//p' bot_bottle/contrib/codex/Dockerfile
)
test -n "$codex_version"
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
for target in \
aarch64-unknown-linux-musl \
x86_64-unknown-linux-musl
do
asset="codex-package-${target}.tar.gz"
expected=$(
awk -v asset="$asset" '$2 == asset { print $1 }' \
bot_bottle/contrib/codex/codex-package_SHA256SUMS
)
test -n "$expected"
curl -fsSL \
"https://github.com/openai/codex/releases/download/rust-v${codex_version}/${asset}" \
-o "$tmp/$asset"
echo "$expected $tmp/$asset" | sha256sum -c -
tar -tzf "$tmp/$asset" > "$tmp/$asset.contents"
grep -Fx 'bin/codex' "$tmp/$asset.contents"
grep -Fx 'bin/codex-code-mode-host' "$tmp/$asset.contents"
grep -Fx 'codex-package.json' "$tmp/$asset.contents"
done
- name: Build and smoke-test all supported images
run: |
set -euo pipefail
suffix="${GITHUB_RUN_ID:-${GITHUB_RUN_NUMBER:-image-inputs}}"
orchestrator="bot-bottle-orchestrator-inputs:${suffix}"
gateway="bot-bottle-gateway-inputs:${suffix}"
orchestrator_fc="bot-bottle-orchestrator-fc-inputs:${suffix}"
claude="bot-bottle-claude-inputs:${suffix}"
codex="bot-bottle-codex-inputs:${suffix}"
pi="bot-bottle-pi-inputs:${suffix}"
python_base=$(python3 -c \
'import json; print(json.load(open("image-build-args.json"))["PYTHON_BASE_IMAGE"])')
node_base=$(python3 -c \
'import json; print(json.load(open("image-build-args.json"))["NODE_BASE_IMAGE"])')
docker build --build-arg "PYTHON_BASE_IMAGE=$python_base" \
-t "$orchestrator" -f Dockerfile.orchestrator .
orchestrator_id=$(docker image inspect --format '{{.Id}}' "$orchestrator")
case "$orchestrator_id" in sha256:*) ;; *) exit 1 ;; esac
orchestrator_base="bot-bottle-orchestrator-inputs:sha256-${orchestrator_id#sha256:}"
docker image tag "$orchestrator_id" "$orchestrator_base"
test "$(
docker image inspect --format '{{.Id}}' "$orchestrator_base"
)" = "$orchestrator_id"
docker build --build-arg "PYTHON_BASE_IMAGE=$python_base" \
-t "$gateway" -f Dockerfile.gateway .
docker build \
--build-arg "ORCHESTRATOR_BASE_IMAGE=$orchestrator_base" \
-t "$orchestrator_fc" -f Dockerfile.orchestrator.fc .
docker build --build-arg "NODE_BASE_IMAGE=$node_base" \
-t "$claude" -f bot_bottle/contrib/claude/Dockerfile .
docker build --build-arg "NODE_BASE_IMAGE=$node_base" \
-t "$codex" -f bot_bottle/contrib/codex/Dockerfile .
docker build --build-arg "NODE_BASE_IMAGE=$node_base" \
-t "$pi" -f bot_bottle/contrib/pi/Dockerfile .
docker run --rm --entrypoint python3 "$orchestrator" -c \
'import bot_bottle.orchestrator'
docker run --rm --entrypoint mitmdump "$gateway" --version
docker run --rm "$claude" claude --version
docker run --rm "$codex" codex --version
docker run --rm "$pi" pi --version
coverage:
needs: [unit, integration-docker]
timeout-minutes: 15
@@ -131,7 +298,7 @@ jobs:
- name: Combined coverage (unit + docker integration)
run: PYTHON=python3 bash scripts/coverage.sh aggregate critical
- name: Diff-coverage gate (changed lines >= 90%)
- name: Diff-coverage gate (changed lines >= 80%)
run: |
git fetch --no-tags origin main:refs/remotes/origin/main
python3 scripts/diff_coverage.py --base origin/main --min 90
python3 scripts/diff_coverage.py --base origin/main --min 80
+15 -6
View File
@@ -33,19 +33,28 @@ jobs:
- name: Run coverage and extract percentage
id: coverage
run: |
python3 -m coverage run -m unittest discover -t . -s tests/unit > /dev/null 2>&1 || true
PERCENT=$(python3 -m coverage report 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
set -euo pipefail
# Never publish a badge from a failed or partial test run.
python3 -m coverage run -m unittest discover -t . -s tests/unit
REPORT=$(python3 -m coverage report)
printf '%s\n' "$REPORT"
PERCENT=$(printf '%s\n' "$REPORT" | awk '$1 == "TOTAL" {gsub("%", "", $NF); print $NF}')
test -n "$PERCENT"
echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Coverage: $PERCENT%"
- name: Extract core (critical-module) coverage percentage
id: core_coverage
run: |
set -euo pipefail
# Reuses the .coverage data from the previous step. The core list is
# the single source of truth in scripts/critical-modules.txt; every
# core module is unit-tested, so the unit-only run is accurate for it.
INCLUDE=$(grep -vE '^[[:space:]]*(#|$)' scripts/critical-modules.txt | paste -sd, -)
PERCENT=$(python3 -m coverage report --include="$INCLUDE" 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
# validated single source of truth. Fail if a listed path disappeared
# or if the measured core falls below ADR 0004's 90% minimum.
INCLUDE=$(python3 scripts/critical_modules.py)
REPORT=$(python3 -m coverage report --include="$INCLUDE" --fail-under=90)
printf '%s\n' "$REPORT"
PERCENT=$(printf '%s\n' "$REPORT" | awk '$1 == "TOTAL" {gsub("%", "", $NF); print $NF}')
test -n "$PERCENT"
echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Core coverage: $PERCENT%"
+21 -6
View File
@@ -36,11 +36,23 @@
# 9420 git-gate smart HTTP (VM-backend agent-facing transport)
# 9100 supervise (MCP HTTP)
# Based on `python:3.12-slim` (Debian trixie) rather than the
# Based on an exact `python:3.12.13-slim-trixie` multi-architecture manifest
# rather than the
# `mitmproxy/mitmproxy` image (Debian bookworm), matching the trixie base the
# orchestrator image needs for buildah (Dockerfile.orchestrator.fc). mitmproxy
# is pip-installed to the same effect as the upstream image.
FROM python:3.12-slim
ARG PYTHON_BASE_IMAGE
FROM ${PYTHON_BASE_IMAGE}
# Freeze apt's package universe as well as the base filesystem. Without a
# snapshot, the same Dockerfile resolves different package versions over time.
ARG DEBIAN_SNAPSHOT=20260724T000000Z
RUN sed -i \
-e "s|http://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|http://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
/etc/apt/sources.list.d/debian.sources
# Runtime system deps:
# git supplies the `git daemon` subcommand (no separate package)
@@ -48,16 +60,19 @@ FROM python:3.12-slim
# openssh-client supplies the upstream SSH transport the
# pre-receive hook uses to forward accepted refs.
# ca-certificates is needed for mitmdump upstream TLS.
RUN apt-get update \
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends \
git openssh-client ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# mitmdump (the egress data plane). The upstream mitmproxy image baked
# this in; on the plain python base we pip-install the same pinned
# version. Its CA dir is set explicitly via `--set confdir=` in
# this in; on the plain python base we install a fully resolved lock whose
# distributions are all hash-verified. Its CA dir is set explicitly via
# `--set confdir=` in
# egress-entrypoint.sh, so it doesn't depend on a `mitmproxy` home user.
RUN pip install --no-cache-dir mitmproxy==11.1.3
COPY requirements.gateway.lock /tmp/requirements.gateway.lock
RUN pip install --no-cache-dir --require-hashes \
-r /tmp/requirements.gateway.lock
# gitleaks (the pre-receive hook's secret scanner). Installed from its
# official release, pinned by version + SHA256 and verified — rather than
+13 -8
View File
@@ -9,19 +9,24 @@
# Keeping the content in one place means future orchestrator deps (e.g.
# iroh) are added here once, not duplicated per backend.
#
# It stays deliberately lean: the control plane is **stdlib-only** today, so
# no third-party payload — none of the gateway's mitmproxy/git/gitleaks
# (that's Dockerfile.gateway) and no buildah (that's the firecracker
# builder, and lives only in Dockerfile.orchestrator.fc). Keeping the
# secret-dense control plane on a minimal dependency surface is the point
# (PRD 0070's "secret concentration").
# It stays deliberately lean: only the pinned FastAPI/Uvicorn control-plane
# stack is installed here — none of the gateway's mitmproxy/git/gitleaks
# (that's Dockerfile.gateway) and no buildah (that's firecracker-only).
#
# Shares the trixie `python:3.12-slim` base with the gateway image.
# Shares an exact multi-architecture Python/trixie manifest with the gateway
# image. The version-qualified tag keeps the human-readable upstream version;
# the digest makes the bytes immutable.
FROM python:3.12-slim
ARG PYTHON_BASE_IMAGE
FROM ${PYTHON_BASE_IMAGE}
WORKDIR /app
COPY requirements.orchestrator.lock /tmp/requirements.orchestrator.lock
RUN pip install --no-cache-dir --require-hashes \
-r /tmp/requirements.orchestrator.lock \
&& rm /tmp/requirements.orchestrator.lock
# The orchestrator content. Baked so the image is self-contained (runs from
# a built image, no runtime bind-mount); the docker backend may still
# bind-mount /app for dev live-reload, which simply overlays this copy.
+14 -3
View File
@@ -12,10 +12,21 @@
# bare microVM (no fuse-overlayfs / overlay module / subuid maps). The trixie
# base (from Dockerfile.orchestrator's python:3.12-slim) carries buildah 1.39,
# which parses the Dockerfile heredocs agent images use (bookworm's 1.28 can't).
# Matches image_builder.
FROM bot-bottle-orchestrator:latest
# Matches image_builder. There is deliberately no default: the build coordinator
# passes the exact local image ID returned by `docker image inspect`, so this
# stage cannot silently resolve a stale `:latest` tag.
ARG ORCHESTRATOR_BASE_IMAGE
FROM ${ORCHESTRATOR_BASE_IMAGE}
RUN apt-get update \
ARG DEBIAN_SNAPSHOT=20260724T000000Z
RUN sed -i \
-e "s|http://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|http://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
/etc/apt/sources.list.d/debian.sources
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends \
buildah crun netavark aardvark-dns \
&& rm -rf /var/lib/apt/lists/*
+3
View File
@@ -4,5 +4,8 @@
include Dockerfile.gateway
include Dockerfile.orchestrator
include Dockerfile.orchestrator.fc
include image-build-args.json
include requirements.gateway.in
include requirements.gateway.lock
include nix/firecracker-netpool.nix
include scripts/firecracker-netpool.sh
+25 -11
View File
@@ -5,7 +5,7 @@
# bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-83%25-brightgreen)](https://coverage.readthedocs.io/)
[![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-94%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -15,7 +15,7 @@
## Features
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `bot-bottle supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Tokens the agent never sees** — host secrets live in a gateway; the agent dials `http://gateway:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
@@ -39,7 +39,7 @@ On the legacy Docker backend, the same logical bottle is two containers per agen
The Docker topology looks like this:
```
host ( ./cli.py )
host ( bot-bottle )
starts │ stops
@@ -71,11 +71,25 @@ When the agent exits, `cli.py` tears down every gateway and both networks; nothi
## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
```sh
curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
```
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
The installer is a bootstrapper: it finds a suitable Python, installs bot-bottle with `pipx` (falling back to `pip --user`), creates `~/.bot-bottle`, and runs `bot-bottle doctor`. It is idempotent and never uses `sudo`. Python-native users can skip it entirely with `pipx install bot-bottle` or `uv tool install bot-bottle`.
> **CI (macOS Apple Container):** the `integration-macos` job (`.gitea/workflows/test.yml`) runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container` on a self-hosted macOS runner labelled `macos`, because Apple Container needs the host virtualization framework and cannot run in a Linux container (so it can't reuse the `kvm` runner). Provision an Apple Silicon host with the `container` CLI on `PATH` and `container system status` running, then register the runner in **host mode** (not docker mode) with the `macos` label — `brew install gitea-runner` (the `act_runner` rename). Give it a Python ≥ 3.11 with `coverage` importable on the launchd service's `PATH` (a launchd service doesn't inherit your shell profile, so pin `node` and the Python env explicitly). The job is **advisory** — `workflow_dispatch` (manual) only, never triggered by push or PR — since a single laptop that sleeps/roams must not block merges or churn on every push to main; its coverage doesn't feed the gate. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1.
### Requirements
**Python ≥ 3.11**, and this is the one that trips people up on macOS: the `python3` Apple ships at `/usr/bin/python3` is **3.9.6**, which is too old. Bare `python3` resolves to that stub far more often than people expect. `path_helper` builds a login shell's `PATH` from `/etc/paths` and then appends `/etc/paths.d/*`, and `/usr/bin` sits in the former — so even when `/opt/homebrew/bin` *is* on the `PATH` (via `/etc/paths.d/homebrew`), it comes after `/usr/bin` and loses. Prepending a newer Python is something your shell profile does, and a fresh account, a launchd job, or a CI runner has no such profile. So the installer looks past bare `python3` before giving up: it tries `python3`, then the versioned `python3.11``python3.14` names, then `/opt/homebrew/bin`, `/usr/local/bin`, `~/.local/bin`, and python.org framework builds — and tells you which one it picked when it isn't the obvious one. Point it somewhere specific with `BOT_BOTTLE_PYTHON=/path/to/python3`.
**No `pipx` required.** If `pipx` is present the installer uses it and stays out of the way. If it isn't, bot-bottle installs into a private venv at `~/.bot-bottle/venv` (override with `BOT_BOTTLE_VENV`) and symlinks the entry point into `~/.local/bin`. There is deliberately no `pip install --user` path: Homebrew, python.org and Debian/Ubuntu interpreters are all externally managed (PEP 668), which blocks `--user` outright — so on a Mac it is never the fallback it appears to be. A venv is exempt from PEP 668, and `venv` is stdlib, so unlike `pipx` there is nothing to bootstrap first.
**`git`**, because the default install spec is a `git+` URL. Set `BOT_BOTTLE_INSTALL_SPEC` to a wheel path or index name to avoid it.
**A backend**, which the installer deliberately does *not* install for you — `doctor` reports what's missing afterwards. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker bot-bottle start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
> **CI (macOS Apple Container):** the advisory `integration-macos` job in `.gitea/workflows/pre-release-test.yml` runs only on manual dispatch. It targets a self-hosted host-mode runner labelled `macos`; Apple Container cannot run inside the Linux pull-request runner. Provision an Apple Silicon host with the `container` CLI running and Python ≥ 3.11 plus `coverage` on the launchd service's explicit `PATH`. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1. Its coverage is reported separately and never feeds the required pull-request gate.
### Containers inside a bottle
@@ -166,18 +180,18 @@ On Linux, a KVM-capable host defaults to the Firecracker backend. It needs:
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
- **Docker** for the gateway and image build.
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `bot-bottle backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `bot-bottle backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
```sh
BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
BOT_BOTTLE_BACKEND=firecracker bot-bottle start <agent>
```
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
> **CI:** the coverage gate (`.gitea/workflows/test.yml` → `coverage` job) runs on a self-hosted runner labelled `kvm`, because the Firecracker backend's VM/SSH orchestration is exercised only by the integration suite, which needs `/dev/kvm` + the provisioned pool (a container runner would skip it and read as uncovered). Provision that runner exactly like a normal Firecracker host `firecracker` on `PATH`, `/dev/kvm`, the cached guest kernel + static dropbear, and the pool installed as the persistent systemd unit — then register it with the `kvm` label. A Docker-capable hosted job builds the candidate once; KVM tests boot those exact bytes, and a successful main run publishes them. The unit/lint jobs still run on `ubuntu-latest`.
> **CI:** Firecracker integration runs in the manually dispatched `.gitea/workflows/pre-release-test.yml` on a self-hosted runner labelled `kvm`; privileged KVM hosts never execute unreviewed PR code automatically. Provision it like a normal Firecracker host: `firecracker` on `PATH`, `/dev/kvm`, the cached guest kernel and static dropbear, and the persistent TAP/nft pool. The required pull-request workflow runs unit plus the complete Docker integration suite on `ubuntu-latest`; see `docs/ci.md`.
```sh
./cli.py start <agent> # builds the image on first run, drops you into claude
bot-bottle start <agent> # builds the image on first run, drops you into claude
```
## Manifest
@@ -253,7 +267,7 @@ You help maintain Gitea-hosted projects.
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default for manifest routes — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). Agent-provider routes (e.g. `api.anthropic.com`) default to `redact`. |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `bot-bottle supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
+1 -1
View File
@@ -226,7 +226,7 @@ class AgentProvider(ABC):
initial task in a non-interactive (headless) session.
Called only when ``--prompt`` is passed to
``./cli.py start --headless``; the returned args are appended
``bot-bottle start --headless``; the returned args are appended
after the provider's ``bypass_args`` and ``startup_args``."""
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
+3
View File
@@ -30,6 +30,7 @@ if TYPE_CHECKING:
BottleImages,
BottlePlan,
BottleSpec,
EnumerationError,
ExecResult,
)
from .selection import (
@@ -59,6 +60,7 @@ _LAZY_MODULES: dict[str, str] = {
"BottleImages": "base",
"BottleBackend": "base",
"BackendStatus": "base",
"EnumerationError": "base",
"get_bottle_backend": "selection",
"known_backend_names": "selection",
"has_backend": "selection",
@@ -100,6 +102,7 @@ __all__ = [
"BottlePlan",
"BottleSpec",
"ExecResult",
"EnumerationError",
"CommitCancelled",
"Freezer",
"get_freezer",
+36 -78
View File
@@ -23,14 +23,14 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Generator, Generic, Sequence, TypeVar
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
from ..agent_provider import AgentProvisionPlan, get_provider
from ..egress import EgressPlan
from ..git_gate import GitGatePlan
from ..log import die, info
from ..util import expand_tilde
from ..manifest import Manifest, ManifestIndex
from ..supervisor.plan import SupervisePlan
from ..env import resolve_env, ResolvedEnv
from ..env import ResolvedEnv
from ..workspace import WorkspacePlan, workspace_plan
from .print_util import print_multi, visible_agent_env_names
from .util import host_skill_dir
@@ -42,6 +42,10 @@ class BackendStatus(enum.IntEnum):
READY = 0
class EnumerationError(RuntimeError):
"""A backend could not produce an authoritative live-resource snapshot."""
@dataclass(frozen=True)
class BottleSpec:
"""CLI-supplied intent. Backend-agnostic — each backend's prepare
@@ -168,6 +172,10 @@ class BottleCleanupPlan(ABC):
"""True iff there is nothing to clean up; the CLI uses this to
short-circuit before showing the y/N."""
@abstractmethod
def intersect(self, current: "BottleCleanupPlan") -> "BottleCleanupPlan":
"""Resources both displayed to the operator and currently removable."""
@dataclass(frozen=True)
class ExecResult:
@@ -296,82 +304,18 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
backend-specific resolution (names, scratch files, etc.). The
validation step is enforced here so a future backend cannot
accidentally skip it. No remote/runtime resources are created."""
from .resolve_common import (
merge_provision_env_vars,
mint_slug,
prepare_agent_state_dir,
prepare_egress,
prepare_git_gate,
prepare_supervise,
reject_nested_containers,
resolve_manifest_dockerfile,
write_launch_metadata,
)
manifest = self._validate(spec)
if not self.supports_nested_containers:
reject_nested_containers(self.name, manifest)
self._preflight()
from ..git_gate import GitGate
manifest = GitGate().preflight_host_keys(
manifest,
headless=spec.headless,
home_md=spec.manifest.home_md,
)
manifest_bottle = manifest.bottle
manifest_agent_provider = manifest_bottle.agent_provider
agent_provider = get_provider(manifest_agent_provider.template)
resolved_env = resolve_env(manifest)
workspace = workspace_plan(spec, guest_home=agent_provider.guest_home)
slug = mint_slug(spec)
write_launch_metadata(slug, spec, compose_project="", backend=self.name)
# Manifest may override the Dockerfile per-bottle; otherwise fall
# back to the provider plugin's bundled Dockerfile (next to its
# agent_provider.py module).
if manifest_agent_provider.dockerfile:
agent_dockerfile_path = resolve_manifest_dockerfile(
manifest_agent_provider.dockerfile, spec,
)
else:
agent_dockerfile_path = str(agent_provider.dockerfile)
agent_dir, prompt_file = prepare_agent_state_dir(slug, manifest)
agent_provision_plan = build_agent_provision_plan(
template=manifest_agent_provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
instance_name=f"bot-bottle-{slug}",
prompt_file=prompt_file,
guest_env=self._build_guest_env(resolved_env),
forward_host_credentials=manifest_agent_provider.forward_host_credentials,
auth_token=manifest_agent_provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace.workdir,
label=spec.label,
color=spec.color,
provider_settings=manifest_agent_provider.settings,
)
agent_provision_plan = merge_provision_env_vars(agent_provision_plan)
egress_plan = prepare_egress(manifest_bottle, slug, agent_provision_plan)
supervise_plan = prepare_supervise(manifest_bottle, slug)
git_gate_plan = prepare_git_gate(manifest_bottle, slug)
from .preparation import BottlePreparationPlanner
prepared = BottlePreparationPlanner(self).prepare(spec)
return self._resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
manifest=prepared.manifest,
slug=prepared.slug,
resolved_env=prepared.resolved_env,
agent_provision_plan=prepared.agent_provision_plan,
egress_plan=prepared.egress_plan,
supervise_plan=prepared.supervise_plan,
git_gate_plan=prepared.git_gate_plan,
stage_dir=stage_dir,
)
@@ -552,6 +496,20 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
(macos-container) die with a pointer — the default here."""
die(f"backend {self.name!r} has no orchestrator control plane")
def attach_bottled_agents_to_gateway(self) -> None:
"""Reconcile all running bottles against the current gateway.
Called when the gateway is (re)brought up (cold-boot path) to restore
the three gateway-dependent services for every live bottle: CA trust,
git-gate repos/creds, and egress tokens. Per-bottle failures must be
logged and skipped — one unreachable agent must not block the rest.
Default: no-op. Backends that run a gateway (Firecracker, docker,
macOS) override this with a backend-native implementation that reaches
running agents via their transport (SSH for Firecracker, exec/cp for
docker and macOS). PRD 0081."""
return
@abstractmethod
def prepare_cleanup(self) -> CleanupT:
"""Enumerate orphaned resources from previous bottles. No side
@@ -587,7 +545,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
host-appropriate config or commands. Prints to stdout/stderr and
returns a shell exit code (0 = nothing to report / success). A
backend that needs no host setup prints a short note and returns
0. Invoked generically by `./cli.py backend setup [--backend=…]`
0. Invoked generically by `bot-bottle backend setup [--backend=…]`
so operators can provision any backend without a
backend-specific command. Classmethod (like `is_available`) —
it's a host query, not per-bottle state."""
@@ -604,14 +562,14 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
stderr. When quiet=True returns the status code silently —
useful for cheap programmatic checks.
Invoked by `./cli.py backend status [--backend=…]` (quiet=False)
Invoked by `bot-bottle backend status [--backend=…]` (quiet=False)
and by is_backend_ready() (caller-controlled)."""
@classmethod
@abstractmethod
def teardown(cls) -> int:
"""Undo `setup()` — the inverse operation, surfaced as
`./cli.py backend teardown [--backend=…]` (uninstall). Symmetric
`bot-bottle backend teardown [--backend=…]` (uninstall). Symmetric
with setup: where setup is advisory (prints the privileged
commands / declarative config to apply), teardown prints the
commands / config change to remove the host prerequisites. A
+60
View File
@@ -0,0 +1,60 @@
"""Shared destructive-cleanup execution and failure accounting."""
from __future__ import annotations
import os
import shutil
import subprocess
from collections.abc import Sequence
from pathlib import Path
class CleanupError(RuntimeError):
"""One or more approved cleanup mutations did not complete."""
class CleanupFailures:
"""Attempt every approved mutation, then fail with complete diagnostics."""
def __init__(self) -> None:
self._messages: list[str] = []
def run(self, argv: Sequence[str], description: str) -> None:
raw_timeout = os.environ.get(
"BOT_BOTTLE_CLEANUP_COMMAND_TIMEOUT_SECONDS", "120",
)
try:
timeout = float(raw_timeout)
except ValueError:
timeout = 120.0
try:
result = subprocess.run(
list(argv), capture_output=True, text=True, check=False,
timeout=max(timeout, 1.0),
)
except (OSError, subprocess.SubprocessError) as exc:
self._messages.append(f"{description}: {exc}")
return
if result.returncode != 0:
detail = (result.stderr or result.stdout).strip()
self._messages.append(
f"{description}: {detail or f'exit {result.returncode}'}"
)
def remove_tree(self, path: Path, description: str) -> None:
try:
shutil.rmtree(path)
except FileNotFoundError:
return
except OSError as exc:
self._messages.append(f"{description}: {exc}")
def record(self, message: str) -> None:
self._messages.append(message)
def raise_if_any(self) -> None:
if self._messages:
raise CleanupError("; ".join(self._messages))
__all__ = ["CleanupError", "CleanupFailures"]
@@ -46,6 +46,22 @@ class DockerBottleCleanupPlan(BottleCleanupPlan):
and not self.orphan_state_dirs
)
def intersect(self, current: BottleCleanupPlan) -> "DockerBottleCleanupPlan":
if not isinstance(current, DockerBottleCleanupPlan):
raise TypeError("cleanup plans must have the same backend type")
return DockerBottleCleanupPlan(
projects=tuple(x for x in self.projects if x in current.projects),
stray_containers=tuple(
x for x in self.stray_containers if x in current.stray_containers
),
stray_networks=tuple(
x for x in self.stray_networks if x in current.stray_networks
),
orphan_state_dirs=tuple(
x for x in self.orphan_state_dirs if x in current.orphan_state_dirs
),
)
def print(self) -> None:
print(file=sys.stderr)
for name in self.projects:
+38 -38
View File
@@ -23,11 +23,12 @@ Active-agent enumeration lives in `backend/docker/enumerate.py`.
from __future__ import annotations
import shutil
import subprocess
from ...paths import bot_bottle_root
from ...log import info, warn
from ...log import info
from .. import EnumerationError
from ..cleanup_control import CleanupFailures
from . import util as docker_mod
from .bottle_cleanup_plan import DockerBottleCleanupPlan
from ...bottle_state import bottle_state_dir, is_preserved
@@ -36,15 +37,17 @@ from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
def _list_prefixed_containers() -> list[str]:
"""All bot-bottle-prefixed containers, running or stopped."""
result = subprocess.run(
["docker", "ps", "-a",
"--filter", f"name=^{COMPOSE_PROJECT_PREFIX}",
"--format", "{{.Names}}\t{{.Label \"com.docker.compose.project\"}}"],
capture_output=True, text=True, check=False,
)
try:
result = subprocess.run(
["docker", "ps", "-a",
"--filter", f"name=^{COMPOSE_PROJECT_PREFIX}",
"--format", "{{.Names}}\t{{.Label \"com.docker.compose.project\"}}"],
capture_output=True, text=True, check=False,
)
except OSError as exc:
raise EnumerationError(f"docker ps failed: {exc}") from exc
if result.returncode != 0:
warn(f"docker ps failed: {result.stderr.strip()}")
return []
raise EnumerationError(f"docker ps failed: {result.stderr.strip()}")
out: list[str] = []
for line in (result.stdout or "").splitlines():
if not line:
@@ -63,15 +66,19 @@ def _list_prefixed_networks() -> list[str]:
to a compose project. Compose-managed networks have a
`com.docker.compose.project` label; bare ones (from pre-compose
code paths) don't."""
result = subprocess.run(
["docker", "network", "ls",
"--filter", f"name={COMPOSE_PROJECT_PREFIX}",
"--format", "{{.Name}}\t{{.Label \"com.docker.compose.project\"}}"],
capture_output=True, text=True, check=False,
)
try:
result = subprocess.run(
["docker", "network", "ls",
"--filter", f"name={COMPOSE_PROJECT_PREFIX}",
"--format", "{{.Name}}\t{{.Label \"com.docker.compose.project\"}}"],
capture_output=True, text=True, check=False,
)
except OSError as exc:
raise EnumerationError(f"docker network ls failed: {exc}") from exc
if result.returncode != 0:
warn(f"docker network ls failed: {result.stderr.strip()}")
return []
raise EnumerationError(
f"docker network ls failed: {result.stderr.strip()}"
)
out: list[str] = []
for line in (result.stdout or "").splitlines():
if not line:
@@ -120,7 +127,10 @@ def prepare_cleanup() -> DockerBottleCleanupPlan:
`enumerate_active_agents()` so the orphan-state-dir bucket
doesn't include slugs whose non-docker bottle is still up."""
docker_mod.require_docker()
projects = list_compose_projects()
projects = list_compose_projects(
warn_on_error=False,
raise_on_error=True,
)
project_set = set(projects)
# Late import to avoid a circular at module-load time —
# the backend package's __init__ imports this module.
@@ -140,40 +150,30 @@ def cleanup(plan: DockerBottleCleanupPlan) -> None:
"""Remove everything in the plan. Projects first (whose `compose
down` reaps their containers + networks atomically), then stray
legacy resources, then orphan state dirs."""
failures = CleanupFailures()
for project in plan.projects:
info(f"docker compose down ({project})")
result = subprocess.run(
failures.run(
["docker", "compose", "-p", project, "down", "--volumes"],
capture_output=True, text=True, check=False,
f"docker compose down failed for {project}",
)
if result.returncode != 0:
warn(
f"compose down failed for {project}: "
f"{result.stderr.strip()}"
)
for name in plan.stray_containers:
info(f"removing stray container {name}")
subprocess.run(
failures.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
f"removing stray container {name}",
)
for name in plan.stray_networks:
info(f"removing stray network {name}")
subprocess.run(
failures.run(
["docker", "network", "rm", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
f"removing stray network {name}",
)
for identity in plan.orphan_state_dirs:
path = bottle_state_dir(identity)
info(f"removing orphan state dir {path}")
try:
shutil.rmtree(path, ignore_errors=True)
except OSError as e:
warn(f"failed to remove {path}: {e}")
failures.remove_tree(path, f"removing orphan state dir {path}")
failures.raise_if_any()
+30 -13
View File
@@ -16,6 +16,7 @@ from pathlib import Path
from typing import Any
from ...log import die, warn
from ..base import EnumerationError
# --- Lifecycle helpers (PRD 0018 chunk 3) ----------------------------------
@@ -52,19 +53,20 @@ def slug_from_compose_project(project: str) -> str:
def list_compose_projects(
*, include_stopped: bool = True, warn_on_error: bool = True,
*,
include_stopped: bool = True,
warn_on_error: bool = True,
raise_on_error: bool = False,
) -> list[str]:
"""All compose project names starting with `bot-bottle-`.
`include_stopped=True` (default) runs `docker compose ls --all`
so exited projects appear too; pass False to get only projects
with at least one running container.
Returns [] on docker daemon errors or malformed output rather
than raising — callers should treat the empty list as "no
projects discoverable", not "no projects exist". `warn_on_error`
stays true for explicit operator commands like cleanup, but active
discovery paths set it false so dashboard refreshes don't spam
stderr while Docker Desktop is stopped."""
Best-effort callers get ``[]`` on Docker errors or malformed output.
Enumeration callers pass ``raise_on_error=True`` so a failed query is not
reported as an authoritative empty result.
"""
argv = ["docker", "compose", "ls", "--format", "json"]
if include_stopped:
argv.insert(3, "--all")
@@ -72,19 +74,30 @@ def list_compose_projects(
result = subprocess.run(
argv, capture_output=True, text=True, check=False,
)
except FileNotFoundError:
# docker binary not on PATH — same shape as a daemon-down
# error from the caller's POV: no projects discoverable.
except OSError as exc:
# Not only "not found": docker on PATH but not executable by this
# user raises PermissionError. Either way the query never ran, so an
# enumeration caller must not read the empty result as authoritative.
if raise_on_error:
raise EnumerationError(
f"docker compose ls failed: docker unavailable ({exc})"
) from exc
return []
if result.returncode != 0:
message = f"docker compose ls failed: {result.stderr.strip()}"
if raise_on_error:
raise EnumerationError(message)
if warn_on_error:
warn(f"docker compose ls failed: {result.stderr.strip()}")
warn(message)
return []
try:
projects = json.loads(result.stdout or "[]")
except json.JSONDecodeError as e:
message = f"docker compose ls returned malformed JSON: {e}"
if raise_on_error:
raise EnumerationError(message) from e
if warn_on_error:
warn(f"docker compose ls returned malformed JSON: {e}")
warn(message)
return []
names: list[str] = []
for p in projects:
@@ -97,7 +110,10 @@ def list_compose_projects(
def list_active_slugs(
*, include_stopped: bool = False, warn_on_error: bool = True,
*,
include_stopped: bool = False,
warn_on_error: bool = True,
raise_on_error: bool = False,
) -> list[str]:
"""Slugs (project name minus prefix) of currently-running
bottles. Used by the dashboard's operator-edit verbs to choose
@@ -108,6 +124,7 @@ def list_active_slugs(
for p in list_compose_projects(
include_stopped=include_stopped,
warn_on_error=warn_on_error,
raise_on_error=raise_on_error,
)
) if slug
)
@@ -68,6 +68,11 @@ def _network_container_ips(network: str) -> list[str]:
"docker", "network", "inspect", "--format",
"{{range .Containers}}{{.IPv4Address}} {{end}}", network,
])
if proc.returncode != 0:
detail = proc.stderr.strip() or f"exit {proc.returncode}"
raise ConsolidatedLaunchError(
f"could not inspect addresses on gateway network {network}: {detail}"
)
ips: list[str] = []
for entry in proc.stdout.split():
ips.append(entry.split("/", 1)[0])
+13 -12
View File
@@ -1,9 +1,8 @@
"""Active-agent enumeration for the docker backend.
Returns `ActiveAgent` records the CLI `active` command and the
dashboard agents pane consume. Empty when docker isn't reachable
— gated by `has_backend('docker')` at the cross-backend caller
so this module trusts that docker is available when called.
dashboard agents pane consume. Docker query failures raise rather
than masquerading as an authoritative empty result.
The parser (`_parse_services_by_project`) is exposed for direct
unit testing; the docker `docker ps` invocation is in
@@ -13,17 +12,18 @@ from __future__ import annotations
import subprocess
from .. import ActiveAgent
from .. import ActiveAgent, EnumerationError
from ...bottle_state import read_metadata
from .compose import compose_project_name, list_active_slugs
def enumerate_active() -> list[ActiveAgent]:
"""All currently-running docker-backed agents. Caller is
responsible for gating on `has_backend('docker')` if it
matters; if docker is missing the `docker ps` call below
returns an empty list silently."""
slugs = list_active_slugs(include_stopped=False, warn_on_error=False)
"""All currently-running docker-backed agents."""
slugs = list_active_slugs(
include_stopped=False,
warn_on_error=False,
raise_on_error=True,
)
if not slugs:
return []
services_by_project = _query_services_by_project()
@@ -74,8 +74,9 @@ def _query_services_by_project() -> dict[str, set[str]]:
],
capture_output=True, text=True, check=False,
)
except FileNotFoundError:
return {}
except OSError as exc:
# Missing, or on PATH but not executable by this user (PermissionError).
raise EnumerationError(f"docker ps failed: docker unavailable ({exc})") from exc
if r.returncode != 0:
return {}
raise EnumerationError(f"docker ps failed: {r.stderr.strip()}")
return _parse_services_by_project(r.stdout or "")
+89 -7
View File
@@ -17,6 +17,10 @@ from ...gateway import (
DEFAULT_CA_TIMEOUT_SECONDS, CA_POLL_SECONDS, GATEWAY_CA_CERT, GatewayError
)
DEFAULT_GATEWAY_SUBNET = "10.242.255.0/24"
_GATEWAY_SUBNET_LABEL = "bot-bottle.gateway-subnet"
class DockerGateway(Gateway):
"""The consolidated gateway as a single, fixed-name Docker container.
@@ -35,6 +39,8 @@ class DockerGateway(Gateway):
build_context: Path | None = None,
dockerfile: str | None = GATEWAY_DOCKERFILE,
host_port_bindings: tuple[int, ...] = (),
ca_mount_source: str | Path | None = None,
subnet: str | None = None,
) -> None:
self.image_ref = image_ref
self.name = name
@@ -59,6 +65,15 @@ class DockerGateway(Gateway):
# backend's dev-harness gateway so VMs can reach it via their TAP link;
# Docker's DNAT + the nft `ct status dnat accept` rule handle the rest.
self._host_port_bindings = host_port_bindings
self._subnet = (
subnet
or os.environ.get("BOT_BOTTLE_DOCKER_GATEWAY_SUBNET", "").strip()
or DEFAULT_GATEWAY_SUBNET
)
configured_ca = os.environ.get("BOT_BOTTLE_DOCKER_CA_MOUNT", "").strip()
self._ca_mount_source = str(
ca_mount_source or configured_ca or host_gateway_ca_dir()
)
def image_exists(self) -> bool:
return run_docker(["docker", "image", "inspect", self.image_ref]).returncode == 0
@@ -81,6 +96,11 @@ class DockerGateway(Gateway):
str(context)]
if os.environ.get("BOT_BOTTLE_NO_CACHE"):
argv.insert(2, "--no-cache")
for name, value in resources.image_build_args(
self._dockerfile,
context=context,
).items():
argv[-1:-1] = ["--build-arg", f"{name}={value}"]
proc = run_docker(argv)
if proc.returncode != 0:
raise GatewayError(f"gateway image build failed: {proc.stderr.strip()}")
@@ -109,10 +129,72 @@ class DockerGateway(Gateway):
def _ensure_network(self) -> None:
"""Create the shared gateway network if it doesn't exist. Idempotent —
a concurrent create loses harmlessly (the loser sees 'already exists').
Docker picks the subnet; the launcher reads it back to allocate IPs."""
if run_docker(["docker", "network", "inspect", self.network]).returncode == 0:
return
proc = run_docker(["docker", "network", "create", self.network])
The explicit subnet is required because bottle attribution pins source
IPs; Docker rejects static endpoint addresses on an auto-IPAM network."""
inspected = run_docker([
"docker", "network", "inspect",
"--format", f'{{{{index .Labels "{_GATEWAY_SUBNET_LABEL}"}}}}',
self.network,
])
if inspected.returncode == 0:
marker = inspected.stdout.strip()
if marker in {"", self._subnet}:
return
# Inspectable but mislabelled: the stale auto-IPAM network created
# by older releases. Replace it below.
stale = True
else:
# inspect failed. Classify by stderr — do NOT assume "not absent"
# implies "poisoned": a transient daemon/API error, permission
# failure, timeout, or bad context also fails here, and destroying
# the shared gateway on that guess would tear the network out from
# under every live bottle.
err = inspected.stderr.lower()
if "no such network" in err or "not found" in err:
# Absent: nothing to replace — create it below.
stale = False
elif "parseaddr" in err:
# Present but poisoned. A daemon that default-enables IPv6
# attaches an fdd0::/64 subnet whose `::1/64` gateway trips
# docker's own netip.ParseAddr in `network inspect`/`ls`, so the
# command exits non-zero with that signature. A fixed release
# never *creates* such a network, but one can survive on a
# shared host from an older or concurrent launch — and
# `--ipv6=false` alone can't heal it, since the create below only
# no-ops on "already exists". Force-replace it so later reads
# (e.g. `_network_cidr` pinning a source IP) stop failing.
stale = True
else:
# Unrecognized failure: no evidence the network is malformed.
# Surface it rather than mutate shared state on a guess.
raise GatewayError(
f"gateway network {self.network} could not be inspected: "
f"{inspected.stderr.strip()}"
)
if stale:
# Migrate the stale/poisoned network. Removing the fixed gateway is
# safe here: this launch recreates it.
run_docker(["docker", "rm", "--force", self.name])
removed = run_docker(["docker", "network", "rm", self.network])
if removed.returncode != 0 and "no such network" not in removed.stderr.lower():
raise GatewayError(
f"gateway network {self.network} needs explicit subnet "
f"{self._subnet} but could not be replaced: "
f"{removed.stderr.strip()}"
)
proc = run_docker([
"docker", "network", "create",
# bot-bottle attribution pins IPv4 source IPs; it has no IPv6
# support. Disable IPv6 explicitly so a daemon that default-enables
# it (default-address-pools) can't attach an fdd0::/64 subnet — a
# malformed `::1/64` gateway address then trips docker's own
# ParseAddr in `network inspect`/`ls`, which poisons every launch
# that reads this network's subnet.
"--ipv6=false",
"--subnet", self._subnet,
"--label", f"{_GATEWAY_SUBNET_LABEL}={self._subnet}",
self.network,
])
if proc.returncode != 0 and "already exists" not in proc.stderr:
raise GatewayError(
f"gateway network {self.network} failed to create: {proc.stderr.strip()}"
@@ -143,9 +225,9 @@ class DockerGateway(Gateway):
# Recreate when the running container's image is stale (a rebuild),
# so source changes to the gateway's flat daemons take effect — not
# just when the container is absent.
self._ensure_network()
if self.is_running() and self._running_image_is_current():
return
self._ensure_network()
# Clear any stale (stopped OR outdated-image) container holding the
# fixed name, then start fresh. `rm --force` on an absent name is a
# tolerated no-op.
@@ -158,7 +240,7 @@ class DockerGateway(Gateway):
# Persist the self-generated CA on the host so it survives both
# container recreation AND docker volume pruning (agents trust it)
# — see host_gateway_ca_dir / issue #450.
"--volume", f"{host_gateway_ca_dir()}:{MITMPROXY_HOME}",
"--volume", f"{self._ca_mount_source}:{MITMPROXY_HOME}",
# No DB mount: the data plane (egress / supervise / git-gate) reaches
# the supervise queue over the control-plane RPC and never opens
# bot-bottle.db, so the gateway container gets no file handle on it
@@ -253,4 +335,4 @@ class DockerGateway(Gateway):
def provisioning_transport(self) -> GatewayTransport:
"""The exec/cp transport git-gate provisioning stages per-bottle repos +
deploy keys through (over the docker socket)."""
return DockerGatewayTransport(self.name)
return DockerGatewayTransport(self.name)
+11 -4
View File
@@ -33,7 +33,6 @@ from .orchestrator import (
ORCHESTRATOR_NAME,
ORCHESTRATOR_NETWORK,
)
from ...paths import bot_bottle_root
from ... import resources
from ...gateway import (
GATEWAY_IMAGE,
@@ -69,6 +68,8 @@ class DockerInfraService(InfraService):
gateway_image: str = GATEWAY_IMAGE,
repo_root: Path | None = None,
host_root: Path | None = None,
root_mount_source: str | Path | None = None,
gateway_ca_mount_source: str | Path | None = None,
orchestrator_name: str = ORCHESTRATOR_NAME,
orchestrator_label: str = ORCHESTRATOR_LABEL,
gateway_name: str = GATEWAY_NAME,
@@ -78,10 +79,14 @@ class DockerInfraService(InfraService):
self.control_network = control_network
self.orchestrator_image = orchestrator_image
self.gateway_image = gateway_image
# Build context / bind-mount source: the repo root in a checkout, a
# staged copy from the installed wheel otherwise (bot_bottle.resources).
# Build context: the repo root in a checkout, a staged copy from the
# installed wheel otherwise (bot_bottle.resources).
self._repo_root = repo_root if repo_root is not None else resources.build_root()
self._host_root = host_root or bot_bottle_root()
if host_root is not None and root_mount_source is not None:
raise ValueError("pass host_root or root_mount_source, not both")
self._host_root = host_root
self._root_mount_source = root_mount_source
self._gateway_ca_mount_source = gateway_ca_mount_source
self._orchestrator_name = orchestrator_name
self._orchestrator_label = orchestrator_label
self._gateway_name = gateway_name
@@ -98,6 +103,7 @@ class DockerInfraService(InfraService):
control_network=self.control_network,
repo_root=self._repo_root,
host_root=self._host_root,
root_mount_source=self._root_mount_source,
)
def gateway(self) -> DockerGateway:
@@ -112,6 +118,7 @@ class DockerInfraService(InfraService):
network=self.network,
control_network=self.control_network,
build_context=self._repo_root,
ca_mount_source=self._gateway_ca_mount_source,
)
def ensure_running(
+60 -19
View File
@@ -42,13 +42,9 @@ ORCHESTRATOR_IMAGE = os.environ.get(
)
ORCHESTRATOR_DOCKERFILE = "Dockerfile.orchestrator"
# Baked as a container label so `ensure_running` can detect whether the running
# orchestrator is executing the current bind-mounted source.
# orchestrator image was built from the current source.
ORCHESTRATOR_SOURCE_HASH_LABEL = "bot-bottle-orchestrator-source-hash"
# The bind-mount path for the live control-plane source inside the container.
# PYTHONPATH points here so a code change takes effect on the next launch
# without an image rebuild.
_SRC_IN_CONTAINER = "/bot-bottle-src"
# Bot-bottle host-root bind-mount (DB + state) inside the orchestrator. The
# control plane opens bot-bottle.db under here (via BOT_BOTTLE_ROOT ->
# host_db_path()); it is the ONLY container with a handle on it (issue #469).
@@ -72,23 +68,51 @@ class DockerOrchestrator(Orchestrator):
control_network: str = ORCHESTRATOR_NETWORK,
repo_root: Path | None = None,
host_root: Path | None = None,
root_mount_source: str | Path | None = None,
client_host: str | None = None,
client_network: str | None = None,
bind_host: str | None = None,
dockerfile: str | None = ORCHESTRATOR_DOCKERFILE,
) -> None:
if host_root is not None and root_mount_source is not None:
raise ValueError("pass host_root or root_mount_source, not both")
self.image_ref = image_ref
self.name = name
self.label = label
self.port = port
self.control_network = control_network
# Build context / bind-mount source: the repo root in a checkout, a
# staged copy from the installed wheel otherwise (bot_bottle.resources).
# Build context: the repo root in a checkout, a staged copy from the
# installed wheel otherwise (bot_bottle.resources).
self._repo_root = repo_root if repo_root is not None else resources.build_root()
self._host_root = host_root or bot_bottle_root()
configured_root = os.environ.get("BOT_BOTTLE_DOCKER_ROOT_MOUNT", "").strip()
self._root_mount_source = str(
root_mount_source or configured_root or host_root or bot_bottle_root()
)
configured_network = os.environ.get(
"BOT_BOTTLE_DOCKER_CLIENT_NETWORK", ""
).strip()
self._client_network = client_network or configured_network or None
configured_host = os.environ.get(
"BOT_BOTTLE_DOCKER_HOST_ADDRESS", ""
).strip()
self._client_host = (
client_host or configured_host
or (self.name if self._client_network else "127.0.0.1")
)
# A socket-shared CI runner reaches published ports through its Docker
# network rather than its own loopback. Production stays bound to host
# loopback unless a caller explicitly selects another client.
self._bind_host = bind_host or (
"0.0.0.0"
if not self._client_network and self._client_host != "127.0.0.1"
else "127.0.0.1"
)
self._dockerfile = dockerfile
def url(self) -> str:
"""Host-side control-plane URL — the orchestrator's published loopback,
which the CLI reaches."""
return f"http://127.0.0.1:{self.port}"
"""Control-plane URL reachable by this Docker client."""
port = DEFAULT_PORT if self._client_network else self.port
return f"http://{self._client_host}:{port}"
def gateway_url(self) -> str:
"""The URL the gateway's data plane resolves policy against — the
@@ -107,6 +131,11 @@ class DockerOrchestrator(Orchestrator):
str(self._repo_root)]
if os.environ.get("BOT_BOTTLE_NO_CACHE"):
argv.insert(2, "--no-cache")
for name, value in resources.image_build_args(
self._dockerfile,
context=self._repo_root,
).items():
argv[-1:-1] = ["--build-arg", f"{name}={value}"]
proc = run_docker(argv)
if proc.returncode != 0:
raise GatewayError(
@@ -119,8 +148,7 @@ class DockerOrchestrator(Orchestrator):
return self.name in proc.stdout.split()
def _source_current(self, current_hash: str) -> bool:
"""True iff the running orchestrator was started from the current
bind-mounted source."""
"""True iff the running orchestrator image matches current source."""
if not self.is_running():
return False
proc = run_docker([
@@ -182,15 +210,19 @@ class DockerOrchestrator(Orchestrator):
# Control network only — agents are never on it, so they have no
# route to the control plane (the L3 block, not just the JWT).
"--network", self.control_network,
# Host CLI reaches the control plane here (loopback only). The
# Host CLI reaches the control plane here (loopback by default).
# Socket-shared CI joins the container directly to the job network;
# the host-side mapping remains loopback-only in that topology. The
# orchestrator listens on the fixed DEFAULT_PORT inside the
# container; self.port is the host-side published port.
"--publish", f"127.0.0.1:{self.port}:{DEFAULT_PORT}",
# Live control-plane source (code changes without an image rebuild).
"--volume", f"{self._repo_root}:{_SRC_IN_CONTAINER}:ro",
"--env", f"PYTHONPATH={_SRC_IN_CONTAINER}",
"--publish", f"{self._bind_host}:{self.port}:{DEFAULT_PORT}",
# The image was rebuilt from `_repo_root` immediately before this
# launch. Running its baked package avoids a host-path bind mount,
# which is both more production-like and works with socket-shared
# CI where the daemon cannot see the job container's workspace.
# Orchestrator registry DB on the host (sole writer: control plane).
"--volume", f"{self._host_root}:{_ROOT_IN_CONTAINER}",
# `root_mount_source` may be a host path or a named Docker volume.
"--volume", f"{self._root_mount_source}:{_ROOT_IN_CONTAINER}",
"--env", f"BOT_BOTTLE_ROOT={_ROOT_IN_CONTAINER}",
# The signing key — held ONLY by the orchestrator (it verifies
# tokens); the gateway gets the pre-minted `gateway` JWT, never the
@@ -205,6 +237,15 @@ class DockerOrchestrator(Orchestrator):
raise OrchestratorStartError(
f"orchestrator container failed to start: {proc.stderr.strip()}"
)
if self._client_network:
proc = run_docker([
"docker", "network", "connect", self._client_network, self.name,
])
if proc.returncode != 0:
raise OrchestratorStartError(
f"orchestrator container failed to join client network "
f"{self._client_network}: {proc.stderr.strip()}"
)
def stop(self) -> None:
"""Remove the control-plane container (idempotent)."""
+3 -3
View File
@@ -8,7 +8,7 @@ pointer, and `status()` reports whether docker is usable.
This is intentionally minimal; a richer version (daemon config checks,
gVisor/runsc install guidance, rootless-docker hints) is tracked
separately. Reached via `DockerBottleBackend.setup` / `.status`, which
the generic `./cli.py backend {setup,status}` dispatches to.
the generic `bot-bottle backend {setup,status}` dispatches to.
"""
from __future__ import annotations
@@ -69,7 +69,7 @@ def teardown() -> int:
sys.stderr.write(
"Docker backend: nothing to undo — it provisions no privileged host "
"state (networks and the gateway are per-launch and are "
"removed by `./cli.py cleanup`). Docker itself is left installed.\n"
"removed by `bot-bottle cleanup`). Docker itself is left installed.\n"
)
return 0
@@ -89,5 +89,5 @@ def status() -> int:
runsc = _docker_on_path() and _util.runsc_available()
sys.stderr.write(f"gVisor runsc runtime: {'registered' if runsc else 'not registered (optional)'}\n")
if not ok:
sys.stderr.write("\nRun: ./cli.py backend setup --backend=docker\n")
sys.stderr.write("\nRun: bot-bottle backend setup --backend=docker\n")
return 0 if ok else 1
+65 -15
View File
@@ -6,12 +6,18 @@ from __future__ import annotations
import os
from datetime import datetime, timezone
import re
import shutil
import subprocess
from typing import Iterator
from ... import resources
from ...log import die, info
from ...util import slugify as _slugify
def slugify(name: str) -> str:
"""Compatibility wrapper; new generic callers import ``bot_bottle.util``."""
return _slugify(name)
def run_docker(
@@ -114,20 +120,13 @@ def docker_cp(src: str, dest: str) -> None:
f"{(result.stderr or '').strip() or '<no stderr>'}")
_SLUG_RE = re.compile(r"[^a-z0-9]+")
def slugify(name: str) -> str:
"""Lowercase, non-alnum runs → '-', trimmed. Dies on empty result."""
if not name:
die("slugify: missing name")
slug = _SLUG_RE.sub("-", name.lower()).strip("-")
if not slug:
die(f"name '{name}' produced an empty slug; use alphanumeric characters")
return slug
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
def build_image(
ref: str,
context: str,
*,
dockerfile: str = "",
build_args: dict[str, str] | None = None,
) -> None:
"""Invokes `docker build` every call. Layer cache makes no-change
rebuilds cheap; running every time means Dockerfile edits land
without manual `docker rmi`.
@@ -147,10 +146,61 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
args.append("--no-cache")
if dockerfile:
args.extend(["-f", dockerfile])
effective_build_args = resources.image_build_args(
dockerfile,
context=context,
) if dockerfile else {}
effective_build_args.update(build_args or {})
for name, value in effective_build_args.items():
args.extend(["--build-arg", f"{name}={value}"])
args.append(context)
subprocess.run(args, check=True)
def image_id(ref: str) -> str:
"""Return the exact content-addressed ID for a local image.
This is used when one locally built image is another Dockerfile's base:
passing the ID prevents a mutable tag from being resolved between builds.
"""
result = run_docker(["docker", "image", "inspect", "--format", "{{.Id}}", ref])
image = result.stdout.strip()
if result.returncode != 0 or not image.startswith("sha256:"):
detail = (result.stderr or result.stdout or "").strip()
die(f"could not resolve exact image ID for {ref!r}: {detail or '<no detail>'}")
return image
def pinned_local_image_ref(ref: str) -> str:
"""Give a local image a content-derived tag and verify the tag resolves
back to the same image ID.
BuildKit treats a bare ``sha256:...`` ID in ``FROM`` as a registry
repository name. A tag whose complete suffix is the local image ID remains
resolvable by BuildKit, while the post-tag inspection keeps the handoff
fail-closed.
"""
image = image_id(ref)
digest = image.removeprefix("sha256:")
if len(digest) != 64 or any(char not in "0123456789abcdef" for char in digest):
die(f"could not derive a local base tag from invalid image ID {image!r}")
repository = ref.split("@", 1)[0]
last_slash = repository.rfind("/")
last_colon = repository.rfind(":")
if last_colon > last_slash:
repository = repository[:last_colon]
pinned_ref = f"{repository}:sha256-{digest}"
result = run_docker(["docker", "image", "tag", image, pinned_ref])
if result.returncode != 0:
detail = (result.stderr or result.stdout or "").strip()
die(f"could not tag exact local image {image}: {detail or '<no detail>'}")
if image_id(pinned_ref) != image:
die(f"content-derived local tag {pinned_ref!r} did not resolve to {image}")
return pinned_ref
def verify_agent_image(image: str, argv: tuple[str, ...]) -> None:
"""Run `argv` inside a throwaway container of a freshly built agent
image and die loudly if it fails, instead of shipping an image
+2 -1
View File
@@ -11,7 +11,8 @@ from pathlib import Path
from ..bottle_state import egress_state_dir
from ..egress import EGRESS_ROUTES_FILENAME
from ..gateway.egress.addon_core import LOG_OFF, load_config
from ..gateway.egress.schema import load_config
from ..gateway.egress.types import LOG_OFF
class EgressApplyError(RuntimeError):
@@ -27,3 +27,11 @@ class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
@property
def empty(self) -> bool:
return not (self.vm_pids or self.run_dirs)
def intersect(self, current: BottleCleanupPlan) -> "FirecrackerBottleCleanupPlan":
if not isinstance(current, FirecrackerBottleCleanupPlan):
raise TypeError("cleanup plans must have the same backend type")
return FirecrackerBottleCleanupPlan(
vm_pids=tuple(x for x in self.vm_pids if x in current.vm_pids),
run_dirs=tuple(x for x in self.run_dirs if x in current.run_dirs),
)
+101 -27
View File
@@ -11,10 +11,9 @@ Reaps *orphans* only — resources with no live VM behind them:
— a VMM left lingering after its dir was removed.
A run dir with a *live* firecracker process is a running bottle and is
left strictly alone: it is neither killed nor removed. (The backend's
`enumerate_active` registry is still a stub — #354 — so a live process
is the only reliable "this bottle is in use" signal we have. Once the
registry lands, registry-orphaned-but-running VMs can be reaped too.)
left strictly alone: it is neither killed nor removed. Active-agent
enumeration uses this same process snapshot, so cleanup and generic
backend consumers agree about which bottles are running.
TAP slots free themselves (the flock drops when the launcher exits), so
there is nothing to reclaim there.
@@ -22,14 +21,16 @@ there is nothing to reclaim there.
from __future__ import annotations
from collections.abc import Sequence
import os
import shutil
import signal
import subprocess
from pathlib import Path
from ...log import info
from . import util
from .. import EnumerationError
from ..cleanup_control import CleanupError, CleanupFailures
from . import lifecycle_lock, util
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
@@ -37,7 +38,7 @@ def _run_root() -> Path:
return util.cache_dir() / "run"
def _run_dir_of(cmd: str, run_root: Path) -> Path | None:
def _run_dir_of(args: Sequence[str], run_root: Path) -> Path | None:
"""The bottle run dir a firecracker cmdline belongs to, or None.
A bottle VM is launched with `--config-file <run_root>/<slug>/config.json`,
@@ -45,15 +46,35 @@ def _run_dir_of(cmd: str, run_root: Path) -> Path | None:
the run root. Anything else (a builder VM, the infra VM elsewhere) is
not ours to reap here.
"""
toks = cmd.split()
for i, tok in enumerate(toks):
if tok == "--config-file" and i + 1 < len(toks):
parent = Path(toks[i + 1]).parent
for i, arg in enumerate(args):
if arg == "--config-file" and i + 1 < len(args):
parent = Path(args[i + 1]).parent
if parent.parent == run_root:
return parent
return None
def _decode_cmdline(raw: bytes) -> tuple[str, ...]:
"""Decode Linux's NUL-delimited argv without losing embedded spaces."""
return tuple(
value.decode(errors="surrogateescape")
for value in raw.split(b"\0") if value
)
def _process_args(pid: int) -> tuple[str, ...] | None:
"""Read one process's lossless argv, or None when it exited meanwhile."""
try:
raw = Path(f"/proc/{pid}/cmdline").read_bytes()
except FileNotFoundError:
return None
except OSError as exc:
raise EnumerationError(
f"could not inspect Firecracker pid {pid}: {exc}"
) from exc
return _decode_cmdline(raw)
def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
"""Inspect running firecracker VMs under ``run_root``.
@@ -62,23 +83,34 @@ def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
* ``orphan_pids`` — firecracker pids whose run dir no longer exists
(a lingering VMM to kill).
"""
result = subprocess.run(
["pgrep", "-a", "firecracker"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
try:
result = subprocess.run(
["pgrep", "firecracker"],
capture_output=True, text=True, check=False,
)
except OSError as exc:
raise EnumerationError(
f"could not enumerate Firecracker processes: {exc}"
) from exc
if result.returncode == 1:
# pgrep's documented "no processes matched" result.
return set(), []
if result.returncode != 0:
detail = (result.stderr or "").strip() or f"exit {result.returncode}"
raise EnumerationError(
f"could not enumerate Firecracker processes: {detail}"
)
live: set[str] = set()
orphan_pids: list[int] = []
for line in result.stdout.splitlines():
parts = line.split(None, 1)
if len(parts) != 2:
continue
try:
pid = int(parts[0])
pid = int(line.strip())
except ValueError:
continue
run_dir = _run_dir_of(parts[1], run_root)
args = _process_args(pid)
if args is None:
continue
run_dir = _run_dir_of(args, run_root)
if run_dir is None:
continue
if run_dir.is_dir():
@@ -114,12 +146,54 @@ def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
def cleanup(plan: FirecrackerBottleCleanupPlan) -> None:
for pid in plan.vm_pids:
"""Revalidate the preview under the launch lock, then remove its survivors."""
with lifecycle_lock.hold():
fresh = prepare_cleanup()
approved_pids = set(plan.vm_pids).intersection(fresh.vm_pids)
approved_dirs = set(plan.run_dirs).intersection(fresh.run_dirs)
failures = CleanupFailures()
for pid in sorted(approved_pids):
try:
_terminate_orphan(pid, _run_root())
except CleanupError as exc:
failures.record(str(exc))
for path in sorted(approved_dirs):
info(f"rm -rf {path}")
failures.remove_tree(Path(path), f"removing Firecracker run dir {path}")
failures.raise_if_any()
def _terminate_orphan(pid: int, run_root: Path) -> None:
"""Signal exactly the process identity that still owns an orphan config."""
try:
pidfd = os.pidfd_open(pid)
except ProcessLookupError:
return
except OSError as exc:
raise EnumerationError(
f"could not pin Firecracker pid {pid} for cleanup: {exc}"
) from exc
try:
try:
raw = Path(f"/proc/{pid}/cmdline").read_bytes()
except FileNotFoundError:
return
except OSError as exc:
raise EnumerationError(
f"could not revalidate Firecracker pid {pid}: {exc}"
) from exc
args = _decode_cmdline(raw)
run_dir = _run_dir_of(args, run_root)
if run_dir is None or run_dir.is_dir():
return
info(f"kill firecracker VM pid {pid}")
try:
os.kill(pid, signal.SIGTERM)
signal.pidfd_send_signal(pidfd, signal.SIGTERM)
except ProcessLookupError:
pass
for path in plan.run_dirs:
info(f"rm -rf {path}")
shutil.rmtree(path, ignore_errors=True)
return
except OSError as exc:
raise CleanupError(
f"could not signal Firecracker pid {pid}: {exc}"
) from exc
finally:
os.close(pidfd)
@@ -26,22 +26,19 @@ The TAP slot allocation, rootfs build, and VM boot are the caller's job.
from __future__ import annotations
import json
import subprocess
from dataclasses import dataclass
from pathlib import Path
from ...egress import EgressPlan
from ...git_gate import GitGatePlan
from ...log import info
from ...orchestrator.client import OrchestratorClient, OrchestratorClientError
from ...orchestrator.client import OrchestratorClient
from ...orchestrator.lifecycle import (
OrchestratorStartError, # re-exported so callers can catch it
)
from ...orchestrator.reprovision import reprovision_bottles
from ...orchestrator.store.secret_store import ENV_VAR_SECRET_NAME
from ..provision_bottle import deprovision_bottle, provision_bottle
from . import cleanup, util
from . import util
from .gateway import FirecrackerGateway
from .infra import FirecrackerInfraService
@@ -64,17 +61,6 @@ class LaunchContext:
env_var_secret: str = "" # encryption key injected into the agent's env
def _guest_ip_from_config(config_path: Path) -> str:
"""Read the kernel's configured guest IP from a Firecracker config."""
try:
config = json.loads(config_path.read_text())
args = config["boot-source"]["boot_args"]
ip_arg = next(part for part in args.split() if part.startswith("ip="))
return ip_arg.removeprefix("ip=").split(":", 1)[0]
except (OSError, ValueError, KeyError, TypeError, StopIteration):
return ""
def persist_env_var_secret(private_key: Path, guest_ip: str, secret: str) -> None:
"""Mirror the exec-time key into guest tmpfs for restart recovery."""
proc = subprocess.run(
@@ -89,29 +75,6 @@ def persist_env_var_secret(private_key: Path, guest_ip: str, secret: str) -> Non
)
def _reprovision_running_bottles(client: OrchestratorClient) -> None:
"""Read keys from live agent VMs and restore the restarted gateway."""
try:
secrets_by_ip: dict[str, str] = {}
for run_dir in cleanup.live_run_dirs():
guest_ip = _guest_ip_from_config(run_dir / "config.json")
private_key = run_dir / "bottle_id_ed25519"
if not guest_ip or not private_key.is_file():
continue
proc = subprocess.run(
util.ssh_base_argv(private_key, guest_ip)
+ [f"cat {_ENV_VAR_SECRET_PATH}"],
capture_output=True, text=True, check=False,
)
if proc.returncode == 0 and proc.stdout.strip():
secrets_by_ip[guest_ip] = proc.stdout.strip()
count = reprovision_bottles(client, secrets_by_ip)
if count:
info(f"reprovisioned egress tokens for {count} Firecracker bottle(s)")
except (OSError, OrchestratorClientError) as exc:
info(f"egress token reprovision skipped: {exc}")
def launch_consolidated(
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
@@ -126,7 +89,6 @@ def launch_consolidated(
service = FirecrackerInfraService()
url = service.ensure_running()
client = OrchestratorClient(url)
_reprovision_running_bottles(client)
# Read the gateway's provisioning transport + CA off the Gateway service.
gateway = service.gateway()
+22 -4
View File
@@ -1,14 +1,32 @@
"""Active-agent enumeration for the Firecracker backend.
The backend is disabled during the companion-container removal (#385) — it can't
launch bottles, so there are none to enumerate. Real enumeration returns
with the backend's consolidated relaunch (#354).
Running bottles are the Firecracker processes whose ``--config-file`` points
at an existing per-bottle run directory. The same authoritative process scan
protects cleanup from deleting live VMs; operational scan failures propagate
as ``EnumerationError`` instead of masquerading as an empty host.
"""
from __future__ import annotations
from ...bottle_state import read_metadata
from .. import ActiveAgent
from .cleanup import live_run_dirs
def enumerate_active() -> list[ActiveAgent]:
return []
out: list[ActiveAgent] = []
for run_dir in live_run_dirs():
slug = run_dir.name
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="firecracker",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
# Firecracker uses the shared gateway, so there are no
# per-bottle gateway service containers to report.
services=(),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
@@ -19,12 +19,14 @@ from __future__ import annotations
import fcntl
import hashlib
import os
import shlex
import shutil
import subprocess
from contextlib import contextmanager
from pathlib import Path
from typing import Generator
from ... import resources
from ...log import die, info
from . import util
from .infra import FirecrackerInfraService
@@ -55,6 +57,11 @@ def _rootfs_digest(dockerfile: Path) -> str:
h = hashlib.sha256()
h.update(_dockerfile_hash(dockerfile).encode())
h.update(b"\0")
for name, value in resources.image_build_args(dockerfile).items():
h.update(name.encode())
h.update(b"=")
h.update(value.encode())
h.update(b"\0")
h.update(util._GUEST_INIT.encode())
return h.hexdigest()[:16]
@@ -147,7 +154,13 @@ def _build_in_infra(
if prep.returncode != 0:
die(f"preparing build dir in the infra VM failed: {prep.stderr.strip()}")
_send_dockerfile(key, ip, dockerfile, ctx)
_buildah_build(key, ip, ctx, tag)
_buildah_build(
key,
ip,
ctx,
tag,
resources.image_build_args(dockerfile),
)
_smoke_test(key, ip, tag, smoke_ctr, smoke_test)
_stream_rootfs(key, ip, tag, export_ctr, base)
finally:
@@ -184,15 +197,26 @@ def _send_dockerfile(private_key: Path, guest_ip: str, dockerfile: Path, ctx: st
f"{proc.stderr.decode(errors='replace').strip()}")
def _buildah_build(private_key: Path, guest_ip: str, ctx: str, tag: str) -> None:
def _buildah_build(
private_key: Path,
guest_ip: str,
ctx: str,
tag: str,
build_args: dict[str, str],
) -> None:
# Stream buildah's step-by-step output straight to our stderr (like the
# docker backend's `docker build`), so a long first build (base pull +
# apt/npm installs) shows live progress instead of a silent wait. The
# remote stderr is where buildah writes its `STEP i/n` lines.
info(f"buildah build {tag} in the infra VM (streaming output)")
arg_flags = " ".join(
f"--build-arg {shlex.quote(f'{name}={value}')}"
for name, value in build_args.items()
)
rc = _ssh_streamed(
private_key, guest_ip,
f"buildah build {_BUILD_FLAGS} -t {tag} -f {ctx}/Dockerfile {ctx}/ctx",
f"buildah build {_BUILD_FLAGS} {arg_flags} "
f"-t {tag} -f {ctx}/Dockerfile {ctx}/ctx",
timeout=_BUILD_TIMEOUT_SECONDS,
)
if rc != 0:
+4 -1
View File
@@ -17,6 +17,7 @@ from ...orchestrator.lifecycle import DEFAULT_STARTUP_TIMEOUT_SECONDS
from . import infra_vm
from .gateway import FirecrackerGateway
from .orchestrator import FirecrackerOrchestrator
from .reconcile import attach_bottled_agents_to_gateway
class FirecrackerInfraService(InfraService):
@@ -67,9 +68,11 @@ class FirecrackerInfraService(InfraService):
# holds the signing key) mints the role-scoped `gateway` JWT for the
# gateway, which never sees the key (#469).
orchestrator.ensure_running(startup_timeout=startup_timeout)
self.gateway().connect_to_orchestrator(
gateway = self.gateway()
gateway.connect_to_orchestrator(
orchestrator.gateway_url(), orchestrator.mint_gateway_token())
infra_vm.record_booted_version(want)
attach_bottled_agents_to_gateway(url, gateway)
return url
def stop(self) -> None:
@@ -41,6 +41,8 @@ from ... import resources
from ...log import die, info
from . import util
ARTIFACT_HTTP_TIMEOUT_SECONDS = 30.0
# Bump if the on-disk artifact *format* changes (compression, layout) so a new
# scheme can't collide with a cached/published artifact of the old one.
_ARTIFACT_FORMAT = "1"
@@ -49,9 +51,18 @@ _ARTIFACT_FORMAT = "1"
# from its own generic package; the Dockerfiles baked into each differ (only the
# orchestrator rootfs carries buildah), so the versions are hashed separately.
ROLES = ("orchestrator", "gateway")
_DOCKERFILES = {
"orchestrator": ("Dockerfile.orchestrator", "Dockerfile.orchestrator.fc"),
"gateway": ("Dockerfile.gateway",),
_BUILD_INPUTS = {
"orchestrator": (
"image-build-args.json",
"Dockerfile.orchestrator",
"Dockerfile.orchestrator.fc",
"requirements.orchestrator.lock",
),
"gateway": (
"image-build-args.json",
"Dockerfile.gateway",
"requirements.gateway.lock",
),
}
_DEFAULT_BASE = "https://gitea.dideric.is"
@@ -101,7 +112,7 @@ def infra_artifact_version(
h.update(str(path.relative_to(repo_root)).encode())
h.update(b"\0")
h.update(path.read_bytes())
for name in _DOCKERFILES[role]:
for name in _BUILD_INPUTS[role]:
h.update(name.encode())
h.update(b"\0")
h.update((repo_root / name).read_bytes())
@@ -155,7 +166,9 @@ def _download(url: str, dest: Path) -> None:
"""Stream `url` to `dest` (atomic via a `.part` sibling)."""
tmp = dest.with_suffix(dest.suffix + ".part")
try:
with urllib.request.urlopen(_open(url)) as resp, open(tmp, "wb") as out:
with urllib.request.urlopen(
_open(url), timeout=ARTIFACT_HTTP_TIMEOUT_SECONDS,
) as resp, open(tmp, "wb") as out:
shutil.copyfileobj(resp, out, _CHUNK)
except urllib.error.HTTPError as e:
tmp.unlink(missing_ok=True)
+7 -2
View File
@@ -133,10 +133,15 @@ def build_infra_images_with_docker() -> None:
root = str(resources.build_root())
docker_mod.build_image(
_ORCHESTRATOR_IMAGE, root, dockerfile="Dockerfile.orchestrator")
orchestrator_base = docker_mod.pinned_local_image_ref(_ORCHESTRATOR_IMAGE)
docker_mod.build_image(
_GATEWAY_IMAGE, root, dockerfile="Dockerfile.gateway")
docker_mod.build_image(
_ORCHESTRATOR_FC_IMAGE, root, dockerfile="Dockerfile.orchestrator.fc")
_ORCHESTRATOR_FC_IMAGE,
root,
dockerfile="Dockerfile.orchestrator.fc",
build_args={"ORCHESTRATOR_BASE_IMAGE": orchestrator_base},
)
def build_rootfs_dir(role: str) -> Path:
@@ -199,7 +204,7 @@ def boot_vm(
Records the PID."""
if not netpool.tap_present(slot.iface):
die(f"infra link {slot.iface} not present.\n"
f" ./cli.py backend setup --backend=firecracker")
f" bot-bottle backend setup --backend=firecracker")
run_dir.mkdir(parents=True, exist_ok=True)
rootfs = run_dir / "rootfs.ext4"
@@ -108,7 +108,7 @@ def verify_isolation(private_key: Path, guest_ip: str) -> None:
die(f"ISOLATION FAILURE: the VM reached the host canary "
f"{canary_ip}:{canary_port}. The egress boundary is not in "
f"force — refusing to run the agent (fail-closed). Verify the "
f"nft table with: ./cli.py backend setup --backend=firecracker")
f"nft table with: bot-bottle backend setup --backend=firecracker")
if result.returncode == 2:
die("isolation probe inconclusive: the guest has no python3/bash/nc "
"to run the connectivity test. Refusing to continue "
+23 -19
View File
@@ -46,7 +46,7 @@ from ...log import die, info, warn
from ...supervisor.types import SUPERVISE_PORT
from ..docker.egress import EGRESS_PORT
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import firecracker_vm, image_builder, isolation_probe, netpool, util
from . import firecracker_vm, image_builder, isolation_probe, lifecycle_lock, netpool, util
from .bottle import FirecrackerBottle
from .bottle_plan import FirecrackerBottlePlan
from ...orchestrator.store.config_store import resolve_teardown_timeout
@@ -164,25 +164,29 @@ def launch(
)
# Step 6: build the per-bottle rootfs + SSH key, then boot.
run_dir = util.cache_dir() / "run" / plan.slug
run_dir.mkdir(parents=True, exist_ok=True)
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
# doesn't leak. Registered before vm.terminate below so it runs *after*
# it (ExitStack is LIFO): the VM is gone before we rm its rootfs.
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
rootfs = run_dir / "rootfs.ext4"
util.build_rootfs_ext4(agent_base, rootfs)
private_key, pubkey = util.generate_keypair(run_dir)
# Cleanup takes the same lock while refreshing its process snapshot.
# Hold it until the VMM exists so a newly-created run dir can never be
# mistaken for an orphan in the build-before-boot window.
with lifecycle_lock.hold():
run_dir = util.cache_dir() / "run" / plan.slug
run_dir.mkdir(parents=True, exist_ok=True)
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
# doesn't leak. Registered before vm.terminate below so it runs
# *after* it (ExitStack is LIFO).
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
rootfs = run_dir / "rootfs.ext4"
util.build_rootfs_ext4(agent_base, rootfs)
private_key, pubkey = util.generate_keypair(run_dir)
vm = firecracker_vm.boot(
name=plan.container_name,
rootfs=rootfs,
tap=slot.iface,
guest_ip=slot.guest_ip,
host_ip=slot.host_ip,
pubkey=pubkey,
run_dir=run_dir,
)
vm = firecracker_vm.boot(
name=plan.container_name,
rootfs=rootfs,
tap=slot.iface,
guest_ip=slot.guest_ip,
host_ip=slot.host_ip,
pubkey=pubkey,
run_dir=run_dir,
)
stack.callback(vm.terminate)
firecracker_vm.wait_for_ssh(vm, private_key)
persist_env_var_secret(private_key, slot.guest_ip, ctx.env_var_secret)
@@ -0,0 +1,30 @@
"""Serialize Firecracker run-directory creation with orphan cleanup."""
from __future__ import annotations
import fcntl
from contextlib import contextmanager
from pathlib import Path
from typing import Generator
from . import util
def _lock_path() -> Path:
return util.cache_dir() / "run.lifecycle.lock"
@contextmanager
def hold() -> Generator[None]:
"""Exclude cleanup while a launch directory lacks a visible VMM."""
path = _lock_path()
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("a", encoding="utf-8") as handle:
fcntl.flock(handle, fcntl.LOCK_EX)
try:
yield
finally:
fcntl.flock(handle, fcntl.LOCK_UN)
__all__ = ["hold"]
+14 -6
View File
@@ -3,7 +3,7 @@ config renderers (shell command + NixOS module) shown to operators.
The Firecracker backend needs a privileged one-time network setup:
a pool of point-to-point TAP devices (owned by the invoking user, so
`./cli.py start` never needs root) and a dedicated nftables table that
`bot-bottle start` never needs root) and a dedicated nftables table that
isolates every VM. The pool parameters live in exactly one place
`netpool.defaults.env`, a plain KEY=VALUE file next to this module
and every consumer reads *that*: this module (below), the shell script
@@ -177,14 +177,20 @@ def gw_slot() -> Slot:
# --- fail-closed verification ---------------------------------------
def _run_ok(argv: list[str]) -> bool:
"""Run a probe command, treating a missing binary as failure
"""Run a probe command, treating an unavailable binary as failure
(rather than crashing) so callers can stay fail-closed."""
try:
return subprocess.run(
argv, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
).returncode == 0
except FileNotFoundError:
except OSError:
# Not only "missing". A name on PATH that isn't executable by this
# user raises PermissionError, and CPython reports that EACCES in
# preference to the ENOENT from the other PATH entries — which is
# how `doctor` came to die with a traceback on a fresh macOS
# account. Any OSError means the probe couldn't run, which for a
# fail-closed check is indistinguishable from "not present".
return False
@@ -244,7 +250,9 @@ def overlapping_routes() -> list[RouteConflict]:
["ip", "-json", "route", "show", "table", "all"],
capture_output=True, text=True, check=False,
)
except FileNotFoundError:
except OSError:
# Missing, or present-but-not-executable for this user; either way
# there are no routes we can enumerate. See _run_ok.
return []
if proc.returncode != 0 or not proc.stdout.strip():
return []
@@ -307,11 +315,11 @@ def allocate(slug: str) -> tuple[Slot, IO[str]]:
return s, handle
die(f"Firecracker TAP pool exhausted ({pool_size()} slots, all in "
f"use). Stop a running bottle or raise BOT_BOTTLE_FC_POOL_SIZE "
f"and re-run `./cli.py backend setup --backend=firecracker`.")
f"and re-run `bot-bottle backend setup --backend=firecracker`.")
raise AssertionError("unreachable")
# --- config renderers (shown by `./cli.py backend setup`) -----------
# --- config renderers (shown by `bot-bottle backend setup`) -----------
# The persistent unit is the portable install: the same systemd oneshot
# on every systemd distro (Debian/Ubuntu/Fedora/RHEL/Arch/…).
@@ -34,6 +34,7 @@ from pathlib import Path
from . import infra_artifact, infra_vm, util
_CHUNK = 1 << 20
_REGISTRY_HTTP_TIMEOUT_SECONDS = 30.0
_GZ_NAME = "rootfs.ext4.gz"
_SHA_NAME = "rootfs.ext4.gz.sha256"
@@ -91,7 +92,9 @@ def _put(url: str, body: "bytes | Path", token: str) -> None:
req.add_header("Authorization", f"token {token}")
req.add_header("Content-Type", "application/octet-stream")
try:
with urllib.request.urlopen(req) as resp:
with urllib.request.urlopen(
req, timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
) as resp:
print(f" uploaded {url} (HTTP {resp.status})")
except urllib.error.HTTPError as e:
if e.code == 409:
@@ -112,7 +115,9 @@ def _delete(url: str, token: str) -> None:
if token:
req.add_header("Authorization", f"token {token}")
try:
with urllib.request.urlopen(req):
with urllib.request.urlopen(
req, timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
):
pass
except urllib.error.HTTPError as e:
if e.code != 404:
@@ -151,7 +156,10 @@ def _try_download_published(role: str, role_dir: Path) -> str | None:
version = _role_version(role)
sha_url = infra_artifact.artifact_url(version, _SHA_NAME, role=role)
try:
with urllib.request.urlopen(infra_artifact._open(sha_url)):
with urllib.request.urlopen(
infra_artifact._open(sha_url),
timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
):
pass
except urllib.error.HTTPError as e:
if e.code == 404:
@@ -195,7 +203,10 @@ def _publish_bundle(role: str, role_dir: Path, token: str) -> str:
# present, a re-publish is a no-op. Otherwise clear any partial upload left
# by an interrupted prior attempt and upload the complete set.
try:
with urllib.request.urlopen(infra_artifact._open(sha_url)) as resp:
with urllib.request.urlopen(
infra_artifact._open(sha_url),
timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
) as resp:
remote_sha = resp.read().decode("utf-8").split()[0].strip().lower()
except urllib.error.HTTPError as e:
if e.code != 404:
+199
View File
@@ -0,0 +1,199 @@
"""Bring-up reconcile: re-attach all running agent VMs to a freshly-booted gateway.
Called from the cold-boot branch of `FirecrackerInfraService.ensure_running()`
after `gateway.connect_to_orchestrator()` completes. Restores the three
gateway-dependent services for every live bottle:
- **CA** push the current (freshly-minted) gateway CA to each agent's
trust store and run `update-ca-certificates`, so the agent trusts the
new CA on its next egress call.
- **git-gate** re-provision each bottle's bare repos and per-repo creds
in the gateway, rebuilt from the persisted host-side state dir.
- **egress tokens** read the agent's `ENV_VAR_SECRET` and feed
`reprovision_bottles` to restore the orchestrator's in-memory tokens.
Per-bottle failures are caught, logged, and skipped so one unreachable VM
does not block the others or the gateway coming up (PRD 0081).
"""
from __future__ import annotations
import json
import subprocess
from pathlib import Path
from ...bottle_state import git_gate_state_dir
from ...gateway import GatewayError
from ...gateway.git_gate.render import GitGateUpstream
from ...git_gate.plan import GitGatePlan
from ...log import info
from ...orchestrator.client import OrchestratorClient, OrchestratorClientError
from ...orchestrator.reprovision import reprovision_bottles
from ..provision_gateway import provision_git_gate
from ..util import AGENT_CA_PATH
from . import cleanup, util
from .gateway import FirecrackerGateway
# Where persist_env_var_secret writes the key on the agent VM (matches
# consolidated_launch._ENV_VAR_SECRET_PATH — duplicated to avoid a
# circular import through infra.py).
_ENV_VAR_SECRET_PATH = "/run/bot-bottle/env-var-secret"
def _guest_ip_from_config(config_path: Path) -> str:
"""Read the agent VM's guest IP from its Firecracker config file."""
try:
config = json.loads(config_path.read_text())
args = config["boot-source"]["boot_args"]
ip_arg = next(p for p in args.split() if p.startswith("ip="))
return ip_arg.removeprefix("ip=").split(":", 1)[0]
except (OSError, ValueError, KeyError, TypeError, StopIteration):
return ""
def attach_bottled_agents_to_gateway(
orchestrator_url: str, gateway: FirecrackerGateway,
) -> None:
"""Reconcile all live agent VMs against the freshly-booted gateway.
Fetches the new CA, lists registered bottles, then for each live run dir
pushes the CA, re-provisions git-gate, and restores egress tokens. Each
per-bottle step is wrapped so a failure is logged and skipped."""
try:
ca_pem = gateway.ca_cert_pem()
except GatewayError as exc:
info(f"bring-up reconcile: could not fetch gateway CA, skipping: {exc}")
return
client = OrchestratorClient(orchestrator_url)
try:
bottles = client.list_bottles()
except OrchestratorClientError as exc:
info(f"bring-up reconcile: could not list bottles, skipping: {exc}")
return
source_ip_to_bottle_id: dict[str, str] = {}
for b in bottles:
source_ip = b.get("source_ip")
bottle_id = b.get("bottle_id")
if isinstance(source_ip, str) and isinstance(bottle_id, str):
source_ip_to_bottle_id[source_ip] = bottle_id
transport = gateway.provisioning_transport()
secrets_by_ip: dict[str, str] = {}
for run_dir in cleanup.live_run_dirs():
slug = run_dir.name
guest_ip = _guest_ip_from_config(run_dir / "config.json")
private_key = run_dir / "bottle_id_ed25519"
if not guest_ip or not private_key.is_file():
continue
# CA: push the gateway's current certificate to the agent's trust store.
try:
_push_ca(private_key, guest_ip, ca_pem)
except Exception as exc:
info(f"bring-up reconcile: CA push to {slug!r} failed: {exc}")
# git-gate: re-provision repos + creds from the persisted state dir.
bottle_id = source_ip_to_bottle_id.get(guest_ip)
if bottle_id:
try:
_reprovision_git_gate(transport, bottle_id, slug)
except Exception as exc:
info(f"bring-up reconcile: git-gate for {slug!r} failed: {exc}")
# egress tokens: read ENV_VAR_SECRET from the agent's tmpfs.
proc = subprocess.run(
util.ssh_base_argv(private_key, guest_ip) + [f"cat {_ENV_VAR_SECRET_PATH}"],
capture_output=True, text=True, check=False,
)
if proc.returncode == 0 and proc.stdout.strip():
secrets_by_ip[guest_ip] = proc.stdout.strip()
# Restore in-memory egress tokens for all bottles that exposed a key.
if secrets_by_ip:
try:
count = reprovision_bottles(client, secrets_by_ip)
if count:
info(
f"bring-up reconcile: restored egress tokens for {count} bottle(s)"
)
except OrchestratorClientError as exc:
info(f"bring-up reconcile: egress token restore failed: {exc}")
def _push_ca(private_key: Path, guest_ip: str, ca_pem: str) -> None:
"""SSH the gateway CA PEM into the agent VM and update its trust store.
Runs as the SSH root user (the agent VM's dropbear accepts root). Each
step uses check=True so a failure raises and the caller's except clause
logs and continues."""
ssh = util.ssh_base_argv(private_key, guest_ip)
# mkdir is idempotent; rootfs builds may not preserve the target dir.
subprocess.run(
ssh + ["mkdir -p /usr/local/share/ca-certificates"],
capture_output=True, check=True,
)
subprocess.run(
ssh + [f"cat > {AGENT_CA_PATH}"],
input=ca_pem, text=True, capture_output=True, check=True,
)
subprocess.run(
ssh + [f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates"],
capture_output=True, check=True,
)
def _reprovision_git_gate(
transport: object, bottle_id: str, slug: str,
) -> None:
"""Re-provision one bottle's git-gate repos and creds from its state dir.
Reads `upstreams.json` (written by `provision_git_gate_dynamic_keys` at
launch) to reconstruct the GitGateUpstream table, then calls
`provision_git_gate` which places the credential files and inits the bare
repos in the gateway. No-op when there is no `upstreams.json` (the bottle
has no git upstreams)."""
state_dir = git_gate_state_dir(slug)
upstreams_file = state_dir / "upstreams.json"
if not upstreams_file.exists():
return
raw = json.loads(upstreams_file.read_text())
upstreams: list[GitGateUpstream] = []
for u in raw:
name = u["name"]
# The key in the state dir is preferred: gitea deploy keys are written
# there by provision_git_gate_dynamic_keys; static keys keep their
# original manifest path.
key_in_state = state_dir / f"{name}-key"
identity_file = (
str(key_in_state) if key_in_state.is_file()
else u.get("identity_file", "")
)
known_hosts = state_dir / f"{name}-known_hosts"
upstreams.append(GitGateUpstream(
name=name,
upstream_url=u["upstream_url"],
upstream_host=u.get("upstream_host", ""),
upstream_port=u.get("upstream_port", ""),
identity_file=identity_file,
known_host_key=u.get("known_host_key", ""),
known_hosts_file=known_hosts if known_hosts.is_file() else Path(),
))
if not upstreams:
return
plan = GitGatePlan(
slug=slug,
entrypoint_script=state_dir / "git_gate_entrypoint.sh",
hook_script=state_dir / "git_gate_pre_receive.sh",
access_hook_script=state_dir / "git_gate_access_hook.sh",
upstreams=tuple(upstreams),
)
provision_git_gate(transport, bottle_id, plan) # type: ignore[arg-type]
__all__ = ["attach_bottled_agents_to_gateway"]
+5 -5
View File
@@ -8,7 +8,7 @@ bundled setup script. `status()` reports what's present, including
whether the pool range collides with an existing route.
Called through `FirecrackerBottleBackend.setup` / `.status`, which the
generic `./cli.py backend {setup,status}` command dispatches to.
generic `bot-bottle backend {setup,status}` command dispatches to.
"""
from __future__ import annotations
@@ -34,7 +34,7 @@ _UNIT_PATH = Path("/etc/systemd/system") / netpool.SYSTEMD_UNIT
def _owner() -> str:
# Under `sudo`, USER is root but SUDO_USER is the real invoker — the
# TAPs must be owned by them so `./cli.py start` stays rootless.
# TAPs must be owned by them so `bot-bottle start` stays rootless.
return os.environ.get("SUDO_USER") or os.environ.get("USER") or "youruser"
@@ -161,7 +161,7 @@ def _setup_systemd() -> None:
if rc == 0:
sys.stderr.write(
f"Installed and started {netpool.SYSTEMD_UNIT}. Verify with "
f"`./cli.py backend status --backend=firecracker`.\n"
f"`bot-bottle backend status --backend=firecracker`.\n"
)
else:
sys.stderr.write(
@@ -181,7 +181,7 @@ def _setup_systemd() -> None:
)
sys.stderr.write(
f"\n(Or re-run this as root to install it directly: "
f"sudo ./cli.py backend setup --backend=firecracker)\n"
f"sudo bot-bottle backend setup --backend=firecracker)\n"
)
@@ -334,7 +334,7 @@ def status() -> int:
sys.stderr.write(f"range overlap: none (base {netpool.ip_base()})\n")
_report_persistence()
if not ok:
sys.stderr.write("\nRun: ./cli.py backend setup --backend=firecracker\n")
sys.stderr.write("\nRun: bot-bottle backend setup --backend=firecracker\n")
return 0 if ok else 1
+4 -4
View File
@@ -9,7 +9,7 @@ generation.
The privileged network setup (TAP pool + nft table) is a one-time
operator step see `netpool.py`, `scripts/firecracker-netpool.sh`,
and `./cli.py backend setup --backend=firecracker`.
and `bot-bottle backend setup --backend=firecracker`.
"""
from __future__ import annotations
@@ -132,18 +132,18 @@ def _require_network_pool() -> None:
f"{netpool.pool_size()} slots) overlaps existing routes: "
f"{detail}. This can shadow or be shadowed by that route; "
f"set BOT_BOTTLE_FC_IP_BASE to a free range and re-run "
f"./cli.py backend setup --backend=firecracker.")
f"bot-bottle backend setup --backend=firecracker.")
missing = netpool.missing_taps()
if missing:
die(f"network pool incomplete — missing TAP devices: "
f"{', '.join(missing)}.\n ./cli.py backend setup --backend=firecracker")
f"{', '.join(missing)}.\n bot-bottle backend setup --backend=firecracker")
if shutil.which("nft") is not None and not netpool.nft_table_present():
# nft is queryable and says the table is absent — that's a
# definite, catchable misconfiguration; fail early.
warn(f"isolation table `inet {netpool.NFT_TABLE}` not found via nft. "
"If this is a permissions issue it will be re-checked "
"empirically after boot; otherwise run: "
"./cli.py backend setup --backend=firecracker")
"bot-bottle backend setup --backend=firecracker")
# --- rootfs pipeline (rootless) -------------------------------------
+2 -2
View File
@@ -43,7 +43,7 @@ class Freezer(ABC):
Calls _freeze for the backend-specific snapshot, then writes the
committed image reference to per-bottle state and marks the bottle
preserved so the next `./cli.py resume` boots from the snapshot.
preserved so the next `bot-bottle resume` boots from the snapshot.
Raises CommitCancelled if the user declines an interactive
confirmation prompt (e.g. the macos-container stop prompt).
@@ -51,7 +51,7 @@ class Freezer(ABC):
image_ref = self._freeze(agent)
write_committed_image(agent.slug, image_ref)
mark_preserved(agent.slug)
info(f"to resume from this snapshot: ./cli.py resume {agent.slug}")
info(f"to resume from this snapshot: bot-bottle resume {agent.slug}")
self._export_hint(agent.slug, image_ref)
@abstractmethod
@@ -25,3 +25,11 @@ class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
@property
def empty(self) -> bool:
return not self.containers and not self.networks
def intersect(self, current: BottleCleanupPlan) -> "MacosContainerBottleCleanupPlan":
if not isinstance(current, MacosContainerBottleCleanupPlan):
raise TypeError("cleanup plans must have the same backend type")
return MacosContainerBottleCleanupPlan(
containers=tuple(x for x in self.containers if x in current.containers),
networks=tuple(x for x in self.networks if x in current.networks),
)
+13 -12
View File
@@ -4,7 +4,9 @@ from __future__ import annotations
import subprocess
from ...log import info, warn
from .. import EnumerationError
from ..cleanup_control import CleanupFailures
from ...log import info
from . import util as container_mod
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
@@ -19,8 +21,8 @@ def _list_prefixed_containers() -> list[str]:
check=False,
)
if result.returncode != 0:
warn(f"container list failed: {result.stderr.strip()}")
return []
detail = result.stderr.strip() or f"exit {result.returncode}"
raise EnumerationError(f"container list failed: {detail}")
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX)
@@ -35,7 +37,8 @@ def _list_prefixed_networks() -> list[str]:
check=False,
)
if result.returncode != 0:
return []
detail = result.stderr.strip() or f"exit {result.returncode}"
raise EnumerationError(f"container network list failed: {detail}")
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX)
@@ -51,19 +54,17 @@ def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
failures = CleanupFailures()
for name in plan.containers:
info(f"container delete --force {name}")
subprocess.run(
failures.run(
["container", "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
f"deleting container {name}",
)
for name in plan.networks:
info(f"container network delete {name}")
subprocess.run(
failures.run(
["container", "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
f"deleting network {name}",
)
failures.raise_if_any()
+12 -11
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent
from .. import ActiveAgent, EnumerationError
from .infra import INFRA_NAME, ORCHESTRATOR_NAME
# The name every agent container carries: `bot-bottle-<slug>`. Exported
@@ -20,17 +20,18 @@ CONTAINER_NAME_PREFIX = "bot-bottle-"
_INFRA_NAMES = frozenset({INFRA_NAME, ORCHESTRATOR_NAME})
class EnumerationError(RuntimeError):
"""container list failed; the resulting live set is not authoritative."""
def enumerate_active() -> list[ActiveAgent]:
result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
try:
result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
except FileNotFoundError as exc:
raise EnumerationError(
"container list failed: container CLI not found"
) from exc
if result.returncode != 0:
raise EnumerationError(
f"container list failed: "
+6 -2
View File
@@ -107,7 +107,8 @@ def _layer_nested_containers(
"""
if not plan.nested_containers:
return agent_image
derived = f"{agent_image}{nested_containers_mod.IMAGE_SUFFIX}"
pinned_base = container_mod.pinned_local_image_ref(agent_image)
derived = f"{pinned_base}{nested_containers_mod.IMAGE_SUFFIX}"
if plan.spec.image_policy == "cached":
if not container_mod.image_exists(derived):
die(
@@ -116,7 +117,10 @@ def _layer_nested_containers(
)
info(f"using cached nested-container image {derived!r}")
return derived
return nested_containers_mod.build_image(agent_image, container_mod.build_image)
return nested_containers_mod.build_image(
pinned_base,
container_mod.build_image,
)
@contextmanager
@@ -55,7 +55,7 @@ _GUEST_DEVICES = ("/dev/fuse", "/dev/net/tun")
def build_image(
base_image: str,
pinned_base: str,
build: Callable[..., None],
) -> str:
"""Layer the nested-container tooling onto an already-built agent image.
@@ -66,14 +66,15 @@ def build_image(
# TODO(#394): replace this hand-rolled Dockerfile with a docker-layer
# abstraction once that infrastructure exists.
"""
image = f"{base_image}{IMAGE_SUFFIX}"
image = f"{pinned_base}{IMAGE_SUFFIX}"
init_script = Path(__file__).with_name("nested-containers-init.sh")
with tempfile.TemporaryDirectory(prefix="bot-bottle-nested-containers.") as tmp:
context = Path(tmp)
shutil.copy2(init_script, context / "nested-containers-init.sh")
(context / "Dockerfile").write_text(
"FROM docker:28-cli AS docker_cli\n"
f"FROM {base_image}\n"
"ARG DOCKER_CLI_BASE_IMAGE\n"
"FROM ${DOCKER_CLI_BASE_IMAGE} AS docker_cli\n"
f"FROM {pinned_base}\n"
"USER root\n"
"COPY --from=docker_cli /usr/local/bin/docker /usr/local/bin/docker\n"
"COPY --from=docker_cli /usr/local/libexec/docker/cli-plugins/"
+2 -2
View File
@@ -5,7 +5,7 @@ Like Docker, this backend needs no privileged network-pool provisioning
running. `setup()` points at the install/`container system start` steps;
`status()` reports readiness. Reached via
`MacosContainerBottleBackend.setup` / `.status`, dispatched from the
generic `./cli.py backend {setup,status}`.
generic `bot-bottle backend {setup,status}`.
"""
from __future__ import annotations
@@ -75,5 +75,5 @@ def status() -> int:
if not _service_running():
ok = False
if not ok:
sys.stderr.write("\nRun: ./cli.py backend setup --backend=macos-container\n")
sys.stderr.write("\nRun: bot-bottle backend setup --backend=macos-container\n")
return 0 if ok else 1
+56 -1
View File
@@ -13,6 +13,7 @@ import time
from datetime import datetime, timezone
from typing import Iterable
from ... import resources
from ...log import die, info
@@ -60,7 +61,13 @@ def dns_server() -> str:
return _host_ipv4_dns() or _DEFAULT_DNS
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
def build_image(
ref: str,
context: str,
*,
dockerfile: str = "",
build_args: dict[str, str] | None = None,
) -> None:
"""Build an OCI image with Apple's BuildKit-backed `container build`.
Set `BOT_BOTTLE_NO_CACHE=1` (the `start --no-cache` flag) to force
@@ -83,6 +90,13 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args.extend(["-f", dockerfile])
effective_build_args = resources.image_build_args(
dockerfile,
context=context,
) if dockerfile else {}
effective_build_args.update(build_args or {})
for name, value in effective_build_args.items():
args.extend(["--build-arg", f"{name}={value}"])
args.append(context)
subprocess.run(args, check=True)
@@ -668,6 +682,47 @@ def image_id(ref: str) -> str:
raise AssertionError("unreachable")
def pinned_local_image_ref(ref: str) -> str:
"""Tag a local image with its complete content ID for a stable ``FROM``.
Agent images are immediately used as bases for the optional
nested-containers layer. A content-derived tag prevents another concurrent
build from moving the provider's ordinary ``:latest`` tag between those
two builds.
Unlike Docker, `container image tag` only accepts ``image-name[:tag]`` as
its source and rejects a bare image ID ("cannot specify 64 byte hex string
as reference"), so the mutable ``ref`` is what gets tagged here. The
post-tag inspection below is what keeps that fail-closed: if ``ref`` moved
between the two commands, the new tag will not resolve to the ID this call
derived its name from.
"""
image = image_id(ref)
digest = image.removeprefix("sha256:")
if len(digest) != 64 or any(char not in "0123456789abcdef" for char in digest):
die(f"could not derive a local base tag from invalid image ID {image!r}")
repository = ref.split("@", 1)[0]
last_slash = repository.rfind("/")
last_colon = repository.rfind(":")
if last_colon > last_slash:
repository = repository[:last_colon]
pinned_ref = f"{repository}:sha256-{digest}"
result = subprocess.run(
[_CONTAINER, "image", "tag", ref, pinned_ref],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"could not tag exact local image {image} via {ref!r}: "
f"{(result.stderr or result.stdout or '').strip() or '<no detail>'}"
)
if image_id(pinned_ref) != image:
die(f"content-derived local tag {pinned_ref!r} did not resolve to {image}")
return pinned_ref
def image_created_at(ref: str) -> datetime | None:
"""Return the image creation timestamp as an aware UTC datetime, or None
when the field is absent or unparseable (e.g. FROM-scratch images, images
+124
View File
@@ -0,0 +1,124 @@
"""Backend-neutral preparation planner.
This module owns the shared transformation from a CLI ``BottleSpec`` to the
typed inputs consumed by a concrete backend's ``_resolve_plan``. Backend
classes retain only their validation/preflight/env hooks and their
backend-specific final resolution.
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from typing import TYPE_CHECKING, Protocol
from ..agent_provider import AgentProvisionPlan, build_agent_provision_plan, get_provider
from ..egress import EgressPlan
from ..env import ResolvedEnv, resolve_env
from ..git_gate import GitGate, GitGatePlan
from ..manifest import Manifest
from ..supervisor.plan import SupervisePlan
from ..workspace import workspace_plan
from .resolve_common import (
merge_provision_env_vars,
mint_slug,
prepare_agent_state_dir,
prepare_egress,
prepare_git_gate,
prepare_supervise,
reject_nested_containers,
resolve_manifest_dockerfile,
write_launch_metadata,
)
if TYPE_CHECKING:
from .base import BottleSpec
class PreparationBackend(Protocol):
"""Backend hooks needed by the shared planner."""
name: str
supports_nested_containers: bool
def _validate(self, spec: BottleSpec) -> Manifest: ...
def _preflight(self) -> None: ...
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]: ...
@dataclass(frozen=True)
class PreparedBottle:
"""Typed, backend-neutral result of shared launch preparation."""
manifest: Manifest
slug: str
resolved_env: ResolvedEnv
agent_provision_plan: AgentProvisionPlan
egress_plan: EgressPlan
git_gate_plan: GitGatePlan
supervise_plan: SupervisePlan | None
class BottlePreparationPlanner:
"""Run the common, side-effect-limited part of bottle preparation."""
def __init__(self, backend: PreparationBackend) -> None:
self._backend = backend
def prepare(self, spec: BottleSpec) -> PreparedBottle:
backend = self._backend
# These are deliberately protected backend hooks: only this shared
# planner orchestrates them, while concrete backends provide the
# implementation.
manifest = backend._validate(spec) # pylint: disable=protected-access
if not backend.supports_nested_containers:
reject_nested_containers(backend.name, manifest)
backend._preflight() # pylint: disable=protected-access
manifest = GitGate().preflight_host_keys(
manifest,
headless=spec.headless,
home_md=spec.manifest.home_md,
)
bottle = manifest.bottle
provider_config = bottle.agent_provider
provider = get_provider(provider_config.template)
resolved_env = resolve_env(manifest)
workspace = workspace_plan(spec, guest_home=provider.guest_home)
slug = mint_slug(spec)
write_launch_metadata(slug, spec, compose_project="", backend=backend.name)
dockerfile = (
resolve_manifest_dockerfile(provider_config.dockerfile, spec)
if provider_config.dockerfile
else str(provider.dockerfile)
)
agent_dir, prompt_file = prepare_agent_state_dir(slug, manifest)
provision = build_agent_provision_plan(
template=provider_config.template,
dockerfile=dockerfile,
state_dir=agent_dir,
instance_name=f"bot-bottle-{slug}",
prompt_file=prompt_file,
guest_env=backend._build_guest_env( # pylint: disable=protected-access
resolved_env
),
forward_host_credentials=provider_config.forward_host_credentials,
auth_token=provider_config.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace.workdir,
label=spec.label,
color=spec.color,
provider_settings=provider_config.settings,
)
provision = merge_provision_env_vars(provision)
return PreparedBottle(
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=provision,
egress_plan=prepare_egress(bottle, slug, provision),
git_gate_plan=prepare_git_gate(bottle, slug),
supervise_plan=prepare_supervise(bottle, slug),
)
+13 -2
View File
@@ -63,12 +63,23 @@ def provision_git_gate(
transport.exec(["chmod", "+x", "/etc/git-gate/access-hook"])
creds = _creds_dir(bottle_id)
transport.exec(["mkdir", "-p", creds])
transport.exec(["chmod", "700", creds])
credential_paths: list[str] = []
for u in plan.upstreams:
if u.identity_file:
transport.cp_into(u.identity_file, f"{creds}/{u.name}-key")
key_path = f"{creds}/{u.name}-key"
transport.cp_into(u.identity_file, key_path)
credential_paths.append(key_path)
known_hosts = str(u.known_hosts_file)
if known_hosts and known_hosts != ".":
transport.cp_into(known_hosts, f"{creds}/{u.name}-known_hosts")
known_hosts_path = f"{creds}/{u.name}-known_hosts"
transport.cp_into(known_hosts, known_hosts_path)
credential_paths.append(known_hosts_path)
# Copy-mode behavior differs across Docker, Apple Container, and SSH.
# Apply the security contract inside the gateway so every backend produces
# the same private credential namespace.
if credential_paths:
transport.exec(["chmod", "600", *credential_paths])
# Init the bare repos + per-repo credential config for this namespace.
script = git_gate_render_provision(bottle_id, plan.upstreams)
transport.exec(["sh", "-c", script])
+2 -2
View File
@@ -30,6 +30,7 @@ from ..log import die
from ..manifest import Manifest, ManifestBottle
from ..supervisor.plan import SupervisePlan
from ..orchestrator.supervisor import Supervisor
from ..util import slugify
from . import BottleSpec
@@ -44,8 +45,7 @@ def mint_slug(spec: BottleSpec) -> str:
if spec.identity:
return spec.identity
if spec.label:
from .docker import util as docker_mod
return docker_mod.slugify(spec.label)
return slugify(spec.label)
return bottle_identity(spec.agent_name)
+1 -1
View File
@@ -86,7 +86,7 @@ def _print_vm_install_instructions() -> None:
info("Then start the service: container system start")
else:
info("Install Firecracker: https://github.com/firecracker-microvm/firecracker/releases")
info("Configure the host: ./cli.py backend setup")
info("Configure the host: bot-bottle backend setup")
def _auto_select_backend(prompt: bool = True) -> str:
+3 -3
View File
@@ -1,9 +1,9 @@
"""`backend` CLI command — generic host setup/status across backends.
`./cli.py backend setup [--backend=NAME]` provisions (or points at how
`bot-bottle backend setup [--backend=NAME]` provisions (or points at how
to provision) the chosen backend's one-time host prerequisites.
`./cli.py backend status [--backend=NAME]` reports readiness.
`./cli.py backend teardown [--backend=NAME]` undoes setup (uninstall).
`bot-bottle backend status [--backend=NAME]` reports readiness.
`bot-bottle backend teardown [--backend=NAME]` undoes setup (uninstall).
All dispatch to the backend's `setup()` / `status()` / `teardown()`
classmethods, so there are no backend-specific commands swapping
+15 -4
View File
@@ -1,7 +1,7 @@
"""cleanup: stop and remove all orphaned bot-bottle resources.
Walks every registered backend (docker, firecracker, macos-container)
so a single `./cli.py cleanup` reaps every backend's leftovers — a
so a single `bot-bottle cleanup` reaps every backend's leftovers — a
firecracker bottle's VM processes and run dirs won't survive a
docker-only cleanup pass (issue addressed alongside #77).
@@ -22,6 +22,7 @@ from __future__ import annotations
import sys
from ...backend import get_bottle_backend, has_backend, known_backend_names
from ...backend.cleanup_control import CleanupError
from ...log import info
from ...util import read_tty_line
@@ -52,10 +53,20 @@ def cmd_cleanup(_argv: list[str]) -> int:
info("cleanup: skipped")
return 0
for name, backend, plan in prepared:
if plan.empty:
# Confirmation authorizes a fresh authoritative snapshot, not blind use of
# identities that may have changed while the operator reviewed the preview.
failures: list[str] = []
for name, backend, displayed in prepared:
current = backend.prepare_cleanup()
approved = displayed.intersect(current)
if approved.empty:
continue
backend.cleanup(plan)
try:
backend.cleanup(approved)
except CleanupError as exc:
failures.append(f"{name}: {exc}")
if failures:
raise CleanupError("cleanup incomplete: " + "; ".join(failures))
info("cleanup: done")
return 0
+2 -2
View File
@@ -4,7 +4,7 @@ Docker bottles are committed to a local Docker image. Macos-container
bottles are exported and rebuilt as a local Apple Container image.
Firecracker bottles stream the guest rootfs out over SSH and rebuild a
local Docker image. The resulting reference is stored in per-bottle
state so the next `./cli.py resume <slug>` boots from the snapshot
state so the next `bot-bottle resume <slug>` boots from the snapshot
instead of rebuilding from the Dockerfile.
"""
@@ -37,7 +37,7 @@ def cmd_commit(argv: list[str]) -> int:
if slug is None:
active = enumerate_active_agents()
if not active:
die("no active bottles; start one with `./cli.py start`")
die("no active bottles; start one with `bot-bottle start`")
choices = [a.slug for a in active]
slug = tui.filter_select(choices, title="Select bottle to commit")
if slug is None:
+1 -1
View File
@@ -8,7 +8,7 @@ override and transcript snapshot under the same state dir.
Use case: an interrupted or preserved bottle needs to be relaunched;
the operator runs
./cli.py resume <identity>
bot-bottle resume <identity>
to bring up the replacement from the recorded state.
"""
+14 -15
View File
@@ -25,12 +25,11 @@ from typing import Callable
from ...agent_provider import get_provider, runtime_for
from ...backend import (
Bottle,
BottlePlan,
BottleSpec,
enumerate_active_agents,
get_bottle_backend,
)
from ...backend.docker import util as docker_mod
from ...backend.docker.bottle_plan import DockerBottlePlan
from ...bottle_state import (
cleanup_state,
is_preserved,
@@ -40,7 +39,7 @@ from ...image_cache import StaleImageError
from ...log import info, die
from ...manifest import Manifest, ManifestIndex
from ..constants import PROG
from ...util import read_tty_line
from ...util import read_tty_line, slugify
from .. import tui
@@ -204,19 +203,19 @@ def _start_headless(
if not os.isatty(stdin_fd):
die(
"--headless requires a PTY on stdin; run via:\n"
" script -q /dev/null ./cli.py start ..."
" script -q /dev/null bot-bottle start ..."
)
agent_name = args.name
if not agent_name:
die("--headless requires an agent name: ./cli.py start <agent> --headless")
die("--headless requires an agent name: bot-bottle start <agent> --headless")
manifest.require_agent(agent_name) # raises ManifestError if unknown
prompt = args.prompt
if not prompt:
die(
"--headless requires --prompt: "
"./cli.py start <agent> --headless --prompt 'Do the thing'"
"bot-bottle start <agent> --headless --prompt 'Do the thing'"
)
if args.bottle:
@@ -257,10 +256,10 @@ def _uniquify_label_headless(label: str) -> str:
logging the chosen label. Orchestrators fire-and-forget many bottles,
so silently picking a free name beats erroring on every collision."""
active_slugs = {a.slug for a in enumerate_active_agents()}
if docker_mod.slugify(label) not in active_slugs:
if slugify(label) not in active_slugs:
return label
n = 2
while docker_mod.slugify(f"{label}-{n}") in active_slugs:
while slugify(f"{label}-{n}") in active_slugs:
n += 1
chosen = f"{label}-{n}"
info(f"label '{label}' already in use; using '{chosen}'")
@@ -274,11 +273,11 @@ def prepare_with_preflight(
spec: BottleSpec,
*,
stage_dir: Path,
render_preflight: Callable[[DockerBottlePlan, str], None],
render_preflight: Callable[[BottlePlan, str], None],
prompt_yes: Callable[[], bool],
dry_run: bool = False,
backend_name: str | None = None,
) -> tuple[DockerBottlePlan | None, str]:
) -> tuple[BottlePlan | None, str]:
"""Run `backend.prepare`, render the preflight summary via the
injected callable, prompt y/N via the injected callable.
@@ -320,9 +319,9 @@ def attach_agent(
`resume=True` adds `--continue` so claude picks up its most
recent session non-interactively (no session-picker prompt).
First-attach paths (`./cli.py start`) leave it False.
First-attach paths (`bot-bottle start`) leave it False.
Used as the inner step of `./cli.py start`."""
Used as the inner step of `bot-bottle start`."""
runtime = runtime_for(agent_provider_template)
info(
f"attaching interactive {agent_provider_template} session "
@@ -355,7 +354,7 @@ def settle_state(identity: str) -> None:
if not identity:
return
if is_preserved(identity):
info(f"to resume this bottle: ./cli.py resume {identity}")
info(f"to resume this bottle: bot-bottle resume {identity}")
return
cleanup_state(identity)
@@ -405,7 +404,7 @@ def _resolve_unique_label(label: str, color: str) -> tuple[str, str]:
in use among running bottles. Passes through unchanged when no
collision is found on the first check."""
while True:
slug_candidate = docker_mod.slugify(label)
slug_candidate = slugify(label)
active_slugs = {a.slug for a in enumerate_active_agents()}
if slug_candidate not in active_slugs:
return label, color
@@ -432,7 +431,7 @@ def _select_image_policy() -> str | None:
def _text_render_preflight():
def _render(plan: DockerBottlePlan, backend_name: str) -> None:
def _render(plan: BottlePlan, backend_name: str) -> None:
print(file=sys.stderr)
print(f"backend: {backend_name}", file=sys.stderr)
print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
+23 -20
View File
@@ -110,7 +110,7 @@ def discover_pending() -> list[QueuedProposal]:
def _approval_status(qp: QueuedProposal, verb: str) -> str:
"""Status-line text after a successful approval."""
base = f"{verb} {qp.proposal.tool} for [{qp.label}]"
return f"{base}; resume: ./cli.py resume {qp.label}"
return f"{base}; resume: bot-bottle resume {qp.label}"
def _detail_lines(
@@ -368,7 +368,7 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore # pragm
elif key in (curses.KEY_UP, ord("k")):
selected = max(selected - 1, 0)
elif key in (curses.KEY_ENTER, 10, 13):
_detail_view(stdscr, qp, green_attr=green_attr)
status_line = _detail_view(stdscr, qp, green_attr=green_attr)
elif key == ord("a"):
try:
status_line = _approve_from_tui(stdscr, qp)
@@ -456,7 +456,7 @@ def _detail_view(
qp: QueuedProposal,
*,
green_attr: int = 0,
) -> None: # pragma: no cover
) -> str: # pragma: no cover
"""Render the full proposal. Scrollable. Press q to return."""
lines = _detail_lines(qp, green_attr=green_attr)
offset = 0
@@ -473,7 +473,7 @@ def _detail_view(
stdscr.refresh()
key = stdscr.getch()
if key in (ord("q"), 27):
return
return ""
if key in (curses.KEY_DOWN, ord("j")):
offset = min(offset + 1, max(0, len(lines) - 1))
elif key in (curses.KEY_UP, ord("k")):
@@ -484,31 +484,34 @@ def _detail_view(
offset = max(0, len(lines) - 1)
elif key == ord("a"):
try:
_approve_from_tui(stdscr, qp)
except ApplyError:
pass
return
return _approve_from_tui(stdscr, qp)
except ApplyError as exc:
return f"apply failed: {exc}"
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
return
return f"modify unavailable for {qp.proposal.tool}"
edited = _modify(stdscr, qp)
if edited is not None:
try:
_approve_from_tui(
stdscr, qp, final_file=edited,
notes="operator modified before approving",
)
except ApplyError:
pass
return
if edited is None:
return "modify aborted (no change)"
try:
return _approve_from_tui(
stdscr, qp, final_file=edited,
notes="operator modified before approving",
)
except ApplyError as exc:
return f"apply failed: {exc}"
elif key == ord("r"):
reason = _prompt(stdscr, "reject reason: ")
if reason:
reject(qp, reason=reason)
return
return f"rejected {qp.proposal.tool} for [{qp.label}]"
return "reject aborted (empty reason)"
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore # pragma: no cover
def _modify(
stdscr: "curses._CursesWindow", # type: ignore
qp: QueuedProposal,
) -> str | None: # pragma: no cover
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
suffix = _suffix_for_tool(qp.proposal.tool)
curses.endwin()
+32 -6
View File
@@ -16,6 +16,8 @@ import os
import sys
from typing import Any, Optional
from ..log import debug
def filter_multiselect(
items: list[str],
@@ -42,7 +44,11 @@ def filter_multiselect(
try:
tty_fd = open(tty_path, "r+b", buffering=0)
except OSError:
except OSError as exc:
debug(
"multi-select unavailable; treating it as cancellation",
context={"error_type": type(exc).__name__, "tty": tty_path},
)
return None
try:
@@ -73,7 +79,11 @@ def filter_select(
try:
tty_fd = open(tty_path, "r+b", buffering=0)
except OSError:
except OSError as exc:
debug(
"filter-select unavailable; treating it as cancellation",
context={"error_type": type(exc).__name__, "tty": tty_path},
)
return None
try:
@@ -129,7 +139,11 @@ def _run_picker(items: list[str], *, title: str, tty_fd: int) -> Optional[str]:
curses.nocbreak()
curses.echo()
curses.endwin()
except Exception: # noqa: W0718 — curses can raise many error types
except Exception as exc: # noqa: W0718 — curses can raise many error types
debug(
"filter-select display failed; treating it as cancellation",
context={"error_type": type(exc).__name__},
)
return None
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
@@ -292,7 +306,11 @@ def _run_multiselect(
curses.nocbreak()
curses.echo()
curses.endwin()
except Exception: # noqa: W0718
except Exception as exc: # noqa: W0718
debug(
"multi-select display failed; treating it as cancellation",
context={"error_type": type(exc).__name__},
)
return None
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
@@ -558,13 +576,21 @@ def name_color_modal(
"""
try:
tty_fd = open(tty_path, "r+b", buffering=0) # pylint: disable=consider-using-with
except OSError:
except OSError as exc:
debug(
"name/color picker unavailable; using defaults",
context={"error_type": type(exc).__name__, "tty": tty_path},
)
return default_label, ""
try:
fd_dup = os.dup(tty_fd.fileno())
return _run_name_color(default_label, tty_fd=fd_dup, disclaimer=disclaimer)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
except Exception as exc: # noqa: BLE001 # pylint: disable=broad-exception-caught
debug(
"name/color picker failed; using defaults",
context={"error_type": type(exc).__name__},
)
return default_label, ""
finally:
tty_fd.close()
+20 -10
View File
@@ -8,9 +8,17 @@
# Layer ordering is deliberate: the npm install lives in its own layer so
# changes to the rest of the repo (or to the CMD) don't bust it.
# Current Node LTS; slim variant keeps the image small while still
# providing apt-get for any future additions.
FROM node:22-trixie-slim
# Version-qualified Node LTS, pinned to its multi-architecture manifest.
ARG NODE_BASE_IMAGE
FROM ${NODE_BASE_IMAGE}
ARG DEBIAN_SNAPSHOT=20260724T000000Z
RUN sed -i \
-e "s|http://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|http://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
/etc/apt/sources.list.d/debian.sources
# Install runtime system deps. claude-code shells out to git for several
# features (status checks, commits, PR creation) — without git in the
@@ -20,7 +28,7 @@ FROM node:22-trixie-slim
# HTTPS_PROXY-aware tool (curl itself, plus anything that shells out
# to it) works against egress's bumped TLS without the agent needing
# local DNS.
RUN apt-get update \
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends \
git \
ca-certificates \
@@ -35,15 +43,17 @@ RUN apt-get update \
# (claude-code is a Node CLI), but is convenient for the agent to
# shell out to for ad-hoc scripts. Kept on its own layer so it can
# be moved to a downstream image if the base ever needs to shrink.
RUN apt-get update \
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
# Install claude-code globally. Pinned to the version verified in the v1
# build (`claude --version` returns 2.1.126). Bump deliberately when
# rolling forward; an unpinned install would mean rebuilds silently pick
# up new behavior.
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
# Install from the committed npm lock. `npm ci` verifies every registry
# artifact against its lockfile integrity and refuses dependency drift.
COPY bot_bottle/contrib/claude/package.json \
bot_bottle/contrib/claude/package-lock.json /opt/claude/
RUN cd /opt/claude \
&& npm ci --omit=dev --no-fund --no-audit \
&& ln -s /opt/claude/node_modules/.bin/claude /usr/local/bin/claude \
&& npm cache clean --force
# Git reads both ~/.gitconfig and ~/.config/git/config. Keep its XDG config
+140
View File
@@ -0,0 +1,140 @@
{
"name": "bot-bottle-claude-image",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "bot-bottle-claude-image",
"dependencies": {
"@anthropic-ai/claude-code": "2.1.172"
}
},
"node_modules/@anthropic-ai/claude-code": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code/-/claude-code-2.1.172.tgz",
"integrity": "sha512-SfwC+5fQcmNbvvm+1vLiZbfUxt0PQz9lbXapj9+FI+XY/2e+3zgteBM4JFEXBjULZj1DtZXHmKtAROUrMM9GZg==",
"hasInstallScript": true,
"license": "SEE LICENSE IN README.md",
"bin": {
"claude": "bin/claude.exe"
},
"engines": {
"node": ">=18.0.0"
},
"optionalDependencies": {
"@anthropic-ai/claude-code-darwin-arm64": "2.1.172",
"@anthropic-ai/claude-code-darwin-x64": "2.1.172",
"@anthropic-ai/claude-code-linux-arm64": "2.1.172",
"@anthropic-ai/claude-code-linux-arm64-musl": "2.1.172",
"@anthropic-ai/claude-code-linux-x64": "2.1.172",
"@anthropic-ai/claude-code-linux-x64-musl": "2.1.172",
"@anthropic-ai/claude-code-win32-arm64": "2.1.172",
"@anthropic-ai/claude-code-win32-x64": "2.1.172"
}
},
"node_modules/@anthropic-ai/claude-code-darwin-arm64": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-darwin-arm64/-/claude-code-darwin-arm64-2.1.172.tgz",
"integrity": "sha512-pBRgDo8PAgbt2aE4oc6ZrKdOa/Ax36RAduhLCaI8NWD3a0RDb5mETzQciQLwnuenk0bs27vIRh9Yg1jAYG/0+A==",
"cpu": [
"arm64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@anthropic-ai/claude-code-darwin-x64": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-darwin-x64/-/claude-code-darwin-x64-2.1.172.tgz",
"integrity": "sha512-vSgibgeCyvCFiLJSXu/sgcd3L/tUjSiS/tfS9rJLXUjElw8satMJhA5pqPUiBmKMflOWKMufbZgzXvLx+OhvFw==",
"cpu": [
"x64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@anthropic-ai/claude-code-linux-arm64": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-linux-arm64/-/claude-code-linux-arm64-2.1.172.tgz",
"integrity": "sha512-Ql7AbyaXnlA6NwDUaQGO7ZZis21rMjYjbKzpksMCvY35CmsJqanYgbYSn/rE83u/tZpKo+NqOv+bWEDSzsZ02w==",
"cpu": [
"arm64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@anthropic-ai/claude-code-linux-arm64-musl": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-linux-arm64-musl/-/claude-code-linux-arm64-musl-2.1.172.tgz",
"integrity": "sha512-Pa9mGGmp8QCRC2j1cgcWRRkwsK1x4bPsS3CcLRd936Op0k5et62tD3qIYhUPQOIxeuxe9Tt/y7lX1cx7JTDxyA==",
"cpu": [
"arm64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@anthropic-ai/claude-code-linux-x64": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-linux-x64/-/claude-code-linux-x64-2.1.172.tgz",
"integrity": "sha512-RYCY9EHkmtoAlwBKcWRzGIhuus+GM2CIVCfU81cTQAwDcCHunoBeFn3NqAcFV1VKb4dk9TRarAmQcWMKJrpPig==",
"cpu": [
"x64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@anthropic-ai/claude-code-linux-x64-musl": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-linux-x64-musl/-/claude-code-linux-x64-musl-2.1.172.tgz",
"integrity": "sha512-1NflAnV/MqIlD4rzlGDVsJgGK2xJ2ldVd5pkcbu7PsDJBW5dzKYVa4PmD0715K8Yiji8jQovh/nhIo6gYyTfVQ==",
"cpu": [
"x64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@anthropic-ai/claude-code-win32-arm64": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-win32-arm64/-/claude-code-win32-arm64-2.1.172.tgz",
"integrity": "sha512-Gj8mIbHDDSGWnoriqB1Jt1uH6cvNBFDQBtIYarCcv0fs+QGCEP6qm2GIaKqxAbwkUaLT0sCW/7+ukvkNdSltuQ==",
"cpu": [
"arm64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"win32"
]
},
"node_modules/@anthropic-ai/claude-code-win32-x64": {
"version": "2.1.172",
"resolved": "https://registry.npmjs.org/@anthropic-ai/claude-code-win32-x64/-/claude-code-win32-x64-2.1.172.tgz",
"integrity": "sha512-OkheFwagiCEKaO2Sb0j3JTO1NrcR3zTlFik/AN+yQPefhUIGP6vHQDbBG5ksuFe+Dyd9s+P95OUh8WDhccb+Ng==",
"cpu": [
"x64"
],
"license": "SEE LICENSE IN LICENSE.md",
"optional": true,
"os": [
"win32"
]
}
}
}
+7
View File
@@ -0,0 +1,7 @@
{
"name": "bot-bottle-claude-image",
"private": true,
"dependencies": {
"@anthropic-ai/claude-code": "2.1.172"
}
}
+40 -8
View File
@@ -3,9 +3,22 @@
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed for that user.
FROM node:22-trixie-slim
ARG NODE_BASE_IMAGE
FROM ${NODE_BASE_IMAGE}
RUN apt-get update \
# Remote-control requires the standalone package layout. Keep this exact release
# in sync with the committed upstream archive checksums.
ARG CODEX_VERSION=0.145.0
ARG DEBIAN_SNAPSHOT=20260724T000000Z
RUN sed -i \
-e "s|http://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|http://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
/etc/apt/sources.list.d/debian.sources
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends \
git \
ca-certificates \
@@ -19,7 +32,7 @@ RUN apt-get update \
# (codex is a Node CLI), but is convenient for the agent to shell
# out to for ad-hoc scripts. Kept on its own layer so it can be
# moved to a downstream image if the base ever needs to shrink.
RUN apt-get update \
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
@@ -30,10 +43,29 @@ WORKDIR /home/node
ENV PATH="/home/node/.local/bin:${PATH}"
# Remote-control support requires the standalone Codex install layout
# under ~/.codex/packages/standalone/current. The npm package can run
# the TUI, but remote-control commands expect this installer-owned path.
RUN mkdir -p /home/node/.codex \
&& curl -fsSL https://chatgpt.com/codex/install.sh | sh
# Install the exact standalone release archive selected by the target
# architecture. The checksum list is copied from the immutable upstream release
# and committed so a rebuild cannot silently accept changed release bytes.
COPY --chown=node:node bot_bottle/contrib/codex/codex-package_SHA256SUMS /tmp/codex-package_SHA256SUMS
RUN case "$(dpkg --print-architecture)" in \
amd64) codex_target=x86_64-unknown-linux-musl ;; \
arm64) codex_target=aarch64-unknown-linux-musl ;; \
*) echo "unsupported Codex architecture: $(dpkg --print-architecture)" >&2; exit 1 ;; \
esac \
&& codex_asset="codex-package-${codex_target}.tar.gz" \
&& codex_sha256="$(awk -v asset="${codex_asset}" '$2 == asset { print $1 }' /tmp/codex-package_SHA256SUMS)" \
&& test -n "${codex_sha256}" \
&& curl -fsSL \
"https://github.com/openai/codex/releases/download/rust-v${CODEX_VERSION}/${codex_asset}" \
-o "/tmp/${codex_asset}" \
&& echo "${codex_sha256} /tmp/${codex_asset}" | sha256sum -c - \
&& codex_release="/home/node/.codex/packages/standalone/releases/${CODEX_VERSION}-${codex_target}" \
&& mkdir -p "${codex_release}" /home/node/.local/bin \
&& tar -xzf "/tmp/${codex_asset}" -C "${codex_release}" \
&& ln -s bin/codex "${codex_release}/codex" \
&& ln -s "${codex_release}" /home/node/.codex/packages/standalone/current \
&& ln -s /home/node/.codex/packages/standalone/current/bin/codex /home/node/.local/bin/codex \
&& rm "/tmp/${codex_asset}" \
&& test "$(codex --version)" = "codex-cli ${CODEX_VERSION}"
CMD ["codex"]
@@ -0,0 +1,2 @@
54f79a05aba6f9abf8ef988abcae8bf2fcefba20beb549b4ff2b3acdb2cb6f54 codex-package-aarch64-unknown-linux-musl.tar.gz
71a28d362c96ac9829bf8203a2c71be451aeb726adb843167fdaf0eae8fe7dd9 codex-package-x86_64-unknown-linux-musl.tar.gz
+22 -11
View File
@@ -2,9 +2,18 @@
#
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
FROM node:22-trixie-slim
ARG NODE_BASE_IMAGE
FROM ${NODE_BASE_IMAGE}
RUN apt-get update \
ARG DEBIAN_SNAPSHOT=20260724T000000Z
RUN sed -i \
-e "s|http://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian-security|http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT}|g" \
-e "s|http://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
-e "s|https://deb.debian.org/debian|http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT}|g" \
/etc/apt/sources.list.d/debian.sources
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends \
git \
ca-certificates \
@@ -15,13 +24,10 @@ RUN apt-get update \
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update \
RUN apt-get -o Acquire::Check-Valid-Until=false update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
&& npm cache clean --force
RUN install -d -o node -g node -m 755 /home/node/.config /home/node/.config/git \
&& mkdir -p /home/node/.pi/agent \
/home/node/.pi/context-mode/sessions \
@@ -34,10 +40,15 @@ RUN install -d -o node -g node -m 755 /home/node/.config /home/node/.config/git
USER node
WORKDIR /home/node
RUN pi install npm:@harms-haus/pi-cwd \
&& pi install npm:pi-web-access \
&& pi install npm:context-mode \
&& pi install npm:pi-subagents \
&& pi install npm:pi-mcp-adapter
# Pi discovers npm packages from its agent directory. Installing the CLI and
# extensions together from the committed lock makes all direct and transitive
# package versions deterministic, with npm integrity verification.
COPY --chown=node:node bot_bottle/contrib/pi/package.json \
bot_bottle/contrib/pi/package-lock.json /home/node/.pi/agent/
RUN cd /home/node/.pi/agent \
&& npm ci --omit=dev --no-fund --no-audit \
&& npm cache clean --force
ENV PATH="/home/node/.pi/agent/node_modules/.bin:${PATH}"
CMD ["pi"]
File diff suppressed because it is too large Load Diff
+15
View File
@@ -0,0 +1,15 @@
{
"name": "bot-bottle-pi-image",
"private": true,
"dependencies": {
"@earendil-works/pi-coding-agent": "0.81.1",
"@earendil-works/pi-agent-core": "0.81.1",
"@earendil-works/pi-ai": "0.81.1",
"@earendil-works/pi-tui": "0.81.1",
"@harms-haus/pi-cwd": "1.0.0",
"context-mode": "1.0.169",
"pi-mcp-adapter": "2.11.0",
"pi-subagents": "0.35.1",
"pi-web-access": "0.13.0"
}
}
+2 -2
View File
@@ -11,7 +11,7 @@ from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from ..gateway.egress.addon_core import Route
from ..gateway.egress.types import Route
@dataclass(frozen=True)
@@ -19,7 +19,7 @@ class EgressRoute(Route):
"""Host-side extension of the addon's `Route`.
Inherits `host`, `matches`, `auth_scheme`, and `token_env`
from `egress_addon_core.Route` those are the fields that cross the
from the gateway's wire `Route` — those are the fields that cross the
YAML wire into the gateway. The fields below are host-only and
are never serialised to the addon.
+2 -2
View File
@@ -14,8 +14,8 @@ import secrets
from pathlib import Path
from typing import TYPE_CHECKING
from ..gateway.egress.addon_core import (
ON_MATCH_REDACT,
from ..gateway.egress.dlp_config import ON_MATCH_REDACT
from ..gateway.egress.types import (
HeaderMatch as CoreHeaderMatch,
MatchEntry as CoreMatchEntry,
PathMatch as CorePathMatch,
+12 -4
View File
@@ -136,10 +136,18 @@ def _pump(name: str, stream: IO[bytes]) -> None:
"""Read lines from `stream`, prefix with `[name]`, write to
stdout. Runs in its own thread per child; daemon=True so a
blocked read doesn't keep the process alive after main exits."""
for raw in iter(stream.readline, b""):
line = raw.decode("utf-8", errors="replace").rstrip("\n")
sys.stdout.write(f"[{name}] {line}\n")
sys.stdout.flush()
try:
for raw in iter(stream.readline, b""):
line = raw.decode("utf-8", errors="replace").rstrip("\n")
sys.stdout.write(f"[{name}] {line}\n")
sys.stdout.flush()
except (OSError, ValueError) as exc:
# The manager closes a dead child's pipe after wait() and before a
# restart. A pump can be between readline calls at that exact moment;
# closed-stream errors are normal completion, not uncaught thread
# failures. Preserve genuinely unexpected I/O diagnostics.
if not stream.closed:
_log(f"{name} output pump stopped: {type(exc).__name__}: {exc}")
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
+137
View File
@@ -0,0 +1,137 @@
"""Shared resource boundaries for gateway stdlib HTTP services."""
from __future__ import annotations
import http.server
import io
import socket
import threading
import time
from dataclasses import dataclass
from typing import Any, Protocol
class Readable(Protocol):
def read(self, size: int = -1, /) -> bytes: ...
class Writable(Protocol):
def write(self, data: bytes, /) -> object: ...
@dataclass(frozen=True)
class BodyReadError(Exception):
status: int
message: str
def read_declared_body(
stream: Readable,
connection: socket.socket,
raw_length: str | None,
*,
maximum: int,
timeout_seconds: float,
require_length: bool,
) -> bytes:
"""Validate and read exactly one declared body under a read deadline."""
output = io.BytesIO()
copy_declared_body(
stream, output, connection, raw_length, maximum=maximum,
timeout_seconds=timeout_seconds, require_length=require_length,
)
return output.getvalue()
def copy_declared_body(
stream: Readable,
output: Writable,
connection: socket.socket,
raw_length: str | None,
*,
maximum: int,
timeout_seconds: float,
require_length: bool,
) -> int:
"""Copy one declared body to a sink without retaining it in memory."""
if raw_length is None:
if require_length:
raise BodyReadError(411, "Content-Length required")
raw_length = "0"
try:
length = int(raw_length)
except ValueError as exc:
raise BodyReadError(400, "invalid Content-Length") from exc
if length < 0:
raise BodyReadError(400, "invalid Content-Length")
if length > maximum:
raise BodyReadError(413, "request body too large")
previous_timeout = connection.gettimeout()
deadline = time.monotonic() + timeout_seconds
remaining = length
try:
while remaining:
timeout = deadline - time.monotonic()
if timeout <= 0:
raise BodyReadError(408, "request body read timed out")
connection.settimeout(timeout)
chunk = stream.read(min(remaining, 64 * 1024))
if not chunk:
raise BodyReadError(400, "incomplete request body")
output.write(chunk)
remaining -= len(chunk)
except TimeoutError as exc:
raise BodyReadError(408, "request body read timed out") from exc
finally:
connection.settimeout(previous_timeout)
return length
class BoundedThreadingHTTPServer(http.server.ThreadingHTTPServer):
"""ThreadingHTTPServer with a hard cap on in-flight request threads."""
daemon_threads = True
def __init__( # pylint: disable=consider-using-with
self, *args, max_workers: int = 32, **kwargs, # type: ignore[no-untyped-def]
):
if max_workers < 1:
raise ValueError("max_workers must be positive")
self._request_slots = threading.BoundedSemaphore(max_workers)
super().__init__(*args, **kwargs)
def process_request(
self, request: Any, client_address: Any,
) -> None:
if not self._request_slots.acquire( # pylint: disable=consider-using-with
blocking=False,
):
try:
request.sendall(
b"HTTP/1.1 503 Service Unavailable\r\n"
b"Content-Length: 0\r\nConnection: close\r\n\r\n"
)
finally:
self.shutdown_request(request)
return
try:
super().process_request(request, client_address)
except BaseException:
self._request_slots.release()
raise
def process_request_thread(
self, request: Any, client_address: Any,
) -> None:
try:
super().process_request_thread(request, client_address)
finally:
self._request_slots.release()
__all__ = [
"BodyReadError",
"BoundedThreadingHTTPServer",
"copy_declared_body",
"read_declared_body",
]
+79 -116
View File
@@ -16,29 +16,35 @@ import typing
from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
from bot_bottle.constants import IDENTITY_HEADER
from bot_bottle.gateway.egress.dlp_detectors import redact_tokens, strip_crlf
from bot_bottle.gateway.egress.addon_core import (
LOG_BLOCKS,
LOG_FULL,
from bot_bottle.gateway.egress.dlp_detectors import redact_tokens
from bot_bottle.gateway.egress.dlp_config import (
DEFAULT_OUTBOUND_ON_MATCH,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
)
from bot_bottle.gateway.egress.context import resolve_client_context
from bot_bottle.gateway.egress.dlp import (
build_inbound_scan_text,
build_token_allow_payload,
scan_inbound,
scan_outbound,
)
from bot_bottle.gateway.egress.outbound_pipeline import redact_request, scan_request
from bot_bottle.gateway.egress.matching import (
decide,
match_route,
)
from bot_bottle.gateway.egress.request_pipeline import (
evaluate_route_policy,
git_block_reason,
)
from bot_bottle.gateway.egress.schema import route_to_yaml_dict
from bot_bottle.gateway.egress.types import (
LOG_BLOCKS,
LOG_FULL,
Config,
Route,
ScanResult,
build_inbound_scan_text,
build_outbound_scan_text,
build_token_allow_payload,
decide,
decide_git_fetch,
is_git_fetch_request,
is_git_push_request,
match_route,
resolve_client_context,
outbound_scan_headers,
route_to_yaml_dict,
scan_inbound,
scan_outbound,
)
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
from bot_bottle.supervisor.types import (
@@ -383,19 +389,9 @@ class EgressAddon:
self._passthrough_conns.discard(conn_id)
async def request(self, flow: http.HTTPFlow) -> None:
config, slug, env = self._request_context(flow)
request_path, _, query = flow.request.path.partition("?")
# Reuse the context stashed by http_connect for HTTPS flows (one
# orchestrator round-trip per connection). Plain-HTTP flows have no
# prior CONNECT stash, so resolve now and stash for response/websocket.
meta = getattr(flow, "metadata", None)
if isinstance(meta, dict) and _FLOW_CTX_KEY in meta:
config, slug, env = meta[_FLOW_CTX_KEY]
self._request_token(flow) # strip identity headers; token already resolved
else:
config, slug, env = self._resolve_flow(flow)
self._stash_flow_ctx(flow, config, slug, env)
# Introspection ("_egress.local/allowlist") reports the calling bottle's
# own resolved routes — served after resolution so it reflects this
# bottle's policy, not a stale global.
@@ -416,56 +412,66 @@ class EgressAddon:
# the path/query the git checks below rely on.
request_path, _, query = flow.request.path.partition("?")
if is_git_push_request(request_path, query):
self._block(
flow,
"egress: git push over HTTPS is not supported; "
"use the bottle.git SSH path (gitleaks-scanned by "
"git-gate's pre-receive hook).",
ctx=self._req_ctx(flow),
)
if not self._allow_git_request(flow, config, request_path, query):
return
if is_git_fetch_request(request_path, query):
git_decision = decide_git_fetch(
config.routes, flow.request.pretty_host,
)
if git_decision.action == "block":
self._block(
flow,
git_decision.reason,
ctx=self._req_ctx(flow),
)
return
self._apply_route_policy(flow, config, route, request_path, env)
def _request_context(
self, flow: http.HTTPFlow,
) -> tuple[Config, str, "typing.Mapping[str, str]"]:
"""Resolve one bottle context, reusing the HTTPS CONNECT snapshot."""
meta = getattr(flow, "metadata", None)
if isinstance(meta, dict) and _FLOW_CTX_KEY in meta:
config, slug, env = meta[_FLOW_CTX_KEY]
self._request_token(flow)
return config, slug, env
config, slug, env = self._resolve_flow(flow)
self._stash_flow_ctx(flow, config, slug, env)
return config, slug, env
def _allow_git_request(
self, flow: http.HTTPFlow, config: Config,
request_path: str, query: str,
) -> bool:
"""Apply the HTTPS Git push/fetch boundary before general routing."""
reason = git_block_reason(
config.routes, flow.request.pretty_host, request_path, query,
)
if not reason:
return True
self._block(flow, reason, ctx=self._req_ctx(flow))
return False
def _apply_route_policy(
self, flow: http.HTTPFlow, config: Config, route: Route | None,
request_path: str, env: "typing.Mapping[str, str]",
) -> None:
"""Strip agent auth, evaluate the route, then inject gateway auth."""
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject gateway-owned auth below.
# Routes with preserve_auth=True pass the header through as-is so the
# agent's own credentials (e.g. registry bearer tokens) reach the upstream.
if route is None or not route.preserve_auth:
result = evaluate_route_policy(
config,
route,
host=flow.request.pretty_host,
request_path=request_path,
method=flow.request.method,
headers=dict(flow.request.headers),
env=env,
)
if result.strip_authorization:
flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
decision = decide(
config.routes,
flow.request.pretty_host,
request_path,
env,
request_method=flow.request.method,
request_headers=req_headers,
deny_reason=config.deny_reason,
)
if decision.action == "block":
self._block(flow, decision.reason, ctx=self._req_ctx(flow))
if result.block_reason:
self._block(flow, result.block_reason, ctx=self._req_ctx(flow))
return
if decision.inject_authorization is not None:
flow.request.headers["authorization"] = decision.inject_authorization
if result.inject_authorization is not None:
flow.request.headers["authorization"] = result.inject_authorization
if config.log >= LOG_FULL:
if result.log_request:
self._log_request(flow, env)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
@@ -489,20 +495,12 @@ class EgressAddon:
Loops so the supervise policy can re-scan after each approval a
second, un-approved token in the same request is still caught."""
while True:
request_path, _, query = flow.request.path.partition("?")
body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, body,
)
# CRLF is scanned only over the request line + headers, never the
# body (see scan_outbound) — a body is not an injection vector.
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(
route, scan_text, env,
safe_tokens=self._safe_tokens_for(slug), crlf_text=crlf_text,
request_path, _, _ = flow.request.path.partition("?")
result = scan_request(
flow.request,
route,
env,
safe_tokens=self._safe_tokens_for(slug),
)
if result is None or result.severity != "block":
return True
@@ -512,7 +510,7 @@ class EgressAddon:
# redact scrubs every detection (tokens and structural CRLF) and
# forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route, env):
if redact_request(flow.request, route, env):
if self._flow_log(flow) >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_redacted",
@@ -545,41 +543,6 @@ class EgressAddon:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound(
self, flow: http.HTTPFlow, route: Route, env: "typing.Mapping[str, str]",
) -> bool:
"""Scrub detected tokens (and CRLF injection sequences) from the mutable
request surfaces (body, headers, path/query) and re-scan. `env` is the
per-bottle env overlay. Returns True if the request is now clean; False
if a block-severity match remains on a surface redaction cannot rewrite
(the hostname) so the caller fails closed."""
body = flow.request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=env)
if redacted_body != body:
flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()):
if name.lower() == "host":
continue # routing-critical; never a legitimate token
redacted = strip_crlf(redact_tokens(value, env=env))
if redacted != value:
flow.request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=env))
if redacted_path != flow.request.path:
flow.request.path = redacted_path
request_path, _, query = flow.request.path.partition("?")
new_body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, new_body,
)
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(route, scan_text, env, crlf_text=crlf_text)
return result is None or result.severity != "block"
async def _supervise_token_block(
self,
flow: http.HTTPFlow,
File diff suppressed because it is too large Load Diff
+77
View File
@@ -0,0 +1,77 @@
"""Fail-closed resolution of a client's policy and egress credentials."""
from __future__ import annotations
import typing
from ...log import debug
from .types import Config
DENY_UNATTRIBUTED = (
"egress: this request was not attributed to any bottle, so no egress policy "
"applies and every host is denied. Either the bottle's registry row is "
"missing/ambiguous (torn down, or another bottle claimed its source IP), or "
"the request carried no matching identity token — check that the caller's "
"proxy URL includes it. This is not an allowlist problem."
)
DENY_UNPARSEABLE = (
"egress: this bottle's egress policy could not be parsed, so it is being "
"treated as deny-all. Fix the bottle's egress.routes; every host is denied "
"until it loads."
)
DENY_RESOLVER_ERROR = (
"egress: the orchestrator could not be reached to resolve this bottle's "
"egress policy, so every host is denied (fail-closed). Check that the "
"control plane is up; this is not an allowlist problem."
)
class PolicyResolverLike(typing.Protocol):
def resolve(self, source_ip: str, identity_token: str = ...) -> str | None: ...
class ContextResolverLike(typing.Protocol):
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = ...,
) -> tuple[str | None, str | None, dict[str, str]]: ...
def _config_from_policy(policy: str | None) -> Config:
# Local import keeps schema parsing independent of resolver protocols.
from .schema import load_config
if not policy:
return Config(routes=(), deny_reason=DENY_UNATTRIBUTED)
try:
return load_config(policy)
except ValueError:
return Config(routes=(), deny_reason=DENY_UNPARSEABLE)
def resolve_client_config(
resolver: PolicyResolverLike, client_ip: str, identity_token: str = "",
) -> Config:
try:
policy = resolver.resolve(client_ip, identity_token)
except Exception as exc: # noqa: BLE001 - a policy lookup failure must deny
debug(
"egress policy resolution failed; applying deny-all",
context={"error_type": type(exc).__name__},
)
return Config(routes=(), deny_reason=DENY_RESOLVER_ERROR)
return _config_from_policy(policy)
def resolve_client_context(
resolver: ContextResolverLike, client_ip: str, identity_token: str = "",
) -> tuple[Config, str, dict[str, str]]:
try:
policy, bottle_id, tokens = resolver.resolve_policy_and_bottle_id(
client_ip, identity_token)
except Exception as exc: # noqa: BLE001 - a policy lookup failure must deny
debug(
"egress context resolution failed; applying deny-all",
context={"error_type": type(exc).__name__},
)
return Config(routes=(), deny_reason=DENY_RESOLVER_ERROR), "", {}
return _config_from_policy(policy), (bottle_id or ""), tokens
+99
View File
@@ -0,0 +1,99 @@
"""DLP scan dispatch and safe proposal rendering for egress requests."""
from __future__ import annotations
import typing
from .types import Route, ScanResult
def build_outbound_scan_text(host: str, path: str, query: str,
headers: typing.Mapping[str, str], body: str) -> str:
parts = [host, path]
if query:
parts.append(query)
parts.extend(f"{name}: {value}" for name, value in headers.items())
if body:
parts.append(body)
return "\n".join(parts)
def outbound_scan_headers(route: Route, headers: typing.Mapping[str, str]) -> dict[str, str]:
"""Drop agent Authorization when the route injects gateway-owned auth."""
skip_auth = bool(route.auth_scheme and route.token_env)
return {name: value for name, value in headers.items()
if not (skip_auth and name.lower() == "authorization")}
def build_inbound_scan_text(headers: typing.Mapping[str, str], body: str) -> str:
parts = [f"{name}: {value}" for name, value in headers.items()]
if body:
parts.append(body)
return "\n".join(parts)
def _enabled(configured: tuple[str, ...] | None, name: str) -> bool:
return configured is None or name in configured
def scan_outbound(route: Route, body: str | bytes, environ: typing.Mapping[str, str], *,
safe_tokens: typing.AbstractSet[str] | None = None,
crlf_text: str | None = None) -> ScanResult | None:
if not route.inspect:
return None
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection, scan_entropy, scan_known_secrets, scan_token_patterns)
except ImportError: # pragma: no cover - gateway's flat module path
from .dlp_detectors import (
scan_crlf_injection, scan_entropy, scan_known_secrets, scan_token_patterns)
if isinstance(body, bytes):
try:
text = body.decode("utf-8")
except UnicodeDecodeError:
text = body.decode("latin-1")
else:
text = body
result = scan_crlf_injection(text if crlf_text is None else crlf_text)
if result is not None:
return result
if _enabled(route.outbound_detectors, "token_patterns"):
result = scan_token_patterns(text, location="body", safe_tokens=safe_tokens)
if result is not None:
return result
if _enabled(route.outbound_detectors, "known_secrets"):
extra = tuple(prefix for prefix in environ.get(
"BOT_BOTTLE_SENSITIVE_PREFIXES", "").split(",") if prefix)
result = scan_known_secrets(text, location="body", env=environ,
sensitive_prefixes=("EGRESS_TOKEN_",) + extra,
safe_tokens=safe_tokens)
if result is not None:
return result
if route.outbound_detectors is not None and "entropy" in route.outbound_detectors:
return scan_entropy(text, location="body")
return None
def build_token_allow_payload(host: str, method: str, path: str, result: ScanResult) -> str:
"""Render redacted operator context; the raw matched secret is excluded."""
lines = [
"egress blocked an outbound request carrying a detected token",
f"host: {host}", f"method: {method}", f"path: {path}",
f"detector: {result.reason}",
]
if result.context:
lines.append(f"context: {result.context}")
return "\n".join(lines) + "\n"
def scan_inbound(route: Route, body: str | bytes) -> ScanResult | None:
if not route.inspect:
return None
try:
from dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
except ImportError: # pragma: no cover - gateway's flat module path
from .dlp_detectors import scan_naive_injection
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
if _enabled(route.inbound_detectors, "naive_injection_detection"):
return scan_naive_injection(text)
return None
+1 -1
View File
@@ -19,7 +19,7 @@ from math import log2
from collections import Counter
from urllib.parse import quote as url_quote
from .addon_core import ScanResult
from .types import ScanResult
# ---------------------------------------------------------------------------
+112
View File
@@ -0,0 +1,112 @@
"""Route matching and request-policy decisions for the egress gateway."""
from __future__ import annotations
import typing
from .types import Decision, MatchEntry, PathMatch, Route
def _path_matches(pm: PathMatch, request_path: str) -> bool:
if pm.type == "exact":
return request_path == pm.value
if pm.type == "prefix":
if request_path == pm.value:
return True
if not pm.value.endswith("/"):
return request_path.startswith(pm.value + "/")
return request_path.startswith(pm.value)
return (
pm.type == "regex"
and pm.compiled is not None
and pm.compiled.search(request_path) is not None
)
def _entry_matches(
entry: MatchEntry, request_path: str, request_method: str,
request_headers: typing.Mapping[str, str],
) -> bool:
if entry.paths and not any(_path_matches(pm, request_path) for pm in entry.paths):
return False
if entry.methods and request_method.upper() not in entry.methods:
return False
for match in entry.headers:
value = request_headers.get(match.name.lower())
if value is None:
return False
if match.type == "exact" and value != match.value:
return False
if match.type == "regex" and (
match.compiled is None or match.compiled.search(value) is None
):
return False
return True
def evaluate_matches(
route: Route, request_path: str, request_method: str = "GET",
request_headers: typing.Mapping[str, str] | None = None,
) -> bool:
"""Return whether a request satisfies a route's optional match entries."""
if not route.matches:
return True
return any(_entry_matches(entry, request_path, request_method, request_headers or {})
for entry in route.matches)
def is_git_push_request(path: str, query: str) -> bool:
return path.endswith("/git-receive-pack") or (
path.endswith("/info/refs") and any(
pair.partition("=") == ("service", "=", "git-receive-pack")
for pair in query.split("&")
)
)
def is_git_fetch_request(path: str, query: str) -> bool:
return path.endswith("/git-upload-pack") or (
path.endswith("/info/refs") and any(
pair.partition("=") == ("service", "=", "git-upload-pack")
for pair in query.split("&")
)
)
def match_route(routes: typing.Sequence[Route], request_host: str) -> Route | None:
target = request_host.lower()
return next((route for route in routes if route.host.lower() == target), None)
def decide(
routes: typing.Sequence[Route], request_host: str, request_path: str,
environ: typing.Mapping[str, str], *, request_method: str = "GET",
request_headers: typing.Mapping[str, str] | None = None, deny_reason: str = "",
) -> Decision:
route = match_route(routes, request_host)
if route is None:
return Decision("block", deny_reason or (
f"egress: host {request_host!r} is not in the bottle's egress.routes "
"allowlist. Declare a route for it or remove the request."))
if not evaluate_matches(route, request_path, request_method, request_headers):
return Decision("block", (
f"egress: request {request_method} {request_path!r} does not match any "
f"entry in matches for {route.host!r}"))
if route.auth_scheme and route.token_env:
token = environ.get(route.token_env, "")
if not token:
return Decision("block", (
f"egress: route for {route.host!r} declared auth but env var "
f"{route.token_env!r} is unset"))
return Decision("forward", inject_authorization=f"{route.auth_scheme} {token}")
return Decision("forward")
def decide_git_fetch(routes: typing.Sequence[Route], request_host: str) -> Decision:
route = match_route(routes, request_host)
if route is not None and route.git_fetch:
return Decision("forward")
return Decision("block", (
"egress: git fetch/clone over HTTPS is not allowed by default; use git-gate "
"for declared repos or set egress.routes[].git.fetch=true for explicit "
"read-only HTTPS Git access."))
@@ -0,0 +1,83 @@
"""Outbound DLP request scanning and redaction for the egress pipeline."""
from __future__ import annotations
from typing import ItemsView, Mapping, Protocol
from .dlp import (
build_outbound_scan_text,
outbound_scan_headers,
scan_outbound,
)
from .dlp_detectors import redact_tokens, strip_crlf
from .types import Route, ScanResult
class MutableHeaders(Protocol):
def items(self) -> ItemsView[str, str]: ...
def __getitem__(self, name: str, /) -> str: ...
def __setitem__(self, name: str, value: str, /) -> None: ...
class MutableRequest(Protocol):
pretty_host: str
path: str
headers: MutableHeaders
text: str
def get_text(self, strict: bool = False) -> str | None: ...
def scan_request(
request: MutableRequest,
route: Route,
env: Mapping[str, str],
*,
safe_tokens: set[str] | None = None,
) -> ScanResult | None:
"""Scan all mutable outbound request surfaces in their canonical order."""
request_path, _, query = request.path.partition("?")
headers = outbound_scan_headers(route, dict(request.headers.items()))
body = request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
request.pretty_host, request_path, query, headers, body,
)
# Bodies cannot alter HTTP framing, so CRLF detection is deliberately
# restricted to the request line and headers.
crlf_text = build_outbound_scan_text(
request.pretty_host, request_path, query, headers, "",
)
return scan_outbound(
route,
scan_text,
env,
safe_tokens=safe_tokens,
crlf_text=crlf_text,
)
def redact_request(
request: MutableRequest,
route: Route,
env: Mapping[str, str],
) -> bool:
"""Redact mutable request surfaces and return whether the result is clean."""
body = request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=env)
if redacted_body != body:
request.text = redacted_body
for name, value in list(request.headers.items()):
if name.lower() == "host":
continue
redacted = strip_crlf(redact_tokens(value, env=env))
if redacted != value:
request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(request.path, env=env))
if redacted_path != request.path:
request.path = redacted_path
result = scan_request(request, route, env)
return result is None or result.severity != "block"
__all__ = ["MutableRequest", "redact_request", "scan_request"]
@@ -0,0 +1,92 @@
"""Framework-neutral request policy stages for the egress adapter.
The mitmproxy addon owns flow mutation and response construction. This module
owns the ordered Git and route-policy decisions so those rules remain directly
testable without a live proxy flow.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Mapping, Sequence
from .matching import (
decide,
decide_git_fetch,
is_git_fetch_request,
is_git_push_request,
)
from .types import LOG_FULL, Config, Route
GIT_PUSH_BLOCK_REASON = (
"egress: git push over HTTPS is not supported; "
"use the bottle.git SSH path (gitleaks-scanned by "
"git-gate's pre-receive hook)."
)
@dataclass(frozen=True)
class RoutePolicyResult:
"""The flow mutations and outcome produced by general route policy."""
block_reason: str = ""
strip_authorization: bool = False
inject_authorization: str | None = None
log_request: bool = False
def git_block_reason(
routes: Sequence[Route],
host: str,
request_path: str,
query: str,
) -> str:
"""Return the HTTPS Git policy denial, or ``""`` when allowed."""
if is_git_push_request(request_path, query):
return GIT_PUSH_BLOCK_REASON
if not is_git_fetch_request(request_path, query):
return ""
decision = decide_git_fetch(routes, host)
return decision.reason if decision.action == "block" else ""
def evaluate_route_policy(
config: Config,
route: Route | None,
*,
host: str,
request_path: str,
method: str,
headers: Mapping[str, str],
env: Mapping[str, str],
) -> RoutePolicyResult:
"""Evaluate authorization stripping, matching, injection, and logging."""
strip_authorization = route is None or not route.preserve_auth
effective_headers = {
name.lower(): value
for name, value in headers.items()
if not (strip_authorization and name.lower() == "authorization")
}
decision = decide(
config.routes,
host,
request_path,
env,
request_method=method,
request_headers=effective_headers,
deny_reason=config.deny_reason,
)
return RoutePolicyResult(
block_reason=decision.reason if decision.action == "block" else "",
strip_authorization=strip_authorization,
inject_authorization=decision.inject_authorization,
log_request=config.log >= LOG_FULL,
)
__all__ = [
"GIT_PUSH_BLOCK_REASON",
"RoutePolicyResult",
"evaluate_route_policy",
"git_block_reason",
]
+349
View File
@@ -0,0 +1,349 @@
"""Egress policy schema parsing and serialization (PRD 0017 / 0053)."""
from __future__ import annotations
import re
import typing
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
from .dlp_config import parse_inspect_block
from .types import (
HEADER_MATCH_TYPES,
LOG_BLOCKS,
LOG_FULL,
LOG_OFF,
PATH_MATCH_TYPES,
VALID_METHODS,
Config,
HeaderMatch,
MatchEntry,
PathMatch,
Route,
)
# Parsing
# ---------------------------------------------------------------------------
def _parse_path_match(idx: int, j: int, raw: object) -> PathMatch:
label = f"route[{idx}] matches paths[{j}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
ptype = raw_dict.get("type", "prefix")
if not isinstance(ptype, str) or ptype not in PATH_MATCH_TYPES:
raise ValueError(
f"{label}: 'type' must be one of {', '.join(PATH_MATCH_TYPES)} "
f"(got {ptype!r})"
)
value = raw_dict.get("value")
if not isinstance(value, str) or not value:
raise ValueError(f"{label}: 'value' must be a non-empty string")
if ptype in ("exact", "prefix") and not value.startswith("/"):
raise ValueError(
f"{label}: value {value!r} must start with '/' for "
f"type {ptype!r}"
)
compiled: re.Pattern[str] | None = None
if ptype == "regex":
try:
compiled = re.compile(value)
except re.error as e:
raise ValueError(
f"{label}: regex {value!r} failed to compile: {e}"
) from e
for k in raw_dict:
if k not in ("type", "value"):
raise ValueError(f"{label}: unknown key {k!r}")
return PathMatch(type=ptype, value=value, compiled=compiled)
def _parse_header_match(idx: int, j: int, raw: object) -> HeaderMatch:
label = f"route[{idx}] matches headers[{j}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
name = raw_dict.get("name")
if not isinstance(name, str) or not name:
raise ValueError(f"{label}: 'name' must be a non-empty string")
value = raw_dict.get("value")
if not isinstance(value, str):
raise ValueError(f"{label}: 'value' must be a string")
htype = raw_dict.get("type", "exact")
if not isinstance(htype, str) or htype not in HEADER_MATCH_TYPES:
raise ValueError(
f"{label}: 'type' must be one of {', '.join(HEADER_MATCH_TYPES)} "
f"(got {htype!r})"
)
compiled: re.Pattern[str] | None = None
if htype == "regex":
try:
compiled = re.compile(value)
except re.error as e:
raise ValueError(
f"{label}: regex {value!r} failed to compile: {e}"
) from e
for k in raw_dict:
if k not in ("name", "value", "type"):
raise ValueError(f"{label}: unknown key {k!r}")
return HeaderMatch(name=name, value=value, type=htype, compiled=compiled)
def _parse_match_entry(idx: int, k: int, raw: object) -> MatchEntry:
label = f"route[{idx}] matches[{k}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
paths: tuple[PathMatch, ...] = ()
paths_raw = raw_dict.get("paths")
if paths_raw is not None:
if not isinstance(paths_raw, list):
raise ValueError(f"{label}: 'paths' must be a list")
paths_list = typing.cast(list[object], paths_raw)
paths = tuple(_parse_path_match(idx, j, p) for j, p in enumerate(paths_list))
methods: tuple[str, ...] = ()
methods_raw = raw_dict.get("methods")
if methods_raw is not None:
if not isinstance(methods_raw, list):
raise ValueError(f"{label}: 'methods' must be a list")
methods_list = typing.cast(list[object], methods_raw)
normalised: list[str] = []
for j, m in enumerate(methods_list):
if not isinstance(m, str):
raise ValueError(f"{label}: methods[{j}] must be a string")
upper = m.upper()
if upper not in VALID_METHODS:
raise ValueError(
f"{label}: methods[{j}] {m!r} is not a valid HTTP method"
)
normalised.append(upper)
methods = tuple(normalised)
headers: tuple[HeaderMatch, ...] = ()
headers_raw = raw_dict.get("headers")
if headers_raw is not None:
if not isinstance(headers_raw, list):
raise ValueError(f"{label}: 'headers' must be a list")
headers_list = typing.cast(list[object], headers_raw)
headers = tuple(
_parse_header_match(idx, j, h) for j, h in enumerate(headers_list)
)
for key in raw_dict:
if key not in ("paths", "methods", "headers"):
raise ValueError(f"{label}: unknown key {key!r}")
return MatchEntry(paths=paths, methods=methods, headers=headers)
def parse_routes(payload: object) -> tuple[Route, ...]:
if not isinstance(payload, dict):
raise ValueError("routes payload: top-level must be an object")
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
raw: object = payload_dict.get("routes")
if not isinstance(raw, list):
raise ValueError("routes payload: 'routes' must be a list")
raw_list: list[object] = typing.cast(list[object], raw)
out: list[Route] = []
for i, r in enumerate(raw_list):
out.append(_parse_one(i, r))
return tuple(out)
def _parse_one(idx: int, raw: object) -> Route:
label = f"route[{idx}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object (got {type(raw).__name__})")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
host: object = raw_dict.get("host")
if not isinstance(host, str) or not host:
raise ValueError(f"{label}: 'host' must be a non-empty string")
legacy_flat = "inspect" not in raw_dict
inspect_raw = raw_dict.get("inspect", {})
if inspect_raw is False:
inspect = False
settings: dict[str, object] = {}
elif isinstance(inspect_raw, dict):
inspect = True
settings = (
{k: v for k, v in raw_dict.items() if k != "host"}
if legacy_flat
else typing.cast(dict[str, object], inspect_raw)
)
legacy_dlp = settings.pop("dlp", None)
if isinstance(legacy_dlp, dict):
settings.update(typing.cast(dict[str, object], legacy_dlp))
elif legacy_dlp is not None:
raise ValueError(
f"{label} ({host}): legacy 'dlp' must be an object"
)
else:
raise ValueError(f"{label} ({host}): 'inspect' must be false or an object")
# matches
matches: tuple[MatchEntry, ...] = ()
matches_raw = settings.get("matches")
if matches_raw is not None:
if not isinstance(matches_raw, list):
raise ValueError(f"{label} ({host}): 'matches' must be a list")
matches_list = typing.cast(list[object], matches_raw)
matches = tuple(
_parse_match_entry(idx, k, m) for k, m in enumerate(matches_list)
)
# auth (unchanged wire format)
auth_scheme: object = settings.get("auth_scheme", "")
token_env: object = settings.get("token_env", "")
if not isinstance(auth_scheme, str):
raise ValueError(f"{label} ({host}): 'auth_scheme' must be a string")
if not isinstance(token_env, str):
raise ValueError(f"{label} ({host}): 'token_env' must be a string")
if bool(auth_scheme) != bool(token_env):
raise ValueError(
f"{label} ({host}): 'auth_scheme' and 'token_env' must be both "
f"set or both empty (got auth_scheme={auth_scheme!r}, "
f"token_env={token_env!r})"
)
# git-over-HTTPS policy
git_fetch = False
git_raw = settings.get("git")
if git_raw is not None:
if not isinstance(git_raw, dict):
raise ValueError(f"{label} ({host}): 'git' must be an object")
git_dict: dict[str, object] = typing.cast(dict[str, object], git_raw)
fetch_raw = git_dict.get("fetch", False)
if fetch_raw is True or fetch_raw is False:
git_fetch = fetch_raw
else:
raise ValueError(f"{label} ({host}): 'git.fetch' must be a boolean")
for k in git_dict:
if k != "fetch":
raise ValueError(
f"{label} ({host}): git has unknown key {k!r}; "
"accepted key is 'fetch'"
)
# dlp detectors
outbound_detectors, inbound_detectors, outbound_on_match = parse_inspect_block(
idx, host, settings,
)
preserve_auth_raw = settings.get("preserve_auth", False)
if preserve_auth_raw is not True and preserve_auth_raw is not False:
raise ValueError(
f"{label} ({host}): 'preserve_auth' must be a boolean"
)
preserve_auth: bool = preserve_auth_raw
for k in settings:
if k not in (
"matches", "auth_scheme", "token_env", "git", "preserve_auth",
"outbound_detectors", "inbound_detectors", "outbound_on_match",
):
raise ValueError(
f"{label} ({host}): inspect has unknown key {k!r}"
)
for k in raw_dict:
if not legacy_flat and k not in ("host", "inspect"):
raise ValueError(
f"{label} ({host}): unknown key {k!r}; accepted keys "
f"are 'host' and 'inspect'"
)
return Route(
host=host,
matches=matches,
auth_scheme=auth_scheme,
token_env=token_env,
git_fetch=git_fetch,
outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match,
preserve_auth=preserve_auth,
inspect=inspect,
)
def _path_match_to_dict(pm: PathMatch) -> dict[str, object]:
d: dict[str, object] = {"value": pm.value}
if pm.type != "prefix":
d["type"] = pm.type
return d
def _header_match_to_dict(hm: HeaderMatch) -> dict[str, object]:
d: dict[str, object] = {"name": hm.name, "value": hm.value}
if hm.type != "exact":
d["type"] = hm.type
return d
def _match_entry_to_dict(me: MatchEntry) -> dict[str, object]:
d: dict[str, object] = {}
if me.paths:
d["paths"] = [_path_match_to_dict(p) for p in me.paths]
if me.methods:
d["methods"] = list(me.methods)
if me.headers:
d["headers"] = [_header_match_to_dict(h) for h in me.headers]
return d
def route_to_yaml_dict(r: Route) -> dict[str, object]:
"""Serialize a Route to YAML-schema-compatible dict.
Uses the same field names the YAML parser accepts, so the output
can be round-tripped directly into an `allow` or `egress-block`
proposal without translation. Fields that are empty/default are
omitted so the agent doesn't copy irrelevant keys."""
d: dict[str, object] = {"host": r.host}
if not r.inspect:
d["inspect"] = False
return d
inspected: dict[str, object] = {}
if r.auth_scheme:
inspected["auth_scheme"] = r.auth_scheme
inspected["token_env"] = r.token_env
if r.matches:
inspected["matches"] = [_match_entry_to_dict(m) for m in r.matches]
if r.git_fetch:
inspected["git"] = {"fetch": True}
if r.outbound_detectors is not None:
inspected["outbound_detectors"] = list(r.outbound_detectors)
if r.inbound_detectors is not None:
inspected["inbound_detectors"] = list(r.inbound_detectors)
if r.outbound_on_match:
inspected["outbound_on_match"] = r.outbound_on_match
if r.preserve_auth:
inspected["preserve_auth"] = True
if inspected:
d["inspect"] = inspected
return d
def parse_config(payload: object) -> "Config":
"""Parse a full egress config payload (top-level log level + routes)."""
if not isinstance(payload, dict):
raise ValueError("routes payload: top-level must be an object")
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
log_raw: object = payload_dict.get("log", LOG_OFF)
if log_raw is True or log_raw is False or not isinstance(log_raw, int) \
or log_raw not in (LOG_OFF, LOG_BLOCKS, LOG_FULL):
raise ValueError(
f"routes payload: 'log' must be {LOG_OFF}, {LOG_BLOCKS}, or {LOG_FULL}"
)
routes = parse_routes(payload)
return Config(routes=routes, log=log_raw)
def load_config(text: str) -> "Config":
"""Parse YAML text → Config (routes + log flag)."""
try:
payload = parse_yaml_subset(text)
except YamlSubsetError as e:
raise ValueError(f"routes payload: invalid YAML: {e}") from e
return parse_config(payload)
+81
View File
@@ -0,0 +1,81 @@
"""Shared egress policy value objects.
Kept dependency-free so the schema parser, matcher, DLP scanner, and addon
adapter can use the same immutable public shapes without importing each other.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
PATH_MATCH_TYPES = ("exact", "prefix", "regex")
HEADER_MATCH_TYPES = ("exact", "regex")
VALID_METHODS = frozenset({
"GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "TRACE",
"CONNECT",
})
LOG_OFF = 0
LOG_BLOCKS = 1
LOG_FULL = 2
@dataclass(frozen=True)
class PathMatch:
type: str
value: str
compiled: re.Pattern[str] | None = None
@dataclass(frozen=True)
class HeaderMatch:
name: str
value: str
type: str = "exact"
compiled: re.Pattern[str] | None = None
@dataclass(frozen=True)
class MatchEntry:
paths: tuple[PathMatch, ...] = ()
methods: tuple[str, ...] = ()
headers: tuple[HeaderMatch, ...] = ()
@dataclass(frozen=True)
class Route:
host: str
matches: tuple[MatchEntry, ...] = ()
auth_scheme: str = ""
token_env: str = ""
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
outbound_on_match: str = ""
preserve_auth: bool = False
inspect: bool = True
@dataclass(frozen=True)
class Config:
routes: tuple[Route, ...]
log: int = LOG_OFF
deny_reason: str = ""
@dataclass(frozen=True)
class Decision:
action: str
reason: str = ""
inject_authorization: str | None = None
@dataclass(frozen=True)
class ScanResult:
severity: str
reason: str
location: str = ""
context: str = ""
matched: str = ""
+48 -22
View File
@@ -21,12 +21,19 @@ from __future__ import annotations
import os
import subprocess
import sys
import tempfile
import threading
import typing
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from http.server import BaseHTTPRequestHandler
from pathlib import Path
from urllib.parse import urlsplit
from bot_bottle.constants import GIT_GATE_TIMEOUT_SECS, IDENTITY_HEADER
from bot_bottle.gateway.bounded_http import (
BodyReadError,
BoundedThreadingHTTPServer,
copy_declared_body,
)
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
@@ -77,6 +84,10 @@ def resolve_sandbox_root(
# Bound memory use while still allowing ordinary git push packfiles.
MAX_BODY_BYTES = 100 * 1024 * 1024
REQUEST_BODY_TIMEOUT_SECONDS = 30.0
MAX_REQUEST_WORKERS = 16
MAX_BODY_WORKERS = 2
_BODY_WORK_SLOTS = threading.BoundedSemaphore(MAX_BODY_WORKERS)
class GitHttpHandler(BaseHTTPRequestHandler):
@@ -184,27 +195,40 @@ class GitHttpHandler(BaseHTTPRequestHandler):
value = self.headers.get(header)
if value:
env[variable] = value
raw_length = self.headers.get("content-length", "0") or "0"
if not _BODY_WORK_SLOTS.acquire(blocking=False):
self.send_error(503, "git request capacity exhausted")
return
try:
length = int(raw_length)
except ValueError:
self.send_error(400, "Bad Content-Length")
return
if length < 0:
self.send_error(400, "Negative Content-Length")
return
if length > MAX_BODY_BYTES:
self.send_error(413, "Request body too large")
return
body = self.rfile.read(length) if length else b""
proc = subprocess.run(
["git", "http-backend"],
input=body,
env=env,
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
with tempfile.TemporaryFile() as body:
try:
copy_declared_body(
self.rfile,
body,
self.connection,
self.headers.get("content-length"),
maximum=MAX_BODY_BYTES,
timeout_seconds=REQUEST_BODY_TIMEOUT_SECONDS,
require_length=False,
)
except BodyReadError as exc:
self.send_error(exc.status, exc.message)
return
body.seek(0)
try:
proc = subprocess.run(
["git", "http-backend"],
stdin=body,
env=env,
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
except (OSError, subprocess.SubprocessError) as exc:
self.log_message("git http-backend unavailable: %s", exc)
self.send_error(503, "git backend unavailable")
return
finally:
_BODY_WORK_SLOTS.release()
self._write_cgi_response(proc.stdout)
def _repo_dir(self, sandbox_root: Path, path: str) -> Path | None:
@@ -273,7 +297,9 @@ def main() -> int:
"(no single-tenant flat-root fallback)\n"
)
return 1
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
server = BoundedThreadingHTTPServer(
("0.0.0.0", port), GitHttpHandler, max_workers=MAX_REQUEST_WORKERS,
)
# Resolve each request's sandbox namespace by source IP against the
# orchestrator control plane.
server.policy_resolver = PolicyResolver(orch_url) # type: ignore[attr-defined]
+22 -1
View File
@@ -252,6 +252,27 @@ cat > "$refs_file"
zero=0000000000000000000000000000000000000000
# Phase 0: reject Gitea AGit review refs before scanning or forwarding.
# A push to refs/for/*, refs/draft/*, or refs/for-review/* asks Gitea to
# open a pull request backed by a server-managed refs/pull/<n>/head rather
# than an ordinary refs/heads/* branch. That breaks the git-gate workflow:
# follow-up commits can't be pushed back through the branch, and Gitea
# rejects later direct updates to the generated review ref. Fail the whole
# push here (before any gitleaks scan or upstream forward) so the caller
# pushes a real branch and opens the PR against it instead. Deletions
# (new == zero) stay allowed so stale AGit refs can still be cleaned up.
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
[ "$new" = "$zero" ] && continue
case "$ref" in
refs/for/*|refs/draft/*|refs/for-review/*)
echo "git-gate: refusing AGit review ref $ref" >&2
echo "git-gate: push to refs/heads/<branch> and open a branch-backed pull request instead" >&2
exit 1
;;
esac
done < "$refs_file"
supervise_gitleaks_allow() {
log_opts=$1
ref=$2
@@ -364,7 +385,7 @@ PY
;;
esac
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
echo "git-gate: approve with 'bot-bottle supervise' to continue this push" >&2
waited=0
while [ "$waited" -lt "$timeout" ]; do
status=$(PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$proposal_id" <<'PY'
@@ -0,0 +1,92 @@
"""Framework-neutral MCP method and tool dispatch."""
from __future__ import annotations
import json
from dataclasses import dataclass
from typing import Callable, Protocol
from bot_bottle.gateway.egress.schema import load_config, route_to_yaml_dict
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
from bot_bottle.supervisor import types as _sv
class Request(Protocol):
@property
def method(self) -> str: ...
@property
def params(self) -> dict[str, object]: ...
class MethodNotFoundError(Exception):
"""Raised when a JSON-RPC method has no MCP handler."""
class RouteResolutionError(Exception):
"""The caller's live route table could not be resolved authoritatively."""
Handler = Callable[[dict[str, object]], object]
@dataclass(frozen=True)
class Handlers:
initialize: Handler
tools_list: Handler
list_routes: Handler
check_proposal: Handler
propose: Handler
def dispatch(request: Request, handlers: Handlers) -> object:
"""Route one parsed request without depending on the HTTP server."""
if request.method == "initialize":
return handlers.initialize(request.params)
if request.method == "notifications/initialized":
return None
if request.method == "tools/list":
return handlers.tools_list(request.params)
if request.method != "tools/call":
raise MethodNotFoundError(request.method)
tool = request.params.get("name")
if tool == _sv.TOOL_LIST_EGRESS_ROUTES:
return handlers.list_routes(request.params)
if tool == _sv.TOOL_CHECK_PROPOSAL:
return handlers.check_proposal(request.params)
return handlers.propose(request.params)
def resolved_routes_payload(
resolver: PolicyResolver,
source_ip: str,
identity_token: str,
) -> dict[str, object]:
"""Render an authoritatively resolved route table for the calling bottle."""
try:
policy, bottle_id, _tokens = resolver.resolve_policy_and_bottle_id(
source_ip, identity_token,
)
except PolicyResolveError as exc:
raise RouteResolutionError("orchestrator unavailable") from exc
if not bottle_id:
raise RouteResolutionError("request source is not attributed to a bottle")
try:
config = load_config(policy or "")
except ValueError as exc:
raise RouteResolutionError("resolved policy is invalid") from exc
body = json.dumps(
{"routes": [route_to_yaml_dict(route) for route in config.routes]},
indent=2,
)
return {"content": [{"type": "text", "text": body}], "isError": False}
__all__ = [
"Handlers",
"MethodNotFoundError",
"RouteResolutionError",
"dispatch",
"resolved_routes_payload",
]
+69 -63
View File
@@ -51,17 +51,27 @@ from __future__ import annotations
import http.server
import json
import os
import socketserver
import sys
import time
import typing
from dataclasses import dataclass
from bot_bottle.constants import IDENTITY_HEADER
from bot_bottle.gateway.egress.addon_core import (
LOG_OFF, load_config, resolve_client_context, route_to_yaml_dict,
from bot_bottle.gateway.bounded_http import (
BodyReadError,
BoundedThreadingHTTPServer,
read_declared_body,
)
from bot_bottle.gateway.egress.schema import load_config
from bot_bottle.gateway.egress.types import LOG_OFF
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
from bot_bottle.gateway.supervisor.mcp_dispatch import (
Handlers as DispatchHandlers,
MethodNotFoundError,
RouteResolutionError,
dispatch,
resolved_routes_payload,
)
from bot_bottle.supervisor import types as _sv
@@ -565,6 +575,8 @@ def format_unknown_proposal_text(proposal_id: str) -> str:
# Max request body the server accepts. 1 MB is well above any realistic
# routes.yaml proposal.
MAX_BODY_BYTES = 1 * 1024 * 1024
REQUEST_BODY_TIMEOUT_SECONDS = 10.0
MAX_REQUEST_WORKERS = 32
class MCPHandler(http.server.BaseHTTPRequestHandler):
@@ -587,19 +599,18 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
self._write_text(405, "use POST for MCP requests\n")
def do_POST(self) -> None:
length_header = self.headers.get("Content-Length")
if length_header is None:
self._write_text(411, "Content-Length required\n")
return
try:
length = int(length_header)
except ValueError:
self._write_text(400, "invalid Content-Length\n")
body = read_declared_body(
self.rfile,
self.connection,
self.headers.get("Content-Length"),
maximum=MAX_BODY_BYTES,
timeout_seconds=REQUEST_BODY_TIMEOUT_SECONDS,
require_length=True,
)
except BodyReadError as exc:
self._write_text(exc.status, exc.message + "\n")
return
if length < 0 or length > MAX_BODY_BYTES:
self._write_text(413, "request body too large\n")
return
body = self.rfile.read(length)
try:
req = parse_jsonrpc(body)
@@ -611,6 +622,11 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
result = self._dispatch(req, config)
except MethodNotFoundError as e:
self._write_jsonrpc(
jsonrpc_error(req.id, ERR_METHOD_NOT_FOUND, f"method not found: {e}"),
)
return
except _RpcClientError as e:
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
return
@@ -633,41 +649,42 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
self._write_jsonrpc(jsonrpc_result(req.id, result))
def _dispatch(self, req: JsonRpcRequest, config: ServerConfig) -> object:
method = req.method
if method == "initialize":
return handle_initialize(req.params)
if method == "notifications/initialized":
return None # ack-only
if method == "tools/list":
return handle_tools_list(req.params)
if method == "tools/call":
# `list-egress-routes` is read-only introspection. The shared gateway
# has no static route table (routes are resolved per request by
# source IP), so answer it from the calling bottle's resolved policy.
# Otherwise the agent sees an empty allowlist and composes an egress
# proposal that *replaces* the live routes instead of extending them
# — silently dropping base routes like api.anthropic.com on approval.
if req.params.get("name") == _sv.TOOL_LIST_EGRESS_ROUTES:
return self._resolved_routes_payload()
resolver = self._resolver_or_fail()
source_ip = self.client_address[0]
token = self._identity_token()
# `check-proposal` is a non-blocking read of the calling bottle's
# own queue — attributed by (source_ip, identity_token) like a
# proposal, but it never queues or blocks.
if req.params.get("name") == _sv.TOOL_CHECK_PROPOSAL:
return handle_check_proposal(
req.params, resolver=resolver,
source_ip=source_ip, identity_token=token,
)
# The control plane attributes the proposal to the source-IP + token
# resolved bottle, so the one shared queue holds each bottle's
# proposal under its own id — no slug is asserted by this daemon.
return handle_tools_call(
req.params, config, resolver=resolver,
source_ip=source_ip, identity_token=token,
def check(params: dict[str, object]) -> object:
return handle_check_proposal(
params,
resolver=self._resolver_or_fail(),
source_ip=self.client_address[0],
identity_token=self._identity_token(),
)
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def propose(params: dict[str, object]) -> object:
return handle_tools_call(
params,
config,
resolver=self._resolver_or_fail(),
source_ip=self.client_address[0],
identity_token=self._identity_token(),
)
def list_routes(_params: dict[str, object]) -> object:
try:
return resolved_routes_payload(
self._resolver_or_fail(),
self.client_address[0],
self._identity_token(),
)
except RouteResolutionError as exc:
raise _RpcInternalError(
f"could not resolve live egress routes: {exc}"
) from exc
return dispatch(req, DispatchHandlers(
initialize=handle_initialize,
tools_list=handle_tools_list,
list_routes=list_routes,
check_proposal=check,
propose=propose,
))
def _identity_token(self) -> str:
"""The agent's per-bottle identity token from the request header (the
@@ -686,20 +703,6 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
raise _RpcInternalError("supervise server has no policy resolver")
return resolver
def _resolved_routes_payload(self) -> dict[str, object]:
"""The calling bottle's live egress routes as the `list-egress-routes`
JSON payload, resolved by (source_ip, identity token). Fail-closed: an
unattributed source or an unreachable orchestrator yields an empty route
list (never another bottle's), courtesy of `resolve_client_context`."""
resolver = self._resolver_or_fail()
conf, _slug, _tokens = resolve_client_context(
resolver, self.client_address[0], self._identity_token(),
)
body = json.dumps(
{"routes": [route_to_yaml_dict(r) for r in conf.routes]}, indent=2,
)
return {"content": [{"type": "text", "text": body}], "isError": False}
def _write_jsonrpc(self, body: bytes) -> None:
self.send_response(200)
self.send_header("Content-Type", "application/json")
@@ -719,7 +722,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
self.wfile.write(encoded)
class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
class MCPServer(BoundedThreadingHTTPServer):
allow_reuse_address = True
daemon_threads = True
config: ServerConfig = ServerConfig()
@@ -728,6 +731,9 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
# closed per request (see `_resolver_or_fail`).
policy_resolver: "PolicyResolver | None" = None
def __init__(self, *args, **kwargs): # type: ignore[no-untyped-def]
super().__init__(*args, max_workers=MAX_REQUEST_WORKERS, **kwargs)
# --- Entry point -----------------------------------------------------------
+27 -1
View File
@@ -8,6 +8,7 @@ imported (`deploy_key_provisioner`) to keep its cost off the host path.
from __future__ import annotations
import json
import os
import dataclasses
from pathlib import Path
@@ -138,7 +139,32 @@ def provision_git_gate_dynamic_keys(
if upstream.name not in updated_names:
updated.append(upstream)
return dataclasses.replace(plan, upstreams=tuple(updated))
final_plan = dataclasses.replace(plan, upstreams=tuple(updated))
_write_upstreams_snapshot(stage_dir, final_plan.upstreams)
return final_plan
def _write_upstreams_snapshot(
stage_dir: Path, upstreams: tuple[GitGateUpstream, ...]
) -> None:
"""Persist the fully-resolved upstream table to `upstreams.json` in
`stage_dir` so the bring-up reconcile can re-provision git-gate without
the manifest. Written after dynamic keys are provisioned so every
identity_file is set."""
data = [
{
"name": u.name,
"upstream_url": u.upstream_url,
"upstream_host": u.upstream_host,
"upstream_port": u.upstream_port,
"identity_file": u.identity_file,
"known_host_key": u.known_host_key,
}
for u in upstreams
]
snapshot = stage_dir / "upstreams.json"
snapshot.write_text(json.dumps(data, indent=2))
snapshot.chmod(0o600)
__all__ = [
+3 -3
View File
@@ -44,7 +44,7 @@ if TYPE_CHECKING:
from ..gateway import Gateway, GatewayError
from .lifecycle import Orchestrator
from .service import OrchestratorCore
from .server import OrchestratorServer, dispatch, make_server
from .server import OrchestratorServer, create_app, make_server
# Facade name -> submodule that defines it. Lazy so importing a leaf (or the
@@ -67,8 +67,8 @@ _LAZY: dict[str, str] = {
"GatewayError": "..gateway",
"Orchestrator": ".lifecycle",
"OrchestratorCore": ".service",
"create_app": ".server",
"OrchestratorServer": ".server",
"dispatch": ".server",
"make_server": ".server",
}
@@ -100,7 +100,7 @@ __all__ = [
"GatewayError",
"Orchestrator",
"OrchestratorCore",
"create_app",
"OrchestratorServer",
"dispatch",
"make_server",
]
+21 -38
View File
@@ -1,6 +1,7 @@
"""Run the orchestrator control plane as a plain process (PRD 0070 dev-harness).
python -m bot_bottle.orchestrator [--host H] [--port P] [--db PATH]
BOT_BOTTLE_ORCHESTRATOR_TOKEN=<signing-key> \
python -m bot_bottle.orchestrator [--host H] [--port P] [--db PATH]
The PRD sequences the orchestrator as a plain-process dev-harness first, so
the consolidation core (registry + attribution + HTTP control plane + live
@@ -16,14 +17,13 @@ import secrets
from pathlib import Path
from .. import log
from .store.store_manager import StoreManager
from .broker import StubBroker, SubmitBroker
from .broker_client import BrokerClient
from .host_server import BROKER_SECRET_ENV, DEFAULT_PORT, broker_secret_from_env
from .server import make_server
from ..trust_domain import CONTROL_PLANE
from .broker import LaunchBroker, StubBroker
from .docker_broker import DockerBroker
from .store.registry_store import RegistryStore, default_db_path
from .server import make_server
from .service import OrchestratorCore
from .store.store_manager import StoreManager
from .store.registry_store import RegistryStore, default_db_path
def main(argv: list[str] | None = None) -> int:
@@ -36,15 +36,15 @@ def main(argv: list[str] | None = None) -> int:
help=f"registry DB path (default: {default_db_path()})",
)
parser.add_argument(
"--broker", choices=("stub", "docker", "http"), default="stub",
help="launch broker: 'stub' records requests; 'docker' runs containers "
"in-process; 'http' relays signed requests to a host control server",
)
parser.add_argument(
"--host-controller-url", default=f"http://127.0.0.1:{DEFAULT_PORT}",
help="host control server URL (used only with --broker http)",
"--broker", choices=("stub", "docker"), default="stub",
help="launch broker: 'stub' records requests; 'docker' runs containers",
)
args = parser.parse_args(argv)
if not CONTROL_PLANE.key_from_env():
log.die(
f"{CONTROL_PLANE.key_env} is required; refusing to start the "
"orchestrator without caller authentication"
)
registry = RegistryStore(args.db)
registry.migrate()
@@ -54,39 +54,22 @@ def main(argv: list[str] | None = None) -> int:
# operator reaches it over HTTP (never a second, disconnected DB).
StoreManager(registry.db_path).migrate()
# A signing secret ties the orchestrator (signer) to its broker (verifier).
# 'stub' records launches instead of starting anything; 'docker' runs real
# containers in-process; 'http' relays signed requests to a separate host
# control server, which verifies and launches. For 'stub'/'docker' the
# secret is ephemeral (signer and verifier share this process); for 'http'
# it must be the SAME secret the host controller holds, so it is read from
# the shared env var (the chunk-1 stand-in for out-of-band provisioning).
broker: SubmitBroker
if args.broker == "http":
secret = broker_secret_from_env()
if secret is None:
parser.error(
f"--broker http requires a shared signing secret in "
f"${BROKER_SECRET_ENV} (hex), matching the host control server"
)
broker = BrokerClient(args.host_controller_url)
else:
secret = secrets.token_bytes(32)
broker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
# An ephemeral signing secret ties the orchestrator (signer) to its
# broker (verifier). 'stub' records launches instead of starting
# anything; 'docker' runs real containers (firecracker drops in later).
secret = secrets.token_bytes(32)
broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
orchestrator = OrchestratorCore(registry, broker, secret)
server = make_server(orchestrator, host=args.host, port=args.port)
bound_host, bound_port = server.server_address[0], server.server_address[1]
log.info(
"orchestrator control plane listening",
context={"host": bound_host, "port": bound_port, "db": str(registry.db_path)},
context={"host": args.host, "port": args.port, "db": str(registry.db_path)},
)
try:
server.serve_forever()
server.run()
except KeyboardInterrupt:
log.info("orchestrator shutting down")
finally:
server.server_close()
return 0
+374
View File
@@ -0,0 +1,374 @@
"""FastAPI control-plane routes for the orchestrator."""
# pyright: reportUnusedFunction=false
from __future__ import annotations
import asyncio
import math
import sys
from fastapi import FastAPI, HTTPException
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from pydantic import BaseModel, ConfigDict, StrictStr
from starlette.types import ASGIApp, Message, Receive, Scope, Send
from ..orchestrator_auth import ROLE_CLI, ROLES
from ..supervisor.types import TOOLS
from ..trust_domain import CONTROL_PLANE
from .http_contract import (
MAX_BODY_BYTES,
ORCHESTRATOR_AUTH_HEADER,
REQUEST_BODY_TIMEOUT_SECONDS,
)
from .service import OrchestratorCore
_GATEWAY_ROUTES = frozenset({
("POST", "/resolve"),
("POST", "/supervise/propose"),
("POST", "/supervise/poll"),
})
class _StrictModel(BaseModel):
model_config = ConfigDict(extra="ignore", strict=True)
class LaunchBody(_StrictModel):
source_ip: StrictStr
image_ref: StrictStr = ""
metadata: StrictStr = ""
policy: StrictStr = ""
tokens: dict[StrictStr, StrictStr] = {}
env_var_secret: StrictStr = ""
class PolicyBody(_StrictModel):
policy: StrictStr
class ReprovisionBody(_StrictModel):
env_var_secret: StrictStr
class SecretBody(_StrictModel):
name: StrictStr
value: StrictStr
env_var_secret: StrictStr
class ReconcileBody(_StrictModel):
live_source_ips: list[StrictStr]
grace_seconds: float | None = None
class IdentityBody(_StrictModel):
source_ip: StrictStr
identity_token: StrictStr = ""
class AttributeBody(_StrictModel):
source_ip: StrictStr
identity_token: StrictStr
class RespondBody(_StrictModel):
proposal_id: StrictStr
bottle_slug: StrictStr
decision: StrictStr
notes: StrictStr = ""
final_file: StrictStr | None = None
class ProposeBody(IdentityBody):
tool: StrictStr
proposed_file: StrictStr
justification: StrictStr
class PollBody(IdentityBody):
proposal_id: StrictStr
class ControlPlaneBoundary:
"""Reject unauthenticated and oversized requests before reading a body."""
def __init__(self, app: ASGIApp, signing_key: str) -> None:
self.app = app
self.signing_key = signing_key
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
method = scope["method"]
route = scope["path"].rstrip("/") or "/"
if not (method == "GET" and route == "/health"):
headers = dict(scope["headers"])
presented = headers.get(
ORCHESTRATOR_AUTH_HEADER.encode(), b"",
).decode(errors="ignore")
role = CONTROL_PLANE.verify(presented, self.signing_key)
if role is None:
await self._reject(
scope, send, 401, "control-plane authentication required",
)
return
allowed = ROLES if (method, route) in _GATEWAY_ROUTES else {ROLE_CLI}
if role not in allowed:
await self._reject(scope, send, 403, "insufficient role for this route")
return
scope.setdefault("state", {})["role"] = role
raw_length = dict(scope["headers"]).get(b"content-length")
if raw_length is not None:
try:
length = int(raw_length)
except ValueError:
await self._reject(scope, send, 400, "invalid Content-Length")
return
if length < 0:
await self._reject(scope, send, 400, "invalid Content-Length")
return
if length > MAX_BODY_BYTES:
await self._reject(scope, send, 413, "request body too large")
return
try:
body = await self._read_body(receive)
except _BodyTooLarge:
await self._reject(scope, send, 413, "request body too large")
return
except TimeoutError:
await self._reject(scope, send, 408, "request body read timed out")
return
try:
await self.app(scope, self._replay_body(body), send)
except Exception as exc: # noqa: BLE001 - redact control-plane failures
sys.stderr.write(
f"orchestrator: {method} {route} failed "
f"[error_type={type(exc).__name__}]\n"
)
sys.stderr.flush()
await self._reject(scope, send, 500, "internal error")
@staticmethod
async def _reject(
scope: Scope, send: Send, status: int, error: str,
) -> None:
response = JSONResponse({"error": error}, status_code=status)
await response(scope, ControlPlaneBoundary._empty_receive, send)
@staticmethod
async def _empty_receive() -> Message:
return {"type": "http.disconnect"}
@staticmethod
async def _read_body(receive: Receive) -> bytes:
body = bytearray()
async with asyncio.timeout(REQUEST_BODY_TIMEOUT_SECONDS):
while True:
message = await receive()
if message["type"] != "http.request":
break
body.extend(message.get("body", b""))
if len(body) > MAX_BODY_BYTES:
raise _BodyTooLarge
if not message.get("more_body", False):
break
return bytes(body)
@staticmethod
def _replay_body(body: bytes) -> Receive:
sent = False
async def replay() -> Message:
nonlocal sent
if sent:
return {"type": "http.disconnect"}
sent = True
return {"type": "http.request", "body": body, "more_body": False}
return replay
class _BodyTooLarge(Exception):
"""The streamed request exceeded the control-plane body limit."""
def _required(value: str, name: str) -> str:
if not value:
raise HTTPException(400, f"{name} (string) is required")
return value
def create_app(orch: OrchestratorCore, *, signing_key: str) -> FastAPI:
"""Build the authenticated orchestrator ASGI application."""
key = signing_key.strip()
if not key:
raise ValueError(
"orchestrator control-plane signing key is required; "
"refusing to start without caller authentication"
)
app = FastAPI(
title="bot-bottle orchestrator",
docs_url=None,
redoc_url=None,
openapi_url=None,
)
app.add_middleware(ControlPlaneBoundary, signing_key=key)
@app.exception_handler(RequestValidationError)
async def invalid_request(
_request: object, exc: RequestValidationError,
) -> JSONResponse:
errors = exc.errors()
location = errors[0].get("loc", ()) if errors else ()
field = str(location[1]) if len(location) > 1 else ""
suffix = f": {field}" if field else ""
return JSONResponse(
{"error": f"invalid request body{suffix}"},
status_code=400,
)
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok"}
@app.get("/gateway")
def gateway() -> dict[str, object]:
return orch.gateway_status()
@app.get("/bottles")
def bottles() -> dict[str, object]:
return {"bottles": [record.redacted() for record in orch.registry.all()]}
@app.post("/bottles", status_code=201)
def launch(body: LaunchBody) -> dict[str, str]:
rec = orch.launch_bottle(
_required(body.source_ip, "source_ip"),
image_ref=body.image_ref,
metadata=body.metadata,
policy=body.policy,
tokens=dict(body.tokens),
env_var_secret=body.env_var_secret,
)
return {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
@app.put("/bottles/{bottle_id}/policy")
def set_policy(bottle_id: str, body: PolicyBody) -> dict[str, object]:
if orch.set_policy(bottle_id, body.policy):
return {"updated": True}
raise HTTPException(404, "no such bottle")
@app.post("/bottles/{bottle_id}/reprovision_gateway")
def reprovision(bottle_id: str, body: ReprovisionBody) -> dict[str, object]:
secret = _required(body.env_var_secret, "env_var_secret")
if orch.reprovision_from_secret(bottle_id, secret):
return {"reprovisioned": True}
raise HTTPException(404, "no stored secrets for this bottle")
@app.post("/bottles/{bottle_id}/secret")
def update_secret(bottle_id: str, body: SecretBody) -> dict[str, object]:
# Update ONE egress token for a running bottle in place — the
# single-secret form of reprovision, for refreshing a short-lived host
# credential (e.g. the Codex access token) without a relaunch. cli-only
# (not in _GATEWAY_ROUTES): a bottle must never set its own tokens.
if orch.update_agent_secret(
bottle_id,
_required(body.name, "name"),
_required(body.value, "value"),
_required(body.env_var_secret, "env_var_secret"),
):
return {"updated": True}
raise HTTPException(404, "no such bottle")
@app.delete("/bottles/{bottle_id}")
def teardown(bottle_id: str) -> dict[str, object]:
if orch.teardown_bottle(bottle_id):
return {"torn_down": True}
raise HTTPException(404, "no such bottle")
@app.post("/reconcile")
def reconcile(body: ReconcileBody) -> dict[str, object]:
if any(not ip for ip in body.live_source_ips):
raise HTTPException(400, "live_source_ips must contain non-empty strings")
kwargs: dict[str, float] = {}
if body.grace_seconds is not None:
if not math.isfinite(body.grace_seconds) or body.grace_seconds < 0:
raise HTTPException(
400, "grace_seconds must be a non-negative finite number",
)
kwargs["grace_seconds"] = body.grace_seconds
return {"reaped": orch.reconcile(body.live_source_ips, **kwargs)}
@app.post("/attribute")
def attribute(body: AttributeBody) -> dict[str, str]:
rec = orch.attribute(body.source_ip, body.identity_token)
if rec is None:
raise HTTPException(403, "unattributed")
return {"bottle_id": rec.bottle_id}
@app.get("/supervise/proposals")
def proposals() -> dict[str, object]:
return {"proposals": orch.supervise_pending()}
@app.post("/supervise/respond")
def respond(body: RespondBody) -> dict[str, object]:
ok, error = orch.supervise_respond(
_required(body.proposal_id, "proposal_id"),
bottle_slug=_required(body.bottle_slug, "bottle_slug"),
decision=_required(body.decision, "decision"),
notes=body.notes,
final_file=body.final_file,
)
if not ok:
raise HTTPException(409, error)
return {"responded": True}
@app.post("/supervise/propose", status_code=201)
def propose(body: ProposeBody) -> dict[str, str]:
source_ip = _required(body.source_ip, "source_ip")
if body.tool not in TOOLS:
raise HTTPException(400, f"tool (string) must be one of {TOOLS}")
rec = orch.resolve(source_ip, body.identity_token)
if rec is None:
raise HTTPException(403, "unattributed")
proposal_id = orch.supervise_queue_proposal(
rec.bottle_id,
tool=body.tool,
proposed_file=_required(body.proposed_file, "proposed_file"),
justification=_required(body.justification, "justification"),
)
return {"proposal_id": proposal_id}
@app.post("/supervise/poll")
def poll(body: PollBody) -> dict[str, object]:
rec = orch.resolve(
_required(body.source_ip, "source_ip"), body.identity_token,
)
if rec is None:
raise HTTPException(403, "unattributed")
return orch.supervise_poll_response(
rec.bottle_id, _required(body.proposal_id, "proposal_id"),
)
@app.post("/resolve")
def resolve(body: IdentityBody) -> dict[str, object]:
rec = orch.resolve(
_required(body.source_ip, "source_ip"), body.identity_token,
)
if rec is None:
raise HTTPException(403, "unattributed")
return {
"bottle_id": rec.bottle_id,
"policy": rec.policy,
"tokens": orch.tokens_for(rec.bottle_id),
}
return app
__all__ = [
"ControlPlaneBoundary",
"MAX_BODY_BYTES",
"ORCHESTRATOR_AUTH_HEADER",
"create_app",
]
+1 -28
View File
@@ -29,7 +29,6 @@ import json
import secrets
import time
from dataclasses import dataclass
from typing import Protocol
_JWT_HEADER = {"alg": "HS256", "typ": "JWT"}
_ALLOWED_OPS = ("launch", "teardown")
@@ -38,21 +37,7 @@ _ALLOWED_OPS = ("launch", "teardown")
class BrokerAuthError(Exception):
"""A broker request failed provenance or schema verification —
bad/absent signature, malformed token, or a payload that doesn't match
the fixed launch-request shape. Fail-closed: the broker must not act.
A **definite** negative: nothing was launched, so a caller may safely roll
back as if the op never happened."""
class BrokerUnavailableError(Exception):
"""A brokered request could not be carried to a verdict: the broker (or the
wire to it) was unreachable, timed out, or dropped the response.
Crucially **ambiguous** unlike `BrokerAuthError`, the op MAY already have
taken effect on the backend before the response was lost, so a caller must
NOT assume it did nothing (e.g. must not roll a registry row back as if no
launch happened, which would orphan a running container). Only the in-process
brokers never raise this; the out-of-process `BrokerClient` does."""
the fixed launch-request shape. Fail-closed: the broker must not act."""
@dataclass(frozen=True)
@@ -138,16 +123,6 @@ def verify_request(token: str, secret: bytes) -> LaunchRequest:
# --- the broker itself ------------------------------------------------------
class SubmitBroker(Protocol):
"""The single method `OrchestratorCore` depends on: verify a signed token and
perform its op, returning the verified request. Both the in-process
`LaunchBroker` and the out-of-process `BrokerClient` (which relays the token
to the host control server) satisfy it structurally, so the core is unchanged
whether the backend is local or a real host service."""
def submit(self, token: str) -> LaunchRequest: ...
class LaunchBroker(abc.ABC):
"""Verifies a signed request came from the orchestrator, then performs
the backend-native launch/teardown. Subclasses implement `_launch` /
@@ -193,9 +168,7 @@ class StubBroker(LaunchBroker):
__all__ = [
"BrokerAuthError",
"BrokerUnavailableError",
"LaunchRequest",
"SubmitBroker",
"LaunchBroker",
"StubBroker",
"sign_request",
-126
View File
@@ -1,126 +0,0 @@
"""Orchestrator-side broker transport (issue #468, chunk 1).
The signer's half of the launch-broker transport gap. `BrokerClient` satisfies
the exact `submit(token)` contract `OrchestratorCore` already depends on (see
`broker.SubmitBroker`), but instead of verifying and launching in-process it POSTs
the signed token to the host control server over HTTP (stdlib `urllib`, like
`orchestrator/client.py`). Because it is drop-in for that interface, wiring a real
out-of-process backend does not change the core: it still signs a request and
calls `submit()`; only the wire is new.
A provenance/schema rejection from the host controller (HTTP 401) is re-raised as
the same `BrokerAuthError` the in-process broker raises, so the launch path's
rollback-on-failure (`OrchestratorCore.launch_bottle`) behaves identically whether
the broker is local or remote.
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from .broker import BrokerAuthError, BrokerUnavailableError, LaunchRequest
DEFAULT_TIMEOUT_SECONDS = 5.0
class BrokerClientError(RuntimeError):
"""The host control server *responded*, but with an unexpected status other
than the fail-closed 401 (which surfaces as `BrokerAuthError`) e.g. a 502
backend failure or a malformed body. A definite negative: the host processed
the request and it did not launch. (A *no-response* failure unreachable /
timeout / dropped is the ambiguous `BrokerUnavailableError` instead.)"""
class BrokerClient:
"""Drop-in `submit(token)` that relays a signed request to the host control
server. Holds no secret provenance rides entirely in the signed token, so a
caller that can reach this client still cannot forge a launch."""
def __init__(self, base_url: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS) -> None:
self._base = base_url.rstrip("/")
self._timeout = timeout
def submit(self, token: str) -> LaunchRequest:
"""POST the signed token to the host controller and return the request it
verified and acted on.
Raises `BrokerAuthError` on a fail-closed 401 (bad provenance/schema
the same exception the in-process broker raises); `BrokerClientError` if
the host *responds* with any other non-success status or a malformed
body (a definite negative); or `BrokerUnavailableError` if no response is
obtained (unreachable / timeout / dropped) the **ambiguous** case, where
the host may already have acted, so the caller must not roll back."""
data = json.dumps({"token": token}).encode()
req = urllib.request.Request(
f"{self._base}/broker", data=data, method="POST",
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=self._timeout) as resp:
return _request_from(_json_object(resp.read()))
except urllib.error.HTTPError as e:
detail = _error_detail(e)
if e.code == 401:
raise BrokerAuthError(
detail or "host controller rejected the request"
) from e
raise BrokerClientError(
f"POST /broker: HTTP {e.code} {detail}".rstrip()
) from e
except (urllib.error.URLError, TimeoutError, OSError) as e:
# No usable response — unreachable, timed out, or the connection
# dropped mid-exchange. Ambiguous: the request may already have
# launched the bottle, so this is NOT a definite failure.
raise BrokerUnavailableError(f"POST /broker: {e}") from e
def _json_object(raw: bytes) -> dict[str, object]:
"""Parse a JSON object, tolerating an empty or malformed body (→ {}), like
the orchestrator client a bad body becomes a clean 'missing field' error
downstream rather than an opaque JSON crash."""
if not raw:
return {}
try:
obj = json.loads(raw)
except ValueError:
return {}
return obj if isinstance(obj, dict) else {}
def _error_detail(e: urllib.error.HTTPError) -> str:
"""The `error` string from a structured error response, best-effort — an
error body may be absent or unreadable, in which case there is no detail."""
try:
detail = _json_object(e.read()).get("error", "")
except Exception: # noqa: BLE001 — the error body is advisory only
return ""
return detail if isinstance(detail, str) else ""
def _request_from(payload: dict[str, object]) -> LaunchRequest:
"""Reconstruct the verified `LaunchRequest` the controller echoed, so the
returned value matches the in-process broker's (which returns the request it
acted on). A missing op/bottle_id means a malformed response."""
op = payload.get("op")
bottle_id = payload.get("bottle_id")
if not isinstance(op, str) or not isinstance(bottle_id, str) or not bottle_id:
raise BrokerClientError("host controller response missing op/bottle_id")
source_ip = payload.get("source_ip")
image_ref = payload.get("image_ref")
slot = payload.get("slot")
return LaunchRequest(
op=op,
bottle_id=bottle_id,
source_ip=source_ip if isinstance(source_ip, str) else "",
image_ref=image_ref if isinstance(image_ref, str) else "",
slot=slot if isinstance(slot, int) and not isinstance(slot, bool) else None,
)
__all__ = [
"BrokerClient",
"BrokerClientError",
"DEFAULT_TIMEOUT_SECONDS",
]
+55 -7
View File
@@ -18,9 +18,10 @@ import urllib.request
from collections.abc import Iterable
from dataclasses import dataclass
from ..log import debug
from ..orchestrator_auth import ROLE_CLI
from ..trust_domain import CONTROL_PLANE
from .server import ORCHESTRATOR_AUTH_HEADER
from .http_contract import ORCHESTRATOR_AUTH_HEADER
DEFAULT_TIMEOUT_SECONDS = 5.0
@@ -53,6 +54,23 @@ class RegisteredBottle:
env_var_secret: str = ""
@dataclass(frozen=True)
class BackendProbeFailure:
"""Safe diagnostic for an optional backend discovery probe."""
backend: str
error_type: str
def _probe_failure(backend: str, exc: BaseException) -> BackendProbeFailure:
failure = BackendProbeFailure(backend, type(exc).__name__)
debug(
"orchestrator discovery probe unavailable",
context={"backend": failure.backend, "error_type": failure.error_type},
)
return failure
class OrchestratorClient:
"""Trusted host-side client for the orchestrator control plane.
@@ -166,6 +184,27 @@ class OrchestratorClient:
)
return True
def update_agent_secret(
self, bottle_id: str, name: str, value: str, env_var_secret: str,
) -> bool:
"""Update ONE egress token for a running bottle in place
(`POST /bottles/<id>/secret`) the single-secret form of
`reprovision_gateway`, for pushing a freshly-refreshed host credential
into a bottle without a relaunch. Returns True on success, False when the
orchestrator doesn't know the bottle (404)."""
status, _ = self._request(
"POST",
f"/bottles/{bottle_id}/secret",
{"name": name, "value": value, "env_var_secret": env_var_secret},
)
if status == 404:
return False
if not 200 <= status < 300:
raise OrchestratorClientError(
f"update_agent_secret {bottle_id}: HTTP {status}"
)
return True
def teardown_bottle(self, bottle_id: str) -> bool:
"""Tear a bottle down (`DELETE /bottles/<id>`). False if the
orchestrator didn't know it (404) — idempotent for cleanup paths."""
@@ -245,32 +284,41 @@ def discover_orchestrator_url(*, timeout: float = 2.0) -> str:
orchestrator TAP. Returns the first that answers `/health`; raises if none
do (no orchestrator up launch a bottle first)."""
candidates: list[str] = []
failures: list[BackendProbeFailure] = []
try: # docker: loopback-published control plane
from .lifecycle import DEFAULT_PORT as _DOCKER_PORT
candidates.append(f"http://127.0.0.1:{_DOCKER_PORT}")
except Exception: # noqa: BLE001 — backend optional
except Exception as exc: # noqa: BLE001 — backend optional
failures.append(_probe_failure("docker", exc))
candidates.append("http://127.0.0.1:8099")
try: # firecracker: infra VM control plane on the orchestrator TAP
from ..backend.firecracker import netpool
from ..backend.firecracker.infra_vm import ORCHESTRATOR_PORT
candidates.append(
f"http://{netpool.orch_slot().guest_ip}:{ORCHESTRATOR_PORT}")
except Exception: # noqa: BLE001 — backend optional / not firecracker
pass
except Exception as exc: # noqa: BLE001 — backend optional / not firecracker
failures.append(_probe_failure("firecracker", exc))
try: # macOS: orchestrator container on its host-only address
from ..backend.macos_container.infra import probe_orchestrator_url
url = probe_orchestrator_url()
if url:
candidates.append(url)
except Exception: # noqa: BLE001 — backend optional / not macOS
pass
except Exception as exc: # noqa: BLE001 — backend optional / not macOS
failures.append(_probe_failure("macos-container", exc))
for url in candidates:
if OrchestratorClient(url, timeout=timeout).health():
return url
detail = ""
if failures:
detail = "; optional probes unavailable: " + ", ".join(
f"{failure.backend} ({failure.error_type})" for failure in failures
)
raise OrchestratorClientError(
"no running orchestrator control plane found (tried "
+ ", ".join(candidates)
+ "); launch a bottle first"
+ ")"
+ detail
+ "; launch a bottle first"
)
-270
View File
@@ -1,270 +0,0 @@
"""Host control server (issue #468) — the launch broker as a real host service.
Chunk 1 of the host-control-server stack closes the **transport** gap the PRD
opens with: today `LaunchBroker.submit(token)` is an in-process method call from
`OrchestratorCore`, and a real host service needs it reachable over the wire.
This module is that service the single privileged host component reached over
**HTTP** (the universal transport 0070 chose), mirroring the orchestrator control
plane's shape (`orchestrator/server.py`): a pure `dispatch()` for socket-free
testing, wrapped by a thin stdlib `http.server` adapter.
GET /health -> 200 {"status": "ok"}
POST /broker -> 200 {"op", "bottle_id", "source_ip", "image_ref", "slot"}
400 (bad body) | 401 (bad provenance/schema) | 502 (backend)
body: {"token": "<signed launch/teardown JWT>"}
Only the **signed token** crosses the wire; the server holds the shared HS256
secret and a real `LaunchBroker` (e.g. `DockerBroker`) and runs the existing
`verify_request` + `_launch`/`_teardown` path behind the endpoint, so nothing
free-form ever reaches it. Provenance/schema failures are fail-closed 401s that
never touch the backend (`LaunchBroker.submit` verifies before acting), and a
backend launch failure is a 502 the caller must surface neither takes the
controller down.
The signed launch token *is* the endpoint's authentication (its provenance is the
whole point of the JWS), so `/broker` needs no separate caller credential; the
host controller's own lifecycle endpoints, which do, arrive with the durable
`TrustDomain` key in a later chunk.
The shared signing secret is read from `$BOT_BOTTLE_BROKER_SECRET` (hex). That is
a **chunk-1 stopgap**: it must be provisioned to signer and verifier out of band,
which is exactly what the durable `TrustDomain` key in chunk 2 (#476) replaces.
"""
from __future__ import annotations
import argparse
import http.server
import json
import os
import socketserver
import sys
import typing
from urllib.parse import urlsplit
from .. import log
from .broker import BrokerAuthError, LaunchBroker
from .docker_broker import DockerBroker
# JSON body payload type (parsed request / rendered response).
Json = dict[str, object]
# The hex-encoded HS256 secret shared with the request signer (the orchestrator).
# Chunk-1 stopgap for the durable, out-of-band `TrustDomain` key of chunk 2.
BROKER_SECRET_ENV = "BOT_BOTTLE_BROKER_SECRET"
# Default host-controller port. Distinct from the orchestrator control plane
# (8099) — a separate privileged component listening on its own socket.
DEFAULT_PORT = 8091
# Cap on the request body. A signed broker request is tiny, so rejecting anything
# larger *before reading it* keeps a caller that can merely reach the socket (no
# signed token needed) from exhausting memory or a handler thread with a huge
# Content-Length — the signed token, not mere reachability, is the authority.
MAX_BODY_BYTES = 64 * 1024
# Per-request socket timeout, bounding how long a stalled / slow-loris caller can
# hold a handler thread on this privileged listener.
REQUEST_TIMEOUT_SECONDS = 15
def _parse_json_object(body: bytes) -> Json:
"""Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
if not body:
return {}
obj = json.loads(body) # raises json.JSONDecodeError (a ValueError)
if not isinstance(obj, dict):
raise ValueError("request body must be a JSON object")
return obj
def broker_secret_from_env(environ: typing.Mapping[str, str] | None = None) -> bytes | None:
"""The shared HS256 secret from `$BOT_BOTTLE_BROKER_SECRET` (hex), or None
when unset or not valid hex. The signer (orchestrator, `--broker http`) and
the verifier (this server) read the same env var so both hold the same key
the chunk-1 stand-in for out-of-band provisioning."""
env = os.environ if environ is None else environ
raw = env.get(BROKER_SECRET_ENV, "").strip()
if not raw:
return None
try:
return bytes.fromhex(raw)
except ValueError:
return None
def dispatch( # pylint: disable=too-many-return-statements
broker: LaunchBroker, method: str, path: str, body: bytes,
) -> tuple[int, Json]:
"""Route one host-control request to a (status, payload) pair. Pure — the
only side effect is the broker's own backend launch — so routing is testable
without a socket.
Total by design: a provenance/schema failure becomes 401 and a backend launch
failure becomes 502 rather than raising, so one bad request can neither act
on the backend nor take the controller down for the next caller."""
route = urlsplit(path).path.rstrip("/") or "/"
if method == "GET" and route == "/health":
return 200, {"status": "ok"}
if method == "POST" and route == "/broker":
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
token = data.get("token")
if not isinstance(token, str) or not token:
return 400, {"error": "token (string) is required"}
try:
req = broker.submit(token)
except BrokerAuthError as e:
# Fail-closed: bad signature, malformed token, or off-schema payload.
# `submit` verifies before acting, so nothing was launched.
return 401, {"error": f"broker auth failed: {e}"}
except Exception as e: # noqa: BLE001 — a backend launch failure (docker
# down, image gone) is operational, not a control-plane bug; the
# caller must see it as a distinct 502, and the server must stay up.
return 502, {"error": f"backend launch failed: {e}"}
return 200, {
"op": req.op,
"bottle_id": req.bottle_id,
"source_ip": req.source_ip,
"image_ref": req.image_ref,
"slot": req.slot,
}
return 404, {"error": "not found"}
class Handler(http.server.BaseHTTPRequestHandler):
"""Thin stdlib adapter: read the body, call `dispatch`, write JSON."""
# Socket timeout per request (applied by StreamRequestHandler.setup) so a
# stalled caller can't pin a handler thread on this privileged listener.
timeout = REQUEST_TIMEOUT_SECONDS
# Quiet by default; opt back into stdlib access logging with
# BOT_BOTTLE_HOST_CONTROLLER_DEBUG (the controller has its own logging).
def log_message(self, format: str, *args: typing.Any) -> None: # noqa: A002
if os.environ.get("BOT_BOTTLE_HOST_CONTROLLER_DEBUG"):
super().log_message(format, *args)
def _serve(self, method: str) -> None:
"""Read the request body (bounded), dispatch it, and write the JSON
reply. A dispatch that raises (it shouldn't — dispatch is total) still
returns a 500 rather than dropping the connection."""
server = self.server
assert isinstance(server, HostControlServer)
try:
length = int(self.headers.get("Content-Length") or 0)
except ValueError:
self._reply(400, {"error": "invalid Content-Length"})
return
if length < 0 or length > MAX_BODY_BYTES:
# Reject before reading: nothing legitimate is this big, so an
# oversized declared length is a bug or a resource-exhaustion attempt.
self._reply(413, {"error": "request body too large"})
return
body = self.rfile.read(length) if length > 0 else b""
try:
status, payload = dispatch(server.broker, method, self.path, body)
except Exception as e: # noqa: BLE001 — the controller must stay up
sys.stderr.write(f"host controller: {method} {self.path} failed: {e!r}\n")
sys.stderr.flush()
status, payload = 500, {"error": f"internal error: {e}"}
self._reply(status, payload)
def _reply(self, status: int, payload: typing.Mapping[str, object]) -> None:
"""Write one JSON response with an explicit Content-Length."""
data = json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self) -> None:
self._serve("GET")
def do_POST(self) -> None:
self._serve("POST")
class HostControlServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
"""Threading HTTP server that carries the launch broker for its handlers.
The broker holds the shared signing secret and performs the backend-native
launch/teardown; the server itself keeps no secret of its own provenance
rides entirely in each request's signed token."""
daemon_threads = True
allow_reuse_address = True
def __init__(self, address: tuple[str, int], broker: LaunchBroker) -> None:
self.broker = broker
super().__init__(address, Handler)
def make_host_server(
broker: LaunchBroker, host: str = "127.0.0.1", port: int = DEFAULT_PORT
) -> HostControlServer:
"""Build (but do not start) a host control server. `port=0` binds an
ephemeral port read `server.server_address` for the actual one."""
return HostControlServer((host, port), broker)
def main(argv: list[str] | None = None) -> int:
"""Run the host control server as a plain process (dev-harness).
python -m bot_bottle.orchestrator.host_server [--host H] [--port P]
Fail-closed: without a shared `$BOT_BOTTLE_BROKER_SECRET` the server can
verify no request's provenance, so it refuses to start rather than run a
launcher that accepts unsigned input."""
parser = argparse.ArgumentParser(prog="bot_bottle.orchestrator.host_server")
parser.add_argument("--host", default="127.0.0.1", help="bind address")
parser.add_argument("--port", type=int, default=DEFAULT_PORT, help="bind port (0 = ephemeral)")
args = parser.parse_args(argv)
secret = broker_secret_from_env()
if secret is None:
sys.stderr.write(
f"host controller: refusing to start without a shared signing secret "
f"(${BROKER_SECRET_ENV}, hex) — it could verify no request's "
"provenance and would relay unsigned launches to the backend\n"
)
sys.stderr.flush()
return 2
broker = DockerBroker(secret)
server = make_host_server(broker, host=args.host, port=args.port)
bound_host, bound_port = server.server_address[0], server.server_address[1]
log.info(
"host control server listening",
context={"host": bound_host, "port": bound_port},
)
try:
server.serve_forever()
except KeyboardInterrupt:
log.info("host controller shutting down")
finally:
server.server_close()
return 0
__all__ = [
"dispatch",
"Handler",
"HostControlServer",
"make_host_server",
"broker_secret_from_env",
"main",
"Json",
"BROKER_SECRET_ENV",
"DEFAULT_PORT",
]
if __name__ == "__main__":
raise SystemExit(main())
+11
View File
@@ -0,0 +1,11 @@
"""Dependency-free constants shared by orchestrator HTTP clients and server."""
ORCHESTRATOR_AUTH_HEADER = "x-bot-bottle-orchestrator-auth"
MAX_BODY_BYTES = 1 * 1024 * 1024
REQUEST_BODY_TIMEOUT_SECONDS = 10.0
__all__ = [
"MAX_BODY_BYTES",
"ORCHESTRATOR_AUTH_HEADER",
"REQUEST_BODY_TIMEOUT_SECONDS",
]
+9 -1
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
from ..log import debug
from .client import OrchestratorClient, OrchestratorClientError
@@ -27,7 +28,14 @@ def reprovision_bottles(
try:
if client.reprovision_gateway(bottle_id, secret):
restored += 1
except OrchestratorClientError:
except OrchestratorClientError as exc:
debug(
"gateway secret reprovision failed; continuing with other bottles",
context={
"bottle_id": bottle_id,
"error_type": type(exc).__name__,
},
)
continue
return restored
+55 -427
View File
@@ -1,452 +1,80 @@
"""Orchestrator HTTP control plane (PRD 0070).
The backend-agnostic control-plane RPC (CLI / console -> orchestrator) over
**HTTP** the universal transport chosen in 0070 (works on every host; no
vsock / unix-socket portability caveats):
GET /health -> 200 {"status": "ok"}
GET /gateway -> 200 {"configured", ["name","running"]}
GET /bottles -> 200 {"bottles": [ <redacted record>, ...]}
POST /bottles -> 201 {"bottle_id","identity_token"} (launch)
body: {"source_ip", ["image_ref"],
["metadata"], ["policy"],
["tokens"], ["env_var_secret"]}
PUT /bottles/<bottle_id>/policy -> 200 {"updated": true} | 404 (live reload)
body: {"policy"}
POST /bottles/<bottle_id>/reprovision_gateway
-> 200 {"reprovisioned": true} | 404
body: {"env_var_secret"}
DELETE /bottles/<bottle_id> -> 200 {"torn_down": true} | 404 (teardown)
POST /reconcile -> 200 {"reaped": [bottle_id, ...]}
body: {"live_source_ips": [...],
["grace_seconds"]}
POST /attribute -> 200 {"bottle_id"} | 403
POST /resolve -> 200 {"bottle_id","policy"} | 403
body: {"source_ip","identity_token"}
GET /supervise/proposals -> 200 {"proposals": [ <proposal>, ...]}
POST /supervise/respond -> 200 {"responded": true} | 409 (operator)
body: {"proposal_id","bottle_slug",
"decision", ["notes"],["final_file"]}
POST /supervise/propose -> 201 {"proposal_id"} | 403 (agent)
body: {"source_ip","identity_token",
"tool","proposed_file","justification"}
POST /supervise/poll -> 200 {"status", ["notes"],["final_file"]} | 403
body: {"source_ip","identity_token",
"proposal_id"}
The `/supervise/propose` + `/supervise/poll` pair is the **agent** half of the
supervise flow: the data plane (supervise / egress / git-gate) queues a proposal
and polls for its response over RPC instead of opening `bot-bottle.db` directly.
`poll` is idempotent it never archives, so a dropped connection can't lose an
operator decision (the row is reaped when the bottle is torn down / reconciled).
Both attribute the caller by `(source_ip, identity_token)` exactly like
`/resolve`, so a bottle can only ever queue or read its own proposals.
`POST /bottles` / `DELETE` drive the full launch lifecycle: they mint (or
tear down) the bottle in the registry AND broker the backend-native launch
via the orchestrator. Register/deregister without a launch are internal to
`OrchestratorCore`, not exposed here.
Routing/handling is the pure function `dispatch()` so it is unit-testable
without a socket; `Handler` / `OrchestratorServer` / `make_server` are a
thin stdlib adapter around it. Listing redacts identity tokens they are
returned only once, to the caller that launches the bottle.
"""
"""Uvicorn transport for the FastAPI orchestrator control plane."""
from __future__ import annotations
import http.server
import json
import os
import socketserver
import sys
import typing
from urllib.parse import urlsplit
import socket
import threading
import uvicorn
from ..orchestrator_auth import ROLE_CLI, ROLES
from ..trust_domain import CONTROL_PLANE
from ..supervisor.types import TOOLS
from .api import create_app
from .http_contract import MAX_BODY_BYTES, ORCHESTRATOR_AUTH_HEADER
from .service import OrchestratorCore
# JSON body payload type (parsed request / rendered response).
Json = dict[str, object]
# The request header carrying the caller's role-scoped control-plane token (a
# signed JWT naming the caller's role — see orchestrator_auth). The role gates which
# routes the caller may reach: the data plane holds a `gateway` token good only
# for the agent-facing lookups; the host CLI holds a `cli` token for the
# operator/mutating routes. An agent that can merely *reach* the port holds no
# token at all, and a compromised gateway holds only `gateway` — neither can
# drive the operator routes (approve proposals, rewrite policy, read tokens).
ORCHESTRATOR_AUTH_HEADER = "x-bot-bottle-orchestrator-auth"
# The routes the data plane (role `gateway`) is allowed to reach — exactly the
# per-request lookups PolicyResolver makes. Every other authenticated route is
# operator-only. `cli` is a superset role: it may reach any route.
_GATEWAY_ROUTES: frozenset[tuple[str, str]] = frozenset({
("POST", "/resolve"),
("POST", "/supervise/propose"),
("POST", "/supervise/poll"),
})
MAX_REQUESTS = 32
KEEP_ALIVE_TIMEOUT_SECONDS = 10
def _allowed_roles(method: str, route: str) -> frozenset[str]:
"""The roles permitted on `(method, route)`: `gateway` or `cli` on the
data-plane routes, `cli`-only everywhere else."""
if (method, route) in _GATEWAY_ROUTES:
return ROLES
return frozenset({ROLE_CLI})
class OrchestratorServer:
"""Small lifecycle wrapper around Uvicorn with an eagerly bound socket."""
def __init__(self, config: uvicorn.Config) -> None:
self._server = uvicorn.Server(config)
self._stopped = threading.Event()
self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._socket.bind((config.host, config.port))
self._socket.listen(config.backlog)
self.server_address = self._socket.getsockname()
def _parse_json_object(body: bytes) -> Json:
"""Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
if not body:
return {}
obj = json.loads(body) # raises json.JSONDecodeError (a ValueError)
if not isinstance(obj, dict):
raise ValueError("request body must be a JSON object")
return obj
def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
orch: OrchestratorCore, method: str, path: str, body: bytes, *, role: str | None = ROLE_CLI,
) -> tuple[int, Json]:
"""Route one control-plane request to a (status, payload) pair. Pure —
no I/O beyond the orchestrator so it is fully testable without a socket.
`role` is the caller's verified control-plane role (`gateway` or `cli`), or
None for an unauthenticated request; an open-mode server (no signing key
configured see `OrchestratorServer`) passes `cli`. Every route except
`GET /health` requires a role: a missing role is 401, and a role that
doesn't cover the route is 403 — so a `gateway` data-plane token can reach
`/resolve` + `/supervise/{propose,poll}` but not the operator routes
(rewrite policy, read injected tokens, approve its own supervise proposals).
The source-IP + identity-token checks inside `/resolve` and `/attribute`
authenticate the *bottle* a request is about, not the *caller*, so this role
gate is what protects the caller-privileged routes. Defaults `cli` so unit
tests of the routing logic don't have to thread it through."""
route = urlsplit(path).path.rstrip("/") or "/"
if method == "GET" and route == "/health":
return 200, {"status": "ok"}
# Role gate — every route below is a trusted-caller operation. Deny before
# touching the registry / broker / supervise store.
if role is None:
return 401, {"error": "control-plane authentication required"}
if role not in _allowed_roles(method, route):
return 403, {"error": "insufficient role for this route"}
if method == "GET" and route == "/gateway":
return 200, orch.gateway_status()
if method == "GET" and route == "/bottles":
return 200, {"bottles": [r.redacted() for r in orch.registry.all()]}
if method == "POST" and route == "/bottles":
def run(self) -> None:
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
image_ref = data.get("image_ref")
metadata = data.get("metadata")
policy = data.get("policy")
raw_tokens = data.get("tokens")
tokens = {
k: v for k, v in raw_tokens.items() if isinstance(k, str) and isinstance(v, str)
} if isinstance(raw_tokens, dict) else {}
env_var_secret = data.get("env_var_secret", "")
rec = orch.launch_bottle(
source_ip,
image_ref=image_ref if isinstance(image_ref, str) else "",
metadata=metadata if isinstance(metadata, str) else "",
policy=policy if isinstance(policy, str) else "",
tokens=tokens,
env_var_secret=env_var_secret if isinstance(env_var_secret, str) else "",
)
return 201, {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
self._server.run(sockets=[self._socket])
finally:
self._stopped.set()
if method == "PUT" and route.startswith("/bottles/") and route.endswith("/policy"):
bottle_id = route[len("/bottles/"):-len("/policy")]
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
policy = data.get("policy")
if not isinstance(policy, str):
return 400, {"error": "policy (string) is required"}
if orch.set_policy(bottle_id, policy):
return 200, {"updated": True}
return 404, {"error": "no such bottle"}
def serve_forever(self) -> None:
self.run()
if (
method == "POST"
and route.startswith("/bottles/")
and route.endswith("/reprovision_gateway")
):
bottle_id = route[len("/bottles/") : -len("/reprovision_gateway")]
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
env_var_secret = data.get("env_var_secret")
if not isinstance(env_var_secret, str) or not env_var_secret:
return 400, {"error": "env_var_secret (string) is required"}
if orch.reprovision_from_secret(bottle_id, env_var_secret):
return 200, {"reprovisioned": True}
return 404, {"error": "no stored secrets for this bottle"}
def shutdown(self) -> None:
self._server.should_exit = True
self._stopped.wait(timeout=5)
if method == "DELETE" and route.startswith("/bottles/"):
bottle_id = route[len("/bottles/"):]
if orch.teardown_bottle(bottle_id):
return 200, {"torn_down": True}
return 404, {"error": "no such bottle"}
if method == "POST" and route == "/reconcile":
# Host-driven self-heal: the caller enumerates its live bottles (only
# the host can see the backend) and the orchestrator drops rows for
# every other active bottle. Trusted-caller only — an agent that could
# reach this would be able to unregister its neighbours.
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
raw_ips = data.get("live_source_ips")
if not isinstance(raw_ips, list):
return 400, {"error": "live_source_ips (list of strings) is required"}
live = [ip for ip in raw_ips if isinstance(ip, str) and ip]
grace = data.get("grace_seconds")
kwargs = (
{"grace_seconds": float(grace)}
if isinstance(grace, (int, float)) and not isinstance(grace, bool)
else {}
)
return 200, {"reaped": orch.reconcile(live, **kwargs)}
if method == "POST" and route == "/attribute":
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
if not isinstance(source_ip, str) or not isinstance(token, str):
return 400, {"error": "source_ip and identity_token (strings) required"}
rec = orch.attribute(source_ip, token)
if rec is None:
return 403, {"error": "unattributed"}
return 200, {"bottle_id": rec.bottle_id}
if method == "GET" and route == "/supervise/proposals":
# Operator TUI: pending supervise proposals across all bottles.
return 200, {"proposals": orch.supervise_pending()}
if method == "POST" and route == "/supervise/respond":
# Operator decision: apply (approve/modify rewrites egress policy),
# write the queued response, audit — all server-side on the one DB.
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
proposal_id = data.get("proposal_id")
bottle_slug = data.get("bottle_slug")
decision = data.get("decision")
if not (isinstance(proposal_id, str) and proposal_id):
return 400, {"error": "proposal_id (string) is required"}
if not (isinstance(bottle_slug, str) and bottle_slug):
return 400, {"error": "bottle_slug (string) is required"}
if not (isinstance(decision, str) and decision):
return 400, {"error": "decision (string) is required"}
notes = data.get("notes")
final_file = data.get("final_file")
ok, err = orch.supervise_respond(
proposal_id,
bottle_slug=bottle_slug,
decision=decision,
notes=notes if isinstance(notes, str) else "",
final_file=final_file if isinstance(final_file, str) else None,
)
if ok:
return 200, {"responded": True}
return 409, {"error": err}
if method == "POST" and route == "/supervise/propose":
# Agent half: queue a proposal, attributed to the caller resolved from
# (source_ip, identity_token) — never a caller-supplied slug — so the
# data plane can't forge attribution. Fail-closed 403 when unattributed.
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
tool = data.get("tool")
proposed_file = data.get("proposed_file")
justification = data.get("justification")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
if not isinstance(tool, str) or tool not in TOOLS:
return 400, {"error": f"tool (string) must be one of {TOOLS}"}
if not isinstance(proposed_file, str) or not proposed_file:
return 400, {"error": "proposed_file (string) is required"}
if not isinstance(justification, str) or not justification:
return 400, {"error": "justification (string) is required"}
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
if rec is None:
return 403, {"error": "unattributed"}
proposal_id = orch.supervise_queue_proposal(
rec.bottle_id, tool=tool, proposed_file=proposed_file,
justification=justification,
)
return 201, {"proposal_id": proposal_id}
if method == "POST" and route == "/supervise/poll":
# Agent half: non-blocking read of the caller's own proposal decision.
# Attributed like /propose, and scoped to the resolved bottle id, so a
# guessed proposal_id can never read another bottle's response.
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
proposal_id = data.get("proposal_id")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
if not isinstance(proposal_id, str) or not proposal_id:
return 400, {"error": "proposal_id (string) is required"}
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
if rec is None:
return 403, {"error": "unattributed"}
return 200, orch.supervise_poll_response(rec.bottle_id, proposal_id)
if method == "POST" and route == "/resolve":
# The per-request lookup the multi-tenant gateway makes: returns the
# bottle's policy. Requires a matching (source_ip, identity_token)
# pair — a missing/empty/mismatched token fail-closes (403), no
# source-IP-only fallback.
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
if rec is None:
return 403, {"error": "unattributed"}
# tokens are the in-memory per-bottle egress auth values the gateway
# injects; served here, never persisted.
return 200, {
"bottle_id": rec.bottle_id,
"policy": rec.policy,
"tokens": orch.tokens_for(rec.bottle_id),
}
return 404, {"error": "not found"}
class Handler(http.server.BaseHTTPRequestHandler):
"""Thin stdlib adapter: read the body, call `dispatch`, write JSON."""
# Quiet by default (the orchestrator has its own logging); opt back into
# stdlib access logging with BOT_BOTTLE_ORCHESTRATOR_DEBUG.
def log_message(self, format: str, *args: typing.Any) -> None: # noqa: A002
if os.environ.get("BOT_BOTTLE_ORCHESTRATOR_DEBUG"):
super().log_message(format, *args)
def _serve(self, method: str) -> None:
"""Read the request body, dispatch it, and write the JSON reply. A
dispatch failure (e.g. a broker error) returns a 500 rather than
crashing the connection, so one bad request can't take the control
plane down for the caller."""
server = self.server
assert isinstance(server, OrchestratorServer)
length = int(self.headers.get("Content-Length") or 0)
body = self.rfile.read(length) if length > 0 else b""
role = server.role_for(self.headers.get(ORCHESTRATOR_AUTH_HEADER, ""))
try:
status, payload = dispatch(
server.orchestrator, method, self.path, body, role=role)
except Exception as e: # noqa: BLE001 — the control plane must stay up
sys.stderr.write(f"orchestrator: {method} {self.path} failed: {e!r}\n")
sys.stderr.flush()
status, payload = 500, {"error": f"internal error: {e}"}
data = json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self) -> None:
self._serve("GET")
def do_POST(self) -> None:
self._serve("POST")
def do_PUT(self) -> None:
self._serve("PUT")
def do_DELETE(self) -> None:
self._serve("DELETE")
class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
"""Threading HTTP server that carries the orchestrator for its handlers.
Holds the per-host control-plane *signing key* (from
`$BOT_BOTTLE_ORCHESTRATOR_TOKEN`, injected by the launcher into the
orchestrator process only) and verifies each request's role-scoped token
against it. When a key is set, every route but `/health` requires a valid
token whose role covers the route; when it is unset the server runs **open**
(full `cli` access) and says so loudly at startup a fail-visible fallback
for tests and any backend that hasn't wired the key yet (e.g. Firecracker,
whose nft boundary already blocks agents from the control-plane port)."""
daemon_threads = True
allow_reuse_address = True
def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None:
self.orchestrator = orchestrator
# The control-plane trust domain's signing key, as injected into THIS
# (the owning) process by the launcher (#476). Unset → open mode below.
self._signing_key = CONTROL_PLANE.key_from_env()
if not self._signing_key:
sys.stderr.write(
"orchestrator: WARNING — no control-plane signing key "
f"(${CONTROL_PLANE.key_env}); running WITHOUT caller "
"authentication. Any client that can reach this port can drive "
"it. Backends that put the control plane on an agent-reachable "
"network MUST set this.\n"
)
sys.stderr.flush()
super().__init__(address, Handler)
def role_for(self, presented: str) -> str | None:
"""The role the request is authorized as, or None if unauthenticated.
Open mode (no signing key) grants full `cli` access the fail-visible
fallback. Otherwise verify the presented signed token; a missing/invalid
token yields None ( 401), a valid one yields its `gateway`/`cli`
role ( per-route 401/403 in `dispatch`)."""
if not self._signing_key:
return ROLE_CLI
return CONTROL_PLANE.verify(presented, self._signing_key)
def server_close(self) -> None:
self._socket.close()
def make_server(
orchestrator: OrchestratorCore, host: str = "127.0.0.1", port: int = 0
orchestrator: OrchestratorCore,
host: str = "127.0.0.1",
port: int = 0,
*,
signing_key: str | None = None,
) -> OrchestratorServer:
"""Build (but do not start) a control-plane server. `port=0` binds an
ephemeral port read `server.server_address` for the actual one."""
return OrchestratorServer((host, port), orchestrator)
"""Build a bounded Uvicorn server around the orchestrator application."""
key = CONTROL_PLANE.key_from_env() if signing_key is None else signing_key
app = create_app(orchestrator, signing_key=key)
config = uvicorn.Config(
app,
host=host,
port=port,
access_log=bool(os.environ.get("BOT_BOTTLE_ORCHESTRATOR_DEBUG")),
log_level="info",
limit_concurrency=MAX_REQUESTS,
timeout_keep_alive=KEEP_ALIVE_TIMEOUT_SECONDS,
server_header=False,
)
return OrchestratorServer(config)
__all__ = [
"dispatch", "Handler", "OrchestratorServer", "make_server", "Json",
"KEEP_ALIVE_TIMEOUT_SECONDS",
"MAX_BODY_BYTES",
"MAX_REQUESTS",
"ORCHESTRATOR_AUTH_HEADER",
"OrchestratorServer",
"create_app",
"make_server",
]
+36 -19
View File
@@ -25,7 +25,7 @@ import json
from collections.abc import Iterable
from datetime import datetime, timezone
from .broker import BrokerUnavailableError, LaunchRequest, SubmitBroker, sign_request
from .broker import LaunchBroker, LaunchRequest, sign_request
from .store.registry_store import DEFAULT_REAP_GRACE_SECONDS, BottleRecord, RegistryStore
from .supervisor import (
AuditEntry,
@@ -62,7 +62,7 @@ class OrchestratorCore:
def __init__(
self,
registry: RegistryStore,
broker: SubmitBroker,
broker: LaunchBroker,
sign_secret: bytes,
supervisor: Supervisor | None = None,
) -> None:
@@ -111,23 +111,14 @@ class OrchestratorCore:
image_ref=image_ref,
slot=slot,
)
launched = False
try:
self._broker.submit(sign_request(req, self._secret))
except BrokerUnavailableError:
# Ambiguous delivery failure (timeout / dropped response): the broker
# may already have launched the bottle before the response was lost.
# Do NOT deregister — that would orphan a running container with no
# registry row (reconcile reaps rows, never containers). Keep the row
# so reconcile reaps it iff the bottle is not actually live; surface
# the error so the caller knows the launch is unconfirmed.
raise
except Exception:
# A definite failure — a fail-closed rejection, a backend launch
# error, or the host reporting it did not launch: nothing is running,
# so roll the registry entry back to leave no orphan.
self.registry.deregister(rec.bottle_id)
self._tokens.pop(rec.bottle_id, None)
raise
launched = True
finally:
if not launched:
self.registry.deregister(rec.bottle_id)
self._tokens.pop(rec.bottle_id, None)
return rec
def teardown_bottle(self, bottle_id: str) -> bool:
@@ -380,10 +371,36 @@ class OrchestratorCore:
if not encrypted:
return False
try:
self._tokens[bottle_id] = {k: decrypt_value(env_var_secret, v)
for k, v in encrypted.items()}
decrypted = {
k: decrypt_value(env_var_secret, v) for k, v in encrypted.items()
}
except ValueError:
return False
self._tokens[bottle_id] = decrypted
return True
def update_agent_secret(
self, bottle_id: str, name: str, value: str, env_var_secret: str,
) -> bool:
"""Update ONE egress token for a known bottle in place — the
single-secret form of ``reprovision_from_secret``.
Sets the in-memory token AND upserts the single re-encrypted row under
*env_var_secret* (the same key the rest of the rows are encrypted with, so
the whole set stays decryptable by a later ``reprovision_from_secret``),
leaving every other token untouched. Returns False if the bottle is
unknown.
Unlike ``reprovision_from_secret`` (which restores the values captured at
launch), this pushes a *caller-supplied* value used to refresh a
short-lived host credential (e.g. the Codex access token) into a
still-running bottle without a relaunch."""
from .store.secret_store import encrypt_value
if self.registry.get(bottle_id) is None:
return False
self._tokens.setdefault(bottle_id, {})[name] = value
self.registry.store_agent_secret(
bottle_id, name, encrypt_value(env_var_secret, value))
return True
# --- consolidated gateway ----------------------------------------------

Some files were not shown because too many files have changed in this diff Show More