Splits the Linux clean-install harness into two variants, selected with
BB_TEST_PREREQS:
with (default) — install python3 + git + pipx first, then install.sh.
The ready-host happy path; PASS = install.sh exits 0 and
leaves a runnable bot-bottle entry point.
without — run install.sh on the BARE cloud image. Exercises
install.sh's own prerequisite-guard logic (python gate,
git-for-git-specs gate, pipx/pip PEP-668 handling). PASS =
install.sh either fully succeeds OR declines with one of
its own recognized, actionable prerequisite errors; a
crash or unrecognized failure is a FAIL.
cmd_run now captures install.sh's exit code + full output (install.rc /
install.log) instead of aborting on non-zero, so the verdict step applies the
variant's criterion. The old assert_installed is replaced by a quiet
entry_point_runnable helper plus classify_outcome, which matches the bare-host
declines against the exact die() messages install.sh prints. doctor still runs
for visibility, but only when an entry point exists.
test-all now runs the full matrix -- every distro x both variants -- each cell
in its own throwaway VM, and can be pinned to one variant via BB_TEST_PREREQS.
Research note updated to document both variants and their pass criteria.
Validated with `bash -n` and `shellcheck`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds scripts/linux-install-test.sh, a throwaway-VM harness that exercises
install.sh the way a brand-new user would across a Linux distro matrix
(Ubuntu, Fedora, Arch, Alpine, NixOS), plus the research note
docs/research/testing-clean-install-on-linux.md that motivates the approach.
This is the Linux counterpart to the macOS clean-install harness. On Linux
the boundary of choice flips from a throwaway user account to a disposable
KVM VM: it's a genuine kernel + userland + package-manager boundary that
wipes to nothing on teardown, and a single harness can swap distro cloud
images to cover the several package-management regimes Linux fragments into.
The host already needs KVM for the Firecracker backend, so a per-run,
copy-on-write VM (qemu-img create -b base) is cheap here -- the equivalent
of `docker run --rm`, for a whole machine.
Per run the harness caches one read-only base image, boots a throwaway
overlay via QEMU/KVM with user-mode networking (no root, no bridge), injects
an ephemeral SSH key + passwordless login through a cloud-init seed ISO,
installs the distro's prerequisites (python3 + git + pipx), pipes THIS
checkout's install.sh into the guest exactly as `curl ... | sh` would, and
asserts the CLI installed cleanly. The overlay is deleted on teardown, so
even the OS-level prerequisites are wiped -- unlike a throwaway user, the
reset is total.
Subcommands mirror the macOS harness (up/run/status/down/test) plus test-all
(the matrix) and ssh (an interactive guest shell). test arms an EXIT/INT/TERM
trap the moment the VM exists, so a failure or Ctrl-C still tears it down.
Scope is installer correctness, not runtime: there is no nested KVM/Docker in
the VM, so `bot-bottle doctor` correctly reports every backend not-ready and
exits non-zero by design. The pass criterion is therefore install.sh exiting
0, the bot-bottle entry point being present on the fresh user's PATH, and
`bot-bottle --version` running -- not a green doctor. doctor's output is still
printed so a real installer regression (broken shim, import error) stays
visible.
Validated with `bash -n` and `shellcheck`. Runtime is host-only (needs
/dev/kvm, qemu, cloud-localds), so like the macOS harness it isn't exercised
by the Linux PR CI. The cloud-image URLs in the DISTRO table are the one
place to bump when a distro cuts a newer build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lower the diff-coverage gate from 90% to 80% and the critical-module
target from 90% to 85%. The 90% diff gate forced back-fill tests on
nearly every changed line; 80% keeps new code honest without the churn.
Global coverage stays informational per ADR 0004 (no new gate added).
Updates scripts/diff_coverage.py, scripts/coverage.sh,
scripts/critical-modules.txt, .gitea/workflows/test.yml, and records the
change as a dated revision in ADR 0004.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the third review round on PR #481.
- `bot-bottle doctor` now checks `is_backend_ready()` (a full backend
status() probe: daemon reachable, network pool present, KVM usable)
instead of the cheap PATH-only `is_backend_available()`. A host with a
stopped Docker daemon or half-configured Firecracker no longer reports
`ok: backend` / exit 0 when `start` can't actually work; each not-ready
backend prints its own diagnostics, and doctor passes only if at least
one backend is ready.
- `install.sh` resolves the pip `--user` scripts directory from the
interpreter (`sysconfig.get_path("scripts", get_preferred_scheme("user"))`)
instead of hardcoding `~/.local/bin`, which is wrong on a python.org
macOS interpreter (`~/Library/Python/<X.Y>/bin`). The PATH guidance now
prints the actual directory.
Tests: doctor tests mock `is_backend_ready` (the readiness contract) and
cover the not-ready → fail path; a new install-script test drives the
macOS `osx_framework_user` scheme and asserts it resolves a
non-~/.local/bin directory.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the review on PR #481.
Self-contained wheel (review point 1): the gateway/infra/orchestrator
images build from a context that must hold bot_bottle/, pyproject.toml,
and the root-level Dockerfiles. Modules previously located these by
walking __file__ to the repo root, so an installed wheel (package in
site-packages, no repo root) passed `doctor` but failed `start`.
- Add bot_bottle/resources.py: build_root() returns the repo root in a
checkout (unchanged) or a staged copy from the wheel's bundled
_resources/ otherwise; dockerfile()/nix_netpool_module()/
netpool_script() derive from it.
- setup.py bundles the root Dockerfiles, nix module, netpool script, and
pyproject.toml into bot_bottle/_resources/ at build; MANIFEST.in ships
them in the sdist.
- Route every _REPO_ROOT/_REPO_DIR call site (docker/macos launch, macos
infra, firecracker infra_vm/infra_artifact/setup, orchestrator
lifecycle/gateway) through resources. Checkout behavior is unchanged.
install.sh prerequisites (review point 2): check for git when installing
a git+ spec, and — before the pip fallback — that pip is usable and the
interpreter isn't externally managed (PEP 668), pointing at pipx.
Tests: test_resources covers checkout + staged-wheel layouts;
test_wheel_install builds the wheel, installs it into an isolated venv,
and asserts `doctor` runs and build_root() yields a valid context.
Running `start` end-to-end still needs a Docker/KVM host (CI).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Give bot-bottle a real distribution path so new users can install
without cloning the repo:
- pyproject.toml: full project metadata, a `bot-bottle` console-script
entry point (bot_bottle.cli:main), and package-data for the runtime
assets (Dockerfiles, egress entrypoint, netpool defaults, macos init).
Still zero runtime pip dependencies.
- install.sh: POSIX, sudo-free, idempotent bootstrapper — checks Python
>= 3.11, creates ~/.bot-bottle/{agents,bottles,contrib}, installs via
pipx (pip --user fallback), then runs `bot-bottle doctor`.
- `bot-bottle doctor`: new store-free subcommand reporting Python
version, backend availability (reuses is_backend_available rather than
hardcoding Docker), and config-dir presence. Exits non-zero when a hard
prerequisite is unmet.
- PRD prd-new-install-script and unit tests for doctor, the packaging
contract, and the install script.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the advisory macOS Apple Container job run on workflow_dispatch
only, removing the push-to-`main` trigger so a single non-redundant
laptop never runs unattended on every push. Updates the job comment,
docs/ci.md, README CI note, and the PRD accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
git-gate is fully implemented on the macos-container backend via the
gateway's consolidated git-http daemon with dynamic key
provisioning/revocation; only the legacy per-bottle git:// daemon is
unused. Fixes a stale README claim that git-gate is "deferred" and
removes the PRD open-question built on that false premise (sandbox-escape
attack 5 runs the same on macOS as on the other backends).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an `integration-macos` job that runs the integration suite against
BOT_BOTTLE_BACKEND=macos-container on a self-hosted host-mode macOS runner
(label `macos`), plus the PRD and provisioning docs.
The job is advisory (push-to-main + workflow_dispatch only, never PRs, not
in coverage.needs) since it targets a single non-redundant laptop. It
preflights `container`/`backend status` so a misprovisioned runner fails
loudly, serializes on a concurrency group, and tears down the
`bot-bottle-mac-infra` singleton on exit (#425).
Also relaxes the TestSandboxEscape CI skip guard: it skipped every backend
but firecracker under GITEA_ACTIONS, which would also skip on a host-mode
macOS runner. The guard's real target is the containerized act_runner, so
it now allows both host-mode backends (firecracker, macos-container)
through — otherwise the macOS job would go green while skipping the one
end-to-end test that proves the backend launches.
Closes#426
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the empty-key opt-out codex flagged: ControlPlaneProvisioning
no longer lets the orchestrator start without a signing key when a
backend declares an "isolated" topology. Host separation is not the
safety condition — the gateway (and any other caller) must reach the
control-plane listener for /resolve, so an open orchestrator would
still grant them full `cli`. The signing key is now mandatory for
every backend.
Drops the now-purposeless Topology/COLOCATED machinery (no backend
declared a non-default topology) and the contractual
test_orchestrator_key_allows_empty_when_isolated. Updates the PRD's
invariant 4 and lifecycle docstring accordingly. Docs/behaviour only
otherwise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review on #482: drop the generic "credential boundary" framing in the
PRD and docstrings and talk about the specific services — the orchestrator holds
the control-plane key; the host controller (#468) gets a separate key the
orchestrator never holds, so the orchestrator can't mint the credentials it uses
to talk to the host controller that owns its lifecycle. Lead with that concrete
win. No code behaviour change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hoist control-plane auth provisioning out of the per-backend launchers into
one shared contract, parameterized per trust domain (#476). Every blocking
finding in PR #471 was the same integration-bug class: each launcher
re-derived, by hand, how to generate the signing key, scope it to the
orchestrator, mint the gateway JWT, and keep the host key canonical.
Introduces `trust_domain.py`:
* `TrustDomain` — one credential boundary (host-canonical key file + role
set + env vars). `mint`/`verify` are scoped to the domain's roles, so a
future host-controller domain (#468) uses its own key/verifier/roles
rather than a `host` role on the control plane's frozenset (which the
orchestrator key could then forge).
* `ControlPlaneProvisioning` — the single seam answering the four
invariants: host-canonical key, split key-vs-token credential, CLI token
valid across co-running backends, and fail-closed (no open mode) for any
co-located topology.
* `Topology` — the backend declares what it is; the default is co-located +
fail-closed, so a backend need not redeclare it.
The `Orchestrator` ABC gets `control_plane_key()` (fail-closed) and routes
`mint_gateway_token()` through the contract; docker/macOS/firecracker
orchestrators, the server (verify), and the host CLI client (mint cli) all go
through the domain instead of reading the host key directly. `orchestrator_auth`
gains an optional `roles=` arg (default unchanged) so a domain scopes its own
role set; `paths.host_signing_key(filename)` generalizes host_orchestrator_token.
Adds unit coverage for the domain boundary + provisioning invariants and a PRD
capturing the durable rationale. No change to the auth primitive's HMAC, the
plane split, or the server's documented open-mode fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes#476
Reevaluated against the actual nft ruleset: the firecracker split's isolation
is nearly free, not "the real work." Agent VMs are already dropped except the
DNAT'd gateway ports, so re-pointing that single DNAT rule at the gateway VM
(dnat to $(gw_guest)) isolates agents from a separate orchestrator VM with zero
new agent rules; the only added nft is a mirrored gateway link + one
gateway->orchestrator forward rule. The effort is the mechanical second-VM
lifecycle, and it's validatable on a Firecracker host.
Sharpened the Access bullet accordingly and added a note to Sequencing
decoupling the original consolidation's "real work" (the broker shim) from the
split, whose difficulty actually runs the other way (docker/macOS do the real
work — new control network, dual-homed gateway, the token-only->L3 upgrade;
firecracker is nearly free).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings in main's 8 commits — #470 (backend-agnostic CI guards: BackendStatus
enum, quiet status(), is_backend_available/is_backend_ready, tests/_backend.py
skip_unless_backend) plus firecracker status() readiness checks (kvm/kernel/
dropbear/mke2fs).
Reconciled #470's additions into the reorg'd backend structure:
* BackendStatus enum + the status(*, quiet=) ABC signature -> backend/base.py
* is_backend_available / is_backend_ready -> backend/selection.py
* exposed all three through the lazy backend facade (__init__).
Integration-test conflicts were our control_plane->orchestrator renames vs
main's skip_unless_docker -> skip_unless_backend swap: kept our refactored
import paths + main's guard. Repointed main's new is_backend_* tests to patch
selection._backends (our moved home) instead of the facade.
pyright: 0 errors. Full unit suite green (2272).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review feedback: the hand-rolled `/dev/kvm` + docker-daemon capability
probes duplicated logic the CLI already owns. Delegate instead.
- tests/_backend.py skip guards now gate on `bot_bottle.backend.has_backend`
(each backend's `is_available()` classmethod — the same probe behind
`./cli.py backend status`), dropping the bespoke `Capability` probes.
- Remove tests/backend_preflight.py; the docker integration job runs
`./cli.py backend status --backend=docker` as its preflight (clear
per-check summary, non-zero exit when unready), matching the firecracker
job. The firecracker preflight reverts to its original binary/KVM checks
(backend status doesn't cover those).
- Unit test + docs updated to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Integration tests now select their backend from BOT_BOTTLE_BACKEND and
skip on the capability that backend actually needs, instead of gating
every backend on unrelated Docker availability.
Task 1 — backend-agnostic guards (tests/_backend.py):
- Capability probes: docker_capability() (reachable daemon) and
firecracker_capability() (accessible /dev/kvm + firecracker on PATH,
Docker-independent). backend_capability()/selected_backend() resolve
the target from BOT_BOTTLE_BACKEND (default docker).
- skip_unless_selected_backend_available() for backend-agnostic tests
(test_sandbox_escape) — runs through whichever backend is selected and
checks that backend's real capability.
- skip_unless_backend("docker") for Docker-implementation tests
(DockerBroker, DockerGateway, backend.docker.*) — they no-op under a
non-Docker run rather than testing internals that run doesn't target.
- Retires tests/_docker.py; the KVM job no longer needs SKIP_DOCKER_TESTS
to steer Docker-only classes.
Task 2 — explicit per-backend skip visibility:
- tests/backend_preflight.py prints a clear PASS/FAIL capability line and
exits non-zero when the selected backend is missing.
- Both integration jobs run it as a preflight, so absent infrastructure
is surfaced at the job level instead of hidden among unittest.skip
lines. The docker job replaces its soft "Show environment" step; the
firecracker job keeps its richer backend-status check.
Docs (tests/README.md, docs/ci.md) updated; unit coverage for the probes,
guards, and preflight in test_backend_skip_guards.py.
Closes#414
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remote terminal is an orchestrator operation, not a data-plane one: the
agent PTY is exec_agent-sourced (ssh -t / container exec --tty / docker
exec), which the orchestrator owns — the gateway never holds a PTY. A
--remote flag publishes a running agent's session; local + remote clients
share it via a session multiplexer (one PTY, many clients),
operator-role-gated, so the split's "one authenticated remote door" holds
for terminals too. Detailed design (multiplexer location, input
arbitration, per-client rendering, always-on vs --remote) tracked in #478.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
v1 decision: agent-image builds run in the orchestrator (the control-plane
side already owns launches and hosts builds in today's combined unit; the
data-plane gateway must not build). Consequence recorded in the memory
section: the orchestrator keeps the ~4 GB buildah ceiling and the gateway
is the slim unit (~1 GB), so the split is ~memory-neutral-to-+1 GB for now
— the ~1.5 GB win waits on a later move to a dedicated slim build unit
(#468-adjacent). Also flags the tension of co-locating buildah with the
control plane. Removed the corresponding open question.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Now that #469 got the DB off the data plane (removing the cross-kernel
SQLite-coherence reason the planes were consolidated), separating them
into distinct runtimes is possible again. Adds a Design subsection to
0070 covering:
* Access — the only hop that changes is gateway->orchestrator (loopback
-> network). A dedicated `bot-bottle-orchestrator` --internal control
network with a dual-homed gateway gives Docker/macOS the L3 agent
isolation that only Firecracker's nft currently has (both are
token-only today); Firecracker gets it nearly free (agents already
can't reach non-DNAT'd ports).
* Memory — firecracker's mem_size_mib is a fixed ceiling (no reclaim
without a balloon); the 4 GB combined VM is really buildah headroom.
Right-sized: ~512 MiB orchestrator + ~1 GB gateway < today's 4 GB,
if agent-image builds move off the gateway.
* iroh belongs in the orchestrator (control-plane remote entry point);
keep the data-plane gateway slim and P2P-free.
Plus open questions: where agent-image builds run post-split, and whether
agent-PTY-to-mobile relays through the orchestrator's iroh endpoint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings in the two commits main gained: #472 (CLI cleanup — remove `info`,
split `list active` into a top-level `active`, simplify `list`) and the
terminology glossary. #472 was written against the old flat cli/ layout,
so its semantics are reconciled into the reorg'd cli/commands/ structure:
* new `active` command lives at cli/commands/active.py, registered in
the lazy registry and listed in `help` (not left at the flat path).
* `info` removed: deleted cli/commands/info.py, dropped from the
registry and `help`.
* `list` simplified to list-available-only, using the reorg's
os.getcwd()/constants.PROG conventions.
* the `list active` -> `active` doc wording applied to the backend
docstrings that our split moved into base.py / selection.py.
Our reorg wins for the fully-rewritten dispatcher (cli/__init__ shim,
backend facade). pyright: 0 errors. Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Defines Agent Provider, Agent Runtime, Agent/Agent Definition,
Bottle/Bottle Definition, Sealed Bottle, Bottled Agent, and Active Bottle.
Links from docs/README.md and AGENTS.md for discoverability.
Closes#474
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PRD 0070's rule — only the orchestrator opens bot-bottle.db; the data
plane reaches state through the control-plane RPC — was not in force.
Three data-plane daemons held a direct read-write handle on the shared
SQLite file: the supervise MCP server, the egress DLP addon (the most
attack-exposed process, TLS-bumping hostile traffic), and the git-gate
pre-receive hook. An RCE in any of them could read every bottle's
plaintext identity_token and forge attribution fleet-wide (issue #469).
Add the agent half of the supervise flow to the control plane:
POST /supervise/propose -> queue a proposal, 201 {proposal_id}
POST /supervise/poll -> non-blocking decision poll, 200 {status,...}
Both attribute the caller by (source_ip, identity_token) exactly like
/resolve — never a caller-supplied slug — so a bottle can only ever queue
or read its own proposals even if the data plane is compromised. A decided
poll archives server-side, preserving the archive-after-read contract.
Data plane: the supervise server, egress addon, and git-gate hook now
queue/poll through PolicyResolver.propose_supervise / poll_supervise
instead of opening the DB. supervise_server keeps its ~30s grace window
by polling the RPC; egress keeps its safelist keyed by resolved bottle;
the git-gate hook gets (source_ip, identity_token) from the CGI env.
Packaging: drop the DB bind-mount and SUPERVISE_DB_PATH from the
data-plane containers/VMs (docker gateway + infra, macOS infra,
firecracker infra). The orchestrator remains the sole opener of the one
file via BOT_BOTTLE_ROOT / host_db_path().
Update PRD 0070: the rule is now in force; remove the transitional caveat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
didericis
didericis-codex
didericis-claude
didericis-claude