docs(ci): document the assurance topology

README.md

@@ -75,7 +75,7 @@ On compatible macOS hosts, the default backend requires Apple's `container` CLI
 
 Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
 
-> **CI (macOS Apple Container):** the `integration-macos` job (`.gitea/workflows/test.yml`) runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container` on a self-hosted macOS runner labelled `macos`, because Apple Container needs the host virtualization framework and cannot run in a Linux container (so it can't reuse the `kvm` runner). Provision an Apple Silicon host with the `container` CLI on `PATH` and `container system status` running, then register the runner in **host mode** (not docker mode) with the `macos` label — `brew install gitea-runner` (the `act_runner` rename). Give it a Python ≥ 3.11 with `coverage` importable on the launchd service's `PATH` (a launchd service doesn't inherit your shell profile, so pin `node` and the Python env explicitly). The job is **advisory** — `workflow_dispatch` (manual) only, never triggered by push or PR — since a single laptop that sleeps/roams must not block merges or churn on every push to main; its coverage doesn't feed the gate. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1.
+> **CI (macOS Apple Container):** the advisory `integration-macos` job in `.gitea/workflows/pre-release-test.yml` runs only on manual dispatch. It targets a self-hosted host-mode runner labelled `macos`; Apple Container cannot run inside the Linux pull-request runner. Provision an Apple Silicon host with the `container` CLI running and Python ≥ 3.11 plus `coverage` on the launchd service's explicit `PATH`. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1. Its coverage is reported separately and never feeds the required pull-request gate.
 
 ### Containers inside a bottle
 
@@ -174,7 +174,7 @@ BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
 
 > **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
 
-> **CI:** the coverage gate (`.gitea/workflows/test.yml` → `coverage` job) runs on a self-hosted runner labelled `kvm`, because the Firecracker backend's VM/SSH orchestration is exercised only by the integration suite, which needs `/dev/kvm` + the provisioned pool (a container runner would skip it and read as uncovered). Provision that runner exactly like a normal Firecracker host — `firecracker` on `PATH`, `/dev/kvm`, the cached guest kernel + static dropbear, and the pool installed as the persistent systemd unit — then register it with the `kvm` label. A Docker-capable hosted job builds the candidate once; KVM tests boot those exact bytes, and a successful main run publishes them. The unit/lint jobs still run on `ubuntu-latest`.
+> **CI:** Firecracker integration runs in the manually dispatched `.gitea/workflows/pre-release-test.yml` on a self-hosted runner labelled `kvm`; privileged KVM hosts never execute unreviewed PR code automatically. Provision it like a normal Firecracker host: `firecracker` on `PATH`, `/dev/kvm`, the cached guest kernel and static dropbear, and the persistent TAP/nft pool. The required pull-request workflow runs unit plus the complete Docker integration suite on `ubuntu-latest`; see `docs/ci.md`.
 
 ```sh
 ./cli.py start <agent>   # builds the image on first run, drops you into claude

docs/ci.md

@@ -1,50 +1,49 @@
 # CI
 
-The test workflow lives at [`.gitea/workflows/test.yml`](../.gitea/workflows/test.yml).
+## Required pull-request gate
-It runs the unit suite plus one integration job per backend
-(`integration-docker`, `integration-firecracker`, `integration-macos`) on:
 
-- every push to a branch with an open pull request, and
+[`.gitea/workflows/test.yml`](../.gitea/workflows/test.yml) runs the unit
-- every push to `main`.
+suite, Docker integration suite, combined coverage report, and diff-coverage
+gate when tested package/build inputs change on a pull request or on `main`.
 
-`integration-macos` is the exception: it is **advisory**, running only on
+The Docker job preflights the backend before discovery. Gitea's `act_runner`
-`workflow_dispatch` (manual dispatch), never on push or pull requests. It targets the
+runs the job in a container with the host Docker socket, so the test process
-Apple Container backend on a self-hosted macOS runner (label `macos`,
+reaches host-published ports through its bridge gateway and uses named Docker
-registered in host mode — Apple Container can't run in a Linux container, so it
+volumes for orchestrator/CA state the host daemon must mount. The orchestrator
-can't reuse the `kvm` runner). A single non-redundant laptop must not be able
+runs the package baked into the image built from the checkout; it does not bind
-to block a PR merge, so the job stays out of the `coverage` job's `needs` and
+the job container's invisible workspace into a sibling container.
-its coverage never feeds the diff-coverage gate. Because the infra container is
-a singleton (`bot-bottle-mac-infra`), the job declares a `concurrency` group
-and tears the container down on exit; keep runner concurrency at 1. See the
-README "macOS Apple Container" CI note for runner provisioning.
 
-Each integration job selects its backend via `BOT_BOTTLE_BACKEND` and
+`scripts.unittest_gate` enforces the Docker job's contract: all 22 integration
-runs a **preflight** (`./cli.py backend status --backend=<name>`) that
+tests must execute and none may skip. This includes the real gateway-image,
-prints a clear per-check readiness summary and fails the job when the
+control-plane authentication, multitenant policy/token isolation,
-backend is missing — so absent infrastructure is visible at the job level
+sandbox-escape, and orphan-network tests. Backend skip decorators remain useful
-rather than hidden among per-test `unittest.skip` lines. The skip guards in
+for local runs, but the CI preflight plus execution-count gate prevents a
-[`tests/_backend.py`](../tests/_backend.py) gate on the same readiness
+missing backend or runner-topology regression from becoming a green job.
-check (`bot_bottle.backend.has_backend`): backend-agnostic tests use
-`skip_unless_selected_backend_available()` and run through whichever
-backend is selected (checking, e.g., Linux + `/dev/kvm` for Firecracker
-rather than unrelated Docker availability); Docker-implementation tests use
-`skip_unless_backend("docker")` and no-op under a non-Docker run.
 
-A small subset of integration tests skip when running specifically
+Combined unit + Docker coverage is informational globally. Two focused gates
-under Gitea Actions (`GITEA_ACTIONS=true`), because `act_runner` runs
+are enforced:
-the job inside a container with the host's `/var/run/docker.sock`
-mounted in. That topology breaks two assumptions those tests make:
 
-- networks created via the host daemon aren't always visible to a
+- changed executable Python lines must be at least 90% covered; and
-  same-process `docker network ls` call from inside the job container,
+- the validated critical security/logic core must remain at least 90% covered.
-  and
-- ports published by sibling containers land on the host's loopback,
-  not on the job container's `127.0.0.1` — so HTTP probes against
-  `http://127.0.0.1:<host_port>` from inside the job time out.
 
-The affected tests (`test_orphan_cleanup.test_create_and_remove`,
+## Privileged pre-release matrix
-`test_gateway_image.TestGatewayImage`) still run
+
-locally where the test process and Docker daemon share a host.
+[`.gitea/workflows/pre-release-test.yml`](../.gitea/workflows/pre-release-test.yml)
-Making them work in CI is a follow-up: either re-write them to
+is manually dispatched before a release. It repeats unit and Docker integration
-discover container IPs via `docker inspect`, or reconfigure the
+coverage, then runs:
-runner with host networking.
+
+- Firecracker integration on the self-hosted `kvm` runner; and
+- advisory Apple Container integration on the self-hosted `macos` runner.
+
+These privileged host-mode runners never execute unreviewed pull-request code
+automatically. Firecracker coverage is combined in the manual pre-release
+report; macOS reports advisory coverage in its own job. The macOS infra
+container is a singleton, so its job uses a concurrency group and always tears
+the service down.
+
+## Scheduled canary
+
+[`.gitea/workflows/canaries.yml`](../.gitea/workflows/canaries.yml) runs weekly
+and on manual dispatch. It verifies the pinned gitleaks release URL, checksum,
+archive shape, and executable. The same unittest execution gate requires at
+least one executed canary and rejects skips.

tests/README.md

@@ -20,10 +20,11 @@ tests/
     ...                             # many others; see unit/ directory
   integration/
     test_gateway_image.py
-    test_dry_run_plan.py
+    test_sandbox_escape.py
     test_orphan_cleanup.py
     ...
-  canaries/                         # opt-in; see below (currently empty)
+  canaries/
+    test_gitleaks_release.py        # opt-in upstream artifact check
 ```
 
 Classification falls out of the directory — no hand-maintained list to
@@ -43,24 +44,27 @@ Discovery is invoked with `-t .` (top-level dir = repo root) so the
 
 ## What the integration tests cover
 
-- `test_dry_run_plan.py` — `cli.py start --dry-run --format=json` emits
-  a structured plan that contains the resolved egress allowlist and
-  the bottle's runtime, and creates zero Docker resources.
 - `test_orphan_cleanup.py` — `network_remove` is idempotent against
   missing resources, so the EXIT trap can call it unconditionally.
 - `test_gateway_image.py` — builds Dockerfile.gateway and
   probes that gitleaks / mitmdump / supervise are all reachable
   inside the gateway image.
+- `test_orchestrator_docker_auth.py` — drives the real control-plane
+  container and verifies role-scoped authentication.
+- `test_multitenant_isolation.py` and `test_sandbox_escape.py` — exercise
+  token/allowlist separation and end-to-end escape attempts.
 
 ## Canaries
 
 `tests/canaries/` holds upstream-regression checks gated on
 `BOT_BOTTLE_RUN_CANARIES=1` and not part of the per-push suite.
-They're invoked by the scheduled `canaries` workflow. Currently
+They're invoked by the scheduled `canaries` workflow. The gitleaks canary
-no canaries are defined.
+downloads the exact release archive pinned by `Dockerfile.gateway`, verifies
+its architecture-specific checksum, and executes the binary.
 
 ```bash
-BOT_BOTTLE_RUN_CANARIES=1 python -m unittest discover -t . -s tests/canaries -v
+BOT_BOTTLE_RUN_CANARIES=1 python -m scripts.unittest_gate \
+  -t . -s tests/canaries -v --minimum-executed 1 --fail-on-skip
 ```
 
 ## What's NOT covered