Compare commits

...

4 Commits

Author SHA1 Message Date
didericis-claude 4190082778 test(integration): lift GITEA_ACTIONS skip for Firecracker backend
lint / lint (push) Successful in 42s
test / coverage (pull_request) Has been skipped
test / integration (pull_request) Successful in 8s
test / unit (pull_request) Successful in 1m19s
The sandbox-escape test was unconditionally skipped when GITEA_ACTIONS=true,
which prevented Firecracker orchestration coverage from being measured even
when BOT_BOTTLE_BACKEND=firecracker is set on the KVM runner.

Narrow the skip to: GITEA_ACTIONS=true AND BOT_BOTTLE_BACKEND != firecracker.
When BOT_BOTTLE_BACKEND=firecracker the test is explicitly opted in to run on
the self-hosted KVM runner where the required /dev/kvm + TAP pool exist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-18 05:33:51 -04:00
didericis-claude 3f1f7cfc8f ci(coverage): address review findings from PR #349
test / coverage (pull_request) Has been skipped
test / integration (pull_request) Successful in 7s
test / unit (pull_request) Successful in 30s
- Finding 1: set BOT_BOTTLE_BACKEND=firecracker on the coverage step so
  the integration suite actually exercises the Firecracker orchestration
  paths rather than defaulting to Docker
- Finding 2: restrict the coverage job to push+workflow_dispatch only;
  PR-controlled code no longer executes on the privileged KVM runner
  automatically — maintainers trigger workflow_dispatch for trusted PRs
- Finding 3: expand path filters to include workflow files, scripts, and
  README so changes to CI configuration trigger the workflow itself

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-18 05:33:41 -04:00
didericis-claude 62474f2664 ci(coverage): install dev requirements on the KVM runner
lint / lint (push) Successful in 39s
The self-hosted KVM runner is a persistent machine, so
--break-system-packages is inappropriate. Use --user instead so
coverage (and pyright/pylint for future jobs) land in ~/.local
and survive between runs without touching the system Python.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-18 05:16:45 -04:00
didericis 25057bbb3a ci(coverage): run the diff-coverage gate on a self-hosted KVM runner
Re-land the coverage gate deferred from #343. The Firecracker VM/SSH
orchestration (~230 lines) is only exercised by the integration suite,
which needs /dev/kvm + the provisioned TAP/nft pool — a container runner
skips it and those lines read uncovered, so the 90% diff gate can't pass
on ubuntu-latest. Move the `coverage` job to a self-hosted `kvm` runner
with a firecracker-readiness preflight (binary + /dev/kvm + `backend
status`) so the integration test actually runs. Unit/lint stay on
ubuntu-latest. README documents the runner prerequisites.

Depends on a registered self-hosted runner labelled `kvm`; until one is
provisioned this gate will not run. See PRD 0069 / #348.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-18 05:16:45 -04:00
3 changed files with 59 additions and 22 deletions
+51 -17
View File
@@ -23,9 +23,16 @@ on:
- main - main
paths: paths:
- '**.py' - '**.py'
- '.gitea/workflows/**.yml'
- 'scripts/**'
- 'README.md'
pull_request: pull_request:
paths: paths:
- '**.py' - '**.py'
- '.gitea/workflows/**.yml'
- 'scripts/**'
- 'README.md'
workflow_dispatch:
jobs: jobs:
unit: unit:
@@ -68,30 +75,57 @@ jobs:
- name: Run integration tests - name: Run integration tests
run: python3 -m unittest discover -t . -s tests/integration -v run: python3 -m unittest discover -t . -s tests/integration -v
# Combined unit+integration coverage report (informational). See # Combined unit+integration coverage + the diff-coverage gate (the hard
# docs/decisions/0004-coverage-policy.md. # gate: new/changed lines >= 90%). See docs/decisions/0004-coverage-policy.md.
# #
# The hard diff-coverage gate (changed lines >= 90%) is DEFERRED: the # This runs on a self-hosted KVM runner (label `kvm`), NOT ubuntu-latest,
# Firecracker backend's VM/SSH orchestration is covered by the integration # because the Firecracker backend's subprocess/VM orchestration
# suite, which needs /dev/kvm + the provisioned TAP/nft pool — a # (launch/boot/SSH/isolation-probe) is covered by the integration suite,
# container-based runner skips it and those lines read uncovered, so the # and that suite needs `/dev/kvm` + the provisioned TAP/nft pool — which a
# gate can't pass here. Re-enabling it on a self-hosted KVM runner is # container-based runner doesn't have. On such a runner the firecracker
# tracked separately (see PRD 0069 / #348 and the ci-runner branch). # integration test skips and its ~230 orchestration lines read as
# uncovered, so the gate can't pass there.
#
# Runner prerequisites (provision once on the host; see the README
# "Firecracker on Linux" section): the `firecracker` binary on PATH,
# `/dev/kvm` accessible to the runner user, Docker, the cached guest
# kernel + static dropbear (BOT_BOTTLE_FC_KERNEL / BOT_BOTTLE_FC_DROPBEAR),
# and the network pool installed as the persistent systemd unit
# (`./cli.py backend setup --backend=firecracker`). The preflight step
# below fails fast with instructions if anything is missing.
#
# Security: this job executes PR-controlled code on a privileged runner
# (Docker, /dev/kvm, TAP/nft). It is restricted to push events (main
# branch) and manual workflow_dispatch by maintainers — it does NOT run
# on pull_request. Trusted PRs are validated by triggering workflow_dispatch
# on the PR branch before merging.
coverage: coverage:
runs-on: ubuntu-latest runs-on: [self-hosted, kvm]
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
# No actions/setup-python: the runner image already ships Python 3.12, - name: Preflight — Firecracker host is ready
# and older act_runner engines mishandle setup-python's PATH (coverage run: |
# lands in one interpreter, `python3` resolves to another). Install command -v firecracker >/dev/null || {
# straight into the ephemeral job container's system Python — echo "firecracker not on PATH — provision the runner (README: Firecracker on Linux)"; exit 1; }
# --break-system-packages is safe because the container is disposable. test -e /dev/kvm || { echo "/dev/kvm missing — KVM not available on this runner"; exit 1; }
- name: Install dev requirements # `backend status` exits non-zero unless the TAP pool is up + no
run: python3 -m pip install --break-system-packages -r requirements-dev.txt # range overlap; it prints the exact `backend setup` fix.
python3 cli.py backend status --backend=firecracker
- name: Combined coverage report (unit + integration) - name: Install dev requirements
run: python3 -m pip install --user -r requirements-dev.txt
- name: Combined coverage (unit + integration, incl. firecracker)
env:
BOT_BOTTLE_BACKEND: firecracker
run: PYTHON=python3 bash scripts/coverage.sh critical run: PYTHON=python3 bash scripts/coverage.sh critical
- name: Diff-coverage gate (changed lines >= 90%)
run: |
git fetch --no-tags origin main:refs/remotes/origin/main
python3 scripts/diff_coverage.py --base origin/main --min 90
+2
View File
@@ -90,6 +90,8 @@ BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`. > **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
> **CI:** the coverage gate (`.gitea/workflows/test.yml` → `coverage` job) runs on a self-hosted runner labelled `kvm`, because the Firecracker backend's VM/SSH orchestration is exercised only by the integration suite, which needs `/dev/kvm` + the provisioned pool (a container runner would skip it and read as uncovered). Provision that runner exactly like a normal Firecracker host — `firecracker` on `PATH`, `/dev/kvm`, Docker, the cached guest kernel + static dropbear, and the pool installed as the persistent systemd unit — then register it with the `kvm` label. The unit/lint jobs still run on `ubuntu-latest`.
```sh ```sh
./cli.py start <agent> # builds the image on first run, drops you into claude ./cli.py start <agent> # builds the image on first run, drops you into claude
``` ```
+6 -5
View File
@@ -69,11 +69,12 @@ _DUMMY_HOST_KEY = (
@skip_unless_docker() @skip_unless_docker()
@unittest.skipIf( @unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true", os.environ.get("GITEA_ACTIONS") == "true"
"skipped under act_runner: egress_tls_init uses a host bind mount " and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker",
"the runner container can't see, and the network topology hides " "skipped under act_runner unless BOT_BOTTLE_BACKEND=firecracker: "
"sibling-gateway visibility — same constraint as the other " "egress_tls_init uses a host bind mount the runner container can't "
"bottle-bringup integration tests", "see, and the network topology hides sibling-gateway visibility — "
"these constraints don't apply on the self-hosted KVM runner",
) )
class TestSandboxEscape(unittest.TestCase): class TestSandboxEscape(unittest.TestCase):
"""End-to-end attacks against a real bottle. The bottle stays """End-to-end attacks against a real bottle. The bottle stays