The require-numbered-prds gate previously ran on every pull request.
Scope its trigger to PRs whose base branch is main, so numbering is
only enforced at the point of merging into main.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A control-plane failure (e.g. the orchestrator becoming unreachable at
bottle registration) is undiagnosable from CI as-is: the `if: always()`
teardown runs `MacosInfraService().stop()`, which deletes the
orchestrator and gateway containers — and their logs — on every run.
Add an `if: failure()` step that dumps `container ls`/`network ls`, plus
`inspect` and `logs` for `bot-bottle-mac-orchestrator` and
`bot-bottle-mac-infra`, ordered before the teardown so the evidence is
captured while the containers still exist. Best-effort (never fails the
job; tolerates an already-removed container).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the advisory macOS Apple Container job run on workflow_dispatch
only, removing the push-to-`main` trigger so a single non-redundant
laptop never runs unattended on every push. Updates the job comment,
docs/ci.md, README CI note, and the PRD accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an `integration-macos` job that runs the integration suite against
BOT_BOTTLE_BACKEND=macos-container on a self-hosted host-mode macOS runner
(label `macos`), plus the PRD and provisioning docs.
The job is advisory (push-to-main + workflow_dispatch only, never PRs, not
in coverage.needs) since it targets a single non-redundant laptop. It
preflights `container`/`backend status` so a misprovisioned runner fails
loudly, serializes on a concurrency group, and tears down the
`bot-bottle-mac-infra` singleton on exit (#425).
Also relaxes the TestSandboxEscape CI skip guard: it skipped every backend
but firecracker under GITEA_ACTIONS, which would also skip on a host-mode
macOS runner. The guard's real target is the containerized act_runner, so
it now allows both host-mode backends (firecracker, macos-container)
through — otherwise the macOS job would go green while skipping the one
end-to-end test that proves the backend launches.
Closes#426
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review feedback: the hand-rolled `/dev/kvm` + docker-daemon capability
probes duplicated logic the CLI already owns. Delegate instead.
- tests/_backend.py skip guards now gate on `bot_bottle.backend.has_backend`
(each backend's `is_available()` classmethod — the same probe behind
`./cli.py backend status`), dropping the bespoke `Capability` probes.
- Remove tests/backend_preflight.py; the docker integration job runs
`./cli.py backend status --backend=docker` as its preflight (clear
per-check summary, non-zero exit when unready), matching the firecracker
job. The firecracker preflight reverts to its original binary/KVM checks
(backend status doesn't cover those).
- Unit test + docs updated to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Integration tests now select their backend from BOT_BOTTLE_BACKEND and
skip on the capability that backend actually needs, instead of gating
every backend on unrelated Docker availability.
Task 1 — backend-agnostic guards (tests/_backend.py):
- Capability probes: docker_capability() (reachable daemon) and
firecracker_capability() (accessible /dev/kvm + firecracker on PATH,
Docker-independent). backend_capability()/selected_backend() resolve
the target from BOT_BOTTLE_BACKEND (default docker).
- skip_unless_selected_backend_available() for backend-agnostic tests
(test_sandbox_escape) — runs through whichever backend is selected and
checks that backend's real capability.
- skip_unless_backend("docker") for Docker-implementation tests
(DockerBroker, DockerGateway, backend.docker.*) — they no-op under a
non-Docker run rather than testing internals that run doesn't target.
- Retires tests/_docker.py; the KVM job no longer needs SKIP_DOCKER_TESTS
to steer Docker-only classes.
Task 2 — explicit per-backend skip visibility:
- tests/backend_preflight.py prints a clear PASS/FAIL capability line and
exits non-zero when the selected backend is missing.
- Both integration jobs run it as a preflight, so absent infrastructure
is surfaced at the job level instead of hidden among unittest.skip
lines. The docker job replaces its soft "Show environment" step; the
firecracker job keeps its richer backend-status check.
Docs (tests/README.md, docs/ci.md) updated; unit coverage for the probes,
guards, and preflight in test_backend_skip_guards.py.
Closes#414
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
upload-artifact@v3's glob silently skips hidden files, so uploading a
bare `.coverage.unit` logged "No files were found. No artifacts will be
uploaded" and registered nothing — the coverage job's download then 404'd
("List Artifacts failed: 404"). The coverage report step read the same
file fine, confirming it existed; only the leading dot broke the upload.
The old pipeline's cross-job artifacts (infra-candidate/, firecracker-
inputs) worked precisely because they were non-dotfiles.
Each test job now copies its .coverage.<suffix> to a non-dot
coverage-<suffix>.dat before upload (the cp also fails loudly if coverage
never wrote the file), and the coverage job renames them back to
.coverage.* before `coverage combine`.
coverage run's --data-file flag can be overridden or ignored in some
runner environments (Nix Python, older act-based runners). Switching to
the COVERAGE_FILE env var with an absolute ${{ github.workspace }} path
ensures coverage.py writes to a known location in every runner context,
so upload-artifact can find the file.
Also adds --reuse-published to the infra build step: if the artifact for
this content hash already exists in the registry, download it instead of
running the full docker build → mke2fs → gzip pipeline.
The delphi-ci runner resolves relative paths in upload-artifact and
download-artifact from a different CWD than run: shell steps, so
'.coverage.unit' etc. were never found. Using ${{ github.workspace }}
gives an absolute path that does not depend on the JS action's CWD.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each test job now runs once under coverage and uploads a small .coverage.*
artifact. The coverage job combines them on ubuntu-latest — no test reruns,
no KVM dependency. The infra candidate is built directly on the KVM runner,
eliminating the build-infra job and the ~70 s upload + ~83 s combined
download. For PRs, no rootfs artifact is transferred at all. Main-branch
pushes upload the tested rootfs and matching dropbear so publish-infra
publishes the byte-identical artifact. relative_files = True in .coveragerc
lets coverage files from different runners combine without path remapping.
Closes#446
Review follow-ups for the infra candidate-artifact flow:
1. publish-infra would fail on the first merge to main. The rootfs version now
hashes the dropbear bytes, and build-infra sets BOT_BOTTLE_FC_DROPBEAR to the
staged dropbear — but publish-infra (ubuntu-latest) set none, so _publish_
bundle re-derived the version with a "<missing>" dropbear and rejected the
candidate as "does not match checkout". Download the same firecracker-inputs
dropbear and export BOT_BOTTLE_FC_DROPBEAR in publish-infra (and add
stage-firecracker-inputs to its needs, since it now consumes that artifact).
2. Guard stage-firecracker-inputs with the same fork-PR check as the other
KVM-runner jobs, so a fork PR can't spin the privileged runner (it only
copies a static binary, but keep the posture consistent; it gates the whole
Firecracker chain via needs).
3. ensure_artifact_gz / _publish_bundle read version.txt before checking it
exists — a bundle missing it raised a raw FileNotFoundError instead of the
intended "bundle is incomplete" die. Guard the read (kept before the gz/sha
completeness check so a wrong-version bundle still reports the version
mismatch, not "incomplete").
4. test_infra_artifact.setUp didn't isolate BOT_BOTTLE_INFRA_ARTIFACT_DIR, so
an ambient candidate dir (the coverage job exports one) would send the
registry-pull tests down the local-bundle path. Pin it off in setUp; the
candidate-path cases set it explicitly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
The test jobs were pulling a pre-published infra rootfs artifact from the
registry, which made CI dependent on an out-of-band publish step. Test suites
should be self-contained.
- Set BOT_BOTTLE_INFRA_BUILD=local in integration-firecracker and coverage
so both jobs build agent and infra images from the checked-out source.
- Add publish-infra.yml: a separate workflow that fires after the test
workflow succeeds on main, builds the same rootfs, and uploads it to the
Gitea generic-package registry for operators who want the pull path.
- Registry credentials (BOT_BOTTLE_INFRA_ARTIFACT_TOKEN) are only needed
by the post-merge publish job, never by test jobs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The Firecracker integration + coverage jobs pull a prebuilt infra rootfs
artifact (PRD 0069 Stage 2) versioned by a content hash of the Dockerfiles,
bot_bottle/, and the guest init. Building that artifact (publish_infra ->
docker build Dockerfile.gateway) has been broken since 5ad3449, so the
artifact was never published and the KVM runner's integration test 404'd on
the pull — the failure this branch surfaced once it stopped falsely skipping.
Two build-time bugs, both from 5ad3449, neither exercised since:
- pyproject.toml declared build-backend "setuptools.backends.legacy:build",
which is not an importable module; `pip install /src/` failed with
BackendUnavailable. Use the real backend, "setuptools.build_meta"
(the project has proper [project] metadata + flat-layout autodiscovery).
Not part of the artifact hash, so this alone doesn't move the version.
- Dockerfile.gateway wrote /app/egress_addon.py before /app existed (the
mkdir/WORKDIR came later), so the RUN redirect died with exit 2. Move
WORKDIR /app above the shim write (WORKDIR creates it) and drop the now
redundant later WORKDIR. This changes the gateway Dockerfile, so the infra
artifact version moves 3c9e7b23260992db -> 01e6aaa714756fce; the matching
artifact has been built and published to the generic package registry.
Also add Dockerfile* and pyproject.toml to test.yml's path filters: these
inputs determine what the firecracker jobs build/pull, so a change to them
must re-run the suite (and lets this push trigger a pull_request run).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9qa3xoavjQScufDfZaXKR
Gitea fires the pull_request push event as 'synchronize' (GitHub spec),
not 'synchronized'. The typo meant the workflow only ran on opened/
edited/reopened, leaving the required check yellow with no details link
after every commit push.
The self-hosted runner's Nix python env has no `pip` module, so
`python3 -m pip install -r requirements-dev.txt` failed with "No module
named pip" in both firecracker jobs. Neither job needs that install:
- integration-firecracker runs the stdlib `unittest` suite (no deps);
- coverage needs only `coverage`, which the runner's Nix python env
already ships (7.12.0) — verified `coverage run`/`coverage json` work.
pylint/pyright are lint.yml's concern, not test.yml's. The ubuntu-latest
`unit` job keeps its `--break-system-packages` install unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1qRZTJC6qgBsUSjNrBdkX
Add separate `integration-docker` and `integration-firecracker` jobs,
each with an explicit BOT_BOTTLE_BACKEND env var, so the backend used
is visible in CI output and skipped backends surface as a distinct job
rather than silent unittest.skip lines.
- integration-docker: ubuntu-latest, BOT_BOTTLE_BACKEND=docker
- integration-firecracker: [self-hosted, kvm], BOT_BOTTLE_BACKEND=firecracker,
same-repo PRs + push + workflow_dispatch only (untrusted fork PRs do
not execute on the privileged KVM runner)
- coverage: same same-repo restriction; refs #414 for the planned
follow-up that moves coverage to ubuntu-latest via artifact combination
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Finding 1: set BOT_BOTTLE_BACKEND=firecracker on the coverage step so
the integration suite actually exercises the Firecracker orchestration
paths rather than defaulting to Docker
- Finding 2: restrict the coverage job to push+workflow_dispatch only;
PR-controlled code no longer executes on the privileged KVM runner
automatically — maintainers trigger workflow_dispatch for trusted PRs
- Finding 3: expand path filters to include workflow files, scripts, and
README so changes to CI configuration trigger the workflow itself
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The self-hosted KVM runner is a persistent machine, so
--break-system-packages is inappropriate. Use --user instead so
coverage (and pyright/pylint for future jobs) land in ~/.local
and survive between runs without touching the system Python.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-land the coverage gate deferred from #343. The Firecracker VM/SSH
orchestration (~230 lines) is only exercised by the integration suite,
which needs /dev/kvm + the provisioned TAP/nft pool — a container runner
skips it and those lines read uncovered, so the 90% diff gate can't pass
on ubuntu-latest. Move the `coverage` job to a self-hosted `kvm` runner
with a firecracker-readiness preflight (binary + /dev/kvm + `backend
status`) so the integration test actually runs. Unit/lint stay on
ubuntu-latest. README documents the runner prerequisites.
Depends on a registered self-hosted runner labelled `kvm`; until one is
provisioned this gate will not run. See PRD 0069 / #348.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
Gitea Actions reports skipped jobs as a non-success status, which caused
label-issue to block PRs even though its if-condition correctly excluded it.
Two dedicated workflows eliminate the skipped-job problem entirely.
After merge, update the branch-protection required status context from
`tracker-policy / check-pr` to `tracker-policy-pr / check-pr`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Applies the same fix as test/lint to every workflow that still used
actions/setup-python, which the old act_runner engine mishandles:
- update-badges: was broken identically to lint — setup-python + pip
install hit the image's externally-managed system Python. Drop
setup-python, install with --break-system-packages, and use `python3`
(not bare `python`) for the coverage steps.
- canaries, prd-number: no pip install, so not failing, but they carried
the same fragile (and network-heavy) setup-python for stdlib-only work.
Removed — the image's system Python 3.12 runs them directly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
Same fix as the test workflow: the old act_runner engine mishandles
actions/setup-python's PATH, so `pip install` hit the image's
externally-managed system Python and failed with
"externally-managed-environment" on the "Install dev dependencies" step.
The runner image already ships Python 3.12 and the job container is
ephemeral, so drop setup-python and install straight into system Python
with --break-system-packages. pylint/pyright console scripts land on
/usr/local/bin (on PATH), so the lint steps still resolve. Also drops the
now-pointless `pip install --upgrade pip`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
The old act_runner engine (v0.2.13 on the delphi-ci runner) mishandles
actions/setup-python's PATH injection: pip installs coverage into the
toolcache interpreter while `python3` in later steps resolves back to the
image's system Python, so unit/coverage jobs failed with "No module named
coverage". Newer runners (TrueNAS's v0.6.1) don't, which is why it only
broke on delphi.
The runner-images/act container already ships Python 3.12, and the job
container is ephemeral, so drop setup-python entirely and install straight
into the system Python with --break-system-packages. Every step now uses
one interpreter consistently, on any runner version. Also removes the
redundant setup-python step from the integration job (stdlib-only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ
didericis-codex
didericis-claude
didericis