ci(coverage): run the diff-coverage gate on a KVM runner
The Firecracker VM/SSH orchestration (launch/boot/SSH/isolation-probe, ~230 lines) is covered by the integration suite, which needs /dev/kvm + the provisioned pool — a container runner skips it, so those lines read uncovered and the 90% diff gate can't pass there (it's been red since the backend landed). Move the `coverage` job to a self-hosted `kvm` runner with a firecracker-readiness preflight (binary + /dev/kvm + `backend status`), so the integration test actually runs and the orchestration is covered. Unit/lint stay on ubuntu-latest. README documents the runner prerequisites. Also close the last unit-coverable gaps (bottle exec/close, bottle_plan properties, docker status/setup branches, netpool span/conflict) so the gate clears 90% (90.3%) with the firecracker integration test running. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
+28
-14
@@ -71,28 +71,42 @@ jobs:
|
||||
- name: Run integration tests
|
||||
run: python3 -m unittest discover -t . -s tests/integration -v
|
||||
|
||||
# Combined unit+integration coverage + the diff-coverage gate.
|
||||
# See docs/decisions/0004-coverage-policy.md. The hard gate is diff
|
||||
# coverage (new/changed lines >= 90%); the combined + critical reports
|
||||
# are informational and degrade gracefully when the runner has no
|
||||
# Docker (integration tests skip, those modules just read lower).
|
||||
# Combined unit+integration coverage + the diff-coverage gate (the hard
|
||||
# gate: new/changed lines >= 90%). See docs/decisions/0004-coverage-policy.md.
|
||||
#
|
||||
# This runs on a self-hosted KVM runner (label `kvm`), NOT ubuntu-latest,
|
||||
# because the Firecracker backend's subprocess/VM orchestration
|
||||
# (launch/boot/SSH/isolation-probe) is covered by the integration suite,
|
||||
# and that suite needs `/dev/kvm` + the provisioned TAP/nft pool — which a
|
||||
# container-based runner doesn't have. On such a runner the firecracker
|
||||
# integration test skips and its ~230 orchestration lines read as
|
||||
# uncovered, so the gate can't pass there.
|
||||
#
|
||||
# Runner prerequisites (provision once on the host; see the README
|
||||
# "Firecracker on Linux" section): the `firecracker` binary on PATH,
|
||||
# `/dev/kvm` accessible to the runner user, Docker, the cached guest
|
||||
# kernel + static dropbear (BOT_BOTTLE_FC_KERNEL / BOT_BOTTLE_FC_DROPBEAR),
|
||||
# and the network pool installed as the persistent systemd unit
|
||||
# (`./cli.py backend setup --backend=firecracker`). The preflight step
|
||||
# below fails fast with instructions if anything is missing.
|
||||
coverage:
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, kvm]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Preflight — Firecracker host is ready
|
||||
run: |
|
||||
command -v firecracker >/dev/null || {
|
||||
echo "firecracker not on PATH — provision the runner (README: Firecracker on Linux)"; exit 1; }
|
||||
test -e /dev/kvm || { echo "/dev/kvm missing — KVM not available on this runner"; exit 1; }
|
||||
# `backend status` exits non-zero unless the TAP pool is up + no
|
||||
# range overlap; it prints the exact `backend setup` fix.
|
||||
python3 cli.py backend status --backend=firecracker
|
||||
|
||||
- name: Install dev requirements
|
||||
run: python3 -m pip install -r requirements-dev.txt
|
||||
|
||||
- name: Combined coverage (unit + integration)
|
||||
- name: Combined coverage (unit + integration, incl. firecracker)
|
||||
run: PYTHON=python3 bash scripts/coverage.sh critical
|
||||
|
||||
- name: Diff-coverage gate (changed lines >= 90%)
|
||||
|
||||
@@ -90,6 +90,8 @@ BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
|
||||
|
||||
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
|
||||
|
||||
> **CI:** the coverage gate (`.gitea/workflows/test.yml` → `coverage` job) runs on a self-hosted runner labelled `kvm`, because the Firecracker backend's VM/SSH orchestration is exercised only by the integration suite, which needs `/dev/kvm` + the provisioned pool (a container runner would skip it and read as uncovered). Provision that runner exactly like a normal Firecracker host — `firecracker` on `PATH`, `/dev/kvm`, Docker, the cached guest kernel + static dropbear, and the pool installed as the persistent systemd unit — then register it with the `kvm` label. The unit/lint jobs still run on `ubuntu-latest`.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
```
|
||||
|
||||
@@ -215,6 +215,32 @@ class TestDockerSetupStatus(unittest.TestCase):
|
||||
with patch.object(dk.shutil, "which", return_value=None):
|
||||
self.assertFalse(dk._daemon_reachable())
|
||||
|
||||
def test_status_reports_missing_docker(self):
|
||||
with patch.object(dk.shutil, "which", return_value=None):
|
||||
rc, out = _cap(dk.status)
|
||||
self.assertEqual(1, rc)
|
||||
self.assertIn("docker on PATH: NO", out)
|
||||
|
||||
def test_setup_missing_docker_prints_install(self):
|
||||
with patch.object(dk.shutil, "which", return_value=None):
|
||||
rc, out = _cap(dk.setup)
|
||||
self.assertEqual(1, rc)
|
||||
self.assertIn("docker.com", out)
|
||||
|
||||
|
||||
class TestNetpoolSpanConflict(unittest.TestCase):
|
||||
def test_pool_span_bounds(self):
|
||||
with patch.dict("os.environ", {"BOT_BOTTLE_FC_IP_BASE": "10.243.0.0",
|
||||
"BOT_BOTTLE_FC_POOL_SIZE": "8"}):
|
||||
lo, hi = netpool._pool_span()
|
||||
# base .. base + 2*8 - 1 (16 addresses)
|
||||
self.assertEqual(15, hi - lo)
|
||||
|
||||
def test_route_conflict_fields(self):
|
||||
c = netpool.RouteConflict(dst="10.243.0.0/24", dev="eth0")
|
||||
self.assertEqual("10.243.0.0/24", c.dst)
|
||||
self.assertEqual("eth0", c.dev)
|
||||
|
||||
|
||||
# --- macos-container ------------------------------------------------
|
||||
|
||||
|
||||
@@ -303,5 +303,74 @@ class TestBootArgs(unittest.TestCase):
|
||||
self.assertEqual("bbfc0", cfg["network-interfaces"][0]["host_dev_name"])
|
||||
|
||||
|
||||
class TestBottleExecClose(unittest.TestCase):
|
||||
def test_exec_runs_as_node_and_returns_result(self):
|
||||
from bot_bottle.backend.firecracker import bottle as bmod
|
||||
with patch.object(bmod.subprocess, "run",
|
||||
return_value=bmod.subprocess.CompletedProcess(
|
||||
[], 0, stdout="hi\n", stderr="")) as run:
|
||||
r = _bottle().exec("echo hi")
|
||||
self.assertEqual(0, r.returncode)
|
||||
self.assertEqual("hi\n", r.stdout)
|
||||
# runs `runuser -u node` over ssh, script piped on stdin.
|
||||
self.assertIn("runuser", run.call_args.args[0])
|
||||
self.assertEqual("echo hi", run.call_args.kwargs["input"])
|
||||
|
||||
def test_exec_respects_root_user(self):
|
||||
from bot_bottle.backend.firecracker import bottle as bmod
|
||||
with patch.object(bmod.subprocess, "run",
|
||||
return_value=bmod.subprocess.CompletedProcess([], 0,
|
||||
stdout="", stderr="")) as run:
|
||||
_bottle().exec("id", user="root")
|
||||
joined = " ".join(run.call_args.args[0])
|
||||
self.assertIn("runuser -u root", joined)
|
||||
|
||||
def test_close_is_idempotent_noop(self):
|
||||
b = _bottle()
|
||||
b.close()
|
||||
b.close() # must not raise
|
||||
|
||||
|
||||
class TestBottlePlanProperties(unittest.TestCase):
|
||||
def _plan(self, **overrides: Any):
|
||||
from bot_bottle.backend.firecracker.bottle_plan import FirecrackerBottlePlan
|
||||
ap = cast(Any, MagicMock())
|
||||
ap.instance_name = "bot-bottle-demo-x"
|
||||
ap.image = "bot-bottle-claude:latest"
|
||||
ap.dockerfile = "Dockerfile.claude"
|
||||
ap.prompt_file = Path("/stage/prompt.txt")
|
||||
ap.command = "claude"
|
||||
ap.prompt_mode = "append_file"
|
||||
ap.template = "claude"
|
||||
fields = dict(
|
||||
spec=cast(Any, MagicMock()), manifest=cast(Any, MagicMock()),
|
||||
stage_dir=Path("/stage"), git_gate_plan=cast(Any, MagicMock()),
|
||||
egress_plan=cast(Any, MagicMock()), supervise_plan=None,
|
||||
agent_provision=ap, slug="demo-x", forwarded_env={},
|
||||
)
|
||||
fields.update(overrides)
|
||||
return FirecrackerBottlePlan(**fields) # type: ignore[arg-type]
|
||||
|
||||
def test_provision_backed_properties(self):
|
||||
p = self._plan()
|
||||
self.assertEqual("bot-bottle-demo-x", p.container_name)
|
||||
self.assertEqual("bot-bottle-claude:latest", p.image)
|
||||
self.assertEqual("Dockerfile.claude", p.dockerfile_path)
|
||||
self.assertEqual(Path("/stage/prompt.txt"), p.prompt_file)
|
||||
self.assertEqual("claude", p.agent_command)
|
||||
self.assertEqual("append_file", p.agent_prompt_mode)
|
||||
self.assertEqual("claude", p.agent_provider_template)
|
||||
|
||||
def test_git_gate_insteadof_defaults(self):
|
||||
p = self._plan()
|
||||
self.assertEqual("git-gate", p.git_gate_insteadof_host)
|
||||
self.assertEqual("git", p.git_gate_insteadof_scheme)
|
||||
|
||||
def test_git_gate_insteadof_http_override(self):
|
||||
p = self._plan(agent_git_gate_url="http://10.243.0.0:9420/")
|
||||
self.assertEqual("10.243.0.0:9420", p.git_gate_insteadof_host)
|
||||
self.assertEqual("http", p.git_gate_insteadof_scheme)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user