From 39353cefce8e678bb51b4f520cd24470b775b150 Mon Sep 17 00:00:00 2001 From: didericis Date: Mon, 27 Jul 2026 10:47:09 -0400 Subject: [PATCH] fix: correct harness defects found running the matrix MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First full run on the KVM host surfaced three harness bugs and two stale image URLs; all fixed here. Harness bugs: - Liveness probe used `bot-bottle --version`, which the CLI does not implement (unknown args die non-zero) — so every SUCCESSFUL install was misreported as "no runnable entry point". Switched to `bot-bottle --help`, which exits 0 before any DB/migration/network work. - cmd_down never removed serial.log, so its rmdir failed and every run left an orphan scratch dir behind. Added serial.log to the cleanup. - The teardown trap was armed AFTER cmd_up, but cmd_up's wait_for_ssh can fail with QEMU already running (a guest that never opens SSH) — leaking the VM. Arm the trap before cmd_up. Stale image URLs: - Fedora 41 is EOL and 404s; bumped to Fedora 44 (44-1.7). - Alpine bumped to 3.21.7; its cloud images ship a .sha512 only (this verifier is sha256), so SUM_URL is now empty (skip) with a note. Validated on delphi (QEMU 11.0.2): ubuntu/fedora/arch pass BOTH test and test-ready. Alpine (cloud-init seed not applied → no SSH) and NixOS (no upstream cloud qcow2) remain blocked on image provisioning, documented in the research note as follow-ups. Co-Authored-By: Claude Opus 4.8 --- .../testing-clean-install-on-linux.md | 34 +++++++++++++++++++ scripts/linux-install-test.sh | 20 +++++++---- 2 files changed, 47 insertions(+), 7 deletions(-) diff --git a/docs/research/testing-clean-install-on-linux.md b/docs/research/testing-clean-install-on-linux.md index 80df9f0d..c46fc541 100644 --- a/docs/research/testing-clean-install-on-linux.md +++ b/docs/research/testing-clean-install-on-linux.md @@ -84,6 +84,40 @@ Each distro exercises a different corner of the installer: | **Alpine** | Alpine "cloud" (cloudinit) image | musl libc + BusyBox `sh` — the harshest POSIX-`sh` host for a `#!/bin/sh` installer. | | **NixOS** | `channels.nixos.org` OpenStack image | No FHS `~/.local` on PATH by default; `nix-env` user-profile prereqs; pipx laying a self-contained venv on a non-FHS host. | +### Validation run (2026-07-27) + +First full run on the delphi KVM host, QEMU 11.0.2: + +| Distro | `test` (bare) | `test-ready` (prepared) | +|---|---|---| +| Ubuntu 24.04 | ✅ declines at git gate | ✅ installs, doctor python+config green | +| Fedora 44 | ✅ declines at git gate | ✅ installs, doctor python+config green | +| Arch (latest) | ✅ declines at git gate | ✅ installs, doctor python+config green | +| Alpine 3.21 | ⚠️ blocked (boot) | ⚠️ blocked (boot) | +| NixOS 24.11 | ⚠️ blocked (no image) | ⚠️ blocked (no image) | + +On all three working distros the bare `test` sees `install.sh` exit 1 at the +git-for-git-specs gate (cloud images ship python3 but not git) — a sound +decline — and `test-ready` installs cleanly with `doctor` reporting a usable +python and config (backends all not-ready, as expected in a plain VM). + +**Two known gaps, both in image provisioning rather than the installer:** + +- **Alpine** boots to a `localhost login:` prompt but its cloud image does not + apply our NoCloud (`cloud-localds`) seed the way Ubuntu/Fedora/Arch do, so + the SSH key is never injected and `wait_for_ssh` times out. Needs an + Alpine-specific seed path (e.g. presenting the seed as a CD-ROM, or Alpine's + `tiny-cloud` expectations) before it can run. +- **NixOS** publishes no downloadable cloud qcow2 on `channels.nixos.org` or + the release bucket (its cloud images are Hydra-built AMIs). The table URL + 404s; a working NixOS cell needs a locally built image (`nixos-generators`, + baking the test key), which is a follow-up. + +The run also fixed real defects it surfaced: the pinned Fedora 41 was EOL/404 +(bumped to 44), and the liveness probe used `bot-bottle --version` — which the +CLI does not implement, so every successful install was misreported as failed +until it was switched to `bot-bottle --help`. + In `test-ready` the harness installs `python3 + git + pipx` first on each distro (install.sh installs none of them), so all five drive the recommended pipx path. `test` then removes that scaffolding and lets each distro's bare diff --git a/scripts/linux-install-test.sh b/scripts/linux-install-test.sh index 8f1e98b4..02b9babd 100755 --- a/scripts/linux-install-test.sh +++ b/scripts/linux-install-test.sh @@ -122,8 +122,8 @@ SUM_URL[ubuntu]="https://cloud-images.ubuntu.com/noble/current/SHA256SUMS" SSH_USER[ubuntu]="ubuntu" PREREQ[ubuntu]="sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y python3 git pipx" -IMAGE_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/41/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-41-1.4.x86_64.qcow2" -SUM_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/41/Cloud/x86_64/images/Fedora-Cloud-41-1.4-x86_64-CHECKSUM" +IMAGE_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/44/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-44-1.7.x86_64.qcow2" +SUM_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/44/Cloud/x86_64/images/Fedora-Cloud-44-1.7-x86_64-CHECKSUM" SSH_USER[fedora]="fedora" PREREQ[fedora]="sudo dnf install -y python3 git pipx" @@ -132,8 +132,8 @@ SUM_URL[arch]="https://geo.mirror.pkgbuild.com/images/latest/Arch-Linux-x86_64-c SSH_USER[arch]="arch" PREREQ[arch]="sudo pacman -Sy --noconfirm python git python-pipx" -IMAGE_URL[alpine]="https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.2-x86_64-bios-cloudinit-r0.qcow2" -SUM_URL[alpine]="https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.2-x86_64-bios-cloudinit-r0.qcow2.sha256" +IMAGE_URL[alpine]="https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.7-x86_64-bios-cloudinit-r0.qcow2" +SUM_URL[alpine]="" # Alpine cloud images ship .sha512 only; this verifier is sha256. Set BB_TEST_SKIP_VERIFY handling applies. SSH_USER[alpine]="alpine" PREREQ[alpine]="sudo apk add --no-cache python3 git pipx" @@ -404,7 +404,10 @@ entry_point_runnable() { guest ' for bb in "$HOME/.local/bin/bot-bottle" "$HOME/.bot-bottle/venv/bin/bot-bottle" "$(command -v bot-bottle 2>/dev/null)"; do [ -n "$bb" ] && [ -x "$bb" ] || continue - "$bb" --version >/dev/null 2>&1 && exit 0 + # --help exits 0 before any DB/migration/network work; it is the + # cheapest proof the package imports and the shim runs. (bot-bottle + # has no --version: an unknown arg would die non-zero.) + "$bb" --help >/dev/null 2>&1 && exit 0 done exit 1 ' >/dev/null 2>&1 @@ -543,7 +546,7 @@ cmd_down() { # Deleting the overlay + seed is the reset; the read-only base stays cached. rm -f "$(_overlay)" "$(_seed)" "$(_ssh_key)" "$(_ssh_key).pub" \ "$RUN_DIR/user-data" "$RUN_DIR/meta-data" \ - "$RUN_DIR/install.rc" "$RUN_DIR/install.log" + "$RUN_DIR/install.rc" "$RUN_DIR/install.log" "$(_serial)" # Only remove a scratch dir we created (leave a user-provided one alone). [ -n "${BB_TEST_RUN_DIR:-}" ] || rmdir "$RUN_DIR" 2>/dev/null || true rm -f "$(_marker)" @@ -595,9 +598,12 @@ _test_cycle() { IN_TEST=1 RUN_DIR="$(mktemp -d "${TMPDIR:-/tmp}/bb-install-test.$DISTRO.XXXXXX")" + # Arm teardown BEFORE cmd_up: its wait_for_ssh can fail after QEMU is + # already running (e.g. a guest that never opens SSH), and without the trap + # in place that would leak the VM. + trap _test_teardown EXIT INT TERM echo "== [1/$_STEPS] up ($DISTRO) ==" cmd_up - trap _test_teardown EXIT INT TERM local step=2 if [ "$with_prereqs" = 1 ]; then