fix: correct harness defects found running the matrix
First full run on the KVM host surfaced three harness bugs and two stale image URLs; all fixed here. Harness bugs: - Liveness probe used `bot-bottle --version`, which the CLI does not implement (unknown args die non-zero) — so every SUCCESSFUL install was misreported as "no runnable entry point". Switched to `bot-bottle --help`, which exits 0 before any DB/migration/network work. - cmd_down never removed serial.log, so its rmdir failed and every run left an orphan scratch dir behind. Added serial.log to the cleanup. - The teardown trap was armed AFTER cmd_up, but cmd_up's wait_for_ssh can fail with QEMU already running (a guest that never opens SSH) — leaking the VM. Arm the trap before cmd_up. Stale image URLs: - Fedora 41 is EOL and 404s; bumped to Fedora 44 (44-1.7). - Alpine bumped to 3.21.7; its cloud images ship a .sha512 only (this verifier is sha256), so SUM_URL is now empty (skip) with a note. Validated on delphi (QEMU 11.0.2): ubuntu/fedora/arch pass BOTH test and test-ready. Alpine (cloud-init seed not applied → no SSH) and NixOS (no upstream cloud qcow2) remain blocked on image provisioning, documented in the research note as follow-ups. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -84,6 +84,40 @@ Each distro exercises a different corner of the installer:
|
||||
| **Alpine** | Alpine "cloud" (cloudinit) image | musl libc + BusyBox `sh` — the harshest POSIX-`sh` host for a `#!/bin/sh` installer. |
|
||||
| **NixOS** | `channels.nixos.org` OpenStack image | No FHS `~/.local` on PATH by default; `nix-env` user-profile prereqs; pipx laying a self-contained venv on a non-FHS host. |
|
||||
|
||||
### Validation run (2026-07-27)
|
||||
|
||||
First full run on the delphi KVM host, QEMU 11.0.2:
|
||||
|
||||
| Distro | `test` (bare) | `test-ready` (prepared) |
|
||||
|---|---|---|
|
||||
| Ubuntu 24.04 | ✅ declines at git gate | ✅ installs, doctor python+config green |
|
||||
| Fedora 44 | ✅ declines at git gate | ✅ installs, doctor python+config green |
|
||||
| Arch (latest) | ✅ declines at git gate | ✅ installs, doctor python+config green |
|
||||
| Alpine 3.21 | ⚠️ blocked (boot) | ⚠️ blocked (boot) |
|
||||
| NixOS 24.11 | ⚠️ blocked (no image) | ⚠️ blocked (no image) |
|
||||
|
||||
On all three working distros the bare `test` sees `install.sh` exit 1 at the
|
||||
git-for-git-specs gate (cloud images ship python3 but not git) — a sound
|
||||
decline — and `test-ready` installs cleanly with `doctor` reporting a usable
|
||||
python and config (backends all not-ready, as expected in a plain VM).
|
||||
|
||||
**Two known gaps, both in image provisioning rather than the installer:**
|
||||
|
||||
- **Alpine** boots to a `localhost login:` prompt but its cloud image does not
|
||||
apply our NoCloud (`cloud-localds`) seed the way Ubuntu/Fedora/Arch do, so
|
||||
the SSH key is never injected and `wait_for_ssh` times out. Needs an
|
||||
Alpine-specific seed path (e.g. presenting the seed as a CD-ROM, or Alpine's
|
||||
`tiny-cloud` expectations) before it can run.
|
||||
- **NixOS** publishes no downloadable cloud qcow2 on `channels.nixos.org` or
|
||||
the release bucket (its cloud images are Hydra-built AMIs). The table URL
|
||||
404s; a working NixOS cell needs a locally built image (`nixos-generators`,
|
||||
baking the test key), which is a follow-up.
|
||||
|
||||
The run also fixed real defects it surfaced: the pinned Fedora 41 was EOL/404
|
||||
(bumped to 44), and the liveness probe used `bot-bottle --version` — which the
|
||||
CLI does not implement, so every successful install was misreported as failed
|
||||
until it was switched to `bot-bottle --help`.
|
||||
|
||||
In `test-ready` the harness installs `python3 + git + pipx` first on each
|
||||
distro (install.sh installs none of them), so all five drive the recommended
|
||||
pipx path. `test` then removes that scaffolding and lets each distro's bare
|
||||
|
||||
@@ -122,8 +122,8 @@ SUM_URL[ubuntu]="https://cloud-images.ubuntu.com/noble/current/SHA256SUMS"
|
||||
SSH_USER[ubuntu]="ubuntu"
|
||||
PREREQ[ubuntu]="sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y python3 git pipx"
|
||||
|
||||
IMAGE_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/41/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-41-1.4.x86_64.qcow2"
|
||||
SUM_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/41/Cloud/x86_64/images/Fedora-Cloud-41-1.4-x86_64-CHECKSUM"
|
||||
IMAGE_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/44/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-44-1.7.x86_64.qcow2"
|
||||
SUM_URL[fedora]="https://download.fedoraproject.org/pub/fedora/linux/releases/44/Cloud/x86_64/images/Fedora-Cloud-44-1.7-x86_64-CHECKSUM"
|
||||
SSH_USER[fedora]="fedora"
|
||||
PREREQ[fedora]="sudo dnf install -y python3 git pipx"
|
||||
|
||||
@@ -132,8 +132,8 @@ SUM_URL[arch]="https://geo.mirror.pkgbuild.com/images/latest/Arch-Linux-x86_64-c
|
||||
SSH_USER[arch]="arch"
|
||||
PREREQ[arch]="sudo pacman -Sy --noconfirm python git python-pipx"
|
||||
|
||||
IMAGE_URL[alpine]="https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.2-x86_64-bios-cloudinit-r0.qcow2"
|
||||
SUM_URL[alpine]="https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.2-x86_64-bios-cloudinit-r0.qcow2.sha256"
|
||||
IMAGE_URL[alpine]="https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/cloud/generic_alpine-3.21.7-x86_64-bios-cloudinit-r0.qcow2"
|
||||
SUM_URL[alpine]="" # Alpine cloud images ship .sha512 only; this verifier is sha256. Set BB_TEST_SKIP_VERIFY handling applies.
|
||||
SSH_USER[alpine]="alpine"
|
||||
PREREQ[alpine]="sudo apk add --no-cache python3 git pipx"
|
||||
|
||||
@@ -404,7 +404,10 @@ entry_point_runnable() {
|
||||
guest '
|
||||
for bb in "$HOME/.local/bin/bot-bottle" "$HOME/.bot-bottle/venv/bin/bot-bottle" "$(command -v bot-bottle 2>/dev/null)"; do
|
||||
[ -n "$bb" ] && [ -x "$bb" ] || continue
|
||||
"$bb" --version >/dev/null 2>&1 && exit 0
|
||||
# --help exits 0 before any DB/migration/network work; it is the
|
||||
# cheapest proof the package imports and the shim runs. (bot-bottle
|
||||
# has no --version: an unknown arg would die non-zero.)
|
||||
"$bb" --help >/dev/null 2>&1 && exit 0
|
||||
done
|
||||
exit 1
|
||||
' >/dev/null 2>&1
|
||||
@@ -543,7 +546,7 @@ cmd_down() {
|
||||
# Deleting the overlay + seed is the reset; the read-only base stays cached.
|
||||
rm -f "$(_overlay)" "$(_seed)" "$(_ssh_key)" "$(_ssh_key).pub" \
|
||||
"$RUN_DIR/user-data" "$RUN_DIR/meta-data" \
|
||||
"$RUN_DIR/install.rc" "$RUN_DIR/install.log"
|
||||
"$RUN_DIR/install.rc" "$RUN_DIR/install.log" "$(_serial)"
|
||||
# Only remove a scratch dir we created (leave a user-provided one alone).
|
||||
[ -n "${BB_TEST_RUN_DIR:-}" ] || rmdir "$RUN_DIR" 2>/dev/null || true
|
||||
rm -f "$(_marker)"
|
||||
@@ -595,9 +598,12 @@ _test_cycle() {
|
||||
IN_TEST=1
|
||||
RUN_DIR="$(mktemp -d "${TMPDIR:-/tmp}/bb-install-test.$DISTRO.XXXXXX")"
|
||||
|
||||
# Arm teardown BEFORE cmd_up: its wait_for_ssh can fail after QEMU is
|
||||
# already running (e.g. a guest that never opens SSH), and without the trap
|
||||
# in place that would leak the VM.
|
||||
trap _test_teardown EXIT INT TERM
|
||||
echo "== [1/$_STEPS] up ($DISTRO) =="
|
||||
cmd_up
|
||||
trap _test_teardown EXIT INT TERM
|
||||
|
||||
local step=2
|
||||
if [ "$with_prereqs" = 1 ]; then
|
||||
|
||||
Reference in New Issue
Block a user