bot-bottle/docs/research/testing-clean-install-on-linux.md

Testing a clean bot-bottle install on Linux

How do you exercise install.sh the way a brand-new user would — on a pristine Linux environment you can throw away afterward — without polluting your daily-driver host, and across the several package-management regimes Linux fragments into? This is the Linux counterpart to testing-clean-install-on-macos.md; the conclusion is different because Linux gives us a boundary macOS doesn't.

Summary

On macOS the honest options were a throwaway user or a VM, and the throwaway user won on pragmatics (nested virtualization is gated to M3+). On Linux the calculus flips: a disposable KVM virtual machine, booted from a distro cloud image and deleted per run, is both the cleanest boundary and the one that lets a single harness cover Ubuntu, Fedora, Arch, Alpine, and NixOS. The host already requires KVM for the Firecracker backend, so the VM is cheap here.

The harness lives at scripts/linux-install-test.sh. Per run it caches one read-only base image, boots a throwaway copy-on-write overlay (qemu-img create -b base), installs the distro's prerequisites, pipes this checkout's install.sh into the guest exactly as curl … | sh would, asserts the CLI installed, and deletes the overlay — the Linux equivalent of docker run --rm, for a whole machine.

Why a VM, not a container or a throwaway user

Mechanism	Why it's the wrong boundary here
Container (docker run --rm)	Shares the host kernel and ships a deliberately minimal userland — no systemd, a stubbed-out package manager story, and (crucially) it doesn't reproduce the externally-managed Python (PEP 668) that real desktop/server installs put in front of the user. It tests "does install.sh run in a container," not "does it run on a real distro."
Throwaway user (useradd/userdel)	The macOS pick, but weaker on Linux: it reaches the real host, yet every system package it installs (python, pipx, git via apt/dnf/…) stays behind, and it can only ever test the one distro the host runs. The whole Linux-specific value is the cross-distro matrix.
Disposable KVM VM (this harness)	A genuine kernel + userland + package-manager boundary that wipes to nothing on teardown, and swaps freely between distro cloud images. The one real cost — nested virtualization for the backend — doesn't apply, because we gate the installer, not the runtime (below).

Scope: installer correctness, not runtime

install.sh never installs a backend — it installs the bot-bottle Python package and runs doctor, which reports what's missing (install.sh header, bot_bottle/cli/commands/doctor.py). Inside the test VM there is no nested KVM or Docker, so every backend is correctly reported not-ready and doctor exits non-zero by design. That is expected and is not a test failure.

The harness's pass criterion is therefore not a green doctor but:

1. the distro prerequisites install,
2. install.sh exits 0,
3. the bot-bottle entry point is present on the fresh user's PATH (or at the pipx/pip location install.sh prints), and
4. bot-bottle --version runs — proving the package imports and the shim works.

doctor's full output is still printed every run, so a genuine installer regression (a broken shim, an import error, a botched PATH) is visible even though the backend line is red.

The distro matrix is the point

Each distro exercises a different corner of the installer:

Distro	Cloud image	What it stresses
Ubuntu (noble)	cloud-images.ubuntu.com	The common case; apt's pipx, externally-managed Python (PEP 668) → install.sh's pipx path.
Fedora	Fedora Cloud Base Generic	dnf packaging, a different default Python, BSD-style checksum file.
Arch	geo.mirror.pkgbuild.com/images/latest	Rolling / newest Python; python-pipx.
Alpine	Alpine "cloud" (cloudinit) image	musl libc + BusyBox sh — the harshest POSIX-sh host for a #!/bin/sh installer.
NixOS	channels.nixos.org OpenStack image	No FHS ~/.local on PATH by default; nix-env user-profile prereqs; pipx laying a self-contained venv on a non-FHS host.

Per the decision to have a clean install succeed everywhere, the harness installs python3 + git + pipx first on each distro (install.sh installs none of them), so all five drive the recommended pipx path rather than the PEP-668-blocked pip fallback. Exercising that fallback deliberately (skip the pipx prereq, expect the "externally managed" die message) is a natural future toggle but is out of scope here.

What a clean install touches (the footprint that decides "wipeable")

Artifact	Location	In the guest's $HOME?	Survives VM teardown?
Config / state / db	~/.bot-bottle/{agents,bottles,contrib,…} (install.sh)	✅	❌ overlay deleted
pipx venv + shim	~/.local/pipx/venvs/bot-bottle, shim in ~/.local/bin	✅	❌ overlay deleted
pip --user fallback	~/.local/lib + ~/.local/bin	✅	❌ overlay deleted
Distro prerequisites (python/git/pipx)	system paths via apt/dnf/pacman/apk/nix-env	❌	❌ overlay deleted

Unlike the macOS throwaway user (whose Homebrew / Apple-Container / Rosetta footprint survives), every row here dies with the overlay — that is the VM's whole advantage. The cached base image is read-only backing and is the only thing that persists between runs, on purpose.

Design notes baked into the harness

• User-mode networking (-netdev user,hostfwd=tcp:127.0.0.1:PORT-:22): no root, no bridge, no host network state touched. Only SSH is forwarded.
• cloud-init seed ISO (cloud-localds) injects an ephemeral SSH keypair and a passwordless-sudo login. The keypair is generated per run and deleted on teardown; the guest can't be logged into after it's gone.
• Copy-on-write overlay: the cached base is never mutated, so a corrupt or interrupted run can't poison the cache; downloads land at *.partial and are renamed only after checksum verification.
• Checksums: verified against each vendor's published sums file at download time (GNU hash file, bare-hash, and Fedora's BSD SHA256 (file) = hash formats are all handled). The NixOS channel image ships no stable sums file, so that one needs BB_TEST_SKIP_VERIFY=1.
• test-all runs each distro in its own subshell on its own forwarded port, so one distro's failure (or teardown trap) can't abort the matrix, and prints a per-distro PASS/FAIL summary.

Not wired into PR CI

Like the macOS harness, the runtime is host-specific (needs /dev/kvm, qemu, and cloud-localds) and is not exercised by the Linux pull-request runner. It is validated statically (bash -n, shellcheck) and run by hand on a KVM-capable host. The cloud-image URLs in the DISTRO table are the one place to bump when a distro cuts a newer build.