feat: add Linux clean-install test harness

Adds scripts/linux-install-test.sh, a throwaway-VM harness that exercises
install.sh the way a brand-new user would across a Linux distro matrix
(Ubuntu, Fedora, Arch, Alpine, NixOS), plus the research note
docs/research/testing-clean-install-on-linux.md that motivates the approach.

This is the Linux counterpart to the macOS clean-install harness. On Linux
the boundary of choice flips from a throwaway user account to a disposable
KVM VM: it's a genuine kernel + userland + package-manager boundary that
wipes to nothing on teardown, and a single harness can swap distro cloud
images to cover the several package-management regimes Linux fragments into.
The host already needs KVM for the Firecracker backend, so a per-run,
copy-on-write VM (qemu-img create -b base) is cheap here -- the equivalent
of `docker run --rm`, for a whole machine.

Per run the harness caches one read-only base image, boots a throwaway
overlay via QEMU/KVM with user-mode networking (no root, no bridge), injects
an ephemeral SSH key + passwordless login through a cloud-init seed ISO,
installs the distro's prerequisites (python3 + git + pipx), pipes THIS
checkout's install.sh into the guest exactly as `curl ... | sh` would, and
asserts the CLI installed cleanly. The overlay is deleted on teardown, so
even the OS-level prerequisites are wiped -- unlike a throwaway user, the
reset is total.

Subcommands mirror the macOS harness (up/run/status/down/test) plus test-all
(the matrix) and ssh (an interactive guest shell). test arms an EXIT/INT/TERM
trap the moment the VM exists, so a failure or Ctrl-C still tears it down.

Scope is installer correctness, not runtime: there is no nested KVM/Docker in
the VM, so `bot-bottle doctor` correctly reports every backend not-ready and
exits non-zero by design. The pass criterion is therefore install.sh exiting
0, the bot-bottle entry point being present on the fresh user's PATH, and
`bot-bottle --version` running -- not a green doctor. doctor's output is still
printed so a real installer regression (broken shim, import error) stays
visible.

Validated with `bash -n` and `shellcheck`. Runtime is host-only (needs
/dev/kvm, qemu, cloud-localds), so like the macOS harness it isn't exercised
by the Linux PR CI. The cloud-image URLs in the DISTRO table are the one
place to bump when a distro cuts a newer build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs/research/testing-clean-install-on-linux.md

@@ -0,0 +1,115 @@
+# Testing a clean bot-bottle install on Linux
+
+How do you exercise `install.sh` the way a brand-new user would — on a
+pristine Linux environment you can throw away afterward — *without*
+polluting your daily-driver host, and across the several package-management
+regimes Linux fragments into? This is the Linux counterpart to
+[`testing-clean-install-on-macos.md`](testing-clean-install-on-macos.md);
+the conclusion is different because Linux gives us a boundary macOS doesn't.
+
+## Summary
+
+On macOS the honest options were a throwaway user or a VM, and the throwaway
+user won on pragmatics (nested virtualization is gated to M3+). On Linux the
+calculus flips: a **disposable KVM virtual machine, booted from a distro
+cloud image and deleted per run, is both the cleanest boundary and the one
+that lets a single harness cover Ubuntu, Fedora, Arch, Alpine, and NixOS**.
+The host already requires KVM for the Firecracker backend, so the VM is cheap
+here.
+
+The harness lives at [`scripts/linux-install-test.sh`](../../scripts/linux-install-test.sh).
+Per run it caches one read-only base image, boots a throwaway copy-on-write
+overlay (`qemu-img create -b base`), installs the distro's prerequisites,
+pipes *this checkout's* `install.sh` into the guest exactly as `curl … | sh`
+would, asserts the CLI installed, and deletes the overlay — the Linux
+equivalent of `docker run --rm`, for a whole machine.
+
+## Why a VM, not a container or a throwaway user
+
+| Mechanism | Why it's the wrong boundary here |
+|---|---|
+| **Container** (`docker run --rm`) | Shares the host kernel and ships a deliberately minimal userland — no systemd, a stubbed-out package manager story, and (crucially) it doesn't reproduce the *externally-managed Python* (PEP 668) that real desktop/server installs put in front of the user. It tests "does install.sh run in a container," not "does it run on a real distro." |
+| **Throwaway user** (`useradd`/`userdel`) | The macOS pick, but weaker on Linux: it reaches the real host, yet every system package it installs (python, pipx, git via `apt`/`dnf`/…) stays behind, and it can only ever test the *one* distro the host runs. The whole Linux-specific value is the cross-distro matrix. |
+| **Disposable KVM VM** (this harness) | A genuine kernel + userland + package-manager boundary that wipes to nothing on teardown, and swaps freely between distro cloud images. The one real cost — nested virtualization for the *backend* — doesn't apply, because we gate the installer, not the runtime (below). |
+
+## Scope: installer correctness, not runtime
+
+`install.sh` never installs a backend — it installs the `bot-bottle` Python
+package and runs `doctor`, which *reports* what's missing
+([`install.sh`](../../install.sh) header,
+[`bot_bottle/cli/commands/doctor.py`](../../bot_bottle/cli/commands/doctor.py)).
+Inside the test VM there is no nested KVM or Docker, so **every backend is
+correctly reported not-ready and `doctor` exits non-zero by design**. That is
+expected and is *not* a test failure.
+
+The harness's pass criterion is therefore not a green doctor but:
+
+1. the distro prerequisites install,
+2. `install.sh` exits 0,
+3. the `bot-bottle` entry point is present on the fresh user's PATH (or at the
+   pipx/pip location install.sh prints), and
+4. `bot-bottle --version` runs — proving the package imports and the shim
+   works.
+
+`doctor`'s full output is still printed every run, so a genuine installer
+regression (a broken shim, an import error, a botched PATH) is visible even
+though the backend line is red.
+
+## The distro matrix is the point
+
+Each distro exercises a different corner of the installer:
+
+| Distro | Cloud image | What it stresses |
+|---|---|---|
+| **Ubuntu** (noble) | `cloud-images.ubuntu.com` | The common case; `apt`'s `pipx`, externally-managed Python (PEP 668) → install.sh's pipx path. |
+| **Fedora** | Fedora Cloud Base Generic | `dnf` packaging, a different default Python, BSD-style checksum file. |
+| **Arch** | `geo.mirror.pkgbuild.com/images/latest` | Rolling / newest Python; `python-pipx`. |
+| **Alpine** | Alpine "cloud" (cloudinit) image | musl libc + BusyBox `sh` — the harshest POSIX-`sh` host for a `#!/bin/sh` installer. |
+| **NixOS** | `channels.nixos.org` OpenStack image | No FHS `~/.local` on PATH by default; `nix-env` user-profile prereqs; pipx laying a self-contained venv on a non-FHS host. |
+
+Per the decision to have a clean install *succeed* everywhere, the harness
+installs `python3 + git + pipx` first on each distro (install.sh installs
+none of them), so all five drive the recommended pipx path rather than the
+PEP-668-blocked pip fallback. Exercising that fallback deliberately (skip the
+pipx prereq, expect the "externally managed" die message) is a natural future
+toggle but is out of scope here.
+
+## What a clean install touches (the footprint that decides "wipeable")
+
+| Artifact | Location | In the guest's `$HOME`? | Survives VM teardown? |
+|---|---|---|---|
+| Config / state / db | `~/.bot-bottle/{agents,bottles,contrib,…}` ([`install.sh`](../../install.sh)) | ✅ | ❌ overlay deleted |
+| pipx venv + shim | `~/.local/pipx/venvs/bot-bottle`, shim in `~/.local/bin` | ✅ | ❌ overlay deleted |
+| pip `--user` fallback | `~/.local/lib` + `~/.local/bin` | ✅ | ❌ overlay deleted |
+| **Distro prerequisites** (python/git/pipx) | system paths via `apt`/`dnf`/`pacman`/`apk`/`nix-env` | ❌ | ❌ **overlay deleted** |
+
+Unlike the macOS throwaway user (whose Homebrew / Apple-Container / Rosetta
+footprint *survives*), **every row here dies with the overlay** — that is the
+VM's whole advantage. The cached base image is read-only backing and is the
+only thing that persists between runs, on purpose.
+
+## Design notes baked into the harness
+
+- **User-mode networking** (`-netdev user,hostfwd=tcp:127.0.0.1:PORT-:22`):
+  no root, no bridge, no host network state touched. Only SSH is forwarded.
+- **cloud-init seed ISO** (`cloud-localds`) injects an ephemeral SSH keypair
+  and a passwordless-sudo login. The keypair is generated per run and deleted
+  on teardown; the guest can't be logged into after it's gone.
+- **Copy-on-write overlay**: the cached base is never mutated, so a corrupt or
+  interrupted run can't poison the cache; downloads land at `*.partial` and
+  are renamed only after checksum verification.
+- **Checksums**: verified against each vendor's published sums file at
+  download time (GNU `hash  file`, bare-hash, and Fedora's BSD
+  `SHA256 (file) = hash` formats are all handled). The NixOS channel image
+  ships no stable sums file, so that one needs `BB_TEST_SKIP_VERIFY=1`.
+- **`test-all`** runs each distro in its own subshell on its own forwarded
+  port, so one distro's failure (or teardown trap) can't abort the matrix,
+  and prints a per-distro PASS/FAIL summary.
+
+## Not wired into PR CI
+
+Like the macOS harness, the runtime is host-specific (needs `/dev/kvm`,
+`qemu`, and `cloud-localds`) and is not exercised by the Linux pull-request
+runner. It is validated statically (`bash -n`, `shellcheck`) and run by hand
+on a KVM-capable host. The cloud-image URLs in the `DISTRO` table are the one
+place to bump when a distro cuts a newer build.