10 Commits

Author	SHA1	Message	Date
didericis	0500ae5f4c	fix(harness): make the prereq step non-interactive ... The launchd fix worked. `container system start` now gets past "Testing access to container-apiserver" and "Verifying machine API server is running" — the two steps it could not reach before — so the XPC domain problem is solved. It then failed somewhere new: No default kernel configured. Error: failed to read user input Install the recommended default kernel from [...kata-static...]? [Y/n]: `container system start` PROMPTS for the default Linux kernel when given neither --enable-kernel-install nor --disable-kernel-install ("default: prompt user"). A throwaway account always hits that prompt, because the kernel lives in the per-user app root and a fresh home has none — and with no TTY the prompt dies immediately. Pass --enable-kernel-install. That is also the honest choice for what this variant claims: the backend cannot run a bottle without a guest kernel, so "prerequisites satisfied" has to include it. The cost is a kernel download per run, since the previous run's copy went with the deleted home; BB_TEST_KERNEL_INSTALL=0 switches to --disable-kernel-install when you only care that the service comes up. This also means the per-user prerequisite is not one step but two — start the service, install a kernel — which is worth knowing for anyone setting up a second account by hand. The rig's stub now models the prompt: an unflagged start fails the way the real CLI does. Verified that guard bites by removing the flag and watching the scenario fail, so a future edit cannot silently reintroduce an interactive prereq step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 10:00:49 -04:00
didericis	5614a56ccd	fix(harness): run as the user inside their launchd domain ... `test` passes. `test-ready` got as far as the service start and failed: Launching container-apiserver... Error: failed to get a response from apiserver: invalidState: "unauthorized request" which is the limitation this script's own header predicted — "may need a full launchd user session (`launchctl asuser`)". `container system start` registers com.apple.container.apiserver as a per-USER launchd agent and then talks to it over XPC. `launchctl list` confirms the shape: the apiserver and every container-network/runtime job are agents in the invoking user's domain. Under plain `sudo -u` the caller stays in root's bootstrap namespace, so the lookup crosses domains and the apiserver rejects it as unauthorized — the agent launched fine, the client just could not reach it. run_as_user now enters the target user's domain with `launchctl asuser <uid>`, which is also what a real user gets from Terminal, so it is the more faithful way to run everything here rather than a special case for the service start. A never-GUI-logged-in account may have no bootstrappable domain at all, so availability is probed once and cached, and everything falls back to plain `sudo -u` when it is missing. Without that fallback a missing domain would break `test` — which passes today and does not need a session — for a reason unrelated to what it tests. The rig covers that path explicitly. If the start still fails, the error now names the cause and the way out (log into the account once to get it a domain) instead of just relaying the CLI's message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:56:51 -04:00
didericis	71c0f8ed88	feat(harness): add test-ready, the with-prerequisites variant ... "Does the installer work" and "can a new user actually run a bottle" are different questions, and collapsing them is what made the previous run ambiguous. Split them into two cycles over the same throwaway account: test a macOS system WITHOUT the prerequisites set up for this user, which is the default state of every new account since the Apple `container` service is per-user. Asserts the install is sound; reports backend readiness without failing on it, because install.sh provides no backend and cannot regress one. test-ready the same system WITH them. Runs `container system start` for the throwaway user, then demands doctor go fully green, backend included — the end-to-end claim. The new `prereqs` step runs `container system start` rather than `bot-bottle backend setup --backend=macos-container`, because that subcommand does not start anything: it checks, then tells you to run `container system start` yourself (backend/macos_container/setup.py, "no privileged host setup required"). The harness runs what the product actually asks for. test-ready sets BB_TEST_REQUIRE_BACKEND, so it demands exactly what `test` merely reports. Both share one cycle function; the step count and the PASS claim are the only differences beyond the extra step, so neither variant can drift from the other's teardown or dirty-account guarantees. Rig covers the distinguishing case directly: identical host state where `test` passes on install soundness and `test-ready` starts the service and reaches a green backend, plus the service-start failure and an install failure under test-ready. 22 scenarios, all passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:51:35 -04:00
didericis	556217ae7b	fix(harness): judge the install separately from backend readiness ... The first run against the branch's own code confirmed the doctor fix — no traceback, the firecracker probes now report "TAP pool: 0/8" instead of dying on PermissionError — and then failed for a reason the installer cannot fix. The Apple `container` service is per-USER. `container system status` reports an appRoot under ~/Library/Application Support/com.apple.container/, and bbtest sees "container system service: NOT running" while the creating admin's is running. A brand-new account therefore has no backend until it runs `container system start` once, doctor rightly fails, and `test` would be permanently red for host state that install.sh neither creates nor can regress. So stop treating doctor's exit code as one verdict. `test` now fails when the install is broken — no entry point, doctor crashed, or doctor could not report a usable python and config dir — and reports backend readiness as a note. BB_TEST_REQUIRE_BACKEND=1 restores the strict behaviour. A traceback is checked for explicitly rather than inferred from the exit code, because those are the same value. The PermissionError bug exited non-zero exactly like a missing prerequisite does; only the traceback distinguishes a defect from an environment fact, and that distinction is the whole point of this change. The research doc's footprint table had the service as a host-wide launchd service that survives user deletion. It does not. The state lives in the home and goes with it, so the reset is more complete than claimed — but the corollary is that a fresh account cannot run a bottle until the service is started for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:45:57 -04:00
didericis	7461802b62	fix(harness): install the branch under test, and stop mangling snippets ... Two harness bugs, both of which made the last run lie about what it verified. The run installed main. install.sh's default spec is the repo's default branch, so `run` piped *this checkout's* installer into the throwaway user and then had it install a package that does not contain the branch's changes — which is why the doctor PermissionError fix appeared not to work: it was never in the code under test. The clone line said so plainly (commit 420184b, not this branch's HEAD) and I read past it. The spec now pins to the current branch, over https because the throwaway user has neither our SSH key nor read access to this mode-700 checkout. Since what it clones is whatever the remote has, a dirty tree or an unpushed branch now prints a warning rather than quietly testing something other than what is on disk. The doctor probe also died with "sh: -c: line 1: syntax error: unexpected end of file". `sudo -i` joins its argv into one string for the login shell's -c, so an argument's quoting is not preserved and a newline terminates the command. The multi-line snippet added in the previous commit could not survive that. run_as_user now feeds snippets on stdin to `sh -s`, which is the same trick install.sh already arrives by, and is immune to the joining. The install spec moves into the stream as an export for the same reason — as an argument it had the identical latent quoting bug, unnoticed only because the default spec has no shell metacharacters in it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:38:39 -04:00
didericis	5c12c6bf4e	fix(doctor): survive a PATH entry the user can't execute ... The install itself now works end to end on a fresh macOS account — venv built, package installed from git, entry point linked — and `doctor` then died with an unhandled traceback: PermissionError: [Errno 13] Permission denied: 'ip' netpool's probe helpers caught only FileNotFoundError. That is not the only way a probe binary can be unavailable: when a name on PATH exists but this user cannot execute it, exec fails with EACCES, and CPython reports that in preference to the ENOENT from the other PATH entries. So `except FileNotFoundError` misses it and the crash propagates all the way out of `doctor`. Reproduced directly: a mode-000 file named `ip` on PATH yields exactly the error above. _run_ok's docstring already stated the intent — treat an unavailable binary as failure rather than crashing — so this widens the catch to OSError to match what it says. Any OSError means the probe could not run, which for a fail-closed check is indistinguishable from "not present". The same narrow catch is fixed in overlapping_routes and in the two docker probes (compose ls, docker ps), which are the same shape and equally reachable. Deliberately not touched: the FileNotFoundError catches around file I/O in bottle_state and orchestrator/service, where the narrow exception is correct. The harness also required `bot-bottle` on PATH before running doctor, which could never be true: install.sh prints the PATH line rather than editing a shell profile, by design, so on a fresh account the entry point is installed and working but not on PATH. It now looks where the installer actually puts it (~/.local/bin, then the venv) and notes when it's running by absolute path. That was the harness failing a run for a reason the installer intends. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:38:39 -04:00
didericis	8d0782d1c7	fix(install): install into a private venv when pipx is absent ... The harness's second run hit the wall the first one predicted: a fresh account has no pipx, so install.sh fell to `pip install --user`, and every Python a Mac offers — Homebrew and python.org alike — is externally managed, so PEP 668 blocked it. That fallback was never a fallback on macOS; it was a dead end that printed instructions. Replace it with a venv at ~/.bot-bottle/venv (BOT_BOTTLE_VENV to move it), with the console script symlinked into ~/.local/bin. PEP 668 does not apply inside a venv, and venv is stdlib, so unlike pipx there is nothing to bootstrap first. pipx stays the preferred path when present, so anyone already managing their Python apps that way is unaffected — and the post-install PATH check now asks pipx for PIPX_BIN_DIR instead of assuming ~/.local/bin. Keeping the venv under ~/.bot-bottle rather than ~/.local/share means the whole footprint stays in one directory, which is what lets the throwaway-account teardown remain a complete reset. This removes the PEP 668 pre-flight and the sysconfig user-scheme lookup, both of which existed only to serve the --user path. Their tests go with them: * `detects_externally_managed_python` asserted the check that is now moot; replaced by one asserting pipx is still preferred when present. * `checks_pip_usable_before_fallback` pinned a pip probe that no longer runs; replaced by one asserting the venv's own pip does the install, since using the base interpreter's would install outside the venv. * `resolves_user_scripts_dir_not_hardcoded` and `macos_user_scheme_is_not_dot_local_bin` guarded the ~/Library/Python scripts-dir lookup. Nothing installs there now. The surviving "don't hardcode" concern is pipx's bin dir, which has its own test. Five tests are added for the new path: the venv fallback exists, no --user path survives, the venv is under the config dir, venv creation failure names python3-venv (Debian ships it separately), and the entry point is exposed outside the venv. Verified end to end in a sandbox HOME with a fresh-account PATH and no pipx: venv built, package installed, symlink created, `doctor` reached and green (python 3.14.5, macos-container ready), exit 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:38:20 -04:00
didericis	6793a09e2b	docs: correct two claims the live harness run disproved ... The PATH story was wrong in an instructive way. I said a Homebrew Python is on PATH only because of a shell-profile line; on this host /etc/paths.d/homebrew puts /opt/homebrew/bin on every login shell's PATH, fresh accounts included. The stub still wins, because path_helper appends /etc/paths.d/* *after* /etc/paths and /usr/bin is in the latter. Ordering, not absence, is what makes bare `python3` the 3.9.6 stub — which is also why the versioned `python3.14` candidate is the one that matched during the real run, rather than the /opt/homebrew/bin/python3 fallback I expected. The harness header also credited install.sh with writing a PATH line into the login shell. It does not write one. The reset argument is unaffected (such a line would live in the deleted home either way), but the claim was untrue. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:38:20 -04:00
didericis	d415581adc	feat: add a one-shot test cycle to the macOS install harness ... `test` chains up -> run -> status -> down, which is the loop you actually want when verifying a clean install. Three things make it more than a convenience wrapper: * It refuses to start against an existing account. A reused home is not a clean install, so testing one silently would defeat the harness. * It tears the account down from an EXIT/INT trap armed the moment the account exists, so a failed run — or a Ctrl-C mid-install — still leaves the machine clean. BB_TEST_KEEP=1 opts out to poke at a failure. * Its verdict is stricter than the installer's. install.sh exits 0 when it finishes but `doctor` reports unmet prerequisites, so "the installer succeeded" is not a useful assertion; `test` fails if the install fails, if bot-bottle never reached the new user's PATH, or if doctor is unhappy. That meant giving cmd_status a real exit status instead of swallowing doctor's. Also fixes two bugs in `run`'s installer staging, by removing the staging entirely and feeding install.sh in on stdin: * `mktemp /tmp/bb-install.XXXXXX.sh` did not do what it looks like. BSD mktemp only substitutes trailing Xs, so every run wrote the *same* predictable path, as root, mode 644, in a world-writable directory. * The cleanup only ran on the normal and failure returns, so an interrupted run leaked the file. Piping on stdin sidesteps both: root opens the redirect before sudo drops privileges, so the mode-700 home that motivated the staging is a non-issue, there is no file to leak, and `sh -s` matches the documented `curl … | sh` shape more closely than executing a staged copy did. The command-level `exit 1`s become `return 1` so the steps compose under the trap, and the "next, run this" hints are suppressed inside `test`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5	2026-07-27 09:38:20 -04:00
didericis-claude	27b9ba5247	feat: add macOS clean-install test harness ... Add scripts/macos-install-test.sh, a throwaway-user harness for exercising install.sh the way a brand-new user would on macOS, plus the research note that motivates the approach. The harness has up/run/status/down/deep-reset subcommands. Because install.sh writes only to the user home (pipx venv, ~/.bot-bottle, a PATH line) and never installs the backend, deleting the account is a complete, deterministic reset of the install surface. A disposable macOS VM can't stand in on M1/M2: the Apple `container` backend needs Virtualization.framework, and running it inside a guest VM requires nested virtualization (M3+ only), so a throwaway user is the only way to reach the real host backend from a clean $HOME. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-27 09:38:20 -04:00