Files

T

didericis 8d0782d1c7 fix(install): install into a private venv when pipx is absent

The harness's second run hit the wall the first one predicted: a fresh account
has no pipx, so install.sh fell to `pip install --user`, and every Python a Mac
offers — Homebrew and python.org alike — is externally managed, so PEP 668
blocked it. That fallback was never a fallback on macOS; it was a dead end that
printed instructions.

Replace it with a venv at ~/.bot-bottle/venv (BOT_BOTTLE_VENV to move it),
with the console script symlinked into ~/.local/bin. PEP 668 does not apply
inside a venv, and venv is stdlib, so unlike pipx there is nothing to bootstrap
first. pipx stays the preferred path when present, so anyone already managing
their Python apps that way is unaffected — and the post-install PATH check now
asks pipx for PIPX_BIN_DIR instead of assuming ~/.local/bin.

Keeping the venv under ~/.bot-bottle rather than ~/.local/share means the whole
footprint stays in one directory, which is what lets the throwaway-account
teardown remain a complete reset.

This removes the PEP 668 pre-flight and the sysconfig user-scheme lookup, both
of which existed only to serve the --user path. Their tests go with them:

* `detects_externally_managed_python` asserted the check that is now moot;
  replaced by one asserting pipx is still preferred when present.
* `checks_pip_usable_before_fallback` pinned a pip probe that no longer runs;
  replaced by one asserting the venv's own pip does the install, since using
  the base interpreter's would install outside the venv.
* `resolves_user_scripts_dir_not_hardcoded` and
  `macos_user_scheme_is_not_dot_local_bin` guarded the ~/Library/Python
  scripts-dir lookup. Nothing installs there now. The surviving "don't
  hardcode" concern is pipx's bin dir, which has its own test.

Five tests are added for the new path: the venv fallback exists, no --user path
survives, the venv is under the config dir, venv creation failure names
python3-venv (Debian ships it separately), and the entry point is exposed
outside the venv.

Verified end to end in a sandbox HOME with a fresh-account PATH and no pipx:
venv built, package installed, symlink created, `doctor` reached and green
(python 3.14.5, macos-container ready), exit 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5

2026-07-27 09:38:20 -04:00

15 KiB

Raw Blame History

Testing a clean bot-bottle install on macOS

How do you exercise install.sh (and, ideally, a first bot-bottle start) the way a brand-new user would — on a pristine macOS environment you can throw away afterward — without permanently polluting your daily-driver Mac? The user's framing: is there a VM or boundary that avoids creating a separate account, or is spinning up and tearing down a throwaway macOS user on the CLI easy enough to just do that?

Summary

There is no lightweight, in-place macOS sandbox that hands you a clean home directory and wipeable system state without either a VM or a separate user account. sandbox-exec (Seatbelt) is deprecated and confines a process, not an environment; App Sandbox is for shipping apps, not for provisioning a fresh dev host. So the real choice is exactly the two the user named: a disposable macOS VM or a throwaway user account — and which one is right turns on a detail specific to this project.

bot-bottle's default macOS backend is Apple's container, which runs each container in its own lightweight VM via Virtualization.framework (README.md:27, apple-container-backend.md). That means a full end-to-end test — install and bot-bottle start — needs virtualization to work wherever bot-bottle runs. Inside a macOS guest VM that requires nested virtualization, which Apple gates to M3 or newer chips on macOS 15+. On M1/M2 you cannot run the Apple Container backend (or Docker Desktop, same reason) inside a macOS VM at all.

The recommendation splits on what you're testing and what silicon you have:

Install-script correctness only (does curl | sh → pipx → config dir → doctor's Python/config checks pass?): a disposable Tart VM is the cleanest boundary and works on any Apple Silicon Mac. doctor will report the backend as not-ready inside the VM on M1/M2, which is fine — you're testing the installer, not the runtime.
Full runtime (actually launch a bottle) on M3/M4: a disposable Tart VM from a golden base image, cloned per run is the gold standard — a genuine kernel/state boundary that wipes to nothing.
Full runtime on M1/M2, or when you'd rather not fight nested virt: a throwaway admin user via sysadminctl is the pragmatic pick. It tests the real backend because the backend runs on the host hypervisor — but it is a hygiene boundary, not a security one, and it does not clean the system-level footprint (see below).

Prefer the VM. Reach for the throwaway user only when nested virt is off the table and you accept an imperfect wipe.

Why "a boundary without a separate user" doesn't really exist on macOS

macOS has no namespace/overlay story like Linux unshare + tmpfs. The options that sound like in-place sandboxes don't fit:

Mechanism	Why it doesn't give you a clean, wipeable env
`sandbox-exec` / Seatbelt	Officially deprecated; confines one process's syscalls against a profile. It cannot present a fresh `$HOME` or a pristine `/usr/local`, and it won't let the Apple Container system service work.
App Sandbox	Entitlement-based confinement for signed `.app` bundles, not a provisioning tool for a CLI dev environment.
A second `$HOME` via `HOME=/tmp/foo`	Redirects only what honors `$HOME`. `install.sh` mostly does (it writes `~/.bot-bottle` and pipx/pip `--user` paths), but the Apple `container` install lands in `/usr/local` + a system service, and Homebrew lands in `/opt/homebrew` — all outside any `$HOME` you set. You'd get a false sense of "clean."
APFS snapshot rollback (`tmutil localsnapshot`)	You can't roll the live boot volume back to a local snapshot without booting to Recovery; it's not a per-run userspace undo.

So the honest answer to "is there some boundary that avoids a separate user?": yes — a VM — and it's the stronger boundary anyway. The only lighter-weight option is the separate user, with the caveats below.

What a clean install actually touches (the footprint that decides "wipeable")

Grounding the teardown story in what install.sh and the backend create:

Artifact	Location	In `$HOME`?	Survives user deletion?
Config / state / db	`~/.bot-bottle/{agents,bottles,contrib,state,db}` (`install.sh:80-83`, `bot_bottle/paths.py:59`)	✅	❌ removed with home
pipx venv + shim	`~/.local/pipx/venvs/bot-bottle`, shim in `~/.local/bin` (`install.sh:87-89`)	✅	❌ removed with home
private venv fallback (no pipx)	`~/.bot-bottle/venv` + symlink in `~/.local/bin` (`install.sh`)	✅	❌ removed with home
PATH / token exports	shell profile (`~/.zprofile`, etc.); `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` (`README.md:74`)	✅	❌ removed with home
Apple `container` install	`/usr/local/...` + notarized `.pkg` receipts	❌	✅ stays
Apple `container` system service	launchd system service (`container system start`)	❌	✅ stays
Homebrew (if used for `container`/python)	`/opt/homebrew`	❌	✅ stays
Rosetta 2 (needed for image builds)	system	❌	✅ stays

The three bold rows are the crux: deleting the throwaway user does not uninstall the Apple Container runtime, its system service, Homebrew, or Rosetta. A VM, by contrast, wipes 100% of the above by definition — that's its entire advantage for this task.

Option A — Disposable Tart VM (recommended)

Tart is a CLI-first macOS/Linux VM manager built on Virtualization.framework, purpose-built for exactly this "does it work on a clean macOS, without my settings/permissions/data" workflow. Keep one pristine golden image, clone a throwaway per run, delete it after.

brew install cirruslabs/cli/tart

# One-time: build a golden base (either a prebuilt image or a vanilla IPSW).
tart clone ghcr.io/cirruslabs/macos-tahoe-base:latest golden   # ~25 GB pull
# — or a truly vanilla install you click through once —
# tart create golden --from-ipsw latest --disk-size 60

# Per test run: clone → boot → test → destroy.
tart clone golden test-run
tart run test-run &
ssh admin@"$(tart ip test-run)"
#   inside the guest:
#   curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
#   bot-bottle doctor
tart stop test-run
tart delete test-run          # back to pristine; golden is untouched

Cloning is cheap (sparse files), so the golden image is your reset button — every tart clone is a fresh macOS. This is the closest thing to a Linux docker run --rm for a whole Mac.

The nested-virt caveat (read before relying on it for runtime tests). The Apple Container backend inside the guest needs Virtualization.framework to work inside the VM. Apple enables nested virtualization only on M3 or newer, on macOS 15 (Sequoia) or later; M2 and earlier are excluded by Apple, confirmed by Apple DTS. Consequences:

M3/M4 host: full runtime works in the guest. bot-bottle doctor reports the backend ready and start can launch a bottle. Gold standard.
M1/M2 host: the guest can install bot-bottle and pass the Python / config-dir checks, but doctor's backend check will fail and you cannot launch a bottle in the VM. Still perfectly good for testing the installer; not for the runtime.
M4-specific: a known bug blocks pre-Ventura guests on M4; use a current macOS guest (which you want anyway, since Apple container targets macOS 26 Tahoe).

UTM is the GUI equivalent on the same framework (and was first to expose nested virt) if you'd rather click; Tart wins for a scriptable spin-up/tear-down loop.

Option B — Throwaway user via `sysadminctl` (pragmatic fallback)

Creating and deleting a user from the CLI is genuinely a two-liner, and it tests the real backend on any Apple Silicon Mac because the backend runs on the host hypervisor — no nested virt needed.

# Create a self-contained admin user (admin needed for the container service).
sudo sysadminctl -addUser bbtest -fullName "bot-bottle test" \
    -password 'throwaway' -admin

# Log into that account (fast-user-switch or the login window), then run the
# installer as bbtest exactly as a new user would. When done:

sudo sysadminctl -deleteUser bbtest -secure   # -secure erases the home dir

Honest accounting of what this does and doesn't buy you:

Boundary strength: it's a hygiene / fresh-$HOME boundary, not a security boundary. Same kernel, same admin group; an admin test user can touch system state. If the point is "clean environment," fine. If the point is "contain something untrusted," this is the wrong tool — use a VM.
Wipe completeness: -secure erases the home dir (so ~/.bot-bottle, the pipx venv, and profile exports go away), but as the footprint table shows, the Apple Container runtime, its launchd system service, Homebrew, and Rosetta persist. For a truly repeatable "did a system with nothing installed work?" test, that residue defeats the purpose — the second run isn't clean.
Operational gotchas: don't pass real passwords on the command line (they land in ps and history — this is a throwaway credential, so it's tolerable here). Deletion must run as root from a normally-booted, admin- logged-in session; the Terminal needs Full Disk Access or you'll hit error -14120 and a half-deleted account. Prefer letting the system place the home dir (don't pass -home), or deletion can orphan it.

Use this when you're on M1/M2, you specifically want to exercise the live backend, and you can tolerate the system-level runtime staying installed between runs (or you uninstall Apple container / brew by hand to reset).

Honorable mentions

External bootable macOS volume. A fresh macOS on an external SSD (or a separate APFS volume) is bare-metal disposable: no nested-virt limit, real backend works, and you diskutil the volume away to reset. Cost is reboot friction per run — good for an occasional thorough pass, poor for a tight loop.
Rented / cloud Mac. AWS EC2 Mac (dedicated Mac minis), Scaleway Apple silicon, or MacStadium give a genuinely throwaway host you release when done. Overkill for local iteration, but this is essentially what the project's own advisory integration-macos CI job needs — a self-hosted Apple Silicon runner with the container CLI, Python ≥ 3.11, and coverage on the launchd service's PATH (README.md:78). If you end up standing up a cloud Mac for install testing, it doubles as that runner.

Recommendation

Default to a disposable Tart VM — it's the only option that wipes the entire footprint (including the Apple Container system service that a user deletion leaves behind), it's a real boundary, and the spin-up/tear-down loop is a two-command tart clone / tart delete. Confirm your chip first: on M3/M4 it tests install and runtime end-to-end; on M1/M2 it still cleanly tests install.sh + doctor's Python/config path, and you fall back to a throwaway sysadminctl admin user for live-backend testing — accepting that it's a hygiene boundary and that you'll manually uninstall the Apple Container runtime / Homebrew between runs to get back to truly clean.

There is no third, lighter-weight "in-place boundary without a user" that actually delivers a clean, wipeable macOS — the VM is that answer, and it's the better one.

Harness

The throwaway-user loop is scripted in scripts/macos-install-test.sh: up creates the account, run pipes this checkout's install.sh into it headlessly (so a PR is verifiable before it lands) and lets the installer run doctor, down deletes the account and its home (the full reset), and deep-reset additionally uninstalls the host container runtime. It leans on the footprint analysis above — the reset is just user deletion because everything install.sh writes is user-home-local.

test chains up → run → status → down into the one-shot cycle you normally want:

sudo ./scripts/macos-install-test.sh test

It refuses to start against an existing account (a reused home is not a clean install), and it tears the account down from an EXIT/INT trap armed the moment the account exists, so a failed or Ctrl-C'd run still leaves the machine clean. Its verdict is deliberately stricter than the installer's own: note that install.sh exits 0 when it finishes but doctor reports unmet prerequisites, so "the installer succeeded" is not the assertion — test fails if the install fails, if bot-bottle never reached the new user's PATH, or if doctor is unhappy. BB_TEST_KEEP=1 skips the teardown to poke at a failure.

What a fresh account actually inherits

Expect the first honest run on a developer Mac to fail at the Python gate, and expect that to be correct. A new account's PATH is just /etc/paths (/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin), which notably does not include /opt/homebrew/bin. Homebrew's shellenv line lives in the installing user's ~/.zprofile and is not inherited, so a throwaway user resolves python3 to /usr/bin/python3 — the Command Line Tools stub, still 3.9.6 on macOS 26 — and install.sh correctly dies on its 3.11+ requirement. Your own shell resolving python3 to a 3.14 Homebrew build says nothing about what a new user sees; that gap is exactly what this harness exists to expose.

15 KiB Raw Blame History