From c62aa7b8059099e2417929be7bb42b2ab55930f0 Mon Sep 17 00:00:00 2001 From: didericis Date: Mon, 27 Jul 2026 00:50:21 -0400 Subject: [PATCH] fix(install): find a usable python instead of dead-ending on PATH's MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The macOS install harness caught this on its first real run: a throwaway account gets `bot-bottle install: error: python3 3.11 or newer is required` and stops. A fresh account's PATH is just /etc/paths, which excludes /opt/homebrew/bin, so `python3` resolves to the Command Line Tools stub — still 3.9.6 on macOS 26. Homebrew's shellenv line lives in the *installing* user's ~/.zprofile and is inherited by nobody. The documented `curl … | sh` path therefore dead-ends for anyone whose profile isn't already set up, which is every new user, launchd job, and CI runner. So look past PATH before giving up: try `python3`, then python3.11-3.14, then /opt/homebrew/bin, /usr/local/bin, ~/.local/bin, and python.org framework builds, and say which one was picked when it isn't the obvious one. On this host that turns the failure into a successful install. The chosen interpreter is now threaded through everything downstream — the pip probe, the PEP 668 check, the pip --user fallback, and the user-scripts-dir lookup — which were all still hardcoding `python3` and would otherwise have run against the 3.9 stub we just rejected. `pipx install` gains `--python`, since pipx otherwise builds the venv with whichever interpreter pipx itself was installed with, not the one that passed the version check. When nothing usable is found the error is now actionable: what was found and why it's insufficient, where else it looked, a platform-appropriate install command, and BOT_BOTTLE_PYTHON to point at an interpreter directly. An explicit BOT_BOTTLE_PYTHON that is too old or unusable is an error rather than a silent fallback to a different interpreter than the caller asked for. Three existing tests asserted on incidental literals rather than the behaviour they describe, and are narrowed to their actual intent: * `never_uses_sudo` matched the word anywhere, including the new "sudo apt install python3.12" remediation *advice*. It now strips string literals and comments first, so it still catches sudo as a bare command, in a pipeline, and in a command substitution — verified by mutation — while allowing the script to print the word. * `checks_pip_usable_before_fallback` pinned the literal `python3 -m pip`. * `resolves_user_scripts_dir_not_hardcoded` banned ".local/bin" script-wide; it's now scoped to the USER_SCRIPTS assignment it exists to guard, since ~/.local/bin is a legitimate place to *find an interpreter*. README grows the install command and a Requirements section it never had, leading with the Python floor and why a working `python3` in your own shell says nothing about what a fresh account sees. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5 --- README.md | 16 +++- install.sh | 129 +++++++++++++++++++++++++----- tests/unit/test_install_script.py | 65 ++++++++++++--- 3 files changed, 175 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index 90c34fd2..8aeef05a 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,21 @@ When the agent exits, `cli.py` tears down every gateway and both networks; nothi ## Quickstart -On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`. +```sh +curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh +``` + +The installer is a bootstrapper: it finds a suitable Python, installs bot-bottle with `pipx` (falling back to `pip --user`), creates `~/.bot-bottle`, and runs `bot-bottle doctor`. It is idempotent and never uses `sudo`. Python-native users can skip it entirely with `pipx install bot-bottle` or `uv tool install bot-bottle`. + +### Requirements + +**Python ≥ 3.11**, and this is the one that trips people up on macOS: the `python3` Apple ships at `/usr/bin/python3` is **3.9.6**, which is too old. A Homebrew or python.org Python is only on your `PATH` because of a line in your shell profile, so anything running without that profile — a fresh user account, a launchd job, a CI runner — sees the 3.9 stub instead. The installer therefore looks past `PATH` before giving up: it tries `python3`, then `python3.11`–`python3.14`, then `/opt/homebrew/bin`, `/usr/local/bin`, `~/.local/bin`, and python.org framework builds, and tells you which one it picked when it isn't the obvious one. Point it somewhere specific with `BOT_BOTTLE_PYTHON=/path/to/python3`. + +**`pipx`** is strongly preferred over the `pip --user` fallback. Homebrew and Debian/Ubuntu interpreters are externally managed (PEP 668), which blocks `pip install --user` outright; the installer detects this and tells you to install `pipx` rather than failing deep inside pip. + +**`git`**, because the default install spec is a `git+` URL. Set `BOT_BOTTLE_INSTALL_SPEC` to a wheel path or index name to avoid it. + +**A backend**, which the installer deliberately does *not* install for you — `doctor` reports what's missing afterwards. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`. Use `BOT_BOTTLE_BACKEND=docker ./cli.py start ` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend. diff --git a/install.sh b/install.sh index 058c2109..452c5195 100755 --- a/install.sh +++ b/install.sh @@ -8,11 +8,15 @@ # pipx install bot-bottle # from a checkout or a published index # uv tool install bot-bottle # -# This script is a thin bootstrapper: it checks prerequisites, installs the -# package with pipx (falling back to pip --user), creates the config dir, and -# runs `bot-bottle doctor`. It is idempotent (safe to re-run) and never uses -# sudo. It does NOT install Docker or a VM backend for you — `doctor` reports -# what's missing after install. +# This script is a thin bootstrapper: it finds a Python 3.11+ interpreter, +# installs the package with pipx (falling back to pip --user), creates the +# config dir, and runs `bot-bottle doctor`. It is idempotent (safe to re-run) +# and never uses sudo. It does NOT install Docker or a VM backend for you — +# `doctor` reports what's missing after install. +# +# Env: +# BOT_BOTTLE_PYTHON interpreter to install with (skips the search) +# BOT_BOTTLE_INSTALL_SPEC pip/git spec to install instead of the default set -eu PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}" @@ -28,17 +32,97 @@ die() { exit 1 } -# --- prerequisites ----------------------------------------------------------- +# --- prerequisites: find an interpreter new enough ---------------------------- -command -v python3 >/dev/null 2>&1 \ - || die "python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR}+ is required but was not found" - -python3 - "$MIN_PYTHON_MAJOR" "$MIN_PYTHON_MINOR" <<'PY' || die "python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} or newer is required" +# Is $1 an interpreter that exists and meets the floor? +python_ok() { + [ -n "${1:-}" ] || return 1 + command -v "$1" >/dev/null 2>&1 || return 1 + "$1" - "$MIN_PYTHON_MAJOR" "$MIN_PYTHON_MINOR" <<'PY' >/dev/null 2>&1 import sys want = (int(sys.argv[1]), int(sys.argv[2])) raise SystemExit(0 if sys.version_info[:2] >= want else 1) PY +} + +python_version() { + "$1" -c 'import sys; print("%d.%d.%d" % sys.version_info[:3])' 2>/dev/null +} + +# `python3` on PATH is often NOT the newest interpreter installed, and on macOS +# it is usually the oldest: a fresh login shell's PATH is just /etc/paths, so +# python3 resolves to the Command Line Tools stub (3.9.x) while the usable +# 3.11+ build sits in /opt/homebrew/bin or a python.org framework directory, +# reachable only via a line in the *installing* user's shell profile. A new +# account inherits none of that. Look past PATH before giving up, so the common +# case installs instead of dead-ending on a version error. +find_python() { + for candidate in \ + "${BOT_BOTTLE_PYTHON:-}" \ + python3 \ + python3.14 python3.13 python3.12 python3.11 \ + /opt/homebrew/bin/python3 \ + /usr/local/bin/python3 \ + "${HOME}/.local/bin/python3" \ + /Library/Frameworks/Python.framework/Versions/*/bin/python3 + do + # An unmatched glob arrives here literally; python_ok rejects it. + if python_ok "$candidate"; then + command -v "$candidate" + return 0 + fi + done + return 1 +} + +# An explicit choice that doesn't work is an error, not a reason to quietly +# search elsewhere and install somewhere the caller didn't ask for. +if [ -n "${BOT_BOTTLE_PYTHON:-}" ] && ! python_ok "${BOT_BOTTLE_PYTHON}"; then + if command -v "${BOT_BOTTLE_PYTHON}" >/dev/null 2>&1; then + die "BOT_BOTTLE_PYTHON=${BOT_BOTTLE_PYTHON} is $(python_version "${BOT_BOTTLE_PYTHON}"), "\ +"below the ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} floor. Unset it to search for a newer one." + fi + die "BOT_BOTTLE_PYTHON=${BOT_BOTTLE_PYTHON} is not an executable interpreter." +fi + +PYTHON="$(find_python || true)" + +if [ -z "${PYTHON}" ]; then + path_python="$(command -v python3 2>/dev/null || true)" + if [ -n "${path_python}" ]; then + found="the python3 on your PATH is ${path_python} ($(python_version "${path_python}")), which is too old" + else + found="no python3 was found on your PATH" + fi + case "$(uname -s)" in + Darwin) fix=" brew install python@3.12 + # or install from https://www.python.org/downloads/macos/ + # macOS itself ships only /usr/bin/python3, which is too old" ;; + *) fix=" sudo apt install python3.12 # Debian/Ubuntu + sudo dnf install python3.12 # Fedora/RHEL" ;; + esac + die "bot-bottle needs python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} or newer, and none was found. + ${found}. + Also checked: python3.11-3.14, /opt/homebrew/bin, /usr/local/bin, + ~/.local/bin, and python.org framework builds. + +Install a newer Python, then re-run this installer: +${fix} + +Already have one somewhere? Point at it directly: + BOT_BOTTLE_PYTHON=/path/to/python3 sh install.sh" +fi + +# Be explicit when the interpreter isn't the obvious one, so nobody is left +# wondering which Python their install ended up on. +path_python="$(command -v python3 2>/dev/null || true)" +if [ "${PYTHON}" != "${path_python}" ]; then + say "using ${PYTHON} ($(python_version "${PYTHON}"))" + if [ -n "${path_python}" ]; then + say "note: 'python3' on your PATH is ${path_python} ($(python_version "${path_python}")), which is below the ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} floor" + fi +fi # Installing a `git+` spec (the default) shells out to git under the hood, # whether via pipx or pip. Fail early with a clear message rather than deep @@ -55,10 +139,10 @@ esac # (PEP 668, common on Debian/Ubuntu/Homebrew) reject `pip install --user`; # pipx sidesteps that, so recommend it when pip can't be used. if ! command -v pipx >/dev/null 2>&1; then - python3 -m pip --version >/dev/null 2>&1 || die \ - "neither pipx nor a usable 'python3 -m pip' was found. Install pipx "\ -"(recommended): 'python3 -m pip install --user pipx' or your OS package manager." - if python3 - <<'PY' + "${PYTHON}" -m pip --version >/dev/null 2>&1 || die \ + "neither pipx nor a usable '${PYTHON} -m pip' was found. Install pipx "\ +"(recommended): '${PYTHON} -m pip install --user pipx' or your OS package manager." + if "${PYTHON}" - <<'PY' import os import sys import sysconfig @@ -69,8 +153,8 @@ marker = os.path.join(sysconfig.get_path("stdlib"), "EXTERNALLY-MANAGED") raise SystemExit(0 if os.path.exists(marker) else 1) PY then - die "this Python is externally managed (PEP 668), so 'pip install --user' is "\ -"blocked. Install pipx and re-run: 'python3 -m pip install --user --break-system-packages pipx', "\ + die "${PYTHON} is externally managed (PEP 668), so 'pip install --user' is "\ +"blocked. Install pipx and re-run: '${PYTHON} -m pip install --user --break-system-packages pipx', "\ "then 'pipx ensurepath'." fi fi @@ -85,11 +169,14 @@ mkdir -p \ # --- install ----------------------------------------------------------------- if command -v pipx >/dev/null 2>&1; then - say "installing with pipx" - pipx install --force "${PACKAGE_SPEC}" + # --python pins the venv to the interpreter we vetted. Without it pipx uses + # whichever Python it was itself installed with, which is not necessarily + # the one that passed the version check above. + say "installing with pipx (python: ${PYTHON})" + pipx install --python "${PYTHON}" --force "${PACKAGE_SPEC}" else - say "pipx not found; installing with 'python3 -m pip install --user'" - python3 -m pip install --user --upgrade "${PACKAGE_SPEC}" + say "pipx not found; installing with '${PYTHON} -m pip install --user'" + "${PYTHON}" -m pip install --user --upgrade "${PACKAGE_SPEC}" fi # --- locate the entry point -------------------------------------------------- @@ -97,7 +184,7 @@ fi # The pip --user scripts directory is platform-specific: ~/.local/bin on Linux, # but ~/Library/Python//bin on a python.org macOS interpreter. Ask the # interpreter for its own user-scheme scripts dir instead of hardcoding. -USER_SCRIPTS="$(python3 - <<'PY' +USER_SCRIPTS="$("${PYTHON}" - <<'PY' import sysconfig print(sysconfig.get_path("scripts", sysconfig.get_preferred_scheme("user"))) PY diff --git a/tests/unit/test_install_script.py b/tests/unit/test_install_script.py index a0a27905..32a3c27a 100644 --- a/tests/unit/test_install_script.py +++ b/tests/unit/test_install_script.py @@ -9,6 +9,7 @@ create the config tree, install the package, and verify with `doctor`. from __future__ import annotations import os +import re import sysconfig import unittest from pathlib import Path @@ -17,6 +18,22 @@ REPO_ROOT = Path(__file__).resolve().parents[2] INSTALL_SH = REPO_ROOT / "install.sh" +def code_only(text: str) -> str: + """Script text with string literals and comments removed. + + Both are places the script *talks about* commands rather than running + them — remediation advice quite reasonably says "sudo apt install …" — + so assertions about what the script actually executes must not see them. + Strings are stripped before comments because a '#' inside a quoted string + is not a comment, and several literals here span multiple lines. + """ + without_strings = re.sub(r"\"(?:[^\"\\]|\\.)*\"|'[^']*'", "", text) + return "\n".join( + ln for ln in without_strings.splitlines() + if ln.strip() and not ln.lstrip().startswith("#") + ) + + class TestInstallScript(unittest.TestCase): @classmethod def setUpClass(cls): @@ -32,12 +49,9 @@ class TestInstallScript(unittest.TestCase): self.assertIn("set -eu", self.text) def test_never_uses_sudo(self): - # Only executable lines matter; the header comment may mention sudo. - code = [ - ln for ln in self.text.splitlines() - if ln.strip() and not ln.lstrip().startswith("#") - ] - self.assertNotIn("sudo", "\n".join(code)) + # The installer must never *invoke* sudo. It may print it: the "no + # usable python" error suggests 'sudo apt install python3.12'. + self.assertNotIn("sudo", code_only(self.text)) def test_creates_config_tree(self): self.assertIn(".bot-bottle/agents", self.text) @@ -61,7 +75,9 @@ class TestInstallScript(unittest.TestCase): self.assertIn("git+*|*.git", self.text) def test_checks_pip_usable_before_fallback(self): - self.assertIn("python3 -m pip --version", self.text) + # Against the discovered interpreter, which is not necessarily the + # `python3` on PATH — hence no literal command name here. + self.assertIn("-m pip --version", self.text) def test_detects_externally_managed_python(self): # PEP 668: 'pip install --user' is blocked on externally-managed @@ -75,12 +91,35 @@ class TestInstallScript(unittest.TestCase): # hardcoding Linux's ~/.local/bin (which is wrong on macOS python.org). self.assertIn("get_preferred_scheme", self.text) self.assertIn("sysconfig", self.text) - # No hardcoded Linux path in executable lines (a comment may mention it). - code = "\n".join( - ln for ln in self.text.splitlines() - if ln.strip() and not ln.lstrip().startswith("#") - ) - self.assertNotIn(".local/bin", code) + # ~/.local/bin appears legitimately elsewhere — it's one of the places + # the interpreter search looks. What must never happen is USER_SCRIPTS + # itself being hardcoded to it. + for line in code_only(self.text).splitlines(): + if "USER_SCRIPTS" in line: + self.assertNotIn(".local/bin", line) + + def test_searches_beyond_path_for_an_interpreter(self): + # `python3` on PATH is the *oldest* interpreter on a stock Mac: a fresh + # account's PATH is /etc/paths, so python3 is the 3.9.6 CLT stub while + # the usable build sits somewhere only a shell profile puts on PATH. + # Giving up at that point dead-ends every new macOS user. + for candidate in ("python3.11", "/opt/homebrew/bin", "Python.framework"): + self.assertIn(candidate, self.text) + + def test_interpreter_is_overridable(self): + self.assertIn("BOT_BOTTLE_PYTHON", self.text) + + def test_pipx_is_pinned_to_the_vetted_interpreter(self): + # Without --python, pipx builds the venv with whichever interpreter + # pipx itself was installed with, which need not be the one that + # passed the version check. + self.assertIn("pipx install --python", self.text) + + def test_version_failure_is_actionable(self): + # The failure a new macOS user actually hits must say what to do about + # it, not just state the requirement. + self.assertIn("brew install python@", self.text) + self.assertIn("BOT_BOTTLE_PYTHON=/path/to/python3", self.text) def test_macos_user_scheme_is_not_dot_local_bin(self): # The case the fix exists for: a python.org macOS interpreter uses the