README.md

@@ -81,7 +81,7 @@ The installer is a bootstrapper: it finds a suitable Python, installs bot-bottle
 
 **Python ≥ 3.11**, and this is the one that trips people up on macOS: the `python3` Apple ships at `/usr/bin/python3` is **3.9.6**, which is too old. Bare `python3` resolves to that stub far more often than people expect. `path_helper` builds a login shell's `PATH` from `/etc/paths` and then appends `/etc/paths.d/*`, and `/usr/bin` sits in the former — so even when `/opt/homebrew/bin` *is* on the `PATH` (via `/etc/paths.d/homebrew`), it comes after `/usr/bin` and loses. Prepending a newer Python is something your shell profile does, and a fresh account, a launchd job, or a CI runner has no such profile. So the installer looks past bare `python3` before giving up: it tries `python3`, then the versioned `python3.11`–`python3.14` names, then `/opt/homebrew/bin`, `/usr/local/bin`, `~/.local/bin`, and python.org framework builds — and tells you which one it picked when it isn't the obvious one. Point it somewhere specific with `BOT_BOTTLE_PYTHON=/path/to/python3`.
 
-**`pipx`** is strongly preferred over the `pip --user` fallback. Homebrew and Debian/Ubuntu interpreters are externally managed (PEP 668), which blocks `pip install --user` outright; the installer detects this and tells you to install `pipx` rather than failing deep inside pip.
+**No `pipx` required.** If `pipx` is present the installer uses it and stays out of the way. If it isn't, bot-bottle installs into a private venv at `~/.bot-bottle/venv` (override with `BOT_BOTTLE_VENV`) and symlinks the entry point into `~/.local/bin`. There is deliberately no `pip install --user` path: Homebrew, python.org and Debian/Ubuntu interpreters are all externally managed (PEP 668), which blocks `--user` outright — so on a Mac it is never the fallback it appears to be. A venv is exempt from PEP 668, and `venv` is stdlib, so unlike `pipx` there is nothing to bootstrap first.
 
 **`git`**, because the default install spec is a `git+` URL. Set `BOT_BOTTLE_INSTALL_SPEC` to a wheel path or index name to avoid it.
 

docs/research/testing-clean-install-on-macos.md

@@ -69,7 +69,7 @@ Grounding the teardown story in what `install.sh` and the backend create:
 |---|---|---|---|
 | Config / state / db | `~/.bot-bottle/{agents,bottles,contrib,state,db}` ([`install.sh:80-83`](../install.sh), [`bot_bottle/paths.py:59`](../bot_bottle/paths.py)) | ✅ | ❌ removed with home |
 | pipx venv + shim | `~/.local/pipx/venvs/bot-bottle`, shim in `~/.local/bin` ([`install.sh:87-89`](../install.sh)) | ✅ | ❌ removed with home |
-| pip `--user` fallback | `~/Library/Python/<X.Y>/{lib,bin}` ([`install.sh:100-104`](../install.sh)) | ✅ | ❌ removed with home |
+| private venv fallback (no pipx) | `~/.bot-bottle/venv` + symlink in `~/.local/bin` ([`install.sh`](../install.sh)) | ✅ | ❌ removed with home |
 | PATH / token exports | shell profile (`~/.zprofile`, etc.); `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` ([`README.md:74`](../README.md)) | ✅ | ❌ removed with home |
 | **Apple `container` install** | `/usr/local/...` + notarized `.pkg` receipts | ❌ | ✅ **stays** |
 | **Apple `container` system service** | launchd system service (`container system start`) | ❌ | ✅ **stays** |

install.sh

@@ -9,7 +9,7 @@
 #   uv tool install bot-bottle
 #
 # This script is a thin bootstrapper: it finds a Python 3.11+ interpreter,
-# installs the package with pipx (falling back to pip --user), creates the
+# installs the package with pipx (falling back to a private venv), creates the
 # config dir, and runs `bot-bottle doctor`. It is idempotent (safe to re-run)
 # and never uses sudo. It does NOT install Docker or a VM backend for you —
 # `doctor` reports what's missing after install.
@@ -17,9 +17,11 @@
 # Env:
 #   BOT_BOTTLE_PYTHON        interpreter to install with (skips the search)
 #   BOT_BOTTLE_INSTALL_SPEC  pip/git spec to install instead of the default
+#   BOT_BOTTLE_VENV          where the non-pipx install lives (~/.bot-bottle/venv)
 set -eu
 
 PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}"
+VENV="${BOT_BOTTLE_VENV:-${HOME}/.bot-bottle/venv}"
 MIN_PYTHON_MAJOR=3
 MIN_PYTHON_MINOR=11
 
@@ -135,30 +137,6 @@ case "${PACKAGE_SPEC}" in
         ;;
 esac
 
-# The pip fallback needs a usable pip. Externally-managed interpreters
-# (PEP 668, common on Debian/Ubuntu/Homebrew) reject `pip install --user`;
-# pipx sidesteps that, so recommend it when pip can't be used.
-if ! command -v pipx >/dev/null 2>&1; then
-    "${PYTHON}" -m pip --version >/dev/null 2>&1 || die \
-        "neither pipx nor a usable '${PYTHON} -m pip' was found. Install pipx "\
-"(recommended): '${PYTHON} -m pip install --user pipx' or your OS package manager."
-    if "${PYTHON}" - <<'PY'
-import os
-import sys
-import sysconfig
-
-# PEP 668: an EXTERNALLY-MANAGED marker in the stdlib dir means pip refuses
-# to install into this interpreter without --break-system-packages.
-marker = os.path.join(sysconfig.get_path("stdlib"), "EXTERNALLY-MANAGED")
-raise SystemExit(0 if os.path.exists(marker) else 1)
-PY
-    then
-        die "${PYTHON} is externally managed (PEP 668), so 'pip install --user' is "\
-"blocked. Install pipx and re-run: '${PYTHON} -m pip install --user --break-system-packages pipx', "\
-"then 'pipx ensurepath'."
-    fi
-fi
-
 # --- config directories ------------------------------------------------------
 
 mkdir -p \
@@ -168,35 +146,43 @@ mkdir -p \
 
 # --- install -----------------------------------------------------------------
 
+BIN_DIR="${HOME}/.local/bin"
+
 if command -v pipx >/dev/null 2>&1; then
     # --python pins the venv to the interpreter we vetted. Without it pipx uses
     # whichever Python it was itself installed with, which is not necessarily
     # the one that passed the version check above.
     say "installing with pipx (python: ${PYTHON})"
     pipx install --python "${PYTHON}" --force "${PACKAGE_SPEC}"
+    # Ask pipx where it puts entry points rather than assuming ~/.local/bin.
+    pipx_bin="$(pipx environment --value PIPX_BIN_DIR 2>/dev/null || true)"
+    [ -n "${pipx_bin}" ] && BIN_DIR="${pipx_bin}"
 else
-    say "pipx not found; installing with '${PYTHON} -m pip install --user'"
-    "${PYTHON}" -m pip install --user --upgrade "${PACKAGE_SPEC}"
+    # No `pip install --user` fallback: PEP 668 makes it unusable on nearly
+    # every interpreter a Mac offers (Homebrew and python.org are both
+    # externally managed), and on Debian/Ubuntu too. A private venv sidesteps
+    # that entirely — PEP 668 does not apply inside a venv — and `venv` is
+    # stdlib, so unlike pipx there is nothing to bootstrap first.
+    say "pipx not found; installing into a managed venv at ${VENV}"
+    "${PYTHON}" -m venv --clear "${VENV}" || die \
+        "could not create a virtualenv at ${VENV} using ${PYTHON}. On Debian/Ubuntu "\
+"the venv module ships separately: 'sudo apt install python3-venv'."
+    "${VENV}/bin/python" -m pip install --upgrade "${PACKAGE_SPEC}"
+    # Expose the entry point outside the venv, the way pipx would.
+    mkdir -p "${BIN_DIR}"
+    ln -sf "${VENV}/bin/bot-bottle" "${BIN_DIR}/bot-bottle"
 fi
 
 # --- locate the entry point --------------------------------------------------
 
-# The pip --user scripts directory is platform-specific: ~/.local/bin on Linux,
-# but ~/Library/Python/<X.Y>/bin on a python.org macOS interpreter. Ask the
-# interpreter for its own user-scheme scripts dir instead of hardcoding.
-USER_SCRIPTS="$("${PYTHON}" - <<'PY'
-import sysconfig
-print(sysconfig.get_path("scripts", sysconfig.get_preferred_scheme("user")))
-PY
-)"
-
 if command -v bot-bottle >/dev/null 2>&1; then
     BOT_BOTTLE_BIN="bot-bottle"
-elif [ -n "${USER_SCRIPTS}" ] && [ -x "${USER_SCRIPTS}/bot-bottle" ]; then
-    BOT_BOTTLE_BIN="${USER_SCRIPTS}/bot-bottle"
-    say "note: add ${USER_SCRIPTS} to your PATH to run 'bot-bottle' directly"
+elif [ -x "${BIN_DIR}/bot-bottle" ]; then
+    BOT_BOTTLE_BIN="${BIN_DIR}/bot-bottle"
+    say "note: add ${BIN_DIR} to your PATH to run 'bot-bottle' directly:"
+    say "    echo 'export PATH=\"${BIN_DIR}:\$PATH\"' >> ~/.zprofile"
 else
-    die "bot-bottle was installed but is not on PATH; add ${USER_SCRIPTS:-your user scripts dir} to PATH and re-run"
+    die "bot-bottle was installed but no entry point turned up in ${BIN_DIR}"
 fi
 
 # --- verify ------------------------------------------------------------------

scripts/macos-install-test.sh

@@ -3,9 +3,10 @@
 #
 # Exercises install.sh the way a brand-new user would, inside a throwaway
 # macOS account you create and delete from the CLI. install.sh's entire
-# footprint is user-home-local — the pipx venv under ~/.local (or the pip
-# --user tree under ~/Library/Python) and the ~/.bot-bottle config dir. It
-# writes no shell-profile PATH line, and never installs the backend (see
+# footprint is user-home-local — the pipx venv under ~/.local, or the private
+# venv at ~/.bot-bottle/venv plus a ~/.local/bin symlink, and the ~/.bot-bottle
+# config dir. It writes no shell-profile PATH line, and never installs the
+# backend (see
 # the header of install.sh), so deleting the user is a complete,
 # deterministic reset of everything the installer touched. The Apple
 # `container` runtime is a HOST prerequisite installed once and kept;

tests/unit/test_install_script.py

@@ -10,7 +10,6 @@ from __future__ import annotations
 
 import os
 import re
-import sysconfig
 import unittest
 from pathlib import Path
 
@@ -57,9 +56,37 @@ class TestInstallScript(unittest.TestCase):
         self.assertIn(".bot-bottle/agents", self.text)
         self.assertIn(".bot-bottle/bottles", self.text)
 
-    def test_installs_via_pipx_with_pip_fallback(self):
+    def test_installs_via_pipx_with_venv_fallback(self):
         self.assertIn("pipx install", self.text)
-        self.assertIn("pip install --user", self.text)
+        self.assertIn("-m venv", self.text)
+
+    def test_no_pip_user_fallback(self):
+        # `pip install --user` is not a fallback, it's a dead end: PEP 668
+        # blocks it on Homebrew, python.org and Debian/Ubuntu interpreters,
+        # which is every Python a Mac realistically offers. A private venv is
+        # exempt from PEP 668 and needs no bootstrap, since venv is stdlib.
+        # code_only, because the comment explaining the absence says the words.
+        code = code_only(self.text)
+        self.assertNotIn("pip install --user", code)
+        self.assertNotIn("--break-system-packages", code)
+
+    def test_venv_lives_under_the_config_dir(self):
+        # Keeps the whole install footprint inside ~/.bot-bottle (plus the
+        # entry-point symlink), which is what makes deleting a throwaway
+        # account a complete reset in scripts/macos-install-test.sh.
+        self.assertIn(".bot-bottle/venv", self.text)
+        self.assertIn("BOT_BOTTLE_VENV", self.text)
+
+    def test_venv_failure_is_actionable(self):
+        # Debian/Ubuntu ship venv separately; failing there must say so rather
+        # than dumping ensurepip's error.
+        self.assertIn("python3-venv", self.text)
+
+    def test_entry_point_is_exposed_outside_the_venv(self):
+        # A venv's bin dir is never on PATH, so the console script has to be
+        # linked somewhere conventional or `bot-bottle` is unreachable.
+        self.assertIn(".local/bin", self.text)
+        self.assertIn("ln -sf", self.text)
 
     def test_runs_doctor_after_install(self):
         self.assertIn("doctor", self.text)
@@ -74,29 +101,20 @@ class TestInstallScript(unittest.TestCase):
         self.assertIn("command -v git", self.text)
         self.assertIn("git+*|*.git", self.text)
 
-    def test_checks_pip_usable_before_fallback(self):
-        # Against the discovered interpreter, which is not necessarily the
-        # `python3` on PATH — hence no literal command name here.
-        self.assertIn("-m pip --version", self.text)
+    def test_installs_into_the_venv_with_its_own_pip(self):
+        # The venv's pip, not the base interpreter's — the base one may not
+        # exist, and using it would install outside the venv.
+        self.assertIn("${VENV}/bin/python\" -m pip install", self.text)
 
-    def test_detects_externally_managed_python(self):
-        # PEP 668: 'pip install --user' is blocked on externally-managed
-        # interpreters; the script must detect this and point at pipx.
-        self.assertIn("EXTERNALLY-MANAGED", self.text)
-        self.assertIn("pipx", self.text)
+    def test_pipx_is_preferred_when_present(self):
+        # The venv is a fallback, not a takeover: someone who already manages
+        # their Python apps with pipx keeps doing so.
+        self.assertIn("command -v pipx", self.text)
 
-    def test_resolves_user_scripts_dir_not_hardcoded(self):
-        # The pip --user scripts dir differs by platform; the script must ask
-        # the interpreter (sysconfig + the preferred *user* scheme) rather than
-        # hardcoding Linux's ~/.local/bin (which is wrong on macOS python.org).
-        self.assertIn("get_preferred_scheme", self.text)
-        self.assertIn("sysconfig", self.text)
-        # ~/.local/bin appears legitimately elsewhere — it's one of the places
-        # the interpreter search looks. What must never happen is USER_SCRIPTS
-        # itself being hardcoded to it.
-        for line in code_only(self.text).splitlines():
-            if "USER_SCRIPTS" in line:
-                self.assertNotIn(".local/bin", line)
+    def test_asks_pipx_where_its_bin_dir_is(self):
+        # PIPX_BIN_DIR is configurable, so the post-install "is it on PATH?"
+        # check must ask rather than assume ~/.local/bin.
+        self.assertIn("PIPX_BIN_DIR", self.text)
 
     def test_searches_beyond_path_for_an_interpreter(self):
         # `python3` on PATH is the *oldest* interpreter on a stock Mac: a fresh
@@ -121,20 +139,5 @@ class TestInstallScript(unittest.TestCase):
         self.assertIn("brew install python@", self.text)
         self.assertIn("BOT_BOTTLE_PYTHON=/path/to/python3", self.text)
 
-    def test_macos_user_scheme_is_not_dot_local_bin(self):
-        # The case the fix exists for: a python.org macOS interpreter uses the
-        # osx_framework_user scheme, whose scripts land under
-        # ~/Library/Python/<X.Y>/bin — NOT ~/.local/bin. Drive the same
-        # sysconfig lookup install.sh uses, with a mac-like userbase, to prove
-        # it resolves a non-~/.local/bin directory.
-        self.assertIn("osx_framework_user", sysconfig.get_scheme_names())
-        scripts = sysconfig.get_path(
-            "scripts", "osx_framework_user",
-            vars={"userbase": "/Users/dev/Library/Python/3.11"},
-        )
-        self.assertEqual("/Users/dev/Library/Python/3.11/bin", scripts)
-        self.assertNotIn("/.local/bin", scripts)
-
-
 if __name__ == "__main__":
     unittest.main()