Compare commits
57 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ccfb935b19 | |||
| 91f6cbabdc | |||
| cf15500fb8 | |||
| 5cb1c190f4 | |||
| bb4075558f | |||
| e4d8f041e7 | |||
| 161a9d0f1b | |||
| a0f2c938b9 | |||
| 4598e62836 | |||
| 99dcba92ea | |||
| c89847b626 | |||
| cd9f023f3d | |||
| 5c499d290b | |||
| 0500ae5f4c | |||
| 5614a56ccd | |||
| 71c0f8ed88 | |||
| 556217ae7b | |||
| 7461802b62 | |||
| 5c12c6bf4e | |||
| 4096409495 | |||
| 8d0782d1c7 | |||
| 6793a09e2b | |||
| af293d7036 | |||
| d415581adc | |||
| 27b9ba5247 | |||
| 420184b874 | |||
| 7938b90d19 | |||
| d879258f62 | |||
| 78d3b43061 | |||
| cdc5c8203d | |||
| d04bbf1454 | |||
| c00097d998 | |||
| 6630a1f701 | |||
| a7ad82f03a | |||
| 9b8f126b8f | |||
| d167c03215 | |||
| 479b93bd0b | |||
| 2de61eeb17 | |||
| a4bc5d4508 | |||
| 7ebd067d2c | |||
| be50354100 | |||
| e1163184c9 | |||
| 70f6a9be20 | |||
| aa43fd032f | |||
| be5c803e8e | |||
| 38ff4a5e88 | |||
| 8df76f6126 | |||
| b63c41db9c | |||
| 6c52686069 | |||
| a52245af95 | |||
| d4e48a4169 | |||
| 53cc3ca492 | |||
| bafb73fdb9 | |||
| b0f317c499 | |||
| 57eb4394dc | |||
| 82e1a353bd | |||
| a6e1aebda1 |
@@ -15,7 +15,7 @@
|
||||
## Features
|
||||
|
||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
|
||||
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
|
||||
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `bot-bottle supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
|
||||
- **Tokens the agent never sees** — host secrets live in a gateway; the agent dials `http://gateway:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
||||
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
||||
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
||||
@@ -39,7 +39,7 @@ On the legacy Docker backend, the same logical bottle is two containers per agen
|
||||
The Docker topology looks like this:
|
||||
|
||||
```
|
||||
host ( ./cli.py )
|
||||
host ( bot-bottle )
|
||||
│
|
||||
starts │ stops
|
||||
▼
|
||||
@@ -71,9 +71,23 @@ When the agent exits, `cli.py` tears down every gateway and both networks; nothi
|
||||
|
||||
## Quickstart
|
||||
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
```sh
|
||||
curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
|
||||
```
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
|
||||
The installer is a bootstrapper: it finds a suitable Python, installs bot-bottle with `pipx` (falling back to `pip --user`), creates `~/.bot-bottle`, and runs `bot-bottle doctor`. It is idempotent and never uses `sudo`. Python-native users can skip it entirely with `pipx install bot-bottle` or `uv tool install bot-bottle`.
|
||||
|
||||
### Requirements
|
||||
|
||||
**Python ≥ 3.11**, and this is the one that trips people up on macOS: the `python3` Apple ships at `/usr/bin/python3` is **3.9.6**, which is too old. Bare `python3` resolves to that stub far more often than people expect. `path_helper` builds a login shell's `PATH` from `/etc/paths` and then appends `/etc/paths.d/*`, and `/usr/bin` sits in the former — so even when `/opt/homebrew/bin` *is* on the `PATH` (via `/etc/paths.d/homebrew`), it comes after `/usr/bin` and loses. Prepending a newer Python is something your shell profile does, and a fresh account, a launchd job, or a CI runner has no such profile. So the installer looks past bare `python3` before giving up: it tries `python3`, then the versioned `python3.11`–`python3.14` names, then `/opt/homebrew/bin`, `/usr/local/bin`, `~/.local/bin`, and python.org framework builds — and tells you which one it picked when it isn't the obvious one. Point it somewhere specific with `BOT_BOTTLE_PYTHON=/path/to/python3`.
|
||||
|
||||
**No `pipx` required.** If `pipx` is present the installer uses it and stays out of the way. If it isn't, bot-bottle installs into a private venv at `~/.bot-bottle/venv` (override with `BOT_BOTTLE_VENV`) and symlinks the entry point into `~/.local/bin`. There is deliberately no `pip install --user` path: Homebrew, python.org and Debian/Ubuntu interpreters are all externally managed (PEP 668), which blocks `--user` outright — so on a Mac it is never the fallback it appears to be. A venv is exempt from PEP 668, and `venv` is stdlib, so unlike `pipx` there is nothing to bootstrap first.
|
||||
|
||||
**`git`**, because the default install spec is a `git+` URL. Set `BOT_BOTTLE_INSTALL_SPEC` to a wheel path or index name to avoid it.
|
||||
|
||||
**A backend**, which the installer deliberately does *not* install for you — `doctor` reports what's missing afterwards. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker bot-bottle start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
|
||||
|
||||
> **CI (macOS Apple Container):** the advisory `integration-macos` job in `.gitea/workflows/pre-release-test.yml` runs only on manual dispatch. It targets a self-hosted host-mode runner labelled `macos`; Apple Container cannot run inside the Linux pull-request runner. Provision an Apple Silicon host with the `container` CLI running and Python ≥ 3.11 plus `coverage` on the launchd service's explicit `PATH`. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1. Its coverage is reported separately and never feeds the required pull-request gate.
|
||||
|
||||
@@ -166,10 +180,10 @@ On Linux, a KVM-capable host defaults to the Firecracker backend. It needs:
|
||||
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
|
||||
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
|
||||
- **Docker** for the gateway and image build.
|
||||
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
|
||||
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `bot-bottle backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `bot-bottle backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
|
||||
|
||||
```sh
|
||||
BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
|
||||
BOT_BOTTLE_BACKEND=firecracker bot-bottle start <agent>
|
||||
```
|
||||
|
||||
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
|
||||
@@ -177,12 +191,14 @@ BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
|
||||
> **CI:** Firecracker integration runs in the manually dispatched `.gitea/workflows/pre-release-test.yml` on a self-hosted runner labelled `kvm`; privileged KVM hosts never execute unreviewed PR code automatically. Provision it like a normal Firecracker host: `firecracker` on `PATH`, `/dev/kvm`, the cached guest kernel and static dropbear, and the persistent TAP/nft pool. The required pull-request workflow runs unit plus the complete Docker integration suite on `ubuntu-latest`; see `docs/ci.md`.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
bot-bottle start <agent> # builds the image on first run, drops you into claude
|
||||
```
|
||||
|
||||
## Manifest
|
||||
|
||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Bottles live in `~/.bot-bottle/bottles/`; agents may also be shipped by a repo at `<repo>/.bot-bottle/agents/<name>.md`.
|
||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Both bottles and agents are **home-only**: they live under `~/.bot-bottle/bottles/` and `~/.bot-bottle/agents/`. A `<repo>/.bot-bottle/agents/<name>.md` shipped by a workspace is ignored with a warning — since an agent may select a host identity and forge secret, checked-out content must not define one (PRD 0082). Keep repo-specific behavioral instructions in `AGENTS.md` instead.
|
||||
|
||||
Identity is **agent-owned**: the author name/email and named forge accounts live on the agent, not under `git-gate` (which now carries only Git transport policy). A bottle repo may optionally name one of the selected agent's forge aliases via `forge:`.
|
||||
|
||||
**Bottle** (`~/.bot-bottle/bottles/gitea-dev.md`):
|
||||
|
||||
@@ -190,37 +206,19 @@ Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle
|
||||
---
|
||||
extends: claude # inherit the Claude provider boundary
|
||||
|
||||
env:
|
||||
GIT_AUTHOR_NAME: didericis
|
||||
|
||||
git:
|
||||
user:
|
||||
name: "Eric Bauerfeld"
|
||||
email: "eric+claude@dideric.is"
|
||||
remotes:
|
||||
gitea.dideric.is:
|
||||
Name: bot-bottle
|
||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
||||
KnownHostKey: ssh-ed25519 AAAA...
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
inspect:
|
||||
auth:
|
||||
scheme: token # Bearer | token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
matches: # optional — restrict to specific paths/methods/headers
|
||||
- paths:
|
||||
- {type: prefix, value: /api/v1/}
|
||||
methods: [GET, POST, PATCH, DELETE]
|
||||
outbound_detectors: [token_patterns, known_secrets]
|
||||
inbound_detectors: false # disable response scanning for this host
|
||||
git-gate:
|
||||
repos:
|
||||
bot-bottle:
|
||||
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
key:
|
||||
provider: gitea
|
||||
forge_token_env: GITEA_DEPLOY_TOKEN # deploy-key admin (push), PRD 0048
|
||||
host_key: "ssh-ed25519 AAAA..."
|
||||
forge: didericis-gitea # ← selects the agent's forge alias
|
||||
---
|
||||
|
||||
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
||||
gitea over SSH for push, token over HTTPS for the API.
|
||||
The `gitea-dev` bottle. Gitea over SSH for push; the API credential and
|
||||
workflow guidance come from the agent's `forge: didericis-gitea` association.
|
||||
````
|
||||
|
||||
**Agent** (`~/.bot-bottle/agents/gitea-helper.md`):
|
||||
@@ -230,11 +228,27 @@ gitea over SSH for push, token over HTTPS for the API.
|
||||
bottle: gitea-dev
|
||||
skills:
|
||||
- init-prd
|
||||
author:
|
||||
name: didericis-claude
|
||||
email: eric+claude@dideric.is
|
||||
forge-accounts:
|
||||
didericis-gitea:
|
||||
url: https://gitea.dideric.is/api/v1
|
||||
auth:
|
||||
type: token
|
||||
token_secret: GITEA_CLAUDE_TOKEN # host env var; value never enters the bottle
|
||||
---
|
||||
|
||||
You help maintain Gitea-hosted projects.
|
||||
````
|
||||
|
||||
`author` populates the bottle's `git config user.name/user.email`. When a
|
||||
selected repo names a `forge` alias, bot-bottle resolves the alias's
|
||||
`token_secret` from the host env into the egress proxy only (never the bottle),
|
||||
adds a scoped, proxy-authenticated route to the Gitea API origin, and appends
|
||||
non-secret forge workflow guidance to the agent's system prompt. Neither the
|
||||
token value nor its `token_secret` name appears in the bottle env or prompt.
|
||||
|
||||
**Egress route fields:**
|
||||
|
||||
| Field | Required | Description |
|
||||
@@ -253,7 +267,7 @@ You help maintain Gitea-hosted projects.
|
||||
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default for manifest routes — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). Agent-provider routes (e.g. `api.anthropic.com`) default to `redact`. |
|
||||
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
|
||||
|
||||
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
|
||||
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `bot-bottle supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
|
||||
|
||||
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
|
||||
|
||||
|
||||
@@ -226,7 +226,7 @@ class AgentProvider(ABC):
|
||||
initial task in a non-interactive (headless) session.
|
||||
|
||||
Called only when ``--prompt`` is passed to
|
||||
``./cli.py start --headless``; the returned args are appended
|
||||
``bot-bottle start --headless``; the returned args are appended
|
||||
after the provider's ``bypass_args`` and ``startup_args``."""
|
||||
|
||||
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||
|
||||
@@ -172,6 +172,10 @@ class BottleCleanupPlan(ABC):
|
||||
"""True iff there is nothing to clean up; the CLI uses this to
|
||||
short-circuit before showing the y/N."""
|
||||
|
||||
@abstractmethod
|
||||
def intersect(self, current: "BottleCleanupPlan") -> "BottleCleanupPlan":
|
||||
"""Resources both displayed to the operator and currently removable."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecResult:
|
||||
@@ -527,7 +531,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
host-appropriate config or commands. Prints to stdout/stderr and
|
||||
returns a shell exit code (0 = nothing to report / success). A
|
||||
backend that needs no host setup prints a short note and returns
|
||||
0. Invoked generically by `./cli.py backend setup [--backend=…]`
|
||||
0. Invoked generically by `bot-bottle backend setup [--backend=…]`
|
||||
so operators can provision any backend without a
|
||||
backend-specific command. Classmethod (like `is_available`) —
|
||||
it's a host query, not per-bottle state."""
|
||||
@@ -544,14 +548,14 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
stderr. When quiet=True returns the status code silently —
|
||||
useful for cheap programmatic checks.
|
||||
|
||||
Invoked by `./cli.py backend status [--backend=…]` (quiet=False)
|
||||
Invoked by `bot-bottle backend status [--backend=…]` (quiet=False)
|
||||
and by is_backend_ready() (caller-controlled)."""
|
||||
|
||||
@classmethod
|
||||
@abstractmethod
|
||||
def teardown(cls) -> int:
|
||||
"""Undo `setup()` — the inverse operation, surfaced as
|
||||
`./cli.py backend teardown [--backend=…]` (uninstall). Symmetric
|
||||
`bot-bottle backend teardown [--backend=…]` (uninstall). Symmetric
|
||||
with setup: where setup is advisory (prints the privileged
|
||||
commands / declarative config to apply), teardown prints the
|
||||
commands / config change to remove the host prerequisites. A
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Shared destructive-cleanup execution and failure accounting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class CleanupError(RuntimeError):
|
||||
"""One or more approved cleanup mutations did not complete."""
|
||||
|
||||
|
||||
class CleanupFailures:
|
||||
"""Attempt every approved mutation, then fail with complete diagnostics."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._messages: list[str] = []
|
||||
|
||||
def run(self, argv: Sequence[str], description: str) -> None:
|
||||
raw_timeout = os.environ.get(
|
||||
"BOT_BOTTLE_CLEANUP_COMMAND_TIMEOUT_SECONDS", "120",
|
||||
)
|
||||
try:
|
||||
timeout = float(raw_timeout)
|
||||
except ValueError:
|
||||
timeout = 120.0
|
||||
try:
|
||||
result = subprocess.run(
|
||||
list(argv), capture_output=True, text=True, check=False,
|
||||
timeout=max(timeout, 1.0),
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
self._messages.append(f"{description}: {exc}")
|
||||
return
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
self._messages.append(
|
||||
f"{description}: {detail or f'exit {result.returncode}'}"
|
||||
)
|
||||
|
||||
def remove_tree(self, path: Path, description: str) -> None:
|
||||
try:
|
||||
shutil.rmtree(path)
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except OSError as exc:
|
||||
self._messages.append(f"{description}: {exc}")
|
||||
|
||||
def record(self, message: str) -> None:
|
||||
self._messages.append(message)
|
||||
|
||||
def raise_if_any(self) -> None:
|
||||
if self._messages:
|
||||
raise CleanupError("; ".join(self._messages))
|
||||
|
||||
|
||||
__all__ = ["CleanupError", "CleanupFailures"]
|
||||
@@ -46,6 +46,22 @@ class DockerBottleCleanupPlan(BottleCleanupPlan):
|
||||
and not self.orphan_state_dirs
|
||||
)
|
||||
|
||||
def intersect(self, current: BottleCleanupPlan) -> "DockerBottleCleanupPlan":
|
||||
if not isinstance(current, DockerBottleCleanupPlan):
|
||||
raise TypeError("cleanup plans must have the same backend type")
|
||||
return DockerBottleCleanupPlan(
|
||||
projects=tuple(x for x in self.projects if x in current.projects),
|
||||
stray_containers=tuple(
|
||||
x for x in self.stray_containers if x in current.stray_containers
|
||||
),
|
||||
stray_networks=tuple(
|
||||
x for x in self.stray_networks if x in current.stray_networks
|
||||
),
|
||||
orphan_state_dirs=tuple(
|
||||
x for x in self.orphan_state_dirs if x in current.orphan_state_dirs
|
||||
),
|
||||
)
|
||||
|
||||
def print(self) -> None:
|
||||
print(file=sys.stderr)
|
||||
for name in self.projects:
|
||||
|
||||
@@ -23,11 +23,12 @@ Active-agent enumeration lives in `backend/docker/enumerate.py`.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
from ...paths import bot_bottle_root
|
||||
from ...log import info, warn
|
||||
from ...log import info
|
||||
from .. import EnumerationError
|
||||
from ..cleanup_control import CleanupFailures
|
||||
from . import util as docker_mod
|
||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from ...bottle_state import bottle_state_dir, is_preserved
|
||||
@@ -36,15 +37,17 @@ from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
|
||||
|
||||
def _list_prefixed_containers() -> list[str]:
|
||||
"""All bot-bottle-prefixed containers, running or stopped."""
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "-a",
|
||||
"--filter", f"name=^{COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Names}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "-a",
|
||||
"--filter", f"name=^{COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Names}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
raise EnumerationError(f"docker ps failed: {exc}") from exc
|
||||
if result.returncode != 0:
|
||||
warn(f"docker ps failed: {result.stderr.strip()}")
|
||||
return []
|
||||
raise EnumerationError(f"docker ps failed: {result.stderr.strip()}")
|
||||
out: list[str] = []
|
||||
for line in (result.stdout or "").splitlines():
|
||||
if not line:
|
||||
@@ -63,15 +66,19 @@ def _list_prefixed_networks() -> list[str]:
|
||||
to a compose project. Compose-managed networks have a
|
||||
`com.docker.compose.project` label; bare ones (from pre-compose
|
||||
code paths) don't."""
|
||||
result = subprocess.run(
|
||||
["docker", "network", "ls",
|
||||
"--filter", f"name={COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Name}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "network", "ls",
|
||||
"--filter", f"name={COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Name}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
raise EnumerationError(f"docker network ls failed: {exc}") from exc
|
||||
if result.returncode != 0:
|
||||
warn(f"docker network ls failed: {result.stderr.strip()}")
|
||||
return []
|
||||
raise EnumerationError(
|
||||
f"docker network ls failed: {result.stderr.strip()}"
|
||||
)
|
||||
out: list[str] = []
|
||||
for line in (result.stdout or "").splitlines():
|
||||
if not line:
|
||||
@@ -120,7 +127,10 @@ def prepare_cleanup() -> DockerBottleCleanupPlan:
|
||||
`enumerate_active_agents()` so the orphan-state-dir bucket
|
||||
doesn't include slugs whose non-docker bottle is still up."""
|
||||
docker_mod.require_docker()
|
||||
projects = list_compose_projects()
|
||||
projects = list_compose_projects(
|
||||
warn_on_error=False,
|
||||
raise_on_error=True,
|
||||
)
|
||||
project_set = set(projects)
|
||||
# Late import to avoid a circular at module-load time —
|
||||
# the backend package's __init__ imports this module.
|
||||
@@ -140,40 +150,30 @@ def cleanup(plan: DockerBottleCleanupPlan) -> None:
|
||||
"""Remove everything in the plan. Projects first (whose `compose
|
||||
down` reaps their containers + networks atomically), then stray
|
||||
legacy resources, then orphan state dirs."""
|
||||
failures = CleanupFailures()
|
||||
for project in plan.projects:
|
||||
info(f"docker compose down ({project})")
|
||||
result = subprocess.run(
|
||||
failures.run(
|
||||
["docker", "compose", "-p", project, "down", "--volumes"],
|
||||
capture_output=True, text=True, check=False,
|
||||
f"docker compose down failed for {project}",
|
||||
)
|
||||
if result.returncode != 0:
|
||||
warn(
|
||||
f"compose down failed for {project}: "
|
||||
f"{result.stderr.strip()}"
|
||||
)
|
||||
|
||||
for name in plan.stray_containers:
|
||||
info(f"removing stray container {name}")
|
||||
subprocess.run(
|
||||
failures.run(
|
||||
["docker", "rm", "-f", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
f"removing stray container {name}",
|
||||
)
|
||||
|
||||
for name in plan.stray_networks:
|
||||
info(f"removing stray network {name}")
|
||||
subprocess.run(
|
||||
failures.run(
|
||||
["docker", "network", "rm", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
f"removing stray network {name}",
|
||||
)
|
||||
|
||||
for identity in plan.orphan_state_dirs:
|
||||
path = bottle_state_dir(identity)
|
||||
info(f"removing orphan state dir {path}")
|
||||
try:
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
except OSError as e:
|
||||
warn(f"failed to remove {path}: {e}")
|
||||
failures.remove_tree(path, f"removing orphan state dir {path}")
|
||||
failures.raise_if_any()
|
||||
|
||||
@@ -74,10 +74,13 @@ def list_compose_projects(
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True, check=False,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
except OSError as exc:
|
||||
# Not only "not found": docker on PATH but not executable by this
|
||||
# user raises PermissionError. Either way the query never ran, so an
|
||||
# enumeration caller must not read the empty result as authoritative.
|
||||
if raise_on_error:
|
||||
raise EnumerationError(
|
||||
"docker compose ls failed: docker not found"
|
||||
f"docker compose ls failed: docker unavailable ({exc})"
|
||||
) from exc
|
||||
return []
|
||||
if result.returncode != 0:
|
||||
|
||||
@@ -74,8 +74,9 @@ def _query_services_by_project() -> dict[str, set[str]]:
|
||||
],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
raise EnumerationError("docker ps failed: docker not found") from exc
|
||||
except OSError as exc:
|
||||
# Missing, or on PATH but not executable by this user (PermissionError).
|
||||
raise EnumerationError(f"docker ps failed: docker unavailable ({exc})") from exc
|
||||
if r.returncode != 0:
|
||||
raise EnumerationError(f"docker ps failed: {r.stderr.strip()}")
|
||||
return _parse_services_by_project(r.stdout or "")
|
||||
|
||||
@@ -8,7 +8,7 @@ pointer, and `status()` reports whether docker is usable.
|
||||
This is intentionally minimal; a richer version (daemon config checks,
|
||||
gVisor/runsc install guidance, rootless-docker hints) is tracked
|
||||
separately. Reached via `DockerBottleBackend.setup` / `.status`, which
|
||||
the generic `./cli.py backend {setup,status}` dispatches to.
|
||||
the generic `bot-bottle backend {setup,status}` dispatches to.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -69,7 +69,7 @@ def teardown() -> int:
|
||||
sys.stderr.write(
|
||||
"Docker backend: nothing to undo — it provisions no privileged host "
|
||||
"state (networks and the gateway are per-launch and are "
|
||||
"removed by `./cli.py cleanup`). Docker itself is left installed.\n"
|
||||
"removed by `bot-bottle cleanup`). Docker itself is left installed.\n"
|
||||
)
|
||||
return 0
|
||||
|
||||
@@ -89,5 +89,5 @@ def status() -> int:
|
||||
runsc = _docker_on_path() and _util.runsc_available()
|
||||
sys.stderr.write(f"gVisor runsc runtime: {'registered' if runsc else 'not registered (optional)'}\n")
|
||||
if not ok:
|
||||
sys.stderr.write("\nRun: ./cli.py backend setup --backend=docker\n")
|
||||
sys.stderr.write("\nRun: bot-bottle backend setup --backend=docker\n")
|
||||
return 0 if ok else 1
|
||||
|
||||
@@ -27,3 +27,11 @@ class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not (self.vm_pids or self.run_dirs)
|
||||
|
||||
def intersect(self, current: BottleCleanupPlan) -> "FirecrackerBottleCleanupPlan":
|
||||
if not isinstance(current, FirecrackerBottleCleanupPlan):
|
||||
raise TypeError("cleanup plans must have the same backend type")
|
||||
return FirecrackerBottleCleanupPlan(
|
||||
vm_pids=tuple(x for x in self.vm_pids if x in current.vm_pids),
|
||||
run_dirs=tuple(x for x in self.run_dirs if x in current.run_dirs),
|
||||
)
|
||||
|
||||
@@ -11,10 +11,9 @@ Reaps *orphans* only — resources with no live VM behind them:
|
||||
— a VMM left lingering after its dir was removed.
|
||||
|
||||
A run dir with a *live* firecracker process is a running bottle and is
|
||||
left strictly alone: it is neither killed nor removed. (The backend's
|
||||
`enumerate_active` registry is still a stub — #354 — so a live process
|
||||
is the only reliable "this bottle is in use" signal we have. Once the
|
||||
registry lands, registry-orphaned-but-running VMs can be reaped too.)
|
||||
left strictly alone: it is neither killed nor removed. Active-agent
|
||||
enumeration uses this same process snapshot, so cleanup and generic
|
||||
backend consumers agree about which bottles are running.
|
||||
|
||||
TAP slots free themselves (the flock drops when the launcher exits), so
|
||||
there is nothing to reclaim there.
|
||||
@@ -22,15 +21,16 @@ there is nothing to reclaim there.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import info
|
||||
from .. import EnumerationError
|
||||
from . import util
|
||||
from ..cleanup_control import CleanupError, CleanupFailures
|
||||
from . import lifecycle_lock, util
|
||||
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ def _run_root() -> Path:
|
||||
return util.cache_dir() / "run"
|
||||
|
||||
|
||||
def _run_dir_of(cmd: str, run_root: Path) -> Path | None:
|
||||
def _run_dir_of(args: Sequence[str], run_root: Path) -> Path | None:
|
||||
"""The bottle run dir a firecracker cmdline belongs to, or None.
|
||||
|
||||
A bottle VM is launched with `--config-file <run_root>/<slug>/config.json`,
|
||||
@@ -46,15 +46,35 @@ def _run_dir_of(cmd: str, run_root: Path) -> Path | None:
|
||||
the run root. Anything else (a builder VM, the infra VM elsewhere) is
|
||||
not ours to reap here.
|
||||
"""
|
||||
toks = cmd.split()
|
||||
for i, tok in enumerate(toks):
|
||||
if tok == "--config-file" and i + 1 < len(toks):
|
||||
parent = Path(toks[i + 1]).parent
|
||||
for i, arg in enumerate(args):
|
||||
if arg == "--config-file" and i + 1 < len(args):
|
||||
parent = Path(args[i + 1]).parent
|
||||
if parent.parent == run_root:
|
||||
return parent
|
||||
return None
|
||||
|
||||
|
||||
def _decode_cmdline(raw: bytes) -> tuple[str, ...]:
|
||||
"""Decode Linux's NUL-delimited argv without losing embedded spaces."""
|
||||
return tuple(
|
||||
value.decode(errors="surrogateescape")
|
||||
for value in raw.split(b"\0") if value
|
||||
)
|
||||
|
||||
|
||||
def _process_args(pid: int) -> tuple[str, ...] | None:
|
||||
"""Read one process's lossless argv, or None when it exited meanwhile."""
|
||||
try:
|
||||
raw = Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not inspect Firecracker pid {pid}: {exc}"
|
||||
) from exc
|
||||
return _decode_cmdline(raw)
|
||||
|
||||
|
||||
def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
|
||||
"""Inspect running firecracker VMs under ``run_root``.
|
||||
|
||||
@@ -65,7 +85,7 @@ def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pgrep", "-a", "firecracker"],
|
||||
["pgrep", "firecracker"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
@@ -83,14 +103,14 @@ def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
|
||||
live: set[str] = set()
|
||||
orphan_pids: list[int] = []
|
||||
for line in result.stdout.splitlines():
|
||||
parts = line.split(None, 1)
|
||||
if len(parts) != 2:
|
||||
continue
|
||||
try:
|
||||
pid = int(parts[0])
|
||||
pid = int(line.strip())
|
||||
except ValueError:
|
||||
continue
|
||||
run_dir = _run_dir_of(parts[1], run_root)
|
||||
args = _process_args(pid)
|
||||
if args is None:
|
||||
continue
|
||||
run_dir = _run_dir_of(args, run_root)
|
||||
if run_dir is None:
|
||||
continue
|
||||
if run_dir.is_dir():
|
||||
@@ -126,12 +146,54 @@ def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
|
||||
|
||||
|
||||
def cleanup(plan: FirecrackerBottleCleanupPlan) -> None:
|
||||
for pid in plan.vm_pids:
|
||||
"""Revalidate the preview under the launch lock, then remove its survivors."""
|
||||
with lifecycle_lock.hold():
|
||||
fresh = prepare_cleanup()
|
||||
approved_pids = set(plan.vm_pids).intersection(fresh.vm_pids)
|
||||
approved_dirs = set(plan.run_dirs).intersection(fresh.run_dirs)
|
||||
failures = CleanupFailures()
|
||||
for pid in sorted(approved_pids):
|
||||
try:
|
||||
_terminate_orphan(pid, _run_root())
|
||||
except CleanupError as exc:
|
||||
failures.record(str(exc))
|
||||
for path in sorted(approved_dirs):
|
||||
info(f"rm -rf {path}")
|
||||
failures.remove_tree(Path(path), f"removing Firecracker run dir {path}")
|
||||
failures.raise_if_any()
|
||||
|
||||
|
||||
def _terminate_orphan(pid: int, run_root: Path) -> None:
|
||||
"""Signal exactly the process identity that still owns an orphan config."""
|
||||
try:
|
||||
pidfd = os.pidfd_open(pid)
|
||||
except ProcessLookupError:
|
||||
return
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not pin Firecracker pid {pid} for cleanup: {exc}"
|
||||
) from exc
|
||||
try:
|
||||
try:
|
||||
raw = Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not revalidate Firecracker pid {pid}: {exc}"
|
||||
) from exc
|
||||
args = _decode_cmdline(raw)
|
||||
run_dir = _run_dir_of(args, run_root)
|
||||
if run_dir is None or run_dir.is_dir():
|
||||
return
|
||||
info(f"kill firecracker VM pid {pid}")
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
signal.pidfd_send_signal(pidfd, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
for path in plan.run_dirs:
|
||||
info(f"rm -rf {path}")
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
return
|
||||
except OSError as exc:
|
||||
raise CleanupError(
|
||||
f"could not signal Firecracker pid {pid}: {exc}"
|
||||
) from exc
|
||||
finally:
|
||||
os.close(pidfd)
|
||||
|
||||
@@ -41,6 +41,8 @@ from ... import resources
|
||||
from ...log import die, info
|
||||
from . import util
|
||||
|
||||
ARTIFACT_HTTP_TIMEOUT_SECONDS = 30.0
|
||||
|
||||
# Bump if the on-disk artifact *format* changes (compression, layout) so a new
|
||||
# scheme can't collide with a cached/published artifact of the old one.
|
||||
_ARTIFACT_FORMAT = "1"
|
||||
@@ -164,7 +166,9 @@ def _download(url: str, dest: Path) -> None:
|
||||
"""Stream `url` to `dest` (atomic via a `.part` sibling)."""
|
||||
tmp = dest.with_suffix(dest.suffix + ".part")
|
||||
try:
|
||||
with urllib.request.urlopen(_open(url)) as resp, open(tmp, "wb") as out:
|
||||
with urllib.request.urlopen(
|
||||
_open(url), timeout=ARTIFACT_HTTP_TIMEOUT_SECONDS,
|
||||
) as resp, open(tmp, "wb") as out:
|
||||
shutil.copyfileobj(resp, out, _CHUNK)
|
||||
except urllib.error.HTTPError as e:
|
||||
tmp.unlink(missing_ok=True)
|
||||
|
||||
@@ -204,7 +204,7 @@ def boot_vm(
|
||||
Records the PID."""
|
||||
if not netpool.tap_present(slot.iface):
|
||||
die(f"infra link {slot.iface} not present.\n"
|
||||
f" ./cli.py backend setup --backend=firecracker")
|
||||
f" bot-bottle backend setup --backend=firecracker")
|
||||
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
rootfs = run_dir / "rootfs.ext4"
|
||||
|
||||
@@ -108,7 +108,7 @@ def verify_isolation(private_key: Path, guest_ip: str) -> None:
|
||||
die(f"ISOLATION FAILURE: the VM reached the host canary "
|
||||
f"{canary_ip}:{canary_port}. The egress boundary is not in "
|
||||
f"force — refusing to run the agent (fail-closed). Verify the "
|
||||
f"nft table with: ./cli.py backend setup --backend=firecracker")
|
||||
f"nft table with: bot-bottle backend setup --backend=firecracker")
|
||||
if result.returncode == 2:
|
||||
die("isolation probe inconclusive: the guest has no python3/bash/nc "
|
||||
"to run the connectivity test. Refusing to continue "
|
||||
|
||||
@@ -46,7 +46,7 @@ from ...log import die, info, warn
|
||||
from ...supervisor.types import SUPERVISE_PORT
|
||||
from ..docker.egress import EGRESS_PORT
|
||||
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
||||
from . import firecracker_vm, image_builder, isolation_probe, netpool, util
|
||||
from . import firecracker_vm, image_builder, isolation_probe, lifecycle_lock, netpool, util
|
||||
from .bottle import FirecrackerBottle
|
||||
from .bottle_plan import FirecrackerBottlePlan
|
||||
from ...orchestrator.store.config_store import resolve_teardown_timeout
|
||||
@@ -164,25 +164,29 @@ def launch(
|
||||
)
|
||||
|
||||
# Step 6: build the per-bottle rootfs + SSH key, then boot.
|
||||
run_dir = util.cache_dir() / "run" / plan.slug
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
|
||||
# doesn't leak. Registered before vm.terminate below so it runs *after*
|
||||
# it (ExitStack is LIFO): the VM is gone before we rm its rootfs.
|
||||
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
|
||||
rootfs = run_dir / "rootfs.ext4"
|
||||
util.build_rootfs_ext4(agent_base, rootfs)
|
||||
private_key, pubkey = util.generate_keypair(run_dir)
|
||||
# Cleanup takes the same lock while refreshing its process snapshot.
|
||||
# Hold it until the VMM exists so a newly-created run dir can never be
|
||||
# mistaken for an orphan in the build-before-boot window.
|
||||
with lifecycle_lock.hold():
|
||||
run_dir = util.cache_dir() / "run" / plan.slug
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
|
||||
# doesn't leak. Registered before vm.terminate below so it runs
|
||||
# *after* it (ExitStack is LIFO).
|
||||
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
|
||||
rootfs = run_dir / "rootfs.ext4"
|
||||
util.build_rootfs_ext4(agent_base, rootfs)
|
||||
private_key, pubkey = util.generate_keypair(run_dir)
|
||||
|
||||
vm = firecracker_vm.boot(
|
||||
name=plan.container_name,
|
||||
rootfs=rootfs,
|
||||
tap=slot.iface,
|
||||
guest_ip=slot.guest_ip,
|
||||
host_ip=slot.host_ip,
|
||||
pubkey=pubkey,
|
||||
run_dir=run_dir,
|
||||
)
|
||||
vm = firecracker_vm.boot(
|
||||
name=plan.container_name,
|
||||
rootfs=rootfs,
|
||||
tap=slot.iface,
|
||||
guest_ip=slot.guest_ip,
|
||||
host_ip=slot.host_ip,
|
||||
pubkey=pubkey,
|
||||
run_dir=run_dir,
|
||||
)
|
||||
stack.callback(vm.terminate)
|
||||
firecracker_vm.wait_for_ssh(vm, private_key)
|
||||
persist_env_var_secret(private_key, slot.guest_ip, ctx.env_var_secret)
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Serialize Firecracker run-directory creation with orphan cleanup."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator
|
||||
|
||||
from . import util
|
||||
|
||||
|
||||
def _lock_path() -> Path:
|
||||
return util.cache_dir() / "run.lifecycle.lock"
|
||||
|
||||
|
||||
@contextmanager
|
||||
def hold() -> Generator[None]:
|
||||
"""Exclude cleanup while a launch directory lacks a visible VMM."""
|
||||
path = _lock_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("a", encoding="utf-8") as handle:
|
||||
fcntl.flock(handle, fcntl.LOCK_EX)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
fcntl.flock(handle, fcntl.LOCK_UN)
|
||||
|
||||
|
||||
__all__ = ["hold"]
|
||||
@@ -3,7 +3,7 @@ config renderers (shell command + NixOS module) shown to operators.
|
||||
|
||||
The Firecracker backend needs a privileged one-time network setup:
|
||||
a pool of point-to-point TAP devices (owned by the invoking user, so
|
||||
`./cli.py start` never needs root) and a dedicated nftables table that
|
||||
`bot-bottle start` never needs root) and a dedicated nftables table that
|
||||
isolates every VM. The pool parameters live in exactly one place —
|
||||
`netpool.defaults.env`, a plain KEY=VALUE file next to this module —
|
||||
and every consumer reads *that*: this module (below), the shell script
|
||||
@@ -177,14 +177,20 @@ def gw_slot() -> Slot:
|
||||
# --- fail-closed verification ---------------------------------------
|
||||
|
||||
def _run_ok(argv: list[str]) -> bool:
|
||||
"""Run a probe command, treating a missing binary as failure
|
||||
"""Run a probe command, treating an unavailable binary as failure
|
||||
(rather than crashing) so callers can stay fail-closed."""
|
||||
try:
|
||||
return subprocess.run(
|
||||
argv, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
).returncode == 0
|
||||
except FileNotFoundError:
|
||||
except OSError:
|
||||
# Not only "missing". A name on PATH that isn't executable by this
|
||||
# user raises PermissionError, and CPython reports that EACCES in
|
||||
# preference to the ENOENT from the other PATH entries — which is
|
||||
# how `doctor` came to die with a traceback on a fresh macOS
|
||||
# account. Any OSError means the probe couldn't run, which for a
|
||||
# fail-closed check is indistinguishable from "not present".
|
||||
return False
|
||||
|
||||
|
||||
@@ -244,7 +250,9 @@ def overlapping_routes() -> list[RouteConflict]:
|
||||
["ip", "-json", "route", "show", "table", "all"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except FileNotFoundError:
|
||||
except OSError:
|
||||
# Missing, or present-but-not-executable for this user; either way
|
||||
# there are no routes we can enumerate. See _run_ok.
|
||||
return []
|
||||
if proc.returncode != 0 or not proc.stdout.strip():
|
||||
return []
|
||||
@@ -307,11 +315,11 @@ def allocate(slug: str) -> tuple[Slot, IO[str]]:
|
||||
return s, handle
|
||||
die(f"Firecracker TAP pool exhausted ({pool_size()} slots, all in "
|
||||
f"use). Stop a running bottle or raise BOT_BOTTLE_FC_POOL_SIZE "
|
||||
f"and re-run `./cli.py backend setup --backend=firecracker`.")
|
||||
f"and re-run `bot-bottle backend setup --backend=firecracker`.")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
# --- config renderers (shown by `./cli.py backend setup`) -----------
|
||||
# --- config renderers (shown by `bot-bottle backend setup`) -----------
|
||||
|
||||
# The persistent unit is the portable install: the same systemd oneshot
|
||||
# on every systemd distro (Debian/Ubuntu/Fedora/RHEL/Arch/…).
|
||||
|
||||
@@ -34,6 +34,7 @@ from pathlib import Path
|
||||
from . import infra_artifact, infra_vm, util
|
||||
|
||||
_CHUNK = 1 << 20
|
||||
_REGISTRY_HTTP_TIMEOUT_SECONDS = 30.0
|
||||
|
||||
_GZ_NAME = "rootfs.ext4.gz"
|
||||
_SHA_NAME = "rootfs.ext4.gz.sha256"
|
||||
@@ -91,7 +92,9 @@ def _put(url: str, body: "bytes | Path", token: str) -> None:
|
||||
req.add_header("Authorization", f"token {token}")
|
||||
req.add_header("Content-Type", "application/octet-stream")
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
with urllib.request.urlopen(
|
||||
req, timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
) as resp:
|
||||
print(f" uploaded {url} (HTTP {resp.status})")
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 409:
|
||||
@@ -112,7 +115,9 @@ def _delete(url: str, token: str) -> None:
|
||||
if token:
|
||||
req.add_header("Authorization", f"token {token}")
|
||||
try:
|
||||
with urllib.request.urlopen(req):
|
||||
with urllib.request.urlopen(
|
||||
req, timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
):
|
||||
pass
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code != 404:
|
||||
@@ -151,7 +156,10 @@ def _try_download_published(role: str, role_dir: Path) -> str | None:
|
||||
version = _role_version(role)
|
||||
sha_url = infra_artifact.artifact_url(version, _SHA_NAME, role=role)
|
||||
try:
|
||||
with urllib.request.urlopen(infra_artifact._open(sha_url)):
|
||||
with urllib.request.urlopen(
|
||||
infra_artifact._open(sha_url),
|
||||
timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
):
|
||||
pass
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
@@ -195,7 +203,10 @@ def _publish_bundle(role: str, role_dir: Path, token: str) -> str:
|
||||
# present, a re-publish is a no-op. Otherwise clear any partial upload left
|
||||
# by an interrupted prior attempt and upload the complete set.
|
||||
try:
|
||||
with urllib.request.urlopen(infra_artifact._open(sha_url)) as resp:
|
||||
with urllib.request.urlopen(
|
||||
infra_artifact._open(sha_url),
|
||||
timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
) as resp:
|
||||
remote_sha = resp.read().decode("utf-8").split()[0].strip().lower()
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code != 404:
|
||||
|
||||
@@ -8,7 +8,7 @@ bundled setup script. `status()` reports what's present, including
|
||||
whether the pool range collides with an existing route.
|
||||
|
||||
Called through `FirecrackerBottleBackend.setup` / `.status`, which the
|
||||
generic `./cli.py backend {setup,status}` command dispatches to.
|
||||
generic `bot-bottle backend {setup,status}` command dispatches to.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -34,7 +34,7 @@ _UNIT_PATH = Path("/etc/systemd/system") / netpool.SYSTEMD_UNIT
|
||||
|
||||
def _owner() -> str:
|
||||
# Under `sudo`, USER is root but SUDO_USER is the real invoker — the
|
||||
# TAPs must be owned by them so `./cli.py start` stays rootless.
|
||||
# TAPs must be owned by them so `bot-bottle start` stays rootless.
|
||||
return os.environ.get("SUDO_USER") or os.environ.get("USER") or "youruser"
|
||||
|
||||
|
||||
@@ -161,7 +161,7 @@ def _setup_systemd() -> None:
|
||||
if rc == 0:
|
||||
sys.stderr.write(
|
||||
f"Installed and started {netpool.SYSTEMD_UNIT}. Verify with "
|
||||
f"`./cli.py backend status --backend=firecracker`.\n"
|
||||
f"`bot-bottle backend status --backend=firecracker`.\n"
|
||||
)
|
||||
else:
|
||||
sys.stderr.write(
|
||||
@@ -181,7 +181,7 @@ def _setup_systemd() -> None:
|
||||
)
|
||||
sys.stderr.write(
|
||||
f"\n(Or re-run this as root to install it directly: "
|
||||
f"sudo ./cli.py backend setup --backend=firecracker)\n"
|
||||
f"sudo bot-bottle backend setup --backend=firecracker)\n"
|
||||
)
|
||||
|
||||
|
||||
@@ -334,7 +334,7 @@ def status() -> int:
|
||||
sys.stderr.write(f"range overlap: none (base {netpool.ip_base()})\n")
|
||||
_report_persistence()
|
||||
if not ok:
|
||||
sys.stderr.write("\nRun: ./cli.py backend setup --backend=firecracker\n")
|
||||
sys.stderr.write("\nRun: bot-bottle backend setup --backend=firecracker\n")
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ generation.
|
||||
|
||||
The privileged network setup (TAP pool + nft table) is a one-time
|
||||
operator step — see `netpool.py`, `scripts/firecracker-netpool.sh`,
|
||||
and `./cli.py backend setup --backend=firecracker`.
|
||||
and `bot-bottle backend setup --backend=firecracker`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -132,18 +132,18 @@ def _require_network_pool() -> None:
|
||||
f"{netpool.pool_size()} slots) overlaps existing routes: "
|
||||
f"{detail}. This can shadow or be shadowed by that route; "
|
||||
f"set BOT_BOTTLE_FC_IP_BASE to a free range and re-run "
|
||||
f"./cli.py backend setup --backend=firecracker.")
|
||||
f"bot-bottle backend setup --backend=firecracker.")
|
||||
missing = netpool.missing_taps()
|
||||
if missing:
|
||||
die(f"network pool incomplete — missing TAP devices: "
|
||||
f"{', '.join(missing)}.\n ./cli.py backend setup --backend=firecracker")
|
||||
f"{', '.join(missing)}.\n bot-bottle backend setup --backend=firecracker")
|
||||
if shutil.which("nft") is not None and not netpool.nft_table_present():
|
||||
# nft is queryable and says the table is absent — that's a
|
||||
# definite, catchable misconfiguration; fail early.
|
||||
warn(f"isolation table `inet {netpool.NFT_TABLE}` not found via nft. "
|
||||
"If this is a permissions issue it will be re-checked "
|
||||
"empirically after boot; otherwise run: "
|
||||
"./cli.py backend setup --backend=firecracker")
|
||||
"bot-bottle backend setup --backend=firecracker")
|
||||
|
||||
|
||||
# --- rootfs pipeline (rootless) -------------------------------------
|
||||
|
||||
@@ -43,7 +43,7 @@ class Freezer(ABC):
|
||||
|
||||
Calls _freeze for the backend-specific snapshot, then writes the
|
||||
committed image reference to per-bottle state and marks the bottle
|
||||
preserved so the next `./cli.py resume` boots from the snapshot.
|
||||
preserved so the next `bot-bottle resume` boots from the snapshot.
|
||||
|
||||
Raises CommitCancelled if the user declines an interactive
|
||||
confirmation prompt (e.g. the macos-container stop prompt).
|
||||
@@ -51,7 +51,7 @@ class Freezer(ABC):
|
||||
image_ref = self._freeze(agent)
|
||||
write_committed_image(agent.slug, image_ref)
|
||||
mark_preserved(agent.slug)
|
||||
info(f"to resume from this snapshot: ./cli.py resume {agent.slug}")
|
||||
info(f"to resume from this snapshot: bot-bottle resume {agent.slug}")
|
||||
self._export_hint(agent.slug, image_ref)
|
||||
|
||||
@abstractmethod
|
||||
|
||||
@@ -25,3 +25,11 @@ class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not self.containers and not self.networks
|
||||
|
||||
def intersect(self, current: BottleCleanupPlan) -> "MacosContainerBottleCleanupPlan":
|
||||
if not isinstance(current, MacosContainerBottleCleanupPlan):
|
||||
raise TypeError("cleanup plans must have the same backend type")
|
||||
return MacosContainerBottleCleanupPlan(
|
||||
containers=tuple(x for x in self.containers if x in current.containers),
|
||||
networks=tuple(x for x in self.networks if x in current.networks),
|
||||
)
|
||||
|
||||
@@ -4,7 +4,9 @@ from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...log import info, warn
|
||||
from .. import EnumerationError
|
||||
from ..cleanup_control import CleanupFailures
|
||||
from ...log import info
|
||||
from . import util as container_mod
|
||||
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
|
||||
|
||||
@@ -19,8 +21,8 @@ def _list_prefixed_containers() -> list[str]:
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
warn(f"container list failed: {result.stderr.strip()}")
|
||||
return []
|
||||
detail = result.stderr.strip() or f"exit {result.returncode}"
|
||||
raise EnumerationError(f"container list failed: {detail}")
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX)
|
||||
@@ -35,7 +37,8 @@ def _list_prefixed_networks() -> list[str]:
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
detail = result.stderr.strip() or f"exit {result.returncode}"
|
||||
raise EnumerationError(f"container network list failed: {detail}")
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX)
|
||||
@@ -51,19 +54,17 @@ def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
|
||||
|
||||
|
||||
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
|
||||
failures = CleanupFailures()
|
||||
for name in plan.containers:
|
||||
info(f"container delete --force {name}")
|
||||
subprocess.run(
|
||||
failures.run(
|
||||
["container", "delete", "--force", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
f"deleting container {name}",
|
||||
)
|
||||
for name in plan.networks:
|
||||
info(f"container network delete {name}")
|
||||
subprocess.run(
|
||||
failures.run(
|
||||
["container", "network", "delete", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
f"deleting network {name}",
|
||||
)
|
||||
failures.raise_if_any()
|
||||
|
||||
@@ -5,7 +5,7 @@ Like Docker, this backend needs no privileged network-pool provisioning
|
||||
running. `setup()` points at the install/`container system start` steps;
|
||||
`status()` reports readiness. Reached via
|
||||
`MacosContainerBottleBackend.setup` / `.status`, dispatched from the
|
||||
generic `./cli.py backend {setup,status}`.
|
||||
generic `bot-bottle backend {setup,status}`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -75,5 +75,5 @@ def status() -> int:
|
||||
if not _service_running():
|
||||
ok = False
|
||||
if not ok:
|
||||
sys.stderr.write("\nRun: ./cli.py backend setup --backend=macos-container\n")
|
||||
sys.stderr.write("\nRun: bot-bottle backend setup --backend=macos-container\n")
|
||||
return 0 if ok else 1
|
||||
|
||||
@@ -689,6 +689,13 @@ def pinned_local_image_ref(ref: str) -> str:
|
||||
nested-containers layer. A content-derived tag prevents another concurrent
|
||||
build from moving the provider's ordinary ``:latest`` tag between those
|
||||
two builds.
|
||||
|
||||
Unlike Docker, `container image tag` only accepts ``image-name[:tag]`` as
|
||||
its source and rejects a bare image ID ("cannot specify 64 byte hex string
|
||||
as reference"), so the mutable ``ref`` is what gets tagged here. The
|
||||
post-tag inspection below is what keeps that fail-closed: if ``ref`` moved
|
||||
between the two commands, the new tag will not resolve to the ID this call
|
||||
derived its name from.
|
||||
"""
|
||||
image = image_id(ref)
|
||||
digest = image.removeprefix("sha256:")
|
||||
@@ -701,14 +708,14 @@ def pinned_local_image_ref(ref: str) -> str:
|
||||
repository = repository[:last_colon]
|
||||
pinned_ref = f"{repository}:sha256-{digest}"
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "image", "tag", image, pinned_ref],
|
||||
[_CONTAINER, "image", "tag", ref, pinned_ref],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"could not tag exact local image {image}: "
|
||||
f"could not tag exact local image {image} via {ref!r}: "
|
||||
f"{(result.stderr or result.stdout or '').strip() or '<no detail>'}"
|
||||
)
|
||||
if image_id(pinned_ref) != image:
|
||||
|
||||
@@ -118,7 +118,7 @@ class BottlePreparationPlanner:
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=provision,
|
||||
egress_plan=prepare_egress(bottle, slug, provision),
|
||||
egress_plan=prepare_egress(manifest, slug, provision),
|
||||
git_gate_plan=prepare_git_gate(bottle, slug),
|
||||
supervise_plan=prepare_supervise(bottle, slug),
|
||||
)
|
||||
|
||||
@@ -63,12 +63,23 @@ def provision_git_gate(
|
||||
transport.exec(["chmod", "+x", "/etc/git-gate/access-hook"])
|
||||
creds = _creds_dir(bottle_id)
|
||||
transport.exec(["mkdir", "-p", creds])
|
||||
transport.exec(["chmod", "700", creds])
|
||||
credential_paths: list[str] = []
|
||||
for u in plan.upstreams:
|
||||
if u.identity_file:
|
||||
transport.cp_into(u.identity_file, f"{creds}/{u.name}-key")
|
||||
key_path = f"{creds}/{u.name}-key"
|
||||
transport.cp_into(u.identity_file, key_path)
|
||||
credential_paths.append(key_path)
|
||||
known_hosts = str(u.known_hosts_file)
|
||||
if known_hosts and known_hosts != ".":
|
||||
transport.cp_into(known_hosts, f"{creds}/{u.name}-known_hosts")
|
||||
known_hosts_path = f"{creds}/{u.name}-known_hosts"
|
||||
transport.cp_into(known_hosts, known_hosts_path)
|
||||
credential_paths.append(known_hosts_path)
|
||||
# Copy-mode behavior differs across Docker, Apple Container, and SSH.
|
||||
# Apply the security contract inside the gateway so every backend produces
|
||||
# the same private credential namespace.
|
||||
if credential_paths:
|
||||
transport.exec(["chmod", "600", *credential_paths])
|
||||
# Init the bare repos + per-repo credential config for this namespace.
|
||||
script = git_gate_render_provision(bottle_id, plan.upstreams)
|
||||
transport.exec(["sh", "-c", script])
|
||||
|
||||
@@ -24,10 +24,11 @@ from ..bottle_state import (
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from ..egress import Egress, EgressPlan
|
||||
from ..egress import Egress, EgressPlan, egress_forge_routes
|
||||
from ..git_gate import GitGate, GitGatePlan
|
||||
from ..log import die
|
||||
from ..manifest import Manifest, ManifestBottle
|
||||
from ..manifest.forge import render_forge_guidance
|
||||
from ..supervisor.plan import SupervisePlan
|
||||
from ..orchestrator.supervisor import Supervisor
|
||||
from ..util import slugify
|
||||
@@ -71,12 +72,21 @@ def write_launch_metadata(
|
||||
|
||||
def prepare_agent_state_dir(slug: str, manifest: Manifest) -> tuple[Path, Path]:
|
||||
"""Create the agent state subdir, write the prompt file.
|
||||
Returns (agent_dir, prompt_file)."""
|
||||
Returns (agent_dir, prompt_file).
|
||||
|
||||
For repositories associated with a forge, appends generated, non-secret
|
||||
provider-specific workflow guidance to the prompt (PRD
|
||||
0082). The guidance carries neither the
|
||||
token value nor its `token_secret` name."""
|
||||
agent = manifest.agent
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
prompt_file.write_text(agent.prompt or "")
|
||||
prompt = agent.prompt or ""
|
||||
guidance = render_forge_guidance(manifest.forge_associations)
|
||||
if guidance:
|
||||
prompt = f"{prompt.rstrip()}\n\n{guidance}" if prompt.strip() else guidance
|
||||
prompt_file.write_text(prompt)
|
||||
prompt_file.chmod(0o600)
|
||||
return agent_dir, prompt_file
|
||||
|
||||
@@ -88,11 +98,18 @@ def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
|
||||
|
||||
|
||||
def prepare_egress(
|
||||
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
|
||||
manifest: Manifest, slug: str, provision: AgentProvisionPlan,
|
||||
) -> EgressPlan:
|
||||
"""Build the egress plan, adding a scoped, proxy-held Gitea API route for
|
||||
each forge alias referenced by a selected git-gate repo (PRD
|
||||
0082). The token is resolved from the host
|
||||
env at launch and never enters the bottle."""
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
|
||||
forge_routes = egress_forge_routes(manifest.forge_associations)
|
||||
return Egress().prepare(
|
||||
manifest.bottle, slug, egress_dir, provision.egress_routes, forge_routes,
|
||||
)
|
||||
|
||||
|
||||
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
|
||||
|
||||
@@ -86,7 +86,7 @@ def _print_vm_install_instructions() -> None:
|
||||
info("Then start the service: container system start")
|
||||
else:
|
||||
info("Install Firecracker: https://github.com/firecracker-microvm/firecracker/releases")
|
||||
info("Configure the host: ./cli.py backend setup")
|
||||
info("Configure the host: bot-bottle backend setup")
|
||||
|
||||
|
||||
def _auto_select_backend(prompt: bool = True) -> str:
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
"""`backend` CLI command — generic host setup/status across backends.
|
||||
|
||||
`./cli.py backend setup [--backend=NAME]` provisions (or points at how
|
||||
`bot-bottle backend setup [--backend=NAME]` provisions (or points at how
|
||||
to provision) the chosen backend's one-time host prerequisites.
|
||||
`./cli.py backend status [--backend=NAME]` reports readiness.
|
||||
`./cli.py backend teardown [--backend=NAME]` undoes setup (uninstall).
|
||||
`bot-bottle backend status [--backend=NAME]` reports readiness.
|
||||
`bot-bottle backend teardown [--backend=NAME]` undoes setup (uninstall).
|
||||
|
||||
All dispatch to the backend's `setup()` / `status()` / `teardown()`
|
||||
classmethods, so there are no backend-specific commands — swapping
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""cleanup: stop and remove all orphaned bot-bottle resources.
|
||||
|
||||
Walks every registered backend (docker, firecracker, macos-container)
|
||||
so a single `./cli.py cleanup` reaps every backend's leftovers — a
|
||||
so a single `bot-bottle cleanup` reaps every backend's leftovers — a
|
||||
firecracker bottle's VM processes and run dirs won't survive a
|
||||
docker-only cleanup pass (issue addressed alongside #77).
|
||||
|
||||
@@ -22,6 +22,7 @@ from __future__ import annotations
|
||||
import sys
|
||||
|
||||
from ...backend import get_bottle_backend, has_backend, known_backend_names
|
||||
from ...backend.cleanup_control import CleanupError
|
||||
from ...log import info
|
||||
from ...util import read_tty_line
|
||||
|
||||
@@ -52,10 +53,20 @@ def cmd_cleanup(_argv: list[str]) -> int:
|
||||
info("cleanup: skipped")
|
||||
return 0
|
||||
|
||||
for name, backend, plan in prepared:
|
||||
if plan.empty:
|
||||
# Confirmation authorizes a fresh authoritative snapshot, not blind use of
|
||||
# identities that may have changed while the operator reviewed the preview.
|
||||
failures: list[str] = []
|
||||
for name, backend, displayed in prepared:
|
||||
current = backend.prepare_cleanup()
|
||||
approved = displayed.intersect(current)
|
||||
if approved.empty:
|
||||
continue
|
||||
backend.cleanup(plan)
|
||||
try:
|
||||
backend.cleanup(approved)
|
||||
except CleanupError as exc:
|
||||
failures.append(f"{name}: {exc}")
|
||||
if failures:
|
||||
raise CleanupError("cleanup incomplete: " + "; ".join(failures))
|
||||
info("cleanup: done")
|
||||
return 0
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ Docker bottles are committed to a local Docker image. Macos-container
|
||||
bottles are exported and rebuilt as a local Apple Container image.
|
||||
Firecracker bottles stream the guest rootfs out over SSH and rebuild a
|
||||
local Docker image. The resulting reference is stored in per-bottle
|
||||
state so the next `./cli.py resume <slug>` boots from the snapshot
|
||||
state so the next `bot-bottle resume <slug>` boots from the snapshot
|
||||
instead of rebuilding from the Dockerfile.
|
||||
"""
|
||||
|
||||
@@ -37,7 +37,7 @@ def cmd_commit(argv: list[str]) -> int:
|
||||
if slug is None:
|
||||
active = enumerate_active_agents()
|
||||
if not active:
|
||||
die("no active bottles; start one with `./cli.py start`")
|
||||
die("no active bottles; start one with `bot-bottle start`")
|
||||
choices = [a.slug for a in active]
|
||||
slug = tui.filter_select(choices, title="Select bottle to commit")
|
||||
if slug is None:
|
||||
|
||||
@@ -8,7 +8,7 @@ override and transcript snapshot under the same state dir.
|
||||
|
||||
Use case: an interrupted or preserved bottle needs to be relaunched;
|
||||
the operator runs
|
||||
./cli.py resume <identity>
|
||||
bot-bottle resume <identity>
|
||||
to bring up the replacement from the recorded state.
|
||||
"""
|
||||
|
||||
|
||||
@@ -128,7 +128,7 @@ def cmd_start(argv: list[str]) -> int:
|
||||
if not manifest.all_agent_names:
|
||||
print(
|
||||
"bot-bottle: no agents defined. "
|
||||
"Add an agent to ~/.bot-bottle/agents/ or ./bot-bottle/agents/ to get started.",
|
||||
"Add an agent to ~/.bot-bottle/agents/ to get started.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
@@ -203,19 +203,19 @@ def _start_headless(
|
||||
if not os.isatty(stdin_fd):
|
||||
die(
|
||||
"--headless requires a PTY on stdin; run via:\n"
|
||||
" script -q /dev/null ./cli.py start ..."
|
||||
" script -q /dev/null bot-bottle start ..."
|
||||
)
|
||||
|
||||
agent_name = args.name
|
||||
if not agent_name:
|
||||
die("--headless requires an agent name: ./cli.py start <agent> --headless")
|
||||
die("--headless requires an agent name: bot-bottle start <agent> --headless")
|
||||
manifest.require_agent(agent_name) # raises ManifestError if unknown
|
||||
|
||||
prompt = args.prompt
|
||||
if not prompt:
|
||||
die(
|
||||
"--headless requires --prompt: "
|
||||
"./cli.py start <agent> --headless --prompt 'Do the thing'"
|
||||
"bot-bottle start <agent> --headless --prompt 'Do the thing'"
|
||||
)
|
||||
|
||||
if args.bottle:
|
||||
@@ -319,9 +319,9 @@ def attach_agent(
|
||||
|
||||
`resume=True` adds `--continue` so claude picks up its most
|
||||
recent session non-interactively (no session-picker prompt).
|
||||
First-attach paths (`./cli.py start`) leave it False.
|
||||
First-attach paths (`bot-bottle start`) leave it False.
|
||||
|
||||
Used as the inner step of `./cli.py start`."""
|
||||
Used as the inner step of `bot-bottle start`."""
|
||||
runtime = runtime_for(agent_provider_template)
|
||||
info(
|
||||
f"attaching interactive {agent_provider_template} session "
|
||||
@@ -354,7 +354,7 @@ def settle_state(identity: str) -> None:
|
||||
if not identity:
|
||||
return
|
||||
if is_preserved(identity):
|
||||
info(f"to resume this bottle: ./cli.py resume {identity}")
|
||||
info(f"to resume this bottle: bot-bottle resume {identity}")
|
||||
return
|
||||
cleanup_state(identity)
|
||||
|
||||
@@ -383,12 +383,9 @@ def _peek_agent_bottle(manifest: ManifestIndex, agent_name: str) -> str:
|
||||
from ...manifest.loader import scan_agent_names
|
||||
from ...yaml_subset import YamlSubsetError, parse_frontmatter
|
||||
|
||||
# Agents are home-only (PRD 0082).
|
||||
home_agents = scan_agent_names(manifest.home_md / "agents")
|
||||
cwd_agents: dict[str, Path] = {}
|
||||
if manifest.cwd_md is not None:
|
||||
cwd_agents = scan_agent_names(manifest.cwd_md / "agents")
|
||||
merged = {**home_agents, **cwd_agents}
|
||||
path = merged.get(agent_name)
|
||||
path = home_agents.get(agent_name)
|
||||
if path is None:
|
||||
return ""
|
||||
try:
|
||||
@@ -488,13 +485,19 @@ def _manifest_to_yaml(manifest: Manifest) -> str:
|
||||
lines.append(" skills:")
|
||||
for s in agent.skills:
|
||||
lines.append(f" - {s}")
|
||||
if not agent.git_user.is_empty():
|
||||
lines.append(" git-gate:")
|
||||
lines.append(" user:")
|
||||
if agent.git_user.name:
|
||||
lines.append(f" name: {agent.git_user.name}")
|
||||
if agent.git_user.email:
|
||||
lines.append(f" email: {agent.git_user.email}")
|
||||
if agent.author is not None:
|
||||
lines.append(" author:")
|
||||
lines.append(f" name: {agent.author.name}")
|
||||
lines.append(f" email: {agent.author.email}")
|
||||
if agent.forge_accounts:
|
||||
lines.append(" forge-accounts:")
|
||||
for alias, acct in sorted(agent.forge_accounts.items()):
|
||||
lines.append(f" {alias}:")
|
||||
lines.append(f" url: {acct.url}")
|
||||
lines.append(" auth:")
|
||||
lines.append(f" type: {acct.auth_type}")
|
||||
# token_secret name is host config; show the name, never a value.
|
||||
lines.append(f" token_secret: {acct.token_secret}")
|
||||
|
||||
bottle = manifest.bottle
|
||||
lines.append("bottle:")
|
||||
@@ -510,20 +513,14 @@ def _manifest_to_yaml(manifest: Manifest) -> str:
|
||||
for k, v in sorted(bottle.env.items()):
|
||||
lines.append(f" {k}: {v}")
|
||||
|
||||
has_git_gate = not bottle.git_user.is_empty() or bottle.git
|
||||
if has_git_gate:
|
||||
if bottle.git:
|
||||
lines.append(" git-gate:")
|
||||
if not bottle.git_user.is_empty():
|
||||
lines.append(" user:")
|
||||
if bottle.git_user.name:
|
||||
lines.append(f" name: {bottle.git_user.name}")
|
||||
if bottle.git_user.email:
|
||||
lines.append(f" email: {bottle.git_user.email}")
|
||||
if bottle.git:
|
||||
lines.append(" repos:")
|
||||
for entry in bottle.git:
|
||||
lines.append(f" {entry.Name}:")
|
||||
lines.append(f" url: {entry.Upstream}")
|
||||
lines.append(" repos:")
|
||||
for entry in bottle.git:
|
||||
lines.append(f" {entry.Name}:")
|
||||
lines.append(f" url: {entry.Upstream}")
|
||||
if entry.Forge:
|
||||
lines.append(f" forge: {entry.Forge}")
|
||||
|
||||
if bottle.egress.routes:
|
||||
lines.append(" egress:")
|
||||
|
||||
@@ -110,7 +110,7 @@ def discover_pending() -> list[QueuedProposal]:
|
||||
def _approval_status(qp: QueuedProposal, verb: str) -> str:
|
||||
"""Status-line text after a successful approval."""
|
||||
base = f"{verb} {qp.proposal.tool} for [{qp.label}]"
|
||||
return f"{base}; resume: ./cli.py resume {qp.label}"
|
||||
return f"{base}; resume: bot-bottle resume {qp.label}"
|
||||
|
||||
|
||||
def _detail_lines(
|
||||
|
||||
@@ -32,6 +32,7 @@ if TYPE_CHECKING:
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
Egress,
|
||||
egress_agent_env_entries,
|
||||
egress_forge_routes,
|
||||
egress_gateway_env_entries,
|
||||
egress_manifest_routes,
|
||||
egress_render_routes,
|
||||
@@ -51,6 +52,7 @@ _LAZY: dict[str, str] = {
|
||||
"EGRESS_ROUTES_FILENAME": ".service",
|
||||
"EGRESS_ROUTES_IN_CONTAINER": ".service",
|
||||
"egress_agent_env_entries": ".service",
|
||||
"egress_forge_routes": ".service",
|
||||
"egress_gateway_env_entries": ".service",
|
||||
"egress_manifest_routes": ".service",
|
||||
"egress_render_routes": ".service",
|
||||
@@ -80,6 +82,7 @@ __all__ = [
|
||||
"Egress",
|
||||
"EgressPlan",
|
||||
"EgressRoute",
|
||||
"egress_forge_routes",
|
||||
"egress_manifest_routes",
|
||||
"egress_render_routes",
|
||||
"egress_resolve_token_values",
|
||||
|
||||
@@ -26,7 +26,7 @@ from ..log import die
|
||||
from .plan import EgressPlan, EgressRoute
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..manifest import ManifestBottle
|
||||
from ..manifest import ManifestBottle, ResolvedForgeAssociation
|
||||
|
||||
|
||||
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||
@@ -119,15 +119,61 @@ def egress_manifest_routes(
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def egress_forge_routes(
|
||||
associations: "tuple[ResolvedForgeAssociation, ...]",
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
"""Synthesize one inspected, token-authenticated egress route per distinct
|
||||
forge alias referenced by a selected git-gate repo (PRD
|
||||
0082).
|
||||
|
||||
The route is scoped to the canonical forge origin (host) and API prefix
|
||||
(`/api/v1`): the proxy injects the Gitea `token` scheme using the value of
|
||||
the host env var named by the account's `token_secret`, resolved at launch
|
||||
from the host environment — the token never enters the bottle. Aliases that
|
||||
canonicalize to the same host share a single route (deduplicated).
|
||||
|
||||
Composition (`resolve_forge_associations`) has already rejected referenced
|
||||
aliases that share a host but disagree on origin/auth/token_secret, so this
|
||||
host-dedup only ever collapses genuinely identical credentials — it never
|
||||
silently discards a distinct one."""
|
||||
out: list[EgressRoute] = []
|
||||
seen_hosts: set[str] = set()
|
||||
for assoc in associations:
|
||||
acct = assoc.account
|
||||
host_key = acct.host.lower()
|
||||
if host_key in seen_hosts:
|
||||
continue
|
||||
seen_hosts.add(host_key)
|
||||
out.append(EgressRoute(
|
||||
host=acct.host,
|
||||
matches=(CoreMatchEntry(
|
||||
paths=(CorePathMatch(type="prefix", value=acct.api_prefix),),
|
||||
),),
|
||||
auth_scheme=acct.auth_type,
|
||||
token_ref=acct.token_secret,
|
||||
inspect=True,
|
||||
))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def egress_routes_for_bottle(
|
||||
bottle: ManifestBottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
forge_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
manifest = egress_manifest_routes(bottle)
|
||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||
merged = list(_default_provider_on_match(provider_routes)) + [
|
||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||
]
|
||||
# Provider routes (LLM API) default to redact-on-match; forge routes are
|
||||
# host-injected but keep the default DLP policy. Both take precedence over
|
||||
# a manifest route to the same host.
|
||||
reserved_hosts = (
|
||||
{pr.host.lower() for pr in provider_routes}
|
||||
| {fr.host.lower() for fr in forge_routes}
|
||||
)
|
||||
merged = (
|
||||
list(_default_provider_on_match(provider_routes))
|
||||
+ list(forge_routes)
|
||||
+ [r for r in manifest if r.host.lower() not in reserved_hosts]
|
||||
)
|
||||
return _assign_token_slots(merged)
|
||||
|
||||
|
||||
@@ -367,8 +413,9 @@ class Egress:
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
forge_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> EgressPlan:
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes, forge_routes)
|
||||
log = bottle.egress.Log
|
||||
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
|
||||
routes_path.write_text(egress_render_routes(routes, log=log))
|
||||
|
||||
@@ -136,10 +136,18 @@ def _pump(name: str, stream: IO[bytes]) -> None:
|
||||
"""Read lines from `stream`, prefix with `[name]`, write to
|
||||
stdout. Runs in its own thread per child; daemon=True so a
|
||||
blocked read doesn't keep the process alive after main exits."""
|
||||
for raw in iter(stream.readline, b""):
|
||||
line = raw.decode("utf-8", errors="replace").rstrip("\n")
|
||||
sys.stdout.write(f"[{name}] {line}\n")
|
||||
sys.stdout.flush()
|
||||
try:
|
||||
for raw in iter(stream.readline, b""):
|
||||
line = raw.decode("utf-8", errors="replace").rstrip("\n")
|
||||
sys.stdout.write(f"[{name}] {line}\n")
|
||||
sys.stdout.flush()
|
||||
except (OSError, ValueError) as exc:
|
||||
# The manager closes a dead child's pipe after wait() and before a
|
||||
# restart. A pump can be between readline calls at that exact moment;
|
||||
# closed-stream errors are normal completion, not uncaught thread
|
||||
# failures. Preserve genuinely unexpected I/O diagnostics.
|
||||
if not stream.closed:
|
||||
_log(f"{name} output pump stopped: {type(exc).__name__}: {exc}")
|
||||
|
||||
|
||||
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Shared resource boundaries for gateway stdlib HTTP services."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import http.server
|
||||
import io
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Protocol
|
||||
|
||||
|
||||
class Readable(Protocol):
|
||||
def read(self, size: int = -1, /) -> bytes: ...
|
||||
|
||||
|
||||
class Writable(Protocol):
|
||||
def write(self, data: bytes, /) -> object: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BodyReadError(Exception):
|
||||
status: int
|
||||
message: str
|
||||
|
||||
|
||||
def read_declared_body(
|
||||
stream: Readable,
|
||||
connection: socket.socket,
|
||||
raw_length: str | None,
|
||||
*,
|
||||
maximum: int,
|
||||
timeout_seconds: float,
|
||||
require_length: bool,
|
||||
) -> bytes:
|
||||
"""Validate and read exactly one declared body under a read deadline."""
|
||||
output = io.BytesIO()
|
||||
copy_declared_body(
|
||||
stream, output, connection, raw_length, maximum=maximum,
|
||||
timeout_seconds=timeout_seconds, require_length=require_length,
|
||||
)
|
||||
return output.getvalue()
|
||||
|
||||
|
||||
def copy_declared_body(
|
||||
stream: Readable,
|
||||
output: Writable,
|
||||
connection: socket.socket,
|
||||
raw_length: str | None,
|
||||
*,
|
||||
maximum: int,
|
||||
timeout_seconds: float,
|
||||
require_length: bool,
|
||||
) -> int:
|
||||
"""Copy one declared body to a sink without retaining it in memory."""
|
||||
if raw_length is None:
|
||||
if require_length:
|
||||
raise BodyReadError(411, "Content-Length required")
|
||||
raw_length = "0"
|
||||
try:
|
||||
length = int(raw_length)
|
||||
except ValueError as exc:
|
||||
raise BodyReadError(400, "invalid Content-Length") from exc
|
||||
if length < 0:
|
||||
raise BodyReadError(400, "invalid Content-Length")
|
||||
if length > maximum:
|
||||
raise BodyReadError(413, "request body too large")
|
||||
previous_timeout = connection.gettimeout()
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
remaining = length
|
||||
try:
|
||||
while remaining:
|
||||
timeout = deadline - time.monotonic()
|
||||
if timeout <= 0:
|
||||
raise BodyReadError(408, "request body read timed out")
|
||||
connection.settimeout(timeout)
|
||||
chunk = stream.read(min(remaining, 64 * 1024))
|
||||
if not chunk:
|
||||
raise BodyReadError(400, "incomplete request body")
|
||||
output.write(chunk)
|
||||
remaining -= len(chunk)
|
||||
except TimeoutError as exc:
|
||||
raise BodyReadError(408, "request body read timed out") from exc
|
||||
finally:
|
||||
connection.settimeout(previous_timeout)
|
||||
return length
|
||||
|
||||
|
||||
class BoundedThreadingHTTPServer(http.server.ThreadingHTTPServer):
|
||||
"""ThreadingHTTPServer with a hard cap on in-flight request threads."""
|
||||
|
||||
daemon_threads = True
|
||||
|
||||
def __init__( # pylint: disable=consider-using-with
|
||||
self, *args, max_workers: int = 32, **kwargs, # type: ignore[no-untyped-def]
|
||||
):
|
||||
if max_workers < 1:
|
||||
raise ValueError("max_workers must be positive")
|
||||
self._request_slots = threading.BoundedSemaphore(max_workers)
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
def process_request(
|
||||
self, request: Any, client_address: Any,
|
||||
) -> None:
|
||||
if not self._request_slots.acquire( # pylint: disable=consider-using-with
|
||||
blocking=False,
|
||||
):
|
||||
try:
|
||||
request.sendall(
|
||||
b"HTTP/1.1 503 Service Unavailable\r\n"
|
||||
b"Content-Length: 0\r\nConnection: close\r\n\r\n"
|
||||
)
|
||||
finally:
|
||||
self.shutdown_request(request)
|
||||
return
|
||||
try:
|
||||
super().process_request(request, client_address)
|
||||
except BaseException:
|
||||
self._request_slots.release()
|
||||
raise
|
||||
|
||||
def process_request_thread(
|
||||
self, request: Any, client_address: Any,
|
||||
) -> None:
|
||||
try:
|
||||
super().process_request_thread(request, client_address)
|
||||
finally:
|
||||
self._request_slots.release()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"BodyReadError",
|
||||
"BoundedThreadingHTTPServer",
|
||||
"copy_declared_body",
|
||||
"read_declared_body",
|
||||
]
|
||||
@@ -21,12 +21,19 @@ from __future__ import annotations
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import typing
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from http.server import BaseHTTPRequestHandler
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from bot_bottle.constants import GIT_GATE_TIMEOUT_SECS, IDENTITY_HEADER
|
||||
from bot_bottle.gateway.bounded_http import (
|
||||
BodyReadError,
|
||||
BoundedThreadingHTTPServer,
|
||||
copy_declared_body,
|
||||
)
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
|
||||
|
||||
|
||||
@@ -77,6 +84,10 @@ def resolve_sandbox_root(
|
||||
|
||||
# Bound memory use while still allowing ordinary git push packfiles.
|
||||
MAX_BODY_BYTES = 100 * 1024 * 1024
|
||||
REQUEST_BODY_TIMEOUT_SECONDS = 30.0
|
||||
MAX_REQUEST_WORKERS = 16
|
||||
MAX_BODY_WORKERS = 2
|
||||
_BODY_WORK_SLOTS = threading.BoundedSemaphore(MAX_BODY_WORKERS)
|
||||
|
||||
|
||||
class GitHttpHandler(BaseHTTPRequestHandler):
|
||||
@@ -184,27 +195,40 @@ class GitHttpHandler(BaseHTTPRequestHandler):
|
||||
value = self.headers.get(header)
|
||||
if value:
|
||||
env[variable] = value
|
||||
raw_length = self.headers.get("content-length", "0") or "0"
|
||||
if not _BODY_WORK_SLOTS.acquire(blocking=False):
|
||||
self.send_error(503, "git request capacity exhausted")
|
||||
return
|
||||
try:
|
||||
length = int(raw_length)
|
||||
except ValueError:
|
||||
self.send_error(400, "Bad Content-Length")
|
||||
return
|
||||
if length < 0:
|
||||
self.send_error(400, "Negative Content-Length")
|
||||
return
|
||||
if length > MAX_BODY_BYTES:
|
||||
self.send_error(413, "Request body too large")
|
||||
return
|
||||
body = self.rfile.read(length) if length else b""
|
||||
proc = subprocess.run(
|
||||
["git", "http-backend"],
|
||||
input=body,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
timeout=GIT_GATE_TIMEOUT_SECS,
|
||||
)
|
||||
with tempfile.TemporaryFile() as body:
|
||||
try:
|
||||
copy_declared_body(
|
||||
self.rfile,
|
||||
body,
|
||||
self.connection,
|
||||
self.headers.get("content-length"),
|
||||
maximum=MAX_BODY_BYTES,
|
||||
timeout_seconds=REQUEST_BODY_TIMEOUT_SECONDS,
|
||||
require_length=False,
|
||||
)
|
||||
except BodyReadError as exc:
|
||||
self.send_error(exc.status, exc.message)
|
||||
return
|
||||
body.seek(0)
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
["git", "http-backend"],
|
||||
stdin=body,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
timeout=GIT_GATE_TIMEOUT_SECS,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
self.log_message("git http-backend unavailable: %s", exc)
|
||||
self.send_error(503, "git backend unavailable")
|
||||
return
|
||||
finally:
|
||||
_BODY_WORK_SLOTS.release()
|
||||
self._write_cgi_response(proc.stdout)
|
||||
|
||||
def _repo_dir(self, sandbox_root: Path, path: str) -> Path | None:
|
||||
@@ -273,7 +297,9 @@ def main() -> int:
|
||||
"(no single-tenant flat-root fallback)\n"
|
||||
)
|
||||
return 1
|
||||
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
|
||||
server = BoundedThreadingHTTPServer(
|
||||
("0.0.0.0", port), GitHttpHandler, max_workers=MAX_REQUEST_WORKERS,
|
||||
)
|
||||
# Resolve each request's sandbox namespace by source IP against the
|
||||
# orchestrator control plane.
|
||||
server.policy_resolver = PolicyResolver(orch_url) # type: ignore[attr-defined]
|
||||
|
||||
@@ -385,7 +385,7 @@ PY
|
||||
;;
|
||||
esac
|
||||
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
|
||||
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
|
||||
echo "git-gate: approve with 'bot-bottle supervise' to continue this push" >&2
|
||||
waited=0
|
||||
while [ "$waited" -lt "$timeout" ]; do
|
||||
status=$(PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$proposal_id" <<'PY'
|
||||
|
||||
@@ -6,9 +6,8 @@ import json
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Protocol
|
||||
|
||||
from bot_bottle.gateway.egress.context import resolve_client_context
|
||||
from bot_bottle.gateway.egress.schema import route_to_yaml_dict
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolver
|
||||
from bot_bottle.gateway.egress.schema import load_config, route_to_yaml_dict
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
|
||||
from bot_bottle.supervisor import types as _sv
|
||||
|
||||
|
||||
@@ -24,6 +23,10 @@ class MethodNotFoundError(Exception):
|
||||
"""Raised when a JSON-RPC method has no MCP handler."""
|
||||
|
||||
|
||||
class RouteResolutionError(Exception):
|
||||
"""The caller's live route table could not be resolved authoritatively."""
|
||||
|
||||
|
||||
Handler = Callable[[dict[str, object]], object]
|
||||
|
||||
|
||||
@@ -60,10 +63,19 @@ def resolved_routes_payload(
|
||||
source_ip: str,
|
||||
identity_token: str,
|
||||
) -> dict[str, object]:
|
||||
"""Render the calling bottle's routes, failing closed to an empty list."""
|
||||
config, _slug, _tokens = resolve_client_context(
|
||||
resolver, source_ip, identity_token,
|
||||
)
|
||||
"""Render an authoritatively resolved route table for the calling bottle."""
|
||||
try:
|
||||
policy, bottle_id, _tokens = resolver.resolve_policy_and_bottle_id(
|
||||
source_ip, identity_token,
|
||||
)
|
||||
except PolicyResolveError as exc:
|
||||
raise RouteResolutionError("orchestrator unavailable") from exc
|
||||
if not bottle_id:
|
||||
raise RouteResolutionError("request source is not attributed to a bottle")
|
||||
try:
|
||||
config = load_config(policy or "")
|
||||
except ValueError as exc:
|
||||
raise RouteResolutionError("resolved policy is invalid") from exc
|
||||
body = json.dumps(
|
||||
{"routes": [route_to_yaml_dict(route) for route in config.routes]},
|
||||
indent=2,
|
||||
@@ -74,6 +86,7 @@ def resolved_routes_payload(
|
||||
__all__ = [
|
||||
"Handlers",
|
||||
"MethodNotFoundError",
|
||||
"RouteResolutionError",
|
||||
"dispatch",
|
||||
"resolved_routes_payload",
|
||||
]
|
||||
|
||||
@@ -51,19 +51,24 @@ from __future__ import annotations
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
import socketserver
|
||||
import sys
|
||||
import time
|
||||
import typing
|
||||
from dataclasses import dataclass
|
||||
|
||||
from bot_bottle.constants import IDENTITY_HEADER
|
||||
from bot_bottle.gateway.bounded_http import (
|
||||
BodyReadError,
|
||||
BoundedThreadingHTTPServer,
|
||||
read_declared_body,
|
||||
)
|
||||
from bot_bottle.gateway.egress.schema import load_config
|
||||
from bot_bottle.gateway.egress.types import LOG_OFF
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
|
||||
from bot_bottle.gateway.supervisor.mcp_dispatch import (
|
||||
Handlers as DispatchHandlers,
|
||||
MethodNotFoundError,
|
||||
RouteResolutionError,
|
||||
dispatch,
|
||||
resolved_routes_payload,
|
||||
)
|
||||
@@ -570,6 +575,8 @@ def format_unknown_proposal_text(proposal_id: str) -> str:
|
||||
# Max request body the server accepts. 1 MB is well above any realistic
|
||||
# routes.yaml proposal.
|
||||
MAX_BODY_BYTES = 1 * 1024 * 1024
|
||||
REQUEST_BODY_TIMEOUT_SECONDS = 10.0
|
||||
MAX_REQUEST_WORKERS = 32
|
||||
|
||||
|
||||
class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
@@ -592,19 +599,18 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
self._write_text(405, "use POST for MCP requests\n")
|
||||
|
||||
def do_POST(self) -> None:
|
||||
length_header = self.headers.get("Content-Length")
|
||||
if length_header is None:
|
||||
self._write_text(411, "Content-Length required\n")
|
||||
return
|
||||
try:
|
||||
length = int(length_header)
|
||||
except ValueError:
|
||||
self._write_text(400, "invalid Content-Length\n")
|
||||
body = read_declared_body(
|
||||
self.rfile,
|
||||
self.connection,
|
||||
self.headers.get("Content-Length"),
|
||||
maximum=MAX_BODY_BYTES,
|
||||
timeout_seconds=REQUEST_BODY_TIMEOUT_SECONDS,
|
||||
require_length=True,
|
||||
)
|
||||
except BodyReadError as exc:
|
||||
self._write_text(exc.status, exc.message + "\n")
|
||||
return
|
||||
if length < 0 or length > MAX_BODY_BYTES:
|
||||
self._write_text(413, "request body too large\n")
|
||||
return
|
||||
body = self.rfile.read(length)
|
||||
|
||||
try:
|
||||
req = parse_jsonrpc(body)
|
||||
@@ -660,20 +666,25 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
identity_token=self._identity_token(),
|
||||
)
|
||||
|
||||
return dispatch(
|
||||
req,
|
||||
DispatchHandlers(
|
||||
initialize=handle_initialize,
|
||||
tools_list=handle_tools_list,
|
||||
list_routes=lambda _params: resolved_routes_payload(
|
||||
def list_routes(_params: dict[str, object]) -> object:
|
||||
try:
|
||||
return resolved_routes_payload(
|
||||
self._resolver_or_fail(),
|
||||
self.client_address[0],
|
||||
self._identity_token(),
|
||||
),
|
||||
check_proposal=check,
|
||||
propose=propose,
|
||||
),
|
||||
)
|
||||
)
|
||||
except RouteResolutionError as exc:
|
||||
raise _RpcInternalError(
|
||||
f"could not resolve live egress routes: {exc}"
|
||||
) from exc
|
||||
|
||||
return dispatch(req, DispatchHandlers(
|
||||
initialize=handle_initialize,
|
||||
tools_list=handle_tools_list,
|
||||
list_routes=list_routes,
|
||||
check_proposal=check,
|
||||
propose=propose,
|
||||
))
|
||||
|
||||
def _identity_token(self) -> str:
|
||||
"""The agent's per-bottle identity token from the request header (the
|
||||
@@ -711,7 +722,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
self.wfile.write(encoded)
|
||||
|
||||
|
||||
class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
||||
class MCPServer(BoundedThreadingHTTPServer):
|
||||
allow_reuse_address = True
|
||||
daemon_threads = True
|
||||
config: ServerConfig = ServerConfig()
|
||||
@@ -720,6 +731,9 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
||||
# closed per request (see `_resolver_or_fail`).
|
||||
policy_resolver: "PolicyResolver | None" = None
|
||||
|
||||
def __init__(self, *args, **kwargs): # type: ignore[no-untyped-def]
|
||||
super().__init__(*args, max_workers=MAX_REQUEST_WORKERS, **kwargs)
|
||||
|
||||
|
||||
# --- Entry point -----------------------------------------------------------
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
"""Manifest dataclasses (PRD 0011 layout).
|
||||
|
||||
Reads the per-file manifest tree:
|
||||
Reads the per-file manifest tree (home-only —
|
||||
PRD 0082):
|
||||
|
||||
$HOME/.bot-bottle/bottles/<name>.md — one bottle per file
|
||||
$HOME/.bot-bottle/agents/<name>.md — home-resident agents
|
||||
$CWD/.bot-bottle/agents/<name>.md — cwd-supplied agents
|
||||
$HOME/.bot-bottle/agents/<name>.md — agents
|
||||
|
||||
Each file is Markdown with YAML frontmatter. The frontmatter holds
|
||||
the structured config (see schema below); for agents the body is
|
||||
@@ -15,27 +15,38 @@ Bottle schema (frontmatter):
|
||||
extends: <bottle-name> # optional (PRD 0025)
|
||||
env: { <NAME>: <env-entry>, ... }
|
||||
git-gate: # optional (PRD 0047)
|
||||
user: { name: <str>, email: <str> } # optional
|
||||
repos: { <name>: <git-gate-entry>, ... } # optional
|
||||
# git-gate-entry keys: url, key, host_key, forge
|
||||
# `forge`: optional alias into the selected agent's forge-accounts
|
||||
egress: { routes: [ <egress-route>, ... ] }
|
||||
# route keys: host, matches, auth, role, dlp
|
||||
supervise: <bool> # optional (default true)
|
||||
nested_containers: <bool> # optional (default false)
|
||||
|
||||
Agent schema (frontmatter):
|
||||
bottle: <bottle-name> # required
|
||||
bottle: <bottle-name> # optional
|
||||
skills: [ <skill-name>, ... ] # optional
|
||||
git-gate:
|
||||
user: { name: <str>, email: <str> } # optional; overlays bottle
|
||||
author: # optional; agent git identity
|
||||
name: <str> # required when author is present
|
||||
email: <str> # required when author is present
|
||||
forge-accounts: # optional; alias -> forge account
|
||||
<alias>:
|
||||
url: <https Gitea /api/v1 base>
|
||||
auth: { type: token, token_secret: <host env var name> }
|
||||
# Claude Code subagent passthrough fields — accepted, ignored:
|
||||
name, description, model, color, memory
|
||||
|
||||
`author` populates the bottle's git user.name/user.email; `forge-accounts`
|
||||
maps a forge alias to a Gitea API origin plus a host token reference. Identity
|
||||
is agent-owned — `git-gate` is no longer accepted on an agent (git-gate.user
|
||||
moved to `author`; git-gate.repos is bottle-only).
|
||||
|
||||
The agent file's Markdown body is the system prompt (stripped).
|
||||
Unknown top-level frontmatter keys raise ManifestError with a hint.
|
||||
|
||||
Bottles can ONLY live under $HOME. A bottles/ dir under $CWD is a
|
||||
warn at load time and contributes nothing. The trust boundary is
|
||||
expressed as filesystem layout rather than resolver logic.
|
||||
Both bottles and agents can ONLY live under $HOME. An agents/ or bottles/
|
||||
dir under $CWD is a warn at load time and contributes nothing. The trust
|
||||
boundary is expressed as filesystem layout rather than resolver logic.
|
||||
|
||||
Two types are exported:
|
||||
|
||||
@@ -66,6 +77,11 @@ if TYPE_CHECKING:
|
||||
from .agent import ManifestAgent, ManifestAgentProvider
|
||||
from .bottle import ManifestBottle
|
||||
from .egress import EGRESS_AUTH_SCHEMES, ManifestEgressConfig, ManifestEgressRoute
|
||||
from .forge import (
|
||||
ManifestAuthor,
|
||||
ManifestForgeAccount,
|
||||
ResolvedForgeAssociation,
|
||||
)
|
||||
from .git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig
|
||||
|
||||
|
||||
@@ -81,6 +97,9 @@ _LAZY_MODULES: dict[str, str] = {
|
||||
"EGRESS_AUTH_SCHEMES": "egress",
|
||||
"ManifestEgressRoute": "egress",
|
||||
"ManifestEgressConfig": "egress",
|
||||
"ManifestAuthor": "forge",
|
||||
"ManifestForgeAccount": "forge",
|
||||
"ResolvedForgeAssociation": "forge",
|
||||
"ManifestGitEntry": "git",
|
||||
"ManifestGitUser": "git",
|
||||
"ManifestKeyConfig": "git",
|
||||
@@ -115,4 +134,7 @@ __all__ = [
|
||||
"EGRESS_AUTH_SCHEMES",
|
||||
"ManifestEgressRoute",
|
||||
"ManifestEgressConfig",
|
||||
"ManifestAuthor",
|
||||
"ManifestForgeAccount",
|
||||
"ResolvedForgeAssociation",
|
||||
]
|
||||
|
||||
@@ -3,11 +3,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import cast
|
||||
from typing import Mapping, cast
|
||||
|
||||
from ..agent_provider import PROVIDER_TEMPLATES
|
||||
from .util import ManifestError, as_json_object
|
||||
from .git import ManifestGitUser
|
||||
from .forge import ManifestAuthor, ManifestForgeAccount
|
||||
from .schema import AGENT_MODEL_KEYS, is_valid_entity_name
|
||||
|
||||
|
||||
@@ -119,15 +119,29 @@ class ManifestAgent:
|
||||
bottle: str = ""
|
||||
skills: tuple[str, ...] = ()
|
||||
prompt: str = ""
|
||||
# Per-agent git identity (issue #94). Overlays the referenced
|
||||
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
|
||||
# `user` is allowed at the agent level; `repos` stays bottle-only
|
||||
# because it carries credentials and host trust.
|
||||
git_user: ManifestGitUser = ManifestGitUser()
|
||||
# Agent-owned identity (PRD 0082).
|
||||
# `author` populates the bottle's git user.name/user.email;
|
||||
# `forge_accounts` maps a forge alias to a canonical Gitea API origin and
|
||||
# a host token reference. Both live only on the agent — never under
|
||||
# `git-gate`, which is bottle-only transport policy.
|
||||
author: ManifestAuthor | None = None
|
||||
forge_accounts: Mapping[str, ManifestForgeAccount] = field(
|
||||
default_factory=dict
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "ManifestAgent":
|
||||
d = as_json_object(raw, f"agent '{name}'")
|
||||
# git-gate is no longer accepted on an agent (checked before the
|
||||
# generic unknown-key error so the migration pointer is surfaced):
|
||||
# identity moved to `author`, and git-gate.repos is bottle-only.
|
||||
if "git-gate" in d:
|
||||
raise ManifestError(
|
||||
f"agent '{name}' has a 'git-gate' block, which is no longer "
|
||||
f"accepted on an agent (PRD 0082). "
|
||||
f"Move git-gate.user name/email into the 'author' block; "
|
||||
f"git-gate.repos stays on the bottle."
|
||||
)
|
||||
unknown = set(d.keys()) - AGENT_MODEL_KEYS
|
||||
if unknown:
|
||||
allowed = ", ".join(sorted(AGENT_MODEL_KEYS))
|
||||
@@ -191,24 +205,30 @@ class ManifestAgent:
|
||||
f"(was {type(prompt_raw).__name__})"
|
||||
)
|
||||
|
||||
# git-gate: agents may declare only `git-gate.user` (name/email).
|
||||
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
||||
git_user = ManifestGitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
||||
for k in gd:
|
||||
if k != "user":
|
||||
raise ManifestError(
|
||||
f"agent '{name}' git-gate.{k} is not allowed at the "
|
||||
f"agent level; only git-gate.user (name/email) may be "
|
||||
f"set on an agent. git-gate.repos is bottle-only "
|
||||
f"(it carries credentials and host trust)."
|
||||
)
|
||||
if "user" in gd:
|
||||
git_user = ManifestGitUser.from_dict(name, gd["user"])
|
||||
# author: agent-owned git identity (optional; both fields required
|
||||
# when present). Populates the bottle's user.name/user.email.
|
||||
author = (
|
||||
ManifestAuthor.from_dict(name, d["author"])
|
||||
if "author" in d else None
|
||||
)
|
||||
|
||||
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
|
||||
# forge-accounts: alias -> Gitea API origin + host token reference.
|
||||
forge_accounts: dict[str, ManifestForgeAccount] = {}
|
||||
forge_raw = d.get("forge-accounts")
|
||||
if forge_raw is not None:
|
||||
forge_d = as_json_object(forge_raw, f"agent '{name}' forge-accounts")
|
||||
for alias, entry in forge_d.items():
|
||||
forge_accounts[alias] = ManifestForgeAccount.from_dict(
|
||||
name, alias, entry,
|
||||
)
|
||||
|
||||
return cls(
|
||||
bottle=bottle,
|
||||
skills=skills,
|
||||
prompt=prompt,
|
||||
author=author,
|
||||
forge_accounts=forge_accounts,
|
||||
)
|
||||
|
||||
|
||||
def _parse_provider_settings(
|
||||
|
||||
@@ -107,11 +107,14 @@ class ManifestBottle:
|
||||
)
|
||||
env[var] = value
|
||||
|
||||
# `git_user` is now an internal resolved carrier populated from the
|
||||
# selected agent's `author` at composition time — it is never parsed
|
||||
# from the bottle manifest (PRD 0082).
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
git_user = ManifestGitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
git, git_user = parse_git_gate_config(name, git_raw)
|
||||
git = parse_git_gate_config(name, git_raw)
|
||||
|
||||
agent_provider = (
|
||||
ManifestAgentProvider.from_dict(name, d["agent_provider"])
|
||||
|
||||
@@ -210,7 +210,7 @@ def _fold_two_bottles(
|
||||
for n in names
|
||||
}
|
||||
if merged_repos_raw:
|
||||
merged_git, _ = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
|
||||
merged_git = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
|
||||
else:
|
||||
merged_git = ()
|
||||
|
||||
|
||||
@@ -0,0 +1,312 @@
|
||||
"""Agent-owned author identity and forge accounts (PRD 0082).
|
||||
|
||||
`ManifestAuthor` is the agent's git author identity (name/email) — it moved off
|
||||
`git-gate.user` (PRD 0027 / ADR 0002) and onto the trusted agent definition.
|
||||
|
||||
`ManifestForgeAccount` maps a **forge alias** to a canonical HTTPS Gitea API
|
||||
origin plus the *name* of a host env var holding the operator-provided API
|
||||
token. The token value never lives on the manifest; the host resolves it only
|
||||
when a selected bottle repository references the alias, and hands it to the
|
||||
egress proxy — never to the bottle.
|
||||
|
||||
`ResolvedForgeAssociation` is the composed view: one distinct forge alias that
|
||||
at least one selected git-gate repository points at, with the repository names
|
||||
that reference it. The proxy-provisioning and prompt-guidance steps consume it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import urllib.parse
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .schema import is_valid_entity_name
|
||||
from .util import ManifestError, as_json_object
|
||||
|
||||
# Only the Gitea `/api/v1` base is supported today. A future provider adds a
|
||||
# validator + auth scheme + guidance renderer in code rather than accepting a
|
||||
# repository-supplied prompt or path (PRD non-goal: provider-generic prompts).
|
||||
GITEA_API_PREFIX = "/api/v1"
|
||||
|
||||
# Auth schemes an agent forge account may declare. Gitea uses `token`.
|
||||
FORGE_AUTH_TYPES = ("token",)
|
||||
|
||||
|
||||
def canonicalize_forge_url(
|
||||
agent_name: str, alias: str, url: object,
|
||||
) -> tuple[str, str, str, str]:
|
||||
"""Validate and canonicalize a forge account's API base URL.
|
||||
|
||||
Returns `(canonical, origin, host, api_prefix)` where `origin` is
|
||||
`https://host[:port]` and `canonical` is `origin + api_prefix`.
|
||||
|
||||
Fails closed on: non-string, non-`https`, embedded userinfo, query, or
|
||||
fragment, a missing host, or any path other than the supported Gitea API
|
||||
base (`/api/v1`, trailing slash tolerated)."""
|
||||
label = f"agent '{agent_name}' forge-accounts.{alias}.url"
|
||||
if not isinstance(url, str) or not url:
|
||||
raise ManifestError(f"{label} is required (non-empty string)")
|
||||
try:
|
||||
parts = urllib.parse.urlsplit(url)
|
||||
except ValueError as e:
|
||||
raise ManifestError(f"{label} is not a valid URL ({e}): {url!r}") from e
|
||||
|
||||
if parts.scheme != "https":
|
||||
raise ManifestError(
|
||||
f"{label} must use https (got scheme {parts.scheme!r} in {url!r})"
|
||||
)
|
||||
if parts.username or parts.password:
|
||||
raise ManifestError(
|
||||
f"{label} must not embed userinfo (user:pass@); the token is held "
|
||||
f"host-side via auth.token_secret. Got {url!r}"
|
||||
)
|
||||
if parts.query:
|
||||
raise ManifestError(f"{label} must not contain a query string: {url!r}")
|
||||
if parts.fragment:
|
||||
raise ManifestError(f"{label} must not contain a fragment: {url!r}")
|
||||
|
||||
host = parts.hostname
|
||||
if not host:
|
||||
raise ManifestError(f"{label} must include a hostname: {url!r}")
|
||||
|
||||
path = parts.path.rstrip("/")
|
||||
if path != GITEA_API_PREFIX:
|
||||
raise ManifestError(
|
||||
f"{label} path must be the Gitea API base '{GITEA_API_PREFIX}' "
|
||||
f"(got {parts.path!r} in {url!r}); other providers and API paths "
|
||||
f"are not yet supported."
|
||||
)
|
||||
|
||||
host = host.lower()
|
||||
netloc = host if parts.port is None else f"{host}:{parts.port}"
|
||||
origin = f"https://{netloc}"
|
||||
return f"{origin}{path}", origin, host, path
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManifestAuthor:
|
||||
"""The agent's git author identity. Both fields are required and
|
||||
non-empty — an author that is declared must fully identify the agent.
|
||||
Populates `user.name` / `user.email` inside the bottle."""
|
||||
|
||||
name: str
|
||||
email: str
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, agent_name: str, raw: object) -> "ManifestAuthor":
|
||||
d = as_json_object(raw, f"agent '{agent_name}' author")
|
||||
for k in d:
|
||||
if k not in {"name", "email"}:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author has unknown key {k!r}; "
|
||||
f"allowed: name, email"
|
||||
)
|
||||
name = d.get("name")
|
||||
if not isinstance(name, str) or not name:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.name must be a non-empty string"
|
||||
)
|
||||
email = d.get("email")
|
||||
if not isinstance(email, str) or not email:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.email must be a non-empty string"
|
||||
)
|
||||
# Git identity validation: reject values that would break the
|
||||
# `git config user.email` line or smuggle a second config directive.
|
||||
if any(c in email for c in ("\n", "\r", " ", "\t")):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.email must not contain whitespace "
|
||||
f"or newlines (got {email!r})"
|
||||
)
|
||||
if any(c in name for c in ("\n", "\r")):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.name must not contain newlines "
|
||||
f"(got {name!r})"
|
||||
)
|
||||
return cls(name=name, email=email)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManifestForgeAccount:
|
||||
"""One forge alias on an agent: a canonical Gitea API origin plus the
|
||||
host env var name that holds the agent-specific API token. The
|
||||
authenticated account is whoever owns that token — there is no separate
|
||||
account-name field."""
|
||||
|
||||
alias: str
|
||||
url: str # canonical https base incl. /api/v1, no trailing slash
|
||||
origin: str # https://host[:port]
|
||||
host: str # lowercased hostname
|
||||
api_prefix: str # /api/v1
|
||||
auth_type: str # "token"
|
||||
token_secret: str # host env var name holding the API token
|
||||
|
||||
@classmethod
|
||||
def from_dict(
|
||||
cls, agent_name: str, alias: str, raw: object,
|
||||
) -> "ManifestForgeAccount":
|
||||
if not is_valid_entity_name(alias):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' forge-accounts key {alias!r} is not a "
|
||||
f"valid alias; must match [a-z][a-z0-9-]*"
|
||||
)
|
||||
label = f"agent '{agent_name}' forge-accounts.{alias}"
|
||||
d = as_json_object(raw, label)
|
||||
for k in d:
|
||||
if k not in {"url", "auth"}:
|
||||
raise ManifestError(
|
||||
f"{label} has unknown key {k!r}; allowed: url, auth"
|
||||
)
|
||||
|
||||
canonical, origin, host, api_prefix = canonicalize_forge_url(
|
||||
agent_name, alias, d.get("url"),
|
||||
)
|
||||
|
||||
if "auth" not in d:
|
||||
raise ManifestError(f"{label} missing required 'auth' block")
|
||||
auth = as_json_object(d.get("auth"), f"{label}.auth")
|
||||
for k in auth:
|
||||
if k not in {"type", "token_secret"}:
|
||||
raise ManifestError(
|
||||
f"{label}.auth has unknown key {k!r}; allowed: type, "
|
||||
f"token_secret"
|
||||
)
|
||||
auth_type = auth.get("type")
|
||||
if auth_type not in FORGE_AUTH_TYPES:
|
||||
raise ManifestError(
|
||||
f"{label}.auth.type must be one of "
|
||||
f"{', '.join(FORGE_AUTH_TYPES)} (got {auth_type!r})"
|
||||
)
|
||||
token_secret = auth.get("token_secret")
|
||||
if not isinstance(token_secret, str) or not token_secret:
|
||||
raise ManifestError(
|
||||
f"{label}.auth.token_secret must be a non-empty string (the "
|
||||
f"name of a host env var holding the API token)"
|
||||
)
|
||||
return cls(
|
||||
alias=alias,
|
||||
url=canonical,
|
||||
origin=origin,
|
||||
host=host,
|
||||
api_prefix=api_prefix,
|
||||
auth_type=str(auth_type),
|
||||
token_secret=token_secret,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ResolvedForgeAssociation:
|
||||
"""A distinct forge alias referenced by one or more selected git-gate
|
||||
repositories. `repo_names` lists the git-gate repo names pointing at
|
||||
`account.alias` (sorted, deduplicated)."""
|
||||
|
||||
account: ManifestForgeAccount
|
||||
repo_names: tuple[str, ...]
|
||||
|
||||
@property
|
||||
def alias(self) -> str:
|
||||
return self.account.alias
|
||||
|
||||
|
||||
def resolve_forge_associations(
|
||||
agent_name: str,
|
||||
forge_accounts: "dict[str, ManifestForgeAccount]",
|
||||
git_entries: "tuple[object, ...]",
|
||||
) -> tuple[ResolvedForgeAssociation, ...]:
|
||||
"""Compose the agent's forge accounts with the effective bottle's
|
||||
git-gate repos. Each git entry that declares a `forge` alias must match a
|
||||
forge account on the selected agent — otherwise fail closed before launch.
|
||||
|
||||
Returns one association per distinct referenced alias, carrying the repo
|
||||
names that reference it. Unreferenced accounts produce nothing (no token
|
||||
is resolved and no route is created for them)."""
|
||||
by_alias: dict[str, list[str]] = {}
|
||||
for entry in git_entries:
|
||||
alias = getattr(entry, "Forge", "")
|
||||
if not alias:
|
||||
continue
|
||||
if alias not in forge_accounts:
|
||||
available = ", ".join(sorted(forge_accounts)) or "(none)"
|
||||
raise ManifestError(
|
||||
f"git-gate.repos['{getattr(entry, 'Name', '?')}'].forge "
|
||||
f"references forge alias {alias!r}, which is not defined on "
|
||||
f"agent '{agent_name}'. Available forge-accounts: {available}"
|
||||
)
|
||||
by_alias.setdefault(alias, []).append(getattr(entry, "Name", ""))
|
||||
|
||||
associations = tuple(
|
||||
ResolvedForgeAssociation(
|
||||
account=forge_accounts[alias],
|
||||
repo_names=tuple(sorted(set(names))),
|
||||
)
|
||||
for alias, names in sorted(by_alias.items())
|
||||
)
|
||||
|
||||
# The egress proxy routes by host, so two referenced aliases that resolve
|
||||
# to the same host must agree on the credential and target — otherwise the
|
||||
# generated plan would authenticate every call to that host as whichever
|
||||
# alias happened to win, silently acting as the wrong forge account. Fail
|
||||
# closed here (before the bottle is created) rather than dropping a
|
||||
# credential at route-synthesis time.
|
||||
by_host: dict[str, ManifestForgeAccount] = {}
|
||||
for assoc in associations:
|
||||
acct = assoc.account
|
||||
prev = by_host.get(acct.host)
|
||||
if prev is None:
|
||||
by_host[acct.host] = acct
|
||||
continue
|
||||
if (prev.origin, prev.api_prefix, prev.auth_type, prev.token_secret) != (
|
||||
acct.origin, acct.api_prefix, acct.auth_type, acct.token_secret
|
||||
):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' forge-accounts '{prev.alias}' and "
|
||||
f"'{acct.alias}' both resolve to host '{acct.host}' but differ "
|
||||
f"in origin/auth/token_secret. The egress proxy routes by host, "
|
||||
f"so the intended credential would be ambiguous — every request "
|
||||
f"to '{acct.host}' would authenticate as one account. Give the "
|
||||
f"aliases distinct hosts, or make their url/auth identical."
|
||||
)
|
||||
|
||||
return associations
|
||||
|
||||
|
||||
def render_forge_guidance(
|
||||
associations: tuple[ResolvedForgeAssociation, ...],
|
||||
) -> str:
|
||||
"""Render the non-secret, provider-specific forge workflow guidance
|
||||
appended to the agent's system prompt for associated repositories only.
|
||||
|
||||
Derived entirely from validated typed fields — never repository-supplied
|
||||
Markdown. Contains neither the token value nor its `token_secret` name.
|
||||
Returns "" when there are no associations (no section is generated)."""
|
||||
if not associations:
|
||||
return ""
|
||||
lines: list[str] = [
|
||||
"## Forge access (managed by bot-bottle)",
|
||||
"",
|
||||
"bot-bottle authenticates the forge API requests below for you "
|
||||
"through the egress proxy. Do not read, print, or manually attach "
|
||||
"an authorization token — the proxy injects it. Never place a token "
|
||||
"in a request.",
|
||||
]
|
||||
for assoc in associations:
|
||||
acct = assoc.account
|
||||
for repo in assoc.repo_names:
|
||||
lines.extend([
|
||||
"",
|
||||
f"### Repository `{repo}` (forge alias `{acct.alias}`)",
|
||||
f"- Forge API base URL: `{acct.url}`.",
|
||||
f"- This git-gate repository (`{repo}`) is tied to forge "
|
||||
f"`{acct.alias}`; use its API for forge actions on this repo.",
|
||||
f"- Call the API at `{acct.url}` over HTTPS through the proxy. "
|
||||
"The proxy adds authentication; you never supply a token "
|
||||
"yourself.",
|
||||
"- Git pushes still use the git-gate remote, not the API.",
|
||||
"- To propose changes: push a normal branch "
|
||||
"(`refs/heads/<branch>`) through the git-gate remote, then open "
|
||||
"a branch-backed pull request through the API.",
|
||||
"- Do NOT push AGit review refs (`refs/for/*`, `refs/draft/*`, "
|
||||
"`refs/for-review/*`); they are prohibited here.",
|
||||
"- Use the API for reviews and comments, and verify the "
|
||||
"returned object state before claiming the task is complete.",
|
||||
])
|
||||
return "\n".join(lines) + "\n"
|
||||
+38
-12
@@ -117,6 +117,11 @@ class ManifestGitEntry:
|
||||
UpstreamHost: str = ""
|
||||
UpstreamPort: str = ""
|
||||
UpstreamPath: str = ""
|
||||
# Optional forge alias (PRD 0082). When
|
||||
# set, it must match a `forge-accounts` alias on the selected agent; the
|
||||
# composition enables a scoped proxy-held API credential and forge
|
||||
# workflow guidance for this repo. Empty = no forge association.
|
||||
Forge: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_repos_entry(
|
||||
@@ -139,10 +144,10 @@ class ManifestGitEntry:
|
||||
label = f"git-gate.repos[{repo_name!r}]"
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
||||
for k in d:
|
||||
if k not in {"url", "key", "host_key"}:
|
||||
if k not in {"url", "key", "host_key", "forge"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
||||
f"allowed: url, key, host_key"
|
||||
f"allowed: url, key, host_key, forge"
|
||||
)
|
||||
upstream = d.get("url")
|
||||
if not isinstance(upstream, str) or not upstream:
|
||||
@@ -150,6 +155,21 @@ class ManifestGitEntry:
|
||||
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
||||
)
|
||||
|
||||
forge = d.get("forge", "")
|
||||
if not isinstance(forge, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} forge must be a string "
|
||||
f"(was {type(forge).__name__})"
|
||||
)
|
||||
if forge and not _GIT_NAME_RE.match(forge):
|
||||
# forge aliases follow the kebab-case identifier grammar; the
|
||||
# cross-check against the agent's forge-accounts happens at
|
||||
# composition time (it needs the resolved agent).
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} forge {forge!r} is not a "
|
||||
f"valid forge alias; allowed characters: A-Z a-z 0-9 . _ -"
|
||||
)
|
||||
|
||||
if "key" not in d:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} missing required 'key' block"
|
||||
@@ -176,6 +196,7 @@ class ManifestGitEntry:
|
||||
UpstreamHost=host,
|
||||
UpstreamPort=port,
|
||||
UpstreamPath=path,
|
||||
Forge=forge,
|
||||
)
|
||||
|
||||
|
||||
@@ -286,21 +307,26 @@ class ManifestGitUser:
|
||||
def parse_git_gate_config(
|
||||
bottle_name: str,
|
||||
raw: object,
|
||||
) -> tuple[tuple[ManifestGitEntry, ...], ManifestGitUser]:
|
||||
) -> tuple[ManifestGitEntry, ...]:
|
||||
"""Parse `git-gate` on a bottle. Only `repos` is accepted; `git-gate.user`
|
||||
moved to the agent's `author` block (PRD 0082)."""
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
||||
if "user" in d:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.user is no longer supported "
|
||||
f"(PRD 0082). Move name/email into "
|
||||
f"the selected home agent's 'author' block:\n"
|
||||
f" author:\n name: <name>\n email: <email>\n"
|
||||
f"Identity is agent-owned; git-gate now carries only transport "
|
||||
f"policy (repos)."
|
||||
)
|
||||
for k in d:
|
||||
if k not in {"user", "repos"}:
|
||||
if k != "repos":
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
||||
f"allowed: user, repos"
|
||||
f"allowed: repos"
|
||||
)
|
||||
|
||||
git_user = (
|
||||
ManifestGitUser.from_dict(bottle_name, d["user"])
|
||||
if "user" in d
|
||||
else ManifestGitUser()
|
||||
)
|
||||
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
repos_raw = d.get("repos")
|
||||
if repos_raw is not None:
|
||||
@@ -311,4 +337,4 @@ def parse_git_gate_config(
|
||||
)
|
||||
validate_unique_git_names(bottle_name, git)
|
||||
|
||||
return git, git_user
|
||||
return git
|
||||
|
||||
@@ -19,6 +19,7 @@ from .util import ManifestError, as_json_object
|
||||
from .agent import ManifestAgent
|
||||
from .bottle import ManifestBottle
|
||||
from .extends import merge_bottles_runtime, resolve_bottles
|
||||
from .forge import ResolvedForgeAssociation, resolve_forge_associations
|
||||
from .git import ManifestGitUser
|
||||
from .loader import (
|
||||
check_stale_json,
|
||||
@@ -37,30 +38,50 @@ def _section_dict(value: object, label: str) -> dict[str, object]:
|
||||
return as_json_object(value, label)
|
||||
|
||||
|
||||
def _merge_git_user(
|
||||
agent_user: ManifestGitUser, base_user: ManifestGitUser
|
||||
) -> ManifestGitUser:
|
||||
"""Merge the agent's git.user over the bottle's, agent-wins-on-non-empty."""
|
||||
if agent_user.is_empty():
|
||||
return base_user
|
||||
return ManifestGitUser(
|
||||
name=agent_user.name or base_user.name,
|
||||
email=agent_user.email or base_user.email,
|
||||
def _warn_ignored_cwd_dir(cwd_dir: Path, kind: str, home_path: str) -> None:
|
||||
"""Warn (once) that manifest files of `kind` under `$CWD/.bot-bottle/`
|
||||
are ignored — the filesystem layout IS the trust boundary. `kind` is the
|
||||
subdir name (`bottles`/`agents`); `home_path` is where they belong."""
|
||||
stale = cwd_dir / kind
|
||||
if not stale.is_dir():
|
||||
return
|
||||
files = sorted(stale.glob("*.md"))
|
||||
if not files:
|
||||
return
|
||||
names = ", ".join(p.name for p in files)
|
||||
warn(
|
||||
f"ignoring {kind[:-1]} file(s) under {stale}: {names}. "
|
||||
f"{kind.capitalize()} can only live under {home_path} "
|
||||
f"(PRD 0082). Move them or delete."
|
||||
)
|
||||
|
||||
|
||||
def _manifest_with_merged_git_user(
|
||||
agent: "ManifestAgent", raw_bottle: "ManifestBottle"
|
||||
def _compose_manifest(
|
||||
agent_name: str,
|
||||
agent: "ManifestAgent",
|
||||
raw_bottle: "ManifestBottle",
|
||||
) -> "Manifest":
|
||||
"""Build the single-value Manifest, overlaying the agent's git-gate.user
|
||||
onto the bottle (agent wins on non-empty, per-field). Shared by the eager
|
||||
and lazy load_for_agent paths."""
|
||||
merged = _merge_git_user(agent.git_user, raw_bottle.git_user)
|
||||
bottle = (
|
||||
raw_bottle if merged == raw_bottle.git_user
|
||||
else replace(raw_bottle, git_user=merged)
|
||||
"""Build the single-value Manifest from the selected agent and its
|
||||
effective bottle (PRD 0082):
|
||||
|
||||
- the agent's `author` populates the bottle's git user.name/user.email;
|
||||
- each git-gate repo's `forge` alias is resolved against the agent's
|
||||
`forge-accounts` (failing closed on an unknown alias) into the
|
||||
Manifest's forge associations.
|
||||
|
||||
Shared by the eager (from_json_obj) and lazy (from_md_dirs) paths."""
|
||||
identity = (
|
||||
ManifestGitUser(name=agent.author.name, email=agent.author.email)
|
||||
if agent.author is not None else ManifestGitUser()
|
||||
)
|
||||
return Manifest(agent=agent, bottle=bottle)
|
||||
bottle = (
|
||||
raw_bottle if identity == raw_bottle.git_user
|
||||
else replace(raw_bottle, git_user=identity)
|
||||
)
|
||||
associations = resolve_forge_associations(
|
||||
agent_name, dict(agent.forge_accounts), bottle.git,
|
||||
)
|
||||
return Manifest(agent=agent, bottle=bottle, forge_associations=associations)
|
||||
|
||||
|
||||
def _resolve_effective_bottle_eager(
|
||||
@@ -121,26 +142,28 @@ def _resolve_effective_bottle_lazy(
|
||||
class Manifest:
|
||||
"""Single-agent/bottle value type. Returned by ManifestIndex.load_for_agent().
|
||||
|
||||
`bottle` is the effective bottle with the agent's git-gate.user already
|
||||
overlaid per-field (agent wins on non-empty). Backends and provisioners
|
||||
use this directly — no agent_name lookup needed."""
|
||||
`bottle` is the effective bottle with the agent's `author` already
|
||||
populated into its git identity. `forge_associations` holds the distinct
|
||||
forge aliases referenced by the effective bottle's git-gate repos, resolved
|
||||
against the agent's `forge-accounts`. Backends and provisioners use this
|
||||
directly — no agent_name lookup needed."""
|
||||
|
||||
agent: ManifestAgent
|
||||
bottle: ManifestBottle
|
||||
forge_associations: tuple[ResolvedForgeAssociation, ...] = ()
|
||||
|
||||
def git_identity_summary(self) -> str | None:
|
||||
"""One-line effective git identity with per-field provenance, e.g.
|
||||
`name=claude (agent), email=eric@dideric.is (bottle)`.
|
||||
Returns None when neither agent nor bottle sets an identity."""
|
||||
over = self.agent.git_user # agent's declared git_user (pre-merge)
|
||||
merged = self.bottle.git_user # effective git_user (post-merge)
|
||||
if merged.is_empty():
|
||||
"""One-line effective git identity, e.g.
|
||||
`name=claude, email=eric@dideric.is`. Sourced from the agent's
|
||||
`author` block. Returns None when the agent declares no author."""
|
||||
gu = self.bottle.git_user
|
||||
if gu.is_empty():
|
||||
return None
|
||||
parts: list[str] = []
|
||||
if merged.name:
|
||||
parts.append(f"name={merged.name} ({'agent' if over.name else 'bottle'})")
|
||||
if merged.email:
|
||||
parts.append(f"email={merged.email} ({'agent' if over.email else 'bottle'})")
|
||||
if gu.name:
|
||||
parts.append(f"name={gu.name}")
|
||||
if gu.email:
|
||||
parts.append(f"email={gu.email}")
|
||||
return ", ".join(parts)
|
||||
|
||||
|
||||
@@ -164,15 +187,15 @@ class ManifestIndex:
|
||||
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "ManifestIndex":
|
||||
"""Walk the per-file manifest tree and build a ManifestIndex.
|
||||
|
||||
Layout (PRD 0011):
|
||||
Layout:
|
||||
$HOME/.bot-bottle/bottles/<name>.md — bottles (home-only)
|
||||
$HOME/.bot-bottle/agents/<name>.md — home agents
|
||||
$CWD/.bot-bottle/agents/<name>.md — cwd agents
|
||||
$HOME/.bot-bottle/agents/<name>.md — agents (home-only)
|
||||
|
||||
Cwd agents merge into the home agents on the same name
|
||||
(cwd wins). A bottles/ subdir under $CWD is logged as a
|
||||
warning and ignored — the filesystem layout IS the trust
|
||||
boundary.
|
||||
Both agents and bottles are home-only
|
||||
(PRD 0082): a `bottles/` or `agents/`
|
||||
subdir under $CWD is logged as a warning and ignored — the filesystem
|
||||
layout IS the trust boundary, since an agent may now select a host
|
||||
identity and forge secret.
|
||||
|
||||
If `missing_ok` is true, a missing `$HOME/.bot-bottle/`
|
||||
returns an empty index instead of dying. This is for
|
||||
@@ -223,17 +246,12 @@ class ManifestIndex:
|
||||
Used by tests to build a ManifestIndex from fixture directories
|
||||
without touching `os.environ`."""
|
||||
if cwd_dir is not None:
|
||||
stale_bottles = cwd_dir / "bottles"
|
||||
if stale_bottles.is_dir():
|
||||
files = sorted(stale_bottles.glob("*.md"))
|
||||
if files:
|
||||
names = ", ".join(p.name for p in files)
|
||||
warn(
|
||||
f"ignoring bottle file(s) under "
|
||||
f"{stale_bottles}: {names}. Bottles can only "
|
||||
f"live under $HOME/.bot-bottle/bottles/ "
|
||||
f"(PRD 0011). Move them or delete."
|
||||
)
|
||||
_warn_ignored_cwd_dir(cwd_dir, "bottles", "$HOME/.bot-bottle/bottles/")
|
||||
# Agents became home-only in
|
||||
# PRD 0082: a cwd agent file that
|
||||
# once shadowed a home agent could select a host identity/secret,
|
||||
# so it is now ignored with a migration pointer.
|
||||
_warn_ignored_cwd_dir(cwd_dir, "agents", "$HOME/.bot-bottle/agents/")
|
||||
return cls(bottles={}, agents={}, home_md=home_dir, cwd_md=cwd_dir)
|
||||
|
||||
@classmethod
|
||||
@@ -275,13 +293,12 @@ class ManifestIndex:
|
||||
|
||||
In names-only mode (from resolve/from_md_dirs) this scans agent
|
||||
filenames without reading their content. In eager mode (from
|
||||
from_json_obj) it returns the pre-parsed agents' names."""
|
||||
from_json_obj) it returns the pre-parsed agents' names.
|
||||
|
||||
Agents are home-only (PRD 0082): cwd
|
||||
agent files never contribute names."""
|
||||
if self.home_md is not None:
|
||||
home_names = set(scan_agent_names(self.home_md / "agents").keys())
|
||||
cwd_names: set[str] = set()
|
||||
if self.cwd_md is not None:
|
||||
cwd_names = set(scan_agent_names(self.cwd_md / "agents").keys())
|
||||
return sorted(home_names | cwd_names)
|
||||
return sorted(scan_agent_names(self.home_md / "agents").keys())
|
||||
return sorted(self.agents.keys())
|
||||
|
||||
def load_for_agent(
|
||||
@@ -326,7 +343,7 @@ class ManifestIndex:
|
||||
raw_bottle = _resolve_effective_bottle_eager(
|
||||
agent_name, agent, bottle_names, self.bottles
|
||||
)
|
||||
return _manifest_with_merged_git_user(agent, raw_bottle)
|
||||
return _compose_manifest(agent_name, agent, raw_bottle)
|
||||
|
||||
def _load_for_agent_lazy(
|
||||
self, agent_name: str, bottle_names: tuple[str, ...]
|
||||
@@ -334,20 +351,17 @@ class ManifestIndex:
|
||||
"""Lazy path (resolve/from_md_dirs): read and parse the agent file and
|
||||
its bottle chain from disk for the first time here."""
|
||||
assert self.home_md is not None # guaranteed by load_for_agent dispatch
|
||||
# Locate the agent file; cwd wins over home on name collision.
|
||||
# Agents are home-only (PRD 0082):
|
||||
# a cwd agent file must not select a host identity or forge secret.
|
||||
home_agents = scan_agent_names(self.home_md / "agents")
|
||||
cwd_agents: dict[str, Path] = {}
|
||||
if self.cwd_md is not None:
|
||||
cwd_agents = scan_agent_names(self.cwd_md / "agents")
|
||||
merged_agents = {**home_agents, **cwd_agents}
|
||||
|
||||
if agent_name not in merged_agents:
|
||||
available = ", ".join(sorted(merged_agents.keys())) or "(none)"
|
||||
if agent_name not in home_agents:
|
||||
available = ", ".join(sorted(home_agents.keys())) or "(none)"
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' not defined. Available: {available}"
|
||||
)
|
||||
|
||||
agent_path = merged_agents[agent_name]
|
||||
agent_path = home_agents[agent_name]
|
||||
try:
|
||||
fm, body = parse_frontmatter(agent_path.read_text())
|
||||
except OSError as e:
|
||||
@@ -374,15 +388,18 @@ class ManifestIndex:
|
||||
}
|
||||
if agent_bottle:
|
||||
agent_dict["bottle"] = agent_bottle
|
||||
if "git-gate" in fm:
|
||||
agent_dict["git-gate"] = fm["git-gate"]
|
||||
# Surface agent-owned identity keys (and any stale git-gate, so
|
||||
# ManifestAgent.from_dict raises the migration error).
|
||||
for key in ("author", "forge-accounts", "git-gate"):
|
||||
if key in fm:
|
||||
agent_dict[key] = fm[key]
|
||||
# Pass the effective bottle name as the known-bottles set so agents
|
||||
# that have bottle: set are validated; agents without bottle: pass {}
|
||||
# since bottle_names were already resolved above.
|
||||
known = {effective_bottle_name} if effective_bottle_name else set()
|
||||
agent = ManifestAgent.from_dict(agent_name, agent_dict, known)
|
||||
|
||||
return _manifest_with_merged_git_user(agent, raw_bottle)
|
||||
return _compose_manifest(agent_name, agent, raw_bottle)
|
||||
|
||||
def has_agent(self, name: str) -> bool:
|
||||
return name in self.agents
|
||||
@@ -394,13 +411,9 @@ class ManifestIndex:
|
||||
if self.has_agent(name):
|
||||
return
|
||||
if self.home_md is not None:
|
||||
# Names-only mode: check file existence without parsing.
|
||||
home_path = self.home_md / "agents" / f"{name}.md"
|
||||
cwd_path = (
|
||||
self.cwd_md / "agents" / f"{name}.md"
|
||||
if self.cwd_md else None
|
||||
)
|
||||
if home_path.is_file() or (cwd_path and cwd_path.is_file()):
|
||||
# Names-only mode: check home file existence without parsing.
|
||||
# Agents are home-only; a cwd agent file is never selectable.
|
||||
if (self.home_md / "agents" / f"{name}.md").is_file():
|
||||
return
|
||||
available = ", ".join(self.all_agent_names) or "(none)"
|
||||
raise ManifestError(
|
||||
|
||||
@@ -22,7 +22,10 @@ BOTTLE_KEYS = frozenset(
|
||||
}
|
||||
)
|
||||
AGENT_KEYS_REQUIRED: frozenset[str] = frozenset()
|
||||
AGENT_KEYS_OPTIONAL = frozenset({"bottle", "skills", "git-gate"})
|
||||
# `author` / `forge-accounts` are agent-owned identity (PRD
|
||||
# 0082). `git-gate` is no longer accepted on an
|
||||
# agent: `git-gate.user` moved to `author`, and `git-gate.repos` is bottle-only.
|
||||
AGENT_KEYS_OPTIONAL = frozenset({"bottle", "skills", "author", "forge-accounts"})
|
||||
|
||||
# Claude Code subagent fields bot-bottle ignores at launch but does
|
||||
# not reject. This lets the same file double as
|
||||
|
||||
@@ -7,6 +7,7 @@ import asyncio
|
||||
import math
|
||||
import sys
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import BaseModel, ConfigDict, StrictStr
|
||||
from starlette.types import ASGIApp, Message, Receive, Scope, Send
|
||||
@@ -208,6 +209,19 @@ def create_app(orch: OrchestratorCore, *, signing_key: str) -> FastAPI:
|
||||
)
|
||||
app.add_middleware(ControlPlaneBoundary, signing_key=key)
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
async def invalid_request(
|
||||
_request: object, exc: RequestValidationError,
|
||||
) -> JSONResponse:
|
||||
errors = exc.errors()
|
||||
location = errors[0].get("loc", ()) if errors else ()
|
||||
field = str(location[1]) if len(location) > 1 else ""
|
||||
suffix = f": {field}" if field else ""
|
||||
return JSONResponse(
|
||||
{"error": f"invalid request body{suffix}"},
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
@app.get("/health")
|
||||
def health() -> dict[str, str]:
|
||||
return {"status": "ok"}
|
||||
|
||||
@@ -366,7 +366,7 @@ class OrchestratorCore:
|
||||
value with *env_var_secret*, and restores ``_tokens[bottle_id]``.
|
||||
Returns True on success, False when no stored secrets exist for this
|
||||
bottle or decryption fails (wrong key / corrupt data)."""
|
||||
from .store.secret_store import decrypt_value, encrypt_value, is_legacy_blob
|
||||
from .store.secret_store import decrypt_value
|
||||
encrypted = self.registry.get_agent_secrets(bottle_id)
|
||||
if not encrypted:
|
||||
return False
|
||||
@@ -377,12 +377,6 @@ class OrchestratorCore:
|
||||
except ValueError:
|
||||
return False
|
||||
self._tokens[bottle_id] = decrypted
|
||||
if any(is_legacy_blob(value) for value in encrypted.values()):
|
||||
migrated = {
|
||||
key: encrypt_value(env_var_secret, value)
|
||||
for key, value in decrypted.items()
|
||||
}
|
||||
self.registry.store_agent_secrets(bottle_id, migrated)
|
||||
return True
|
||||
|
||||
# --- consolidated gateway ----------------------------------------------
|
||||
|
||||
@@ -129,6 +129,10 @@ _MIGRATIONS = TableMigrations(
|
||||
# v5 — index for fast per-bottle lookups and bulk DELETE on teardown.
|
||||
"CREATE INDEX IF NOT EXISTS idx_bottled_agent_secrets_id "
|
||||
"ON bottled_agent_secrets (bottled_agent_id, type)",
|
||||
# v6 — unauthenticated legacy ciphertext must never be selected by
|
||||
# attacker-controlled blob contents. Existing local agents are
|
||||
# intentionally reprovisioned instead of retaining downgrade support.
|
||||
"DELETE FROM bottled_agent_secrets",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@@ -18,9 +18,9 @@ value is encrypted independently. New output blobs are:
|
||||
|
||||
``version || nonce (16 bytes) || ciphertext || tag (32 bytes)``
|
||||
|
||||
encoded as URL-safe base64 (no padding). The version marker lets the reader
|
||||
accept legacy ``nonce || ciphertext`` rows long enough to rewrite them in the
|
||||
authenticated format after a successful reprovision.
|
||||
encoded as URL-safe base64 (no padding). Unversioned legacy ciphertext is
|
||||
rejected; the registry migration clears those rows rather than allowing blob
|
||||
contents to select an unauthenticated decoder.
|
||||
|
||||
keystream_block_i = HMAC-SHA256(key, nonce || i.to_bytes(4, "big"))
|
||||
ciphertext_i = plaintext_i XOR keystream_block_i[:len(plaintext_i)]
|
||||
@@ -84,44 +84,18 @@ def encrypt_value(secret_b64: str, plaintext: str) -> str:
|
||||
return base64.urlsafe_b64encode(authenticated + tag).rstrip(b"=").decode()
|
||||
|
||||
|
||||
def is_legacy_blob(blob_b64: str) -> bool:
|
||||
"""Whether *blob_b64* uses the pre-authentication storage format."""
|
||||
try:
|
||||
return not _b64dec(blob_b64).startswith(_VERSION)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
|
||||
def _decrypt_legacy(key: bytes, blob: bytes) -> str:
|
||||
"""Read the original ``nonce || ciphertext`` format for migration only."""
|
||||
if len(blob) < _NONCE_BYTES:
|
||||
raise ValueError("ciphertext blob too short")
|
||||
nonce, ciphertext = blob[:_NONCE_BYTES], blob[_NONCE_BYTES:]
|
||||
pt = bytearray()
|
||||
# The legacy format used the byte offset as the PRF counter.
|
||||
for i in range(0, len(ciphertext), _BLOCK):
|
||||
chunk = ciphertext[i : i + _BLOCK]
|
||||
ks = _keystream(key, nonce, i)[: len(chunk)]
|
||||
pt.extend(c ^ k for c, k in zip(chunk, ks))
|
||||
try:
|
||||
return bytes(pt).decode()
|
||||
except UnicodeDecodeError as exc:
|
||||
raise ValueError(f"decryption produced non-UTF-8 output: {exc}") from exc
|
||||
|
||||
|
||||
def decrypt_value(secret_b64: str, blob_b64: str) -> str:
|
||||
"""Decrypt a blob produced by :func:`encrypt_value`.
|
||||
|
||||
Returns the original plaintext string. Raises ``ValueError`` for malformed
|
||||
input, authentication failure, or a key mismatch. Legacy unauthenticated
|
||||
rows remain readable so callers can migrate them immediately."""
|
||||
input, authentication failure, or a key mismatch."""
|
||||
key = _b64dec(secret_b64)
|
||||
try:
|
||||
blob = _b64dec(blob_b64)
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise ValueError(f"invalid ciphertext blob: {exc}") from exc
|
||||
if not blob.startswith(_VERSION):
|
||||
return _decrypt_legacy(key, blob)
|
||||
raise ValueError("unsupported ciphertext format")
|
||||
minimum = len(_VERSION) + _NONCE_BYTES + _TAG_BYTES
|
||||
if len(blob) < minimum:
|
||||
raise ValueError("ciphertext blob too short")
|
||||
@@ -152,5 +126,4 @@ __all__ = [
|
||||
"new_env_var_secret",
|
||||
"encrypt_value",
|
||||
"decrypt_value",
|
||||
"is_legacy_blob",
|
||||
]
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
import stat
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
@@ -19,9 +21,62 @@ class DbStore:
|
||||
def __init__(self, db_path: Path, migrations: TableMigrations) -> None:
|
||||
self.db_path = db_path
|
||||
self._migrations = migrations
|
||||
self.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._secure_parent()
|
||||
if self.db_path.exists():
|
||||
self._chmod()
|
||||
|
||||
def _secure_parent(self) -> None:
|
||||
"""Create and verify the private parent directory."""
|
||||
parent = self.db_path.parent
|
||||
parent.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if parent.is_symlink():
|
||||
raise PermissionError(f"database directory must not be a symlink: {parent}")
|
||||
parent.chmod(0o700)
|
||||
parent_stat = parent.lstat()
|
||||
if not stat.S_ISDIR(parent_stat.st_mode):
|
||||
raise PermissionError(f"database parent is not a directory: {parent}")
|
||||
if stat.S_IMODE(parent_stat.st_mode) != 0o700:
|
||||
raise PermissionError(f"database directory is not mode 0700: {parent}")
|
||||
|
||||
def _secure_db_file(self) -> None:
|
||||
"""Create the database without a permissive filesystem window.
|
||||
|
||||
SQLite otherwise creates a missing database using the process umask.
|
||||
This store contains control-plane identity tokens, so both creation and
|
||||
repair are fail-closed rather than best-effort.
|
||||
"""
|
||||
flags = os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW
|
||||
fd = os.open(
|
||||
self.db_path,
|
||||
flags,
|
||||
stat.S_IRUSR | stat.S_IWUSR,
|
||||
)
|
||||
try:
|
||||
self._secure_open_file(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
def _chmod(self) -> None:
|
||||
"""Enforce and verify the private database mode after every write."""
|
||||
fd = os.open(self.db_path, os.O_RDWR | os.O_NOFOLLOW)
|
||||
try:
|
||||
self._secure_open_file(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
def _secure_open_file(self, fd: int) -> None:
|
||||
"""Pin, validate, and secure an opened database filesystem object."""
|
||||
file_stat = os.fstat(fd)
|
||||
if not stat.S_ISREG(file_stat.st_mode):
|
||||
raise PermissionError(
|
||||
f"database must be a regular file: {self.db_path}"
|
||||
)
|
||||
os.fchmod(fd, stat.S_IRUSR | stat.S_IWUSR)
|
||||
if stat.S_IMODE(os.fstat(fd).st_mode) != 0o600:
|
||||
raise PermissionError(f"database is not mode 0600: {self.db_path}")
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
self._secure_db_file()
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
conn.row_factory = sqlite3.Row
|
||||
return conn
|
||||
@@ -51,16 +106,9 @@ class DbStore:
|
||||
return version == len(self._migrations.migrations)
|
||||
|
||||
def migrate(self) -> None:
|
||||
"""Apply any pending migrations and set permissions on the DB file."""
|
||||
"""Apply any pending migrations to the already-secured DB file."""
|
||||
with self._connection() as conn:
|
||||
self._migrations.apply(conn)
|
||||
self._chmod()
|
||||
|
||||
def _chmod(self) -> None:
|
||||
try:
|
||||
self.db_path.chmod(0o600)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
__all__ = ["DbStore", "DbVersionError"]
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
# VHS tape — drives `./cli.py start demo` interactively and asks
|
||||
# VHS tape — drives `bot-bottle start demo` interactively and asks
|
||||
# claude (the AI) to run four probes via natural-language prompts.
|
||||
# Setup (manifest + dummy SSH key + image pre-warm) and teardown
|
||||
# happen outside the tape; record via `bash scripts/demo-record.sh`,
|
||||
@@ -29,7 +29,7 @@ Show
|
||||
# defaults), one git upstream (unreachable on purpose so gitleaks runs
|
||||
# before the gate would forward), and a FAKE_TOKEN env var shaped like
|
||||
# a GitHub PAT.
|
||||
Type "./cli.py start demo"
|
||||
Type "bot-bottle start demo"
|
||||
Enter
|
||||
Sleep 8s
|
||||
|
||||
|
||||
@@ -1,9 +1,19 @@
|
||||
# PRD 0011: Per-file Markdown manifest
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Active (agent cwd-discovery superseded)
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-24
|
||||
|
||||
> **Superseded in part by PRD 0082.**
|
||||
> The `$CWD/.bot-bottle/agents/<name>.md` discovery/override path described
|
||||
> below is removed: agents are now **home-only**, like bottles. Once an agent
|
||||
> definition can select a host identity (`author`) and a host forge secret
|
||||
> (`forge-accounts`), letting checked-out workspace content define or override
|
||||
> an agent would let untrusted content select host credentials. A cwd
|
||||
> `agents/` (or `bottles/`) directory is now warned-about and ignored. The
|
||||
> filesystem-layout trust boundary still holds — it just admits nothing from
|
||||
> `$CWD`.
|
||||
|
||||
## Summary
|
||||
|
||||
Replace the single-file `bot-bottle.json` manifest with a
|
||||
@@ -132,7 +142,7 @@ Each test runs against a temporary `$HOME` and a temporary `$CWD`:
|
||||
can revisit, but the v1 of this PRD is one file = one bottle.
|
||||
|
||||
- **Hot-reload.** Changes to manifest files take effect at next
|
||||
`./cli.py start`; we do not watch the directory.
|
||||
`bot-bottle start`; we do not watch the directory.
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
@@ -290,7 +290,7 @@ After this PRD:
|
||||
|
||||
### Cleanup CLI
|
||||
|
||||
`./cli.py cleanup` switches from "list every container with prefix
|
||||
`bot-bottle cleanup` switches from "list every container with prefix
|
||||
`bot-bottle-` and every network with prefix `bot-bottle-net-`
|
||||
or `bot-bottle-egress-`" to:
|
||||
|
||||
@@ -369,7 +369,7 @@ Sized for one PR each, in order.
|
||||
`docker compose up -d` + attach + teardown. Per-sidecar `start()`/
|
||||
`stop()` lifecycle methods deleted in the same chunk. Compose-
|
||||
log dump on teardown added.
|
||||
4. **Cleanup CLI on compose.** Switch `./cli.py cleanup` to
|
||||
4. **Cleanup CLI on compose.** Switch `bot-bottle cleanup` to
|
||||
`docker compose ls`-based discovery; keep prefix-scan as
|
||||
fallback for one release.
|
||||
5. **Dashboard.** Decide on the discovery question (open question
|
||||
|
||||
@@ -37,7 +37,7 @@ Two rough edges in the current dashboard:
|
||||
shows only pending proposals. If no agent has called a tool,
|
||||
the screen reads "no pending proposals" — even when five
|
||||
bottles are quietly working. The operator has to `docker
|
||||
compose ls` (or `./cli.py cleanup -n` to see the y/N preview)
|
||||
compose ls` (or `bot-bottle cleanup -n` to see the y/N preview)
|
||||
to find out what's actually live.
|
||||
|
||||
2. **`e` / `p` re-discover-and-disambiguate every invocation.**
|
||||
@@ -82,12 +82,12 @@ the "operator wants to make an unprompted change" case.
|
||||
global across bottles. Filtering ("show me only this agent's
|
||||
proposals") might be a follow-up but isn't this PRD.
|
||||
- **Agent lifecycle from the dashboard.** Starting / stopping
|
||||
agents stays in `./cli.py start` / `./cli.py cleanup`. The
|
||||
agents stays in `bot-bottle start` / `bot-bottle cleanup`. The
|
||||
dashboard reads state; it doesn't change it.
|
||||
- **Preserved-but-not-running bottles.** The active-agents list
|
||||
is strictly "what's running now" (cross-referenced from
|
||||
`docker compose ls`). Preserved state dirs without a live
|
||||
project don't appear — `./cli.py resume <identity>` is the
|
||||
project don't appear — `bot-bottle resume <identity>` is the
|
||||
path for those.
|
||||
- **A separate per-agent detail view.** The agent rows are
|
||||
one-line summaries. Pressing Enter on a proposal still drops
|
||||
@@ -125,7 +125,7 @@ the "operator wants to make an unprompted change" case.
|
||||
- Changes to proposal handling (`a` / `m` / `r` / Enter all
|
||||
unchanged).
|
||||
- Changes to the queue-dir / supervise sidecar protocol.
|
||||
- New CLI surface beyond what's in `./cli.py dashboard`.
|
||||
- New CLI surface beyond what's in `bot-bottle dashboard`.
|
||||
- Touching the manifest, compose renderer, launch lifecycle.
|
||||
|
||||
## Proposed design
|
||||
|
||||
@@ -14,8 +14,8 @@
|
||||
Today the dashboard is read-only: it surfaces pending proposals
|
||||
and active agents (PRD 0019) but can't *start* an agent or
|
||||
*re-enter* one. The operator's path is split — they launch
|
||||
agents from one terminal (`./cli.py start <name>`), and watch
|
||||
them from another (`./cli.py dashboard`).
|
||||
agents from one terminal (`bot-bottle start <name>`), and watch
|
||||
them from another (`bot-bottle dashboard`).
|
||||
|
||||
This PRD collapses that split. The dashboard becomes the
|
||||
operator's single surface: pressing a key opens an agent picker,
|
||||
@@ -31,7 +31,7 @@ claude session AND the dashboard process. Exit claude → back to
|
||||
dashboard, bottle still running. Start another agent → two
|
||||
bottles up at once. Quit the dashboard → bottles continue
|
||||
running. Teardown is **always explicit**: the operator presses
|
||||
`x` on an agent, or runs `./cli.py cleanup` later.
|
||||
`x` on an agent, or runs `bot-bottle cleanup` later.
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -45,7 +45,7 @@ Two real frictions today:
|
||||
open and the dashboard's "active agents" pane is hopelessly
|
||||
behind reality because they just spawned three in a row.
|
||||
|
||||
2. **`./cli.py start` ties the bottle to a single claude
|
||||
2. **`bot-bottle start` ties the bottle to a single claude
|
||||
session.** The start command's `ExitStack` brings the bottle
|
||||
up, runs claude, and tears down on Ctrl-D — fine for a one-
|
||||
shot session, wrong for "let me bounce in and out of this
|
||||
@@ -60,7 +60,7 @@ captures full-merged logs per bottle (PRD 0018). It already
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
1. From inside `./cli.py dashboard`, pressing `n` (new) opens
|
||||
1. From inside `bot-bottle dashboard`, pressing `n` (new) opens
|
||||
an agent picker listing every agent defined in the manifest.
|
||||
Selecting one runs `prepare → preflight → launch`.
|
||||
2. The preflight Y/N summary renders cleanly — either as a
|
||||
@@ -83,7 +83,7 @@ captures full-merged logs per bottle (PRD 0018). It already
|
||||
state cleanup) without quitting the dashboard.
|
||||
7. Quitting the dashboard (`q`) leaves every running bottle
|
||||
running. Bottle teardown is always explicit (per-bottle `x`
|
||||
or `./cli.py cleanup`). The next `./cli.py dashboard`
|
||||
or `bot-bottle cleanup`). The next `bot-bottle dashboard`
|
||||
invocation re-discovers them via `list_active_slugs()` and
|
||||
surfaces re-attach for any it can reconstruct context for
|
||||
(see "Cross-dashboard re-attach" below).
|
||||
@@ -94,13 +94,13 @@ captures full-merged logs per bottle (PRD 0018). It already
|
||||
embedded-emulator option from the research doc is out of
|
||||
scope. The handoff (option 1) is the v1; option 2 is a
|
||||
separate PRD if and when handoff is observably insufficient.
|
||||
- **Adopting bottles started by an out-of-dashboard `./cli.py
|
||||
- **Adopting bottles started by an out-of-dashboard `bot-bottle
|
||||
start` invocation.** Those have their own ExitStack-owner and
|
||||
the dashboard treats them as read-only-watch (already does
|
||||
today). Re-attach only applies to bottles the *current
|
||||
dashboard process* started.
|
||||
- **Resurrecting an out-of-process bottle into a new dashboard
|
||||
with full re-attach.** A bottle started by `./cli.py start`
|
||||
with full re-attach.** A bottle started by `bot-bottle start`
|
||||
in another terminal — or by a previous dashboard run, now
|
||||
exited — appears in the agents pane (already does, PRD 0019)
|
||||
and can be re-attached via `docker exec -it claude` because
|
||||
@@ -109,12 +109,12 @@ captures full-merged logs per bottle (PRD 0018). It already
|
||||
context object to drive teardown — e.g., the
|
||||
ExitStack-tracked CA + state cleanup `_settle_state` performs
|
||||
today. Cross-dashboard re-attach uses the existing
|
||||
`./cli.py cleanup` for teardown, not an `x` keypress (see
|
||||
`bot-bottle cleanup` for teardown, not an `x` keypress (see
|
||||
open questions).
|
||||
- **Multi-window UI.** Single curses window, two existing
|
||||
panes (proposals + agents); the agent picker is a modal, not
|
||||
a third pane.
|
||||
- **Removing `./cli.py start`.** Stays as the script-friendly /
|
||||
- **Removing `bot-bottle start`.** Stays as the script-friendly /
|
||||
legacy entry point. The dashboard is the new default.
|
||||
|
||||
## Scope
|
||||
@@ -140,7 +140,7 @@ captures full-merged logs per bottle (PRD 0018). It already
|
||||
|
||||
### Out of scope
|
||||
|
||||
- Changes to `./cli.py start` itself. It keeps its current
|
||||
- Changes to `bot-bottle start` itself. It keeps its current
|
||||
shape; the dashboard reuses its internal pieces (backend.
|
||||
prepare / backend.launch) without reaching through the CLI
|
||||
layer.
|
||||
@@ -157,7 +157,7 @@ captures full-merged logs per bottle (PRD 0018). It already
|
||||
Today's flow:
|
||||
|
||||
```
|
||||
./cli.py start agent
|
||||
bot-bottle start agent
|
||||
└─ with backend.launch(plan) as bottle: ← bottle alive while inside `with`
|
||||
bottle.exec_agent([...], tty=True) ← blocks until claude exits
|
||||
# context exits → compose down → state cleanup
|
||||
@@ -166,7 +166,7 @@ Today's flow:
|
||||
The proposed dashboard-driven flow:
|
||||
|
||||
```
|
||||
./cli.py dashboard
|
||||
bot-bottle dashboard
|
||||
└─ bottles: dict[str, tuple[ContextManager, DockerBottle]] = {}
|
||||
|
||||
# operator presses `n`, picks agent
|
||||
@@ -205,7 +205,7 @@ Two shifts:
|
||||
evaluation, state-dir reap) doesn't fire on a quit-while-
|
||||
running bottle. It DOES fire when the operator explicitly
|
||||
stops via `x`, because that calls `cm.__exit__`. For
|
||||
bottles a previous dashboard quit on, `./cli.py cleanup`
|
||||
bottles a previous dashboard quit on, `bot-bottle cleanup`
|
||||
is the path — its compose-down + state-reap logic
|
||||
already covers the case.
|
||||
|
||||
@@ -213,7 +213,7 @@ Two shifts:
|
||||
|
||||
When the dashboard discovers a bottle in `discover_active_agents`
|
||||
that it didn't itself start (a previous-dashboard or external
|
||||
`./cli.py start` bottle), Enter still attaches via `docker exec
|
||||
`bot-bottle start` bottle), Enter still attaches via `docker exec
|
||||
-it … claude` — the agent container is running `sleep infinity`
|
||||
exactly the same way regardless of who started it. The only
|
||||
thing the current dashboard lacks for those bottles is the
|
||||
@@ -221,8 +221,8 @@ launch-context object needed to drive a clean teardown via
|
||||
`x`.
|
||||
|
||||
For v1 we surface this honestly: pressing `x` on a non-owned
|
||||
agent shows a status hint pointing at `./cli.py cleanup` (or
|
||||
`./cli.py cleanup` targeted at the slug if we add that flag
|
||||
agent shows a status hint pointing at `bot-bottle cleanup` (or
|
||||
`bot-bottle cleanup` targeted at the slug if we add that flag
|
||||
later). The agent stays alive; the operator handles teardown
|
||||
out-of-band. Enter (re-attach) works for both owned and
|
||||
non-owned bottles.
|
||||
@@ -288,7 +288,7 @@ agents pane.
|
||||
|
||||
`x` on a non-owned agent (discovered via `list_active_slugs`
|
||||
but not in `bottles` dict): no-op with status hint pointing
|
||||
at `./cli.py cleanup` (the existing path that tears down
|
||||
at `bot-bottle cleanup` (the existing path that tears down
|
||||
ANY bot-bottle compose project plus reaps state dirs).
|
||||
|
||||
### Dashboard quit
|
||||
@@ -300,7 +300,7 @@ the `docker compose` project keeps running. The next dashboard
|
||||
invocation discovers the bottles via `list_active_slugs` and
|
||||
surfaces re-attach.
|
||||
|
||||
This is a real departure from today's `./cli.py start`
|
||||
This is a real departure from today's `bot-bottle start`
|
||||
semantics (which couples bottle lifetime to the process via
|
||||
ExitStack). It's intentional: the dashboard is a watching +
|
||||
acting surface, not a lifetime owner.
|
||||
@@ -322,7 +322,7 @@ Sized for one PR each.
|
||||
dashboard's ExitStack; handoff invokes `attach_agent`.
|
||||
3. **Re-attach via Enter on owned agents-pane row.** Looks up
|
||||
the slug in the dashboard's `bottles` map; if present →
|
||||
handoff; else → status-line hint pointing at `./cli.py
|
||||
handoff; else → status-line hint pointing at `bot-bottle
|
||||
resume`.
|
||||
4. **Explicit per-bottle stop (`x` keybinding).** Pop the
|
||||
bottle's `close` callback off the stack, call it, refresh.
|
||||
@@ -369,7 +369,7 @@ Sized for one PR each.
|
||||
bottles dict goes out of scope without invoking `__exit__`,
|
||||
so the `docker compose` projects keep running. Bottle
|
||||
teardown is always explicit: per-bottle `x` (for
|
||||
dashboard-owned), or `./cli.py cleanup` (for everything).
|
||||
dashboard-owned), or `bot-bottle cleanup` (for everything).
|
||||
|
||||
## Open questions
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ window, two panes, no terminal handoff.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
1. When the operator runs `./cli.py dashboard` from inside a
|
||||
1. When the operator runs `bot-bottle dashboard` from inside a
|
||||
tmux session (`$TMUX` set), the dashboard establishes a
|
||||
two-pane layout: dashboard in the left pane, an initially-
|
||||
empty right pane reserved for claude sessions.
|
||||
@@ -313,7 +313,7 @@ Sized small.
|
||||
3. **Dashboard launched OUTSIDE tmux but tmux is installed.**
|
||||
Should the dashboard auto-exec itself inside a fresh tmux
|
||||
session to get the split-pane experience? Convenient but
|
||||
surprising (`./cli.py dashboard` shouldn't silently
|
||||
surprising (`bot-bottle dashboard` shouldn't silently
|
||||
change what session you're in). v1 leaves this off —
|
||||
operators who want split-pane mode start tmux themselves
|
||||
and then run the dashboard.
|
||||
@@ -332,7 +332,7 @@ Sized small.
|
||||
PRD-0019 focus indicator?
|
||||
|
||||
6. **Concurrent dashboards in different tmux windows.**
|
||||
Multiple `./cli.py dashboard` invocations in different
|
||||
Multiple `bot-bottle dashboard` invocations in different
|
||||
tmux windows would each create their own right pane —
|
||||
probably fine, each has its own state, but worth
|
||||
verifying that `tmux list-panes` is scoped to the right
|
||||
|
||||
@@ -14,7 +14,7 @@ Today bot-bottle is hard-wired around Claude Code assumptions. When Claude runs
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- A Codex agent can be started from the dashboard and via `./cli.py start` alongside a Claude agent.
|
||||
- A Codex agent can be started from the dashboard and via `bot-bottle start` alongside a Claude agent.
|
||||
- The manifest can express the agent provider/template and, where needed, a custom agent Dockerfile.
|
||||
- Claude-specific default egress/auth behavior is no longer implicit; provider-specific auth is expressed through explicit bottle egress routes and roles.
|
||||
- The launcher preserves required infrastructure behavior for sidecars, egress, pipelock, supervisor MCP, CA handling, git, and shell basics.
|
||||
|
||||
@@ -30,7 +30,7 @@ across every bottle spin-up. This has several consequences:
|
||||
to grant that access.
|
||||
- **Manual rotation burden.** Operators must manage key files on disk, keeping
|
||||
them secure, rotating them on a schedule, and distributing them across hosts
|
||||
that run `./cli.py start`.
|
||||
that run `bot-bottle start`.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
## Summary
|
||||
|
||||
The `./cli.py dashboard` command has grown from its PRD 0013 roots
|
||||
The `bot-bottle dashboard` command has grown from its PRD 0013 roots
|
||||
(triage supervise proposals) into a parallel-agent control surface
|
||||
(PRDs 0019/0020/0021): an active-agents pane, agent picker + start,
|
||||
re-attach, per-bottle stop, tmux split-pane handoff, operator-
|
||||
@@ -21,7 +21,7 @@ proposals, approve / modify / reject each one, write audit entries,
|
||||
deliver the response that unblocks the agent's tool call. Everything
|
||||
that's about *starting / re-entering / stopping* bottles, or about
|
||||
*operator-initiated* config edits, comes out. The command is renamed
|
||||
`./cli.py supervise` so the name matches what it does after the cut.
|
||||
`bot-bottle supervise` so the name matches what it does after the cut.
|
||||
|
||||
Future agent-management UX is explicitly punted: if and when a
|
||||
control surface for parallel agents resurfaces, the working
|
||||
@@ -41,9 +41,9 @@ Three concrete pains, all downstream of the dashboard's growth:
|
||||
ExitStack-free bottle ownership are intricate enough that
|
||||
shipping the next polish increment costs more than it returns.
|
||||
2. **No clear ownership of "starts and stops bottles".** Today
|
||||
that responsibility is split: `./cli.py start` owns one-shot
|
||||
that responsibility is split: `bot-bottle start` owns one-shot
|
||||
sessions; the dashboard owns multi-session bottles it started
|
||||
itself; `./cli.py cleanup` owns everything else. The dashboard
|
||||
itself; `bot-bottle cleanup` owns everything else. The dashboard
|
||||
tracking its own `bottles: dict[str, (cm, bottle, identity)]`
|
||||
that doesn't survive a quit is a confusing third lane.
|
||||
3. **Wrong target shape for a "manage many agents" UI.** The
|
||||
@@ -97,12 +97,12 @@ problem is everything that got bolted onto that core after.
|
||||
dashboard. After this PRD they don't exist anywhere — operators
|
||||
who need ad-hoc edits use the same path the agents do (call the
|
||||
supervise tool from inside the bottle) or hand-edit the host-
|
||||
side files and restart the sidecar. Adding a `./cli.py routes
|
||||
side files and restart the sidecar. Adding a `bot-bottle routes
|
||||
edit <slug>` verb is a follow-up if the loss bites.
|
||||
- **Removing `./cli.py start` or changing its semantics.** Start
|
||||
- **Removing `bot-bottle start` or changing its semantics.** Start
|
||||
remains the one-shot launch path. PRD 0020's bottle-outlives-
|
||||
process model is removed; the only path to a long-running
|
||||
bottle is `./cli.py start` (foreground) plus `cli.py cleanup`
|
||||
bottle is `bot-bottle start` (foreground) plus `cli.py cleanup`
|
||||
for teardown.
|
||||
- **Removing the supervise-sidecar protocol or any of the three
|
||||
block-remediation engines.** PRDs 0013–0016 stay Active. The
|
||||
@@ -122,8 +122,8 @@ problem is everything that got bolted onto that core after.
|
||||
|
||||
### In scope
|
||||
|
||||
- **Rename the subcommand.** `./cli.py dashboard` becomes
|
||||
`./cli.py supervise`. The module moves from `bot_bottle/cli/
|
||||
- **Rename the subcommand.** `bot-bottle dashboard` becomes
|
||||
`bot-bottle supervise`. The module moves from `bot_bottle/cli/
|
||||
dashboard.py` to `bot_bottle/cli/supervise.py`. The dispatcher
|
||||
in `bot_bottle/cli/__init__.py` and the help text both update.
|
||||
- **Strip the curses loop to proposal-only.** The remaining
|
||||
@@ -167,7 +167,7 @@ problem is everything that got bolted onto that core after.
|
||||
|
||||
- Any new feature in the supervise TUI. The cut is purely
|
||||
subtractive (except for the rename).
|
||||
- Behavior changes in `./cli.py start`, `cli.py cleanup`,
|
||||
- Behavior changes in `bot-bottle start`, `cli.py cleanup`,
|
||||
`cli.py resume`, `cli.py list`, `cli.py info`, `cli.py edit`,
|
||||
`cli.py init` — unchanged.
|
||||
- Changes to the supervise sidecar (`supervise_server.py`,
|
||||
@@ -181,7 +181,7 @@ problem is everything that got bolted onto that core after.
|
||||
|
||||
### Final shape of the TUI
|
||||
|
||||
After this PRD the `./cli.py supervise` curses surface is:
|
||||
After this PRD the `bot-bottle supervise` curses surface is:
|
||||
|
||||
```
|
||||
bot-bottle supervise (3 pending)
|
||||
@@ -307,8 +307,8 @@ The PR closes issue #174.
|
||||
|
||||
1. **`e` / `p` operator-initiated edits — gone for good or
|
||||
moved to a separate CLI verb?** The PRD removes them with no
|
||||
replacement. The simplest replacement is `./cli.py routes
|
||||
edit <slug>` and `./cli.py pipelock edit <slug>`, sharing
|
||||
replacement. The simplest replacement is `bot-bottle routes
|
||||
edit <slug>` and `bot-bottle pipelock edit <slug>`, sharing
|
||||
the existing `apply_routes_change` / `apply_allowlist_change`
|
||||
engines. If the loss is felt within the first parallel
|
||||
run after this lands, that follow-up is a small PR. Leaving
|
||||
|
||||
@@ -7,12 +7,12 @@
|
||||
|
||||
## Summary
|
||||
|
||||
When `./cli.py start` is run without an agent name, or without a backend
|
||||
When `bot-bottle start` is run without an agent name, or without a backend
|
||||
explicitly specified, the user currently gets an argparse error (missing
|
||||
positional) or falls through to the `docker` default silently. This PRD
|
||||
adds a terminal UI that appears in those gaps: a filter-select screen
|
||||
built with `curses` that lets the operator pick the agent and/or backend
|
||||
interactively rather than memorising names or consulting `./cli.py list`.
|
||||
interactively rather than memorising names or consulting `bot-bottle list`.
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -29,15 +29,15 @@ visible.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
1. `./cli.py start` (no arguments) shows an interactive agent selector;
|
||||
1. `bot-bottle start` (no arguments) shows an interactive agent selector;
|
||||
the selected name is used exactly as if it had been passed on the
|
||||
command line.
|
||||
2. `./cli.py start <name>` (no `--backend`, no `BOT_BOTTLE_BACKEND`)
|
||||
2. `bot-bottle start <name>` (no `--backend`, no `BOT_BOTTLE_BACKEND`)
|
||||
shows an interactive backend selector; the selected backend is used
|
||||
exactly as if `--backend=<selected>` had been passed.
|
||||
3. `./cli.py start <name> --backend=<b>` (both explicit) shows neither
|
||||
3. `bot-bottle start <name> --backend=<b>` (both explicit) shows neither
|
||||
screen — no behavioural change from today.
|
||||
4. `./cli.py start` (no arguments, no env backend) shows the agent
|
||||
4. `bot-bottle start` (no arguments, no env backend) shows the agent
|
||||
selector first, then the backend selector.
|
||||
5. The filter-select widget is a standalone utility
|
||||
(`bot_bottle/cli/tui.py`) shared by both selectors.
|
||||
@@ -57,7 +57,7 @@ visible.
|
||||
- No pagination beyond what fits in the terminal window (scroll via
|
||||
cursor movement is sufficient for typical agent counts).
|
||||
- No multi-select; exactly one item is chosen per invocation.
|
||||
- No changes to `./cli.py resume`, `./cli.py list`, or any other
|
||||
- No changes to `bot-bottle resume`, `bot-bottle list`, or any other
|
||||
subcommand.
|
||||
|
||||
## Design
|
||||
@@ -83,7 +83,7 @@ def filter_select(
|
||||
|
||||
The widget renders to the tty file descriptor opened via `curses.initscr`
|
||||
(or `curses.newterm` on the tty fd so stdout remains clean for callers
|
||||
that pipe `./cli.py`).
|
||||
that pipe `bot-bottle`).
|
||||
|
||||
Layout (full-width, minimal):
|
||||
|
||||
@@ -140,7 +140,7 @@ agent picker can populate itself from the real manifest. The same
|
||||
`filter_select` opens `/dev/tty` and feeds it as the input file to
|
||||
`curses.wrapper`-equivalent code (using `curses.newterm` to avoid
|
||||
clobbering the caller's stdout/stderr). This keeps the picker
|
||||
composable — callers can pipe `./cli.py` output without the curses
|
||||
composable — callers can pipe `bot-bottle` output without the curses
|
||||
draw sequences contaminating the pipe.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
@@ -30,12 +30,12 @@ snapshot before a planned host reboot or hardware migration.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `./cli.py commit [<slug>]` takes a snapshot of the running agent and
|
||||
- `bot-bottle commit [<slug>]` takes a snapshot of the running agent and
|
||||
stores it as a local artifact.
|
||||
- Without a slug argument the command shows the same interactive picker
|
||||
as `start` (the list of active slugs).
|
||||
- The committed artifact reference is stored in per-bottle state so
|
||||
that the next `./cli.py resume <slug>` automatically uses the
|
||||
that the next `bot-bottle resume <slug>` automatically uses the
|
||||
snapshot instead of rebuilding from the Dockerfile.
|
||||
- `mark_preserved` is called so the state dir survives the normal
|
||||
session-end cleanup.
|
||||
@@ -81,7 +81,7 @@ to the committed `.smolmachine` artifact.
|
||||
### `commit` command
|
||||
|
||||
```
|
||||
./cli.py commit [<slug>]
|
||||
bot-bottle commit [<slug>]
|
||||
```
|
||||
|
||||
1. Resolve slug (arg or interactive picker from `enumerate_active_agents`).
|
||||
|
||||
@@ -37,7 +37,7 @@ egress policy), the operator must duplicate the agent file and change the
|
||||
selection order, as the effective bottle for the session.
|
||||
5. Confirming with an empty selection falls back to the agent's `bottle:` field.
|
||||
If neither is set, a ManifestError is raised pointing the operator at the fix.
|
||||
6. The ordered bottle list is stored in launch metadata so `./cli.py resume`
|
||||
6. The ordered bottle list is stored in launch metadata so `bot-bottle resume`
|
||||
uses the same bottles.
|
||||
7. The preflight summary (`y/N` screen) shows the effective bottle name(s).
|
||||
8. The multi-select picker supports incremental filtering, Space/Enter to toggle
|
||||
@@ -52,7 +52,7 @@ egress policy), the operator must duplicate the agent file and change the
|
||||
- Reordering the selection list from within the picker (order = insertion order;
|
||||
drag-and-drop is out of scope).
|
||||
- Storing bottle selection history / MRU.
|
||||
- Changes to `./cli.py edit`, `./cli.py list`, or `./cli.py info`.
|
||||
- Changes to `bot-bottle edit`, `bot-bottle list`, or `bot-bottle info`.
|
||||
- Removing the `bottle:` key from the agent schema (it stays, now optional).
|
||||
|
||||
## Design
|
||||
|
||||
@@ -74,7 +74,7 @@ macOS-only for v1. Three concrete blockers:
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` launches,
|
||||
- `BOT_BOTTLE_BACKEND=smolmachines bot-bottle start <agent>` launches,
|
||||
runs, and tears down a bottle on a Linux host with `/dev/kvm`.
|
||||
- The TSI allowlist is enforced on Linux: PRD 0022's
|
||||
`tests/integration/test_sandbox_escape.py` passes against
|
||||
|
||||
@@ -39,7 +39,7 @@ end-to-end runner that would catch the *next* macOS-only launch regression.)
|
||||
PR #470 (#414) already made the integration suite backend-agnostic:
|
||||
`skip_unless_selected_backend_available()` gates on the *selected* backend's
|
||||
own `is_backend_ready()` rather than `docker_available()`, and each
|
||||
integration job runs `./cli.py backend status --backend=<name>` as a preflight
|
||||
integration job runs `bot-bottle backend status --backend=<name>` as a preflight
|
||||
that fails loudly when the backend is missing. That is the machinery this job
|
||||
plugs into; this PRD supplies the runner and the job.
|
||||
|
||||
@@ -98,7 +98,7 @@ Modeled on `integration-firecracker`:
|
||||
- `concurrency: { group: integration-macos-infra, cancel-in-progress: false }`
|
||||
to serialize runs against the singleton.
|
||||
- **Preflight** — `command -v container`, `container system status`, then
|
||||
`./cli.py backend status --backend=macos-container`; any failure exits
|
||||
`bot-bottle backend status --backend=macos-container`; any failure exits
|
||||
non-zero so a misprovisioned runner fails loudly instead of silently
|
||||
skipping.
|
||||
- Run the integration suite under coverage with
|
||||
|
||||
@@ -16,7 +16,7 @@ verifies host prerequisites after install.
|
||||
## Problem
|
||||
|
||||
There is currently no install path for new users. The only way to run
|
||||
bot-bottle is to clone the repo and invoke `./cli.py`. This blocks any
|
||||
bot-bottle is to clone the repo and invoke `bot-bottle`. This blocks any
|
||||
public demo: readers want `curl | sh` or `pipx install`, not a manual
|
||||
clone-and-configure flow. There is also no single command that tells a
|
||||
user whether their host is actually ready to run a bottle.
|
||||
|
||||
@@ -0,0 +1,391 @@
|
||||
# PRD 0082: Trusted agent forge identity and guidance
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Author:** didericis-claude
|
||||
- **Created:** 2026-07-25
|
||||
- **Issue:** #423
|
||||
|
||||
## Summary
|
||||
|
||||
Move author identity and named forge configurations into host-trusted agent
|
||||
definitions. A bottle may optionally associate a git-gate repository with one
|
||||
of the selected agent's forge aliases. When associated, bot-bottle gives the
|
||||
agent non-secret, provider-specific system guidance for that repository and
|
||||
routes authenticated forge API calls through the egress proxy without exposing
|
||||
the forge token to the bottle. The authenticated account is inferred from the
|
||||
identity that owns that token; it is not separately declared in the manifest.
|
||||
|
||||
Repository-local agent and bottle definitions are no longer discovered. Once
|
||||
agent definitions can select a host forge credential, allowing checked-out
|
||||
repository content to define or override an agent would let untrusted workspace
|
||||
content select host identities and secrets.
|
||||
|
||||
This PRD is deliberately limited to identity ownership, manifest trust, forge
|
||||
API access, and generated guidance. Per-activation signing, signature
|
||||
enforcement, commit attribution, and the commit audit model move to a follow-up
|
||||
PRD.
|
||||
|
||||
Successor to:
|
||||
|
||||
- **PRD 0011 (per-file manifests)** — allowed repository-local agent files to
|
||||
override home agents. This PRD removes that trust path: agents and bottles are
|
||||
loaded only from the host-owned `~/.bot-bottle` tree.
|
||||
- **PRD 0027 (agent git identity, #94)** / **ADR 0002** — put name/email in
|
||||
`git-gate.user` while keeping them claimed rather than vouched. This PRD moves
|
||||
those agent properties out of git-gate and into the trusted agent definition.
|
||||
- **PRD 0048 (deploy-key provisioning, #169)** — remains the Git push
|
||||
capability. A forge actor token is a separate API credential and never
|
||||
replaces a deploy key.
|
||||
|
||||
## Problem
|
||||
|
||||
### Forge workflow context is missing
|
||||
|
||||
The agent prompt does not know which forge backs a git-gate repository, which
|
||||
API base URL to use, or that authenticated requests must go through the egress
|
||||
proxy. Bespoke prompt text has drifted between agents. Missing guidance has
|
||||
already caused incorrect PR behavior, including attempts to use Gitea AGit
|
||||
review refs instead of a normal branch-backed pull request.
|
||||
|
||||
### Identity is owned by the wrong layer
|
||||
|
||||
`git-gate.user` puts an agent property on a repository transport component. An
|
||||
author identity and set of forge credentials should follow the agent across
|
||||
bottles and repositories. Git-gate should own Git transport policy, not decide
|
||||
who the agent is.
|
||||
|
||||
### Repository-local agents become a credential-selection path
|
||||
|
||||
Today `$CWD/.bot-bottle/agents/*.md` can define new agents and override
|
||||
home-resident agents. If an agent definition may reference an operator-provided
|
||||
forge token, a malicious repository could select a host credential merely by
|
||||
being the current workspace. Repository-local bottles are already ignored;
|
||||
agents need the same host-only boundary.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- **Agent-owned identity.** Author name/email and named forge configurations
|
||||
live on the agent definition, not under `git-gate`.
|
||||
- **Trusted definitions only.** Agent and bottle files are discovered only
|
||||
under `~/.bot-bottle/{agents,bottles}`. Repository-local definitions never
|
||||
contribute names, defaults, or overrides.
|
||||
- **Optional repository association.** A bottle repository may name one forge
|
||||
alias from the selected agent. Repositories without `forge` retain current
|
||||
behavior and generate no forge guidance or credential route.
|
||||
- **Proxy-held forge credential.** The host resolves the selected forge
|
||||
alias's token reference and gives the value only to the egress proxy. The
|
||||
bottle receives neither the token nor a credential file containing it.
|
||||
- **Forge-aware system guidance.** For associated repositories, bot-bottle
|
||||
generates provider-specific instructions describing the API base URL,
|
||||
repository/account association, proxy-authenticated access, and correct PR
|
||||
workflow.
|
||||
- **No secret prompt material.** Neither token values nor token
|
||||
environment-variable names appear in the prompt or bottle environment.
|
||||
- **Fail closed.** Unknown account references, unsupported/non-HTTPS URLs,
|
||||
missing host secrets, and misplaced agent/bottle fields fail before creating
|
||||
the bottle.
|
||||
- **Push capability unchanged.** Git transport remains PRD 0048 deploy keys.
|
||||
The forge actor token is only for API actions such as opening, reviewing, and
|
||||
commenting on pull requests.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Commit signing or signature enforcement.** No signing key is minted and
|
||||
git-gate does not verify commit signatures in this PRD.
|
||||
- **Commit attribution or audit tables.** There is no trustworthy commit
|
||||
observation event in this slice. A follow-up signing PRD owns activation keys,
|
||||
control-plane verification, and any configured-author-versus-claimed-author
|
||||
audit model.
|
||||
- **Cryptographically vouched author identity.** `author` configures Git and
|
||||
identifies the agent actor, but commit author/committer fields remain claims
|
||||
under ADR 0002.
|
||||
- **Forge account or token minting.** The operator creates the agent-specific
|
||||
account/token out of band. Bot-bottle references an existing host secret; it
|
||||
does not create, rotate, or revoke that credential.
|
||||
- **Forge-side commit attribution surfaces.** No commit status, signing-key
|
||||
registration, or "Verified" badge.
|
||||
- **Provider-generic arbitrary prompts.** Gitea is the first supported forge.
|
||||
A future provider adds typed validation and generated guidance in code rather
|
||||
than accepting repository-supplied prompt text.
|
||||
|
||||
## Design
|
||||
|
||||
### Ownership model
|
||||
|
||||
The resolved bottled agent is an agent definition composed with a bottle:
|
||||
|
||||
| Part | Source | Role |
|
||||
|------|--------|------|
|
||||
| Author identity | agent `author` | configures Git name/email and identifies the agent's claimed author |
|
||||
| Forge configurations | agent `forge-accounts` | maps forge aliases to API origins and host token references available to this agent; token ownership determines account identity |
|
||||
| Git repository | bottle `git-gate.repos` | configures Git transport, host verification, and push capability |
|
||||
| Repository/forge association | bottle repo `forge` | optionally selects an agent forge alias for guidance and API access |
|
||||
|
||||
This boundary keeps identity on the agent, capability/policy on the bottle, and
|
||||
transport enforcement in git-gate.
|
||||
|
||||
### Agent manifest
|
||||
|
||||
Agent identity and forge accounts live only in
|
||||
`~/.bot-bottle/agents/<name>.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
author:
|
||||
name: didericis-claude
|
||||
email: eric+claude@dideric.is
|
||||
forge-accounts:
|
||||
didericis-gitea:
|
||||
auth:
|
||||
type: token
|
||||
token_secret: GITEA_CLAUDE_TOKEN
|
||||
url: https://gitea.dideric.is/api/v1
|
||||
---
|
||||
```
|
||||
|
||||
`author` contains:
|
||||
|
||||
- `name`: non-empty string;
|
||||
- `email`: non-empty string passing the existing Git identity validation.
|
||||
|
||||
The resolved values populate `user.name` and `user.email`. Existing
|
||||
`git-gate.user` fields fail with migration guidance to move the values into the
|
||||
selected home agent's `author` block. There is no compatibility period where a
|
||||
bottle identity silently overrides the agent identity.
|
||||
|
||||
`forge-accounts` is a map whose keys are **forge aliases** and follow the
|
||||
manifest's existing kebab-case identifier grammar
|
||||
(`[a-z][a-z0-9-]*`). Each forge entry contains:
|
||||
|
||||
- `url`: a canonical HTTPS Gitea API base URL;
|
||||
- `auth.type`: `token` in this slice;
|
||||
- `auth.token_secret`: the name of a host environment variable containing the
|
||||
operator-provided, agent-specific API token.
|
||||
|
||||
The token's owner determines the authenticated forge account; there is no
|
||||
separate account-name field. The token secret name is host configuration, not
|
||||
bottle configuration. The token value is resolved only if a selected bottle
|
||||
repository references the forge alias.
|
||||
|
||||
### Bottle manifest
|
||||
|
||||
Repository policy remains in `~/.bot-bottle/bottles/<name>.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
bot-bottle:
|
||||
url: ssh://git@100.78.141.42:30009/didericis/bot-bottle.git
|
||||
provisioned_key:
|
||||
provider: gitea
|
||||
token_env: GITEA_DEPLOY_TOKEN
|
||||
host_key: "ssh-ed25519 AAAA..."
|
||||
forge: didericis-gitea
|
||||
---
|
||||
```
|
||||
|
||||
`git-gate.repos.<name>.forge` is optional:
|
||||
|
||||
- When absent, the repository behaves exactly as it does today. Bot-bottle does
|
||||
not resolve a forge actor token and does not add forge-specific instructions
|
||||
for that repository.
|
||||
- When present, it must match a forge alias in `forge-accounts` on the selected
|
||||
agent.
|
||||
The resolved association enables a scoped proxy credential route and adds the
|
||||
repository/forge relationship to generated system guidance.
|
||||
|
||||
`provisioned_key.token_env` remains the deploy-key administration credential
|
||||
from PRD 0048. It is separate from `forge-accounts.*.auth.token_secret`: the
|
||||
former provisions Git push capability, while the latter performs API actions as
|
||||
the agent.
|
||||
|
||||
`author` and `forge-accounts` are agent-only. `git-gate.repos`, including
|
||||
`forge`, is bottle-only. Validation errors point to the correct file and trust
|
||||
domain rather than ignoring misplaced keys.
|
||||
|
||||
### Definition trust and discovery
|
||||
|
||||
Only the host-owned manifest tree is authoritative:
|
||||
|
||||
- Agents: `~/.bot-bottle/agents/*.md`
|
||||
- Bottles: `~/.bot-bottle/bottles/*.md`
|
||||
|
||||
`$CWD/.bot-bottle/agents/*.md` no longer contributes new agents and no longer
|
||||
overrides a home agent. `$CWD/.bot-bottle/bottles/*.md` remains unusable. If
|
||||
either repository-local directory contains manifest files, bot-bottle emits a
|
||||
warning that they are ignored and points to the corresponding home path.
|
||||
|
||||
Every discovery and resolution surface uses the same home-only index:
|
||||
|
||||
- agent enumeration and selectors;
|
||||
- `require_agent`;
|
||||
- lazy `load_for_agent`;
|
||||
- default-agent/default-bottle resolution;
|
||||
- dashboard and headless launch paths.
|
||||
|
||||
There must be no alternate direct-path load that can still select a workspace
|
||||
definition.
|
||||
|
||||
Workspace instructions remain repository content (for example `AGENTS.md`),
|
||||
but runtime policy, host secret references, and actor identity do not.
|
||||
Programmatic in-memory manifests remain available for tests and trusted internal
|
||||
composition; they are not a filesystem discovery path.
|
||||
|
||||
### Forge URL validation
|
||||
|
||||
The host parses and canonicalizes each referenced forge alias's URL before
|
||||
creating any runtime resources:
|
||||
|
||||
- scheme must be `https`;
|
||||
- userinfo, query, and fragment are forbidden;
|
||||
- hostname must be present;
|
||||
- the path must be a supported Gitea API base (initially `/api/v1`, with
|
||||
normalization of a trailing slash);
|
||||
- visually different inputs that canonicalize to the same origin/prefix are
|
||||
deduplicated;
|
||||
- unsupported providers or path shapes fail closed.
|
||||
|
||||
The provider is determined by typed support in bot-bottle, not by prompt text
|
||||
from a repository. Adding another provider requires a validator, auth scheme,
|
||||
and guidance renderer.
|
||||
|
||||
### Proxy credential provisioning
|
||||
|
||||
For each distinct forge alias referenced by at least one selected bottle
|
||||
repository, the host:
|
||||
|
||||
1. Resolves `auth.token_secret` from the host environment and rejects a missing
|
||||
or empty value.
|
||||
2. Copies the token value only into the egress proxy's credential environment.
|
||||
3. Adds an inspected route scoped to the canonical forge origin and API prefix.
|
||||
4. Configures the provider authentication scheme (`token` for Gitea) so the
|
||||
proxy injects authentication.
|
||||
|
||||
The bottle makes an unauthenticated request to the configured HTTPS API URL.
|
||||
The token is not copied into the bottle, `.gitconfig`, generated prompt,
|
||||
workspace, or process environment visible to the agent.
|
||||
|
||||
If several repositories reference the same forge alias, they share one
|
||||
credential route. Unreferenced forge aliases resolve no secret and create no
|
||||
route.
|
||||
|
||||
### Generated system guidance
|
||||
|
||||
Bot-bottle appends a generated, non-secret section to its existing system
|
||||
prompt file. It is derived from validated typed fields, not copied Markdown from
|
||||
a repository.
|
||||
|
||||
For each associated repository, Gitea guidance includes:
|
||||
|
||||
- forge alias (for example `didericis-gitea`);
|
||||
- API base URL (for example `https://gitea.dideric.is/api/v1`);
|
||||
- the git-gate repository name tied to that forge;
|
||||
- the instruction to call the configured HTTPS API through the proxy without
|
||||
reading, printing, or manually attaching an authorization token;
|
||||
- the distinction that Git pushes still use the git-gate remote;
|
||||
- the requirement to create/update a normal `refs/heads/<branch>` and open a
|
||||
branch-backed pull request through the API;
|
||||
- the prohibition on pushing `refs/for/*`, `refs/draft/*`, or
|
||||
`refs/for-review/*`;
|
||||
- the instruction to use the API for reviews/comments and verify returned
|
||||
object state before claiming completion.
|
||||
|
||||
The prompt contains neither the token value nor its `token_secret` name.
|
||||
Repositories without `forge` are omitted from this section. If no selected
|
||||
repository has a forge association, no forge guidance section is generated.
|
||||
|
||||
### Failure and lifecycle behavior
|
||||
|
||||
Forge configuration is validated before bottle creation. A bad association or
|
||||
credential must not leave a partially-created bottle or proxy.
|
||||
|
||||
The operator-owned token is not minted, rotated, or revoked by bot-bottle. On
|
||||
teardown, stopping the egress proxy discards the activation's in-memory/runtime
|
||||
copy. Later activations resolve the current host secret again.
|
||||
|
||||
Logs may include the forge alias, canonical API origin, and repository name.
|
||||
They must never contain the token value. Errors for missing secrets name the
|
||||
configuration field and host environment variable, but do not print any value.
|
||||
|
||||
## Migration
|
||||
|
||||
This change is intentionally breaking at the manifest trust boundary:
|
||||
|
||||
1. Move each home bottle/agent `git-gate.user.name` and `.email` into the
|
||||
corresponding home agent's `author`.
|
||||
2. Add agent-specific `forge-accounts` only to home agent definitions.
|
||||
3. Add optional `forge` associations to home bottle repository entries.
|
||||
4. Move any `$CWD/.bot-bottle/agents/*.md` that should remain selectable into
|
||||
`~/.bot-bottle/agents/`. Repository copies are ignored thereafter.
|
||||
5. Keep repository-specific behavioral instructions in `AGENTS.md` or another
|
||||
workspace instruction file; do not put runtime identity or secret references
|
||||
there.
|
||||
|
||||
Errors and warnings link to this migration rather than silently changing which
|
||||
identity or definition is active.
|
||||
|
||||
## Follow-up: signed commits and attribution
|
||||
|
||||
A separate PRD will consume the trusted agent `author` introduced here and own:
|
||||
|
||||
- per-activation signing key minting and sidecar isolation;
|
||||
- commit-time signing through a forwarded signing capability;
|
||||
- git-gate rejection of unsigned/wrong-key new commits;
|
||||
- independent control-plane object-ID recomputation and signature verification;
|
||||
- activation and attributed-commit audit tables;
|
||||
- storage of configured agent author separately from each commit's unenforced
|
||||
claimed author/committer;
|
||||
- post-teardown verification and key-retention policy.
|
||||
|
||||
That PRD must not reintroduce identity under git-gate or expand repository-local
|
||||
manifest trust.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **This PRD.** Establish the identity, forge, and filesystem trust boundary.
|
||||
2. **Home-only definitions.** Remove cwd agents from discovery, override,
|
||||
enumeration, lazy loading, defaults, and selectors. Warn on ignored cwd
|
||||
agent/bottle files and provide migration guidance.
|
||||
3. **Manifest schema.** Add agent-only `author` and `forge-accounts`; remove
|
||||
`git-gate.user`; add optional bottle-only
|
||||
`git-gate.repos.<name>.forge`. Validate account names, composition
|
||||
references, field placement, and Gitea API URLs.
|
||||
4. **Proxy provisioning.** Lazily resolve only referenced token secrets and
|
||||
create scoped authenticated Gitea API routes without putting credentials in
|
||||
the bottle.
|
||||
5. **Prompt generation.** Render typed Gitea/repository workflow guidance into
|
||||
the existing bot-bottle prompt path for associated repositories only.
|
||||
6. **Docs and migration.** Update README examples, PRD 0011-facing discovery
|
||||
documentation, agent/bottle schema docs, and error guidance.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Trust boundary:** cwd agent files are ignored with a warning, cannot
|
||||
override a home agent, are absent from enumeration/selectors/defaults, and
|
||||
cannot be loaded by name. Cwd bottle behavior remains home-only.
|
||||
- **Agent schema:** `author` and `forge-accounts` parse; malformed identities,
|
||||
non-kebab account names, unknown fields, invalid auth types, and misplaced
|
||||
git-gate fields fail clearly.
|
||||
- **Bottle schema:** `forge` is optional; absent associations preserve current
|
||||
behavior; present associations resolve against the selected agent; unknown or
|
||||
misplaced associations fail before launch.
|
||||
- **URL validation:** HTTPS Gitea API bases pass and canonicalize; HTTP,
|
||||
userinfo, query, fragment, missing host, unsupported paths, and unsupported
|
||||
providers fail closed.
|
||||
- **Secret handling:** only referenced accounts resolve environment secrets;
|
||||
missing/empty secrets fail before runtime creation; token values are absent
|
||||
from bottle env, prompt, generated config, logs, and workspace; token secret
|
||||
names are absent from the bottle and prompt.
|
||||
- **Proxy behavior:** associated API requests receive proxy-injected Gitea
|
||||
authentication scoped to the configured origin/prefix; unrelated hosts and
|
||||
paths receive no credential.
|
||||
- **Prompt behavior:** guidance names account/API/repository associations,
|
||||
branch-backed PR workflow, prohibited AGit refs, and mutation verification;
|
||||
repositories without `forge` are omitted; no associations means no section.
|
||||
- **Migration:** legacy `git-gate.user` fails with an `author` migration pointer;
|
||||
ignored cwd definitions warn with the target home path.
|
||||
|
||||
## Open questions
|
||||
|
||||
- None.
|
||||
@@ -32,9 +32,17 @@ not a principled scope exclusion: both are major hosted sandbox platforms and
|
||||
belong in this landscape even though they target platform builders rather than
|
||||
bot-bottle's local single-operator workflow.
|
||||
|
||||
Updated 2026-07-27 after a scan of recent Show HN launches: **Black LLAB,
|
||||
Eve, CloudRouter, Nucleus, yolo-cage, and Sandbox Agent SDK** added as a
|
||||
dated entrant cohort. They sharpen the comparison on three axes the original
|
||||
table underweighted: the browser/preview loop, parallel-agent operator UX, and
|
||||
a provider-neutral automation/session API.
|
||||
|
||||
## Summary
|
||||
|
||||
The main table compares bot-bottle against fifteen isolation/sandbox tools.
|
||||
The main table compares bot-bottle against fifteen canonical
|
||||
isolation/sandbox tools; a later section evaluates six recent HN entrants
|
||||
without widening an already unwieldy table.
|
||||
Governance/pre-action authorization and credential-only layers are covered
|
||||
separately because they don't provide VM or container isolation. None
|
||||
duplicate bot-bottle's combination of local
|
||||
@@ -542,6 +550,199 @@ them.
|
||||
framework runtime is not compromised.
|
||||
- **Maturity**: Specification + reference implementation, 2026.
|
||||
|
||||
## Recent HN entrants (added 2026-07-27)
|
||||
|
||||
These are grouped by launch date rather than promoted into the main table.
|
||||
Several are young or sparsely documented, and putting them beside mature
|
||||
runtime platforms with false precision would obscure the useful comparison.
|
||||
The HN launch posts are the evidence snapshot; feature claims should be
|
||||
rechecked against their repositories before relying on them for a security
|
||||
decision.
|
||||
|
||||
### Black LLAB
|
||||
|
||||
- **Source**: https://github.com/isaacdear/black-llab ;
|
||||
HN launch https://news.ycombinator.com/item?id=47402394
|
||||
- **Isolation/locality**: Local Docker environment, with an isolated container
|
||||
created for each agent task. Shared host kernel; no stronger boundary is
|
||||
claimed.
|
||||
- **Agent integration**: General local/cloud model workspace. Its headline is
|
||||
dynamic routing of simple prompts to local models and complex prompts to
|
||||
hosted models, with code execution and web scraping inside the task
|
||||
container.
|
||||
- **Network/credentials**: No default-deny egress, payload inspection, or
|
||||
host-side credential injection documented in the launch.
|
||||
- **Competitive read**: Superficial overlap ("a container per agent task"),
|
||||
but not a direct security-policy competitor. Its useful challenge is the
|
||||
integrated model-selection UX, which bot-bottle intentionally leaves to the
|
||||
selected agent provider.
|
||||
- **Maturity**: Early solo project; HN launch received 1 point.
|
||||
|
||||
### Eve
|
||||
|
||||
- **Source**: https://eve.new/ ;
|
||||
HN launch https://news.ycombinator.com/item?id=47721255
|
||||
- **Isolation/locality**: Managed, hosted Linux sandbox per user/session
|
||||
(claimed 2 vCPU, 4 GB RAM, 10 GB disk), with filesystem, code execution,
|
||||
headless Chromium, and service connectors.
|
||||
- **Agent integration**: End-user OpenClaw-style agent product. An orchestrator
|
||||
routes subtasks to specialist models and can run parallel subagents that
|
||||
coordinate through a shared filesystem. Web UI and iMessage are primary
|
||||
interaction surfaces.
|
||||
- **Network/credentials**: Broad connectors are a product feature; the launch
|
||||
does not document bot-bottle-style default-deny route policy, content DLP,
|
||||
or credentials held outside the sandbox.
|
||||
- **Competitive read**: Adjacent, not direct. Eve sells a managed colleague;
|
||||
bot-bottle lets an operator run existing coding-agent CLIs under local
|
||||
containment. Eve nevertheless demonstrates the appeal of background work,
|
||||
live progress, browser capability, and mobile notification.
|
||||
- **Maturity**: Commercial hosted product; HN launch received 71 points and
|
||||
39 comments.
|
||||
|
||||
### CloudRouter
|
||||
|
||||
- **Source**: https://github.com/manaflow-ai/manaflow/tree/main/packages/cloudrouter ;
|
||||
HN launch https://news.ycombinator.com/item?id=47006393
|
||||
- **Isolation/locality**: Claude Code or Codex runs locally and provisions
|
||||
remote cloud VMs/GPUs for execution. Project files are uploaded to the VM;
|
||||
each machine exposes auth-protected VNC, VS Code, and Jupyter surfaces.
|
||||
- **Agent integration**: A skill plus CLI lets the coding agent itself start,
|
||||
command, inspect, and tear down machines. Browser automation is integrated,
|
||||
including snapshots and screenshots. Parallel disposable compute is the
|
||||
central workflow.
|
||||
- **Network/credentials**: The launch emphasizes remote resource isolation and
|
||||
authenticated UI endpoints, not default-deny guest egress, payload DLP, or
|
||||
proxy-held application credentials.
|
||||
- **Competitive read**: The closest recent workflow competitor. It directly
|
||||
addresses parallel coding agents, environmental conflict, and closing the
|
||||
browser/test loop, but trades local custody for elastic cloud compute.
|
||||
Cloud VMs and GPUs could be a future bot-bottle backend; they do not replace
|
||||
its manifest/policy layer.
|
||||
- **Maturity**: Active open-source monorepo project; HN launch received
|
||||
138 points and 36 comments.
|
||||
|
||||
### Nucleus
|
||||
|
||||
- **Source**: https://github.com/coproduct-opensource/nucleus ;
|
||||
HN launch https://news.ycombinator.com/item?id=46855770
|
||||
- **Isolation/locality**: Firecracker microVM with an enforcing MCP tool proxy.
|
||||
- **Agent integration/config**: Compositional permission envelope for
|
||||
read/write/run actions. The envelope is non-escalating and can tighten or
|
||||
terminate, with scoped approval tokens for gated operations.
|
||||
- **Network/credentials**: Default-deny egress, DNS allowlist, iptables drift
|
||||
detection, time/budget caps, and hash-chained audit logging are claimed.
|
||||
Remote append-only audit storage and attestation were roadmap items at
|
||||
launch.
|
||||
- **Competitive read**: Direct on security architecture, especially
|
||||
non-escalating policy and tamper-evident audit. It is an early execution/tool
|
||||
proxy rather than a provider-neutral, one-command coding-agent product. Its
|
||||
tool-level action envelope is semantically finer than bot-bottle's network
|
||||
boundary; bot-bottle is stronger on turnkey agent/provider integration,
|
||||
credential custody, Git mediation, and long-running operator workflow.
|
||||
- **Maturity**: Early OSS experiment; HN launch received 3 points.
|
||||
|
||||
### yolo-cage
|
||||
|
||||
- **Source**: https://github.com/borenstein/yolo-cage ;
|
||||
HN launch https://news.ycombinator.com/item?id=46706796
|
||||
- **Isolation/locality**: Local sandbox for running multiple coding agents in
|
||||
YOLO mode. The launch discussion describes a VM boundary.
|
||||
- **Agent integration**: Built around the native Claude Code experience and
|
||||
motivated by running many agents in parallel without permission-prompt
|
||||
fatigue.
|
||||
- **Network/Git/credentials**: Strict egress filtering, configurable HTTP
|
||||
middleware, and mediated `git`/`gh` dispatch are the main value. The launch
|
||||
discussion explicitly identifies provider credential handling as unfinished
|
||||
and difficult because Claude state spans multiple host paths.
|
||||
- **Competitive read**: The closest new threat-model competitor. It shares
|
||||
bot-bottle's premise that filesystem isolation alone is insufficient and
|
||||
that Git plus authorized HTTP channels need mediation. bot-bottle currently
|
||||
leads on cross-provider support, proxy-held Claude/Codex/forge credentials,
|
||||
typed per-role manifests, content DLP, and supervision. yolo-cage's simpler
|
||||
pitch and narrower Claude-first setup may be easier to explain.
|
||||
- **Maturity**: Early local tool; HN launch received 60 points and 76 comments.
|
||||
|
||||
### Sandbox Agent SDK
|
||||
|
||||
- **Source**: https://github.com/rivet-dev/sandbox-agent ;
|
||||
HN launch https://news.ycombinator.com/item?id=46795584
|
||||
- **Isolation/locality**: Does not provide the isolation primitive. It runs
|
||||
inside E2B, Daytona, Modal, Cloudflare Containers, Agent Computer, BoxLite,
|
||||
Docker, or another sandbox provider. Embedded mode can also run locally
|
||||
without a sandbox.
|
||||
- **Agent integration**: Provider-neutral Rust server/SDK exposing a common
|
||||
HTTP/SSE/OpenAPI interface across Claude Code, Codex, OpenCode, Cursor, Amp,
|
||||
and Pi, plus a universal event/session schema for external storage and
|
||||
replay. It also exposes filesystem, managed-process, terminal, MCP, skills,
|
||||
custom-tool, and computer-use APIs. TypeScript is the primary SDK surface.
|
||||
- **Network/credentials**: Delegated to the chosen sandbox provider.
|
||||
- **Credential posture**: Its documented convenience command extracts real
|
||||
OpenAI/Anthropic credentials from local agent configuration and passes them
|
||||
as environment variables into the sandbox. That is materially weaker than
|
||||
bot-bottle's host-side credential custody, but it is an integration choice,
|
||||
not a structural limitation: a sandbox provider could put a credential
|
||||
proxy underneath the same SDK.
|
||||
- **Competitive read**: A serious architectural threat despite not supplying
|
||||
isolation. Sandbox Agent is trying to standardize the boundary *above* the
|
||||
sandbox: one client protocol, session model, and UI/control surface across
|
||||
every coding agent and runtime. If that boundary becomes the ecosystem
|
||||
standard, users and application builders may choose a sandbox provider plus
|
||||
Sandbox Agent rather than a vertically integrated launcher. bot-bottle's
|
||||
manifests would then be valuable chiefly as a local policy/backend
|
||||
implementation unless they expose an equally usable control contract.
|
||||
- **Maturity**: Apache 2.0, ~1.5k stars and 426 commits at the 2026-07-27
|
||||
check; HN launch received 41 points.
|
||||
|
||||
#### Why the Sandbox Agent architecture is strategically different
|
||||
|
||||
The manifest and the universal control protocol solve different layers:
|
||||
|
||||
- A bot-bottle manifest is a **trusted launch-time policy composition**. It
|
||||
selects the agent role, isolation backend, image, skills, egress routes,
|
||||
credentials, Git mediation, and supervision policy. Crucially, identity and
|
||||
secret references live on the host side of the trust boundary.
|
||||
- Sandbox Agent is a **runtime control and observation protocol**. A remote
|
||||
client creates sessions, sends messages, handles permissions, configures
|
||||
skills/MCP, manipulates files/processes/desktops, and streams normalized
|
||||
events. It deliberately delegates sandbox lifecycle, Git management,
|
||||
storage, network policy, and credential security to other products.
|
||||
|
||||
That makes it complementary in a component diagram but competitive in product
|
||||
architecture. The layer that becomes the stable integration point tends to own
|
||||
the ecosystem. Three plausible threat paths matter:
|
||||
|
||||
1. **Standard control plane, interchangeable runtimes.** Applications integrate
|
||||
once with Sandbox Agent and treat E2B, Daytona, BoxLite, Docker, or a future
|
||||
local microVM as replaceable compute. A provider that bundles adequate
|
||||
egress and credential custody makes bot-bottle's end-to-end launcher less
|
||||
necessary.
|
||||
2. **Policy grows upward.** Sandbox Agent already configures permissions,
|
||||
skills, MCP, custom tools, filesystem/process access, and computer use. If
|
||||
it adds a declarative, host-verifiable policy document, the overlap with
|
||||
agent/bottle manifests becomes substantial even if enforcement remains
|
||||
delegated.
|
||||
3. **UI and session ownership.** Its universal transcript schema, Inspector,
|
||||
React components, event replay, and remote terminal/computer APIs can become
|
||||
the natural basis for desktop, web, and mobile agent managers. bot-bottle's
|
||||
security layer could remain stronger while losing the operator surface and
|
||||
distribution channel.
|
||||
|
||||
The counter-position is not to claim that manifests and an API are mutually
|
||||
exclusive. The defensible split is:
|
||||
|
||||
- bot-bottle owns the trusted policy and enforcement plane outside the agent;
|
||||
- a provider-neutral protocol owns agent process control and normalized
|
||||
events; and
|
||||
- the operator UI consumes both.
|
||||
|
||||
This suggests an explicit compatibility decision rather than parallel,
|
||||
accidental protocol design: evaluate running Sandbox Agent inside a bottle and
|
||||
exposing it only through the authenticated bot-bottle control plane. If its
|
||||
schema is suitable, adopting it could turn a threat into an integration while
|
||||
keeping manifests as the higher-trust policy source. If it is unsuitable,
|
||||
bot-bottle should still publish a stable provider-neutral session/event API so
|
||||
frontends do not depend on Claude/Codex/Pi-specific process behavior.
|
||||
|
||||
## Comparison table
|
||||
|
||||
*Isolation/sandbox tools only. AGT and OAP are governance layers — see their per-project notes above.*
|
||||
@@ -616,6 +817,70 @@ would be a *backend* bot-bottle could call, not a competitor to its
|
||||
manifest layer. endo-familiar is in a different paradigm entirely:
|
||||
capability passing rather than kernel boundaries.
|
||||
|
||||
**Recent entrants change two parts of this read.** yolo-cage is closer to the
|
||||
actual threat model than agent-safehouse or litterbox: it combines a VM-style
|
||||
boundary with mediated Git and filtered HTTP specifically for parallel coding
|
||||
agents. Sandbox Agent SDK is the more important strategic entrant even though
|
||||
it supplies no isolation. It can become the standard agent-control layer above
|
||||
all of these runtimes, including a future bot-bottle backend. CloudRouter is
|
||||
the clearest workflow challenge because its browser/desktop/GPU loop makes
|
||||
parallel agents visibly more capable, not merely safer.
|
||||
|
||||
## Gap evaluation after the 2026-07-27 entrant scan
|
||||
|
||||
### Material gaps
|
||||
|
||||
1. **A stable provider-neutral control and event protocol.** This is the
|
||||
largest newly visible gap. bot-bottle normalizes launch/provisioning across
|
||||
providers, but an external UI or orchestrator still lacks one documented
|
||||
contract for creating a Claude/Codex/Pi session, sending input, handling
|
||||
permission/supervision events, streaming normalized output, reconnecting,
|
||||
and replaying history. Sandbox Agent SDK addresses exactly this layer and
|
||||
is already portable across many sandbox providers.
|
||||
2. **Browser/preview closure.** CloudRouter and Eve make a browser or desktop
|
||||
part of the standard agent environment and expose screenshots/live viewing
|
||||
to the operator. bot-bottle can run dev servers and supports nested
|
||||
containers, but it does not present a first-class browser/computer-use
|
||||
primitive or an auth-protected preview surface. For coding agents expected
|
||||
to verify UI work, this is a real product gap.
|
||||
3. **Unified parallel-session operator UX.** Named persistent bottles and
|
||||
supervision provide the substrate, but the recent products make task
|
||||
switching, live progress, notifications, terminal attach, diffs, and
|
||||
session history the product. Security depth will not compensate for a
|
||||
visibly rougher daily loop.
|
||||
4. **Normalized transcript persistence and replay.** bot-bottle preserves
|
||||
provider-specific state for resume; it does not expose a provider-neutral
|
||||
event record suitable for audit, replay, analytics, or a web/mobile client.
|
||||
This is both a UX gap and an audit gap.
|
||||
|
||||
### Important, but not necessarily bot-bottle features
|
||||
|
||||
- **Cloud VM/GPU provisioning.** Valuable for elastic workloads and could be a
|
||||
backend, but it conflicts with the local-custody default and should not
|
||||
displace core policy work.
|
||||
- **Automatic model routing.** Black LLAB and Eve sell task-to-model routing.
|
||||
bot-bottle's provider-template boundary can host that choice without making
|
||||
it part of the trusted sandbox policy.
|
||||
- **A thousand SaaS connectors.** This broadens capability and blast radius.
|
||||
The bot-bottle-native answer should remain explicit, scoped forge/egress
|
||||
associations rather than connector count as a goal.
|
||||
- **SDK-driven sandbox lifecycle as the primary configuration model.** Useful
|
||||
for platform builders, but not a replacement for reviewable, host-owned
|
||||
manifests. A control API and a declarative policy source are compatible;
|
||||
neither should silently become the other.
|
||||
|
||||
### Areas where bot-bottle remains ahead
|
||||
|
||||
- real provider and forge credentials remain outside the agent process rather
|
||||
than being extracted into its environment;
|
||||
- authorized HTTP payloads are scanned, not merely destination-filtered;
|
||||
- Git writes traverse a distinct gate with secret scanning and host-held
|
||||
upstream credentials;
|
||||
- role policy is host-owned, composable, and separate from untrusted repo
|
||||
content; and
|
||||
- local Firecracker/Apple Container execution preserves operator custody
|
||||
without requiring a hosted sandbox platform.
|
||||
|
||||
## Borrowable ideas
|
||||
|
||||
### Already shipped or otherwise addressed
|
||||
@@ -642,6 +907,19 @@ capability passing rather than kernel boundaries.
|
||||
|
||||
### Still worth considering
|
||||
|
||||
- **Sandbox Agent compatibility or an equivalent stable protocol (highest
|
||||
priority):** spike running its server inside a bottle behind bot-bottle's
|
||||
authenticated control plane. Compare its session/event schema, permission
|
||||
model, restore semantics, and provider coverage with current provider
|
||||
adapters. Adopt compatibility if it preserves the host-owned trust boundary;
|
||||
otherwise specify bot-bottle's own stable API before building another UI.
|
||||
- **First-class browser/preview loop** (from CloudRouter and Eve): give a
|
||||
bottle an optional browser/computer-use capability plus an operator-visible,
|
||||
authenticated preview/screenshot surface. Treat its network access as part
|
||||
of the bottle policy, not an implicit bypass.
|
||||
- **Provider-neutral transcript/event persistence** (from Sandbox Agent SDK):
|
||||
retain enough normalized structure for replay and audit while preserving the
|
||||
provider-native state needed for exact resume.
|
||||
- **Live network activity in the supervisor TUI** (from Docker sbx): show
|
||||
allowed and blocked connections and let the operator propose policy changes
|
||||
from the existing supervision surface.
|
||||
@@ -652,10 +930,11 @@ capability passing rather than kernel boundaries.
|
||||
closer review. This needs a carefully specified trust model before it can be
|
||||
more than a heuristic.
|
||||
|
||||
Not worth borrowing: the SDK-first programmatic API style of boxlite /
|
||||
microsandbox (cuts against the declarative-manifest stance), and the
|
||||
hosted-SaaS dashboard model of tilde.run (cuts against the
|
||||
"infrastructure I control" goal).
|
||||
Not worth borrowing: SDK-first *policy configuration* as used by boxlite /
|
||||
microsandbox (cuts against the reviewable declarative-manifest stance), and
|
||||
the hosted-SaaS custody model of tilde.run (cuts against the "infrastructure I
|
||||
control" goal). A provider-neutral runtime-control API is a separate concern
|
||||
and is worth borrowing.
|
||||
|
||||
## Publishing and positioning verdict
|
||||
|
||||
@@ -679,9 +958,15 @@ bot-bottle remains unusual in combining:
|
||||
The practical wedge is “as easy as native yolo, with declarative role policy
|
||||
and self-hosted custody,” including scoped access to private LAN/Tailnet
|
||||
services that cloud-first runtimes cannot provide without additional network
|
||||
plumbing. The main competitive risks are a local wrapper such as claudebox or
|
||||
Docker sbx growing a role-manifest layer, and GUI products such as SuperHQ
|
||||
adding equivalent policy and audit depth.
|
||||
plumbing. The main competitive risks are now:
|
||||
|
||||
- a local wrapper such as yolo-cage, claudebox, or Docker sbx growing a
|
||||
role-manifest and credential-custody layer;
|
||||
- Sandbox Agent SDK becoming the standard control/session boundary and making
|
||||
the runtime beneath it interchangeable; and
|
||||
- GUI products such as SuperHQ or CloudRouter adding equivalent policy and
|
||||
audit depth before bot-bottle closes the browser/preview and
|
||||
parallel-session UX gaps.
|
||||
|
||||
## Caveats
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ Can bot-bottle grow a built-in supervisor — TUI inventory plus PR-feedback rou
|
||||
|
||||
## Context
|
||||
|
||||
bot-bottle today is a fleet *executor*: `./cli.py start <agent>` brings up one bottle (agent container + pipelock + optional git-gate + optional cred-proxy on a per-bottle internal network), and `cli.py` tears it down when the session ends. There is no inventory view, no idle-detection, no automated reaction to PR or CI events. In parallel use, a human is the supervisor — opening one terminal per bottle, switching between them, and watching upstream PR state by hand.
|
||||
bot-bottle today is a fleet *executor*: `bot-bottle start <agent>` brings up one bottle (agent container + pipelock + optional git-gate + optional cred-proxy on a per-bottle internal network), and `cli.py` tears it down when the session ends. There is no inventory view, no idle-detection, no automated reaction to PR or CI events. In parallel use, a human is the supervisor — opening one terminal per bottle, switching between them, and watching upstream PR state by hand.
|
||||
|
||||
A separate survey of the broader ecosystem ([agent control dashboards research, mid-2026](https://gitea.dideric.is/didericis/consilium-research/src/branch/main/developer-workflow/agent-control-dashboards-2026-05-24.md)) sorts dashboards into five tiers (session managers, parallel runners, Kanban boards, mission-control SPAs, observability backends). The earlier first-pass conclusion was that a full SPA tier conflicts with bot-bottle's isolation model. This doc reconsiders the smaller question: a TUI supervisor in the existing Python CLI.
|
||||
|
||||
@@ -25,13 +25,13 @@ A supervisor doesn't have to be heavy. A TUI built into the existing Python CLI,
|
||||
|
||||
Three layers, each independently useful, in order of ambition:
|
||||
|
||||
### 1. `./cli.py status` — read-only inventory
|
||||
### 1. `bot-bottle status` — read-only inventory
|
||||
|
||||
Reads `docker ps` filtered by a bottle label and tails each bottle's session log. Reports per bottle: name, agent, uptime, last-activity timestamp, token spend if available, associated PR/branch if recorded.
|
||||
|
||||
No new daemons. No new ports. No new credentials. ~100 lines.
|
||||
|
||||
### 2. `./cli.py watch` — TUI over the same data
|
||||
### 2. `bot-bottle watch` — TUI over the same data
|
||||
|
||||
Same data as `status`, rendered with auto-refresh and keyboard shortcuts that shell out to the existing `cli.py attach / stop / start` commands.
|
||||
|
||||
@@ -39,7 +39,7 @@ Library choice: prefer the stdlib `curses` module to stay stdlib-first; fall bac
|
||||
|
||||
This is the Claude Squad / tmux-agent-status pattern, applied to bottles instead of tmux sessions. The whole category exists *because* a TUI is the lightweight shape that doesn't require what the SPA tier requires.
|
||||
|
||||
### 3. `./cli.py supervise` — PR feedback router
|
||||
### 3. `bot-bottle supervise` — PR feedback router
|
||||
|
||||
The optional, more ambitious layer. The bottle manifest gains an optional field:
|
||||
|
||||
@@ -49,7 +49,7 @@ pr_watch:
|
||||
branch: agent/task-42
|
||||
```
|
||||
|
||||
`./cli.py supervise` polls the named upstream for new review comments and CI failures on `branch`. When one fires, it surfaces as a desktop notification or a flash in the TUI. The human decides what to do with the feedback — there is no autonomous loop that feeds the comment back into a bottle's next prompt (see "Where to be conservative" for why).
|
||||
`bot-bottle supervise` polls the named upstream for new review comments and CI failures on `branch`. When one fires, it surfaces as a desktop notification or a flash in the TUI. The human decides what to do with the feedback — there is no autonomous loop that feeds the comment back into a bottle's next prompt (see "Where to be conservative" for why).
|
||||
|
||||
The polling token is a **host** token (the same `GH_PAT` / Gitea token the host already keeps in shell env), not a bottle credential. The supervisor never holds bottle secrets.
|
||||
|
||||
@@ -62,7 +62,7 @@ The load-bearing question is whether the supervisor introduces the privileged-ch
|
||||
| Reaching into running bottles | Supervisor reads `docker ps` and host-side log files. The host already sees both — Docker is the trust boundary, the supervisor is on the host side of it. |
|
||||
| Holding bottle credentials | The polling token is a host token. The supervisor never receives `bottle.cred_proxy.routes` entries; it has no path to them. |
|
||||
| Bridging between bottles | The supervisor does not relay state from bottle A to bottle B. It relays *upstream PR state* to a bottle's next prompt — and only if the manifest opts in. |
|
||||
| New attack surface | All "control" actions go through `./cli.py start <agent>`, which already enforces the manifest. The supervisor is an automated caller of the existing CLI, not a parallel control plane. |
|
||||
| New attack surface | All "control" actions go through `bot-bottle start <agent>`, which already enforces the manifest. The supervisor is an automated caller of the existing CLI, not a parallel control plane. |
|
||||
|
||||
The boundary stays at the bottle wall. The supervisor looks outward at git/PR state and downward at Docker; it does not look *inward* through pipelock.
|
||||
|
||||
|
||||
@@ -13,11 +13,11 @@ What's the cheapest path to that, and where does it bottom out?
|
||||
|
||||
## What "interact" means
|
||||
|
||||
Today the flow is bimodal. `./cli.py start <agent>` brings the
|
||||
Today the flow is bimodal. `bot-bottle start <agent>` brings the
|
||||
bottle up and immediately drops you into an interactive
|
||||
`docker exec -it bot-bottle-<slug> claude ...` — claude-code
|
||||
owns the whole terminal until you Ctrl-D out, at which point the
|
||||
bottle tears down. The dashboard (`./cli.py dashboard`) is a
|
||||
bottle tears down. The dashboard (`bot-bottle dashboard`) is a
|
||||
*separate* invocation that watches across bottles but never
|
||||
exposes the claude TUI itself.
|
||||
|
||||
@@ -107,7 +107,7 @@ What's not good:
|
||||
|
||||
This is the v1 the project's existing code-shape strongly
|
||||
prefers. It clears the bar of "let me talk to claude-code
|
||||
without quitting `./cli.py dashboard`."
|
||||
without quitting `bot-bottle dashboard`."
|
||||
|
||||
## Option 2: Embedded emulator
|
||||
|
||||
|
||||
@@ -84,7 +84,7 @@ dangerous changes before they left*.
|
||||
(`supervise_gitleaks_allow`, [`git_gate_render.py`](../../bot_bottle/git_gate_render.py)).
|
||||
Extend the same flow to **high-risk file classes**: any commit touching
|
||||
CI/build/deploy scripts, auth/crypto code, egress config, or
|
||||
adding/changing dependencies → route to `./cli.py supervise`. This is
|
||||
adding/changing dependencies → route to `bot-bottle supervise`. This is
|
||||
attribution/policy, not detection, and it's the strongest thing here —
|
||||
a human on exactly the temporal-escape surfaces.
|
||||
4. **LLM semantic diff-review — the behavioral backstop.** The only
|
||||
@@ -146,7 +146,7 @@ preserves the bottom-up distribution funnel.
|
||||
**governed code-egress review**, not "we resell inference" (the
|
||||
monetization notes explicitly warn against reselling compute).
|
||||
- **The web-console supervise/review flow — the strongest anchor.** Turn
|
||||
the CLI `./cli.py supervise` approval into a real review surface:
|
||||
the CLI `bot-bottle supervise` approval into a real review surface:
|
||||
rendered diff + finding context, approve/reject, **who-approved audit
|
||||
trail, RBAC on approvers, mobile/phone-control** (ties to the
|
||||
dashboard/vault north star). This is "central enforcement +
|
||||
|
||||
@@ -100,7 +100,7 @@ resolver globs each directory.
|
||||
becomes file ops (mkdir, mv, rm) instead of editing one file.
|
||||
Power users prefer that; new users may not.
|
||||
- Discovery requires `ls`, not "grep one file." Tooling helps
|
||||
(e.g. `./cli.py list`) but the manifest is no longer a single
|
||||
(e.g. `bot-bottle list`) but the manifest is no longer a single
|
||||
artifact to email or ship.
|
||||
- Atomicity: swapping a bottle name across agents touches
|
||||
multiple files. Git handles this fine; a one-shot text editor
|
||||
@@ -364,7 +364,7 @@ wins; none of the body-prose or dependency story.
|
||||
warns / ignores / breaks. If it warns, we'd want a different
|
||||
field name (e.g. `bot-bottle-bottle`) or a namespaced block.
|
||||
- **Migration story.** Is the project willing to ship a one-shot
|
||||
`./cli.py migrate-manifest` command that does the JSON → MD
|
||||
`bot-bottle migrate-manifest` command that does the JSON → MD
|
||||
conversion? Or do users just rewrite by hand from the new docs?
|
||||
- **Bottle file body content.** If most bottle .md files have an
|
||||
empty body, is the MD-with-frontmatter format still warranted?
|
||||
|
||||
@@ -34,7 +34,7 @@ on top of working onboarding.
|
||||
|
||||
A first-time user today goes through five steps: install Docker,
|
||||
install `uv`, set `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`, write
|
||||
`bot-bottle.json`, run `./cli.py start`. One of those is
|
||||
`bot-bottle.json`, run `bot-bottle start`. One of those is
|
||||
"author a JSON manifest." Polished tools in this category let
|
||||
users skip that step on day one. The fix is an `init` subcommand
|
||||
that drops a working `bot-bottle.json` with a default `coder`
|
||||
|
||||
@@ -131,7 +131,7 @@ The minimum-viable workflow, no bot-bottle code changes:
|
||||
3. SSH in.
|
||||
4. `git clone` bot-bottle on the VM, drop a manifest in place,
|
||||
inject `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` via the provider's secrets path.
|
||||
5. `./cli.py start <agent>` — the existing launcher handles the rest.
|
||||
5. `bot-bottle start <agent>` — the existing launcher handles the rest.
|
||||
6. On exit: destroy the VM. No host artifacts persist.
|
||||
|
||||
For the "VPN pivot" failure mode, see
|
||||
|
||||
@@ -0,0 +1,536 @@
|
||||
# Sandbox Agent SDK and bot-bottle: protocol versus product
|
||||
|
||||
This note asks whether [Sandbox Agent SDK](https://github.com/rivet-dev/sandbox-agent)
|
||||
and bot-bottle compete for the same architectural layer, whether bot-bottle
|
||||
can productize the turnkey ecosystem/DX layer above it, and how far the
|
||||
Docker/OCI analogy actually holds.
|
||||
|
||||
Research conducted 2026-07-27. Sandbox Agent SDK was at the `0.4.x` line,
|
||||
Apache 2.0, and documented support for Claude Code, Codex, OpenCode, Cursor,
|
||||
Amp, and Pi at the time of review.
|
||||
|
||||
## Summary
|
||||
|
||||
**The projects are complementary at the component boundary and competitive at
|
||||
the product boundary.** Sandbox Agent SDK normalizes how software controls a
|
||||
coding-agent process inside an arbitrary sandbox. bot-bottle decides what
|
||||
sandbox to create, what trusted role and policy it receives, how credentials
|
||||
and Git access cross the boundary, how traffic is constrained, and how an
|
||||
operator launches and supervises the result.
|
||||
|
||||
The Docker analogy is useful with one correction:
|
||||
|
||||
- Sandbox Agent SDK is not equivalent to Linux container APIs or OCI itself.
|
||||
It is closer to a **containerd shim plus a portable exec/session API for
|
||||
coding agents**. It adapts incompatible agent processes to one HTTP/SSE
|
||||
contract.
|
||||
- A future independent agent-session specification would be the closer OCI
|
||||
analogue.
|
||||
- bot-bottle can credibly occupy the **Docker Engine / Compose / Desktop**
|
||||
layer: packaging, policy composition, lifecycle, networking, credentials,
|
||||
storage, operator UX, and a one-command experience above interchangeable
|
||||
agent adapters and isolation runtimes.
|
||||
|
||||
That is a viable position, but “turnkey wrapper” undersells it. A thin wrapper
|
||||
is replaceable. The valuable product is a **turnkey, policy-first coding-agent
|
||||
runtime** whose manifest compiles trusted operator intent into multiple
|
||||
enforcement planes. Sandbox Agent SDK may be one internal process-control
|
||||
component of that product.
|
||||
|
||||
The recommended direction is:
|
||||
|
||||
1. Keep the bot-bottle manifest as the host-owned source of trusted policy.
|
||||
2. Spike Sandbox Agent SDK as the in-bottle provider/session adapter.
|
||||
3. Expose a stable, provider-neutral bot-bottle control API, compatible with
|
||||
Sandbox Agent where practical.
|
||||
4. Keep security decisions and authoritative audit outside the sandbox.
|
||||
5. Build the ecosystem around policy packs, agent images, skills, backends,
|
||||
operator UI, and trusted integrations—not around a proprietary transcript
|
||||
protocol.
|
||||
|
||||
## What each project is today
|
||||
|
||||
### Sandbox Agent SDK
|
||||
|
||||
Sandbox Agent is a Rust server that runs alongside the coding agent. A client
|
||||
connects over HTTP, streams events over SSE, and uses one API across agent
|
||||
implementations. Its documented surface includes:
|
||||
|
||||
- creating and restoring agent sessions;
|
||||
- sending messages and streaming normalized events;
|
||||
- handling permissions;
|
||||
- configuring MCP servers, skills, and custom tools;
|
||||
- filesystem and managed-process APIs;
|
||||
- interactive terminal access;
|
||||
- computer-use/desktop operations;
|
||||
- a universal session/transcript schema;
|
||||
- an Inspector UI, React components, CLI, TypeScript SDK, and OpenAPI spec.
|
||||
|
||||
It can run in embedded mode or inside E2B, Daytona, Modal, Cloudflare
|
||||
Containers, Agent Computer, BoxLite, Docker, and other environments. It
|
||||
explicitly leaves these concerns to the caller or sandbox provider:
|
||||
|
||||
- sandbox creation and lifecycle;
|
||||
- Git repository management;
|
||||
- durable session storage;
|
||||
- network policy;
|
||||
- isolation strength; and
|
||||
- secure credential delivery.
|
||||
|
||||
Its documented credential convenience path extracts real provider credentials
|
||||
from local agent configuration and passes them into the sandbox environment.
|
||||
That is convenient but is not an acceptable security boundary for bot-bottle.
|
||||
|
||||
Sources:
|
||||
|
||||
- [Sandbox Agent repository and architecture](https://github.com/rivet-dev/sandbox-agent)
|
||||
- [Sandbox Agent documentation](https://sandboxagent.dev/docs)
|
||||
- [HTTP API](https://sandboxagent.dev/docs/api-reference)
|
||||
- [Universal session/transcript schema](https://sandboxagent.dev/docs/session-transcript-schema)
|
||||
|
||||
### bot-bottle
|
||||
|
||||
bot-bottle is a host-side launch, policy, and enforcement system for existing
|
||||
coding-agent CLIs. Its current architecture includes:
|
||||
|
||||
- agent and bottle manifests with composition via `extends:`;
|
||||
- a host-only trust boundary for roles, identity, and secret references;
|
||||
- provider templates and plugins for Claude Code, Codex, Pi, and custom
|
||||
providers;
|
||||
- Firecracker on KVM Linux and Apple Container on macOS, with Docker fallback;
|
||||
- image construction and provider-specific provisioning;
|
||||
- default-deny inspected egress with path/method/header policy;
|
||||
- payload DLP on authorized channels;
|
||||
- real credentials held outside the agent and injected by the gateway;
|
||||
- Git mediation, upstream credential custody, and gitleaks scanning;
|
||||
- a per-host authenticated orchestrator and shared gateway;
|
||||
- named bottle lifecycle, resume, supervision, and audit state; and
|
||||
- a CLI/TUI intended to make full-permission agents operationally tolerable.
|
||||
|
||||
The provider layer currently normalizes launch-time concerns—command, image,
|
||||
prompt delivery, files, skills, environment, verification, and provider-owned
|
||||
egress routes. It does **not** yet expose a stable provider-neutral runtime
|
||||
contract for sessions, messages, transcripts, terminals, or normalized events.
|
||||
That is the gap Sandbox Agent directly illuminates.
|
||||
|
||||
Sources in this repository:
|
||||
|
||||
- [`README.md`](../../README.md)
|
||||
- [`0070-per-host-orchestrator.md`](../prds/0070-per-host-orchestrator.md)
|
||||
- [`0026-agent-provider-templates.md`](../prds/0026-agent-provider-templates.md)
|
||||
- [`0053-user-provider-plugins.md`](../prds/0053-user-provider-plugins.md)
|
||||
- [`agent_provider.py`](../../bot_bottle/agent_provider.py)
|
||||
|
||||
## The layer model
|
||||
|
||||
The cleanest architecture has four layers:
|
||||
|
||||
| Layer | Responsibility | Likely owner |
|
||||
|---|---|---|
|
||||
| Operator product | Install, select a role, launch, observe, intervene, resume, review changes | bot-bottle |
|
||||
| Trusted policy and lifecycle | Compose manifest, choose backend/image, hold credentials, enforce egress/Git, persist authoritative audit | bot-bottle |
|
||||
| Agent control protocol | Start provider process, create session, send input, stream normalized events, terminal/computer operations | Sandbox Agent or a compatible protocol |
|
||||
| Isolation primitive | VM/container/process boundary, filesystem, CPU/memory, networking substrate | Firecracker, Apple Container, Docker, E2B, Daytona, BoxLite, etc. |
|
||||
|
||||
The important boundary is between trusted policy/lifecycle and agent control.
|
||||
The agent-control daemon runs in the environment being treated as untrusted.
|
||||
It can report what the agent says happened, but it cannot authoritatively prove
|
||||
that policy was enforced. Egress decisions, credential custody, Git scanning,
|
||||
bottle identity, and security audit must remain outside it.
|
||||
|
||||
### Proposed composition
|
||||
|
||||
```text
|
||||
operator UI / CLI / API
|
||||
|
|
||||
v
|
||||
bot-bottle orchestrator (trusted)
|
||||
- resolves manifest
|
||||
- owns bottle identity and lifecycle
|
||||
- stores authoritative audit
|
||||
- authenticates clients
|
||||
|
|
||||
+--------------------------+
|
||||
| |
|
||||
v v
|
||||
isolation backend shared gateway (trusted)
|
||||
Firecracker / Apple / Docker - egress policy + DLP
|
||||
| - credential injection
|
||||
| - Git mediation
|
||||
v
|
||||
bottle / guest (untrusted)
|
||||
- Sandbox Agent server
|
||||
- Claude Code / Codex / Pi subprocess
|
||||
- workspace, skills, MCP configuration
|
||||
```
|
||||
|
||||
The bot-bottle manifest would compile into both sides:
|
||||
|
||||
- **outside the bottle:** backend, network, egress, credentials, Git,
|
||||
supervision, identity, and authoritative lifecycle;
|
||||
- **inside the bottle:** selected provider, prompt, skills, MCP configuration,
|
||||
startup arguments, and non-secret session metadata.
|
||||
|
||||
Sandbox Agent should never receive real secrets merely because its API offers
|
||||
a credential extraction helper. Provider and forge requests should continue
|
||||
to use bot-bottle's placeholder/proxy pattern.
|
||||
|
||||
## How accurate is the Docker/OCI analogy?
|
||||
|
||||
### The useful part
|
||||
|
||||
The container ecosystem separates low-level execution from a product that
|
||||
ordinary developers operate. OCI defines interoperable image, runtime, and
|
||||
distribution specifications. Docker Engine adds a daemon, API, CLI, object
|
||||
model, images, networks, volumes, and lifecycle; Docker Desktop and related
|
||||
products add installation, updates, UI, integrations, policy, and team
|
||||
workflows.
|
||||
|
||||
The same separation can exist for coding agents:
|
||||
|
||||
| Container ecosystem | Agent-sandbox ecosystem |
|
||||
|---|---|
|
||||
| OCI/runtime contract | A future open agent session/event contract |
|
||||
| `runc` / runtime adapter | Claude/Codex/Pi adapter |
|
||||
| containerd shim and task/exec API | Sandbox Agent server and HTTP/SSE session API |
|
||||
| containerd / CRI-style lifecycle | Sandbox-provider lifecycle APIs |
|
||||
| Docker Engine / Compose | bot-bottle orchestrator + manifests + backends + gateway |
|
||||
| Docker Desktop / Hub ecosystem | bot-bottle desktop/mobile UX, policy packs, agent images, skills, trusted integrations |
|
||||
|
||||
Sandbox Agent makes coding-agent processes portable in roughly the way a shim
|
||||
makes runtimes consumable through a common lifecycle interface. bot-bottle can
|
||||
make the entire safe-agent system usable without asking the operator to
|
||||
assemble that plumbing.
|
||||
|
||||
Official container references:
|
||||
|
||||
- [Open Container Initiative](https://opencontainers.org/)
|
||||
- [OCI Runtime Specification](https://github.com/opencontainers/runtime-spec)
|
||||
- [Docker Engine architecture](https://docs.docker.com/engine/)
|
||||
- [Docker alternative runtimes and containerd shims](https://docs.docker.com/engine/daemon/alternative-runtimes/)
|
||||
|
||||
### Where the analogy breaks
|
||||
|
||||
1. **Sandbox Agent is an implementation, not an independent standard.**
|
||||
Its OpenAPI document is public, but the project currently owns the server,
|
||||
adapters, schema, and evolution. OCI is an independently governed set of
|
||||
specifications with multiple implementations.
|
||||
2. **It sits above, not below, the isolation boundary.** Linux namespaces,
|
||||
cgroups, VMs, and OCI runtimes create the boundary. Sandbox Agent controls a
|
||||
process after some other system has created that boundary.
|
||||
3. **It reaches into product territory.** Inspector, React components,
|
||||
computer-use APIs, skills/MCP configuration, transcripts, and restoration
|
||||
are not merely low-level primitives. Sandbox Agent can continue growing
|
||||
upward into the same UI and orchestration space bot-bottle might occupy.
|
||||
4. **Coding agents are semantically uneven.** Normalizing a container
|
||||
lifecycle is easier than claiming full behavioral parity across Claude
|
||||
Code, Codex, Cursor, Amp, OpenCode, and Pi. A universal schema can become a
|
||||
lowest common denominator or accumulate provider-specific escape hatches.
|
||||
5. **The security contract is not standardized.** An agent-session API says
|
||||
little about whether credentials are visible, egress is controlled, Git is
|
||||
mediated, or audit is trustworthy. Those are core bot-bottle concerns.
|
||||
|
||||
The positioning should therefore say “Docker-like product layer above an open
|
||||
agent-control protocol,” not “Sandbox Agent is OCI” or “bot-bottle implements
|
||||
OCI for agents.”
|
||||
|
||||
## Can bot-bottle be the turnkey product layer?
|
||||
|
||||
Yes, if it owns substantially more than launch syntax.
|
||||
|
||||
The turnkey promise is:
|
||||
|
||||
> Choose a trusted role, point it at a project, and run any supported coding
|
||||
> agent with full permissions. bot-bottle builds the environment, isolates it,
|
||||
> supplies only the capabilities it needs, keeps credentials outside, mediates
|
||||
> external writes, and gives the operator one place to watch and intervene.
|
||||
|
||||
That product has several defensible jobs:
|
||||
|
||||
### 1. Packaging and reproducibility
|
||||
|
||||
- provider and toolchain images;
|
||||
- pinned, verified build inputs;
|
||||
- skills and MCP configuration;
|
||||
- role/bottle composition;
|
||||
- cached startup and portable environment definitions; and
|
||||
- compatibility testing across agents and backends.
|
||||
|
||||
### 2. Trusted policy compilation
|
||||
|
||||
The manifest is valuable because one reviewable document compiles into:
|
||||
|
||||
- an isolation plan;
|
||||
- gateway routes and DLP policy;
|
||||
- credential slots;
|
||||
- Git-gate repositories and identities;
|
||||
- provider configuration;
|
||||
- supervision behavior; and
|
||||
- operator-facing preflight.
|
||||
|
||||
Sandbox Agent's runtime configuration does not replace this. The policy must
|
||||
be resolved before an untrusted guest or agent-control daemon exists.
|
||||
|
||||
### 3. Security enforcement
|
||||
|
||||
- dedicated-kernel isolation where available;
|
||||
- no direct guest route to the internet;
|
||||
- credentials injected outside the agent;
|
||||
- content inspection on allowed destinations;
|
||||
- Git secrets scanning and upstream-key custody;
|
||||
- fail-closed policy resolution; and
|
||||
- authoritative host-side audit.
|
||||
|
||||
This is the strongest current differentiation from a generic
|
||||
“Sandbox Agent + Docker/E2B” assembly.
|
||||
|
||||
### 4. Lifecycle and operations
|
||||
|
||||
- install and host preflight;
|
||||
- image build/update;
|
||||
- start, stop, resume, cleanup, and migration;
|
||||
- concurrent named agents;
|
||||
- state recovery after crashes;
|
||||
- live supervision and policy remediation; and
|
||||
- backend selection without changing the role definition.
|
||||
|
||||
### 5. Ecosystem and DX
|
||||
|
||||
A product layer can support:
|
||||
|
||||
- curated provider images;
|
||||
- signed policy/bottle packs;
|
||||
- reusable role templates;
|
||||
- skills and MCP bundles;
|
||||
- backend plugins;
|
||||
- an authenticated desktop/web/mobile operator client;
|
||||
- browser/preview integration;
|
||||
- normalized transcripts and change review; and
|
||||
- team policy distribution and compliance exports.
|
||||
|
||||
The analogy to Docker is strongest here: users adopt the coherent workflow and
|
||||
ecosystem, not because the low-level process API is proprietary.
|
||||
|
||||
## Business and product positioning
|
||||
|
||||
“Turnkey wrapper” is understandable internally but weak externally. It implies
|
||||
that the hard work lives underneath and that another wrapper can replace it.
|
||||
Prefer one of:
|
||||
|
||||
- **The policy-first runtime for coding agents**
|
||||
- **Run any coding agent with full permissions, without giving it your host or
|
||||
credentials**
|
||||
- **A turnkey local control plane for isolated coding agents**
|
||||
- **Docker-like packaging and operations for coding agents, with the security
|
||||
boundary outside the agent**
|
||||
|
||||
The open/product split could resemble the container ecosystem:
|
||||
|
||||
### Open foundation
|
||||
|
||||
- manifest schema and composition;
|
||||
- local CLI and core orchestrator;
|
||||
- provider adapters;
|
||||
- Firecracker/Apple Container/Docker backends;
|
||||
- gateway policy format and enforcement;
|
||||
- Sandbox Agent compatibility;
|
||||
- local audit and supervision; and
|
||||
- conformance tests for providers/backends/policy.
|
||||
|
||||
### Productizable ecosystem/DX
|
||||
|
||||
- polished desktop and mobile clients;
|
||||
- fleet/remote-host management;
|
||||
- signed and curated role/image/policy registry;
|
||||
- team policy distribution and administrative controls;
|
||||
- durable searchable transcripts and audit exports;
|
||||
- SSO, RBAC, retention, and tamper-evident audit;
|
||||
- managed update/compatibility channels;
|
||||
- remote browser/preview relay;
|
||||
- enterprise support; and
|
||||
- optional managed build/cache infrastructure.
|
||||
|
||||
OCI itself is not the thing Docker sells. Interoperability expands the market;
|
||||
the product captures value through reliable packaging, workflow, distribution,
|
||||
management, and trust. bot-bottle should follow that logic rather than trying
|
||||
to make its session protocol the moat.
|
||||
|
||||
## Strategic threat from Sandbox Agent
|
||||
|
||||
Sandbox Agent is a real threat for three reasons:
|
||||
|
||||
1. **It can become the integration default.** A frontend or agent platform can
|
||||
integrate one API and choose among many agents and sandbox vendors.
|
||||
2. **It can own session data and UI.** The universal event schema, Inspector,
|
||||
React components, restoration, terminal, and computer-use APIs give it a
|
||||
natural path toward the operator surface.
|
||||
3. **Sandbox providers can move upward.** If E2B, Daytona, BoxLite, or another
|
||||
runtime combines Sandbox Agent with adequate network policy and credential
|
||||
custody, it can offer much of the turnkey stack.
|
||||
|
||||
The threat is not that its manifest syntax is better. It currently has no
|
||||
equivalent trusted policy composition. The threat is that **the ecosystem may
|
||||
standardize around its API before bot-bottle has a stable external control
|
||||
surface**. In that world bot-bottle is evaluated as one sandbox provider,
|
||||
while the SDK and its consumers own the user relationship.
|
||||
|
||||
## Why bot-bottle can still win its layer
|
||||
|
||||
Sandbox Agent's scope exclusions align with bot-bottle's deepest work:
|
||||
|
||||
- it does not choose or operate the sandbox provider;
|
||||
- it does not mediate Git;
|
||||
- it does not own network policy;
|
||||
- it does not securely deliver credentials;
|
||||
- it does not durably store sessions; and
|
||||
- it cannot make guest-generated telemetry authoritative.
|
||||
|
||||
Those are not incidental features. Together they define the trusted system
|
||||
around an untrusted coding agent. bot-bottle also has a narrower and coherent
|
||||
initial customer: a developer or small operator who wants existing agent CLIs
|
||||
to run locally with broad permissions and bounded consequences.
|
||||
|
||||
The durable advantage is therefore:
|
||||
|
||||
> Sandbox Agent makes agents controllable. bot-bottle makes them safe and
|
||||
> operable.
|
||||
|
||||
That sentence remains true only if bot-bottle closes its operator-DX gaps.
|
||||
Security without a browser/preview loop, stable API, normalized session view,
|
||||
and good parallel-task UX risks becoming an invisible backend feature.
|
||||
|
||||
## Integration options
|
||||
|
||||
### Option A — Embed Sandbox Agent inside each bottle
|
||||
|
||||
bot-bottle launches Sandbox Agent as the provider process supervisor and
|
||||
connects it to the host orchestrator through a bottle-scoped authenticated
|
||||
channel.
|
||||
|
||||
**Benefits**
|
||||
|
||||
- immediate provider-neutral session API;
|
||||
- more supported agents;
|
||||
- normalized streaming and transcripts;
|
||||
- terminal, filesystem, process, and computer-use primitives;
|
||||
- Inspector/React ecosystem; and
|
||||
- less provider-specific reverse engineering in bot-bottle.
|
||||
|
||||
**Risks**
|
||||
|
||||
- `0.x` API/schema churn;
|
||||
- extra binary and release-supply-chain dependency;
|
||||
- lowest-common-denominator normalization;
|
||||
- conflict with provider-native resume state;
|
||||
- an in-guest daemon is attacker-controlled after guest compromise;
|
||||
- duplicate orchestration responsibilities; and
|
||||
- upstream can move into policy/lifecycle and compete more directly.
|
||||
|
||||
**Security rule**
|
||||
|
||||
Treat every event and state claim from Sandbox Agent as untrusted telemetry.
|
||||
Never delegate egress authorization, credential release, bottle identity,
|
||||
authoritative audit, or Git policy to it.
|
||||
|
||||
### Option B — Implement a Sandbox Agent-compatible endpoint
|
||||
|
||||
bot-bottle maps the external protocol onto its existing provider adapters and
|
||||
process model without running the upstream server.
|
||||
|
||||
**Benefits**
|
||||
|
||||
- ecosystem compatibility with tighter component control;
|
||||
- no in-guest daemon dependency; and
|
||||
- room to preserve bot-bottle-native lifecycle semantics.
|
||||
|
||||
**Risks**
|
||||
|
||||
- large and continuing compatibility burden;
|
||||
- “full feature coverage” is expensive across all providers;
|
||||
- accidental protocol fork; and
|
||||
- effort diverted from policy and UX differentiation.
|
||||
|
||||
### Option C — Define an independent bot-bottle session API
|
||||
|
||||
Build only the control surface bot-bottle needs.
|
||||
|
||||
**Benefits**
|
||||
|
||||
- clean fit with the trust model and persistent named bottles;
|
||||
- no upstream dependency; and
|
||||
- deliberate support for supervision and security events.
|
||||
|
||||
**Risks**
|
||||
|
||||
- recreates a fast-growing open-source project;
|
||||
- no existing client ecosystem;
|
||||
- slower browser/desktop/mobile work; and
|
||||
- increases the chance that Sandbox Agent becomes the de facto standard first.
|
||||
|
||||
### Recommendation
|
||||
|
||||
Start with **Option A as a bounded compatibility spike**, not a product
|
||||
commitment. Do not begin with a clean-room competing protocol.
|
||||
|
||||
The spike should answer:
|
||||
|
||||
1. Can Claude Code, Codex, and Pi retain exact native resume behavior?
|
||||
2. Can Sandbox Agent run without receiving real provider credentials?
|
||||
3. Can its server be reached through a bottle-scoped authenticated channel
|
||||
without exposing the orchestrator or broadening guest egress?
|
||||
4. Which permission events overlap or conflict with bot-bottle supervision?
|
||||
5. Can normalized events be stored while clearly separating untrusted
|
||||
transcript telemetry from authoritative gateway/Git audit?
|
||||
6. Can manifest skills, MCP servers, prompt, and startup arguments compile
|
||||
deterministically into its configuration?
|
||||
7. Does its versioning policy permit a compatibility contract bot-bottle can
|
||||
support?
|
||||
8. What image-size, startup-time, and update burden does the binary add?
|
||||
|
||||
If the answers are favorable, adopt it behind a bot-bottle-owned interface and
|
||||
pin/test the supported version. If not, implement the smallest compatible
|
||||
subset needed by external clients before inventing a wholly separate API.
|
||||
|
||||
## Product roadmap implications
|
||||
|
||||
The competitor scan and this architecture comparison reorder the likely work:
|
||||
|
||||
1. **Provider-neutral control/session compatibility spike**
|
||||
2. **Stable authenticated external bot-bottle API**
|
||||
3. **Normalized transcript/event persistence**
|
||||
4. **Parallel-session operator UI**
|
||||
5. **Browser/preview/computer-use capability**
|
||||
6. **Policy/image/skill distribution and signing**
|
||||
7. **Remote host/fleet management**
|
||||
|
||||
This does not mean pausing security work. It means exposing the shipped
|
||||
security work through a product surface that can compete with the SDK-plus-
|
||||
sandbox ecosystem.
|
||||
|
||||
## Decision
|
||||
|
||||
Treat Sandbox Agent SDK as a potentially standard **agent process-control
|
||||
layer**, not as a sandbox replacement and not as a minor complementary
|
||||
library. Position bot-bottle one layer above it:
|
||||
|
||||
- manifests express trusted role and environment policy;
|
||||
- bot-bottle compiles and enforces that policy across host, gateway, Git, and
|
||||
isolation backends;
|
||||
- Sandbox Agent or a compatible protocol controls the selected agent process;
|
||||
and
|
||||
- bot-bottle owns the turnkey operator experience.
|
||||
|
||||
The Docker analogy is strategically sound when stated as:
|
||||
|
||||
> Sandbox Agent can be the portable task/exec protocol; bot-bottle can be the
|
||||
> opinionated engine, Compose-like policy layer, and Desktop-like operator
|
||||
> product.
|
||||
|
||||
It is not sound when stated as:
|
||||
|
||||
> Sandbox Agent is OCI and bot-bottle is Docker.
|
||||
|
||||
There is no independent OCI-equivalent agent specification yet, and Sandbox
|
||||
Agent already reaches into UI/session territory. Compatibility should be
|
||||
pursued quickly, while the trusted manifest/enforcement plane and operator
|
||||
experience remain the parts bot-bottle deliberately owns.
|
||||
@@ -0,0 +1,305 @@
|
||||
# Testing a clean bot-bottle install on macOS
|
||||
|
||||
How do you exercise `install.sh` (and, ideally, a first `bot-bottle start`)
|
||||
the way a brand-new user would — on a pristine macOS environment you can
|
||||
throw away afterward — *without* permanently polluting your daily-driver
|
||||
Mac? The user's framing: is there a VM or boundary that avoids creating a
|
||||
separate account, or is spinning up and tearing down a throwaway macOS
|
||||
user on the CLI easy enough to just do that?
|
||||
|
||||
## Summary
|
||||
|
||||
There is no lightweight, in-place macOS sandbox that hands you a clean home
|
||||
directory and wipeable system state without *either* a VM or a separate
|
||||
user account. `sandbox-exec` (Seatbelt) is deprecated and confines a
|
||||
process, not an environment; App Sandbox is for shipping apps, not for
|
||||
provisioning a fresh dev host. So the real choice is exactly the two the
|
||||
user named: **a disposable macOS VM** or **a throwaway user account** —
|
||||
and which one is right turns on a detail specific to *this* project.
|
||||
|
||||
bot-bottle's default macOS backend is Apple's `container`, which runs each
|
||||
container in its own lightweight VM via `Virtualization.framework`
|
||||
([`README.md:27`](../README.md), [`apple-container-backend.md`](apple-container-backend.md)).
|
||||
That means a full end-to-end test — install *and* `bot-bottle start` —
|
||||
needs virtualization to work wherever bot-bottle runs. Inside a macOS guest
|
||||
VM that requires **nested virtualization, which Apple gates to M3 or newer
|
||||
chips on macOS 15+**. On M1/M2 you cannot run the Apple Container backend
|
||||
(or Docker Desktop, same reason) inside a macOS VM at all.
|
||||
|
||||
The recommendation splits on what you're testing and what silicon you have:
|
||||
|
||||
- **Install-script correctness only** (does `curl | sh` → pipx → config dir
|
||||
→ `doctor`'s Python/config checks pass?): a **disposable Tart VM** is the
|
||||
cleanest boundary and works on any Apple Silicon Mac. `doctor` will report
|
||||
the backend as not-ready inside the VM on M1/M2, which is fine — you're
|
||||
testing the installer, not the runtime.
|
||||
- **Full runtime** (actually launch a bottle) on **M3/M4**: a **disposable
|
||||
Tart VM from a golden base image, cloned per run** is the gold standard —
|
||||
a genuine kernel/state boundary that wipes to nothing.
|
||||
- **Full runtime** on **M1/M2**, or when you'd rather not fight nested virt:
|
||||
a **throwaway admin user via `sysadminctl`** is the pragmatic pick. It
|
||||
tests the real backend because the backend runs on the host hypervisor —
|
||||
but it is a *hygiene* boundary, not a security one, and it does **not**
|
||||
clean the system-level footprint (see below).
|
||||
|
||||
Prefer the VM. Reach for the throwaway user only when nested virt is off the
|
||||
table and you accept an imperfect wipe.
|
||||
|
||||
## Why "a boundary without a separate user" doesn't really exist on macOS
|
||||
|
||||
macOS has no namespace/overlay story like Linux `unshare` + tmpfs. The
|
||||
options that sound like in-place sandboxes don't fit:
|
||||
|
||||
| Mechanism | Why it doesn't give you a clean, wipeable env |
|
||||
|---|---|
|
||||
| `sandbox-exec` / Seatbelt | Officially deprecated; confines *one process's* syscalls against a profile. It cannot present a fresh `$HOME` or a pristine `/usr/local`, and it won't let the Apple Container system service work. |
|
||||
| App Sandbox | Entitlement-based confinement for signed `.app` bundles, not a provisioning tool for a CLI dev environment. |
|
||||
| A second `$HOME` via `HOME=/tmp/foo` | Redirects only what honors `$HOME`. `install.sh` mostly does (it writes `~/.bot-bottle` and pipx/pip `--user` paths), but the Apple `container` install lands in `/usr/local` + a **system service**, and Homebrew lands in `/opt/homebrew` — all outside any `$HOME` you set. You'd get a false sense of "clean." |
|
||||
| APFS snapshot rollback (`tmutil localsnapshot`) | You can't roll the live boot volume back to a local snapshot without booting to Recovery; it's not a per-run userspace undo. |
|
||||
|
||||
So the honest answer to "is there some boundary that avoids a separate
|
||||
user?": yes — a **VM** — and it's the *stronger* boundary anyway. The only
|
||||
lighter-weight option is the separate user, with the caveats below.
|
||||
|
||||
## What a clean install actually touches (the footprint that decides "wipeable")
|
||||
|
||||
Grounding the teardown story in what `install.sh` and the backend create:
|
||||
|
||||
| Artifact | Location | In `$HOME`? | Survives user deletion? |
|
||||
|---|---|---|---|
|
||||
| Config / state / db | `~/.bot-bottle/{agents,bottles,contrib,state,db}` ([`install.sh:80-83`](../install.sh), [`bot_bottle/paths.py:59`](../bot_bottle/paths.py)) | ✅ | ❌ removed with home |
|
||||
| pipx venv + shim | `~/.local/pipx/venvs/bot-bottle`, shim in `~/.local/bin` ([`install.sh:87-89`](../install.sh)) | ✅ | ❌ removed with home |
|
||||
| private venv fallback (no pipx) | `~/.bot-bottle/venv` + symlink in `~/.local/bin` ([`install.sh`](../install.sh)) | ✅ | ❌ removed with home |
|
||||
| PATH / token exports | shell profile (`~/.zprofile`, etc.); `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` ([`README.md:74`](../README.md)) | ✅ | ❌ removed with home |
|
||||
| **Apple `container` install** | `/usr/local/...` + notarized `.pkg` receipts | ❌ | ✅ **stays** |
|
||||
| Apple `container` **service state** | per-user: `~/Library/Application Support/com.apple.container/` (`container system start`) | ✅ | ❌ removed with home |
|
||||
| **Homebrew** (if used for `container`/python) | `/opt/homebrew` | ❌ | ✅ **stays** |
|
||||
| Rosetta 2 (needed for image builds) | system | ❌ | ✅ **stays** |
|
||||
|
||||
The bold rows are the crux: **deleting the throwaway user does not uninstall
|
||||
the Apple Container runtime, Homebrew, or Rosetta.** A VM, by contrast, wipes
|
||||
100% of the above by definition — that's its entire advantage for this task.
|
||||
|
||||
The service row is the exception, and a live harness run corrected it: the
|
||||
Apple `container` service is **per-user**, not a host-wide launchd service.
|
||||
`container system status` reports an `appRoot` under
|
||||
`~/Library/Application Support/com.apple.container/`, and a freshly created
|
||||
account sees `container system service: NOT running` even while the creating
|
||||
admin's is running. That cuts both ways — the state is genuinely removed with
|
||||
the home, so the reset is *more* complete than this table first claimed, but
|
||||
it also means **no brand-new user can run a bottle until they run
|
||||
`container system start` once**. `doctor` correctly fails until they do, which
|
||||
is why `test` judges backend readiness separately from install correctness.
|
||||
|
||||
## Option A — Disposable Tart VM (recommended)
|
||||
|
||||
[Tart](https://tart.run) is a CLI-first macOS/Linux VM manager built on
|
||||
`Virtualization.framework`, purpose-built for exactly this "does it work on
|
||||
a clean macOS, without my settings/permissions/data" workflow. Keep one
|
||||
pristine *golden* image, clone a throwaway per run, delete it after.
|
||||
|
||||
```sh
|
||||
brew install cirruslabs/cli/tart
|
||||
|
||||
# One-time: build a golden base (either a prebuilt image or a vanilla IPSW).
|
||||
tart clone ghcr.io/cirruslabs/macos-tahoe-base:latest golden # ~25 GB pull
|
||||
# — or a truly vanilla install you click through once —
|
||||
# tart create golden --from-ipsw latest --disk-size 60
|
||||
|
||||
# Per test run: clone → boot → test → destroy.
|
||||
tart clone golden test-run
|
||||
tart run test-run &
|
||||
ssh admin@"$(tart ip test-run)"
|
||||
# inside the guest:
|
||||
# curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
|
||||
# bot-bottle doctor
|
||||
tart stop test-run
|
||||
tart delete test-run # back to pristine; golden is untouched
|
||||
```
|
||||
|
||||
Cloning is cheap (sparse files), so the golden image is your reset button —
|
||||
every `tart clone` is a fresh macOS. This is the closest thing to a Linux
|
||||
`docker run --rm` for a whole Mac.
|
||||
|
||||
**The nested-virt caveat (read before relying on it for runtime tests).**
|
||||
The Apple Container backend inside the guest needs
|
||||
`Virtualization.framework` to work *inside* the VM. Apple enables nested
|
||||
virtualization only on **M3 or newer**, on **macOS 15 (Sequoia) or later**;
|
||||
M2 and earlier are excluded by Apple, confirmed by Apple DTS. Consequences:
|
||||
|
||||
- **M3/M4 host:** full runtime works in the guest. `bot-bottle doctor`
|
||||
reports the backend ready and `start` can launch a bottle. Gold standard.
|
||||
- **M1/M2 host:** the guest can install bot-bottle and pass the Python /
|
||||
config-dir checks, but `doctor`'s backend check will fail and you cannot
|
||||
launch a bottle in the VM. Still perfectly good for testing *the
|
||||
installer*; not for the runtime.
|
||||
- **M4-specific:** a known bug blocks pre-Ventura guests on M4; use a
|
||||
current macOS guest (which you want anyway, since Apple `container`
|
||||
targets macOS 26 Tahoe).
|
||||
|
||||
UTM is the GUI equivalent on the same framework (and was first to expose
|
||||
nested virt) if you'd rather click; Tart wins for a scriptable
|
||||
spin-up/tear-down loop.
|
||||
|
||||
## Option B — Throwaway user via `sysadminctl` (pragmatic fallback)
|
||||
|
||||
Creating and deleting a user from the CLI is genuinely a two-liner, and it
|
||||
tests the **real** backend on any Apple Silicon Mac because the backend runs
|
||||
on the host hypervisor — no nested virt needed.
|
||||
|
||||
```sh
|
||||
# Create a self-contained admin user (admin needed for the container service).
|
||||
sudo sysadminctl -addUser bbtest -fullName "bot-bottle test" \
|
||||
-password 'throwaway' -admin
|
||||
|
||||
# Log into that account (fast-user-switch or the login window), then run the
|
||||
# installer as bbtest exactly as a new user would. When done:
|
||||
|
||||
sudo sysadminctl -deleteUser bbtest -secure # -secure erases the home dir
|
||||
```
|
||||
|
||||
Honest accounting of what this does and doesn't buy you:
|
||||
|
||||
- **Boundary strength:** it's a *hygiene / fresh-`$HOME`* boundary, **not a
|
||||
security boundary.** Same kernel, same admin group; an admin test user can
|
||||
touch system state. If the point is "clean environment," fine. If the point
|
||||
is "contain something untrusted," this is the wrong tool — use a VM.
|
||||
- **Wipe completeness:** `-secure` erases the home dir (so `~/.bot-bottle`,
|
||||
the pipx venv, and profile exports go away), but as the footprint table
|
||||
shows, the **Apple Container runtime, its launchd system service,
|
||||
Homebrew, and Rosetta persist.** For a truly repeatable "did a *system with
|
||||
nothing installed* work?" test, that residue defeats the purpose — the
|
||||
second run isn't clean.
|
||||
- **Operational gotchas:** don't pass real passwords on the command line (they
|
||||
land in `ps` and history — this is a throwaway credential, so it's
|
||||
tolerable here). Deletion must run as root from a normally-booted, admin-
|
||||
logged-in session; the Terminal needs **Full Disk Access** or you'll hit
|
||||
error `-14120` and a half-deleted account. Prefer letting the system place
|
||||
the home dir (don't pass `-home`), or deletion can orphan it.
|
||||
|
||||
Use this when you're on M1/M2, you specifically want to exercise the live
|
||||
backend, and you can tolerate the system-level runtime staying installed
|
||||
between runs (or you uninstall Apple `container` / brew by hand to reset).
|
||||
|
||||
## Honorable mentions
|
||||
|
||||
- **External bootable macOS volume.** A fresh macOS on an external SSD (or a
|
||||
separate APFS volume) is bare-metal disposable: no nested-virt limit, real
|
||||
backend works, and you `diskutil` the volume away to reset. Cost is reboot
|
||||
friction per run — good for an occasional thorough pass, poor for a tight
|
||||
loop.
|
||||
- **Rented / cloud Mac.** AWS EC2 Mac (dedicated Mac minis), Scaleway Apple
|
||||
silicon, or MacStadium give a genuinely throwaway host you release when
|
||||
done. Overkill for local iteration, but this is essentially what the
|
||||
project's own advisory `integration-macos` CI job needs — a self-hosted
|
||||
Apple Silicon runner with the `container` CLI, Python ≥ 3.11, and coverage
|
||||
on the launchd service's PATH ([`README.md:78`](../README.md)). If you end
|
||||
up standing up a cloud Mac for install testing, it doubles as that runner.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Default to a **disposable Tart VM** — it's the only option that wipes the
|
||||
*entire* footprint (including the Apple Container system service that a user
|
||||
deletion leaves behind), it's a real boundary, and the spin-up/tear-down
|
||||
loop is a two-command `tart clone` / `tart delete`. Confirm your chip first:
|
||||
on **M3/M4** it tests install *and* runtime end-to-end; on **M1/M2** it still
|
||||
cleanly tests `install.sh` + `doctor`'s Python/config path, and you fall back
|
||||
to a **throwaway `sysadminctl` admin user** for live-backend testing —
|
||||
accepting that it's a hygiene boundary and that you'll manually uninstall the
|
||||
Apple Container runtime / Homebrew between runs to get back to truly clean.
|
||||
|
||||
There is no third, lighter-weight "in-place boundary without a user" that
|
||||
actually delivers a clean, wipeable macOS — the VM *is* that answer, and it's
|
||||
the better one.
|
||||
|
||||
## Harness
|
||||
|
||||
The throwaway-user loop is scripted in
|
||||
[`scripts/macos-install-test.sh`](../../scripts/macos-install-test.sh):
|
||||
`up` creates the account, `run` pipes *this checkout's* `install.sh` into it
|
||||
headlessly (so a PR is verifiable before it lands) and lets the installer run
|
||||
`doctor`, `down` deletes the account and its home (the full reset), and
|
||||
`deep-reset` additionally uninstalls the host `container` runtime. It leans on
|
||||
the footprint analysis above — the reset is just user deletion because
|
||||
everything `install.sh` writes is user-home-local.
|
||||
|
||||
There are two one-shot cycles, because "does the installer work" and "can a new
|
||||
user actually run a bottle" are different questions:
|
||||
|
||||
```sh
|
||||
sudo ./scripts/macos-install-test.sh test # up → run → status → down
|
||||
sudo ./scripts/macos-install-test.sh test-ready # ... + prereqs before status
|
||||
```
|
||||
|
||||
`test` models a macOS system **without** the prerequisites set up for this user
|
||||
— which is the default state of every new account, since the `container`
|
||||
service is per-user. It asserts the install is sound and *reports* backend
|
||||
readiness without failing on it, because install.sh provides no backend and so
|
||||
cannot regress one.
|
||||
|
||||
`test-ready` models a system **with** them, then demands doctor go fully green,
|
||||
backend included. Both variants pass as of this writing.
|
||||
|
||||
### The per-user prerequisite is two steps, not one
|
||||
|
||||
Running it revealed that "set up the backend for this account" is more than
|
||||
starting a service:
|
||||
|
||||
1. **`container system start`** — the service is per-user. The run confirms it
|
||||
directly: the throwaway account's `appRoot` is
|
||||
`/Users/bbtest/Library/Application Support/com.apple.container/`, its own,
|
||||
and starting it left the admin's service untouched.
|
||||
2. **A guest kernel**, which also lives in that per-user app root. A fresh
|
||||
account has none, so `container system start` prompts to download one —
|
||||
and *only* prompts, since the flags default to asking. Headless callers
|
||||
must pass `--enable-kernel-install` or the command dies on
|
||||
`failed to read user input`.
|
||||
|
||||
Neither step is done by `bot-bottle backend setup --backend=macos-container`,
|
||||
which only checks and then tells you to run `container system start` yourself.
|
||||
So a new account's real path to a working backend is:
|
||||
`container system start --enable-kernel-install`.
|
||||
|
||||
The harness must also enter the user's launchd domain via
|
||||
`launchctl asuser <uid>` to do any of this. `container system start` registers
|
||||
`com.apple.container.apiserver` as a per-user launchd agent and talks to it
|
||||
over XPC; from plain `sudo -u` the caller is still in root's bootstrap
|
||||
namespace, the lookup crosses domains, and the apiserver answers
|
||||
`invalidState: "unauthorized request"` even though the agent started fine.
|
||||
|
||||
It refuses to start against an existing account (a reused home is not a clean
|
||||
install), and it tears the account down from an `EXIT`/`INT` trap armed the
|
||||
moment the account exists, so a failed or Ctrl-C'd run still leaves the machine
|
||||
clean. Its verdict is deliberately stricter than the installer's own: note that
|
||||
`install.sh` exits **0** when it finishes but `doctor` reports unmet
|
||||
prerequisites, so "the installer succeeded" is not the assertion — `test` fails
|
||||
if the install fails, if `bot-bottle` never reached the new user's `PATH`, or if
|
||||
`doctor` is unhappy. `BB_TEST_KEEP=1` skips the teardown to poke at a failure.
|
||||
|
||||
### What a fresh account actually inherits
|
||||
|
||||
Expect the first honest run on a developer Mac to fail at the *Python* gate,
|
||||
and expect that to be correct. A new account's `PATH` is just `/etc/paths`
|
||||
(`/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin`),
|
||||
which notably does **not** include `/opt/homebrew/bin`. Homebrew's `shellenv`
|
||||
line lives in the *installing* user's `~/.zprofile` and is not inherited, so a
|
||||
throwaway user resolves `python3` to `/usr/bin/python3` — the Command Line
|
||||
Tools stub, still **3.9.6** on macOS 26 — and `install.sh` correctly dies on its
|
||||
`3.11+` requirement. Your own shell resolving `python3` to a 3.14 Homebrew
|
||||
build says nothing about what a new user sees; that gap is exactly what this
|
||||
harness exists to expose.
|
||||
|
||||
## Sources
|
||||
|
||||
- [Apple Containers on macOS: technical comparison with Docker — The New Stack](https://thenewstack.io/apple-containers-on-macos-a-technical-comparison-with-docker/)
|
||||
- [How to Set Up Apple Containerization on macOS 26 — Stéphane Paquet](https://spaquet.medium.com/how-to-set-up-apple-containerization-on-macos-26-f870cc8c26cd)
|
||||
- [Install Apple Container CLI (macOS 15/26) — 4sysops](https://4sysops.com/archives/install-apple-container-cli-running-containers-natively-on-macos-15-sequoia-and-macos-26-tahoe/)
|
||||
- [Nested virtualization on Apple Silicon (M3+, macOS 15) — UTM issue #6700](https://github.com/utmapp/UTM/issues/6700)
|
||||
- [macOS 15 Sequoia nested virtualization for M3+ — Parallels Forums](https://forum.parallels.com/threads/macos-15-sequoia-nested-virtualization-for-m3-macs.364397/)
|
||||
- [M2 nested virtualization restriction (Apple DTS) — Apple Developer Forums](https://developer.apple.com/forums/thread/756723)
|
||||
- [M4 can't virtualize older macOS — Yahoo/Tech](https://tech.yahoo.com/computing/articles/m4-mac-computers-cant-virtualize-175122301.html)
|
||||
- [Tart — macOS/Linux VMs on Apple Silicon (Cirrus Labs)](https://tart.run/quick-start/)
|
||||
- [Tart GitHub](https://github.com/cirruslabs/tart)
|
||||
- [macOS VMs in a single command — frr.dev](https://www.frr.dev/posts/tart-macos-vms-from-terminal/)
|
||||
- [sysadminctl reference — SS64](https://ss64.com/mac/sysadminctl.html)
|
||||
- [User management from the macOS command line — macnotes](https://macnotes.wordpress.com/2019/03/28/user-management-create-remove-change-password-secure-token-from-macos-command-line/)
|
||||
@@ -5,10 +5,9 @@ model: opus
|
||||
bottle: dev
|
||||
skills:
|
||||
- init-prd
|
||||
git-gate:
|
||||
user:
|
||||
name: implementer-bot
|
||||
email: eric+implementer@dideric.is
|
||||
author:
|
||||
name: implementer-bot
|
||||
email: eric+implementer@dideric.is
|
||||
---
|
||||
|
||||
You are a feature-implementation agent running inside an ephemeral
|
||||
|
||||
+131
-51
@@ -8,14 +8,20 @@
|
||||
# pipx install bot-bottle # from a checkout or a published index
|
||||
# uv tool install bot-bottle
|
||||
#
|
||||
# This script is a thin bootstrapper: it checks prerequisites, installs the
|
||||
# package with pipx (falling back to pip --user), creates the config dir, and
|
||||
# runs `bot-bottle doctor`. It is idempotent (safe to re-run) and never uses
|
||||
# sudo. It does NOT install Docker or a VM backend for you — `doctor` reports
|
||||
# what's missing after install.
|
||||
# This script is a thin bootstrapper: it finds a Python 3.11+ interpreter,
|
||||
# installs the package with pipx (falling back to a private venv), creates the
|
||||
# config dir, and runs `bot-bottle doctor`. It is idempotent (safe to re-run)
|
||||
# and never uses sudo. It does NOT install Docker or a VM backend for you —
|
||||
# `doctor` reports what's missing after install.
|
||||
#
|
||||
# Env:
|
||||
# BOT_BOTTLE_PYTHON interpreter to install with (skips the search)
|
||||
# BOT_BOTTLE_INSTALL_SPEC pip/git spec to install instead of the default
|
||||
# BOT_BOTTLE_VENV where the non-pipx install lives (~/.bot-bottle/venv)
|
||||
set -eu
|
||||
|
||||
PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}"
|
||||
VENV="${BOT_BOTTLE_VENV:-${HOME}/.bot-bottle/venv}"
|
||||
MIN_PYTHON_MAJOR=3
|
||||
MIN_PYTHON_MINOR=11
|
||||
|
||||
@@ -28,17 +34,97 @@ die() {
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- prerequisites -----------------------------------------------------------
|
||||
# --- prerequisites: find an interpreter new enough ----------------------------
|
||||
|
||||
command -v python3 >/dev/null 2>&1 \
|
||||
|| die "python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR}+ is required but was not found"
|
||||
|
||||
python3 - "$MIN_PYTHON_MAJOR" "$MIN_PYTHON_MINOR" <<'PY' || die "python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} or newer is required"
|
||||
# Is $1 an interpreter that exists and meets the floor?
|
||||
python_ok() {
|
||||
[ -n "${1:-}" ] || return 1
|
||||
command -v "$1" >/dev/null 2>&1 || return 1
|
||||
"$1" - "$MIN_PYTHON_MAJOR" "$MIN_PYTHON_MINOR" <<'PY' >/dev/null 2>&1
|
||||
import sys
|
||||
|
||||
want = (int(sys.argv[1]), int(sys.argv[2]))
|
||||
raise SystemExit(0 if sys.version_info[:2] >= want else 1)
|
||||
PY
|
||||
}
|
||||
|
||||
python_version() {
|
||||
"$1" -c 'import sys; print("%d.%d.%d" % sys.version_info[:3])' 2>/dev/null
|
||||
}
|
||||
|
||||
# `python3` on PATH is often NOT the newest interpreter installed, and on macOS
|
||||
# it is usually the oldest: a fresh login shell's PATH is just /etc/paths, so
|
||||
# python3 resolves to the Command Line Tools stub (3.9.x) while the usable
|
||||
# 3.11+ build sits in /opt/homebrew/bin or a python.org framework directory,
|
||||
# reachable only via a line in the *installing* user's shell profile. A new
|
||||
# account inherits none of that. Look past PATH before giving up, so the common
|
||||
# case installs instead of dead-ending on a version error.
|
||||
find_python() {
|
||||
for candidate in \
|
||||
"${BOT_BOTTLE_PYTHON:-}" \
|
||||
python3 \
|
||||
python3.14 python3.13 python3.12 python3.11 \
|
||||
/opt/homebrew/bin/python3 \
|
||||
/usr/local/bin/python3 \
|
||||
"${HOME}/.local/bin/python3" \
|
||||
/Library/Frameworks/Python.framework/Versions/*/bin/python3
|
||||
do
|
||||
# An unmatched glob arrives here literally; python_ok rejects it.
|
||||
if python_ok "$candidate"; then
|
||||
command -v "$candidate"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# An explicit choice that doesn't work is an error, not a reason to quietly
|
||||
# search elsewhere and install somewhere the caller didn't ask for.
|
||||
if [ -n "${BOT_BOTTLE_PYTHON:-}" ] && ! python_ok "${BOT_BOTTLE_PYTHON}"; then
|
||||
if command -v "${BOT_BOTTLE_PYTHON}" >/dev/null 2>&1; then
|
||||
die "BOT_BOTTLE_PYTHON=${BOT_BOTTLE_PYTHON} is $(python_version "${BOT_BOTTLE_PYTHON}"), "\
|
||||
"below the ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} floor. Unset it to search for a newer one."
|
||||
fi
|
||||
die "BOT_BOTTLE_PYTHON=${BOT_BOTTLE_PYTHON} is not an executable interpreter."
|
||||
fi
|
||||
|
||||
PYTHON="$(find_python || true)"
|
||||
|
||||
if [ -z "${PYTHON}" ]; then
|
||||
path_python="$(command -v python3 2>/dev/null || true)"
|
||||
if [ -n "${path_python}" ]; then
|
||||
found="the python3 on your PATH is ${path_python} ($(python_version "${path_python}")), which is too old"
|
||||
else
|
||||
found="no python3 was found on your PATH"
|
||||
fi
|
||||
case "$(uname -s)" in
|
||||
Darwin) fix=" brew install python@3.12
|
||||
# or install from https://www.python.org/downloads/macos/
|
||||
# macOS itself ships only /usr/bin/python3, which is too old" ;;
|
||||
*) fix=" sudo apt install python3.12 # Debian/Ubuntu
|
||||
sudo dnf install python3.12 # Fedora/RHEL" ;;
|
||||
esac
|
||||
die "bot-bottle needs python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} or newer, and none was found.
|
||||
${found}.
|
||||
Also checked: python3.11-3.14, /opt/homebrew/bin, /usr/local/bin,
|
||||
~/.local/bin, and python.org framework builds.
|
||||
|
||||
Install a newer Python, then re-run this installer:
|
||||
${fix}
|
||||
|
||||
Already have one somewhere? Point at it directly:
|
||||
BOT_BOTTLE_PYTHON=/path/to/python3 sh install.sh"
|
||||
fi
|
||||
|
||||
# Be explicit when the interpreter isn't the obvious one, so nobody is left
|
||||
# wondering which Python their install ended up on.
|
||||
path_python="$(command -v python3 2>/dev/null || true)"
|
||||
if [ "${PYTHON}" != "${path_python}" ]; then
|
||||
say "using ${PYTHON} ($(python_version "${PYTHON}"))"
|
||||
if [ -n "${path_python}" ]; then
|
||||
say "note: 'python3' on your PATH is ${path_python} ($(python_version "${path_python}")), which is below the ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} floor"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Installing a `git+` spec (the default) shells out to git under the hood,
|
||||
# whether via pipx or pip. Fail early with a clear message rather than deep
|
||||
@@ -51,30 +137,6 @@ case "${PACKAGE_SPEC}" in
|
||||
;;
|
||||
esac
|
||||
|
||||
# The pip fallback needs a usable pip. Externally-managed interpreters
|
||||
# (PEP 668, common on Debian/Ubuntu/Homebrew) reject `pip install --user`;
|
||||
# pipx sidesteps that, so recommend it when pip can't be used.
|
||||
if ! command -v pipx >/dev/null 2>&1; then
|
||||
python3 -m pip --version >/dev/null 2>&1 || die \
|
||||
"neither pipx nor a usable 'python3 -m pip' was found. Install pipx "\
|
||||
"(recommended): 'python3 -m pip install --user pipx' or your OS package manager."
|
||||
if python3 - <<'PY'
|
||||
import os
|
||||
import sys
|
||||
import sysconfig
|
||||
|
||||
# PEP 668: an EXTERNALLY-MANAGED marker in the stdlib dir means pip refuses
|
||||
# to install into this interpreter without --break-system-packages.
|
||||
marker = os.path.join(sysconfig.get_path("stdlib"), "EXTERNALLY-MANAGED")
|
||||
raise SystemExit(0 if os.path.exists(marker) else 1)
|
||||
PY
|
||||
then
|
||||
die "this Python is externally managed (PEP 668), so 'pip install --user' is "\
|
||||
"blocked. Install pipx and re-run: 'python3 -m pip install --user --break-system-packages pipx', "\
|
||||
"then 'pipx ensurepath'."
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- config directories ------------------------------------------------------
|
||||
|
||||
mkdir -p \
|
||||
@@ -84,32 +146,50 @@ mkdir -p \
|
||||
|
||||
# --- install -----------------------------------------------------------------
|
||||
|
||||
BIN_DIR="${HOME}/.local/bin"
|
||||
|
||||
if command -v pipx >/dev/null 2>&1; then
|
||||
say "installing with pipx"
|
||||
pipx install --force "${PACKAGE_SPEC}"
|
||||
# --python pins the venv to the interpreter we vetted. Without it pipx uses
|
||||
# whichever Python it was itself installed with, which is not necessarily
|
||||
# the one that passed the version check above.
|
||||
say "installing with pipx (python: ${PYTHON})"
|
||||
pipx install --python "${PYTHON}" --force "${PACKAGE_SPEC}"
|
||||
# Ask pipx where it puts entry points rather than assuming ~/.local/bin.
|
||||
pipx_bin="$(pipx environment --value PIPX_BIN_DIR 2>/dev/null || true)"
|
||||
[ -n "${pipx_bin}" ] && BIN_DIR="${pipx_bin}"
|
||||
else
|
||||
say "pipx not found; installing with 'python3 -m pip install --user'"
|
||||
python3 -m pip install --user --upgrade "${PACKAGE_SPEC}"
|
||||
# No `pip install --user` fallback: PEP 668 makes it unusable on nearly
|
||||
# every interpreter a Mac offers (Homebrew and python.org are both
|
||||
# externally managed), and on Debian/Ubuntu too. A private venv sidesteps
|
||||
# that entirely — PEP 668 does not apply inside a venv — and `venv` is
|
||||
# stdlib, so unlike pipx there is nothing to bootstrap first.
|
||||
say "pipx not found; installing into a managed venv at ${VENV}"
|
||||
"${PYTHON}" -m venv --clear "${VENV}" || die \
|
||||
"could not create a virtualenv at ${VENV} using ${PYTHON}. On Debian/Ubuntu "\
|
||||
"the venv module ships separately: 'sudo apt install python3-venv'."
|
||||
"${VENV}/bin/python" -m pip install --upgrade "${PACKAGE_SPEC}"
|
||||
# Expose the entry point outside the venv, the way pipx would.
|
||||
mkdir -p "${BIN_DIR}"
|
||||
ln -sf "${VENV}/bin/bot-bottle" "${BIN_DIR}/bot-bottle"
|
||||
fi
|
||||
|
||||
# --- locate the entry point --------------------------------------------------
|
||||
|
||||
# The pip --user scripts directory is platform-specific: ~/.local/bin on Linux,
|
||||
# but ~/Library/Python/<X.Y>/bin on a python.org macOS interpreter. Ask the
|
||||
# interpreter for its own user-scheme scripts dir instead of hardcoding.
|
||||
USER_SCRIPTS="$(python3 - <<'PY'
|
||||
import sysconfig
|
||||
print(sysconfig.get_path("scripts", sysconfig.get_preferred_scheme("user")))
|
||||
PY
|
||||
)"
|
||||
|
||||
if command -v bot-bottle >/dev/null 2>&1; then
|
||||
BOT_BOTTLE_BIN="bot-bottle"
|
||||
elif [ -n "${USER_SCRIPTS}" ] && [ -x "${USER_SCRIPTS}/bot-bottle" ]; then
|
||||
BOT_BOTTLE_BIN="${USER_SCRIPTS}/bot-bottle"
|
||||
say "note: add ${USER_SCRIPTS} to your PATH to run 'bot-bottle' directly"
|
||||
elif [ -x "${BIN_DIR}/bot-bottle" ]; then
|
||||
BOT_BOTTLE_BIN="${BIN_DIR}/bot-bottle"
|
||||
# Name the file the user's own login shell actually reads. ~/.profile is
|
||||
# the safe default for non-zsh: bash falls back to it, and suggesting
|
||||
# ~/.bash_profile could shadow an existing ~/.profile.
|
||||
case "${SHELL:-}" in
|
||||
*/zsh) profile="~/.zprofile" ;;
|
||||
*) profile="~/.profile" ;;
|
||||
esac
|
||||
say "note: add ${BIN_DIR} to your PATH to run 'bot-bottle' directly:"
|
||||
say " echo 'export PATH=\"${BIN_DIR}:\$PATH\"' >> ${profile}"
|
||||
else
|
||||
die "bot-bottle was installed but is not on PATH; add ${USER_SCRIPTS:-your user scripts dir} to PATH and re-run"
|
||||
die "bot-bottle was installed but no entry point turned up in ${BIN_DIR}"
|
||||
fi
|
||||
|
||||
# --- verify ------------------------------------------------------------------
|
||||
|
||||
@@ -30,6 +30,7 @@ bot_bottle/manifest/agent.py
|
||||
bot_bottle/manifest/bottle.py
|
||||
bot_bottle/manifest/egress.py
|
||||
bot_bottle/manifest/extends.py
|
||||
bot_bottle/manifest/forge.py
|
||||
bot_bottle/manifest/git.py
|
||||
bot_bottle/manifest/index.py
|
||||
bot_bottle/manifest/loader.py
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
#
|
||||
# Why a pool + one-time setup: creating a TAP and assigning it an IP
|
||||
# needs CAP_NET_ADMIN. Pre-creating user-owned, pre-addressed TAPs
|
||||
# means `./cli.py start` never needs root. The nft table is static
|
||||
# means `bot-bottle start` never needs root. The nft table is static
|
||||
# (keyed on the `bbfc*` interface wildcard), so it covers every slot
|
||||
# without per-launch changes.
|
||||
#
|
||||
|
||||
Executable
+531
@@ -0,0 +1,531 @@
|
||||
#!/usr/bin/env bash
|
||||
# Clean-install test harness for the macOS (Apple `container`) path.
|
||||
#
|
||||
# Exercises install.sh the way a brand-new user would, inside a throwaway
|
||||
# macOS account you create and delete from the CLI. install.sh's entire
|
||||
# footprint is user-home-local — the pipx venv under ~/.local, or the private
|
||||
# venv at ~/.bot-bottle/venv plus a ~/.local/bin symlink, and the ~/.bot-bottle
|
||||
# config dir. It writes no shell-profile PATH line, and never installs the
|
||||
# backend (see
|
||||
# the header of install.sh), so deleting the user is a complete,
|
||||
# deterministic reset of everything the installer touched. The Apple
|
||||
# `container` runtime is a HOST prerequisite installed once and kept;
|
||||
# `deep-reset` is the rare escape hatch that also removes it.
|
||||
#
|
||||
# Why a throwaway user and not a disposable VM: bot-bottle's default macOS
|
||||
# backend is Apple `container`, which runs each container in its own
|
||||
# Virtualization.framework microVM. Running that backend inside a macOS
|
||||
# guest VM needs nested virtualization, which Apple gates to M3+ silicon.
|
||||
# On M1/M2 a separate user account is the only way to get a clean $HOME
|
||||
# while still reaching the real host backend. Full rationale in
|
||||
# docs/research/testing-clean-install-on-macos.md.
|
||||
#
|
||||
# Usage:
|
||||
# sudo ./scripts/macos-install-test.sh test # up -> run -> status -> down
|
||||
# sudo ./scripts/macos-install-test.sh test-ready # ... with prereqs satisfied
|
||||
# sudo ./scripts/macos-install-test.sh up # create the throwaway user
|
||||
# sudo ./scripts/macos-install-test.sh run # run install.sh (+doctor) as it
|
||||
# sudo ./scripts/macos-install-test.sh prereqs # start the per-user backend svc
|
||||
# ./scripts/macos-install-test.sh status # user present? backend ready?
|
||||
# sudo ./scripts/macos-install-test.sh down # delete user + home (the reset)
|
||||
# sudo ./scripts/macos-install-test.sh deep-reset # ALSO uninstall host `container`
|
||||
#
|
||||
# TWO TEST VARIANTS, because "does the installer work" and "can a new user
|
||||
# actually run a bottle" are different questions with different answers:
|
||||
#
|
||||
# test A macOS system WITHOUT the prerequisites set up for this user —
|
||||
# the default state of any brand-new account, since the Apple
|
||||
# `container` service is per-user. Asserts the INSTALL is sound
|
||||
# (entry point present, doctor runs without crashing, python and
|
||||
# config check out). Backend readiness is reported, not required:
|
||||
# install.sh does not provide a backend and cannot regress one,
|
||||
# so failing on it would make this permanently red.
|
||||
#
|
||||
# test-ready A macOS system WITH the prerequisites satisfied — it starts the
|
||||
# per-user `container` service (and installs its guest kernel)
|
||||
# for the throwaway user first, then demands doctor go fully
|
||||
# green, backend included. This is the end-to-end claim: a new
|
||||
# user can actually run a bottle. Note it downloads a guest
|
||||
# kernel per run, because the previous run's went with the
|
||||
# deleted home; BB_TEST_KERNEL_INSTALL=0 skips that.
|
||||
#
|
||||
# Both refuse to start if the account already exists (a reused home is not a
|
||||
# clean install), and both tear the account down on the way out however they
|
||||
# exit, so a failed run never leaves an orphan behind. install.sh itself exits
|
||||
# 0 when doctor reports unmet prerequisites, so either variant is a stricter
|
||||
# gate than running the installer by hand.
|
||||
#
|
||||
# Config via env:
|
||||
# BB_TEST_USER account short name (default: bbtest)
|
||||
# BB_TEST_FULLNAME account full name (default: "bot-bottle install test")
|
||||
# BB_TEST_ADMIN 1=admin (reach container svc), 0=standard (default: 1)
|
||||
# BB_TEST_INSTALL_URL curl this install.sh instead of piping the local checkout
|
||||
# BB_TEST_KEEP 1=`test` skips its teardown, to poke at a failure
|
||||
# BB_TEST_REQUIRE_BACKEND 1=`test` also fails when no backend is ready
|
||||
# BB_TEST_KERNEL_INSTALL 0=`test-ready` skips the guest-kernel download
|
||||
# BB_TEST_REPO_URL https repo the throwaway user clones (default: this one)
|
||||
# BOT_BOTTLE_INSTALL_SPEC passed through to install.sh (pip / git spec)
|
||||
#
|
||||
# Notes:
|
||||
# * Run from a normally-booted admin session. Grant Terminal *Full Disk
|
||||
# Access* (System Settings -> Privacy & Security) or `down` half-fails
|
||||
# with error -14120 and leaves an orphaned account.
|
||||
# * `sysadminctl` always exits 0 even on failure, so `up`/`down` verify
|
||||
# the result with `dscl` and fail loudly on a mismatch.
|
||||
# * The account is created without a password: `run` drives it headlessly
|
||||
# via `sudo -u`, which never needs the target's password. The account
|
||||
# cannot GUI-login, which this harness does not require.
|
||||
# * `run` covers the installer + `bot-bottle doctor`. Actually launching a
|
||||
# bottle from the throwaway user may need a full launchd user session
|
||||
# (`launchctl asuser`); on M1/M2 the backend can't run under nested virt
|
||||
# anyway, so this harness stops at install + doctor.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
USER_NAME="${BB_TEST_USER:-bbtest}"
|
||||
FULL_NAME="${BB_TEST_FULLNAME:-bot-bottle install test}"
|
||||
ADMIN="${BB_TEST_ADMIN:-1}"
|
||||
# https, not the ssh `origin`: the throwaway user has no key of ours.
|
||||
BB_TEST_REPO_URL="${BB_TEST_REPO_URL:-https://gitea.dideric.is/didericis/bot-bottle.git}"
|
||||
|
||||
_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
_REPO_ROOT="$(cd "$_SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
# Set by `test`, which chains the steps itself and so suppresses the
|
||||
# "here's the next command to run" hints the individual steps print.
|
||||
IN_TEST=0
|
||||
|
||||
# --- guards ----------------------------------------------------------
|
||||
require_macos() {
|
||||
[ "$(uname -s)" = "Darwin" ] \
|
||||
|| { echo "error: this harness is macOS-only (uname is $(uname -s))" >&2; exit 1; }
|
||||
}
|
||||
|
||||
require_root() {
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
echo "error: '$1' needs root; re-run under sudo" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
user_exists() { dscl . -read "/Users/$USER_NAME" >/dev/null 2>&1; }
|
||||
|
||||
user_uid() {
|
||||
dscl . -read "/Users/$USER_NAME" UniqueID 2>/dev/null | awk '{print $2}'
|
||||
}
|
||||
|
||||
# Whether we can enter the target user's launchd domain.
|
||||
#
|
||||
# This matters more than it sounds. `container system start` registers
|
||||
# com.apple.container.apiserver as a per-USER launchd agent and then talks to
|
||||
# it over XPC. Under plain `sudo -u` the caller is still in root's bootstrap
|
||||
# namespace, so the lookup crosses domains and the apiserver answers
|
||||
# invalidState: "unauthorized request"
|
||||
# even though it launched fine. `launchctl asuser <uid>` puts the whole
|
||||
# invocation in that user's domain, which is also what a real user gets from
|
||||
# Terminal — so it is the more faithful way to run *everything* here, not just
|
||||
# the service start. Falls back to plain sudo when unavailable, since a
|
||||
# never-logged-in account may have no bootstrappable domain at all.
|
||||
_SESSION_OK=""
|
||||
user_session_available() {
|
||||
if [ -z "$_SESSION_OK" ]; then
|
||||
local uid
|
||||
uid="$(user_uid)"
|
||||
if [ -n "$uid" ] && launchctl asuser "$uid" true >/dev/null 2>&1; then
|
||||
_SESSION_OK=1
|
||||
else
|
||||
_SESSION_OK=0
|
||||
fi
|
||||
fi
|
||||
[ "$_SESSION_OK" = "1" ]
|
||||
}
|
||||
|
||||
# Run a shell snippet as the throwaway user in a fresh login shell.
|
||||
#
|
||||
# The snippet goes in on stdin, never as an argument. `sudo -i` joins its argv
|
||||
# into a single string and hands that to the login shell's -c, so quoting in an
|
||||
# argument is not preserved and a newline ends the command outright ("sh: -c:
|
||||
# line 1: syntax error: unexpected end of file"). Feeding `sh -s` from stdin
|
||||
# sidesteps the joining entirely and lets snippets be multi-line.
|
||||
run_as_user() {
|
||||
if user_session_available; then
|
||||
printf '%s\n' "$1" | launchctl asuser "$(user_uid)" sudo -u "$USER_NAME" -i sh -s
|
||||
else
|
||||
printf '%s\n' "$1" | sudo -u "$USER_NAME" -i sh -s
|
||||
fi
|
||||
}
|
||||
|
||||
# `bot-bottle doctor` as the throwaway user. Non-zero when no entry point was
|
||||
# installed at all, or when doctor itself is unhappy.
|
||||
#
|
||||
# Deliberately does NOT require `bot-bottle` on PATH: install.sh prints the
|
||||
# PATH line rather than editing a shell profile, so on a fresh account the
|
||||
# entry point is installed and working but not on PATH. Demanding PATH here
|
||||
# would fail every run for a reason the installer intends.
|
||||
#
|
||||
# Doctor's own exit code conflates two unrelated things: whether the install
|
||||
# works, and whether this user has a backend ready to run a bottle. Those need
|
||||
# different verdicts here. install.sh explicitly does not install a backend,
|
||||
# and the Apple `container` service is per-USER — its state lives in
|
||||
# ~/Library/Application Support/com.apple.container — so a brand-new account
|
||||
# never has one running, no matter how correct the install is. Failing on that
|
||||
# would leave `test` permanently red for something the installer cannot fix
|
||||
# and cannot regress. So: assert the install, report the backend.
|
||||
# BB_TEST_REQUIRE_BACKEND=1 makes backend readiness fatal too.
|
||||
doctor_as_user() {
|
||||
local out rc=0 bad=0
|
||||
out="$(mktemp "${TMPDIR:-/tmp}/bb-doctor.XXXXXX")"
|
||||
# shellcheck disable=SC2016 # $HOME/$bb must expand in the *target* user's
|
||||
# shell, not in this one — that's the whole point of the single quotes.
|
||||
run_as_user '
|
||||
for bb in bot-bottle "$HOME/.local/bin/bot-bottle" "$HOME/.bot-bottle/venv/bin/bot-bottle"; do
|
||||
if command -v "$bb" >/dev/null 2>&1; then
|
||||
case "$bb" in
|
||||
bot-bottle) : ;;
|
||||
*) echo " (not on PATH — running $bb directly, as install.sh advises)" ;;
|
||||
esac
|
||||
exec "$bb" doctor
|
||||
fi
|
||||
done
|
||||
echo " no bot-bottle entry point found for this user" >&2
|
||||
exit 1
|
||||
' >"$out" 2>&1 || rc=$?
|
||||
cat "$out"
|
||||
|
||||
# An unhandled exception is always an install/product defect, never an
|
||||
# environment fact — this is exactly how the PermissionError on `ip` showed
|
||||
# up, and doctor's non-zero exit alone would not have distinguished it.
|
||||
if grep -q 'Traceback (most recent call last)' "$out"; then
|
||||
echo " doctor crashed (traceback above) — a defect, not a missing prerequisite" >&2
|
||||
bad=1
|
||||
fi
|
||||
grep -qE '^ok: +python:' "$out" \
|
||||
|| { echo " doctor never reported a usable python" >&2; bad=1; }
|
||||
grep -qE '^ok: +config:' "$out" \
|
||||
|| { echo " doctor never reported a usable config dir" >&2; bad=1; }
|
||||
if [ "$rc" -ne 0 ] && ! grep -qE '^(fail|warn): +backend' "$out"; then
|
||||
echo " doctor failed for something other than backend readiness" >&2
|
||||
bad=1
|
||||
fi
|
||||
|
||||
local backend_ready=1
|
||||
grep -qE '^fail: +backend' "$out" && backend_ready=0
|
||||
rm -f "$out"
|
||||
|
||||
[ "$bad" -eq 0 ] || return 1
|
||||
if [ "$backend_ready" -eq 0 ]; then
|
||||
if [ "${BB_TEST_REQUIRE_BACKEND:-0}" = "1" ]; then
|
||||
echo " no backend is ready and BB_TEST_REQUIRE_BACKEND=1" >&2
|
||||
return 1
|
||||
fi
|
||||
echo " note: install is sound; no backend ready for this user."
|
||||
echo " Expected on a fresh account — the Apple 'container' service is"
|
||||
echo " per-user and needs one 'container system start'. Set"
|
||||
echo " BB_TEST_REQUIRE_BACKEND=1 to treat this as a failure."
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# --- commands --------------------------------------------------------
|
||||
cmd_up() {
|
||||
require_macos
|
||||
require_root up
|
||||
if user_exists; then
|
||||
echo "$USER_NAME already exists; nothing to do (run 'down' first to reset)"
|
||||
return 0
|
||||
fi
|
||||
local admin_flag=()
|
||||
[ "$ADMIN" = "1" ] && admin_flag=(-admin)
|
||||
# No -password: the account is only ever driven headlessly via `sudo -u`,
|
||||
# which doesn't need one. sysadminctl warns about FileVault here; that's
|
||||
# irrelevant to a headless test account.
|
||||
sysadminctl -addUser "$USER_NAME" -fullName "$FULL_NAME" "${admin_flag[@]}" || true
|
||||
# sysadminctl exits 0 regardless of outcome, so confirm the account landed.
|
||||
user_exists || { echo "error: failed to create $USER_NAME" >&2; return 1; }
|
||||
if [ "$IN_TEST" = 1 ]; then
|
||||
echo "created $USER_NAME (admin=$ADMIN)"
|
||||
else
|
||||
echo "created $USER_NAME (admin=$ADMIN). Install into it with: sudo $0 run"
|
||||
fi
|
||||
}
|
||||
|
||||
# The package spec to install. install.sh's own default is the repo's DEFAULT
|
||||
# BRANCH, which would mean piping this checkout's installer into the throwaway
|
||||
# user and then installing main's code — verifying the PR's install.sh against
|
||||
# a package that doesn't contain the PR. Pin to the branch under test instead.
|
||||
#
|
||||
# The branch has to be pushed: the throwaway user clones over https and cannot
|
||||
# read this checkout (mode-700 home, and no SSH key for the ssh remote), so
|
||||
# what it gets is whatever the remote has. Say so when that differs from here.
|
||||
resolve_install_spec() {
|
||||
if [ -n "${BOT_BOTTLE_INSTALL_SPEC:-}" ]; then
|
||||
echo "$BOT_BOTTLE_INSTALL_SPEC"
|
||||
return 0
|
||||
fi
|
||||
local branch
|
||||
branch="$(git -C "$_REPO_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null)" || branch=""
|
||||
if [ -z "$branch" ] || [ "$branch" = "HEAD" ]; then
|
||||
echo "note: not on a named branch; installing the repo default" >&2
|
||||
echo "git+${BB_TEST_REPO_URL}"
|
||||
return 0
|
||||
fi
|
||||
if [ -n "$(git -C "$_REPO_ROOT" status --porcelain 2>/dev/null)" ]; then
|
||||
echo "warning: working tree is dirty — the throwaway user installs" >&2
|
||||
echo " $branch as pushed, not what is on disk here." >&2
|
||||
elif ! git -C "$_REPO_ROOT" diff --quiet "@{upstream}" 2>/dev/null; then
|
||||
echo "warning: $branch differs from its upstream — push first, or the" >&2
|
||||
echo " throwaway user installs stale code." >&2
|
||||
fi
|
||||
echo "git+${BB_TEST_REPO_URL}@${branch}"
|
||||
}
|
||||
|
||||
cmd_run() {
|
||||
require_macos
|
||||
require_root run
|
||||
user_exists || { echo "error: $USER_NAME does not exist; run 'sudo $0 up' first" >&2; return 1; }
|
||||
|
||||
local spec
|
||||
spec="$(resolve_install_spec)"
|
||||
|
||||
echo "== installing bot-bottle as $USER_NAME =="
|
||||
echo "== spec: $spec =="
|
||||
if [ -n "${BB_TEST_INSTALL_URL:-}" ]; then
|
||||
run_as_user "BOT_BOTTLE_INSTALL_SPEC='$spec' curl -fsSL '$BB_TEST_INSTALL_URL' | sh"
|
||||
else
|
||||
# Test THIS checkout's install.sh, not the published one, so a PR is
|
||||
# verifiable before it lands. Feed it in on stdin rather than staging a
|
||||
# copy somewhere the throwaway user can read: the redirect is opened by
|
||||
# root before sudo drops privileges, so the tester's mode-700 home is a
|
||||
# non-issue, there's no temp file to leak if the run is interrupted, and
|
||||
# `sh -s` is the same shape as the documented `curl … | sh` install.
|
||||
#
|
||||
# The spec is prepended to the stream as an export rather than passed
|
||||
# as an argument, for the same argv-joining reason as run_as_user.
|
||||
{
|
||||
printf "export BOT_BOTTLE_INSTALL_SPEC='%s'\n" "$spec"
|
||||
cat "$_REPO_ROOT/install.sh"
|
||||
} | sudo -u "$USER_NAME" -i sh -s
|
||||
fi
|
||||
[ "$IN_TEST" = 1 ] \
|
||||
|| echo "== install.sh runs 'doctor' itself; re-check anytime with: $0 status =="
|
||||
}
|
||||
|
||||
# Informational, with one teeth-bearing case: when it can actually reach
|
||||
# doctor (root, account present) its exit status is doctor's, so `test` and
|
||||
# any other caller can use it as the post-install assertion.
|
||||
cmd_status() {
|
||||
require_macos
|
||||
local rc=0
|
||||
if user_exists; then
|
||||
echo "user: $USER_NAME present"
|
||||
if [ "$(id -u)" -eq 0 ]; then
|
||||
echo "doctor (as $USER_NAME):"
|
||||
doctor_as_user || rc=1
|
||||
else
|
||||
echo " (re-run under sudo to run 'bot-bottle doctor' as $USER_NAME)"
|
||||
fi
|
||||
else
|
||||
echo "user: $USER_NAME absent"
|
||||
fi
|
||||
if command -v container >/dev/null 2>&1; then
|
||||
echo "backend: apple 'container' present ($(container --version 2>/dev/null | head -1))"
|
||||
else
|
||||
echo "backend: apple 'container' NOT on PATH (host prerequisite; install once)"
|
||||
fi
|
||||
return "$rc"
|
||||
}
|
||||
|
||||
cmd_down() {
|
||||
require_macos
|
||||
require_root down
|
||||
if ! user_exists; then
|
||||
echo "$USER_NAME not present; nothing to remove"
|
||||
return 0
|
||||
fi
|
||||
# A plain -deleteUser removes the home dir, which is the whole reset.
|
||||
# -secure is a no-op on modern macOS (secure erase of the home folder
|
||||
# was removed in Sierra), so it buys nothing here.
|
||||
sysadminctl -deleteUser "$USER_NAME" || true
|
||||
if user_exists; then
|
||||
echo "error: $USER_NAME still present after delete." >&2
|
||||
echo " - grant Terminal Full Disk Access (System Settings > Privacy & Security), or" >&2
|
||||
echo " - it may hold the last Secure Token (won't happen while another admin exists)" >&2
|
||||
return 1
|
||||
fi
|
||||
echo "removed $USER_NAME and its home — install surface is clean."
|
||||
}
|
||||
|
||||
# Satisfy the throwaway user's backend prerequisites.
|
||||
#
|
||||
# `bot-bottle backend setup --backend=macos-container` deliberately does NOT do
|
||||
# this: it only checks, then tells you to run `container system start` yourself
|
||||
# (see backend/macos_container/setup.py — "no privileged host setup required").
|
||||
# So the harness runs the command the product asks for, as the user who needs
|
||||
# it, which is the whole prerequisite for the macOS backend given the CLI is
|
||||
# already installed host-wide.
|
||||
cmd_prereqs() {
|
||||
require_macos
|
||||
require_root prereqs
|
||||
user_exists || { echo "error: $USER_NAME does not exist" >&2; return 1; }
|
||||
command -v container >/dev/null 2>&1 || {
|
||||
echo "error: the Apple 'container' CLI is not on PATH. It is a HOST" >&2
|
||||
echo " prerequisite this harness does not install; see the header." >&2
|
||||
return 1
|
||||
}
|
||||
echo "starting the per-user Apple 'container' service for $USER_NAME"
|
||||
user_session_available || {
|
||||
echo "warning: no launchd session for $USER_NAME; the service start is" >&2
|
||||
echo " likely to fail with 'unauthorized request'. See user_session_available." >&2
|
||||
}
|
||||
# The kernel flag is not optional for a headless run. `container system
|
||||
# start` PROMPTS for the default Linux kernel when neither flag is given
|
||||
# ("default: prompt user"), and a throwaway account has no kernel because
|
||||
# it lives in the per-user app root — so the prompt always fires here and
|
||||
# dies on "failed to read user input" with no TTY.
|
||||
#
|
||||
# Enabling it is also the honest choice for what this variant claims: the
|
||||
# backend cannot actually run a bottle without a guest kernel. The cost is
|
||||
# that every test-ready run downloads one afresh, since the previous run's
|
||||
# copy went with the deleted home. BB_TEST_KERNEL_INSTALL=0 skips the
|
||||
# download when you only care that the service comes up.
|
||||
local kernel_flag=--enable-kernel-install
|
||||
[ "${BB_TEST_KERNEL_INSTALL:-1}" = "0" ] && kernel_flag=--disable-kernel-install
|
||||
echo " ($kernel_flag)"
|
||||
run_as_user "container system start $kernel_flag" || {
|
||||
echo "error: 'container system start' failed for $USER_NAME" >&2
|
||||
echo " invalidState/\"unauthorized request\" means the CLI could not reach" >&2
|
||||
echo " its per-user apiserver over XPC — a never-GUI-logged-in account may" >&2
|
||||
echo " have no usable launchd domain; log into $USER_NAME once to get one." >&2
|
||||
echo " A download failure means the guest kernel could not be fetched; retry" >&2
|
||||
echo " with network, or BB_TEST_KERNEL_INSTALL=0 to skip it." >&2
|
||||
return 1
|
||||
}
|
||||
run_as_user 'container system status' || {
|
||||
echo "error: the service is still not running for $USER_NAME" >&2
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
# Teardown half of the test cycles, installed as an EXIT trap the moment the
|
||||
# account exists so a failure — or a Ctrl-C — still leaves the machine clean.
|
||||
_test_teardown() {
|
||||
local rc=$?
|
||||
trap - EXIT INT TERM
|
||||
if [ "${BB_TEST_KEEP:-0}" = "1" ]; then
|
||||
echo
|
||||
echo "== [$_STEPS/$_STEPS] down: SKIPPED (BB_TEST_KEEP=1) =="
|
||||
echo " $USER_NAME is still around; remove it with: sudo $0 down"
|
||||
exit "$rc"
|
||||
fi
|
||||
echo
|
||||
echo "== [$_STEPS/$_STEPS] down =="
|
||||
cmd_down || rc=1
|
||||
if [ "$rc" -eq 0 ]; then
|
||||
echo
|
||||
echo "PASS: $_PASS_CLAIM"
|
||||
else
|
||||
echo
|
||||
echo "FAIL: see above (the throwaway account was torn down regardless)." >&2
|
||||
fi
|
||||
exit "$rc"
|
||||
}
|
||||
|
||||
# The two variants differ only in whether the backend prerequisites get
|
||||
# satisfied before doctor runs, which is exactly the question each one asks:
|
||||
#
|
||||
# test a brand-new user on a machine where nothing has been set up for
|
||||
# them. Asserts the INSTALL is sound; backend readiness is
|
||||
# reported, not required, because install.sh does not provide a
|
||||
# backend and cannot regress one.
|
||||
# test-ready the same user with the documented prerequisites satisfied.
|
||||
# Asserts doctor goes fully green, backend included — the
|
||||
# end-to-end "a new user can actually run a bottle" claim.
|
||||
_test_cycle() {
|
||||
local with_prereqs="$1"
|
||||
require_macos
|
||||
require_root test
|
||||
# A pre-existing account means a pre-existing home, which is the one thing
|
||||
# this harness exists to rule out. Don't silently test a dirty install.
|
||||
if user_exists; then
|
||||
echo "error: $USER_NAME already exists, so this would not be a clean install." >&2
|
||||
echo " reset first: sudo $0 down" >&2
|
||||
return 1
|
||||
fi
|
||||
IN_TEST=1
|
||||
|
||||
echo "== [1/$_STEPS] up =="
|
||||
cmd_up
|
||||
trap _test_teardown EXIT INT TERM
|
||||
|
||||
echo
|
||||
echo "== [2/$_STEPS] run =="
|
||||
cmd_run
|
||||
|
||||
if [ "$with_prereqs" = 1 ]; then
|
||||
echo
|
||||
echo "== [3/$_STEPS] prereqs =="
|
||||
cmd_prereqs || {
|
||||
echo "error: could not satisfy the backend prerequisites (see above)." >&2
|
||||
return 1
|
||||
}
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "== [$((_STEPS - 1))/$_STEPS] status =="
|
||||
# install.sh exits 0 even when doctor reports unmet prerequisites, so the
|
||||
# install succeeding is not the verdict — this is.
|
||||
cmd_status || {
|
||||
echo "error: doctor is unhappy for a freshly installed user (see above)." >&2
|
||||
echo " re-run with BB_TEST_KEEP=1 to keep $USER_NAME around and dig in." >&2
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
cmd_test() {
|
||||
_STEPS=4
|
||||
_PASS_CLAIM="a brand-new user can install bot-bottle; the install is sound."
|
||||
_test_cycle 0
|
||||
}
|
||||
|
||||
cmd_test_ready() {
|
||||
_STEPS=5
|
||||
_PASS_CLAIM="a brand-new user with the prerequisites met passes doctor outright."
|
||||
# With the prerequisites satisfied there is no excuse for an unready
|
||||
# backend, so this variant demands the thing `test` only reports.
|
||||
BB_TEST_REQUIRE_BACKEND=1
|
||||
_test_cycle 1
|
||||
}
|
||||
|
||||
cmd_deep_reset() {
|
||||
require_macos
|
||||
require_root deep-reset
|
||||
# Remove the user first (idempotent), then the HOST-level container
|
||||
# runtime that a user deletion leaves behind under /usr/local + launchd.
|
||||
cmd_down || true
|
||||
if command -v container >/dev/null 2>&1; then
|
||||
# The service can run in more than one launchd context (the invoking
|
||||
# user's and root's), so stop both, best-effort.
|
||||
[ -n "${SUDO_USER:-}" ] && sudo -u "$SUDO_USER" container system stop 2>/dev/null || true
|
||||
container system stop 2>/dev/null || true
|
||||
if [ -x /usr/local/bin/uninstall-container.sh ]; then
|
||||
/usr/local/bin/uninstall-container.sh -d || true
|
||||
echo "uninstalled the host Apple 'container' runtime"
|
||||
else
|
||||
echo "note: /usr/local/bin/uninstall-container.sh not found; runtime left as-is" >&2
|
||||
fi
|
||||
else
|
||||
echo "no 'container' runtime on PATH; nothing further to remove"
|
||||
fi
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
test) cmd_test ;;
|
||||
test-ready) cmd_test_ready ;;
|
||||
up) cmd_up ;;
|
||||
run) cmd_run ;;
|
||||
prereqs) cmd_prereqs ;;
|
||||
status) cmd_status ;;
|
||||
down) cmd_down ;;
|
||||
deep-reset) cmd_deep_reset ;;
|
||||
*) echo "usage: $0 {test|test-ready|up|run|prereqs|status|down|deep-reset}" >&2 ; exit 2 ;;
|
||||
esac
|
||||
+2
-2
@@ -95,7 +95,7 @@ BOT_BOTTLE_RUN_CANARIES=1 python -m scripts.unittest_gate \
|
||||
```
|
||||
4. Skip guards live in `tests._backend` and gate on the backend's own
|
||||
readiness check, `bot_bottle.backend.has_backend` — the same probe
|
||||
behind `./cli.py backend status`:
|
||||
behind `bot-bottle backend status`:
|
||||
- Backend-agnostic tests (go through `get_bottle_backend()`) decorate
|
||||
the class with `@skip_unless_selected_backend_available()` — the test
|
||||
runs against whichever backend `BOT_BOTTLE_BACKEND` selects and skips
|
||||
@@ -105,7 +105,7 @@ BOT_BOTTLE_RUN_CANARIES=1 python -m scripts.unittest_gate \
|
||||
`backend.docker.*`, …) decorate with `@skip_unless_backend("docker")`
|
||||
so they no-op under a run targeting a different backend.
|
||||
|
||||
Each CI integration job runs `./cli.py backend status --backend=<name>`
|
||||
Each CI integration job runs `bot-bottle backend status --backend=<name>`
|
||||
as a preflight, which prints a clear per-check summary and exits non-zero
|
||||
when the backend is missing — so absent infrastructure fails the job
|
||||
instead of hiding among per-test `unittest.skip` lines.
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
Each integration test targets the backend named by ``BOT_BOTTLE_BACKEND``
|
||||
(default ``docker``) and gates on that backend's full readiness check —
|
||||
``is_backend_ready()`` (equivalent to ``./cli.py backend status``), not just
|
||||
``is_backend_ready()`` (equivalent to ``bot-bottle backend status``), not just
|
||||
a binary-on-PATH probe. When the backend is not ready, diagnostic output is
|
||||
printed during test discovery so the operator sees a concrete reason for each
|
||||
skip.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Tests for the backend-aware skip guards in ``tests/_backend.py``.
|
||||
|
||||
The guards delegate their readiness check to
|
||||
``bot_bottle.backend.is_backend_ready`` (the probe behind ``./cli.py backend
|
||||
``bot_bottle.backend.is_backend_ready`` (the probe behind ``bot-bottle backend
|
||||
status``); here that probe is mocked so the unit job asserts the
|
||||
selection/skip logic without either backend present on the runner.
|
||||
"""
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Unit: the generic `backend` CLI command.
|
||||
|
||||
`./cli.py backend {setup,status} [--backend=NAME]` resolves a backend
|
||||
`bot-bottle backend {setup,status} [--backend=NAME]` resolves a backend
|
||||
and dispatches to its `setup()` / `status()` classmethods — no
|
||||
backend-specific commands. Also covers the docker backend's own
|
||||
setup/status logic (the firecracker path is exercised by
|
||||
|
||||
@@ -17,6 +17,7 @@ from bot_bottle.cli.commands import cleanup as cmd
|
||||
def _make_backend(empty: bool = True):
|
||||
backend = MagicMock()
|
||||
plan = MagicMock(empty=empty)
|
||||
plan.intersect.return_value = plan
|
||||
backend.prepare_cleanup.return_value = plan
|
||||
backend.cleanup = MagicMock()
|
||||
return backend, plan
|
||||
@@ -41,8 +42,8 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
docker.prepare_cleanup.assert_called_once()
|
||||
fc.prepare_cleanup.assert_called_once()
|
||||
self.assertEqual(2, docker.prepare_cleanup.call_count)
|
||||
self.assertEqual(2, fc.prepare_cleanup.call_count)
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
fc.cleanup.assert_called_once_with(fc_plan)
|
||||
|
||||
@@ -68,7 +69,7 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
docker.prepare_cleanup.assert_called_once()
|
||||
self.assertEqual(2, docker.prepare_cleanup.call_count)
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
macos.prepare_cleanup.assert_not_called()
|
||||
|
||||
@@ -135,6 +136,28 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
fc.cleanup.assert_not_called()
|
||||
|
||||
def test_executes_only_displayed_resources_still_current(self):
|
||||
backend = MagicMock()
|
||||
preview = MagicMock(empty=False)
|
||||
refreshed = MagicMock(empty=False)
|
||||
approved = MagicMock(empty=False)
|
||||
preview.intersect.return_value = approved
|
||||
backend.prepare_cleanup.side_effect = [preview, refreshed]
|
||||
|
||||
with patch.object(
|
||||
cmd, "known_backend_names", return_value=("firecracker",),
|
||||
), patch.object(
|
||||
cmd, "get_bottle_backend", return_value=backend,
|
||||
), patch.object(
|
||||
cmd, "has_backend", return_value=True,
|
||||
), patch.object(
|
||||
cmd, "_prompt_yes", return_value=True,
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
preview.intersect.assert_called_once_with(refreshed)
|
||||
backend.cleanup.assert_called_once_with(approved)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -44,6 +44,17 @@ class TestProjectNaming(unittest.TestCase):
|
||||
|
||||
|
||||
class TestComposeProjectListing(unittest.TestCase):
|
||||
def test_compose_ls_empty_when_docker_unusable(self):
|
||||
# Missing is the obvious case; present-but-not-executable raises
|
||||
# PermissionError instead, which must not escape as a crash.
|
||||
for exc in (FileNotFoundError, PermissionError(13, "Permission denied", "docker")):
|
||||
with self.subTest(exc=type(exc).__name__):
|
||||
with mock.patch(
|
||||
"bot_bottle.backend.docker.compose.subprocess.run",
|
||||
side_effect=exc,
|
||||
):
|
||||
self.assertEqual([], list_compose_projects())
|
||||
|
||||
def test_compose_ls_error_warns_by_default(self):
|
||||
with (
|
||||
mock.patch(
|
||||
|
||||
@@ -3,9 +3,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
import stat
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.store.db_store import DbStore
|
||||
from bot_bottle.store.migrations import TableMigrations
|
||||
@@ -22,6 +24,48 @@ class TestDbStoreIsMigrated(unittest.TestCase):
|
||||
store = _store(Path(d))
|
||||
self.assertFalse(store.is_migrated())
|
||||
|
||||
def test_creates_private_directory_and_database_before_first_open(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
parent = Path(d) / "store"
|
||||
store = _store(parent)
|
||||
self.assertEqual(0o700, stat.S_IMODE(parent.stat().st_mode))
|
||||
self.assertFalse(store.db_path.exists())
|
||||
store.migrate()
|
||||
self.assertEqual(0o600, stat.S_IMODE(store.db_path.stat().st_mode))
|
||||
|
||||
def test_repairs_existing_permissions(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
parent = Path(d) / "store"
|
||||
parent.mkdir(mode=0o755)
|
||||
db_path = parent / "test.db"
|
||||
db_path.touch(mode=0o644)
|
||||
store = _store(parent)
|
||||
self.assertEqual(db_path, store.db_path)
|
||||
self.assertEqual(0o700, stat.S_IMODE(parent.stat().st_mode))
|
||||
self.assertEqual(0o600, stat.S_IMODE(db_path.stat().st_mode))
|
||||
|
||||
def test_permission_repair_failure_is_not_suppressed(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
parent = Path(d) / "store"
|
||||
parent.mkdir()
|
||||
(parent / "test.db").touch()
|
||||
with patch("os.fchmod", side_effect=OSError("denied")):
|
||||
with self.assertRaisesRegex(OSError, "denied"):
|
||||
_store(parent)
|
||||
|
||||
def test_rejects_database_symlink_without_changing_target(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
parent = Path(d) / "store"
|
||||
parent.mkdir()
|
||||
target = Path(d) / "target"
|
||||
target.touch(mode=0o644)
|
||||
(parent / "test.db").symlink_to(target)
|
||||
|
||||
with self.assertRaises(OSError):
|
||||
_store(parent)
|
||||
|
||||
self.assertEqual(0o644, stat.S_IMODE(target.stat().st_mode))
|
||||
|
||||
def test_returns_false_when_schema_versions_missing(self):
|
||||
# DB file exists but has no schema_versions table → OperationalError → False.
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
|
||||
@@ -14,9 +14,12 @@ from __future__ import annotations
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from tests.unit import use_bottle_root
|
||||
from bot_bottle import bottle_state
|
||||
from bot_bottle.backend import EnumerationError
|
||||
from bot_bottle.backend.docker import cleanup
|
||||
from bot_bottle.backend.docker.cleanup import _list_orphan_state_dirs
|
||||
|
||||
|
||||
@@ -116,5 +119,37 @@ class TestOrphanStateDirs(_FakeHomeMixin, unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestAuthoritativeDiscovery(unittest.TestCase):
|
||||
def test_prepare_requires_authoritative_compose_projects(self):
|
||||
with patch.object(cleanup.docker_mod, "require_docker"), \
|
||||
patch.object(
|
||||
cleanup, "list_compose_projects",
|
||||
side_effect=EnumerationError("compose unavailable"),
|
||||
) as projects, self.assertRaisesRegex(
|
||||
EnumerationError, "compose unavailable",
|
||||
):
|
||||
cleanup.prepare_cleanup()
|
||||
projects.assert_called_once_with(
|
||||
warn_on_error=False,
|
||||
raise_on_error=True,
|
||||
)
|
||||
|
||||
def test_container_query_failure_raises(self):
|
||||
failed = cleanup.subprocess.CompletedProcess(
|
||||
[], 1, stdout="", stderr="daemon unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=failed), \
|
||||
self.assertRaisesRegex(EnumerationError, "daemon unavailable"):
|
||||
cleanup._list_prefixed_containers()
|
||||
|
||||
def test_network_query_failure_raises(self):
|
||||
failed = cleanup.subprocess.CompletedProcess(
|
||||
[], 1, stdout="", stderr="daemon unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=failed), \
|
||||
self.assertRaisesRegex(EnumerationError, "daemon unavailable"):
|
||||
cleanup._list_prefixed_networks()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
@@ -21,7 +22,7 @@ from bot_bottle.backend import Bottle, BottleSpec, ExecResult
|
||||
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
||||
from bot_bottle.egress import EgressPlan
|
||||
from bot_bottle.git_gate import GitGatePlan
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
from bot_bottle.manifest import ManifestGitUser, ManifestIndex
|
||||
|
||||
|
||||
class _Provider(AgentProvider):
|
||||
@@ -50,15 +51,39 @@ def _plan(*, git_user: dict | None = None, # type: ignore
|
||||
user_cwd: str = "/tmp/x",
|
||||
stage_dir: Path | None = None) -> DockerBottlePlan:
|
||||
bottle_json: dict = {} # type: ignore
|
||||
if git_user is not None:
|
||||
bottle_json["git-gate"] = {"user": git_user}
|
||||
if git_repos is not None:
|
||||
bottle_json.setdefault("git-gate", {})["repos"] = git_repos
|
||||
# Identity now lives on the agent's `author` block; at composition it
|
||||
# populates manifest.bottle.git_user, which provision_git reads
|
||||
# (production unchanged). When the caller passes a full name+email we
|
||||
# route it through `author`; the name-only / email-only cases (which
|
||||
# exercise provision_git emitting a single `git config` line) can't be
|
||||
# expressed via `author` (both fields required), so we inject the
|
||||
# partial ManifestGitUser onto the composed bottle directly.
|
||||
agent_json: dict = {"skills": [], "prompt": "", "bottle": "dev"} # type: ignore
|
||||
full_author = (
|
||||
git_user
|
||||
if git_user and git_user.get("name") and git_user.get("email")
|
||||
else None
|
||||
)
|
||||
if full_author is not None:
|
||||
agent_json["author"] = full_author
|
||||
index = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": bottle_json},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
"agents": {"demo": agent_json},
|
||||
})
|
||||
manifest = index.load_for_agent("demo")
|
||||
if git_user is not None and full_author is None:
|
||||
manifest = replace(
|
||||
manifest,
|
||||
bottle=replace(
|
||||
manifest.bottle,
|
||||
git_user=ManifestGitUser(
|
||||
name=git_user.get("name", ""),
|
||||
email=git_user.get("email", ""),
|
||||
),
|
||||
),
|
||||
)
|
||||
spec = BottleSpec(
|
||||
manifest=index, agent_name="demo",
|
||||
copy_cwd=copy_cwd, user_cwd=user_cwd,
|
||||
|
||||
@@ -32,15 +32,19 @@ class TestProcessScan(unittest.TestCase):
|
||||
self.assertEqual(
|
||||
Path("/cache/run/dev-a"),
|
||||
fc_cleanup._run_dir_of(
|
||||
f"firecracker --config-file {run_root}/dev-a/config.json", run_root
|
||||
("firecracker", "--config-file",
|
||||
f"{run_root}/dev-a/config.json"), run_root
|
||||
),
|
||||
)
|
||||
# infra/builder VMs elsewhere, or nested paths, are not ours.
|
||||
self.assertIsNone(
|
||||
fc_cleanup._run_dir_of("firecracker --config-file /elsewhere/config.json", run_root)
|
||||
fc_cleanup._run_dir_of(
|
||||
("firecracker", "--config-file", "/elsewhere/config.json"),
|
||||
run_root,
|
||||
)
|
||||
)
|
||||
self.assertIsNone(
|
||||
fc_cleanup._run_dir_of("firecracker --no-config", run_root)
|
||||
fc_cleanup._run_dir_of(("firecracker", "--no-config"), run_root)
|
||||
)
|
||||
|
||||
def test_scan_splits_live_dirs_from_orphan_pids(self):
|
||||
@@ -48,17 +52,40 @@ class TestProcessScan(unittest.TestCase):
|
||||
run_root = Path(tmp)
|
||||
(run_root / "live-a").mkdir() # dir present -> live VM, protected
|
||||
# "gone-b" dir intentionally absent -> lingering VMM, orphan pid
|
||||
out = (
|
||||
f"111 firecracker --config-file {run_root}/live-a/config.json\n"
|
||||
f"222 firecracker --config-file {run_root}/gone-b/config.json\n"
|
||||
"333 firecracker --config-file /elsewhere/config.json\n"
|
||||
"notanint firecracker --config-file x\n"
|
||||
)
|
||||
with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(out)):
|
||||
args = {
|
||||
111: ("firecracker", "--config-file",
|
||||
f"{run_root}/live-a/config.json"),
|
||||
222: ("firecracker", "--config-file",
|
||||
f"{run_root}/gone-b/config.json"),
|
||||
333: ("firecracker", "--config-file", "/elsewhere/config.json"),
|
||||
}
|
||||
with patch.object(
|
||||
fc_cleanup.subprocess, "run",
|
||||
return_value=_proc("111\n222\n333\nnotanint\n"),
|
||||
), patch.object(
|
||||
fc_cleanup, "_process_args", side_effect=args.get,
|
||||
):
|
||||
live, orphan_pids = fc_cleanup._scan_processes(run_root)
|
||||
self.assertEqual({str(run_root / "live-a")}, live)
|
||||
self.assertEqual([222], orphan_pids)
|
||||
|
||||
def test_scan_preserves_spaces_in_config_path(self):
|
||||
with tempfile.TemporaryDirectory(prefix="fc cache ") as tmp:
|
||||
run_root = Path(tmp)
|
||||
live = run_root / "live bottle"
|
||||
live.mkdir()
|
||||
with patch.object(
|
||||
fc_cleanup.subprocess, "run", return_value=_proc("111\n"),
|
||||
), patch.object(
|
||||
fc_cleanup, "_process_args",
|
||||
return_value=("firecracker", "--config-file",
|
||||
str(live / "config.json")),
|
||||
):
|
||||
self.assertEqual(
|
||||
({str(live)}, []),
|
||||
fc_cleanup._scan_processes(run_root),
|
||||
)
|
||||
|
||||
def test_scan_empty_when_pgrep_finds_no_processes(self):
|
||||
with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(returncode=1)):
|
||||
self.assertEqual((set(), []), fc_cleanup._scan_processes(Path("/x")))
|
||||
@@ -117,9 +144,14 @@ class TestProcessScan(unittest.TestCase):
|
||||
run_root = Path(tmp)
|
||||
(run_root / "live-a").mkdir()
|
||||
(run_root / "dead-b").mkdir()
|
||||
out = f"111 firecracker --config-file {run_root}/live-a/config.json\n"
|
||||
with patch.object(fc_cleanup, "_run_root", return_value=run_root), \
|
||||
patch.object(fc_cleanup.subprocess, "run", return_value=_proc(out)):
|
||||
patch.object(fc_cleanup.subprocess, "run",
|
||||
return_value=_proc("111\n")), \
|
||||
patch.object(
|
||||
fc_cleanup, "_process_args",
|
||||
return_value=("firecracker", "--config-file",
|
||||
str(run_root / "live-a/config.json")),
|
||||
):
|
||||
plan = fc_cleanup.prepare_cleanup()
|
||||
self.assertEqual((), plan.vm_pids)
|
||||
self.assertEqual((str(run_root / "dead-b"),), plan.run_dirs)
|
||||
@@ -132,18 +164,46 @@ class TestCleanupRemoval(unittest.TestCase):
|
||||
vm_pids=(101,),
|
||||
run_dirs=("/run/dev-x",),
|
||||
)
|
||||
with patch.object(fc_cleanup.os, "kill") as kill, \
|
||||
patch.object(fc_cleanup.shutil, "rmtree") as rmtree, \
|
||||
with patch.object(fc_cleanup, "prepare_cleanup", return_value=plan), \
|
||||
patch.object(fc_cleanup, "_run_root", return_value=Path("/run")), \
|
||||
patch.object(fc_cleanup, "_terminate_orphan") as terminate, \
|
||||
patch("bot_bottle.backend.cleanup_control.shutil.rmtree") as rmtree, \
|
||||
patch.object(fc_cleanup, "info"):
|
||||
fc_cleanup.cleanup(plan)
|
||||
kill.assert_called_once()
|
||||
rmtree.assert_called_once_with("/run/dev-x", ignore_errors=True)
|
||||
terminate.assert_called_once_with(101, Path("/run"))
|
||||
rmtree.assert_called_once_with(Path("/run/dev-x"))
|
||||
|
||||
def test_cleanup_tolerates_dead_pid(self):
|
||||
plan = FirecrackerBottleCleanupPlan(vm_pids=(999,))
|
||||
with patch.object(fc_cleanup.os, "kill", side_effect=ProcessLookupError), \
|
||||
patch.object(fc_cleanup, "info"):
|
||||
fc_cleanup.cleanup(plan) # must not raise
|
||||
def test_cleanup_skips_resources_no_longer_in_refreshed_plan(self):
|
||||
preview = FirecrackerBottleCleanupPlan(
|
||||
vm_pids=(999,), run_dirs=("/run/reused",),
|
||||
)
|
||||
with patch.object(
|
||||
fc_cleanup, "prepare_cleanup",
|
||||
return_value=FirecrackerBottleCleanupPlan(),
|
||||
), patch.object(fc_cleanup, "_terminate_orphan") as terminate, \
|
||||
patch("bot_bottle.backend.cleanup_control.shutil.rmtree") as rmtree:
|
||||
fc_cleanup.cleanup(preview)
|
||||
terminate.assert_not_called()
|
||||
rmtree.assert_not_called()
|
||||
|
||||
def test_pidfd_prevents_pid_reuse_from_signalling_unrelated_process(self):
|
||||
with patch.object(fc_cleanup.os, "pidfd_open", return_value=7), \
|
||||
patch.object(
|
||||
fc_cleanup.Path, "read_bytes",
|
||||
return_value=b"/usr/bin/python\0worker.py\0",
|
||||
), patch.object(fc_cleanup.signal, "pidfd_send_signal") as send, \
|
||||
patch.object(fc_cleanup.os, "close"):
|
||||
fc_cleanup._terminate_orphan(101, Path("/run"))
|
||||
send.assert_not_called()
|
||||
|
||||
def test_pidfd_signals_revalidated_orphan(self):
|
||||
command = b"firecracker\0--config-file\0/run/gone/config.json\0"
|
||||
with patch.object(fc_cleanup.os, "pidfd_open", return_value=7), \
|
||||
patch.object(fc_cleanup.Path, "read_bytes", return_value=command), \
|
||||
patch.object(fc_cleanup.signal, "pidfd_send_signal") as send, \
|
||||
patch.object(fc_cleanup.os, "close"), patch.object(fc_cleanup, "info"):
|
||||
fc_cleanup._terminate_orphan(101, Path("/run"))
|
||||
send.assert_called_once_with(7, fc_cleanup.signal.SIGTERM)
|
||||
|
||||
|
||||
class TestCleanupPlan(unittest.TestCase):
|
||||
|
||||
@@ -56,6 +56,21 @@ class TestNetpoolProbes(unittest.TestCase):
|
||||
with patch.object(netpool.subprocess, "run", side_effect=FileNotFoundError):
|
||||
self.assertFalse(netpool._run_ok(["nft"]))
|
||||
|
||||
def test_run_ok_false_on_unexecutable_binary(self):
|
||||
# A name on PATH that this user can't execute raises PermissionError,
|
||||
# not FileNotFoundError — CPython reports that EACCES in preference to
|
||||
# the ENOENT from the other PATH entries. Catching only the latter made
|
||||
# `doctor` die with a traceback on a fresh macOS account.
|
||||
with patch.object(netpool.subprocess, "run",
|
||||
side_effect=PermissionError(13, "Permission denied", "ip")):
|
||||
self.assertFalse(netpool._run_ok(["ip", "link", "show", "bbfc0"]))
|
||||
|
||||
def test_overlapping_routes_empty_when_ip_unusable(self):
|
||||
for exc in (FileNotFoundError, PermissionError(13, "Permission denied", "ip")):
|
||||
with self.subTest(exc=type(exc).__name__), \
|
||||
patch.object(netpool.subprocess, "run", side_effect=exc):
|
||||
self.assertEqual([], netpool.overlapping_routes())
|
||||
|
||||
def test_tap_and_nft_probes(self):
|
||||
with patch.object(netpool, "_run_ok", return_value=True) as ok:
|
||||
self.assertTrue(netpool.tap_present("bbfc0"))
|
||||
|
||||
@@ -0,0 +1,426 @@
|
||||
"""Unit: trusted agent forge identity & guidance
|
||||
(PRD 0082).
|
||||
|
||||
Covers the net-new surface: agent `author`, agent `forge-accounts` (Gitea API
|
||||
URL validation + host token reference), the repo→forge association resolved at
|
||||
composition, the synthesized proxy-held egress route, and the generated,
|
||||
non-secret prompt guidance. Legacy git-gate.user / cwd-agent behavior lives in
|
||||
the manifest test modules.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.egress import (
|
||||
egress_forge_routes,
|
||||
egress_render_routes,
|
||||
egress_resolve_token_values,
|
||||
egress_routes_for_bottle,
|
||||
egress_token_env_map,
|
||||
)
|
||||
from bot_bottle.manifest import ManifestError, ManifestIndex
|
||||
from bot_bottle.manifest.forge import (
|
||||
ManifestForgeAccount,
|
||||
canonicalize_forge_url,
|
||||
render_forge_guidance,
|
||||
)
|
||||
|
||||
|
||||
GITEA = {
|
||||
"url": "https://gitea.dideric.is/api/v1",
|
||||
"auth": {"type": "token", "token_secret": "GITEA_CLAUDE_TOKEN"},
|
||||
}
|
||||
|
||||
|
||||
def _repo(*, forge: str | None = None) -> dict[str, object]:
|
||||
entry: dict[str, object] = {
|
||||
"url": "ssh://git@100.78.141.42:30009/didericis/bot-bottle.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
}
|
||||
if forge is not None:
|
||||
entry["forge"] = forge
|
||||
return entry
|
||||
|
||||
|
||||
def _index(
|
||||
*,
|
||||
author: dict[str, object] | None = None,
|
||||
forge_accounts: dict[str, object] | None = None,
|
||||
repo_forge: str | None = None,
|
||||
) -> ManifestIndex:
|
||||
"""Build an eager index with one agent 'claude' + bottle 'bb'."""
|
||||
agent: dict[str, object] = {"bottle": "bb", "prompt": ""}
|
||||
if author is not None:
|
||||
agent["author"] = author
|
||||
if forge_accounts is not None:
|
||||
agent["forge-accounts"] = forge_accounts
|
||||
bottle: dict[str, object] = {
|
||||
"git-gate": {"repos": {"bot-bottle": _repo(forge=repo_forge)}}
|
||||
}
|
||||
return ManifestIndex.from_json_obj(
|
||||
{"bottles": {"bb": bottle}, "agents": {"claude": agent}}
|
||||
)
|
||||
|
||||
|
||||
def _error(
|
||||
callable_: Callable[..., object], *args: object, **kwargs: object,
|
||||
) -> str:
|
||||
try:
|
||||
callable_(*args, **kwargs)
|
||||
except ManifestError as e:
|
||||
return str(e)
|
||||
raise AssertionError("expected ManifestError was not raised")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# author
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAuthor(unittest.TestCase):
|
||||
def test_populates_git_identity(self) -> None:
|
||||
idx = _index(author={"name": "didericis-claude", "email": "e+c@x.is"})
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual("didericis-claude", m.bottle.git_user.name)
|
||||
self.assertEqual("e+c@x.is", m.bottle.git_user.email)
|
||||
self.assertEqual(
|
||||
"name=didericis-claude, email=e+c@x.is",
|
||||
m.git_identity_summary(),
|
||||
)
|
||||
|
||||
def test_absent_author_no_identity(self) -> None:
|
||||
m = _index().load_for_agent("claude", ())
|
||||
self.assertTrue(m.bottle.git_user.is_empty())
|
||||
self.assertIsNone(m.git_identity_summary())
|
||||
|
||||
def test_name_required(self) -> None:
|
||||
msg = _error(_index, author={"email": "e@x.is"})
|
||||
self.assertIn("author.name must be a non-empty string", msg)
|
||||
|
||||
def test_email_required(self) -> None:
|
||||
msg = _error(_index, author={"name": "n"})
|
||||
self.assertIn("author.email must be a non-empty string", msg)
|
||||
|
||||
def test_email_rejects_whitespace(self) -> None:
|
||||
msg = _error(_index, author={"name": "n", "email": "a b@x.is"})
|
||||
self.assertIn("must not contain whitespace", msg)
|
||||
|
||||
def test_name_rejects_newline(self) -> None:
|
||||
msg = _error(_index, author={"name": "a\nb", "email": "e@x.is"})
|
||||
self.assertIn("author.name must not contain newlines", msg)
|
||||
|
||||
def test_unknown_key(self) -> None:
|
||||
msg = _error(
|
||||
_index, author={"name": "n", "email": "e@x.is", "role": "x"}
|
||||
)
|
||||
self.assertIn("author has unknown key", msg)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# forge-accounts URL validation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeUrl(unittest.TestCase):
|
||||
def test_canonical_ok(self) -> None:
|
||||
canonical, origin, host, prefix = canonicalize_forge_url(
|
||||
"a", "g", "https://Gitea.Dideric.is/api/v1/"
|
||||
)
|
||||
# trailing slash normalized; host lowercased.
|
||||
self.assertEqual("https://gitea.dideric.is/api/v1", canonical)
|
||||
self.assertEqual("https://gitea.dideric.is", origin)
|
||||
self.assertEqual("gitea.dideric.is", host)
|
||||
self.assertEqual("/api/v1", prefix)
|
||||
|
||||
def test_port_preserved(self) -> None:
|
||||
canonical, origin, _host, _ = canonicalize_forge_url(
|
||||
"a", "g", "https://gitea.local:3000/api/v1"
|
||||
)
|
||||
self.assertEqual("https://gitea.local:3000/api/v1", canonical)
|
||||
self.assertEqual("https://gitea.local:3000", origin)
|
||||
|
||||
def test_http_fails(self) -> None:
|
||||
self.assertIn("must use https", _error(
|
||||
canonicalize_forge_url, "a", "g", "http://gitea/api/v1"))
|
||||
|
||||
def test_userinfo_fails(self) -> None:
|
||||
self.assertIn("userinfo", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://u:p@gitea/api/v1"))
|
||||
|
||||
def test_query_fails(self) -> None:
|
||||
self.assertIn("query string", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://gitea/api/v1?x=1"))
|
||||
|
||||
def test_fragment_fails(self) -> None:
|
||||
self.assertIn("fragment", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://gitea/api/v1#x"))
|
||||
|
||||
def test_missing_host_fails(self) -> None:
|
||||
self.assertIn("hostname", _error(
|
||||
canonicalize_forge_url, "a", "g", "https:///api/v1"))
|
||||
|
||||
def test_bad_path_fails(self) -> None:
|
||||
self.assertIn("Gitea API base", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://gitea/api/v2"))
|
||||
|
||||
def test_non_string_fails(self) -> None:
|
||||
self.assertIn("required", _error(
|
||||
canonicalize_forge_url, "a", "g", None))
|
||||
|
||||
def test_malformed_url_fails(self) -> None:
|
||||
# An unparseable URL (invalid IPv6 literal) trips urlsplit's ValueError.
|
||||
self.assertIn("not a valid URL", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://[/api/v1"))
|
||||
|
||||
|
||||
class TestForgeAccount(unittest.TestCase):
|
||||
def test_parses(self) -> None:
|
||||
acct = ManifestForgeAccount.from_dict("a", "didericis-gitea", GITEA)
|
||||
self.assertEqual("didericis-gitea", acct.alias)
|
||||
self.assertEqual("gitea.dideric.is", acct.host)
|
||||
self.assertEqual("token", acct.auth_type)
|
||||
self.assertEqual("GITEA_CLAUDE_TOKEN", acct.token_secret)
|
||||
|
||||
def test_non_kebab_alias_fails(self) -> None:
|
||||
self.assertIn("valid alias", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "Bad_Alias", GITEA))
|
||||
|
||||
def test_auth_required(self) -> None:
|
||||
self.assertIn("missing required 'auth'", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1"}))
|
||||
|
||||
def test_bad_auth_type_fails(self) -> None:
|
||||
self.assertIn("auth.type must be", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1",
|
||||
"auth": {"type": "basic", "token_secret": "T"}}))
|
||||
|
||||
def test_token_secret_required(self) -> None:
|
||||
self.assertIn("token_secret must be", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1", "auth": {"type": "token"}}))
|
||||
|
||||
def test_unknown_key_fails(self) -> None:
|
||||
self.assertIn("unknown key", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{**GITEA, "bogus": 1}))
|
||||
|
||||
def test_unknown_auth_key_fails(self) -> None:
|
||||
self.assertIn("auth has unknown key", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1",
|
||||
"auth": {"type": "token", "token_secret": "T", "bogus": 1}}))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# repo -> forge association (composition)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeAssociations(unittest.TestCase):
|
||||
def test_resolves_when_repo_references_alias(self) -> None:
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual(1, len(m.forge_associations))
|
||||
assoc = m.forge_associations[0]
|
||||
self.assertEqual("didericis-gitea", assoc.alias)
|
||||
self.assertEqual(("bot-bottle",), assoc.repo_names)
|
||||
|
||||
def test_unreferenced_account_yields_no_association(self) -> None:
|
||||
idx = _index(forge_accounts={"didericis-gitea": GITEA}) # repo has no forge
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual((), m.forge_associations)
|
||||
|
||||
def test_unknown_alias_fails_closed(self) -> None:
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA}, repo_forge="nope"
|
||||
)
|
||||
msg = _error(idx.load_for_agent, "claude", ())
|
||||
self.assertIn("references forge alias 'nope'", msg)
|
||||
self.assertIn("not defined on agent", msg)
|
||||
|
||||
def test_forge_without_account_fails_closed(self) -> None:
|
||||
idx = _index(repo_forge="didericis-gitea") # no forge-accounts at all
|
||||
self.assertIn("(none)", _error(idx.load_for_agent, "claude", ()))
|
||||
|
||||
def _two_alias_index(self, t1: str, t2: str) -> ManifestIndex:
|
||||
"""Two aliases on the SAME host, each referenced by a distinct repo."""
|
||||
def acct(token: str) -> dict[str, object]:
|
||||
return {
|
||||
"url": "https://same.example/api/v1",
|
||||
"auth": {"type": "token", "token_secret": token},
|
||||
}
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {"bb": {"git-gate": {"repos": {
|
||||
"r1": {"url": "ssh://git@h/x/r1.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
"forge": "g1"},
|
||||
"r2": {"url": "ssh://git@h/x/r2.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
"forge": "g2"},
|
||||
}}}},
|
||||
"agents": {"claude": {"bottle": "bb", "prompt": "",
|
||||
"forge-accounts": {"g1": acct(t1),
|
||||
"g2": acct(t2)}}},
|
||||
})
|
||||
|
||||
def test_conflicting_aliases_same_host_fail_closed(self) -> None:
|
||||
# Two aliases on the same host with DIFFERENT tokens is ambiguous —
|
||||
# the proxy routes by host, so it must fail before launch rather than
|
||||
# silently authenticate every call as one account.
|
||||
idx = self._two_alias_index("FIRST_TOKEN", "SECOND_TOKEN")
|
||||
msg = _error(idx.load_for_agent, "claude", ())
|
||||
self.assertIn("both resolve to host 'same.example'", msg)
|
||||
self.assertIn("ambiguous", msg)
|
||||
|
||||
def test_matching_aliases_same_host_ok(self) -> None:
|
||||
# Identical url/auth under two aliases is unambiguous: one route.
|
||||
idx = self._two_alias_index("SAME_TOKEN", "SAME_TOKEN")
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual(2, len(m.forge_associations)) # both referenced
|
||||
routes = egress_forge_routes(m.forge_associations)
|
||||
self.assertEqual(1, len(routes)) # deduped to a single proxy route
|
||||
self.assertEqual("SAME_TOKEN", routes[0].token_ref)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# synthesized egress route (proxy-held credential)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeEgressRoutes(unittest.TestCase):
|
||||
def _assoc(self):
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
return idx.load_for_agent("claude", ())
|
||||
|
||||
def test_route_shape(self) -> None:
|
||||
routes = egress_forge_routes(self._assoc().forge_associations)
|
||||
self.assertEqual(1, len(routes))
|
||||
r = routes[0]
|
||||
self.assertEqual("gitea.dideric.is", r.host)
|
||||
self.assertEqual("token", r.auth_scheme)
|
||||
self.assertEqual("GITEA_CLAUDE_TOKEN", r.token_ref)
|
||||
self.assertTrue(r.inspect)
|
||||
# scoped to the API prefix.
|
||||
self.assertEqual("/api/v1", r.matches[0].paths[0].value)
|
||||
|
||||
def test_token_slot_and_resolution(self) -> None:
|
||||
m = self._assoc()
|
||||
forge_routes = egress_forge_routes(m.forge_associations)
|
||||
routes = egress_routes_for_bottle(m.bottle, (), forge_routes)
|
||||
token_map = egress_token_env_map(routes)
|
||||
# one slot mapping the proxy env slot -> host env var name.
|
||||
self.assertEqual({"EGRESS_TOKEN_0": "GITEA_CLAUDE_TOKEN"}, token_map)
|
||||
resolved = egress_resolve_token_values(
|
||||
token_map, {"GITEA_CLAUDE_TOKEN": "sekret-value"}
|
||||
)
|
||||
self.assertEqual({"EGRESS_TOKEN_0": "sekret-value"}, resolved)
|
||||
|
||||
def test_rendered_routes_hide_secret_name_and_value(self) -> None:
|
||||
m = self._assoc()
|
||||
routes = egress_routes_for_bottle(
|
||||
m.bottle, (), egress_forge_routes(m.forge_associations)
|
||||
)
|
||||
rendered = egress_render_routes(routes)
|
||||
self.assertIn("gitea.dideric.is", rendered)
|
||||
self.assertIn("EGRESS_TOKEN_0", rendered) # slot, not the host var name
|
||||
self.assertNotIn("GITEA_CLAUDE_TOKEN", rendered)
|
||||
self.assertNotIn("sekret-value", rendered)
|
||||
|
||||
def test_dedup_by_host(self) -> None:
|
||||
# Two repos referencing the same alias share one route/credential.
|
||||
idx = ManifestIndex.from_json_obj({
|
||||
"bottles": {"bb": {"git-gate": {"repos": {
|
||||
"r1": _repo(forge="g"),
|
||||
"r2": {
|
||||
"url": "ssh://git@h/x/other.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
"forge": "g",
|
||||
},
|
||||
}}}},
|
||||
"agents": {"claude": {"bottle": "bb", "prompt": "",
|
||||
"forge-accounts": {"g": GITEA}}},
|
||||
})
|
||||
m = idx.load_for_agent("claude", ())
|
||||
routes = egress_forge_routes(m.forge_associations)
|
||||
self.assertEqual(1, len(routes))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# generated prompt guidance
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeGuidance(unittest.TestCase):
|
||||
def _assoc(self):
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
return idx.load_for_agent("claude", ()).forge_associations
|
||||
|
||||
def test_empty_when_no_associations(self) -> None:
|
||||
self.assertEqual("", render_forge_guidance(()))
|
||||
|
||||
def test_names_account_repo_and_workflow(self) -> None:
|
||||
text = render_forge_guidance(self._assoc())
|
||||
self.assertIn("didericis-gitea", text)
|
||||
self.assertIn("https://gitea.dideric.is/api/v1", text)
|
||||
self.assertIn("bot-bottle", text)
|
||||
# branch-backed PR workflow + AGit prohibition.
|
||||
self.assertIn("refs/heads/", text)
|
||||
self.assertIn("refs/for/*", text)
|
||||
self.assertIn("pull request", text)
|
||||
|
||||
def test_no_token_secret_name_or_value(self) -> None:
|
||||
text = render_forge_guidance(self._assoc())
|
||||
self.assertNotIn("GITEA_CLAUDE_TOKEN", text)
|
||||
|
||||
def test_prompt_file_appends_guidance(self) -> None:
|
||||
from bot_bottle.backend.resolve_common import prepare_agent_state_dir
|
||||
|
||||
idx = _index(
|
||||
author={"name": "n", "email": "e@x.is"},
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
# Give the agent a real prompt body.
|
||||
m = idx.load_for_agent("claude", ())
|
||||
from dataclasses import replace
|
||||
m = replace(m, agent=replace(m.agent, prompt="BASE PROMPT"))
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_ROOT": td}):
|
||||
_dir, prompt_file = prepare_agent_state_dir("slug1", m)
|
||||
body = Path(prompt_file).read_text()
|
||||
self.assertIn("BASE PROMPT", body)
|
||||
self.assertIn("Forge access", body)
|
||||
self.assertIn("https://gitea.dideric.is/api/v1", body)
|
||||
|
||||
def test_prompt_file_no_guidance_without_forge(self) -> None:
|
||||
from bot_bottle.backend.resolve_common import prepare_agent_state_dir
|
||||
|
||||
m = _index(author={"name": "n", "email": "e@x.is"}).load_for_agent(
|
||||
"claude", ()
|
||||
)
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_ROOT": td}):
|
||||
_dir, prompt_file = prepare_agent_state_dir("slug2", m)
|
||||
body = Path(prompt_file).read_text()
|
||||
self.assertNotIn("Forge access", body)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Unit tests for shared gateway stdlib HTTP resource boundaries."""
|
||||
# pylint: disable=protected-access
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import socket
|
||||
import unittest
|
||||
|
||||
from bot_bottle.gateway.bounded_http import (
|
||||
BodyReadError,
|
||||
BoundedThreadingHTTPServer,
|
||||
read_declared_body,
|
||||
)
|
||||
|
||||
|
||||
class _Handler:
|
||||
pass
|
||||
|
||||
|
||||
class _TimeoutStream:
|
||||
def read(self, _size: int = -1, /) -> bytes:
|
||||
raise TimeoutError
|
||||
|
||||
|
||||
class TestDeclaredBody(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.left, self.right = socket.socketpair()
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.left.close()
|
||||
self.right.close()
|
||||
|
||||
def test_rejects_incomplete_body(self) -> None:
|
||||
with self.assertRaisesRegex(BodyReadError, "incomplete"):
|
||||
read_declared_body(
|
||||
io.BytesIO(b"short"),
|
||||
self.left,
|
||||
"10",
|
||||
maximum=100,
|
||||
timeout_seconds=1,
|
||||
require_length=True,
|
||||
)
|
||||
|
||||
def test_maps_read_timeout(self) -> None:
|
||||
with self.assertRaises(BodyReadError) as caught:
|
||||
read_declared_body(
|
||||
_TimeoutStream(),
|
||||
self.left,
|
||||
"1",
|
||||
maximum=100,
|
||||
timeout_seconds=1,
|
||||
require_length=True,
|
||||
)
|
||||
self.assertEqual(408, caught.exception.status)
|
||||
|
||||
|
||||
class TestBoundedServer(unittest.TestCase):
|
||||
def test_saturated_server_rejects_without_spawning_thread(self) -> None:
|
||||
client, peer = socket.socketpair()
|
||||
with BoundedThreadingHTTPServer(
|
||||
("127.0.0.1", 0), _Handler, max_workers=1, # type: ignore[arg-type]
|
||||
) as server:
|
||||
with client, peer:
|
||||
# Directly reserve the only slot to model an in-flight handler.
|
||||
self.assertTrue(
|
||||
server._request_slots.acquire( # pylint: disable=consider-using-with
|
||||
blocking=False,
|
||||
),
|
||||
)
|
||||
try:
|
||||
server.process_request(client, ("127.0.0.1", 1))
|
||||
self.assertIn(b"503 Service Unavailable", peer.recv(1024))
|
||||
finally:
|
||||
server._request_slots.release()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -23,6 +23,7 @@ from bot_bottle.gateway.bootstrap import (
|
||||
_DaemonManager,
|
||||
_argv_for_daemon,
|
||||
_env_for_daemon,
|
||||
_pump,
|
||||
_selected_daemons,
|
||||
)
|
||||
from tests._bin import SLEEP
|
||||
@@ -565,5 +566,26 @@ class TestMainEndToEnd(unittest.TestCase):
|
||||
self.assertIn("no daemons selected", out)
|
||||
|
||||
|
||||
class _FailingStream:
|
||||
def __init__(self, *, closed: bool) -> None:
|
||||
self.closed = closed
|
||||
|
||||
def readline(self) -> bytes:
|
||||
raise ValueError("I/O operation on closed file")
|
||||
|
||||
|
||||
class TestOutputPump(unittest.TestCase):
|
||||
def test_closed_stream_race_is_normal_completion(self) -> None:
|
||||
with patch("bot_bottle.gateway.bootstrap._log") as log:
|
||||
_pump("egress", _FailingStream(closed=True)) # type: ignore[arg-type]
|
||||
log.assert_not_called()
|
||||
|
||||
def test_unexpected_io_failure_is_logged(self) -> None:
|
||||
with patch("bot_bottle.gateway.bootstrap._log") as log:
|
||||
_pump("egress", _FailingStream(closed=False)) # type: ignore[arg-type]
|
||||
log.assert_called_once()
|
||||
self.assertIn("output pump stopped", log.call_args.args[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user