Compare commits

..

6 Commits

Author SHA1 Message Date
didericis bd663196dc docs: reposition README around provider-neutral secure substrate
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 17s
Lead with the agnostic + security story instead of the single-user
security framing. New hero positions bot-bottle as a neutral control
plane that runs any agent (Claude, Codex, or a drop-in contrib plugin)
inside an isolation boundary the agent can't touch.

Restructure Features into three pillars — neutral substrate, isolation
boundary, host-matched isolation — promoting provider-agnosticism (PRD
0053 user plugins) from a buried bullet to a headline. No capability
claims changed; per-provider auth/image detail preserved as a note
linking to Manifest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-24 01:21:05 -04:00
didericis-codex 6b0de88be6 docs: activate install script prd
lint / lint (push) Successful in 1m39s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
2026-06-23 21:47:12 -04:00
didericis-codex 9a941e59be feat: add install script packaging 2026-06-23 21:47:12 -04:00
didericis d7a3539755 ci(prd): rename PRD to prd-new placeholder per new convention 2026-06-23 21:46:44 -04:00
didericis cfe57a50d0 docs(prd): renumber PRD 0054 → 0057 (0054 slot taken by named-labelled-agents) 2026-06-23 21:46:44 -04:00
didericis e5d551861c docs(prd): PRD 0054 - install script 2026-06-23 21:46:44 -04:00
150 changed files with 2305 additions and 12584 deletions
-18
View File
@@ -1,18 +0,0 @@
[run]
branch = True
source = .
[report]
# Coverage policy: see docs/decisions/0004-coverage-policy.md.
#
# `omit` is reserved for genuinely interactive entry-point shells whose
# bodies are `read_tty_line()` / curses prompt loops — there is no
# behaviour to assert that a test wouldn't have to fake wholesale, so a
# test here would inflate the number without buying confidence. This is
# NOT a place to hide subprocess/backend orchestration: that code is
# security-relevant and is measured via the integration suite instead
# (run scripts/coverage.sh for the combined unit+integration number).
omit =
bot_bottle/cli/tui.py
bot_bottle/cli/init.py
tests/*
+1 -36
View File
@@ -39,14 +39,8 @@ jobs:
with:
python-version: "3.12"
- name: Install dev requirements
run: python3 -m pip install -r requirements-dev.txt
- name: Run unit tests
run: python3 -m coverage run -m unittest discover -t . -s tests/unit -v
- name: Report unit coverage
run: python3 -m coverage report -m
run: python3 -m unittest discover -t . -s tests/unit -v
integration:
runs-on: ubuntu-latest
@@ -70,32 +64,3 @@ jobs:
- name: Run integration tests
run: python3 -m unittest discover -t . -s tests/integration -v
# Combined unit+integration coverage + the diff-coverage gate.
# See docs/decisions/0004-coverage-policy.md. The hard gate is diff
# coverage (new/changed lines >= 90%); the combined + critical reports
# are informational and degrade gracefully when the runner has no
# Docker (integration tests skip, those modules just read lower).
coverage:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dev requirements
run: python3 -m pip install -r requirements-dev.txt
- name: Combined coverage (unit + integration)
run: PYTHON=python3 bash scripts/coverage.sh critical
- name: Diff-coverage gate (changed lines >= 90%)
run: |
git fetch --no-tags origin main:refs/remotes/origin/main
python3 scripts/diff_coverage.py --base origin/main --min 90
+24 -26
View File
@@ -6,9 +6,8 @@ on:
- main
paths:
- '**.py'
- '.coveragerc'
# The core-coverage badge reads this list; refresh when it changes.
- 'scripts/critical-modules.txt'
- '.pylintrc'
- 'pyrightconfig.json'
workflow_dispatch:
jobs:
@@ -30,39 +29,38 @@ jobs:
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Run coverage and extract percentage
id: coverage
- name: Run pylint and extract score
id: pylint
run: |
python -m coverage run -m unittest discover -t . -s tests/unit > /dev/null 2>&1 || true
PERCENT=$(python -m coverage report 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Coverage: $PERCENT%"
PYLINT_OUTPUT=$(python -m pylint bot_bottle/ 2>&1) || true
SCORE=$(echo "$PYLINT_OUTPUT" | grep -oP '(?<=rated at )\d+\.\d+/10' | head -1)
echo "score=$SCORE" >> $GITHUB_OUTPUT
echo "Pylint score: $SCORE"
- name: Extract core (critical-module) coverage percentage
id: core_coverage
- name: Run pyright and check errors
id: pyright
run: |
# Reuses the .coverage data from the previous step. The core list is
# the single source of truth in scripts/critical-modules.txt; every
# core module is unit-tested, so the unit-only run is accurate for it.
INCLUDE=$(grep -vE '^[[:space:]]*(#|$)' scripts/critical-modules.txt | paste -sd, -)
PERCENT=$(python -m coverage report --include="$INCLUDE" 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Core coverage: $PERCENT%"
PYRIGHT_OUTPUT=$(python -m pyright 2>&1) || true
ERRORS=$(echo "$PYRIGHT_OUTPUT" | grep -oP '\d+(?= error)' | head -1)
echo "errors=$ERRORS" >> $GITHUB_OUTPUT
echo "Pyright errors: $ERRORS"
- name: Update badges in README
run: |
COVERAGE_PERCENT="${{ steps.coverage.outputs.percent }}"
CORE_COVERAGE_PERCENT="${{ steps.core_coverage.outputs.percent }}"
PYLINT_SCORE="${{ steps.pylint.outputs.score }}"
PYRIGHT_ERRORS="${{ steps.pyright.outputs.errors }}"
if [ -n "$COVERAGE_PERCENT" ]; then
sed -i "s|/badge/coverage-[^)]*|/badge/coverage-${COVERAGE_PERCENT}%25-brightgreen|" README.md
PYLINT_SCORE_ENCODED=$(echo "$PYLINT_SCORE" | sed 's|/|%2F|g')
if [ -n "$PYLINT_SCORE_ENCODED" ]; then
sed -i "s|/badge/pylint-[^)]*|/badge/pylint-${PYLINT_SCORE_ENCODED}-brightgreen|" README.md
fi
if [ -n "$CORE_COVERAGE_PERCENT" ]; then
sed -i "s|/badge/core%20coverage-[^)]*|/badge/core%20coverage-${CORE_COVERAGE_PERCENT}%25-brightgreen|" README.md
if [ -n "$PYRIGHT_ERRORS" ]; then
sed -i "s|/badge/pyright-[^)]*|/badge/pyright-${PYRIGHT_ERRORS}%20errors-brightgreen|" README.md
fi
echo "Updated badges:"
grep -E "coverage" README.md | head -2
grep -E "pylint|pyright" README.md | head -2
- name: Commit and push badge updates
run: |
@@ -75,7 +73,7 @@ jobs:
else
echo "Badge changes detected, committing..."
git add README.md
MSG="chore: update quality badges"$'\n\n'"- Coverage: ${{ steps.coverage.outputs.percent }}%"$'\n'"- Core coverage: ${{ steps.core_coverage.outputs.percent }}%"$'\n\n'"[skip ci]"
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n\n'"[skip ci]"
git commit -m "$MSG"
git push
fi
-1
View File
@@ -22,4 +22,3 @@ venv/
.pytest_cache/
.mypy_cache/
.ruff_cache/
.coverage
-1
View File
@@ -62,7 +62,6 @@ COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
# top-level siblings (absolute imports), matching the prior
# Dockerfile.egress / Dockerfile.supervise layout.
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
COPY bot_bottle/egress_dlp_config.py /app/egress_dlp_config.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
+43 -11
View File
@@ -5,29 +5,43 @@
# bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-83%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-95%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
[![pylint](https://img.shields.io/badge/pylint-9.93%2F10-brightgreen)](https://github.com/PyCQA/pylint)
[![pyright](https://img.shields.io/badge/pyright-0%20errors-brightgreen)](https://github.com/microsoft/pyright)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
**Run any coding agent like it might be compromised — and lose nothing when it is.**
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
bot-bottle is a provider-neutral, security-first substrate for autonomous agents. Bring Claude Code, Codex, or your own harness; each one runs in an ephemeral, per-agent "bottle" it cannot modify, where every byte of egress is scanned for exfiltration and capabilities are narrowed to exactly what the task declares.
## Features
**Problem:** You want to let a coding agent run unsupervised, but a prompt-injected or misbehaving agent — or a poisoned repo, MCP server, or skill — can wreck your environment or exfiltrate your secrets. Locking yourself to one vendor's cloud doesn't fix that; it just moves the blast radius.
**Solution:** A neutral control plane that runs *whatever agent you choose* inside an isolation boundary the agent can't touch: TLS-bumped egress allowlisting, outbound/inbound DLP, gitleaks-gated pushes, and host secrets the agent never sees. Swap the agent; keep the guarantees.
## Why bot-bottle
### A neutral substrate — bring your own agent
- **Provider-agnostic by design** — Claude and Codex ship built in; any other agent (Gemini, Aider, a local-model wrapper) is a drop-in plugin at `~/.bot-bottle/contrib/<name>/` — no fork, no PR against this repo. The manifest accepts any provider template, and the isolation, egress, and git guarantees are identical across all of them.
- **One control plane, every harness** — the same bottle, egress policy, and supervise flow wrap whichever agent you run, so switching or mixing providers doesn't change your security posture.
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
### An isolation boundary the agent can't touch
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
### Isolation that matches your host
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
Per-provider auth (Claude long-lived OAuth token; Codex opt-in host device-auth forwarding) and per-provider images (`Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile) are configured on the bottle — see [Manifest](#manifest).
## Architecture
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
@@ -69,6 +83,27 @@ The Docker topology looks like this:
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
## Install
Install the CLI with the bootstrap script:
```sh
curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
```
The script checks Python 3.11+, checks Docker daemon reachability, creates the `~/.bot-bottle/` config directories, installs the Python package with `pipx` when available or `pip --user` otherwise, then runs:
```sh
bot-bottle doctor
```
Python-native installers can use the package metadata directly:
```sh
pipx install git+https://gitea.dideric.is/didericis/bot-bottle.git
uv tool install git+https://gitea.dideric.is/didericis/bot-bottle.git
```
## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
@@ -149,11 +184,8 @@ You help maintain Gitea-hosted projects.
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default for manifest routes — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). Agent-provider routes (e.g. `api.anthropic.com`) default to `redact`. |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Trademarks
+96
View File
@@ -0,0 +1,96 @@
# Per-bottle sidecar bundle image (PRD 0024).
#
# Collapses the prior per-sidecar images (egress, git-gate,
# supervise) into one. A small stdlib-Python init supervisor at
# /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
# propagates per-daemon stdout/stderr to the container log with a
# `[name]` prefix. See PRD 0024 for the rationale.
#
# Layout:
#
# /usr/bin/gitleaks gitleaks binary
# /app/egress_addon.py + siblings mitmproxy addon (egress)
# /app/egress-entrypoint.sh mitmdump launcher
# /app/supervise_server.py + .py supervise MCP server
# /app/sidecar_init.py PID 1 supervisor
# /etc/egress/routes.yaml bind-mounted at run time
# /etc/git-gate/pre-receive docker-cp'd at start time
# /git-gate-entrypoint.sh docker-cp'd at start time
# /git-gate/creds/* docker-cp'd at start time
# /git/* bare repos, populated at runtime
# /run/supervise/queue/ bind-mounted at run time
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
#
# Exposed ports inside the container:
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
# 9418 git-gate (git-daemon)
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
# 9100 supervise (MCP HTTP)
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
# Python + mitmdump pre-installed — heavier than the others, so
# this stage starts there and pulls the standalone binaries in.
FROM mitmproxy/mitmproxy:11.1.3
# Run as root inside the bundle. The bundle is the isolation
# boundary; per-daemon user separation inside it is not load-bearing
# and complicates the supervisor's spawn path.
USER root
# Runtime system deps:
# git supplies the `git daemon` subcommand (no separate package)
# plus the core `git` binary the pre-receive hook invokes.
# openssh-client supplies the upstream SSH transport the
# pre-receive hook uses to forward accepted refs.
# ca-certificates is needed for mitmdump upstream TLS (the
# base image already has it; listed for explicitness).
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git openssh-client ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# Pull the standalone binaries into the final image.
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
# Project Python: addon + server modules + the init supervisor.
# Kept flat under /app/ so mitmdump's loader resolves them as
# top-level siblings (absolute imports), matching the prior
# Dockerfile.egress / Dockerfile.supervise layout.
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
RUN chmod +x /app/egress-entrypoint.sh
# Pre-create runtime directories the compose renderer + start
# step expect to exist. `docker cp` does not create intermediate
# dirs, and bind mounts won't either if the parent is missing.
RUN mkdir -p \
/etc/egress \
/etc/git-gate \
/git-gate/creds \
/git \
/run/supervise/queue \
/home/mitmproxy/.mitmproxy
# Documentation only — the compose renderer publishes whichever
# subset the bottle uses.
EXPOSE 8888 9099 9418 9420 9100
# WORKDIR matches Dockerfile.supervise's prior layout so the
# in-app same-dir import in supervise_server.py stays deterministic.
WORKDIR /app
# PID 1 is the supervisor. It owns signal handling and exit-code
# propagation; no `exec` chain in the entrypoint itself.
ENTRYPOINT ["python3", "/app/sidecar_init.py"]
+2 -19
View File
@@ -61,6 +61,7 @@ class AgentProviderRuntime:
prompt_mode: PromptMode
bypass_args: tuple[str, ...]
resume_args: tuple[str, ...]
remote_control_args: tuple[str, ...]
@dataclass(frozen=True)
@@ -209,15 +210,6 @@ class AgentProvider(ABC):
the supervise sidecar is reachable. No-op when
`plan.supervise_plan is None`."""
@abstractmethod
def headless_prompt(self, prompt: str) -> list[str]:
"""Return the agent CLI args that deliver `prompt` as the
initial task in a non-interactive (headless) session.
Called only when ``--prompt`` is passed to
``./cli.py start --headless``; the returned args are appended
after the provider's ``bypass_args`` and ``startup_args``."""
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
"""Install the egress MITM CA into the agent's trust store.
@@ -379,15 +371,6 @@ def build_agent_provision_plan(
)
def provider_startup_args(
provider_settings: dict[str, object] | None,
) -> tuple[str, ...]:
raw = (provider_settings or {}).get("startup_args", ())
if not isinstance(raw, (list, tuple)):
return ()
return tuple(arg for arg in raw if isinstance(arg, str))
def prompt_args(
prompt_mode: PromptMode,
prompt_path: str | None,
@@ -399,7 +382,7 @@ def prompt_args(
if prompt_mode == "append_file":
return ["--append-system-prompt-file", prompt_path]
if prompt_mode == "read_prompt_file":
if argv and ("resume" in argv or "remote-control" in argv):
if argv and "resume" in argv:
return []
return [f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "print_read_prompt_file":
+5 -14
View File
@@ -72,9 +72,6 @@ class BottleSpec:
identity: str = ""
label: str = ""
color: str = ""
# Ordered bottle names selected at launch (issue #269). When non-empty
# they are merged in order and replace the agent's `bottle:` field.
bottle_names: tuple[str, ...] = ()
@dataclass(frozen=True)
@@ -112,8 +109,9 @@ class BottlePlan(ABC):
def workspace_plan(self) -> WorkspacePlan:
return workspace_plan(self.spec, guest_home=self.guest_home)
def print(self) -> None:
def print(self, *, remote_control: bool) -> None:
"""Render the y/N preflight summary to stderr."""
del remote_control
spec = self.spec
manifest = self.manifest
agent = manifest.agent
@@ -132,11 +130,7 @@ class BottlePlan(ABC):
info(f"provider : {self.agent_provision.template}")
print_multi("env ", env_names)
print_multi("skills ", list(agent.skills))
effective_bottles = (
list(spec.bottle_names) if spec.bottle_names
else ([agent.bottle] if agent.bottle else [])
)
print_multi("bottle ", effective_bottles)
info(f"bottle : {agent.bottle}")
identity = manifest.git_identity_summary()
if identity:
@@ -370,7 +364,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
Returns the loaded Manifest for the selected agent. Subclasses with
additional preconditions should override and call
`super()._validate(spec)` first."""
manifest = spec.manifest.load_for_agent(spec.agent_name, spec.bottle_names)
manifest = spec.manifest.load_for_agent(spec.agent_name)
self._validate_skills(manifest.agent.skills)
self._validate_agent_provider_dockerfile(spec, manifest)
return manifest
@@ -396,12 +390,9 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
if not path.is_absolute():
path = Path(spec.user_cwd) / path
if not path.is_file():
effective = (
", ".join(spec.bottle_names) if spec.bottle_names else manifest.agent.bottle
)
die(
f"agent_provider.dockerfile for bottle "
f"'{effective}' not found: {path}"
f"'{manifest.agent.bottle}' not found: {path}"
)
@abstractmethod
@@ -0,0 +1,211 @@
"""capability_apply — host-side orchestrator for capability-block
remediation (PRD 0016).
On approval of a capability-block proposal, the dashboard calls
apply_capability_change(slug, new_dockerfile) which:
1. Snapshots the agent's transcript dir to
~/.bot-bottle/state/<slug>/transcript/ (best-effort).
2. Pushes the agent's working tree via `git push` (best-effort —
no upstream / no commits / no git repo all skip with a log).
3. Writes the new Dockerfile to
~/.bot-bottle/state/<slug>/Dockerfile (PRD 0016 Phase 1
state). The next `cli.py start <agent>` picks it up.
4. Force-removes the agent container + all sidecars + the
per-bottle networks. Idempotent — missing resources are not
errors.
Returns (before, after) Dockerfile contents so the dashboard can
record / render the diff. (capability-block has no audit log per
PRD 0013 — the per-bottle Dockerfile state is its own record.)
This is "fire-and-forget" from the agent's perspective: by the time
the dashboard writes the response file the supervise sidecar is
gone, so the agent's tool call connection drops without ever
receiving the response. The replacement agent (next manual
`cli.py start`) sees the new Dockerfile and starts from there.
v1 does not auto-relaunch — see PRD 0016's capability-block return
semantics open question.
"""
from __future__ import annotations
import shutil
import subprocess
from ...agent_provider import get_provider
from ...log import info, warn
from ...bottle_state import (
mark_preserved,
per_bottle_dockerfile,
transcript_snapshot_dir,
write_per_bottle_dockerfile,
)
from .sidecar_bundle import sidecar_bundle_container_name
# Agent home inside the container (per the repo Dockerfile's
# `USER node` + `WORKDIR /home/node`). Used to locate the transcript
# dir + the workspace dir for git push.
_AGENT_HOME_IN_CONTAINER = "/home/node"
_AGENT_TRANSCRIPT_IN_CONTAINER = f"{_AGENT_HOME_IN_CONTAINER}/.claude"
_AGENT_WORKSPACE_IN_CONTAINER = f"{_AGENT_HOME_IN_CONTAINER}/workspace"
# Per-bottle resource name patterns (mirroring prepare.py).
def _agent_container_name(slug: str) -> str:
return f"bot-bottle-{slug}"
def _per_bottle_container_names(slug: str) -> list[str]:
"""All container names that belong to this bottle. Missing
containers are silently skipped by the teardown helper, so it's
fine to include names that don't exist for a given bottle."""
return [
_agent_container_name(slug),
sidecar_bundle_container_name(slug),
]
def _per_bottle_network_names(slug: str) -> list[str]:
return [
f"bot-bottle-net-{slug}",
f"bot-bottle-egress-{slug}",
]
class CapabilityApplyError(RuntimeError):
"""Raised when the apply fails in a way that should keep the
proposal pending (so the operator can retry). Best-effort
failures (transcript snapshot, git push) do not raise — they
just log and proceed."""
# --- Public helpers --------------------------------------------------------
def fetch_current_dockerfile(slug: str) -> str:
"""Return the Dockerfile content the next `cli.py start <agent>`
would use for this bottle. If a per-bottle override exists, that
one; otherwise the repo's Dockerfile.
Used by the operator-edit verb to show the current source of
truth, and by apply_capability_change for the before-diff."""
override = per_bottle_dockerfile(slug)
if override is not None:
return override
repo_dockerfile = get_provider("claude").dockerfile
if repo_dockerfile.is_file():
return repo_dockerfile.read_text()
raise CapabilityApplyError(
f"no per-bottle Dockerfile for {slug} and no provider Dockerfile at "
f"{repo_dockerfile}"
)
def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
"""End-to-end capability-block remediation. See module docstring
for the sequence. Returns (before, after) Dockerfile content."""
if not new_dockerfile.strip():
raise CapabilityApplyError("proposed Dockerfile is empty")
before = fetch_current_dockerfile(slug)
snapshot_transcript(slug)
_push_working_tree(slug)
write_per_bottle_dockerfile(slug, new_dockerfile)
# Set the preserve marker BEFORE teardown so cli.py's session-end
# cleanup sees it and keeps the state dir intact for the
# operator's `cli.py resume <identity>`. Without the marker the
# state dir would be deleted as part of normal session end.
mark_preserved(slug)
_teardown_bottle(slug)
return before, new_dockerfile
# --- Internals -------------------------------------------------------------
def snapshot_transcript(slug: str) -> None:
"""`docker cp` /home/node/.claude out of the agent container into
~/.bot-bottle/state/<slug>/transcript/. Best-effort: missing
container, missing dir, or cp error all log a warning and return.
The transcript is what `claude --resume` reads to pick up where
the agent left off.
Called from two places:
- capability-apply, before tearing the bottle down.
- cli.py's session-end path, before the launch context closes,
so a crash or normal exit also leaves a transcript on disk
(deleted along with the state dir on clean exit, kept on
crash or capability-block per the preserve marker)."""
container = _agent_container_name(slug)
dest = transcript_snapshot_dir(slug)
if dest.exists():
# Remove any prior snapshot so the new one is a clean copy.
shutil.rmtree(dest, ignore_errors=True)
dest.parent.mkdir(parents=True, exist_ok=True)
r = subprocess.run(
["docker", "cp", f"{container}:{_AGENT_TRANSCRIPT_IN_CONTAINER}", str(dest)],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"transcript snapshot skipped "
f"({(r.stderr or '').strip() or 'no transcript dir in container?'})"
)
return
info(f"transcript snapshotted to {dest}")
def _push_working_tree(slug: str) -> None:
"""`docker exec <agent> git push` from /home/node/workspace.
Best-effort: not-a-git-repo, no upstream, nothing-to-push, no
network all log a warning and return. The replacement bottle
will pick up whatever's actually upstream."""
container = _agent_container_name(slug)
r = subprocess.run(
[
"docker", "exec", container, "sh", "-c",
f"cd {_AGENT_WORKSPACE_IN_CONTAINER} && "
f"git rev-parse --is-inside-work-tree >/dev/null 2>&1 && "
f"git push origin HEAD 2>&1 || true",
],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"capability-apply: git push skipped "
f"({(r.stderr or '').strip() or 'docker exec failed'})"
)
return
output = (r.stdout or "").strip()
if output:
info(f"capability-apply: git push: {output}")
else:
info("capability-apply: git push ran (no output — likely not a git workspace)")
def _teardown_bottle(slug: str) -> None:
"""Force-remove all per-bottle docker resources. Idempotent —
`docker rm -f` / `docker network rm` silently ignore missing
names, so this can be called even mid-rebuild."""
info(f"capability-apply: tearing down bottle {slug}")
for name in _per_bottle_container_names(slug):
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
for net in _per_bottle_network_names(slug):
subprocess.run(
["docker", "network", "rm", net],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
__all__ = [
"CapabilityApplyError",
"apply_capability_change",
"fetch_current_dockerfile",
"snapshot_transcript",
]
+21 -6
View File
@@ -28,12 +28,11 @@ from typing import Any
from ...egress import (
EGRESS_HOSTNAME,
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_sidecar_env_entries,
)
from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn
from ...supervise import (
CURRENT_CONFIG_DIR_IN_AGENT,
QUEUE_DIR_IN_CONTAINER,
SUPERVISE_HOSTNAME,
SUPERVISE_PORT,
@@ -59,10 +58,17 @@ from .sidecar_bundle import (
)
# Repo root, used as the build context for the bundle Dockerfile.
# Repo root or installed site-packages root, used as the build context for
# Dockerfiles that COPY bot_bottle source files.
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def _sidecar_bundle_dockerfile() -> str:
if (Path(_REPO_DIR) / SIDECAR_BUNDLE_DOCKERFILE).is_file():
return SIDECAR_BUNDLE_DOCKERFILE
return f"bot_bottle/{SIDECAR_BUNDLE_DOCKERFILE}"
def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
"""Render a Compose v2 spec dict from a fully-resolved
DockerBottlePlan.
@@ -136,7 +142,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
if ep.routes:
volumes.append(_bind(ep.routes_path.parent, str(Path(EGRESS_ROUTES_IN_CONTAINER).parent)))
env.extend(egress_sidecar_env_entries(ep))
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
# --- git-gate -----------------------------------------------------
gp = plan.git_gate_plan
@@ -183,7 +190,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"image": SIDECAR_BUNDLE_IMAGE,
"build": {
"context": _REPO_DIR,
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
"dockerfile": _sidecar_bundle_dockerfile(),
},
"container_name": sidecar_bundle_container_name(plan.slug),
"networks": {
@@ -220,7 +227,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
# never lands on argv or in the compose file.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
service: dict[str, Any] = {
"image": plan.image,
@@ -232,6 +238,15 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
if plan.use_runsc:
service["runtime"] = "runsc"
volumes: list[dict[str, Any]] = []
if plan.supervise_plan is not None:
volumes.append(_bind(
plan.supervise_plan.current_config_dir,
CURRENT_CONFIG_DIR_IN_AGENT,
))
if volumes:
service["volumes"] = volumes
# The init supervisor inside the bundle owns intra-bundle
# daemon ordering, so the agent only waits for the bundle
# container itself.
+1 -9
View File
@@ -37,10 +37,7 @@ from pathlib import Path
from typing import Callable, Generator
from ...egress import egress_resolve_token_values
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import info, warn
from . import network as network_mod
from . import util as docker_mod
@@ -121,11 +118,6 @@ def launch(
git_gate_plan = plan.git_gate_plan
if git_gate_plan.upstreams:
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
git_gate_plan,
git_gate_state_dir(plan.slug),
)
git_gate_plan = dataclasses.replace(
git_gate_plan,
internal_network=internal_network,
+4 -3
View File
@@ -12,9 +12,10 @@ from __future__ import annotations
import os
# Bundle image. Defaults to a built-locally tag (built from the
# repo's Dockerfile.sidecars via compose `build:`). Operators
# pinning to a published digest can override via env.
# Bundle image. Defaults to a built-locally tag. Source checkouts
# build from the repo-root Dockerfile.sidecars; installed packages
# build from the packaged copy under bot_bottle/.
# Operators pinning to a published digest can override via env.
SIDECAR_BUNDLE_IMAGE = os.environ.get(
"BOT_BOTTLE_SIDECAR_IMAGE",
"bot-bottle-sidecars:latest",
+2 -6
View File
@@ -11,7 +11,7 @@ from pathlib import Path
from ..bottle_state import egress_state_dir
from ..egress import EGRESS_ROUTES_FILENAME
from ..egress_addon_core import LOG_OFF, load_config
from ..egress_addon_core import load_routes
class EgressApplyError(RuntimeError):
@@ -33,15 +33,11 @@ class EgressApplicator(ABC):
@staticmethod
def validate_routes_content(content: str) -> None:
try:
config = load_config(content)
load_routes(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
if config.log != LOG_OFF:
raise EgressApplyError(
"proposed routes.yaml must not change egress logging"
)
@staticmethod
def _routes_path(slug: str) -> Path:
+5 -27
View File
@@ -22,16 +22,8 @@ from ...bottle_state import (
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import die, info, warn
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
@@ -101,8 +93,6 @@ def launch(
egress_network = egress_network_name(plan.slug)
_create_networks(internal_network, egress_network, stack)
plan = _provision_git_gate_keys(plan)
sidecar_name = sidecar_container_name(plan.slug)
container_mod.force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
@@ -246,19 +236,6 @@ def _stamp_agent_urls(
)
def _provision_git_gate_keys(
plan: MacosContainerBottlePlan,
) -> MacosContainerBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
@@ -373,7 +350,9 @@ def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
env: list[str] = []
if plan.egress_plan.routes:
env.extend(sorted(plan.egress_plan.token_env_map.keys()))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
@@ -441,7 +420,6 @@ def _agent_env_entries(
env.append(f"{name}={value}")
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
return tuple(env)
@@ -68,11 +68,6 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
_ensure_builder_dns()
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
if dockerfile:
# `container build` resolves -f relative to the current working
# directory, not the build context. Anchor a relative Dockerfile to
# the context so builds work from any cwd.
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args.extend(["-f", dockerfile])
args.append(context)
subprocess.run(args, check=True)
-1
View File
@@ -63,7 +63,6 @@ def write_launch_metadata(
backend=backend,
label=spec.label,
color=spec.color,
bottle_names=spec.bottle_names,
))
+6 -24
View File
@@ -23,9 +23,7 @@ from typing import Callable, Generator
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
@@ -41,10 +39,7 @@ from ..docker.git_gate import (
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import info, warn
from ...bottle_state import (
egress_state_dir,
@@ -177,7 +172,6 @@ def _start_bundle(
) -> SmolmachinesBottlePlan:
"""Build the BundleLaunchSpec, resolve token env, start the
sidecar bundle container, and register teardown."""
plan = _provision_git_gate_keys(plan)
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
token_env = _resolve_token_env(plan, dict(os.environ))
_bundle.ensure_bundle_image(bundle_spec.image)
@@ -186,19 +180,6 @@ def _start_bundle(
return plan
def _provision_git_gate_keys(
plan: SmolmachinesBottlePlan,
) -> SmolmachinesBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _discover_urls(
plan: SmolmachinesBottlePlan,
loopback_ip: str,
@@ -247,9 +228,6 @@ def _discover_urls(
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
name, value = entry.split("=", 1)
guest_env[name] = value
return dataclasses.replace(
plan,
@@ -338,7 +316,11 @@ def _bundle_launch_spec(
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
if ep.routes:
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
env.extend(egress_sidecar_env_entries(ep))
# Bare-name entries for upstream-token slots. Their values
# come from the docker-run subprocess env (inherited from
# the operator's shell), never landing on argv.
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
# --- git-gate ---------------------------------------------
gp = plan.git_gate_plan
+16 -19
View File
@@ -1,7 +1,8 @@
"""Per-bottle persistent state.
"""Per-bottle persistent state (PRD 0016).
Holds optional per-bottle Dockerfile overrides, the transcript snapshot
the state-preservation helper saves before teardown, and the launch metadata that lets
Holds the per-bottle Dockerfile override that capability-block
remediation writes, the transcript snapshot the state-preservation
helper saves before teardown, and the launch metadata that lets
`cli.py resume <identity>` reconstruct a bottle's spec. State
lives at:
@@ -60,7 +61,7 @@ _METADATA_NAME = "metadata.json"
_LIVE_CONFIG_SUBDIR = "live-config"
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
# Empty marker file. Session preservation writes it before teardown so
# Empty marker file. capability_apply writes it before teardown so
# cli.py's session-end cleanup knows to preserve the state dir for
# `cli.py resume <identity>`. Absent = clean up.
_PRESERVE_MARKER = ".preserve"
@@ -111,10 +112,6 @@ class BottleMetadata:
backend: str = ""
label: str = ""
color: str = ""
# Ordered bottle names selected at launch (issue #269). Empty tuple
# for state dirs written before this change; resume falls back to
# the agent's `bottle:` field in that case.
bottle_names: tuple[str, ...] = ()
def metadata_path(identity: str) -> Path:
@@ -142,10 +139,6 @@ def read_metadata(identity: str) -> BottleMetadata | None:
if not isinstance(raw, dict):
return None
raw_typed = cast(dict[str, object], raw)
raw_bottle_names = raw_typed.get("bottle_names", [])
bottle_names: tuple[str, ...] = ()
if isinstance(raw_bottle_names, list):
bottle_names = tuple(str(n) for n in raw_bottle_names if isinstance(n, str))
return BottleMetadata(
identity=str(raw_typed.get("identity", identity)),
agent_name=str(raw_typed.get("agent_name", "")),
@@ -156,7 +149,6 @@ def read_metadata(identity: str) -> BottleMetadata | None:
backend=str(raw_typed.get("backend", "")),
label=str(raw_typed.get("label", "")),
color=str(raw_typed.get("color", "")),
bottle_names=bottle_names,
)
@@ -172,7 +164,8 @@ def per_bottle_dockerfile_path(identity: str) -> Path:
def per_bottle_dockerfile(identity: str) -> str | None:
"""Return the per-bottle Dockerfile content if present, else
None. None means: use the provider or manifest Dockerfile."""
None. None means: use the repo's Dockerfile (the original
pre-capability-block behavior)."""
p = per_bottle_dockerfile_path(identity)
if p.is_file():
return p.read_text()
@@ -256,7 +249,9 @@ def write_live_config(
def transcript_snapshot_dir(identity: str) -> Path:
"""Where agent session snapshots are kept for resume flows."""
"""Where capability_apply stashes the agent's transcript before
teardown, so the next `cli.py start <agent>` can offer to
resume from it."""
return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR
@@ -283,7 +278,8 @@ def git_gate_state_dir(identity: str) -> Path:
def supervise_state_dir(identity: str) -> Path:
"""State subdir reserved for supervise sidecar bind-mount sources.
"""State subdir for the supervise sidecar's current-config dir
(bind-mounted into the agent at /etc/bot-bottle/current-config).
The queue dir is intentionally NOT under here — it lives at
~/.bot-bottle/queue/<slug>/ alongside the audit logs, so it
survives state-dir cleanup."""
@@ -305,8 +301,9 @@ def preserve_marker_path(identity: str) -> Path:
def mark_preserved(identity: str) -> Path:
"""Mark this bottle's state for preservation across session
teardown so cli.py's session-end cleanup leaves the state dir
intact for a subsequent `cli.py resume`."""
teardown. Written by capability_apply.apply_capability_change so
cli.py's session-end cleanup leaves the state dir intact for a
subsequent `cli.py resume`."""
path = preserve_marker_path(identity)
path.parent.mkdir(parents=True, exist_ok=True)
path.touch()
@@ -319,7 +316,7 @@ def is_preserved(identity: str) -> bool:
def clear_preserve_marker(identity: str) -> None:
"""Idempotent removal. Called at fresh launch (start or resume)
so a marker left from a prior preserved session doesn't keep
so a marker left from a prior capability-block doesn't keep
state alive past the next normal session-end."""
try:
preserve_marker_path(identity).unlink()
+4 -1
View File
@@ -1,6 +1,6 @@
"""Main CLI dispatcher.
Commands: cleanup, commit, edit, info, init, list, resume, start, supervise
Commands: cleanup, commit, doctor, edit, info, init, list, resume, start, supervise
"""
from __future__ import annotations
@@ -13,6 +13,7 @@ from ._common import PROG
from . import list as _list_mod
from .cleanup import cmd_cleanup
from .commit import cmd_commit
from .doctor import cmd_doctor
from .edit import cmd_edit
from .info import cmd_info
from .init import cmd_init
@@ -25,6 +26,7 @@ cmd_list = _list_mod.cmd_list
COMMANDS = {
"cleanup": cmd_cleanup,
"commit": cmd_commit,
"doctor": cmd_doctor,
"edit": cmd_edit,
"info": cmd_info,
"init": cmd_init,
@@ -40,6 +42,7 @@ def usage() -> None:
sys.stderr.write("Commands:\n")
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
sys.stderr.write(" doctor check Python, Docker, and bot-bottle config prerequisites\n")
sys.stderr.write(" edit open an agent in vim for editing\n")
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
+1 -1
View File
@@ -6,7 +6,7 @@ import os
import sys
from pathlib import Path
PROG = "cli.py"
PROG = Path(sys.argv[0]).name or "bot-bottle"
USER_CWD = os.getcwd()
REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
+3 -2
View File
@@ -13,8 +13,9 @@ dirs are shared layout, so docker is the single owner of that
bucket.
State dirs with `.preserve` are intentionally never touched — they
hold preserved sessions the operator may want to `resume`. Manual
`rm -rf ~/.bot-bottle/state/<identity>` is the path for those.
hold capability-block rebuilds or crash snapshots the operator may
want to `resume`. Manual `rm -rf ~/.bot-bottle/state/<identity>`
is the path for those.
"""
from __future__ import annotations
+73
View File
@@ -0,0 +1,73 @@
"""doctor: validate host prerequisites for running bot-bottle."""
from __future__ import annotations
import argparse
import shutil
import subprocess
import sys
from pathlib import Path
from ._common import PROG
def _ok(label: str, detail: str) -> None:
print(f"ok: {label}: {detail}")
def _fail(label: str, detail: str) -> None:
print(f"fail: {label}: {detail}")
def _check_python() -> bool:
version = sys.version_info
detail = f"{version.major}.{version.minor}.{version.micro}"
if version >= (3, 11):
_ok("python", detail)
return True
_fail("python", f"{detail}; need 3.11 or newer")
return False
def _check_docker() -> bool:
docker = shutil.which("docker")
if not docker:
_fail("docker", "docker command not found")
return False
try:
result = subprocess.run(
[docker, "info"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
timeout=10,
)
except (OSError, subprocess.TimeoutExpired) as exc:
_fail("docker", f"daemon check failed: {exc}")
return False
if result.returncode == 0:
_ok("docker", "daemon reachable")
return True
_fail("docker", "daemon not reachable")
return False
def _check_config_dir() -> bool:
config = Path.home() / ".bot-bottle"
if config.is_dir():
_ok("config", str(config))
return True
_fail("config", f"{config} does not exist")
return False
def cmd_doctor(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} doctor", add_help=True)
parser.parse_args(argv)
checks = (
_check_python(),
_check_docker(),
_check_config_dir(),
)
return 0 if all(checks) else 1
+7 -5
View File
@@ -4,12 +4,13 @@ Reads ~/.bot-bottle/state/<identity>/metadata.json to recover the
(agent_name, cwd, copy_cwd) the bottle was originally started with,
then runs the same launch core as `start` — but pinned to the
recorded identity so the new bottle picks up any per-bottle Dockerfile
override and transcript snapshot under the same state dir.
(from capability-block apply) and transcript snapshot under the same
state dir.
Use case: an interrupted or preserved bottle needs to be relaunched;
the operator runs
Use case: an agent calls capability-block, the dashboard approves
and tears down the bottle, the operator runs
./cli.py resume <identity>
to bring up the replacement from the recorded state.
to bring up the replacement with the new capabilities baked in.
"""
from __future__ import annotations
@@ -27,6 +28,7 @@ from .start import _launch_bottle
def cmd_resume(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} resume", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"identity",
help="bottle identity from a prior `start` (see its session-end output)",
@@ -49,11 +51,11 @@ def cmd_resume(argv: list[str]) -> int:
copy_cwd=metadata.copy_cwd,
user_cwd=metadata.cwd or USER_CWD,
identity=metadata.identity,
bottle_names=tuple(metadata.bottle_names),
)
backend_name = metadata.backend or None
return _launch_bottle(
spec,
dry_run=args.dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
+24 -300
View File
@@ -2,11 +2,6 @@
interactive claude-code session. The container is torn down when the
session ends.
`--headless` selects a non-interactive launch (agent/bottles/label from
flags, no TUI selectors, no y/N prompt) for orchestrators,
CI, and webhook dispatch. The agent still execs on the inherited
stdio/PTY, so an orchestrator that allocates the PTY drives the session.
The launch core is shared with `cli.py resume <identity>` through
the private orchestrator `_launch_bottle`.
"""
@@ -21,7 +16,7 @@ import tempfile
from pathlib import Path
from typing import Callable
from ..agent_provider import get_provider, runtime_for
from ..agent_provider import runtime_for
from ..backend import (
Bottle,
BottleSpec,
@@ -36,8 +31,9 @@ from ..bottle_state import (
is_preserved,
mark_preserved,
)
from ..log import info, die
from ..manifest import Manifest, ManifestIndex
# from ..backend.docker.capability_apply import snapshot_transcript
from ..log import info
from ..manifest import ManifestIndex
from ._common import PROG, USER_CWD, read_tty_line
from . import tui
@@ -46,6 +42,7 @@ def cmd_start(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"--backend",
choices=known_backend_names(),
@@ -55,39 +52,6 @@ def cmd_start(argv: list[str]) -> int:
"or host auto-selection). Overrides the env var when set."
),
)
parser.add_argument(
"--headless",
action="store_true",
help=(
"non-interactive launch: take agent/bottles/label from flags, "
"skip all prompts. For orchestrators, CI, and webhooks."
),
)
parser.add_argument(
"--bottle",
action="append",
default=None,
metavar="NAME",
help=(
"bottle to compose, repeatable (order = merge order). In "
"--headless, defaults to the agent's own bottle when omitted."
),
)
parser.add_argument(
"--label",
default=None,
help="bottle label / terminal title (--headless default: agent name)",
)
parser.add_argument(
"--color",
default=None,
help="bottle color, one of the 16 ANSI color names (--headless default: none)",
)
parser.add_argument(
"--prompt",
default=None,
help="initial task prompt delivered to the agent (required with --headless)",
)
parser.add_argument(
"name",
nargs="?",
@@ -99,12 +63,6 @@ def cmd_start(argv: list[str]) -> int:
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
manifest = ManifestIndex.resolve(USER_CWD)
backend_name: str | None = args.backend
if args.headless:
return _start_headless(
manifest, args, dry_run=dry_run, backend_name=backend_name
)
agent_name: str | None = args.name
if agent_name is None:
@@ -115,22 +73,7 @@ def cmd_start(argv: list[str]) -> int:
if agent_name is None:
return 0
# Bottle multiselect: always show after agent selection so operators
# can compose bottles at launch time without editing agent manifests.
available_bottles = manifest.all_bottle_names
lineage_map = _bottle_lineage(manifest)
display_labels = [lineage_map.get(n, n) for n in available_bottles]
label_to_name = {lineage_map.get(n, n): n for n in available_bottles}
initial_bottle = _peek_agent_bottle(manifest, agent_name)
initial_labels = [lineage_map.get(initial_bottle, initial_bottle)] if initial_bottle else []
selected_labels = tui.filter_multiselect(
display_labels,
title="Select bottles",
initial=initial_labels,
)
if selected_labels is None:
return 0
bottle_names = tuple(label_to_name.get(lbl, lbl) for lbl in selected_labels)
backend_name: str | None = args.backend
label, color = tui.name_color_modal(default_label=agent_name)
label, color = _resolve_unique_label(label, color)
@@ -142,92 +85,15 @@ def cmd_start(argv: list[str]) -> int:
user_cwd=USER_CWD,
label=label,
color=color,
bottle_names=bottle_names,
)
return _launch_bottle(
spec,
dry_run=dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
# --- Headless launch -----------------------------------------------------
def _start_headless(
manifest: ManifestIndex,
args: argparse.Namespace,
*,
dry_run: bool,
backend_name: str | None,
) -> int:
"""Non-interactive launch path for orchestrators / CI / webhooks.
Resolves agent, bottles, label, and color from flags + manifest
defaults instead of the TUI selectors, and auto-confirms the
preflight. Otherwise runs the same launch core as the interactive
path, so the agent still execs on the inherited stdio/PTY — an
orchestrator allocates that PTY and relays it to its
desktop/mobile clients."""
agent_name = args.name
if not agent_name:
die("--headless requires an agent name: ./cli.py start <agent> --headless")
manifest.require_agent(agent_name) # raises ManifestError if unknown
prompt = args.prompt
if not prompt:
die(
"--headless requires --prompt: "
"./cli.py start <agent> --headless --prompt 'Do the thing'"
)
if args.bottle:
bottle_names: tuple[str, ...] = tuple(args.bottle)
else:
default_bottle = _peek_agent_bottle(manifest, agent_name)
if not default_bottle:
die(
f"--headless: agent '{agent_name}' has no default bottle; "
f"pass one or more --bottle NAME"
)
bottle_names = (default_bottle,)
label = _uniquify_label_headless(args.label or agent_name)
spec = BottleSpec(
manifest=manifest,
agent_name=agent_name,
copy_cwd=args.cwd,
user_cwd=USER_CWD,
label=label,
color=args.color or "",
bottle_names=bottle_names,
)
return _launch_bottle(
spec,
dry_run=dry_run,
backend_name=backend_name,
assume_yes=True,
headless_prompt_text=prompt,
)
def _uniquify_label_headless(label: str) -> str:
"""Non-interactive analog of `_resolve_unique_label`: if the label's
slug collides with a running bottle, append -2, -3, … until free,
logging the chosen label. Orchestrators fire-and-forget many bottles,
so silently picking a free name beats erroring on every collision."""
active_slugs = {a.slug for a in enumerate_active_agents()}
if docker_mod.slugify(label) not in active_slugs:
return label
n = 2
while docker_mod.slugify(f"{label}-{n}") in active_slugs:
n += 1
chosen = f"{label}-{n}"
info(f"label '{label}' already in use; using '{chosen}'")
return chosen
# --- Launch helpers ------------------------------------------------------
@@ -268,7 +134,7 @@ def prepare_with_preflight(
def attach_agent(
bottle: Bottle, *, resume: bool = False,
bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
agent_provider_template: str = "claude",
startup_args: tuple[str, ...] = (),
) -> int:
@@ -287,6 +153,8 @@ def attach_agent(
"(Ctrl-D or 'exit' to leave; container will be removed)"
)
agent_args = list(runtime.bypass_args)
if remote_control:
agent_args.extend(runtime.remote_control_args)
agent_args.extend(startup_args)
if resume:
agent_args.extend(runtime.resume_args)
@@ -326,38 +194,6 @@ def _identity_from_plan(plan: object) -> str:
return getattr(plan, "slug", "")
def _peek_agent_bottle(manifest: ManifestIndex, agent_name: str) -> str:
"""Return the `bottle:` value from the named agent's frontmatter without
fully parsing the agent file, or "" when absent or unreadable.
Used to pre-populate the bottle multiselect with the agent's default
bottle so operators who haven't removed `bottle:` from their manifests
don't need to re-select it every time."""
if manifest.home_md is None:
# Eager mode (from_json_obj): agent is pre-parsed.
if agent_name in manifest.agents:
return manifest.agents[agent_name].bottle
return ""
from ..manifest_loader import scan_agent_names
from ..yaml_subset import YamlSubsetError, parse_frontmatter
home_agents = scan_agent_names(manifest.home_md / "agents")
cwd_agents: dict[str, Path] = {}
if manifest.cwd_md is not None:
cwd_agents = scan_agent_names(manifest.cwd_md / "agents")
merged = {**home_agents, **cwd_agents}
path = merged.get(agent_name)
if path is None:
return ""
try:
fm, _ = parse_frontmatter(path.read_text())
bottle = fm.get("bottle", "")
return str(bottle) if isinstance(bottle, str) else ""
except (OSError, YamlSubsetError):
return ""
def _resolve_unique_label(label: str, color: str) -> tuple[str, str]:
"""Re-prompt with a disclaimer until the label's slug is not already
in use among running bottles. Passes through unchanged when no
@@ -382,140 +218,30 @@ def _text_prompt_yes() -> bool:
return reply in ("y", "Y", "yes", "YES")
def _text_render_preflight():
def _text_render_preflight(*, remote_control: bool):
def _render(plan: DockerBottlePlan) -> None:
print(file=sys.stderr)
print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
plan.print(remote_control=remote_control)
return _render
def _bottle_lineage(manifest: ManifestIndex) -> dict[str, str]:
"""Return {bottle_name: lineage_label} for bottles that have an extends chain.
Bottles without a parent are omitted (the caller falls back to the bare name).
Labels show the chain root-first: e.g. 'dev -> bot-bottle-dev -> claude-dev'."""
if manifest.home_md is None:
return {}
bottles_dir = manifest.home_md / "bottles"
if not bottles_dir.is_dir():
return {}
from ..yaml_subset import YamlSubsetError, parse_frontmatter
extends_of: dict[str, str] = {}
for path in bottles_dir.glob("*.md"):
try:
fm, _ = parse_frontmatter(path.read_text())
parent = fm.get("extends", "")
if isinstance(parent, str) and parent:
extends_of[path.stem] = parent
except (OSError, YamlSubsetError):
pass
labels: dict[str, str] = {}
for name in extends_of:
chain = [name]
seen = {name}
cur = name
while cur in extends_of:
par = extends_of[cur]
if par in seen:
break
chain.append(par)
seen.add(par)
cur = par
labels[name] = " -> ".join(reversed(chain))
return labels
def _manifest_to_yaml(manifest: Manifest) -> str:
"""Serialize the resolved Manifest to a YAML string for preflight display."""
lines: list[str] = []
agent = manifest.agent
lines.append("agent:")
if agent.skills:
lines.append(" skills:")
for s in agent.skills:
lines.append(f" - {s}")
if not agent.git_user.is_empty():
lines.append(" git-gate:")
lines.append(" user:")
if agent.git_user.name:
lines.append(f" name: {agent.git_user.name}")
if agent.git_user.email:
lines.append(f" email: {agent.git_user.email}")
bottle = manifest.bottle
lines.append("bottle:")
if bottle.agent_provider.template != "claude" or bottle.agent_provider.dockerfile:
lines.append(" agent_provider:")
lines.append(f" template: {bottle.agent_provider.template}")
if bottle.agent_provider.dockerfile:
lines.append(f" dockerfile: {bottle.agent_provider.dockerfile}")
if bottle.env:
lines.append(" env:")
for k, v in sorted(bottle.env.items()):
lines.append(f" {k}: {v}")
has_git_gate = not bottle.git_user.is_empty() or bottle.git
if has_git_gate:
lines.append(" git-gate:")
if not bottle.git_user.is_empty():
lines.append(" user:")
if bottle.git_user.name:
lines.append(f" name: {bottle.git_user.name}")
if bottle.git_user.email:
lines.append(f" email: {bottle.git_user.email}")
if bottle.git:
lines.append(" repos:")
for entry in bottle.git:
lines.append(f" {entry.Name}:")
lines.append(f" url: {entry.Upstream}")
if bottle.egress.routes:
lines.append(" egress:")
lines.append(" routes:")
for r in bottle.egress.routes:
lines.append(f" - host: {r.Host}")
if r.AuthScheme:
lines.append(f" auth:")
lines.append(f" scheme: {r.AuthScheme}")
lines.append(f" supervise: {'true' if bottle.supervise else 'false'}")
return "\n".join(lines)
def _launch_bottle(
spec: BottleSpec,
*,
dry_run: bool,
remote_control: bool,
backend_name: str | None = None,
assume_yes: bool = False,
headless_prompt_text: str = "",
) -> int:
"""Shared launch core for `start` and `resume`. Builds the plan,
prints / dry-runs / prompts as appropriate, brings the bottle up,
attaches claude, and prints the resume hint on session end.
`assume_yes` skips the interactive y/N confirmation (headless /
orchestrator launches), where there is no human at the prompt.
`headless_prompt_text` is passed to the provider's `headless_prompt`
method and the resulting args are appended to startup_args so the
agent receives the initial task without interactive input."""
attaches claude, and prints the resume hint on session end."""
stage_dir = Path(tempfile.mkdtemp(prefix="bot-bottle-stage."))
identity = ""
try:
plan, identity = prepare_with_preflight(
spec,
stage_dir=stage_dir,
render_preflight=_text_render_preflight(),
prompt_yes=(lambda: True) if assume_yes else _text_prompt_yes,
render_preflight=_text_render_preflight(remote_control=remote_control),
prompt_yes=_text_prompt_yes,
dry_run=dry_run,
backend_name=backend_name,
)
@@ -525,17 +251,11 @@ def _launch_bottle(
backend = get_bottle_backend(backend_name)
with backend.launch(plan) as bottle:
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
extra_args: tuple[str, ...] = ()
if headless_prompt_text:
extra_args = tuple(
get_provider(agent_provider_template).headless_prompt(
headless_prompt_text
)
)
exit_code = attach_agent(
bottle,
remote_control=remote_control,
agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args + extra_args,
startup_args=plan.agent_provision.startup_args,
)
info(
f"session ended (exit {exit_code}); "
@@ -543,8 +263,12 @@ def _launch_bottle(
)
# While the container is still alive: always snapshot the
# transcript and — if the agent exited non-zero — mark
# the state for preservation. This picks up crashes /
# Ctrl-Cs / OOM kills before cleanup removes the state dir.
# the state for preservation. Capability-block already
# did both before triggering teardown from the dashboard;
# this picks up crashes / Ctrl-Cs / OOM kills the same
# way. snapshot_transcript is best-effort so the
# capability-block path's prior snapshot isn't clobbered
# when the container is already gone.
if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code)
return 0
+46 -28
View File
@@ -2,8 +2,9 @@
act on them (approve / modify / reject).
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
Egress proposals are queued for operator review as full routes.yaml
updates.
approval handler wires to PRD 0016 (capability-block), which rebuilds
the bottle Dockerfile. Egress proposals are queued for operator review
as full routes.yaml updates.
"""
from __future__ import annotations
@@ -21,6 +22,10 @@ from pathlib import Path
from .. import supervise as _supervise
from ..bottle_state import read_metadata
# from ..backend.docker.capability_apply import (
# CapabilityApplyError,
# apply_capability_change,
# )
from ..backend.docker.egress_apply import (
EgressApplyError,
applicator as _docker_applicator,
@@ -33,6 +38,10 @@ from ..backend.smolmachines.egress_apply import (
)
from ..log import Die, error, info
class CapabilityApplyError(RuntimeError):
"""Placeholder while capability_apply is disabled."""
from ..supervise import (
COMPONENT_FOR_TOOL,
AuditEntry,
@@ -41,10 +50,11 @@ from ..supervise import (
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOL_EGRESS_ALLOW,
TOOL_CAPABILITY_BLOCK,
TOOL_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
archive_proposal,
list_pending_proposals,
render_diff,
write_audit_entry,
@@ -55,11 +65,6 @@ from ._common import PROG
_REFRESH_INTERVAL_MS = 1000
# Proposal tools whose payload is a read-only report, not a file the operator
# edits: modify is unavailable and approval requires a recorded reason for the
# audit trail.
_REPORT_ONLY_TOOLS: tuple[str, ...] = (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW)
@dataclass(frozen=True)
class QueuedProposal:
@@ -72,7 +77,7 @@ class QueuedProposal:
# Errors any remediation engine may raise. Caught by the TUI key
# handlers and surfaced in the status line so a failed apply keeps
# the proposal pending rather than crashing curses.
ApplyError = (EgressApplyError,)
ApplyError = (CapabilityApplyError, EgressApplyError)
def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
@@ -132,9 +137,11 @@ def _detail_lines(
def _suffix_for_tool(tool: str) -> str:
if tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
if tool == TOOL_CAPABILITY_BLOCK:
return ".dockerfile"
if tool in (TOOL_ALLOW, TOOL_EGRESS_BLOCK):
return ".yaml"
if tool in (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW):
if tool == TOOL_GITLEAKS_ALLOW:
return ".txt"
return ".txt"
@@ -153,7 +160,18 @@ def approve(
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
diff_before, diff_after = "", ""
if qp.proposal.tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
# if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
# _meta = read_metadata(qp.proposal.bottle_slug)
# if _meta is not None and not _meta.compose_project:
# raise CapabilityApplyError(
# "capability-block remediation is not supported for smolmachines "
# "bottles. Reject this proposal or handle the capability change "
# "manually, then restart the bottle."
# )
# diff_before, diff_after = apply_capability_change(
# qp.proposal.bottle_slug, file_to_apply,
# )
if qp.proposal.tool in (TOOL_ALLOW, TOOL_EGRESS_BLOCK):
diff_before, diff_after = apply_routes_change(
qp.proposal.bottle_slug,
file_to_apply,
@@ -170,6 +188,9 @@ def approve(
qp, action=status, notes=notes,
diff_before=diff_before, diff_after=diff_after,
)
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
archive_proposal(qp.queue_dir, qp.proposal.id)
def reject(qp: QueuedProposal, *, reason: str) -> None:
"""Write a rejection response and an audit entry."""
@@ -191,8 +212,8 @@ def _approve_from_tui(
notes: str = "",
) -> str:
"""Approve from curses, prompting for any tool-specific audit note."""
if qp.proposal.tool in _REPORT_ONLY_TOOLS and final_file is None:
notes = _prompt(stdscr, "allow reason (false positive / legitimately needed): ")
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW and final_file is None:
notes = _prompt(stdscr, "allow reason (test fixture/false positive): ")
if not notes:
return "approve aborted (empty reason)"
approve(qp, final_file=final_file, notes=notes)
@@ -271,10 +292,7 @@ def cmd_supervise(argv: list[str]) -> int:
return e.code if isinstance(e.code, int) else 1
except Exception as e: # noqa: W0718 — catch supervise crash for logging
log_path = _write_crash_log(e)
error(
f"supervise crashed: {type(e).__name__}: {e}",
context={"error_type": type(e).__name__, "crash_log": str(log_path)},
)
error(f"supervise crashed: {type(e).__name__}: {e}")
error(f"full traceback written to {log_path}")
return 1
return 0
@@ -319,7 +337,7 @@ def _list_once() -> int:
return 0
def _try_init_green() -> int: # pragma: no cover
def _try_init_green() -> int:
"""Initialise a green color pair and return its attr, or 0."""
try:
curses.start_color()
@@ -330,7 +348,7 @@ def _try_init_green() -> int: # pragma: no cover
return 0
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore # pragma: no cover
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
curses.curs_set(0)
stdscr.timeout(_REFRESH_INTERVAL_MS)
green_attr = _try_init_green()
@@ -390,8 +408,8 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore # pragm
except ApplyError as e:
status_line = f"apply failed: {e}"
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
status_line = f"modify unavailable for {qp.proposal.tool}"
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW:
status_line = "modify unavailable for gitleaks-allow"
continue
edited = _modify(stdscr, qp)
if edited is None:
@@ -420,7 +438,7 @@ def _render(
status_line: str,
*,
green_attr: int = 0, # noqa: F841 — unused, but required by interface
) -> None: # pragma: no cover
) -> None:
stdscr.erase()
h, w = stdscr.getmaxyx()
header = f"bot-bottle supervise ({len(pending)} pending)"
@@ -471,7 +489,7 @@ def _detail_view(
qp: QueuedProposal,
*,
green_attr: int = 0,
) -> None: # pragma: no cover
) -> None:
"""Render the full proposal. Scrollable. Press q to return."""
lines = _detail_lines(qp, green_attr=green_attr)
offset = 0
@@ -504,7 +522,7 @@ def _detail_view(
pass
return
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW:
return
edited = _modify(stdscr, qp)
if edited is not None:
@@ -523,7 +541,7 @@ def _detail_view(
return
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore # pragma: no cover
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
suffix = _suffix_for_tool(qp.proposal.tool)
curses.endwin()
@@ -534,7 +552,7 @@ def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
return edited
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore # pragma: no cover
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore
"""One-line input at the bottom of the screen."""
curses.curs_set(1)
h, _ = stdscr.getmaxyx()
-300
View File
@@ -17,43 +17,6 @@ import sys
from typing import Any, Optional
def filter_multiselect(
items: list[str],
*,
title: str = "",
initial: Optional[list[str]] = None,
tty_path: str = "/dev/tty",
) -> Optional[list[str]]:
"""Render a multi-select picker over *items*.
Returns the ordered list of selected items, or ``None`` if the user
cancelled (Esc / ``q`` / Ctrl-C / Ctrl-D with no items).
Press Space to toggle the item under the cursor.
Press Enter to confirm the current selection.
Press Ctrl-D to confirm the current selection (returns even if empty).
Press Esc/q to cancel (returns None).
*initial* pre-populates the selection in insertion order. Items
added are appended; removed items leave the remaining order unchanged.
"""
if not items:
return []
try:
tty_fd = open(tty_path, "r+b", buffering=0)
except OSError:
return None
try:
fd_dup = os.dup(tty_fd.fileno())
return _run_multiselect(
items, title=title, initial=list(initial or []), tty_fd=fd_dup
)
finally:
tty_fd.close()
def filter_select(
items: list[str],
*,
@@ -258,269 +221,6 @@ def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.
pass
# ---------------------------------------------------------------------------
# filter_multiselect internals
# ---------------------------------------------------------------------------
_KEY_SPACE = 32
def _run_multiselect(
items: list[str], *, title: str, initial: list[str], tty_fd: int
) -> Optional[list[str]]:
"""Drive a curses multi-select session on *tty_fd*."""
os.environ.setdefault("TERM", "xterm-256color")
orig_stdin = sys.__stdin__
orig_stdout = sys.__stdout__
try:
import io
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode='r+'), write_through=True)
sys.__stdin__ = tty_text # type: ignore[assignment]
sys.__stdout__ = tty_text # type: ignore[assignment]
screen = curses.initscr()
curses.noecho()
curses.cbreak()
screen.keypad(True)
try:
result = _multiselect_loop(screen, items, title=title, initial=initial)
finally:
screen.keypad(False)
curses.nocbreak()
curses.echo()
curses.endwin()
except Exception: # noqa: W0718
return None
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
sys.__stdout__ = orig_stdout # type: ignore[assignment]
return result
def _toggle_membership(items: list[str], item: str) -> None:
"""Add `item` if absent, remove it if present (in place)."""
if item in items:
items.remove(item)
else:
items.append(item)
def _handle_order_key(key: int, selected: list[str], order_cursor: int) -> int:
"""Apply a keypress in 'order' focus: navigate, reorder, or remove the
item at `order_cursor`. Mutates `selected` in place and returns the new
order cursor."""
if key in (curses.KEY_UP, ord("k")):
if order_cursor > 0:
order_cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")):
if order_cursor < len(selected) - 1:
order_cursor += 1
elif key == ord("K"):
# Move selected item up (earlier in order).
if order_cursor > 0:
i = order_cursor
selected[i - 1], selected[i] = selected[i], selected[i - 1]
order_cursor -= 1
elif key == ord("J"):
# Move selected item down (later in order).
if order_cursor < len(selected) - 1:
i = order_cursor
selected[i], selected[i + 1] = selected[i + 1], selected[i]
order_cursor += 1
elif key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r"), _KEY_SPACE):
# Remove item from selection while in order mode.
del selected[order_cursor]
if order_cursor >= len(selected) and order_cursor > 0:
order_cursor -= 1
return order_cursor
def _multiselect_loop(
screen: Any, items: list[str], *, title: str, initial: list[str]
) -> Optional[list[str]]:
query = ""
cursor = 0
selected: list[str] = [s for s in initial if s in items]
# focus = "filter": navigate + toggle items in the filterable list
# focus = "order": navigate + reorder items in the selected list
focus = "filter"
order_cursor = 0
while True:
filtered = _filter_items(items, query)
if not filtered:
cursor = 0
elif cursor >= len(filtered):
cursor = len(filtered) - 1
if not selected:
order_cursor = 0
if focus == "order":
focus = "filter"
elif order_cursor >= len(selected):
order_cursor = len(selected) - 1
try:
_render_multiselect(
screen, filtered, cursor,
query=query, title=title, selected=selected,
focus=focus, order_cursor=order_cursor,
)
except curses.error:
return None
try:
key = screen.getch()
except KeyboardInterrupt:
return None
if key in (_KEY_ESC, _KEY_CTRL_C, ord("q")):
return None
if key == _KEY_CTRL_D:
return list(selected)
# Tab toggles between filter and order focus.
if key == ord("\t"):
if focus == "filter" and selected:
focus = "order"
order_cursor = 0
else:
focus = "filter"
continue
if focus == "filter":
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
return list(selected)
elif key == _KEY_SPACE:
if filtered:
_toggle_membership(selected, filtered[cursor])
elif key in (curses.KEY_UP, ord("k")):
if cursor > 0:
cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")):
if cursor < len(filtered) - 1:
cursor += 1
elif key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
query = query[:-1]
new_filtered = _filter_items(items, query)
if cursor >= len(new_filtered):
cursor = max(0, len(new_filtered) - 1)
elif 32 <= key <= 126 and key != _KEY_SPACE:
query += chr(key)
cursor = 0
else: # focus == "order"
order_cursor = _handle_order_key(key, selected, order_cursor)
def _render_multiselect(
screen: Any,
filtered: list[str],
cursor: int,
*,
query: str,
title: str,
selected: list[str],
focus: str = "filter",
order_cursor: int = 0,
) -> None:
screen.erase()
rows, cols = screen.getmaxyx()
min_rows = 7
if rows < min_rows:
raise curses.error("terminal too small")
sep = "" * min(cols - 1, 40)
row = 0
if title and row < rows - 1:
_addstr_safe(screen, row, 0, title[:cols - 1], curses.A_BOLD)
row += 1
# Filter line — dim when focus is on the order panel.
filter_label = f"Filter: {query}"
filter_hint = " [Tab: reorder]" if focus == "filter" and selected else ""
filter_attr = curses.A_DIM if focus == "order" else curses.A_NORMAL
if row < rows - 1:
_addstr_safe(screen, row, 0, (filter_label + filter_hint)[:cols - 1], filter_attr)
row += 1
if row < rows - 1:
_addstr_safe(screen, row, 0, sep)
row += 1
# Compute how many rows the bottom order panel needs.
# Cap the visible selected list to keep the filter list legible.
order_rows = min(len(selected), max(1, (rows - row) // 3)) if selected else 0
# Bottom reserved: sep + order_rows + sep + help = order_rows + 3
bottom_reserved = order_rows + 3
list_start = row
list_rows = rows - list_start - bottom_reserved
if list_rows < 1:
list_rows = 1
selected_set = set(selected)
filter_dim = focus == "order"
scroll = max(0, cursor - list_rows + 1)
visible = filtered[scroll: scroll + list_rows]
for idx, item in enumerate(visible):
abs_idx = scroll + idx
mark = "[*]" if item in selected_set else "[ ]"
prefix = "> " if (abs_idx == cursor and focus == "filter") else " "
line = (prefix + mark + " " + item)[:cols - 1]
item_attr = curses.A_DIM if filter_dim else (
curses.A_REVERSE if abs_idx == cursor else curses.A_NORMAL
)
if row < rows - bottom_reserved:
_addstr_safe(screen, row, 0, line, item_attr)
row += 1
# Separator before the order panel.
if row < rows - (order_rows + 2):
_addstr_safe(screen, row, 0, sep)
row += 1
# Order panel.
order_scroll = max(0, order_cursor - order_rows + 1)
order_visible = selected[order_scroll: order_scroll + order_rows]
for idx, item in enumerate(order_visible):
abs_idx = order_scroll + idx
is_active = focus == "order" and abs_idx == order_cursor
prefix = "> " if is_active else " "
line = (prefix + item)[:cols - 1]
attr = curses.A_REVERSE if is_active else curses.A_NORMAL
if row < rows - 2:
_addstr_safe(screen, row, 0, line, attr)
row += 1
if row < rows - 1:
_addstr_safe(screen, row, 0, sep)
row += 1
if focus == "filter":
help_line = "[↑↓/jk] move [Space] toggle [Enter] confirm [Tab] reorder [Esc/q] cancel"
else:
help_line = "[↑↓/jk] cursor [K/J] reorder [Space/Enter] remove [Tab] back [Ctrl-D] done"
if row < rows:
_addstr_safe(screen, min(rows - 1, row), 0, help_line[:cols - 1])
screen.refresh()
# ---------------------------------------------------------------------------
# name_color_modal — two-step label + color picker
# ---------------------------------------------------------------------------
+1 -1
View File
@@ -21,7 +21,7 @@ FROM node:22-slim
# to it) works against egress's bumped TLS without the agent needing
# local DNS.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \
&& apt-get install -y --no-install-recommends git ca-certificates curl \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by claude-code itself
+5 -14
View File
@@ -20,7 +20,6 @@ from ...agent_provider import (
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...backend.docker import util as docker_mod
from ...egress import EgressRoute
@@ -91,6 +90,7 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",),
remote_control_args=("--remote-control",),
)
@@ -115,9 +115,8 @@ class ClaudeAgentProvider(AgentProvider):
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del forward_host_credentials, host_env
del forward_host_credentials, host_env, provider_settings
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
@@ -200,7 +199,6 @@ class ClaudeAgentProvider(AgentProvider):
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=dirs,
files=tuple(files),
egress_routes=egress_routes,
@@ -217,7 +215,7 @@ class ClaudeAgentProvider(AgentProvider):
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
bottle.exec(f"mkdir -p {shlex.quote(skills_dir)}", user="root")
bottle.exec(f"mkdir -p {skills_dir}", user="root")
for name in agent.skills:
src = host_skill_dir(name)
if not os.path.isdir(src):
@@ -227,13 +225,9 @@ class ClaudeAgentProvider(AgentProvider):
)
dst = f"{skills_dir}/{name}"
info(f"copying skill {name} into {bottle.name}:{dst}")
# Defense in depth: skill names are validated kebab-case at
# manifest load, but quote the path so a future unvalidated
# field can't inject shell metacharacters here either.
dst_q = shlex.quote(dst)
bottle.exec(f"rm -rf {dst_q} && mkdir -p {dst_q}", user="root")
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
bottle.cp_in(f"{src}/.", f"{dst}/")
bottle.exec(f"chown -R node:node {dst_q}", user="root")
bottle.exec(f"chown -R node:node {dst}", user="root")
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
"""Copy the prompt file into the guest, fix ownership/mode.
@@ -313,9 +307,6 @@ class ClaudeAgentProvider(AgentProvider):
f"claude mcp add --scope user --transport http supervise {supervise_url}"
)
def headless_prompt(self, prompt: str) -> list[str]:
return ["-p", prompt]
def _exec(bottle: "Bottle", script: str, error: str) -> None:
result = bottle.exec(script, user="root")
+6 -9
View File
@@ -1,12 +1,12 @@
# bot-bottle Codex provider image.
#
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed for that user.
# non-root node user, and the provider CLI installed globally.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl procps ripgrep \
&& apt-get install -y --no-install-recommends git ca-certificates curl \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by codex itself
@@ -17,15 +17,12 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
&& npm cache clean --force
USER node
WORKDIR /home/node
ENV PATH="/home/node/.local/bin:${PATH}"
# Remote-control support requires the standalone Codex install layout
# under ~/.codex/packages/standalone/current. The npm package can run
# the TUI, but remote-control commands expect this installer-owned path.
RUN mkdir -p /home/node/.codex \
&& curl -fsSL https://chatgpt.com/codex/install.sh | sh
RUN mkdir -p /home/node/.codex
CMD ["codex"]
+5 -14
View File
@@ -22,7 +22,6 @@ from ...agent_provider import (
AgentProvisionCommand,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
@@ -55,6 +54,7 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"),
remote_control_args=(),
)
@@ -79,9 +79,8 @@ class CodexAgentProvider(AgentProvider):
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, label, color
del auth_token, label, color, provider_settings
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
@@ -164,7 +163,6 @@ class CodexAgentProvider(AgentProvider):
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=tuple(dirs),
files=tuple(files),
pre_copy=tuple(pre_copy),
@@ -183,7 +181,7 @@ class CodexAgentProvider(AgentProvider):
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
bottle.exec(f"mkdir -p {shlex.quote(skills_dir)}", user="root")
bottle.exec(f"mkdir -p {skills_dir}", user="root")
for name in agent.skills:
src = host_skill_dir(name)
if not os.path.isdir(src):
@@ -193,13 +191,9 @@ class CodexAgentProvider(AgentProvider):
)
dst = f"{skills_dir}/{name}"
info(f"copying skill {name} into {bottle.name}:{dst}")
# Defense in depth: skill names are validated kebab-case at
# manifest load, but quote the path so a future unvalidated
# field can't inject shell metacharacters here either.
dst_q = shlex.quote(dst)
bottle.exec(f"rm -rf {dst_q} && mkdir -p {dst_q}", user="root")
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
bottle.cp_in(f"{src}/.", f"{dst}/")
bottle.exec(f"chown -R node:node {dst_q}", user="root")
bottle.exec(f"chown -R node:node {dst}", user="root")
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
"""Copy the prompt file into the guest, fix ownership/mode.
@@ -279,9 +273,6 @@ class CodexAgentProvider(AgentProvider):
f"codex mcp add supervise --url {shlex.quote(supervise_url)}"
)
def headless_prompt(self, prompt: str) -> list[str]:
return [prompt]
def _exec(bottle: "Bottle", script: str, error: str) -> None:
result = bottle.exec(script, user="root")
-52
View File
@@ -1,52 +0,0 @@
"""Scoped forge wrapper: read-anywhere / write-scoped access control.
`ScopedForge` wraps any forge object and restricts write operations to
the set of issue/PR numbers the agent is explicitly assigned to. Read
operations always pass through unconditionally.
"""
from __future__ import annotations
from typing import Any
class ScopedForge:
"""Delegates all forge calls to an inner forge, raising `PermissionError`
on write calls for numbers outside the assigned scope."""
def __init__(
self,
forge: Any,
*,
assigned_issue: int,
assigned_prs: list[int],
) -> None:
self._forge = forge
self._allowed_writes: frozenset[int] = frozenset({assigned_issue, *assigned_prs})
def _check_write(self, number: int) -> None:
if number not in self._allowed_writes:
raise PermissionError(
f"write to #{number} is outside the assigned scope "
f"(allowed: {sorted(self._allowed_writes)})"
)
def is_org_member(self, org: str, username: str) -> bool:
return self._forge.is_org_member(org, username)
def read_issue(self, number: int) -> dict[str, Any]:
return self._forge.read_issue(number)
def read_pr(self, number: int) -> dict[str, Any]:
return self._forge.read_pr(number)
def read_comments(self, number: int) -> list[dict[str, Any]]:
return self._forge.read_comments(number)
def post_comment(self, number: int, body: str) -> None:
self._check_write(number)
self._forge.post_comment(number, body)
def update_description(self, number: int, body: str) -> None:
self._check_write(number)
self._forge.update_description(number, body)
-112
View File
@@ -1,112 +0,0 @@
"""Gitea API client and forge adapter (PRD prd-new: fold orchestrator).
`GiteaClient` is a thin HTTP wrapper (stdlib `urllib.request` only no
new runtime dependencies). `GiteaForge` composes a client and exposes
the forge protocol used by the orchestrator's sidecar and lifecycle.
Required Gitea token scopes:
- Repository: Read & Write (issues, comments, PR descriptions)
- Organization: Read (org membership check)
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from typing import Any
_TIMEOUT_SECS = 30
class GiteaClient:
"""Low-level HTTP wrapper for the Gitea REST API."""
def __init__(
self, *, api_url: str, owner: str, repo: str, token: str
) -> None:
self._base = api_url.rstrip("/")
self._owner = owner
self._repo = repo
self._headers = {
"Authorization": f"token {token}",
"Content-Type": "application/json",
"Accept": "application/json",
}
def _request(
self,
method: str,
path: str,
body: dict[str, Any] | None = None,
) -> Any:
url = f"{self._base}{path}"
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(
url, data=data, headers=self._headers, method=method
)
with urllib.request.urlopen(req, timeout=_TIMEOUT_SECS) as resp:
raw = resp.read()
return json.loads(raw) if raw else None
def is_org_member(self, org: str, username: str) -> bool:
url = f"{self._base}/orgs/{org}/members/{username}"
req = urllib.request.Request(url, headers=self._headers, method="GET")
try:
urllib.request.urlopen(req, timeout=_TIMEOUT_SECS).close()
return True
except urllib.error.HTTPError:
return False
def get_issue(self, number: int) -> dict[str, Any]:
return self._request("GET", f"/repos/{self._owner}/{self._repo}/issues/{number}")
def get_pull(self, number: int) -> dict[str, Any]:
return self._request("GET", f"/repos/{self._owner}/{self._repo}/pulls/{number}")
def list_comments(self, number: int) -> list[dict[str, Any]]:
return self._request("GET", f"/repos/{self._owner}/{self._repo}/issues/{number}/comments")
def create_comment(self, number: int, body: str) -> None:
self._request(
"POST",
f"/repos/{self._owner}/{self._repo}/issues/{number}/comments",
{"body": body},
)
def update_issue(self, number: int, body: str) -> None:
self._request(
"PATCH",
f"/repos/{self._owner}/{self._repo}/issues/{number}",
{"body": body},
)
class GiteaForge:
"""Adapts `GiteaClient` to the forge protocol expected by the orchestrator.
The forge protocol is duck-typed: any object with `is_org_member`,
`read_issue`, `read_pr`, `read_comments`, `post_comment`, and
`update_description` methods satisfies it.
"""
def __init__(self, client: GiteaClient) -> None:
self._client = client
def is_org_member(self, org: str, username: str) -> bool:
return self._client.is_org_member(org, username)
def read_issue(self, number: int) -> dict[str, Any]:
return self._client.get_issue(number)
def read_pr(self, number: int) -> dict[str, Any]:
return self._client.get_pull(number)
def read_comments(self, number: int) -> list[dict[str, Any]]:
return self._client.list_comments(number)
def post_comment(self, number: int, body: str) -> None:
self._client.create_comment(number, body)
def update_description(self, number: int, body: str) -> None:
self._client.update_issue(number, body)
@@ -19,12 +19,7 @@ import urllib.error
import urllib.request
from pathlib import Path
from ...deploy_key_provisioner import DeployKeyCollisionError, DeployKeyProvisioner
# Timeout for ssh-keygen and Gitea API HTTP calls. A hung Gitea instance at
# prepare time would stall bottle launch indefinitely without this bound.
_API_TIMEOUT_SECS = 30
_KEYGEN_TIMEOUT_SECS = 10
from ...deploy_key_provisioner import DeployKeyProvisioner
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
@@ -51,7 +46,6 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=_KEYGEN_TIMEOUT_SECS,
)
private_key = key_path.read_bytes()
public_key = key_path.with_suffix(".pub").read_text().strip()
@@ -73,15 +67,10 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=_API_TIMEOUT_SECS) as resp:
with urllib.request.urlopen(req) as resp:
body = json.loads(resp.read())
except urllib.error.HTTPError as exc:
_body = _read_error_body(exc)
if exc.code == 422:
raise DeployKeyCollisionError(
f"deploy key collision for {owner_repo!r} "
f"(title={title!r}): key title or content already registered — {_body}"
) from exc
raise RuntimeError(
f"failed to create deploy key for {owner_repo}: "
f"HTTP {exc.code}{_body}"
@@ -104,7 +93,7 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
method="DELETE",
)
try:
with urllib.request.urlopen(req, timeout=_API_TIMEOUT_SECS):
with urllib.request.urlopen(req):
pass
except urllib.error.HTTPError as exc:
if exc.code == 404:
-137
View File
@@ -1,137 +0,0 @@
"""Forge state persistence for the orchestrator (PRD prd-new: fold orchestrator).
`ForgeState` is a dataclass that mirrors the orchestrator's `RunRecord`
field-for-field, held here so the store implementation is in bot-bottle
where the Gitea contrib lives.
`SqliteForgeStateStore` backs it with a single SQLite table. The DB path
is optional; passing `None` uses `:memory:` (useful for tests and status
commands that don't need persistence).
"""
from __future__ import annotations
import json
import sqlite3
from dataclasses import dataclass, field
from pathlib import Path
@dataclass
class ForgeState:
"""Persisted state for one forge-targeted issue's bottle lifecycle."""
owner: str
repo: str
issue_number: int
slug: str
agent_name: str
bottle_names: list[str] = field(default_factory=list)
backend_name: str = ""
agent_git_user: str = ""
pr_number: int | None = None
status: str = ""
last_checkin_at: str = ""
_DDL = """
CREATE TABLE IF NOT EXISTS forge_state (
owner TEXT NOT NULL,
repo TEXT NOT NULL,
issue_number INTEGER NOT NULL,
slug TEXT NOT NULL,
agent_name TEXT NOT NULL,
bottle_names TEXT NOT NULL DEFAULT '[]',
backend_name TEXT NOT NULL DEFAULT '',
agent_git_user TEXT NOT NULL DEFAULT '',
pr_number INTEGER,
status TEXT NOT NULL DEFAULT '',
last_checkin_at TEXT NOT NULL DEFAULT '',
PRIMARY KEY (owner, repo, issue_number)
)
"""
class SqliteForgeStateStore:
"""SQLite-backed `ForgeState` store.
Thread-safety: a single connection is used; callers that share a
store across threads must serialise access externally.
"""
def __init__(self, db_path: Path | None) -> None:
path = str(db_path) if db_path is not None else ":memory:"
self._conn = sqlite3.connect(path, check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute(_DDL)
self._conn.commit()
def upsert(self, state: ForgeState) -> None:
self._conn.execute(
"""
INSERT INTO forge_state
(owner, repo, issue_number, slug, agent_name,
bottle_names, backend_name, agent_git_user,
pr_number, status, last_checkin_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(owner, repo, issue_number) DO UPDATE SET
slug = excluded.slug,
agent_name = excluded.agent_name,
bottle_names = excluded.bottle_names,
backend_name = excluded.backend_name,
agent_git_user = excluded.agent_git_user,
pr_number = excluded.pr_number,
status = excluded.status,
last_checkin_at = excluded.last_checkin_at
""",
(
state.owner,
state.repo,
state.issue_number,
state.slug,
state.agent_name,
json.dumps(state.bottle_names),
state.backend_name,
state.agent_git_user,
state.pr_number,
state.status,
state.last_checkin_at,
),
)
self._conn.commit()
def get(self, owner: str, repo: str, issue_number: int) -> ForgeState | None:
row = self._conn.execute(
"SELECT * FROM forge_state WHERE owner=? AND repo=? AND issue_number=?",
(owner, repo, issue_number),
).fetchone()
return _row_to_state(row) if row is not None else None
def delete(self, owner: str, repo: str, issue_number: int) -> None:
self._conn.execute(
"DELETE FROM forge_state WHERE owner=? AND repo=? AND issue_number=?",
(owner, repo, issue_number),
)
self._conn.commit()
def all(self) -> list[ForgeState]:
rows = self._conn.execute(
"SELECT * FROM forge_state ORDER BY owner, repo, issue_number"
).fetchall()
return [_row_to_state(r) for r in rows]
def _row_to_state(row: sqlite3.Row) -> ForgeState:
return ForgeState(
owner=row["owner"],
repo=row["repo"],
issue_number=row["issue_number"],
slug=row["slug"],
agent_name=row["agent_name"],
bottle_names=json.loads(row["bottle_names"]),
backend_name=row["backend_name"],
agent_git_user=row["agent_git_user"],
pr_number=row["pr_number"],
status=row["status"],
last_checkin_at=row["last_checkin_at"],
)
+4 -13
View File
@@ -21,7 +21,6 @@ from ...agent_provider import (
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...egress import EgressRoute
from ...log import die, info
@@ -166,6 +165,7 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="append_system_prompt",
bypass_args=(),
resume_args=(),
remote_control_args=(),
)
@@ -199,7 +199,6 @@ class PiAgentProvider(AgentProvider):
models_payload, base_url, api_key_env, models, provider_name = (
_pi_models_json(settings)
)
extra_startup_args = provider_startup_args(provider_settings)
models_file = state_dir / "pi-models.json"
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
models_file.chmod(0o600)
@@ -220,7 +219,6 @@ class PiAgentProvider(AgentProvider):
startup_args=(
"--models",
",".join(f"{provider_name}/{model}" for model in models),
*extra_startup_args,
),
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
@@ -238,7 +236,7 @@ class PiAgentProvider(AgentProvider):
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
bottle.exec(f"mkdir -p {shlex.quote(skills_dir)}", user="root")
bottle.exec(f"mkdir -p {skills_dir}", user="root")
for name in agent.skills:
src = host_skill_dir(name)
if not os.path.isdir(src):
@@ -248,13 +246,9 @@ class PiAgentProvider(AgentProvider):
)
dst = f"{skills_dir}/{name}"
info(f"copying skill {name} into {bottle.name}:{dst}")
# Defense in depth: skill names are validated kebab-case at
# manifest load, but quote the path so a future unvalidated
# field can't inject shell metacharacters here either.
dst_q = shlex.quote(dst)
bottle.exec(f"rm -rf {dst_q} && mkdir -p {dst_q}", user="root")
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
bottle.cp_in(f"{src}/.", f"{dst}/")
bottle.exec(f"chown -R node:node {dst_q}", user="root")
bottle.exec(f"chown -R node:node {dst}", user="root")
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
prompt_path = _prompt_path(plan.guest_home)
@@ -315,9 +309,6 @@ class PiAgentProvider(AgentProvider):
) -> None:
del plan, bottle, supervise_url
def headless_prompt(self, prompt: str) -> list[str]:
return ["-p", prompt]
def _exec(bottle: "Bottle", script: str, error: str) -> None:
result = bottle.exec(script, user="root")
-4
View File
@@ -11,10 +11,6 @@ from __future__ import annotations
from abc import ABC, abstractmethod
class DeployKeyCollisionError(RuntimeError):
"""Raised when a deploy key title or public key already exists on the repo."""
class DeployKeyProvisioner(ABC):
"""Manages a single deploy-key lifecycle on a remote forge."""
+15 -259
View File
@@ -11,13 +11,10 @@ the same try/except import shim pattern.
from __future__ import annotations
import base64
import functools
import gzip
import re
import typing
import unicodedata
from math import log2
from collections import Counter
from urllib.parse import quote as url_quote
try:
@@ -81,27 +78,16 @@ TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
)
def scan_token_patterns(
text: str,
*,
location: str = "body",
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
def scan_token_patterns(text: str, *, location: str = "body") -> ScanResult | None:
normalized = _normalize_text(text)
for name, pattern in TOKEN_PATTERNS:
for m in pattern.finditer(normalized):
value = m.group(0)
# A value the supervisor has approved (PRD 0062) is no longer a
# block — keep scanning so a second, un-approved token in the
# same request is still caught.
if safe_tokens is not None and value in safe_tokens:
continue
m = pattern.search(normalized)
if m is not None:
return ScanResult(
severity="block",
reason=f"{name} found in {location}",
location=location,
context=_snippet(normalized, m.start(), m.end()),
matched=value,
context=_snippet(text, m.start(), m.end()),
)
return None
@@ -110,46 +96,24 @@ def redact_tokens(
text: str,
*,
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> str:
"""Replace token pattern matches and (if env given) provisioned secrets with REDACT."""
for _, pattern in TOKEN_PATTERNS:
text = pattern.sub(REDACT, text)
if env is not None:
for key, value in env.items():
if any(key.startswith(p) for p in sensitive_prefixes) and value:
if key.startswith("EGRESS_TOKEN_") and value:
for variant in _encoded_variants(value):
text = text.replace(variant, REDACT)
return text
# ---------------------------------------------------------------------------
# Known secrets detector
# Known secrets detector (Phase 1b)
# ---------------------------------------------------------------------------
# Encoded-variant cache. Provisioned secrets are stable for the life of the
# proxy, but `_encoded_variants` is on the per-request hot path — it runs for
# every secret on every redaction and known-secret scan (host, path, each
# header, body). Deriving the variant set is relatively expensive (gzip +
# nine encodings), so memoize it per distinct secret. The proxy process
# already holds these values in `os.environ`, so caching them here adds no
# new exposure. The cache is bounded (lru_cache maxsize) so a long-lived
# proxy that sees rotating secrets evicts the oldest rather than growing
# without limit; 256 comfortably covers the EGRESS_TOKEN_* set in practice.
_VARIANT_CACHE_MAXSIZE = 256
def _encoded_variants(secret: str) -> list[str]:
"""Return the secret plus common encoded variants for exfil detection.
The variant set is computed once per distinct secret and cached; callers
get a fresh list so they can't mutate the shared cached tuple."""
return list(_compute_encoded_variants(secret))
@functools.lru_cache(maxsize=_VARIANT_CACHE_MAXSIZE)
def _compute_encoded_variants(secret: str) -> tuple[str, ...]:
"""Derive the secret plus its encoded variants (memoized, bounded)."""
"""Return the secret plus common encoded variants for exfil detection."""
seen: set[str] = {secret}
variants: list[str] = [secret]
@@ -183,52 +147,7 @@ def _compute_encoded_variants(secret: str) -> tuple[str, ...]:
# gzip + base64 (deterministic: mtime=0); recognisable by H4sI prefix
_add(base64.b64encode(gzip.compress(secret_bytes, mtime=0)).decode("ascii"))
return tuple(variants)
# ---------------------------------------------------------------------------
# Fragmentation-resistant helpers
# ---------------------------------------------------------------------------
# Minimum length of alnum projection for projection-based checks to run.
# Short secrets produce too many false positives in projection space.
_ALNUM_MIN_LEN = 8
# Minimum window length for the partial-substring sliding scan.
PARTIAL_MATCH_MIN_LEN = 12
def _alnum_projection(text: str) -> str:
"""Return text with every non-alphanumeric character stripped.
Used for fragmentation-resistant matching: separator-injected secrets
(spaces, hyphens, dots inserted between characters) are identical to
their originals in alnum projection space.
"""
return "".join(c for c in text if c.isalnum())
def _find_partial_window(secret_alnum: str, text_alnum: str, min_len: int) -> int | None:
"""Return the earliest position in text_alnum holding a min_len-char window
that also appears in secret_alnum, or None.
The secret's set of min_len-grams is small (bounded by the secret length),
so building it once and sweeping the text a single time is O(len(text))
rather than the O(len(secret) * len(text)) of repeated substring searches
which matters because this runs per provisioned secret on every request
body. Coverage is unchanged: a hit still means at least min_len consecutive
alphanumeric characters of the secret leaked into the text.
"""
if len(secret_alnum) < min_len or len(text_alnum) < min_len:
return None
secret_grams = {
secret_alnum[i:i + min_len]
for i in range(len(secret_alnum) - min_len + 1)
}
for pos in range(len(text_alnum) - min_len + 1):
if text_alnum[pos:pos + min_len] in secret_grams:
return pos
return None
return variants
def scan_known_secrets(
@@ -236,134 +155,20 @@ def scan_known_secrets(
*,
location: str = "body",
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
if env is None:
return None
# Pre-compute alnum projection of the scan text once; reused per secret.
text_alnum: str | None = None
for key, value in env.items():
if not any(key.startswith(p) for p in sensitive_prefixes) or not value:
if not key.startswith("EGRESS_TOKEN_") or not value:
continue
# Pass 1: exact match across encoded variants (original behaviour).
approved_exact = False
for variant in _encoded_variants(value):
pos = text.find(variant)
if pos >= 0:
# The supervisor approves the exact encoded variant found
# (PRD 0062); a different encoding of the same secret is a
# fresh block.
if safe_tokens is not None and variant in safe_tokens:
approved_exact = True
continue
return ScanResult(
severity="block",
reason=f"provisioned secret from {key} found in {location}",
location=location,
context=_snippet(text, pos, pos + len(variant)),
matched=variant,
)
if approved_exact:
# Exact match was found and approved; projection passes would
# fire on the same value, so skip them for this secret.
continue
# Pass 2 & 3: fragmentation-resistant projection checks.
secret_alnum = _alnum_projection(value)
if len(secret_alnum) < _ALNUM_MIN_LEN:
continue
if text_alnum is None:
text_alnum = _alnum_projection(text)
# Pass 2: full alnum-projection exact match (catches separator injection).
pos2 = text_alnum.find(secret_alnum)
if pos2 >= 0:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(fragmented match — separator injection)"
),
location=location,
context=_snippet(text_alnum, pos2, pos2 + len(secret_alnum)),
)
# Pass 3: sliding-window partial match (catches chunked-substring leaks).
pos3 = _find_partial_window(secret_alnum, text_alnum, PARTIAL_MATCH_MIN_LEN)
if pos3 is not None:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(partial match — at least {PARTIAL_MATCH_MIN_LEN} consecutive "
f"alphanumeric chars)"
),
location=location,
context=_snippet(text_alnum, pos3, pos3 + PARTIAL_MATCH_MIN_LEN),
)
return None
# ---------------------------------------------------------------------------
# Entropy detector (warn-only)
# ---------------------------------------------------------------------------
# Sliding window size and step for the entropy scan.
ENTROPY_WINDOW = 64
ENTROPY_STEP = 32
# Bits-per-character threshold. Random ASCII printable ≈ 6.6 bits; random
# lowercase hex ≈ 4 bits; random base64url ≈ 6 bits. 5.5 sits above
# typical structured data (JSON, URLs) while staying below truly random
# content.
ENTROPY_BLOCK_THRESHOLD = 5.5
def _shannon_entropy(text: str) -> float:
if not text:
return 0.0
counts = Counter(text)
n = len(text)
return -sum((c / n) * log2(c / n) for c in counts.values())
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW,
threshold: float = ENTROPY_BLOCK_THRESHOLD,
) -> ScanResult | None:
"""Warn-only detector: flag windows of `window` chars with Shannon entropy
above `threshold` bits per character.
Never blocks; always returns severity='warn'. Disabled by default
routes must opt in via dlp.outbound_detectors=['entropy'].
"""
if not text:
return None
step = max(1, window // 2)
end = len(text)
# Scan overlapping windows; also check the final tail if shorter than window.
positions = list(range(0, end - window + 1, step))
if end < window:
positions = [0]
elif (end - window) % step != 0:
positions.append(end - window)
for i in positions:
chunk = text[i:i + window]
if _shannon_entropy(chunk) >= threshold:
return ScanResult(
severity="warn",
reason=f"high-entropy content in {location} (possible encrypted exfil)",
location=location,
context=_snippet(text, i, i + len(chunk)),
)
return None
@@ -392,52 +197,19 @@ JAILBREAK_PHRASES: tuple[re.Pattern[str], ...] = (
PROXIMITY_CHARS = 500
def _match_gap(a: re.Match[str], b: re.Match[str]) -> int:
"""Character gap between two match spans; 0 when they overlap or touch."""
return max(0, max(a.start(), b.start()) - min(a.end(), b.end()))
def _closest_pair(
a_matches: list[re.Match[str]],
b_matches: list[re.Match[str]],
*,
within: int | None = None,
) -> tuple[re.Match[str], re.Match[str]] | None:
"""Return the (a, b) pair with the smallest character gap, or None when
either list is empty.
Runs in O(n log n) sort + O(n) merge rather than the O(n*m) cross product:
both lists are sorted by start offset and swept with a two-pointer merge,
advancing whichever span ends first (it can only get farther from any
later span in the other list). This matters because the inputs are
attacker-controlled response-body matches that have already passed the
body-size cap, so the quadratic form is a latent DoS.
When `within` is set, returns as soon as a pair with gap <= within is
found: the only caller blocks on any pair inside the proximity threshold,
so the exact global minimum past that point doesn't change the decision.
"""
if not a_matches or not b_matches:
return None
a_sorted = sorted(a_matches, key=lambda m: m.start())
b_sorted = sorted(b_matches, key=lambda m: m.start())
i = j = 0
"""Return the pair (a, b) with the smallest character gap, or None."""
best: tuple[re.Match[str], re.Match[str]] | None = None
best_gap: int | None = None
while i < len(a_sorted) and j < len(b_sorted):
a, b = a_sorted[i], b_sorted[j]
gap = _match_gap(a, b)
for a in a_matches:
for b in b_matches:
gap = max(0, max(a.start(), b.start()) - min(a.end(), b.end()))
if best_gap is None or gap < best_gap:
best_gap = gap
best = (a, b)
if within is not None and gap <= within:
return best
# Advance the span that ends first; it cannot form a closer pair with
# any later (further-right) span from the other list.
if a.end() <= b.end():
i += 1
else:
j += 1
return best
@@ -447,9 +219,9 @@ def scan_naive_injection(text: str) -> ScanResult | None:
jailbreak_hits = [m for p in JAILBREAK_PHRASES for m in p.finditer(text)]
if disclosure_hits and jailbreak_hits:
pair = _closest_pair(disclosure_hits, jailbreak_hits, within=PROXIMITY_CHARS)
pair = _closest_pair(disclosure_hits, jailbreak_hits)
if pair is not None:
dist = _match_gap(pair[0], pair[1])
dist = max(0, max(pair[0].start(), pair[1].start()) - min(pair[0].end(), pair[1].end()))
if dist <= PROXIMITY_CHARS:
first = pair[0] if pair[0].start() <= pair[1].start() else pair[1]
return ScanResult(
@@ -493,14 +265,6 @@ _CRLF_ENCODED_RE = re.compile(r"%0[dD]%0[aA]", re.ASCII)
_CRLF_HEADER_INJECT_RE = re.compile(r"\r\n[A-Za-z][A-Za-z0-9\-]+\s*:", re.ASCII)
def strip_crlf(text: str) -> str:
"""Remove URL-encoded and literal CRLF injection sequences from a request
surface (PRD 0062 redact policy). Used to scrub the request line / headers
so the request can be forwarded instead of hard-blocked."""
text = _CRLF_ENCODED_RE.sub("", text)
return _CRLF_HEADER_INJECT_RE.sub(lambda m: m.group(0)[2:], text)
def scan_crlf_injection(text: str) -> ScanResult | None:
if _CRLF_ENCODED_RE.search(text):
return ScanResult(
@@ -516,20 +280,12 @@ def scan_crlf_injection(text: str) -> ScanResult | None:
__all__ = [
"ENTROPY_BLOCK_THRESHOLD",
"ENTROPY_WINDOW",
"ENTROPY_STEP",
"PARTIAL_MATCH_MIN_LEN",
"REDACT",
"SNIPPET_CONTEXT",
"TOKEN_PATTERNS",
"_alnum_projection",
"_shannon_entropy",
"redact_tokens",
"scan_crlf_injection",
"scan_entropy",
"scan_known_secrets",
"scan_naive_injection",
"scan_token_patterns",
"strip_crlf",
]
+11 -102
View File
@@ -10,14 +10,12 @@ specific and lives on concrete subclasses (see
from __future__ import annotations
import dataclasses
import secrets
from abc import ABC
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
from .egress_addon_core import (
ON_MATCH_REDACT,
HeaderMatch as CoreHeaderMatch,
MatchEntry as CoreMatchEntry,
PathMatch as CorePathMatch,
@@ -35,50 +33,6 @@ EGRESS_HOSTNAME = "egress"
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
EGRESS_ROUTES_FILENAME = Path(EGRESS_ROUTES_IN_CONTAINER).name
_CANARY_ENV_WORDS = (
"ACCORD",
"ANCHOR",
"ATLAS",
"CANON",
"CIPHER",
"EMBER",
"FALCON",
"HARBOR",
"LANTERN",
"MARBLE",
"NOVA",
"ORBIT",
"PIVOT",
"RADIUS",
"SUMMIT",
"VECTOR",
)
def _random_canary_env() -> str:
first = secrets.choice(_CANARY_ENV_WORDS)
remaining = tuple(word for word in _CANARY_ENV_WORDS if word != first)
second = secrets.choice(remaining)
return f"{first}_{second}_SECRET"
def egress_sidecar_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return sidecar env entries needed by egress across all backends."""
env: list[str] = []
if plan.routes:
env.extend(sorted(plan.token_env_map.keys()))
if plan.canary and plan.canary_env:
env.append(f"{plan.canary_env}={plan.canary}")
env.append(f"BOT_BOTTLE_SENSITIVE_PREFIXES={plan.canary_env}")
return tuple(env)
def egress_agent_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return agent-visible egress env entries shared by all backends."""
if plan.canary and plan.canary_env:
return (f"{plan.canary_env}={plan.canary}",)
return ()
@dataclass(frozen=True)
class EgressRoute(Route):
@@ -110,8 +64,6 @@ class EgressPlan:
mitmproxy_ca_host_path: Path = Path()
mitmproxy_ca_cert_only_host_path: Path = Path()
log: int = 0
canary: str = ""
canary_env: str = ""
def egress_manifest_routes(
@@ -143,7 +95,6 @@ def egress_manifest_routes(
git_fetch=r.GitFetch,
outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors,
outbound_on_match=r.OutboundOnMatch,
))
return tuple(out)
@@ -154,27 +105,12 @@ def egress_routes_for_bottle(
) -> tuple[EgressRoute, ...]:
manifest = egress_manifest_routes(bottle)
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
merged = list(_default_provider_on_match(provider_routes)) + [
merged = list(provider_routes) + [
r for r in manifest if r.host.lower() not in provisioned_hosts
]
return _assign_token_slots(merged)
def _default_provider_on_match(
provider_routes: tuple[EgressRoute, ...],
) -> tuple[EgressRoute, ...]:
"""Provider routes (the agent talking to its own LLM API) default to the
`redact` on-match policy (PRD 0062): high-volume conversation payloads are
the worst source of token-shaped false positives, so a match is scrubbed
and forwarded rather than hard-blocked or queued for the operator. A
provider that sets `outbound_on_match` explicitly keeps its choice."""
return tuple(
r if r.outbound_on_match
else dataclasses.replace(r, outbound_on_match=ON_MATCH_REDACT)
for r in provider_routes
)
def _assign_token_slots(
routes: list[EgressRoute],
) -> tuple[EgressRoute, ...]:
@@ -210,17 +146,6 @@ def egress_token_env_map(
return out
def _yaml_str_escape(s: str) -> str:
"""Escape a string for use inside a YAML double-quoted scalar."""
return (
s.replace("\\", "\\\\")
.replace('"', '\\"')
.replace("\n", "\\n")
.replace("\r", "\\r")
.replace("\t", "\\t")
)
def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields: dict[str, object] = {"host": r.host}
if r.auth_scheme and r.token_env:
@@ -252,11 +177,7 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields["matches"] = matches_data
if r.git_fetch:
fields["git"] = {"fetch": True}
if (
r.outbound_detectors is not None
or r.inbound_detectors is not None
or r.outbound_on_match
):
if r.outbound_detectors is not None or r.inbound_detectors is not None:
dlp: dict[str, object] = {}
if r.outbound_detectors is not None:
dlp["outbound_detectors"] = (
@@ -268,8 +189,6 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
False if not r.inbound_detectors
else list(r.inbound_detectors)
)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
fields["dlp"] = dlp
return fields
@@ -283,12 +202,12 @@ def _render_match_entry(entry: dict[str, object]) -> list[str]:
for pd in entry["paths"]: # type: ignore[union-attr]
pd_dict: dict[str, str] = pd # type: ignore[assignment]
if "type" in pd_dict:
lines.append(f' - type: "{_yaml_str_escape(pd_dict["type"])}"')
lines.append(f' value: "{_yaml_str_escape(pd_dict["value"])}"')
lines.append(f' - type: "{pd_dict["type"]}"')
lines.append(f' value: "{pd_dict["value"]}"')
else:
lines.append(f' - value: "{_yaml_str_escape(pd_dict["value"])}"')
lines.append(f' - value: "{pd_dict["value"]}"')
if "methods" in entry:
methods_str = ", ".join(f'"{_yaml_str_escape(m)}"' for m in entry["methods"]) # type: ignore[union-attr]
methods_str = ", ".join(f'"{m}"' for m in entry["methods"]) # type: ignore[union-attr]
prefix = " - " if first_key else " "
lines.append(f'{prefix}methods: [{methods_str}]')
first_key = False
@@ -298,8 +217,8 @@ def _render_match_entry(entry: dict[str, object]) -> list[str]:
first_key = False
for hd in entry["headers"]: # type: ignore[union-attr]
hd_dict: dict[str, str] = hd # type: ignore[assignment]
lines.append(f' - name: "{_yaml_str_escape(hd_dict["name"])}"')
lines.append(f' value: "{_yaml_str_escape(hd_dict["value"])}"')
lines.append(f' - name: "{hd_dict["name"]}"')
lines.append(f' value: "{hd_dict["value"]}"')
if first_key:
lines.append(" - {}")
return lines
@@ -319,10 +238,10 @@ def egress_render_routes(
return "\n".join(lines) + "\n"
for r in routes:
f = _route_to_yaml_fields(r)
lines.append(f' - host: "{_yaml_str_escape(str(f["host"]))}"')
lines.append(f' - host: "{f["host"]}"')
if "auth_scheme" in f:
lines.append(f' auth_scheme: "{_yaml_str_escape(str(f["auth_scheme"]))}"')
lines.append(f' token_env: "{_yaml_str_escape(str(f["token_env"]))}"')
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
lines.append(f' token_env: "{f["token_env"]}"')
if "matches" in f:
lines.append(" matches:")
for entry in f["matches"]: # type: ignore[union-attr]
@@ -341,8 +260,6 @@ def egress_render_routes(
elif isinstance(dv, list):
items_str = ", ".join(f'"{x}"' for x in dv)
lines.append(f" {dk}: [{items_str}]")
elif isinstance(dv, str):
lines.append(f' {dk}: "{_yaml_str_escape(dv)}"')
return "\n".join(lines) + "\n"
@@ -382,18 +299,12 @@ class Egress(ABC):
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
routes_path.write_text(egress_render_routes(routes, log=log))
routes_path.chmod(0o600)
# Generate a per-session fake secret under a plausible random env name.
# The sidecar marks that exact env name as sensitive for known-secret
# scanning; the agent receives the same name/value as exfil bait.
canary = secrets.token_urlsafe(32)
return EgressPlan(
slug=slug,
routes_path=routes_path,
routes=routes,
token_env_map=egress_token_env_map(routes),
log=log,
canary=canary,
canary_env=_random_canary_env(),
)
__all__ = [
@@ -408,7 +319,5 @@ __all__ = [
"egress_render_routes",
"egress_resolve_token_values",
"egress_routes_for_bottle",
"egress_agent_env_entries",
"egress_sidecar_env_entries",
"egress_token_env_map",
]
+22 -282
View File
@@ -5,7 +5,6 @@ egress container."""
from __future__ import annotations
import asyncio
import json
import os
import signal
@@ -17,15 +16,9 @@ from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
LOG_BLOCKS,
LOG_FULL,
DEFAULT_OUTBOUND_ON_MATCH,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
Config,
Route,
ScanResult,
build_inbound_scan_text,
build_outbound_scan_text,
build_token_allow_payload,
decide,
decide_git_fetch,
is_git_fetch_request,
@@ -39,55 +32,23 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
)
try:
from dlp_detectors import redact_tokens, strip_crlf # type: ignore[import-not-found]
from dlp_detectors import redact_tokens # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.dlp_detectors import ( # type: ignore[import-not-found]
redact_tokens,
strip_crlf,
)
try:
import supervise as _sv # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
from bot_bottle.dlp_detectors import redact_tokens # type: ignore[import-not-found]
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
INTROSPECT_HOST = "_egress.local"
# Seconds the egress proxy holds a token-blocked request open waiting for the
# operator's supervisor decision (PRD 0062), overridable via env.
DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS = 300.0
# Filesystem poll cadence while awaiting the operator's response.
TOKEN_ALLOW_POLL_INTERVAL_SECONDS = 0.5
# Fixed operator guidance attached to every token-allow proposal.
_TOKEN_ALLOW_JUSTIFICATION = (
"egress DLP blocked an outbound request carrying a detected token. "
"Approve only if this value is a false positive or a credential this "
"request legitimately needs; the value is then allowed for the life of "
"this bottle's egress proxy."
)
class EgressAddon:
def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.config: Config = Config(routes=())
# Tokens the operator has approved this session (PRD 0062). In-memory
# only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed.
self.safe_tokens: set[str] = set()
self._supervise_queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "").strip()
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True)
self._install_sighup()
def _supervise_available(self) -> bool:
return bool(self._supervise_queue_dir and self._supervise_slug)
def _reload(self, *, initial: bool = False) -> None:
try:
text = Path(self.routes_path).read_text(encoding="utf-8")
@@ -160,42 +121,31 @@ class EgressAddon:
)
def _log_request(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.request.headers.items()
if k.lower() != "authorization"
}
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": headers,
"body": body,
"headers": dict(flow.request.headers),
"body": flow.request.get_text(strict=False) or "",
})
+ "\n"
)
def _log_response(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.response.headers.items()
}
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": headers,
"body": body,
"headers": dict(flow.response.headers),
"body": flow.response.get_text(strict=False) or "",
})
+ "\n"
)
async def request(self, flow: http.HTTPFlow) -> None:
def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?")
if flow.request.pretty_host == INTROSPECT_HOST:
@@ -207,11 +157,21 @@ class EgressAddon:
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
if route is not None:
if not await self._handle_outbound_dlp(flow, route):
body = flow.request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
outbound_scan_headers(route, dict(flow.request.headers)),
body,
)
dlp_result = scan_outbound(route, scan_text, os.environ)
if dlp_result is not None and dlp_result.severity == "block":
ctx = self._req_ctx(flow)
if dlp_result.context:
ctx = {**ctx, "context": dlp_result.context}
self._block(flow, f"egress DLP: {dlp_result.reason}", ctx=ctx)
return
# The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on.
request_path, _, query = flow.request.path.partition("?")
if is_git_push_request(request_path, query):
self._block(
@@ -261,202 +221,6 @@ class EgressAddon:
if self.config.log >= LOG_FULL:
self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
ctx = self._req_ctx(flow)
if result.context:
ctx = {**ctx, "context": result.context}
self._block(flow, f"egress DLP: {result.reason}", ctx=ctx)
async def _handle_outbound_dlp(
self,
flow: http.HTTPFlow,
route: Route,
) -> bool:
"""Scan the outbound request and apply the route's on-match policy
(PRD 0062). Returns True if the request may be forwarded, False if a
403 response has been written to `flow`.
Loops so the supervise policy can re-scan after each approval a
second, un-approved token in the same request is still caught."""
while True:
request_path, _, query = flow.request.path.partition("?")
body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, body,
)
# CRLF is scanned only over the request line + headers, never the
# body (see scan_outbound) — a body is not an injection vector.
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(
route, scan_text, os.environ,
safe_tokens=self.safe_tokens, crlf_text=crlf_text,
)
if result is None or result.severity != "block":
return True
policy = route.outbound_on_match or DEFAULT_OUTBOUND_ON_MATCH
# redact scrubs every detection (tokens and structural CRLF) and
# forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route):
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_redacted",
"reason": f"egress DLP: {result.reason}",
**self._req_ctx(flow),
}) + "\n")
return True
self._block(
flow,
f"egress DLP: {result.reason}; redaction could not remove "
"all matches (e.g. a match in the hostname)",
ctx=self._req_ctx(flow),
)
return False
# Structural blocks (CRLF, no safelist-able value) cannot be
# supervised — there is nothing to approve and remember — so under
# block/supervise they are a hard 403.
if policy == ON_MATCH_BLOCK or not result.matched:
self._block_dlp(flow, result)
return False
# supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available():
self._block_dlp(flow, result)
return False
approved = await self._supervise_token_block(flow, request_path, result)
if not approved:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound(self, flow: http.HTTPFlow, route: Route) -> bool:
"""Scrub detected tokens (and CRLF injection sequences) from the mutable
request surfaces (body, headers, path/query) and re-scan. Returns True
if the request is now clean; False if a block-severity match remains on
a surface redaction cannot rewrite (the hostname) so the caller fails
closed."""
body = flow.request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=os.environ)
if redacted_body != body:
flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()):
if name.lower() == "host":
continue # routing-critical; never a legitimate token
redacted = strip_crlf(redact_tokens(value, env=os.environ))
if redacted != value:
flow.request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=os.environ))
if redacted_path != flow.request.path:
flow.request.path = redacted_path
request_path, _, query = flow.request.path.partition("?")
new_body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, new_body,
)
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(route, scan_text, os.environ, crlf_text=crlf_text)
return result is None or result.severity != "block"
async def _supervise_token_block(
self,
flow: http.HTTPFlow,
request_path: str,
result: ScanResult,
) -> bool:
"""Route a token DLP block to the operator's supervisor queue and wait.
Returns True if the operator approved (the matched value is added to
`self.safe_tokens` and the caller re-scans); False if the request must
be blocked (a 403 response has been written to `flow`)."""
host = flow.request.pretty_host
payload = build_token_allow_payload(
redact_tokens(host, env=os.environ),
flow.request.method,
redact_tokens(request_path, env=os.environ),
result,
)
proposal = _sv.Proposal.new(
bottle_slug=self._supervise_slug,
tool=_sv.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file=payload,
justification=_TOKEN_ALLOW_JUSTIFICATION,
current_file_hash=_sv.sha256_hex(payload),
)
queue_dir = Path(self._supervise_queue_dir)
try:
_sv.write_proposal(queue_dir, proposal)
except OSError as e:
sys.stderr.write(
f"egress: could not queue token-allow proposal: {e}; "
"blocking request\n"
)
self._block(flow, f"egress DLP: {result.reason}", ctx=self._req_ctx(flow))
return False
sys.stderr.write(json.dumps({
"event": "egress_token_supervise",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
response = await self._await_token_response(queue_dir, proposal.id)
_sv.archive_proposal(queue_dir, proposal.id)
if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
):
self.safe_tokens.add(result.matched)
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_token_allowed",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
return True
if response is None:
reason = (
f"egress DLP: {result.reason}; supervisor approval timed out "
f"after {self._token_allow_timeout:g}s"
)
else:
reason = f"egress DLP: {result.reason}; supervisor rejected the request"
self._block(flow, reason, ctx=self._req_ctx(flow))
return False
async def _await_token_response(
self,
queue_dir: Path,
proposal_id: str,
) -> "_sv.Response | None":
"""Poll the queue dir for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout."""
loop = asyncio.get_running_loop()
deadline = loop.time() + self._token_allow_timeout
while True:
try:
return _sv.read_response(queue_dir, proposal_id)
except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed.
pass
if loop.time() >= deadline:
return None
await asyncio.sleep(TOKEN_ALLOW_POLL_INTERVAL_SECONDS)
def response(self, flow: http.HTTPFlow) -> None:
"""DLP inbound scan on response headers and body."""
route = match_route(self.config.routes, flow.request.pretty_host)
@@ -508,12 +272,7 @@ class EgressAddon:
message = flow.websocket.messages[-1] # type: ignore[union-attr]
content = message.content.decode("utf-8", errors="replace")
if message.from_client:
# A WebSocket data frame is not an HTTP request line, so CRLF is
# not an injection vector here — scan only for credential leakage.
result = scan_outbound(
route, content, os.environ,
safe_tokens=self.safe_tokens, crlf_text="",
)
result = scan_outbound(route, content, os.environ)
if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
@@ -527,23 +286,4 @@ class EgressAddon:
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
def _token_allow_timeout_from_env(env: "os._Environ[str]") -> float:
"""Read EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS; fall back to the default on an
unset or invalid value (a bad value should not wedge egress at boot)."""
raw = env.get("EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS", "").strip()
if not raw:
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
try:
value = float(raw)
except ValueError:
value = 0.0
if value <= 0:
sys.stderr.write(
"egress: invalid EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS="
f"{raw!r}; using default {DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS:g}s\n"
)
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
return value
addons = [EgressAddon()]
+75 -112
View File
@@ -21,32 +21,6 @@ try:
except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset
# DLP detector-config parsing lives in a sibling module (also flat-bundled
# into the sidecar — see Dockerfile.sidecars). Re-exported below so existing
# `from egress_addon_core import ON_MATCH_*` callers keep working.
try:
from egress_dlp_config import ( # type: ignore[import-not-found]
DEFAULT_OUTBOUND_ON_MATCH,
INBOUND_DETECTOR_NAMES,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
ON_MATCH_SUPERVISE,
OUTBOUND_DETECTOR_NAMES,
OUTBOUND_ON_MATCH_VALUES,
parse_dlp_block,
)
except ImportError: # pragma: no cover - host-side path
from .egress_dlp_config import (
DEFAULT_OUTBOUND_ON_MATCH,
INBOUND_DETECTOR_NAMES,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
ON_MATCH_SUPERVISE,
OUTBOUND_DETECTOR_NAMES,
OUTBOUND_ON_MATCH_VALUES,
parse_dlp_block,
)
# ---------------------------------------------------------------------------
# Match types (Gateway API HTTPRoute vocabulary, PRD 0053)
@@ -60,6 +34,9 @@ VALID_METHODS = frozenset({
"CONNECT",
})
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
@dataclass(frozen=True)
class PathMatch:
@@ -92,8 +69,6 @@ class Route:
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
# "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES.
outbound_on_match: str = ""
LOG_OFF = 0 # no logging
@@ -120,11 +95,6 @@ class ScanResult:
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
matched: str = ""
# ---------------------------------------------------------------------------
@@ -244,6 +214,61 @@ def _parse_match_entry(idx: int, k: int, raw: object) -> MatchEntry:
return MatchEntry(paths=paths, methods=methods, headers=headers)
def _parse_detectors(
idx: int,
host: str,
raw_dict: dict[str, object],
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
"""Parse the optional `dlp` block on a route, returning
(outbound_detectors, inbound_detectors)."""
dlp_raw = raw_dict.get("dlp")
if dlp_raw is None:
return None, None
label = f"route[{idx}] ({host})"
if not isinstance(dlp_raw, dict):
raise ValueError(f"{label}: 'dlp' must be an object")
dlp = typing.cast(dict[str, object], dlp_raw)
def _parse_detector_field(
field: str,
valid_names: frozenset[str],
) -> tuple[str, ...] | None:
val = dlp.get(field)
if val is None:
return None
if val is False:
return ()
if not isinstance(val, list):
raise ValueError(
f"{label}: dlp.{field} must be false, a list, or omitted"
)
items = typing.cast(list[object], val)
names: list[str] = []
for j, item in enumerate(items):
if not isinstance(item, str):
raise ValueError(
f"{label}: dlp.{field}[{j}] must be a string"
)
if item not in valid_names:
raise ValueError(
f"{label}: dlp.{field}[{j}] {item!r} is not a valid "
f"detector name; valid names: {', '.join(sorted(valid_names))}"
)
names.append(item)
return tuple(names)
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
for k in dlp:
if k not in ("outbound_detectors", "inbound_detectors"):
raise ValueError(
f"{label}: dlp has unknown key {k!r}; accepted keys "
f"are 'outbound_detectors', 'inbound_detectors'"
)
return outbound, inbound
def parse_routes(payload: object) -> tuple[Route, ...]:
if not isinstance(payload, dict):
raise ValueError("routes payload: top-level must be an object")
@@ -312,7 +337,7 @@ def _parse_one(idx: int, raw: object) -> Route:
)
# dlp detectors
outbound_detectors, inbound_detectors, outbound_on_match = parse_dlp_block(
outbound_detectors, inbound_detectors = _parse_detectors(
idx, host, raw_dict,
)
@@ -331,7 +356,6 @@ def _parse_one(idx: int, raw: object) -> Route:
git_fetch=git_fetch,
outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match,
)
@@ -380,13 +404,20 @@ def route_to_yaml_dict(r: Route) -> dict[str, object]:
dlp["outbound_detectors"] = list(r.outbound_detectors)
if r.inbound_detectors is not None:
dlp["inbound_detectors"] = list(r.inbound_detectors)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
if dlp:
d["dlp"] = dlp
return d
def load_routes(text: str) -> tuple[Route, ...]:
"""Parse YAML text → routes."""
try:
payload = parse_yaml_subset(text)
except YamlSubsetError as e:
raise ValueError(f"routes payload: invalid YAML: {e}") from e
return parse_routes(payload)
def parse_config(payload: object) -> "Config":
"""Parse a full egress config payload (top-level log level + routes)."""
if not isinstance(payload, dict):
@@ -659,103 +690,43 @@ def scan_outbound(
route: Route,
body: str | bytes,
environ: typing.Mapping[str, str],
*,
safe_tokens: typing.AbstractSet[str] | None = None,
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
except ImportError: # pragma: no cover - host-side path
from .dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
# Binary bodies: latin-1 is a bijective byte↔codepoint mapping that
# preserves every byte value, so ASCII-range secret strings remain
# findable by str.find / regex. Prefer strict UTF-8 for valid text bodies.
if isinstance(body, bytes):
try:
text = body.decode("utf-8")
except UnicodeDecodeError:
text = body.decode("latin-1")
else:
text = body
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
# CRLF injection is only an attack in the request line + headers, never the
# body: an HTTP body is delimited by Content-Length, so CRLF bytes there
# cannot split the request. Scanning the body produces false positives on
# legitimate form-encoded / multi-line content. Callers pass the
# body-excluded surfaces as `crlf_text`; `None` falls back to the full text
# for backward-compatible callers (host-side tests, websocket frames).
crlf_target = text if crlf_text is None else crlf_text
result = scan_crlf_injection(crlf_target)
# CRLF injection is never legitimate — runs unconditionally, not gated
# by outbound_detectors config.
result = scan_crlf_injection(text)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "token_patterns"):
result = scan_token_patterns(text, location="body", safe_tokens=safe_tokens)
result = scan_token_patterns(text, location="body")
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "known_secrets"):
# BOT_BOTTLE_SENSITIVE_PREFIXES lets operators add extra env prefixes
# beyond EGRESS_TOKEN_* without changing the manifest schema.
extra_raw = environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "")
extra = tuple(p for p in extra_raw.split(",") if p)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
result = scan_known_secrets(
text, location="body", env=environ,
sensitive_prefixes=sensitive_prefixes, safe_tokens=safe_tokens,
)
if result is not None:
return result
# Entropy scanning requires explicit opt-in: it is NOT part of the
# default "all detectors" set because it produces false positives on
# legitimate base64 / binary payloads. Routes must list "entropy" in
# dlp.outbound_detectors to enable it.
if (
route.outbound_detectors is not None
and "entropy" in route.outbound_detectors
):
result = scan_entropy(text, location="body")
result = scan_known_secrets(text, location="body", env=environ)
if result is not None:
return result
return None
def build_token_allow_payload(
host: str,
method: str,
path: str,
result: ScanResult,
) -> str:
"""Render the human-readable supervisor proposal body for an outbound
token block (PRD 0062). Carries the host/method/path, the detector
reason, and the redacted context snippet never the raw token value."""
lines = [
"egress blocked an outbound request carrying a detected token",
f"host: {host}",
f"method: {method}",
f"path: {path}",
f"detector: {result.reason}",
]
if result.context:
lines.append(f"context: {result.context}")
return "\n".join(lines) + "\n"
def scan_inbound(
route: Route,
body: str | bytes,
@@ -780,14 +751,6 @@ __all__ = [
"route_to_yaml_dict",
"LOG_FULL",
"LOG_OFF",
"ON_MATCH_BLOCK",
"ON_MATCH_REDACT",
"ON_MATCH_SUPERVISE",
"OUTBOUND_ON_MATCH_VALUES",
"DEFAULT_OUTBOUND_ON_MATCH",
"OUTBOUND_DETECTOR_NAMES",
"INBOUND_DETECTOR_NAMES",
"parse_dlp_block",
"Config",
"Decision",
"HeaderMatch",
@@ -797,13 +760,13 @@ __all__ = [
"ScanResult",
"build_inbound_scan_text",
"build_outbound_scan_text",
"build_token_allow_payload",
"decide",
"decide_git_fetch",
"evaluate_matches",
"is_git_push_request",
"is_git_fetch_request",
"load_config",
"load_routes",
"match_route",
"outbound_scan_headers",
"parse_config",
-92
View File
@@ -1,92 +0,0 @@
"""DLP detector-config parsing for egress routes (PRD 0053, PRD 0062).
A route's optional `dlp:` block names which outbound/inbound detectors run
and what the proxy does when an outbound detector matches a token
(`outbound_on_match`). This module owns parsing and validating that block,
kept apart from the request-time scan/decision flow in `egress_addon_core`
so each half reads top-to-bottom without scrolling past the other.
Stdlib-only; ships flat into the sidecar bundle image alongside
`egress_addon_core.py` see `Dockerfile.sidecars`."""
from __future__ import annotations
import typing
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets", "entropy"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# Per-route policy for what the proxy does when an outbound DLP detector
# matches a token (PRD 0062).
ON_MATCH_BLOCK = "block" # hard 403, never overridable
ON_MATCH_REDACT = "redact" # scrub the matched value, forward the request
ON_MATCH_SUPERVISE = "supervise" # queue for operator approval, hold the request
OUTBOUND_ON_MATCH_VALUES = (ON_MATCH_BLOCK, ON_MATCH_REDACT, ON_MATCH_SUPERVISE)
# Unset resolves to supervise (fall back to block when supervise is not wired).
DEFAULT_OUTBOUND_ON_MATCH = ON_MATCH_SUPERVISE
def parse_dlp_block(
idx: int,
host: str,
raw_dict: dict[str, object],
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
"""Parse the optional `dlp` block on a route, returning
(outbound_detectors, inbound_detectors, outbound_on_match)."""
dlp_raw = raw_dict.get("dlp")
if dlp_raw is None:
return None, None, ""
label = f"route[{idx}] ({host})"
if not isinstance(dlp_raw, dict):
raise ValueError(f"{label}: 'dlp' must be an object")
dlp = typing.cast(dict[str, object], dlp_raw)
def _parse_detector_field(
field: str,
valid_names: frozenset[str],
) -> tuple[str, ...] | None:
val = dlp.get(field)
if val is None:
return None
if val is False:
return ()
if not isinstance(val, list):
raise ValueError(
f"{label}: dlp.{field} must be false, a list, or omitted"
)
items = typing.cast(list[object], val)
names: list[str] = []
for j, item in enumerate(items):
if not isinstance(item, str):
raise ValueError(
f"{label}: dlp.{field}[{j}] must be a string"
)
if item not in valid_names:
raise ValueError(
f"{label}: dlp.{field}[{j}] {item!r} is not a valid "
f"detector name; valid names: {', '.join(sorted(valid_names))}"
)
names.append(item)
return tuple(names)
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = dlp.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ValueError(
f"{label}: dlp.outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in dlp:
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ValueError(
f"{label}: dlp has unknown key {k!r}; accepted keys "
f"are 'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
)
return outbound, inbound, on_match
+571 -47
View File
@@ -27,36 +27,51 @@ dataclass (`GitGatePlan`). The sidecar's start/stop lifecycle is
backend-specific and lives on concrete subclasses (see
`bot_bottle/backend/docker/git_gate.py`)."""
from __future__ import annotations
import dataclasses
import os
import shlex
from abc import ABC
from dataclasses import dataclass
from pathlib import Path
from .manifest import ManifestBottle
from .log import info
from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the sidecar bundle. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate"
# Bound half-open git client sessions. If an agent/tool runner is
# interrupted during push, git daemon should reap the receive-pack
# child instead of keeping the gate wedged indefinitely.
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
@dataclass(frozen=True)
class GitGateUpstream:
"""One bare repo on the gate. `name` drives the bare-repo path
(`/git/<name>.git`), the agent's URL after insteadOf rewrite
(`git://<gate>/<name>.git`), and the per-upstream credential
paths inside the gate (`/git-gate/creds/<name>-key` and
`/git-gate/creds/<name>-known_hosts`).
`identity_file` is the host-side absolute path the gate's start
step will docker-cp into the container. `known_host_key` is the
KnownHostKey string from the manifest; the gate's start step
materialises it into a known_hosts file if non-empty.
the gate credential paths inside the running sidecar."""
name: str
upstream_url: str
upstream_host: str
upstream_port: str
identity_file: str
known_host_key: str
known_hosts_file: Path = Path()
# Rendering and the deploy-key lifecycle live in sibling modules; the
# names are re-exported here (see __all__) so existing
# `from bot_bottle.git_gate import …` callers are unchanged.
from .git_gate_render import (
GIT_GATE_HOSTNAME,
GIT_GATE_TIMEOUT_SECS,
GitGateUpstream,
git_gate_known_hosts_line,
git_gate_render_access_hook,
git_gate_render_entrypoint,
git_gate_render_gitconfig,
git_gate_render_hook,
git_gate_upstreams_for_bottle,
_gitconfig_validate_value,
)
from .git_gate_provision import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
_provision_dynamic_key,
_resolve_identity_file,
)
@dataclass(frozen=True)
class GitGatePlan:
@@ -81,6 +96,529 @@ class GitGatePlan:
egress_network: str = ""
def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstream, ...]:
"""Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name
validation already ran in `manifest.ManifestBottle.from_dict`."""
return tuple(
GitGateUpstream(
name=e.Name,
upstream_url=e.Upstream,
upstream_host=e.UpstreamHost,
upstream_port=e.UpstreamPort,
identity_file=e.IdentityFile,
known_host_key=e.KnownHostKey,
)
for e in bottle.git
)
def git_gate_render_gitconfig(
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
"""Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
exposed for tests + reuse across backends.
`gate_host` is the part of the URL between `<scheme>://` and the
repo path backends differ here:
- docker: `git-gate` (the short network alias)
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
TSI-allowlisted guest)
Empty `entries` returns an empty string so callers can no-op
cleanly without conditional formatting at the call site."""
if not entries:
return ""
out = [
"# bot-bottle git-gate (PRD 0008): every git operation against\n",
"# a declared upstream routes through the gate, which mirrors\n",
"# the upstream bidirectionally (gitleaks-scanned push;\n",
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
]
for entry in entries:
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
out.append(f"\tinsteadOf = {entry.Upstream}\n")
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
port = (
f":{entry.UpstreamPort}"
if entry.UpstreamPort and entry.UpstreamPort != "22"
else ""
)
alias = (
f"ssh://{entry.UpstreamUser}@{entry.RemoteKey}{port}/"
f"{entry.UpstreamPath}"
)
out.append(f"\tinsteadOf = {alias}\n")
return "".join(out)
def git_gate_known_hosts_line(host: str, port: str, key: str) -> str:
"""Format `host[:port] key` for OpenSSH's known_hosts. Non-default
ports use the bracketed `[host]:port` form (the form OpenSSH writes
on disk for hosts reached via a non-22 port)."""
if port and port != "22":
target = f"[{host}]:{port}"
else:
target = host
return f"{target} {key}\n"
def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
"""Posix-sh entrypoint. One `init_repo` call per upstream, then
`exec git daemon`. The function reads
`/git-gate/creds/<name>-{key,known_hosts}` (bind-mounted into
the bundle by the renderer) and wires them into each bare repo's
config; the access-hook + pre-receive hook pick those paths up
at fetch / push time."""
lines = [
"#!/bin/sh",
"set -eu",
"",
"init_repo() {",
" name=$1",
" upstream_url=$2",
" keyfile=/git-gate/creds/${name}-key",
" hostsfile=/git-gate/creds/${name}-known_hosts",
"",
# `|| true`: PRD 0018 chunk 3+ bind-mounts these RO from the
# host, so chmod-syscalls fail with EROFS. The files already
# have the right perms on the host (SSH requires 0600 to load
# the key in the first place), so the chmod is best-effort
# cleanup for the legacy docker-cp path where the file
# landed at the host's umask perms.
" chmod 600 \"$keyfile\" 2>/dev/null || true",
" if [ -f \"$hostsfile\" ]; then",
" chmod 600 \"$hostsfile\" 2>/dev/null || true",
" fi",
"",
" repo=/git/${name}.git",
" if [ ! -d \"$repo\" ]; then",
" git init --bare \"$repo\" >/dev/null",
# --mirror=fetch sets remote.origin.fetch = +refs/*:refs/* so",
# a later `git fetch origin` mirrors the upstream's full ref",
# graph (heads, tags, notes) into the bare repo at canonical",
# paths. It does NOT set remote.origin.mirror=true, so an",
# explicit `git push origin <ref>:<ref>` still pushes one ref.",
" git -C \"$repo\" remote add --mirror=fetch origin \"$upstream_url\"",
" fi",
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
" git -C \"$repo\" config receive.advertisePushOptions true",
" git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}",
"",
"mkdir -p /git",
]
for u in upstreams:
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
lines.extend([
"",
"exec git daemon \\",
" --reuseaddr \\",
f" --timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
" --base-path=/git \\",
" --export-all \\",
" --enable=receive-pack \\",
" --access-hook=/etc/git-gate/access-hook \\",
" --verbose",
])
return "\n".join(lines) + "\n"
def git_gate_render_hook() -> str:
"""The shared pre-receive hook: gitleaks-scan all incoming refs,
then forward each accepted ref to the real upstream (`origin`)
using the per-repo credential. Failure in either phase aborts
the push so the agent sees a real rejection. POSIX sh.
Two phases (scan all, then push all) keeps a hit on ref N from
half-pushing refs 1..N-1; both phases re-read stdin from a temp
file because pre-receive's stdin is a one-shot stream."""
return r"""#!/bin/sh
# git-gate pre-receive (PRD 0008). Stdin: <old> <new> <ref> per line.
set -u
refs_file=$(mktemp)
trap 'rm -f "$refs_file"' EXIT
cat > "$refs_file"
zero=0000000000000000000000000000000000000000
supervise_gitleaks_allow() {
log_opts=$1
ref=$2
report_file=$(mktemp)
if ! gitleaks git \
--log-opts="$log_opts" \
--no-banner \
--redact \
--ignore-gitleaks-allow \
--report-format=json \
--report-path="$report_file" \
--exit-code 0 \
1>&2; then
rm -f "$report_file"
echo "git-gate: gitleaks inline-suppression scan failed for $ref" >&2
return 1
fi
proposal_id=$(
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
import datetime
import hashlib
import json
import os
import sys
import uuid
from pathlib import Path
report_path = Path(sys.argv[1])
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
if not queue_dir or not slug:
sys.exit(2)
try:
raw = json.loads(report_path.read_text() or "[]")
except json.JSONDecodeError:
sys.exit(3)
if not isinstance(raw, list):
sys.exit(3)
if not raw:
sys.exit(0)
ref = os.environ.get("GITLEAKS_ALLOW_REF", "")
lines = [
"gitleaks inline suppression requires supervisor approval",
f"ref: {ref}",
"",
]
for i, finding in enumerate(raw, 1):
if not isinstance(finding, dict):
continue
file_path = finding.get("File", "")
line_no = finding.get("StartLine", finding.get("Line", ""))
rule_id = finding.get("RuleID", "")
commit = finding.get("Commit", "")
line = finding.get("Line", "")
lines.extend([
f"finding {i}:",
f" file: {file_path}",
f" line: {line_no}",
f" rule: {rule_id}",
f" commit: {commit}",
f" code: {line}",
"",
])
payload = "\n".join(lines).rstrip() + "\n"
proposal_id = str(uuid.uuid4())
proposal = {
"id": proposal_id,
"bottle_slug": slug,
"tool": "gitleaks-allow",
"proposed_file": payload,
"justification": (
"git-gate found gitleaks findings hidden by # gitleaks:allow; "
"approve only for dummy test fixtures or confirmed false positives"
),
"arrival_timestamp": datetime.datetime.now(
datetime.timezone.utc
).isoformat(),
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(),
}
queue = Path(queue_dir)
queue.mkdir(parents=True, exist_ok=True)
path = queue / f"{proposal_id}.proposal.json"
tmp = path.with_suffix(path.suffix + ".tmp")
with tmp.open("w", encoding="utf-8") as f:
json.dump(proposal, f, indent=2)
f.write("\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
print(proposal_id)
PY
)
rc=$?
rm -f "$report_file"
if [ "$rc" -eq 0 ] && [ -z "$proposal_id" ]; then
return 0
fi
if [ "$rc" -ne 0 ]; then
echo "git-gate: cannot route # gitleaks:allow finding to supervisor; refusing push" >&2
return 1
fi
queue_dir=${SUPERVISE_QUEUE_DIR:-}
response_file="$queue_dir/${proposal_id}.response.json"
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
case "$timeout" in
''|*[!0-9]*)
echo "git-gate: invalid SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS=$timeout" >&2
return 1
;;
esac
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
waited=0
while [ "$waited" -lt "$timeout" ]; do
if [ -f "$response_file" ]; then
status=$(python3 - "$response_file" <<'PY'
import json
import sys
try:
with open(sys.argv[1], encoding="utf-8") as f:
raw = json.load(f)
except (OSError, json.JSONDecodeError):
sys.exit(1)
status = raw.get("status")
if not isinstance(status, str):
sys.exit(1)
print(status)
PY
) || status=""
case "$status" in
approved|modified)
mkdir -p "$queue_dir/processed"
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
return 0
;;
rejected)
echo "git-gate: supervisor rejected # gitleaks:allow for $ref" >&2
return 1
;;
*)
echo "git-gate: invalid supervisor response for # gitleaks:allow" >&2
return 1
;;
esac
fi
sleep 1
waited=$((waited + 1))
done
echo "git-gate: supervisor approval timed out for # gitleaks:allow; refusing push" >&2
return 1
}
# Phase 1: gitleaks scan each ref's incoming commits.
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
[ "$new" = "$zero" ] && continue
if [ "$old" = "$zero" ]; then
# New ref: scan only the commits this push introduces — those
# reachable from $new but not from any ref the gate already has.
# Everything already on the gate arrived via upstream mirror-fetch
# or a previously gitleaks-scanned push, so it's already-upstream
# or already-scanned; re-scanning it (the old `$new` full-ancestry
# range) only resurfaces historical findings and blocks every new
# branch. See PRD 0028 / issue #106.
log_opts="$new --not --all"
else
log_opts="$old..$new"
fi
echo "git-gate: gitleaks scanning $ref ($log_opts)" >&2
if ! gitleaks git --log-opts="$log_opts" --no-banner --redact 1>&2; then
echo "git-gate: gitleaks rejected push to $ref" >&2
exit 1
fi
if ! supervise_gitleaks_allow "$log_opts" "$ref"; then
exit 1
fi
done < "$refs_file"
# Phase 2: forward each ref to the upstream (`origin`, configured
# in the entrypoint via `git remote add --mirror=fetch`).
keyfile=$(git config --get git-gate.identityFile)
hostsfile=$(git config --get git-gate.knownHosts)
if [ ! -f "$hostsfile" ]; then
echo "git-gate: no KnownHostKey configured for this upstream; refusing to push" >&2
echo "git-gate: add KnownHostKey to the bottle.git entry and restart the bottle" >&2
exit 1
fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
push_option_count=${GIT_PUSH_OPTION_COUNT:-0}
case "$push_option_count" in
''|*[!0-9]*)
echo "git-gate: invalid GIT_PUSH_OPTION_COUNT=$push_option_count" >&2
exit 1
;;
esac
set --
i=0
while [ "$i" -lt "$push_option_count" ]; do
opt=$(printenv "GIT_PUSH_OPTION_$i" || :)
set -- "$@" --push-option="$opt"
i=$((i + 1))
done
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
if [ "$new" = "$zero" ]; then
refspec=":$ref"
elif [ "$old" != "$zero" ] && ! git merge-base --is-ancestor "$old" "$new" 2>/dev/null; then
refspec="+$new:$ref"
else
refspec="$new:$ref"
fi
echo "git-gate: forwarding $ref to origin" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git push "$@" origin "$refspec" 1>&2; then
echo "git-gate: upstream push failed for $ref" >&2
exit 1
fi
done < "$refs_file"
exit 0
"""
def git_gate_render_access_hook() -> str:
"""`git daemon --access-hook` script. Runs before each protocol
service; for `upload-pack` (fetch / clone / ls-remote / pull) it
refreshes the bare repo from upstream first, so the response
reflects upstream's current state. For other services (notably
`receive-pack`) it returns 0 immediately and lets the existing
pre-receive hook gate the operation. POSIX sh.
The hook receives:
$1 service name (`upload-pack`, `receive-pack`, ...)
$2 absolute path to the resolved repo
$3 client hostname (unused)
$4 client tcp address (unused)
Fail-closed on upstream errors: the agent's fetch fails too,
so it never silently sees stale data matches the PRD's
'equivalent to operations against the upstream' contract."""
return r"""#!/bin/sh
# git-gate access-hook (PRD 0008). $1=service $2=repo $3=host $4=peer
set -u
service=$1
repo_dir=$2
# Push path keeps its own gating in pre-receive (gitleaks +
# forward). Only refresh-from-upstream on fetch operations.
if [ "$service" != "upload-pack" ]; then
exit 0
fi
keyfile=$(git -C "$repo_dir" config --get git-gate.identityFile 2>/dev/null || true)
hostsfile=$(git -C "$repo_dir" config --get git-gate.knownHosts 2>/dev/null || true)
if [ -z "$keyfile" ] || [ ! -f "$hostsfile" ]; then
echo "git-gate: missing credentials for $repo_dir; refusing fetch" >&2
exit 1
fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
echo "git-gate: refreshing $repo_dir from upstream" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" fetch origin --prune >&2; then
echo "git-gate: upstream fetch failed for $repo_dir; refusing to serve stale data" >&2
exit 1
fi
# Sync the bare repo's HEAD to upstream's HEAD on the first fetch
# (when it still points at the `git init --bare` default of
# refs/heads/master and upstream uses something else, the cloned
# checkout would fail with "remote HEAD refers to nonexistent ref").
# Costs one extra ls-remote on first fetch only; subsequent fetches
# skip the branch. If upstream's default branch changes after the
# gate has cached it, restart the bottle to resync.
if ! git -C "$repo_dir" rev-parse --verify HEAD >/dev/null 2>&1; then
upstream_head=$(GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" \
ls-remote --symref origin HEAD 2>/dev/null \
| awk '/^ref:/ {print $2; exit}')
if [ -n "$upstream_head" ]; then
git -C "$repo_dir" symbolic-ref HEAD "$upstream_head" || true
fi
fi
exit 0
"""
def _provision_dynamic_key(
entry: ManifestGitEntry,
slug: str,
stage_dir: Path,
) -> str:
"""Generate a fresh ed25519 keypair, register the public half with
the forge, and persist the private key + key ID under `stage_dir`.
Returns the host-side path to the private key file so the caller
can inject it into the GitGateUpstream as `identity_file`."""
from .deploy_key_provisioner import get_provisioner
pk = entry.Key
token = os.environ.get(pk.forge_token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url)
owner_repo = entry.UpstreamPath
if owner_repo.endswith(".git"):
owner_repo = owner_repo[:-4]
title = f"bot-bottle:{slug}:{entry.Name}"
info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]")
key_id, private_key_bytes = provisioner.create(owner_repo, title)
key_file = stage_dir / f"{entry.Name}-key"
key_file.write_bytes(private_key_bytes)
key_file.chmod(0o600)
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
id_file.write_text(key_id)
id_file.chmod(0o600)
info(f"provisioned deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
return str(key_file)
def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) -> None:
"""Revoke all deploy keys provisioned for `bottle` during prepare.
Called at teardown after containers stop. Raises if any revocation
fails a stranded key is a security concern that the operator must
address manually."""
from .deploy_key_provisioner import get_provisioner
for entry in bottle.git:
if entry.Key.provider != "gitea":
continue
pk = entry.Key
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
if not id_file.exists():
continue
key_id = id_file.read_text().strip()
token = os.environ.get(pk.forge_token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set;"
f" cannot revoke deploy key {key_id}"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url)
owner_repo = entry.UpstreamPath
if owner_repo.endswith(".git"):
owner_repo = owner_repo[:-4]
info(f"revoking deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
provisioner.delete(owner_repo, key_id)
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path) -> str:
"""Return the host-side SSH identity file path for this entry.
For gitea entries, provisions a fresh deploy key first."""
if entry.Key.provider == "gitea":
return _provision_dynamic_key(entry, slug, stage_dir)
return entry.IdentityFile
class GitGate(ABC):
"""The per-agent git-gate. Encapsulates the host-side prepare
@@ -93,14 +631,20 @@ class GitGate(ABC):
entrypoint, pre-receive hook, and access-hook scripts (mode
600) under `stage_dir`. Pure host-side, no docker subprocess.
For `gitea` key entries, the returned upstream intentionally
has an empty identity file. Backend launch fills that in after
the operator confirms the preflight.
For `gitea` key entries, also generates and registers
a fresh deploy key via the forge API and writes the private key
+ key ID to `stage_dir`.
Returned plan is incomplete: the launch step must fill
`internal_network` / `egress_network` via `dataclasses.replace`
before passing the plan to `.start`."""
upstreams = git_gate_upstreams_for_bottle(bottle)
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
for i, entry in enumerate(bottle.git):
upstreams_list[i] = dataclasses.replace(
upstreams_list[i],
identity_file=_resolve_identity_file(entry, slug, stage_dir),
)
upstreams = tuple(upstreams_list)
entrypoint = stage_dir / "git_gate_entrypoint.sh"
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
entrypoint.chmod(0o600)
@@ -142,23 +686,3 @@ class GitGate(ABC):
access_hook_script=access_hook,
upstreams=tuple(upstreams_with_files),
)
__all__ = [
"GIT_GATE_HOSTNAME",
"GIT_GATE_TIMEOUT_SECS",
"GitGateUpstream",
"GitGatePlan",
"GitGate",
"git_gate_upstreams_for_bottle",
"git_gate_render_gitconfig",
"git_gate_known_hosts_line",
"git_gate_render_entrypoint",
"git_gate_render_hook",
"git_gate_render_access_hook",
"provision_git_gate_dynamic_keys",
"revoke_git_gate_provisioned_keys",
"_gitconfig_validate_value",
"_provision_dynamic_key",
"_resolve_identity_file",
]
-145
View File
@@ -1,145 +0,0 @@
"""git-gate deploy-key lifecycle for `gitea` upstreams (PRD 0047/0048).
Provisions a fresh ed25519 deploy key via the forge API at prepare time
and revokes it at teardown, so the agent never holds an upstream
credential. Split out of `git_gate.py`; the forge HTTP client is lazily
imported (`deploy_key_provisioner`) to keep its cost off the host path.
`git_gate` re-exports these names for API stability."""
from __future__ import annotations
import os
import dataclasses
from pathlib import Path
from typing import TYPE_CHECKING
from .log import info
from .manifest import ManifestBottle, ManifestGitEntry
from .git_gate_render import GitGateUpstream
if TYPE_CHECKING:
from .git_gate import GitGatePlan
def _provision_dynamic_key(
entry: ManifestGitEntry,
slug: str,
stage_dir: Path,
) -> str:
"""Generate a fresh ed25519 keypair, register the public half with
the forge, and persist the private key + key ID under `stage_dir`.
Returns the host-side path to the private key file so the caller
can inject it into the GitGateUpstream as `identity_file`."""
from .deploy_key_provisioner import get_provisioner
pk = entry.Key
token = os.environ.get(pk.forge_token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url)
owner_repo = entry.UpstreamPath
if owner_repo.endswith(".git"):
owner_repo = owner_repo[:-4]
title = f"bot-bottle:{slug}:{entry.Name}"
info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]")
key_id, private_key_bytes = provisioner.create(owner_repo, title)
key_file = stage_dir / f"{entry.Name}-key"
key_file.write_bytes(private_key_bytes)
key_file.chmod(0o600)
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
id_file.write_text(key_id)
id_file.chmod(0o600)
info(f"provisioned deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
return str(key_file)
def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) -> None:
"""Revoke all deploy keys provisioned for `bottle` during prepare.
Called at teardown after containers stop. Raises if any revocation
fails a stranded key is a security concern that the operator must
address manually."""
from .deploy_key_provisioner import get_provisioner
for entry in bottle.git:
if entry.Key.provider != "gitea":
continue
pk = entry.Key
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
if not id_file.exists():
continue
key_id = id_file.read_text().strip()
token = os.environ.get(pk.forge_token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set;"
f" cannot revoke deploy key {key_id}"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url)
owner_repo = entry.UpstreamPath
if owner_repo.endswith(".git"):
owner_repo = owner_repo[:-4]
info(f"revoking deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
provisioner.delete(owner_repo, key_id)
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path) -> str:
"""Return the host-side SSH identity file path for this entry.
For gitea entries, provisions a fresh deploy key first."""
if entry.Key.provider == "gitea":
return _provision_dynamic_key(entry, slug, stage_dir)
return entry.IdentityFile
def provision_git_gate_dynamic_keys(
bottle: ManifestBottle,
plan: "GitGatePlan",
stage_dir: Path,
) -> "GitGatePlan":
"""Provision dynamic git-gate keys and return an updated plan.
This runs during backend launch, after the operator confirms the
preflight. Plan preparation intentionally stays side-effect-light:
dry-runs and aborted launches must not create remote deploy keys.
"""
if not plan.upstreams:
return plan
upstreams_by_name: dict[str, GitGateUpstream] = {
upstream.name: upstream for upstream in plan.upstreams
}
updated: list[GitGateUpstream] = []
for entry in bottle.git:
upstream = upstreams_by_name.get(entry.Name)
if upstream is None:
continue
if entry.Key.provider == "gitea":
identity_file = _provision_dynamic_key(entry, plan.slug, stage_dir)
upstream = dataclasses.replace(upstream, identity_file=identity_file)
updated.append(upstream)
if len(updated) != len(plan.upstreams):
updated_names = {u.name for u in updated}
for upstream in plan.upstreams:
if upstream.name not in updated_names:
updated.append(upstream)
return dataclasses.replace(plan, upstreams=tuple(updated))
__all__ = [
"revoke_git_gate_provisioned_keys",
"provision_git_gate_dynamic_keys",
"_provision_dynamic_key",
"_resolve_identity_file",
]
-502
View File
@@ -1,502 +0,0 @@
"""Pure host-side rendering for the per-agent git-gate (PRD 0008).
Builds the agent's `.gitconfig` insteadOf rewrites, the known_hosts
line, and the entrypoint / pre-receive / access-hook scripts the sidecar
runs. No docker or forge calls exposed for tests and reuse across
backends. Split out of `git_gate.py` so the control surface (`GitGate`)
and the deploy-key lifecycle (`git_gate_provision`) each read on their
own; `git_gate` re-exports these names for API stability."""
from __future__ import annotations
import shlex
from dataclasses import dataclass
from pathlib import Path
from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the sidecar bundle. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate"
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
# git daemon (--timeout/--init-timeout), the access-hook subprocess in
# git_http_backend, and the git http-backend CGI subprocess.
GIT_GATE_TIMEOUT_SECS = 15
@dataclass(frozen=True)
class GitGateUpstream:
"""One bare repo on the gate. `name` drives the bare-repo path
(`/git/<name>.git`), the agent's URL after insteadOf rewrite
(`git://<gate>/<name>.git`), and the per-upstream credential
paths inside the gate (`/git-gate/creds/<name>-key` and
`/git-gate/creds/<name>-known_hosts`).
`identity_file` is the host-side absolute path the gate's start
step will docker-cp into the container. `known_host_key` is the
KnownHostKey string from the manifest; the gate's start step
materialises it into a known_hosts file if non-empty.
the gate credential paths inside the running sidecar."""
name: str
upstream_url: str
upstream_host: str
upstream_port: str
identity_file: str
known_host_key: str
known_hosts_file: Path = Path()
def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstream, ...]:
"""Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name
validation already ran in `manifest.ManifestBottle.from_dict`."""
return tuple(
GitGateUpstream(
name=e.Name,
upstream_url=e.Upstream,
upstream_host=e.UpstreamHost,
upstream_port=e.UpstreamPort,
identity_file=e.IdentityFile,
known_host_key=e.KnownHostKey,
)
for e in bottle.git
)
def _gitconfig_validate_value(field: str, value: str) -> None:
"""Raise ValueError if value contains characters that break gitconfig line syntax."""
if "\n" in value or "\r" in value:
raise ValueError(
f"git-gate: {field} contains a newline, which would inject "
f"arbitrary gitconfig keys; rejecting manifest entry"
)
def git_gate_render_gitconfig(
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
"""Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
exposed for tests + reuse across backends.
`gate_host` is the part of the URL between `<scheme>://` and the
repo path backends differ here:
- docker: `git-gate` (the short network alias)
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
TSI-allowlisted guest)
Empty `entries` returns an empty string so callers can no-op
cleanly without conditional formatting at the call site."""
if not entries:
return ""
out = [
"# bot-bottle git-gate (PRD 0008): every git operation against\n",
"# a declared upstream routes through the gate, which mirrors\n",
"# the upstream bidirectionally (gitleaks-scanned push;\n",
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
]
for entry in entries:
_gitconfig_validate_value(f"repos[{entry.Name!r}].url", entry.Upstream)
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
out.append(f"\tinsteadOf = {entry.Upstream}\n")
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
port = (
f":{entry.UpstreamPort}"
if entry.UpstreamPort and entry.UpstreamPort != "22"
else ""
)
alias = (
f"ssh://{entry.UpstreamUser}@{entry.RemoteKey}{port}/"
f"{entry.UpstreamPath}"
)
_gitconfig_validate_value(f"repos[{entry.Name!r}].url (resolved alias)", alias)
out.append(f"\tinsteadOf = {alias}\n")
return "".join(out)
def git_gate_known_hosts_line(host: str, port: str, key: str) -> str:
"""Format `host[:port] key` for OpenSSH's known_hosts. Non-default
ports use the bracketed `[host]:port` form (the form OpenSSH writes
on disk for hosts reached via a non-22 port)."""
if port and port != "22":
target = f"[{host}]:{port}"
else:
target = host
return f"{target} {key}\n"
def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
"""Posix-sh entrypoint. One `init_repo` call per upstream, then
`exec git daemon`. The function reads
`/git-gate/creds/<name>-{key,known_hosts}` (bind-mounted into
the bundle by the renderer) and wires them into each bare repo's
config; the access-hook + pre-receive hook pick those paths up
at fetch / push time."""
lines = [
"#!/bin/sh",
"set -eu",
"",
"init_repo() {",
" name=$1",
" upstream_url=$2",
" keyfile=/git-gate/creds/${name}-key",
" hostsfile=/git-gate/creds/${name}-known_hosts",
"",
# `|| true`: PRD 0018 chunk 3+ bind-mounts these RO from the
# host, so chmod-syscalls fail with EROFS. The files already
# have the right perms on the host (SSH requires 0600 to load
# the key in the first place), so the chmod is best-effort
# cleanup for the legacy docker-cp path where the file
# landed at the host's umask perms.
" chmod 600 \"$keyfile\" 2>/dev/null || true",
" if [ -f \"$hostsfile\" ]; then",
" chmod 600 \"$hostsfile\" 2>/dev/null || true",
" fi",
"",
" repo=/git/${name}.git",
" if [ ! -d \"$repo\" ]; then",
" git init --bare \"$repo\" >/dev/null",
# --mirror=fetch sets remote.origin.fetch = +refs/*:refs/* so",
# a later `git fetch origin` mirrors the upstream's full ref",
# graph (heads, tags, notes) into the bare repo at canonical",
# paths. It does NOT set remote.origin.mirror=true, so an",
# explicit `git push origin <ref>:<ref>` still pushes one ref.",
" git -C \"$repo\" remote add --mirror=fetch origin \"$upstream_url\"",
" fi",
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
" git -C \"$repo\" config receive.advertisePushOptions true",
" git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}",
"",
"mkdir -p /git",
]
for u in upstreams:
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
lines.extend([
"",
"exec git daemon \\",
" --reuseaddr \\",
f" --timeout={GIT_GATE_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_TIMEOUT_SECS} \\",
" --base-path=/git \\",
" --export-all \\",
" --enable=receive-pack \\",
" --access-hook=/etc/git-gate/access-hook \\",
" --verbose",
])
return "\n".join(lines) + "\n"
def git_gate_render_hook() -> str:
"""The shared pre-receive hook: gitleaks-scan all incoming refs,
then forward each accepted ref to the real upstream (`origin`)
using the per-repo credential. Failure in either phase aborts
the push so the agent sees a real rejection. POSIX sh.
Two phases (scan all, then push all) keeps a hit on ref N from
half-pushing refs 1..N-1; both phases re-read stdin from a temp
file because pre-receive's stdin is a one-shot stream."""
return r"""#!/bin/sh
# git-gate pre-receive (PRD 0008). Stdin: <old> <new> <ref> per line.
set -u
refs_file=$(mktemp)
trap 'rm -f "$refs_file"' EXIT
cat > "$refs_file"
zero=0000000000000000000000000000000000000000
supervise_gitleaks_allow() {
log_opts=$1
ref=$2
report_file=$(mktemp)
if ! gitleaks git \
--log-opts="$log_opts" \
--no-banner \
--redact \
--ignore-gitleaks-allow \
--report-format=json \
--report-path="$report_file" \
--exit-code 0 \
1>&2; then
rm -f "$report_file"
echo "git-gate: gitleaks inline-suppression scan failed for $ref" >&2
return 1
fi
proposal_id=$(
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
import datetime
import hashlib
import json
import os
import sys
import uuid
from pathlib import Path
report_path = Path(sys.argv[1])
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
if not queue_dir or not slug:
sys.exit(2)
try:
raw = json.loads(report_path.read_text() or "[]")
except json.JSONDecodeError:
sys.exit(3)
if not isinstance(raw, list):
sys.exit(3)
if not raw:
sys.exit(0)
ref = os.environ.get("GITLEAKS_ALLOW_REF", "")
lines = [
"gitleaks inline suppression requires supervisor approval",
f"ref: {ref}",
"",
]
for i, finding in enumerate(raw, 1):
if not isinstance(finding, dict):
continue
file_path = finding.get("File", "")
line_no = finding.get("StartLine", finding.get("Line", ""))
rule_id = finding.get("RuleID", "")
commit = finding.get("Commit", "")
line = finding.get("Line", "")
lines.extend([
f"finding {i}:",
f" file: {file_path}",
f" line: {line_no}",
f" rule: {rule_id}",
f" commit: {commit}",
f" code: {line}",
"",
])
payload = "\n".join(lines).rstrip() + "\n"
proposal_id = str(uuid.uuid4())
proposal = {
"id": proposal_id,
"bottle_slug": slug,
"tool": "gitleaks-allow",
"proposed_file": payload,
"justification": (
"git-gate found gitleaks findings hidden by # gitleaks:allow; "
"approve only for dummy test fixtures or confirmed false positives"
),
"arrival_timestamp": datetime.datetime.now(
datetime.timezone.utc
).isoformat(),
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(),
}
queue = Path(queue_dir)
queue.mkdir(parents=True, exist_ok=True)
path = queue / f"{proposal_id}.proposal.json"
tmp = path.with_suffix(path.suffix + ".tmp")
with tmp.open("w", encoding="utf-8") as f:
json.dump(proposal, f, indent=2)
f.write("\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
print(proposal_id)
PY
)
rc=$?
rm -f "$report_file"
if [ "$rc" -eq 0 ] && [ -z "$proposal_id" ]; then
return 0
fi
if [ "$rc" -ne 0 ]; then
echo "git-gate: cannot route # gitleaks:allow finding to supervisor; refusing push" >&2
return 1
fi
queue_dir=${SUPERVISE_QUEUE_DIR:-}
response_file="$queue_dir/${proposal_id}.response.json"
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
case "$timeout" in
''|*[!0-9]*)
echo "git-gate: invalid SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS=$timeout" >&2
return 1
;;
esac
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
waited=0
while [ "$waited" -lt "$timeout" ]; do
if [ -f "$response_file" ]; then
status=$(python3 - "$response_file" <<'PY'
import json
import sys
try:
with open(sys.argv[1], encoding="utf-8") as f:
raw = json.load(f)
except (OSError, json.JSONDecodeError):
sys.exit(1)
status = raw.get("status")
if not isinstance(status, str):
sys.exit(1)
print(status)
PY
) || status=""
case "$status" in
approved|modified)
mkdir -p "$queue_dir/processed"
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
return 0
;;
rejected)
echo "git-gate: supervisor rejected # gitleaks:allow for $ref" >&2
return 1
;;
*)
echo "git-gate: invalid supervisor response for # gitleaks:allow" >&2
return 1
;;
esac
fi
sleep 1
waited=$((waited + 1))
done
echo "git-gate: supervisor approval timed out for # gitleaks:allow; refusing push" >&2
return 1
}
# Phase 1: gitleaks scan each ref's incoming commits.
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
[ "$new" = "$zero" ] && continue
if [ "$old" = "$zero" ]; then
# New ref: scan only the commits this push introduces — those
# reachable from $new but not from any ref the gate already has.
# Everything already on the gate arrived via upstream mirror-fetch
# or a previously gitleaks-scanned push, so it's already-upstream
# or already-scanned; re-scanning it (the old `$new` full-ancestry
# range) only resurfaces historical findings and blocks every new
# branch. See PRD 0028 / issue #106.
log_opts="$new --not --all"
else
log_opts="$old..$new"
fi
echo "git-gate: gitleaks scanning $ref ($log_opts)" >&2
if ! gitleaks git --log-opts="$log_opts" --no-banner --redact 1>&2; then
echo "git-gate: gitleaks rejected push to $ref" >&2
exit 1
fi
if ! supervise_gitleaks_allow "$log_opts" "$ref"; then
exit 1
fi
done < "$refs_file"
# Phase 2: forward each ref to the upstream (`origin`, configured
# in the entrypoint via `git remote add --mirror=fetch`).
keyfile=$(git config --get git-gate.identityFile)
hostsfile=$(git config --get git-gate.knownHosts)
if [ ! -f "$hostsfile" ]; then
echo "git-gate: no KnownHostKey configured for this upstream; refusing to push" >&2
echo "git-gate: add KnownHostKey to the bottle.git entry and restart the bottle" >&2
exit 1
fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
push_option_count=${GIT_PUSH_OPTION_COUNT:-0}
case "$push_option_count" in
''|*[!0-9]*)
echo "git-gate: invalid GIT_PUSH_OPTION_COUNT=$push_option_count" >&2
exit 1
;;
esac
set --
i=0
while [ "$i" -lt "$push_option_count" ]; do
opt=$(printenv "GIT_PUSH_OPTION_$i" || :)
set -- "$@" --push-option="$opt"
i=$((i + 1))
done
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
if [ "$new" = "$zero" ]; then
refspec=":$ref"
elif [ "$old" != "$zero" ] && ! git merge-base --is-ancestor "$old" "$new" 2>/dev/null; then
refspec="+$new:$ref"
else
refspec="$new:$ref"
fi
echo "git-gate: forwarding $ref to origin" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git push "$@" origin "$refspec" 1>&2; then
echo "git-gate: upstream push failed for $ref" >&2
exit 1
fi
done < "$refs_file"
exit 0
"""
def git_gate_render_access_hook() -> str:
"""`git daemon --access-hook` script. Runs before each protocol
service; for `upload-pack` (fetch / clone / ls-remote / pull) it
refreshes the bare repo from upstream first, so the response
reflects upstream's current state. For other services (notably
`receive-pack`) it returns 0 immediately and lets the existing
pre-receive hook gate the operation. POSIX sh.
The hook receives:
$1 service name (`upload-pack`, `receive-pack`, ...)
$2 absolute path to the resolved repo
$3 client hostname (unused)
$4 client tcp address (unused)
Fail-closed on upstream errors: the agent's fetch fails too,
so it never silently sees stale data matches the PRD's
'equivalent to operations against the upstream' contract."""
return r"""#!/bin/sh
# git-gate access-hook (PRD 0008). $1=service $2=repo $3=host $4=peer
set -u
service=$1
repo_dir=$2
# Push path keeps its own gating in pre-receive (gitleaks +
# forward). Only refresh-from-upstream on fetch operations.
if [ "$service" != "upload-pack" ]; then
exit 0
fi
keyfile=$(git -C "$repo_dir" config --get git-gate.identityFile 2>/dev/null || true)
hostsfile=$(git -C "$repo_dir" config --get git-gate.knownHosts 2>/dev/null || true)
if [ -z "$keyfile" ] || [ ! -f "$hostsfile" ]; then
echo "git-gate: missing credentials for $repo_dir; refusing fetch" >&2
exit 1
fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
echo "git-gate: refreshing $repo_dir from upstream" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" fetch origin --prune >&2; then
echo "git-gate: upstream fetch failed for $repo_dir; refusing to serve stale data" >&2
exit 1
fi
# Sync the bare repo's HEAD to upstream's HEAD on the first fetch
# (when it still points at the `git init --bare` default of
# refs/heads/master and upstream uses something else, the cloned
# checkout would fail with "remote HEAD refers to nonexistent ref").
# Costs one extra ls-remote on first fetch only; subsequent fetches
# skip the branch. If upstream's default branch changes after the
# gate has cached it, restart the bottle to resync.
if ! git -C "$repo_dir" rev-parse --verify HEAD >/dev/null 2>&1; then
upstream_head=$(GIT_SSH_COMMAND="$ssh_cmd" git -C "$repo_dir" \
ls-remote --symref origin HEAD 2>/dev/null \
| awk '/^ref:/ {print $2; exit}')
if [ -n "$upstream_head" ]; then
git -C "$repo_dir" symbolic-ref HEAD "$upstream_head" || true
fi
fi
exit 0
"""
-15
View File
@@ -19,13 +19,6 @@ from urllib.parse import urlsplit
DEFAULT_PORT = 9420
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
# imported: this module ships as a flat top-level sibling in the sidecar
# bundle image (see Dockerfile.sidecars), not as part of the bot_bottle
# package, so `bot_bottle.git_gate` and its dependency chain aren't
# available at runtime.
GIT_GATE_TIMEOUT_SECS = 15
# Bound memory use while still allowing ordinary git push packfiles.
MAX_BODY_BYTES = 100 * 1024 * 1024
@@ -54,7 +47,6 @@ class GitHttpHandler(BaseHTTPRequestHandler):
[hook_path, "upload-pack", str(repo_dir), peer, peer],
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
if hook.returncode != 0:
detail = (hook.stderr or hook.stdout).decode(
@@ -118,7 +110,6 @@ class GitHttpHandler(BaseHTTPRequestHandler):
env=env,
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
self._write_cgi_response(proc.stdout)
@@ -157,13 +148,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
key, _, value = line.decode("latin1").partition(":")
value = value.strip()
if key.lower() == "status":
try:
status = int(value.split()[0])
except (ValueError, IndexError):
self.log_message(
"malformed CGI Status header %r; using 500", value,
)
status = 500
else:
headers.append((key, value))
self.send_response(status)
+10 -96
View File
@@ -1,107 +1,21 @@
"""Tiny logging wrappers. All output goes to stderr.
Two capabilities layer onto the bare wrappers (issue #252):
- **Levels.** `debug` / `info` / `warn` / `error` carry an ordered
severity. Output is gated by `BOT_BOTTLE_LOG_LEVEL` (debug | info |
warn | error; default `info`). A message emits when its severity is
at or above the threshold, so `debug` is silent by default and
`error` always surfaces (nothing sits above it) which keeps the
fatal `die` path visible regardless of the configured level.
- **Context.** Every wrapper takes an optional `context` mapping that
renders as a parseable ` [k=v ...]` suffix (keys sorted; values with
whitespace/quotes are quoted), so failures can be filtered and
correlated instead of being flat strings.
With no `context` and the default level, output is byte-identical to the
original `bot-bottle: <msg>` / `bot-bottle: warning: <msg>` /
`bot-bottle: error: <msg>` lines the 100+ existing call sites are
unaffected.
"""
"""Tiny logging wrappers. All output goes to stderr."""
from __future__ import annotations
import os
import sys
from typing import Mapping, NoReturn
# Ordered severities. Gaps left between values so intermediate levels
# can be added later without renumbering.
DEBUG = 10
INFO = 20
WARN = 30
ERROR = 40
_LEVEL_NAMES: dict[str, int] = {
"debug": DEBUG,
"info": INFO,
"warn": WARN,
"warning": WARN,
"error": ERROR,
}
# Default threshold when BOT_BOTTLE_LOG_LEVEL is unset or unrecognised.
_DEFAULT_THRESHOLD = INFO
_LOG_LEVEL_ENV = "BOT_BOTTLE_LOG_LEVEL"
from typing import NoReturn
def _threshold() -> int:
"""Resolve the active level threshold from the environment.
Read per-call (not cached) so the level can be changed at runtime
and so tests can patch `os.environ` without a reload. Unknown values
fall back to the default rather than raising logging must never be
the thing that crashes the process."""
raw = os.environ.get(_LOG_LEVEL_ENV, "")
return _LEVEL_NAMES.get(raw.strip().lower(), _DEFAULT_THRESHOLD)
def info(msg: str) -> None:
print(f"bot-bottle: {msg}", file=sys.stderr)
def _format_context(context: Mapping[str, object] | None) -> str:
"""Render a context mapping as a ` [k=v k2=v2]` suffix.
Keys are sorted for stable, diffable output. Values that are empty or
contain whitespace or a quote are wrapped in double quotes (with inner
quotes escaped) so each `k=v` pair stays parseable. Empty/None context
renders as the empty string."""
if not context:
return ""
parts: list[str] = []
for key in sorted(context):
value = str(context[key])
if value == "" or any(ch.isspace() for ch in value) or '"' in value:
value = '"' + value.replace('"', '\\"') + '"'
parts.append(f"{key}={value}")
return " [" + " ".join(parts) + "]"
def warn(msg: str) -> None:
print(f"bot-bottle: warning: {msg}", file=sys.stderr)
def _emit(
level: int,
label: str,
msg: str,
context: Mapping[str, object] | None,
) -> None:
if level < _threshold():
return
prefix = f"{label}: " if label else ""
sys.stderr.write(f"bot-bottle: {prefix}{msg}{_format_context(context)}\n")
def debug(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(DEBUG, "debug", msg, context)
def info(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(INFO, "", msg, context)
def warn(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(WARN, "warning", msg, context)
def error(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(ERROR, "error", msg, context)
def error(msg: str) -> None:
print(f"bot-bottle: error: {msg}", file=sys.stderr)
class Die(SystemExit):
@@ -117,6 +31,6 @@ class Die(SystemExit):
self.message = message
def die(msg: str, *, context: Mapping[str, object] | None = None) -> NoReturn:
error(msg, context=context)
def die(msg: str) -> NoReturn:
error(msg)
raise Die(1, msg)
+140 -137
View File
@@ -62,25 +62,15 @@ from dataclasses import dataclass, field, replace
from pathlib import Path
from typing import Mapping
from .log import warn
from .manifest_util import ManifestError, as_json_object
from .manifest_agent import ManifestAgent, ManifestAgentProvider
from .manifest_bottle import ManifestBottle
from .manifest_egress import (
EGRESS_AUTH_SCHEMES,
ManifestEgressConfig,
ManifestEgressRoute,
)
from .manifest_extends import merge_bottles_runtime, resolve_bottles
from .manifest_git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig
from .manifest_loader import (
check_stale_json,
load_bottle_chain_from_dir,
scan_agent_names,
scan_bottle_names,
)
from .manifest_schema import validate_agent_frontmatter_keys
from .yaml_subset import YamlSubsetError, parse_frontmatter
from .manifest_git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig, parse_git_gate_config
from .manifest_schema import BOTTLE_KEYS
# Re-export everything that callers currently import from this module.
__all__ = [
@@ -99,6 +89,10 @@ __all__ = [
]
def _empty_str_dict() -> dict[str, str]:
return {}
def _section_dict(value: object, label: str) -> dict[str, object]:
"""Like as_json_object but treats absent/null as an empty section."""
if value is None:
@@ -106,6 +100,109 @@ def _section_dict(value: object, label: str) -> dict[str, object]:
return as_json_object(value, label)
@dataclass(frozen=True)
class ManifestBottle:
env: Mapping[str, str] = field(default_factory=_empty_str_dict)
agent_provider: ManifestAgentProvider = field(default_factory=ManifestAgentProvider)
git: tuple[ManifestGitEntry, ...] = ()
# Per-bottle git identity (issue #86). Empty default — bottles
# that don't set `git-gate.user:` in the manifest skip the
# `git config --global` step entirely. A bottle can declare a user
# identity without any git-gate.repos upstreams, and vice versa.
git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
# Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
# default, issue #249), the launch step brings up a supervise
# sidecar that exposes MCP tools to the agent (egress-block,
# capability-block) plus mounts the current-config dir read-only
# into the agent at /etc/bot-bottle/current-config. Set
# `supervise: false` to skip the sidecar and mount.
supervise: bool = True
@classmethod
def from_dict(cls, name: str, raw: object) -> "ManifestBottle":
d = as_json_object(raw, f"bottle '{name}'")
if "runtime" in d:
raise ManifestError(
f"bottle '{name}' has a 'runtime' field, which is no longer "
f"supported. gVisor (runsc) is now auto-detected by the "
f"backend; remove the 'runtime' field from the bottle "
f"definition."
)
if "ssh" in d:
raise ManifestError(
f"bottle '{name}' has an 'ssh' field, which has been removed "
f"(PRD 0009). Declare upstreams under 'git-gate.repos' with "
f"url + identity + host_key; the git-gate sidecar (PRD 0008) "
f"holds the credential and gitleaks-scans pushes."
)
if "git" in d:
raise ManifestError(
f"bottle '{name}' uses 'git' which has been replaced by "
f"'git-gate' (PRD 0047). Move git.user → git-gate.user "
f"and git.remotes → git-gate.repos (fields: url, identity, host_key)."
)
if "git_user" in d:
raise ManifestError(
f"bottle '{name}' has a 'git_user' field, which has been "
f"removed. Move it under 'git-gate.user'."
)
unknown = set(d.keys()) - BOTTLE_KEYS
if unknown:
allowed = ", ".join(sorted(BOTTLE_KEYS))
raise ManifestError(
f"bottle '{name}' has unknown key(s) {sorted(unknown)}; "
f"allowed keys are {allowed}."
)
env: dict[str, str] = {}
env_raw = d.get("env")
if env_raw is not None:
env_dict = as_json_object(env_raw, f"bottle '{name}' env")
for var, value in env_dict.items():
if not isinstance(value, str):
raise ManifestError(
f"env entry {var} in bottle '{name}' must be a JSON string "
f"(was {type(value).__name__}). Use \"?<message>\" for prompt-at-runtime."
)
env[var] = value
git: tuple[ManifestGitEntry, ...] = ()
git_user = ManifestGitUser()
git_raw = d.get("git-gate")
if git_raw is not None:
git, git_user = parse_git_gate_config(name, git_raw)
agent_provider = (
ManifestAgentProvider.from_dict(name, d["agent_provider"])
if "agent_provider" in d
else ManifestAgentProvider()
)
egress = (
ManifestEgressConfig.from_dict(name, d["egress"])
if "egress" in d
else ManifestEgressConfig()
)
supervise_raw = d.get("supervise", True)
if not isinstance(supervise_raw, bool):
raise ManifestError(
f"bottle '{name}' supervise must be a boolean "
f"(was {type(supervise_raw).__name__})"
)
return cls(
env=env, agent_provider=agent_provider, git=git,
git_user=git_user, egress=egress, supervise=supervise_raw,
)
def _merge_git_user(
agent_user: ManifestGitUser, base_user: ManifestGitUser
) -> ManifestGitUser:
@@ -118,74 +215,6 @@ def _merge_git_user(
)
def _manifest_with_merged_git_user(
agent: "ManifestAgent", raw_bottle: "ManifestBottle"
) -> "Manifest":
"""Build the single-value Manifest, overlaying the agent's git-gate.user
onto the bottle (agent wins on non-empty, per-field). Shared by the eager
and lazy load_for_agent paths."""
merged = _merge_git_user(agent.git_user, raw_bottle.git_user)
bottle = (
raw_bottle if merged == raw_bottle.git_user
else replace(raw_bottle, git_user=merged)
)
return Manifest(agent=agent, bottle=bottle)
def _resolve_effective_bottle_eager(
agent_name: str,
agent: "ManifestAgent",
bottle_names: "tuple[str, ...]",
bottles: "Mapping[str, ManifestBottle]",
) -> "ManifestBottle":
"""Return the effective ManifestBottle for the eager (from_json_obj) path.
When bottle_names is non-empty they are merged in order. When empty, falls
back to agent.bottle. Raises ManifestError when neither is set."""
if bottle_names:
resolved: list[ManifestBottle] = []
for bn in bottle_names:
if bn not in bottles:
available = ", ".join(sorted(bottles.keys())) or "(none)"
raise ManifestError(
f"bottle '{bn}' not defined. Available: {available}"
)
resolved.append(bottles[bn])
return merge_bottles_runtime(resolved)
if not agent.bottle:
raise ManifestError(
f"agent '{agent_name}' has no 'bottle' field and no bottles were "
f"selected at launch. Select at least one bottle or add "
f"'bottle: <name>' to the agent manifest."
)
return bottles[agent.bottle]
def _resolve_effective_bottle_lazy(
agent_name: str,
agent_bottle: str,
bottle_names: "tuple[str, ...]",
bottles_dir: "Path",
) -> "ManifestBottle":
"""Return the effective ManifestBottle for the lazy (from_md_dirs) path.
When bottle_names is non-empty they are resolved from disk and merged in
order. When empty, falls back to agent_bottle. Raises ManifestError when
neither is set."""
if bottle_names:
resolved = [load_bottle_chain_from_dir(bn, bottles_dir) for bn in bottle_names]
return merge_bottles_runtime(resolved)
if not agent_bottle:
raise ManifestError(
f"agent '{agent_name}' has no 'bottle' field and no bottles were "
f"selected at launch. Select at least one bottle or add "
f"'bottle: <name>' to the agent manifest."
)
return load_bottle_chain_from_dir(agent_bottle, bottles_dir)
@dataclass(frozen=True)
class Manifest:
"""Single-agent/bottle value type. Returned by ManifestIndex.load_for_agent().
@@ -258,6 +287,8 @@ class ManifestIndex:
home_md = home_dir / ".bot-bottle"
cwd_md = cwd_dir / ".bot-bottle"
from .manifest_loader import check_stale_json
check_stale_json(home_dir, home_md, "$HOME")
if cwd_dir.resolve() != home_dir.resolve():
check_stale_json(cwd_dir, cwd_md, "$CWD")
@@ -297,6 +328,7 @@ class ManifestIndex:
files = sorted(stale_bottles.glob("*.md"))
if files:
names = ", ".join(p.name for p in files)
from .log import warn
warn(
f"ignoring bottle file(s) under "
f"{stale_bottles}: {names}. Bottles can only "
@@ -318,6 +350,7 @@ class ManifestIndex:
raw_bottles: dict[str, dict[str, object]] = {}
for n, b in raw_bottles_obj.items():
raw_bottles[n] = as_json_object(b, f"bottle '{n}'")
from .manifest_extends import resolve_bottles
bottles = resolve_bottles(raw_bottles)
@@ -327,17 +360,6 @@ class ManifestIndex:
}
return cls(bottles=bottles, agents=agents)
@property
def all_bottle_names(self) -> list[str]:
"""Sorted list of all discoverable bottle names.
In names-only mode (from resolve/from_md_dirs) this scans bottle
filenames without reading their content. In eager mode (from
from_json_obj) it returns the pre-parsed bottles' names."""
if self.home_md is not None:
return scan_bottle_names(self.home_md / "bottles")
return sorted(self.bottles.keys())
@property
def all_agent_names(self) -> list[str]:
"""Sorted list of all discoverable agent names.
@@ -346,6 +368,7 @@ class ManifestIndex:
filenames without reading their content. In eager mode (from
from_json_obj) it returns the pre-parsed agents' names."""
if self.home_md is not None:
from .manifest_loader import scan_agent_names
home_names = set(scan_agent_names(self.home_md / "agents").keys())
cwd_names: set[str] = set()
if self.cwd_md is not None:
@@ -353,18 +376,9 @@ class ManifestIndex:
return sorted(home_names | cwd_names)
return sorted(self.agents.keys())
def load_for_agent(
self,
agent_name: str,
bottle_names: "tuple[str, ...] | None" = None,
) -> "Manifest":
def load_for_agent(self, agent_name: str) -> "Manifest":
"""Parse the named agent and its bottle; return a single-value Manifest.
`bottle_names` is an ordered list of bottles selected at launch time.
When non-empty they are resolved and merged in order (index 0 = base;
later entries override). When empty or None, falls back to the agent's
own `bottle:` field. Raises ManifestError when neither is set.
In lazy mode (from resolve/from_md_dirs) the agent file and its
bottle chain are read from disk for the first time here. In eager
mode (from_json_obj) the data is already parsed; this just filters
@@ -375,34 +389,25 @@ class ManifestIndex:
Always raises ManifestError if the agent is unknown or invalid.
Backends call this at preflight inside _validate."""
effective_bottle_names: tuple[str, ...] = bottle_names or ()
if self.home_md is None:
return self._load_for_agent_eager(agent_name, effective_bottle_names)
return self._load_for_agent_lazy(agent_name, effective_bottle_names)
def _load_for_agent_eager(
self, agent_name: str, bottle_names: tuple[str, ...]
) -> "Manifest":
"""Eager path (from_json_obj): data is already parsed; filter to the one
requested agent and its bottle so the returned Manifest always holds
exactly one agent and one bottle regardless of path."""
# Eager manifest (from_json_obj): data already parsed; filter to
# the one requested agent and its bottle so the returned Manifest
# always holds exactly one agent and one bottle regardless of path.
if agent_name not in self.agents:
available = ", ".join(sorted(self.agents.keys())) or "(none)"
raise ManifestError(
f"agent '{agent_name}' not defined. Available: {available}"
)
agent = self.agents[agent_name]
raw_bottle = _resolve_effective_bottle_eager(
agent_name, agent, bottle_names, self.bottles
)
return _manifest_with_merged_git_user(agent, raw_bottle)
raw_bottle = self.bottles[agent.bottle]
merged = _merge_git_user(agent.git_user, raw_bottle.git_user)
bottle = raw_bottle if merged == raw_bottle.git_user else replace(raw_bottle, git_user=merged)
return Manifest(agent=agent, bottle=bottle)
from .manifest_loader import load_bottle_chain_from_dir, scan_agent_names
from .manifest_schema import validate_agent_frontmatter_keys
from .yaml_subset import YamlSubsetError, parse_frontmatter
def _load_for_agent_lazy(
self, agent_name: str, bottle_names: tuple[str, ...]
) -> "Manifest":
"""Lazy path (resolve/from_md_dirs): read and parse the agent file and
its bottle chain from disk for the first time here."""
assert self.home_md is not None # guaranteed by load_for_agent dispatch
# Locate the agent file; cwd wins over home on name collision.
home_agents = scan_agent_names(self.home_md / "agents")
cwd_agents: dict[str, Path] = {}
@@ -426,32 +431,30 @@ class ManifestIndex:
validate_agent_frontmatter_keys(agent_path, fm.keys())
# Determine the effective bottle name(s).
agent_bottle = fm.get("bottle") or ""
bottle_name = fm.get("bottle")
if not isinstance(bottle_name, str) or not bottle_name:
raise ManifestError(
f"agent '{agent_name}' must declare a 'bottle' field "
f"naming a defined bottle"
)
# Load the bottle chain (may raise ManifestError).
bottles_dir = self.home_md / "bottles"
raw_bottle = _resolve_effective_bottle_lazy(
agent_name, str(agent_bottle), bottle_names, bottles_dir
)
effective_bottle_name = (
bottle_names[-1] if bottle_names else str(agent_bottle)
)
raw_bottle = load_bottle_chain_from_dir(bottle_name, bottles_dir)
# Build and validate the full ManifestAgent.
agent_dict: dict[str, object] = {
"bottle": bottle_name,
"skills": fm.get("skills", []),
"prompt": body.strip(),
}
if agent_bottle:
agent_dict["bottle"] = agent_bottle
if "git-gate" in fm:
agent_dict["git-gate"] = fm["git-gate"]
# Pass the effective bottle name as the known-bottles set so agents
# that have bottle: set are validated; agents without bottle: pass {}
# since bottle_names were already resolved above.
known = {effective_bottle_name} if effective_bottle_name else set()
agent = ManifestAgent.from_dict(agent_name, agent_dict, known)
agent = ManifestAgent.from_dict(agent_name, agent_dict, {bottle_name})
return _manifest_with_merged_git_user(agent, raw_bottle)
merged_user = _merge_git_user(agent.git_user, raw_bottle.git_user)
bottle = raw_bottle if merged_user == raw_bottle.git_user else replace(raw_bottle, git_user=merged_user)
return Manifest(agent=agent, bottle=bottle)
def has_agent(self, name: str) -> bool:
return name in self.agents
+14 -49
View File
@@ -8,7 +8,7 @@ from typing import cast
from .agent_provider import PROVIDER_TEMPLATES
from .manifest_util import ManifestError, as_json_object
from .manifest_git import ManifestGitUser
from .manifest_schema import AGENT_MODEL_KEYS, is_valid_entity_name
from .manifest_schema import AGENT_MODEL_KEYS
@dataclass(frozen=True)
@@ -109,8 +109,7 @@ class ManifestAgentProvider:
@dataclass(frozen=True)
class ManifestAgent:
# Optional: when empty the operator selects bottles at launch time.
bottle: str = ""
bottle: str
skills: tuple[str, ...] = ()
prompt: str = ""
# Per-agent git identity (issue #94). Overlays the referenced
@@ -130,20 +129,18 @@ class ManifestAgent:
f"allowed keys are {allowed}."
)
bottle_raw = d.get("bottle")
bottle = ""
if bottle_raw is not None:
if not isinstance(bottle_raw, str) or not bottle_raw:
bottle = d.get("bottle")
if not isinstance(bottle, str) or not bottle:
raise ManifestError(
f"agent '{name}' bottle must be a non-empty string when declared"
f"agent '{name}' must declare a 'bottle' field naming a "
f"defined bottle"
)
if bottle_raw not in bottle_names:
if bottle not in bottle_names:
available = ", ".join(sorted(bottle_names)) or "(none defined)"
raise ManifestError(
f"agent '{name}' references bottle '{bottle_raw}', which is not defined. "
f"agent '{name}' references bottle '{bottle}', which is not defined. "
f"Available: {available}"
)
bottle = bottle_raw
skills: tuple[str, ...] = ()
skills_raw = d.get("skills")
@@ -161,16 +158,6 @@ class ManifestAgent:
f"agent '{name}' skills[{i}] must be a string "
f"(was {type(skill).__name__})"
)
# Skill names become host/guest path segments and are
# interpolated into provisioning shell commands, so they
# must fit the same kebab-case convention as bottle/agent
# filenames — rejecting anything that could break out of a
# path segment or inject shell metacharacters.
if not is_valid_entity_name(skill):
raise ManifestError(
f"agent '{name}' skills[{i}] {skill!r} is not a valid "
f"skill name; must match [a-z][a-z0-9-]*"
)
collected.append(skill)
skills = tuple(collected)
@@ -212,10 +199,13 @@ def _parse_provider_settings(
) -> dict[str, object]:
if raw is None:
return {}
if template != "pi":
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings is only "
"supported for template 'pi'"
)
settings = as_json_object(raw, f"bottle '{bottle_name}' agent_provider.settings")
common_allowed = {"startup_args"}
pi_allowed = {
allowed = {
"provider",
"base_url",
"api",
@@ -228,37 +218,12 @@ def _parse_provider_settings(
"supports_developer_role",
"supports_reasoning_effort",
}
if template == "pi":
allowed = common_allowed | pi_allowed
elif template in ("claude", "codex"):
allowed = common_allowed
elif template not in PROVIDER_TEMPLATES:
return dict(settings)
else:
allowed = common_allowed
for key in settings:
if key not in allowed:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings has unknown "
f"key {key!r}; allowed: {', '.join(sorted(allowed))}"
)
startup_args = settings.get("startup_args")
if startup_args is not None:
if not isinstance(startup_args, list):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.startup_args "
f"must be an array of strings"
)
for i, arg in enumerate(startup_args):
if not isinstance(arg, str) or not arg:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings."
f"startup_args[{i}] must be a non-empty string"
)
if template != "pi":
return dict(settings)
for key in ("provider", "base_url", "api", "api_key", "api_key_env"):
value = settings.get(key)
if value is not None and (not isinstance(value, str) or not value):
-129
View File
@@ -1,129 +0,0 @@
"""The `ManifestBottle` value type.
Split out of `manifest.py` so the `extends:`/loader resolvers can import it
without a circular dependency: `manifest.py` imports those resolvers, while
they only need this value type. Everything here depends on leaf modules
(`manifest_util`, `manifest_agent`, `manifest_egress`, `manifest_git`,
`manifest_schema`), so this module sits at the bottom of the manifest layer.
`manifest.py` re-exports `ManifestBottle`, so existing
`from .manifest import ManifestBottle` callers are unaffected.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Mapping
from .manifest_util import ManifestError, as_json_object
from .manifest_agent import ManifestAgentProvider
from .manifest_egress import ManifestEgressConfig
from .manifest_git import ManifestGitEntry, ManifestGitUser, parse_git_gate_config
from .manifest_schema import BOTTLE_KEYS
__all__ = ["ManifestBottle"]
def _empty_str_dict() -> dict[str, str]:
return {}
@dataclass(frozen=True)
class ManifestBottle:
env: Mapping[str, str] = field(default_factory=_empty_str_dict)
agent_provider: ManifestAgentProvider = field(default_factory=ManifestAgentProvider)
git: tuple[ManifestGitEntry, ...] = ()
# Per-bottle git identity (issue #86). Empty default — bottles
# that don't set `git-gate.user:` in the manifest skip the
# `git config --global` step entirely. A bottle can declare a user
# identity without any git-gate.repos upstreams, and vice versa.
git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
# Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
# default, issue #249), the launch step brings up a supervise
# sidecar that exposes egress MCP tools to the agent. Set
# `supervise: false` to skip the sidecar.
supervise: bool = True
@classmethod
def from_dict(cls, name: str, raw: object) -> "ManifestBottle":
d = as_json_object(raw, f"bottle '{name}'")
if "runtime" in d:
raise ManifestError(
f"bottle '{name}' has a 'runtime' field, which is no longer "
f"supported. gVisor (runsc) is now auto-detected by the "
f"backend; remove the 'runtime' field from the bottle "
f"definition."
)
if "ssh" in d:
raise ManifestError(
f"bottle '{name}' has an 'ssh' field, which has been removed "
f"(PRD 0009). Declare upstreams under 'git-gate.repos' with "
f"url + identity + host_key; the git-gate sidecar (PRD 0008) "
f"holds the credential and gitleaks-scans pushes."
)
if "git" in d:
raise ManifestError(
f"bottle '{name}' uses 'git' which has been replaced by "
f"'git-gate' (PRD 0047). Move git.user → git-gate.user "
f"and git.remotes → git-gate.repos (fields: url, identity, host_key)."
)
if "git_user" in d:
raise ManifestError(
f"bottle '{name}' has a 'git_user' field, which has been "
f"removed. Move it under 'git-gate.user'."
)
unknown = set(d.keys()) - BOTTLE_KEYS
if unknown:
allowed = ", ".join(sorted(BOTTLE_KEYS))
raise ManifestError(
f"bottle '{name}' has unknown key(s) {sorted(unknown)}; "
f"allowed keys are {allowed}."
)
env: dict[str, str] = {}
env_raw = d.get("env")
if env_raw is not None:
env_dict = as_json_object(env_raw, f"bottle '{name}' env")
for var, value in env_dict.items():
if not isinstance(value, str):
raise ManifestError(
f"env entry {var} in bottle '{name}' must be a JSON string "
f"(was {type(value).__name__}). Use \"?<message>\" for prompt-at-runtime."
)
env[var] = value
git: tuple[ManifestGitEntry, ...] = ()
git_user = ManifestGitUser()
git_raw = d.get("git-gate")
if git_raw is not None:
git, git_user = parse_git_gate_config(name, git_raw)
agent_provider = (
ManifestAgentProvider.from_dict(name, d["agent_provider"])
if "agent_provider" in d
else ManifestAgentProvider()
)
egress = (
ManifestEgressConfig.from_dict(name, d["egress"])
if "egress" in d
else ManifestEgressConfig()
)
supervise_raw = d.get("supervise", True)
if not isinstance(supervise_raw, bool):
raise ManifestError(
f"bottle '{name}' supervise must be a boolean "
f"(was {type(supervise_raw).__name__})"
)
return cls(
env=env, agent_provider=agent_provider, git=git,
git_user=git_user, egress=egress, supervise=supervise_raw,
)
+5 -22
View File
@@ -21,9 +21,6 @@ VALID_METHODS = frozenset({
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# What the proxy does on an outbound token match (PRD 0062).
OUTBOUND_ON_MATCH_VALUES = ("block", "redact", "supervise")
def validate_egress_routes(
bottle_name: str,
@@ -70,7 +67,6 @@ class ManifestEgressRoute:
GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None
InboundDetectors: tuple[str, ...] | None = None
OutboundOnMatch: str = ""
@classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
@@ -165,9 +161,8 @@ class ManifestEgressRoute:
# --- dlp ---
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
outbound_on_match = ""
if "dlp" in d:
outbound_detectors, inbound_detectors, outbound_on_match = _parse_dlp_block(
outbound_detectors, inbound_detectors = _parse_dlp_block(
label, d.get("dlp"),
)
@@ -206,7 +201,6 @@ class ManifestEgressRoute:
GitFetch=git_fetch,
OutboundDetectors=outbound_detectors,
InboundDetectors=inbound_detectors,
OutboundOnMatch=outbound_on_match,
)
@@ -329,7 +323,7 @@ def _parse_header_match(
def _parse_dlp_block(
route_label: str,
raw: object,
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
label = f"{route_label} dlp"
d = as_json_object(raw, label)
@@ -364,24 +358,13 @@ def _parse_dlp_block(
outbound = _parse_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = d.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ManifestError(
f"{label} outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in d:
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
if k not in ("outbound_detectors", "inbound_detectors"):
raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are "
f"'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
f"'outbound_detectors', 'inbound_detectors'"
)
return outbound, inbound, on_match
return outbound, inbound
LOG_LEVELS = frozenset({0, 1, 2})
+31 -151
View File
@@ -2,59 +2,11 @@
from __future__ import annotations
from .manifest_bottle import ManifestBottle
from .manifest_egress import ManifestEgressConfig, validate_egress_routes
from .manifest_git import ManifestGitUser, parse_git_gate_config
from .manifest_util import ManifestError, as_json_object
from typing import TYPE_CHECKING
def merge_bottles_runtime(bottles: "list[ManifestBottle]") -> "ManifestBottle":
"""Merge an ordered list of pre-resolved ManifestBottle objects.
Index 0 is the base; each subsequent entry is applied on top using
the same field-merge rules as the file-based extends machinery:
env: dict merge, later wins; git_user: per-field overlay, later
wins on non-empty; git (repos): union by name, later wins; egress
routes: concatenate; agent_provider, supervise: later replaces.
"""
if not bottles:
raise ValueError("merge_bottles_runtime requires at least one bottle")
result = bottles[0]
for override in bottles[1:]:
result = _merge_two_bottles_runtime(result, override)
return result
def _merge_two_bottles_runtime(base: "ManifestBottle", override: "ManifestBottle") -> "ManifestBottle":
merged_env = {**base.env, **override.env}
merged_git_user = ManifestGitUser(
name=override.git_user.name or base.git_user.name,
email=override.git_user.email or base.git_user.email,
)
# git repos: union keyed by Name, override wins per-name.
base_repos_by_name = {entry.Name: entry for entry in base.git}
override_repos_by_name = {entry.Name: entry for entry in override.git}
merged_repos_names = list(base_repos_by_name) + [
n for n in override_repos_by_name if n not in base_repos_by_name
]
merged_git = tuple(
override_repos_by_name.get(n, base_repos_by_name[n])
for n in merged_repos_names
)
merged_routes = base.egress.routes + override.egress.routes
merged_egress = ManifestEgressConfig(routes=merged_routes, Log=override.egress.Log)
return ManifestBottle(
env=merged_env,
agent_provider=override.agent_provider,
git=merged_git,
git_user=merged_git_user,
egress=merged_egress,
supervise=override.supervise,
)
if TYPE_CHECKING:
from .manifest import ManifestBottle
from .manifest_egress import ManifestEgressConfig
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, ManifestBottle]:
@@ -77,6 +29,8 @@ def _resolve_one_bottle(
repos_cache: dict[str, dict[str, object]],
seen: tuple[str, ...],
) -> ManifestBottle:
from .manifest import ManifestBottle, ManifestError
if name in cache:
return cache[name]
if name in seen:
@@ -95,120 +49,33 @@ def _resolve_one_bottle(
repos_cache[name] = _resolve_repos_raw({}, child_raw)
return bottle
# Normalize to list, accepting both str and list[str].
raw_list: list[object]
if isinstance(parent_name_raw, str):
raw_list = [parent_name_raw]
elif isinstance(parent_name_raw, list):
raw_list = parent_name_raw
else:
if not isinstance(parent_name_raw, str):
raise ManifestError(
f"bottle '{name}' extends must be a string or list of strings "
f"bottle '{name}' extends must be a string "
f"(was {type(parent_name_raw).__name__})"
)
# Validate each entry before resolving any of them.
parent_names: list[str] = []
for i, pname in enumerate(raw_list):
if not isinstance(pname, str):
parent_name: str = parent_name_raw
if parent_name == name:
raise ManifestError(
f"bottle '{name}' extends[{i}] must be a string "
f"(was {type(pname).__name__})"
f"bottle '{name}' extends itself; remove the "
f"self-reference"
)
parent_names.append(pname)
if pname == name:
raise ManifestError(
f"bottle '{name}' extends itself; remove the self-reference"
)
if pname not in raws:
if parent_name not in raws:
avail = ", ".join(sorted(raws.keys())) or "(none)"
raise ManifestError(
f"bottle '{name}' extends '{pname}' which is not "
f"bottle '{name}' extends '{parent_name}' which is not "
f"defined. Available bottles: {avail}"
)
combined_parent, combined_repos_raw = _fold_parents(
parent_names, raws, cache, repos_cache, seen + (name,)
parent = _resolve_one_bottle(
parent_name, raws, cache, repos_cache, seen + (name,)
)
merged_repos_raw = _resolve_repos_raw(combined_repos_raw, child_raw)
bottle = _merge_bottles(combined_parent, child_raw, merged_repos_raw, name)
merged_repos_raw = _resolve_repos_raw(repos_cache[parent_name], child_raw)
bottle = _merge_bottles(parent, child_raw, merged_repos_raw, name)
cache[name] = bottle
repos_cache[name] = merged_repos_raw
return bottle
def _fold_parents(
parent_names: list[str],
raws: dict[str, dict[str, object]],
cache: dict[str, ManifestBottle],
repos_cache: dict[str, dict[str, object]],
seen: tuple[str, ...],
) -> tuple[ManifestBottle, dict[str, object]]:
"""Resolve each parent and fold them left-to-right.
Later parents win over earlier ones on conflict. The `seen` tuple
carries the current bottle's name so cycle detection works across
every parent edge in the multi-parent graph."""
first = parent_names[0]
effective = _resolve_one_bottle(first, raws, cache, repos_cache, seen)
effective_repos_raw = repos_cache[first]
for pname in parent_names[1:]:
later = _resolve_one_bottle(pname, raws, cache, repos_cache, seen)
later_repos_raw = repos_cache[pname]
effective, effective_repos_raw = _fold_two_bottles(
effective, effective_repos_raw, later, later_repos_raw
)
return effective, effective_repos_raw
def _fold_two_bottles(
earlier: ManifestBottle,
earlier_repos_raw: dict[str, object],
later: ManifestBottle,
later_repos_raw: dict[str, object],
) -> tuple[ManifestBottle, dict[str, object]]:
"""Combine two resolved parent bottles; later wins over earlier."""
merged_env = {**earlier.env, **later.env}
merged_git_user = ManifestGitUser(
name=later.git_user.name or earlier.git_user.name,
email=later.git_user.email or earlier.git_user.email,
)
# Repos: union by name; for same-name entries, later wins per-field.
# Unlike _resolve_repos_raw, an empty later_repos_raw means "no repos
# declared" — it does NOT clear the earlier parent's repos.
names = list(earlier_repos_raw) + [
n for n in later_repos_raw if n not in earlier_repos_raw
]
merged_repos_raw: dict[str, object] = {
n: {
**as_json_object(earlier_repos_raw.get(n, {}), "earlier parent repo"),
**as_json_object(later_repos_raw.get(n, {}), "later parent repo"),
}
for n in names
}
if merged_repos_raw:
merged_git, _ = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
else:
merged_git = ()
# Egress: routes concatenate; scalar fields use last-wins.
merged_egress = ManifestEgressConfig(
routes=earlier.egress.routes + later.egress.routes,
Log=later.egress.Log,
)
return ManifestBottle(
env=merged_env,
agent_provider=later.agent_provider,
git=merged_git,
git_user=merged_git_user,
egress=merged_egress,
supervise=later.supervise,
), merged_repos_raw
def _merge_bottles(
parent: ManifestBottle,
child_raw: dict[str, object],
@@ -216,6 +83,10 @@ def _merge_bottles(
name: str,
) -> ManifestBottle:
"""Apply PRD 0025 merge rules."""
from .manifest import ManifestBottle, ManifestGitUser
from .manifest_egress import validate_egress_routes
from .manifest_util import as_json_object
# git-gate.repos: when the child declares repos, inject the already
# name-merged repo set (computed by _resolve_repos_raw) so the child
# parses with the full inherited+overridden list (issue #237).
@@ -288,6 +159,8 @@ def _resolve_repos_raw(
inherits the parent's set verbatim; an explicit empty dict clears it.
Otherwise parent and child unite by name, with same-name entries
field-merged (parent fields are defaults, child fields win)."""
from .manifest_util import as_json_object
if not _child_declares_git_gate_repos(child_raw):
return parent_repos
child_repos = _declared_repos_raw(child_raw)
@@ -307,6 +180,8 @@ def _resolve_repos_raw(
def _declared_repos_raw(child_raw: dict[str, object]) -> dict[str, object]:
"""Return the child's explicitly declared git-gate.repos as raw dicts,
or an empty dict when none are declared."""
from .manifest_util import as_json_object
if not _child_declares_git_gate_repos(child_raw):
return {}
git_raw = as_json_object(child_raw.get("git-gate", {}), "child git-gate")
@@ -314,6 +189,8 @@ def _declared_repos_raw(child_raw: dict[str, object]) -> dict[str, object]:
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
from .manifest_util import as_json_object
git_raw = child_raw.get("git-gate")
if git_raw is None:
return False
@@ -326,6 +203,9 @@ def _merge_egress(
child: ManifestEgressConfig,
child_raw: dict[str, object],
) -> ManifestEgressConfig:
from .manifest_egress import ManifestEgressConfig
from .manifest_util import as_json_object
child_egress_raw = as_json_object(child_raw.get("egress"), "child egress")
routes = parent.routes + child.routes
log = child.Log if "log" in child_egress_raw else parent.Log
+6 -23
View File
@@ -3,10 +3,9 @@
from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
from .log import warn
from .manifest_bottle import ManifestBottle
from .manifest_extends import resolve_bottles
from .manifest_schema import (
entity_name_from_path,
validate_bottle_frontmatter_keys,
@@ -14,6 +13,9 @@ from .manifest_schema import (
from .manifest_util import ManifestError
from .yaml_subset import YamlSubsetError, parse_frontmatter
if TYPE_CHECKING:
from .manifest import ManifestBottle
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
@@ -30,25 +32,6 @@ def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
)
def scan_bottle_names(bottles_dir: Path) -> list[str]:
"""Scan `<bottles_dir>/*.md` for valid filenames and return sorted bottle names.
No file content is read. Invalid filenames are skipped with a warning."""
result: list[str] = []
if not bottles_dir.is_dir():
return result
for path in sorted(bottles_dir.glob("*.md")):
name = entity_name_from_path(path)
if name is None:
warn(
f"skipping {path}: filename must match "
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
)
continue
result.append(name)
return result
def scan_agent_names(agents_dir: Path) -> dict[str, Path]:
"""Scan `<agents_dir>/*.md` for valid filenames and return `{name: path}`.
@@ -76,6 +59,8 @@ def load_bottle_chain_from_dir(
Only the files in the extends chain are read unrelated bottle files
are never touched. Raises ManifestError on parse or validation failure."""
from .manifest_extends import resolve_bottles
raws: dict[str, dict[str, object]] = {}
to_load = [bottle_name]
while to_load:
@@ -102,7 +87,5 @@ def load_bottle_chain_from_dir(
parent = fm.get("extends")
if isinstance(parent, str):
to_load.append(parent)
elif isinstance(parent, list):
to_load.extend(p for p in parent if isinstance(p, str))
return resolve_bottles(raws)[bottle_name]
+3 -10
View File
@@ -18,8 +18,8 @@ _FILENAME_RX = re.compile(r"^[a-z][a-z0-9-]*$")
BOTTLE_KEYS = frozenset(
{"env", "extends", "agent_provider", "git-gate", "egress", "supervise"}
)
AGENT_KEYS_REQUIRED: frozenset[str] = frozenset()
AGENT_KEYS_OPTIONAL = frozenset({"bottle", "skills", "git-gate"})
AGENT_KEYS_REQUIRED = frozenset({"bottle"})
AGENT_KEYS_OPTIONAL = frozenset({"skills", "git-gate"})
# Claude Code subagent fields bot-bottle ignores at launch but does
# not reject. This lets the same file double as
@@ -33,20 +33,13 @@ AGENT_KEYS = (
AGENT_MODEL_KEYS = AGENT_KEYS | frozenset({"prompt"})
def is_valid_entity_name(name: str) -> bool:
"""True if `name` fits the kebab-case `[a-z][a-z0-9-]*` convention
shared by bottle/agent filenames and skill names. Names that satisfy
this are also safe to interpolate into a host/guest path segment."""
return bool(_FILENAME_RX.match(name))
def entity_name_from_path(path: Path) -> str | None:
"""Return the entity name implied by the filename, or None if the
filename does not fit the [a-z][a-z0-9-]* convention."""
if path.suffix != ".md":
return None
stem = path.stem
if not is_valid_entity_name(stem):
if not _FILENAME_RX.match(stem):
return None
return stem
-8
View File
@@ -1,8 +0,0 @@
"""bot-bottle-orchestrator: forge-native orchestration for bot-bottle.
The package is stdlib-only. The core (events, targeting, lifecycle,
watchdog, sidecar, webhook) depends on its collaborators a forge, a
state store, a bottle runner through duck-typed interfaces, so it runs
and tests without bot-bottle installed. `bootstrap` is the single module
that imports `bot_bottle` and wires the concrete implementations.
"""
-51
View File
@@ -1,51 +0,0 @@
"""CLI entry point: `python -m bot_bottle.orchestrator <command>`.
Commands:
run start the webhook server + watchdog + done-signal relay
status print the tracked runs (issue -> slug, status)
"""
from __future__ import annotations
import argparse
import sys
from .config import Config
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="python -m bot_bottle.orchestrator")
sub = parser.add_subparsers(dest="command", required=True)
sub.add_parser("run", help="start the webhook server, watchdog, and relay")
sub.add_parser("status", help="list tracked runs")
args = parser.parse_args(argv)
config = Config.from_env()
if args.command == "run":
from . import bootstrap # pylint: disable=import-outside-toplevel
print(
f"orchestrator listening on "
f"http://{config.webhook_host}:{config.webhook_port}/webhook",
file=sys.stderr,
)
bootstrap.run(config)
return 0
if args.command == "status":
from .bootstrap import ( # pylint: disable=import-outside-toplevel
BotBottleStateStore,
)
store = BotBottleStateStore(config.db_path)
for r in store.all():
pr = f"PR#{r.pr_number}" if r.pr_number else "-"
print(f"{r.owner}/{r.repo}#{r.issue_number}\t{r.slug}\t{r.status}\t{pr}")
return 0
return 2
if __name__ == "__main__":
sys.exit(main())
-155
View File
@@ -1,155 +0,0 @@
"""Wire the concrete bot-bottle implementations into the core.
This is the ONLY module that imports from `bot_bottle.contrib`. It adapts
`SqliteForgeStateStore` to our `StateStore`, builds `GiteaForge`s (and
scope-wrapped forges for sidecars), constructs the `Orchestrator`, and
runs the webhook server + watchdog + done-signal relay.
Imports are direct (no lazy loading) because the orchestrator is now part
of the same package installation.
"""
from __future__ import annotations
import os
import threading
from pathlib import Path
from typing import Any
from ..contrib.forge.base import ScopedForge
from ..contrib.gitea.client import GiteaClient, GiteaForge
from ..contrib.gitea.forge_state import ForgeState, SqliteForgeStateStore
from .config import Config
from .lifecycle import Orchestrator
from .model import RunRecord
from .runner import ProgrammaticBottleRunner
from .sidecar import ForgeSidecar, OpLog, drain_done_events
from .watchdog import Watchdog
from .webhook import WebhookServer
_RELAY_TICK_SECS = 2.0
def _token() -> str:
tok = os.environ.get("GITEA_TOKEN") or os.environ.get("FORGE_GITEA_TOKEN")
if not tok:
raise RuntimeError("set GITEA_TOKEN (or FORGE_GITEA_TOKEN)")
return tok
class BotBottleStateStore:
"""Adapts `SqliteForgeStateStore` to our `StateStore`, translating
`RunRecord` <-> `ForgeState` field-for-field."""
def __init__(self, db_path: Path | None) -> None:
self._inner = SqliteForgeStateStore(db_path)
def upsert(self, record: RunRecord) -> None:
self._inner.upsert(_to_forge_state(record))
def get(self, owner: str, repo: str, issue_number: int) -> RunRecord | None:
state = self._inner.get(owner, repo, issue_number)
return _to_record(state) if state is not None else None
def delete(self, owner: str, repo: str, issue_number: int) -> None:
self._inner.delete(owner, repo, issue_number)
def all(self) -> list[RunRecord]:
return [_to_record(s) for s in self._inner.all()]
def _to_forge_state(r: RunRecord) -> ForgeState:
return ForgeState(
owner=r.owner, repo=r.repo, issue_number=r.issue_number, slug=r.slug,
agent_name=r.agent_name, bottle_names=list(r.bottle_names),
backend_name=r.backend_name, agent_git_user=r.agent_git_user,
pr_number=r.pr_number, status=r.status, last_checkin_at=r.last_checkin_at,
)
def _to_record(s: ForgeState) -> RunRecord:
return RunRecord(
owner=s.owner, repo=s.repo, issue_number=s.issue_number, slug=s.slug,
agent_name=s.agent_name, bottle_names=list(s.bottle_names),
backend_name=s.backend_name, agent_git_user=s.agent_git_user,
pr_number=s.pr_number, status=s.status, last_checkin_at=s.last_checkin_at,
)
def make_forge(config: Config, owner: str, repo: str) -> Any:
"""A `GiteaForge` bound to one repo."""
client = GiteaClient(
api_url=config.gitea_api, owner=owner, repo=repo, token=_token()
)
return GiteaForge(client)
def make_sidecar(
config: Config, owner: str, repo: str, issue_number: int, assigned_prs: list[int]
) -> ForgeSidecar:
"""A scope-enforced sidecar for one run (read-anywhere / write-scoped)."""
scoped = ScopedForge(
make_forge(config, owner, repo),
assigned_issue=issue_number,
assigned_prs=assigned_prs,
)
op_log = OpLog(config.queue_dir / f"{owner}-{repo}-{issue_number}.oplog.jsonl")
return ForgeSidecar(
forge=scoped,
op_log=op_log,
queue_dir=config.queue_dir,
run_key=(owner, repo, issue_number),
)
def build(config: Config) -> tuple[WebhookServer, Watchdog, Orchestrator]:
store = BotBottleStateStore(config.db_path)
runner = ProgrammaticBottleRunner()
membership_forge = make_forge(config, "_", "_")
orchestrator = Orchestrator(
forge=membership_forge,
store=store,
runner=runner,
org=config.forge_org,
gitea_api=config.gitea_api,
forge_env_base={
"GITEA_TOKEN": _token(),
"FORGE_QUEUE_DIR": str(config.queue_dir),
"FORGE_SIDECAR_SOCKET": str(config.sidecar_socket),
},
)
watchdog = Watchdog(
store=store, runner=runner, timeout_secs=config.watchdog_timeout_secs
)
server = WebhookServer(
(config.webhook_host, config.webhook_port),
orchestrator=orchestrator,
store=store,
)
return server, watchdog, orchestrator
def _relay_loop(config: Config, orchestrator: Orchestrator, stop: threading.Event) -> None:
while not stop.wait(_RELAY_TICK_SECS):
for ev in drain_done_events(config.queue_dir):
orchestrator.on_done_signal(
ev["owner"], ev["repo"], int(ev["issue_number"]),
str(ev.get("status", "")), str(ev.get("summary", "")),
)
def run(config: Config) -> None:
"""Blocking run: webhook server + watchdog + done-signal relay."""
server, watchdog, orchestrator = build(config)
watchdog.start()
stop = threading.Event()
relay = threading.Thread(
target=_relay_loop, args=(config, orchestrator, stop), daemon=True
)
relay.start()
try:
server.serve_forever()
finally:
stop.set()
watchdog.stop()
server.server_close()
-50
View File
@@ -1,50 +0,0 @@
"""Configuration, loaded from the environment (stdlib `os` only).
Everything the orchestrator needs to run is an env var so a deploy is a
process with an environment, no config file to manage. `FORGE_*` names
match the bot-bottle forge-native PRD.
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
# The label that marks an issue as agent-targeted: `bot-bottle:<agent>`.
LABEL_PREFIX = "bot-bottle:"
# Optional bottle override: `bot-bottle-bottle:<name>`.
BOTTLE_LABEL_PREFIX = "bot-bottle-bottle:"
@dataclass(frozen=True)
class Config:
"""Resolved orchestrator configuration."""
forge_org: str
gitea_api: str
watchdog_timeout_secs: int
webhook_host: str
webhook_port: int
queue_dir: Path
sidecar_socket: Path
db_path: Path | None
@staticmethod
def from_env(env: dict[str, str] | None = None) -> "Config":
e = os.environ if env is None else env
home = Path(e.get("HOME", str(Path.home())))
default_root = home / ".bot-bottle"
db = e.get("FORGE_DB_PATH")
return Config(
forge_org=e.get("FORGE_ORG", "bot-bottle"),
gitea_api=e.get("FORGE_GITEA_API", ""),
watchdog_timeout_secs=int(e.get("FORGE_WATCHDOG_TIMEOUT", "1800")),
webhook_host=e.get("FORGE_WEBHOOK_HOST", "127.0.0.1"),
webhook_port=int(e.get("FORGE_WEBHOOK_PORT", "8477")),
queue_dir=Path(e.get("FORGE_QUEUE_DIR", str(default_root / "forge-queue"))),
sidecar_socket=Path(
e.get("FORGE_SIDECAR_SOCKET", str(default_root / "forge-sidecar.sock"))
),
db_path=Path(db) if db else None,
)
-85
View File
@@ -1,85 +0,0 @@
"""Parse Gitea webhook payloads into typed `ForgeEvent`s.
Only the fields the orchestrator acts on are extracted; unknown payloads
and event types return None so the webhook layer can ignore them.
Gitea sends the event kind in the `X-Gitea-Event` header and the payload
as JSON. The relevant kinds:
- `issues` with `action == "assigned"` -> IssueAssigned
- `issue_comment` with `action == "created"` -> CommentCreated
- `pull_request` with `action == "closed"` -> PullRequestClosed
"""
from __future__ import annotations
from typing import Any
from .model import CommentCreated, ForgeEvent, IssueAssigned, PullRequestClosed
def _repo_owner(payload: dict[str, Any]) -> tuple[str, str]:
repo = payload.get("repository") or {}
owner = (repo.get("owner") or {}).get("login", "")
return str(owner), str(repo.get("name", ""))
def parse_event(event_kind: str, payload: dict[str, Any]) -> ForgeEvent | None:
"""Map (X-Gitea-Event, payload) to a `ForgeEvent`, or None to ignore."""
if event_kind == "issues":
return _parse_issue(payload)
if event_kind == "issue_comment":
return _parse_comment(payload)
if event_kind == "pull_request":
return _parse_pull_request(payload)
return None
def _parse_issue(payload: dict[str, Any]) -> IssueAssigned | None:
if payload.get("action") != "assigned":
return None
owner, repo = _repo_owner(payload)
issue = payload.get("issue") or {}
assignees = tuple(
str(a.get("login", "")) for a in (issue.get("assignees") or [])
)
labels = tuple(str(l.get("name", "")) for l in (issue.get("labels") or []))
return IssueAssigned(
owner=owner,
repo=repo,
issue_number=int(issue.get("number", 0)),
title=str(issue.get("title", "")),
body=str(issue.get("body", "") or ""),
assignees=assignees,
labels=labels,
)
def _parse_comment(payload: dict[str, Any]) -> CommentCreated | None:
if payload.get("action") != "created":
return None
owner, repo = _repo_owner(payload)
issue = payload.get("issue") or {}
comment = payload.get("comment") or {}
return CommentCreated(
owner=owner,
repo=repo,
issue_number=int(issue.get("number", 0)),
comment_id=int(comment.get("id", 0)),
author=str((comment.get("user") or {}).get("login", "")),
body=str(comment.get("body", "") or ""),
is_pull=bool(issue.get("pull_request")),
)
def _parse_pull_request(payload: dict[str, Any]) -> PullRequestClosed | None:
if payload.get("action") != "closed":
return None
owner, repo = _repo_owner(payload)
pr = payload.get("pull_request") or {}
return PullRequestClosed(
owner=owner,
repo=repo,
pr_number=int(pr.get("number", 0)),
merged=bool(pr.get("merged", False)),
)
-180
View File
@@ -1,180 +0,0 @@
"""The orchestration lifecycle: forge events -> bottle transitions.
`Orchestrator.handle(event)` is the single entry point the webhook layer
calls. `on_done_signal(...)` is called by the sidecar relay when an agent
signals completion. All collaborators (forge, store, runner) are
injected and duck-typed; `now` and `label_for` are injectable for tests.
Transitions:
IssueAssigned (targeted, new) -> start bottle, record = running
signal_done (running) -> freeze bottle, record = frozen
CommentCreated (frozen) -> resume bottle, record = running
PullRequestClosed (tracked) -> destroy bottle, record removed
"""
from __future__ import annotations
from collections.abc import Callable
from datetime import datetime
from .model import (
STATUS_DESTROYED,
STATUS_FROZEN,
STATUS_RUNNING,
CommentCreated,
ForgeEvent,
IssueAssigned,
PullRequestClosed,
RunRecord,
)
from .runner import BottleRunner
from .store import StateStore
from .targeting import Membership, Target, resolve_target
def _iso_now() -> str:
return datetime.now().astimezone().isoformat(timespec="seconds")
def _default_label(agent: str, event: IssueAssigned) -> str:
# Embed the issue identity so slugs are unique per issue and never
# get renamed on collision.
return f"{agent}-{event.owner}-{event.repo}-{event.issue_number}"
class Orchestrator:
def __init__(
self,
*,
forge: Membership,
store: StateStore,
runner: BottleRunner,
org: str,
gitea_api: str = "",
forge_env_base: dict[str, str] | None = None,
now: Callable[[], str] = _iso_now,
label_for: Callable[[str, IssueAssigned], str] = _default_label,
) -> None:
self._forge = forge
self._store = store
self._runner = runner
self._org = org
self._gitea_api = gitea_api
self._forge_env_base = forge_env_base or {}
self._now = now
self._label_for = label_for
# --- entry points ------------------------------------------------------
def handle(self, event: ForgeEvent) -> None:
if isinstance(event, IssueAssigned):
self._on_issue_assigned(event)
elif isinstance(event, CommentCreated):
self._on_comment(event)
else:
self._on_pr_closed(event)
def on_done_signal( # pylint: disable=unused-argument
self, owner: str, repo: str, issue_number: int, status: str, summary: str
) -> None:
"""Sidecar relay: an agent signalled completion. Freeze the bottle.
`status`/`summary` are recorded by provenance (via the op log), not
acted on here."""
record = self._store.get(owner, repo, issue_number)
if record is None or record.status != STATUS_RUNNING:
return
self._runner.freeze(record.slug)
record.status = STATUS_FROZEN
record.last_checkin_at = self._now()
self._store.upsert(record)
def link_pr(self, owner: str, repo: str, issue_number: int, pr_number: int) -> None:
"""Record the PR a tracked issue produced, so PR comments and the
PR-close event route back to this record."""
record = self._store.get(owner, repo, issue_number)
if record is not None:
record.pr_number = pr_number
self._store.upsert(record)
# --- handlers ----------------------------------------------------------
def _on_issue_assigned(self, event: IssueAssigned) -> None:
target = resolve_target(event, self._forge, self._org)
if target is None:
return
# Idempotent: a webhook redelivery must not launch a second bottle.
if self._store.get(event.owner, event.repo, event.issue_number) is not None:
return
self._launch(event, target)
def _launch(self, event: IssueAssigned, target: Target) -> None:
label = self._label_for(target.agent_name, event)
bottles = [target.bottle_override] if target.bottle_override else []
slug = self._runner.start(
agent=target.agent_name,
bottles=bottles,
label=label,
prompt=event.body,
forge_env=self._forge_env(event.owner, event.repo, event.issue_number),
)
self._store.upsert(
RunRecord(
owner=event.owner,
repo=event.repo,
issue_number=event.issue_number,
slug=slug,
agent_name=target.agent_name,
bottle_names=bottles,
status=STATUS_RUNNING,
last_checkin_at=self._now(),
)
)
def _on_comment(self, event: CommentCreated) -> None:
record = self._route_comment(event)
if record is None or record.status != STATUS_FROZEN:
return
# Echo-loop guard: ignore the agent's own comments.
if record.agent_git_user and event.author == record.agent_git_user:
return
self._runner.resume(record.slug, event.body)
record.status = STATUS_RUNNING
record.last_checkin_at = self._now()
self._store.upsert(record)
def _route_comment(self, event: CommentCreated) -> RunRecord | None:
# A comment on the issue routes by issue number; a comment on a PR
# routes by the recorded pr_number.
direct = self._store.get(event.owner, event.repo, event.issue_number)
if direct is not None:
return direct
if event.is_pull:
return self._find_by_pr(event.owner, event.repo, event.issue_number)
return None
def _on_pr_closed(self, event: PullRequestClosed) -> None:
record = self._find_by_pr(event.owner, event.repo, event.pr_number)
if record is None:
return
self._runner.destroy(record.slug)
record.status = STATUS_DESTROYED
self._store.delete(record.owner, record.repo, record.issue_number)
def _find_by_pr(self, owner: str, repo: str, pr_number: int) -> RunRecord | None:
for record in self._store.all():
if (
record.owner == owner
and record.repo == repo
and record.pr_number == pr_number
):
return record
return None
def _forge_env(self, owner: str, repo: str, issue_number: int) -> dict[str, str]:
env = dict(self._forge_env_base)
if self._gitea_api:
env["FORGE_GITEA_API"] = self._gitea_api
env["FORGE_OWNER"] = owner
env["FORGE_REPO"] = repo
env["FORGE_ISSUE_NUMBER"] = str(issue_number)
return env
-108
View File
@@ -1,108 +0,0 @@
"""Domain model: run records, forge events, provenance.
These are the orchestrator's own dataclasses. `RunRecord` mirrors
bot-bottle's `ForgeState` field-for-field so the bootstrap adapter can
translate between them with no loss; keeping our own copy is what lets
the core stay import-free of bot-bottle.
"""
from __future__ import annotations
from dataclasses import dataclass, field
# Run lifecycle. A bottle is launched (running), frozen on the done
# signal, and destroyed when the PR closes.
STATUS_RUNNING = "running"
STATUS_FROZEN = "frozen"
STATUS_DESTROYED = "destroyed"
@dataclass
class RunRecord:
"""One forge-targeted issue's bottle lifecycle record."""
owner: str
repo: str
issue_number: int
slug: str
agent_name: str
bottle_names: list[str] = field(default_factory=list)
backend_name: str = ""
agent_git_user: str = ""
pr_number: int | None = None
status: str = STATUS_RUNNING
last_checkin_at: str = ""
# --- Forge events (parsed webhook payloads) --------------------------------
@dataclass(frozen=True)
class IssueAssigned:
"""An issue gained an assignee — the trigger to consider a launch."""
owner: str
repo: str
issue_number: int
title: str
body: str
assignees: tuple[str, ...]
labels: tuple[str, ...]
@dataclass(frozen=True)
class CommentCreated:
"""A comment was posted on an issue or PR — a rehydrate trigger."""
owner: str
repo: str
issue_number: int
comment_id: int
author: str
body: str
is_pull: bool
@dataclass(frozen=True)
class PullRequestClosed:
"""A PR closed (merged or not) — the teardown trigger."""
owner: str
repo: str
pr_number: int
merged: bool
# Union of everything the webhook layer can emit.
ForgeEvent = IssueAssigned | CommentCreated | PullRequestClosed
# --- Provenance ------------------------------------------------------------
@dataclass(frozen=True)
class ForgeOp:
"""One semantic forge operation the sidecar recorded."""
at: str # ISO timestamp
op: str # e.g. "post_comment", "read_pr", "signal_done"
target: int | None
detail: str
@dataclass(frozen=True)
class Provenance:
"""The audit record for one run, served by the provenance API. Never
posted into the forge."""
slug: str
owner: str
repo: str
issue_number: int
agent_name: str
bottle_names: tuple[str, ...]
started_at: str
finished_at: str
exit_code: int | None
watchdog_fired: bool
ops: tuple[ForgeOp, ...]
-71
View File
@@ -1,71 +0,0 @@
"""Provenance assembly + serialization.
Provenance is the run's audit record: the `RunRecord` metadata plus the
sidecar's semantic operation log. It is exposed through the provenance
API (see `webhook.ProvenanceHandler`) and deliberately never posted back
into the forge a mutable PR comment is not an audit record.
This module only assembles and serializes; retention/signing of the
record is a control-plane concern out of scope here.
"""
from __future__ import annotations
from typing import Any
from .model import ForgeOp, Provenance, RunRecord
def ops_from_log(entries: list[dict[str, Any]]) -> tuple[ForgeOp, ...]:
return tuple(
ForgeOp(
at=str(e.get("at", "")),
op=str(e.get("op", "")),
target=e.get("target"),
detail=str(e.get("detail", "")),
)
for e in entries
)
def build_provenance(
record: RunRecord,
*,
ops: tuple[ForgeOp, ...],
started_at: str,
finished_at: str,
exit_code: int | None,
watchdog_fired: bool,
) -> Provenance:
return Provenance(
slug=record.slug,
owner=record.owner,
repo=record.repo,
issue_number=record.issue_number,
agent_name=record.agent_name,
bottle_names=tuple(record.bottle_names),
started_at=started_at,
finished_at=finished_at,
exit_code=exit_code,
watchdog_fired=watchdog_fired,
ops=ops,
)
def provenance_to_dict(p: Provenance) -> dict[str, Any]:
return {
"slug": p.slug,
"owner": p.owner,
"repo": p.repo,
"issue_number": p.issue_number,
"agent": p.agent_name,
"bottles": list(p.bottle_names),
"started_at": p.started_at,
"finished_at": p.finished_at,
"exit_code": p.exit_code,
"watchdog_fired": p.watchdog_fired,
"ops": [
{"at": o.at, "op": o.op, "target": o.target, "detail": o.detail}
for o in p.ops
],
}
-83
View File
@@ -1,83 +0,0 @@
"""Bottle runner: drive bot_bottle to manage a bottle's life.
`BottleRunner` is the interface the lifecycle depends on;
`ProgrammaticBottleRunner` calls into the bot_bottle Python API directly
(no subprocess). The slug returned by `start` is the actual slug minted
at launch time not a post-hoc derivation from the label so it is
authoritative even if bot-bottle's slugification logic changes.
`slugify` is retained for `FakeRunner` (tests) and for the label scheme
the orchestrator uses to predict collision-free slugs.
"""
from __future__ import annotations
import re
from collections.abc import Sequence
from typing import Protocol
class BottleRunner(Protocol):
def start(
self,
*,
agent: str,
bottles: Sequence[str],
label: str,
prompt: str,
forge_env: dict[str, str],
) -> str: ...
def freeze(self, slug: str) -> None: ...
def resume(self, slug: str, prompt: str) -> None: ...
def destroy(self, slug: str) -> None: ...
_SLUG_RE = re.compile(r"[^a-z0-9]+")
def slugify(label: str) -> str:
"""Lowercase, collapse non-alphanumerics to single hyphens, strip
leading/trailing hyphens matches bot-bottle's slug rule."""
return _SLUG_RE.sub("-", label.lower()).strip("-")
class ProgrammaticBottleRunner:
"""Calls into the bot_bottle Python API directly — no subprocess.
Imports are deferred to call time so tests can inject a mock into
sys.modules['bot_bottle.api'] before calling runner methods.
bot_bottle.api is added in the forge-native-integration PR (#318),
which merges before this one."""
def start(
self,
*,
agent: str,
bottles: Sequence[str],
label: str,
prompt: str,
forge_env: dict[str, str],
) -> str:
from bot_bottle import api # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
return api.start_headless(
agent,
prompt=prompt,
bottles=list(bottles) or None,
label=label,
forge_env=forge_env,
)
def freeze(self, slug: str) -> None:
from bot_bottle import api # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
api.freeze(slug)
def resume(self, slug: str, prompt: str) -> None:
from bot_bottle import api # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
api.resume_headless(slug, prompt=prompt)
def destroy(self, slug: str) -> None:
from bot_bottle import api # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
api.destroy(slug)
-171
View File
@@ -1,171 +0,0 @@
"""Forge sidecar: the agent's only door to the forge.
The agent calls the sidecar over a line-delimited JSON-RPC AF_UNIX
socket; the sidecar dispatches to an injected `forge` (already
scope-wrapped by bootstrap) and holds the token, so the agent never sees
a credential or a forge endpoint. Every call is appended to a semantic
operation log (the provenance raw material). `signal_done` additionally
drops an event file in the queue dir the orchestrator drains.
`dispatch` is pure and testable; `serve` wraps it in a socket server.
"""
from __future__ import annotations
import dataclasses
import json
import socketserver
import uuid
from collections.abc import Callable
from datetime import datetime
from pathlib import Path
from typing import Any
_READ_METHODS = {"read_issue", "read_pr", "read_comments"}
_WRITE_METHODS = {"post_comment", "update_description"}
def _iso_now() -> str:
return datetime.now().astimezone().isoformat(timespec="seconds")
def _jsonable(value: Any) -> Any:
if dataclasses.is_dataclass(value) and not isinstance(value, type):
return dataclasses.asdict(value)
if isinstance(value, list):
return [_jsonable(v) for v in value]
return value
class OpLog:
"""Append-only JSONL log of semantic forge operations."""
def __init__(self, path: Path, *, now: Callable[[], str] = _iso_now) -> None:
self._path = path
self._now = now
path.parent.mkdir(parents=True, exist_ok=True)
def record(self, op: str, target: int | None, detail: str) -> None:
entry = {"at": self._now(), "op": op, "target": target, "detail": detail}
with self._path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(entry) + "\n")
def read(self) -> list[dict[str, Any]]:
if not self._path.exists():
return []
return [
json.loads(line)
for line in self._path.read_text(encoding="utf-8").splitlines()
if line.strip()
]
def write_done_event(queue_dir: Path, event: dict[str, Any]) -> Path:
"""Atomically drop a done-signal event file in the queue dir."""
queue_dir.mkdir(parents=True, exist_ok=True)
path = queue_dir / f"done-{uuid.uuid4().hex}.json"
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(event), encoding="utf-8")
tmp.replace(path)
return path
def drain_done_events(queue_dir: Path) -> list[dict[str, Any]]:
"""Read and remove every queued done-signal event."""
if not queue_dir.is_dir():
return []
events: list[dict[str, Any]] = []
for path in sorted(queue_dir.glob("done-*.json")):
try:
events.append(json.loads(path.read_text(encoding="utf-8")))
except (OSError, ValueError):
continue
finally:
path.unlink(missing_ok=True)
return events
class ForgeSidecar:
"""Dispatches sidecar protocol calls to the forge, logging each and
relaying `signal_done` to the queue dir. `run_key` is the
(owner, repo, issue_number) the run is bound to."""
def __init__(
self,
*,
forge: object,
op_log: OpLog,
queue_dir: Path,
run_key: tuple[str, str, int],
) -> None:
self._forge = forge
self._log = op_log
self._queue_dir = queue_dir
self._owner, self._repo, self._issue = run_key
def dispatch(self, method: str, params: dict[str, Any]) -> dict[str, Any]:
try:
result = self._invoke(method, params)
except Exception as exc: # noqa: BLE001 # pylint: disable=broad-exception-caught
self._log.record(method, params.get("number"), f"error: {exc}")
return {"ok": False, "error": str(exc)}
return {"ok": True, "result": result}
def _invoke(self, method: str, params: dict[str, Any]) -> Any:
if method in _READ_METHODS:
number = int(params["number"])
result = getattr(self._forge, method)(number)
self._log.record(method, number, "ok")
return _jsonable(result)
if method in _WRITE_METHODS:
number = int(params["number"])
getattr(self._forge, method)(number, params["body"])
self._log.record(method, number, "ok")
return None
if method == "signal_done":
status = str(params.get("status", ""))
summary = str(params.get("summary", ""))
self._log.record("signal_done", None, f"{status}: {summary}")
write_done_event(
self._queue_dir,
{
"owner": self._owner,
"repo": self._repo,
"issue_number": self._issue,
"status": status,
"summary": summary,
},
)
return None
raise ValueError(f"unknown method: {method}")
class _Handler(socketserver.StreamRequestHandler):
def handle(self) -> None:
line = self.rfile.readline()
if not line:
return
try:
req = json.loads(line)
except ValueError:
self.wfile.write(b'{"ok": false, "error": "invalid json"}\n')
return
resp = self.server.sidecar.dispatch( # type: ignore[attr-defined]
str(req.get("method", "")), dict(req.get("params", {}))
)
self.wfile.write((json.dumps(resp) + "\n").encode())
class _Server(socketserver.ThreadingUnixStreamServer):
def __init__(self, socket_path: str, sidecar: ForgeSidecar) -> None:
super().__init__(socket_path, _Handler)
self.sidecar = sidecar
def serve(sidecar: ForgeSidecar, socket_path: Path) -> _Server:
"""Bind a threaded AF_UNIX server for `sidecar`. Caller runs
`serve_forever()` (or `handle_request()` in tests) and closes it."""
if socket_path.exists():
socket_path.unlink()
socket_path.parent.mkdir(parents=True, exist_ok=True)
return _Server(str(socket_path), sidecar)
-48
View File
@@ -1,48 +0,0 @@
"""State store interface + an in-memory implementation.
The orchestrator persists one `RunRecord` per forge-targeted issue. At
runtime `bootstrap` supplies an adapter over bot-bottle's
`SqliteForgeStateStore`; the in-memory store here backs tests and a
`--no-bot-bottle` dry mode.
"""
from __future__ import annotations
from typing import Protocol
from .model import RunRecord
class StateStore(Protocol):
"""Thin CRUD surface. Mirrors bot-bottle's `ForgeStateStore` so the
bootstrap adapter is a straight pass-through."""
def upsert(self, record: RunRecord) -> None: ...
def get(self, owner: str, repo: str, issue_number: int) -> RunRecord | None: ...
def delete(self, owner: str, repo: str, issue_number: int) -> None: ...
def all(self) -> list[RunRecord]: ...
class InMemoryStateStore:
"""Dict-backed `StateStore`, keyed by (owner, repo, issue_number)."""
def __init__(self) -> None:
self._by_key: dict[tuple[str, str, int], RunRecord] = {}
def upsert(self, record: RunRecord) -> None:
self._by_key[(record.owner, record.repo, record.issue_number)] = record
def get(self, owner: str, repo: str, issue_number: int) -> RunRecord | None:
return self._by_key.get((owner, repo, issue_number))
def delete(self, owner: str, repo: str, issue_number: int) -> None:
self._by_key.pop((owner, repo, issue_number), None)
def all(self) -> list[RunRecord]:
return sorted(
self._by_key.values(),
key=lambda r: (r.owner, r.repo, r.issue_number),
)
-51
View File
@@ -1,51 +0,0 @@
"""Decide whether an assigned issue is agent-targeted, and for whom.
An issue is forge-targeted when BOTH hold:
- it carries a `bot-bottle:<agent>` label naming the agent, and
- at least one assignee is a member of the configured org.
An optional `bot-bottle-bottle:<name>` label overrides bottle selection.
The forge is duck-typed: any object with `is_org_member(org, user)`.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol
from .config import BOTTLE_LABEL_PREFIX, LABEL_PREFIX
from .model import IssueAssigned
class Membership(Protocol):
def is_org_member(self, org: str, username: str) -> bool: ...
@dataclass(frozen=True)
class Target:
agent_name: str
bottle_override: str | None
def parse_labels(labels: tuple[str, ...]) -> tuple[str | None, str | None]:
"""Return (agent_name, bottle_override) parsed from labels."""
agent: str | None = None
bottle: str | None = None
for label in labels:
if label.startswith(BOTTLE_LABEL_PREFIX):
bottle = label[len(BOTTLE_LABEL_PREFIX):] or None
elif label.startswith(LABEL_PREFIX):
agent = label[len(LABEL_PREFIX):] or None
return agent, bottle
def resolve_target(
event: IssueAssigned, forge: Membership, org: str
) -> Target | None:
"""Return the `Target` for a forge-targeted issue, or None to ignore."""
agent, bottle = parse_labels(event.labels)
if not agent:
return None
if not any(forge.is_org_member(org, a) for a in event.assignees):
return None
return Target(agent_name=agent, bottle_override=bottle)
-68
View File
@@ -1,68 +0,0 @@
"""Watchdog: freeze runs whose agent exited without signalling done.
`sweep(now)` is the pure, testable core: any `running` record whose
`last_checkin_at` is older than the timeout is frozen as
done-without-self-report and returned so provenance can flag it.
`Watchdog.start()` runs `sweep` on a daemon thread once a minute.
"""
from __future__ import annotations
import threading
from datetime import datetime, timedelta
from .model import STATUS_FROZEN, STATUS_RUNNING, RunRecord
from .runner import BottleRunner
from .store import StateStore
_TICK_SECS = 60.0
def _parse(ts: str) -> datetime | None:
try:
return datetime.fromisoformat(ts)
except (ValueError, TypeError):
return None
class Watchdog:
def __init__(
self,
*,
store: StateStore,
runner: BottleRunner,
timeout_secs: int,
) -> None:
self._store = store
self._runner = runner
self._timeout = timedelta(seconds=timeout_secs)
self._stop = threading.Event()
self._thread: threading.Thread | None = None
def sweep(self, now: datetime) -> list[RunRecord]:
"""Freeze stale running records. Returns the ones fired."""
fired: list[RunRecord] = []
for record in self._store.all():
if record.status != STATUS_RUNNING:
continue
checkin = _parse(record.last_checkin_at)
if checkin is None or now - checkin <= self._timeout:
continue
self._runner.freeze(record.slug)
record.status = STATUS_FROZEN
self._store.upsert(record)
fired.append(record)
return fired
def start(self) -> None:
self._thread = threading.Thread(target=self._loop, daemon=True)
self._thread.start()
def stop(self) -> None:
self._stop.set()
if self._thread is not None:
self._thread.join(timeout=_TICK_SECS)
def _loop(self) -> None:
while not self._stop.wait(_TICK_SECS):
self.sweep(datetime.now().astimezone())
-123
View File
@@ -1,123 +0,0 @@
"""HTTP surface: the Gitea webhook receiver and the provenance API.
`POST /webhook` a Gitea event; parsed and dispatched to the orchestrator.
`GET /healthz` liveness.
`GET /provenance?owner=&repo=&issue=` the run's audit record (never
posted to the forge).
Webhook signature verification is optional: set a secret and the handler
rejects bodies whose `X-Gitea-Signature` HMAC-SHA256 does not match.
"""
from __future__ import annotations
import hmac
import json
from collections.abc import Callable
from hashlib import sha256
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any
from urllib.parse import parse_qs, urlparse
from .events import parse_event
from .lifecycle import Orchestrator
from .provenance import build_provenance, ops_from_log, provenance_to_dict
from .store import StateStore
# (record) -> that run's op-log entries, injected by bootstrap.
OpLogReader = Callable[[Any], list[dict[str, Any]]]
class WebhookServer(ThreadingHTTPServer):
def __init__(
self,
address: tuple[str, int],
*,
orchestrator: Orchestrator,
store: StateStore,
secret: bytes | None = None,
op_log_reader: OpLogReader | None = None,
) -> None:
super().__init__(address, _Handler)
self.orchestrator = orchestrator
self.store = store
self.secret = secret
self.op_log_reader = op_log_reader
def verify_signature(secret: bytes, body: bytes, signature: str) -> bool:
expected = hmac.new(secret, body, sha256).hexdigest()
return hmac.compare_digest(expected, signature or "")
class _Handler(BaseHTTPRequestHandler):
server: WebhookServer # type: ignore[assignment]
def log_message( # pylint: disable=redefined-builtin
self, format: str, *args: Any
) -> None: # quiet by default
pass
def _send(self, code: int, payload: dict[str, Any]) -> None:
body = json.dumps(payload).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_POST(self) -> None: # noqa: N802 # pylint: disable=invalid-name
if urlparse(self.path).path != "/webhook":
self._send(404, {"error": "not found"})
return
length = int(self.headers.get("Content-Length", "0"))
body = self.rfile.read(length)
if self.server.secret is not None:
sig = self.headers.get("X-Gitea-Signature", "")
if not verify_signature(self.server.secret, body, sig):
self._send(401, {"error": "bad signature"})
return
try:
payload = json.loads(body)
except ValueError:
self._send(400, {"error": "invalid json"})
return
kind = self.headers.get("X-Gitea-Event", "")
event = parse_event(kind, payload)
if event is not None:
self.server.orchestrator.handle(event)
self._send(200, {"ok": True, "handled": event is not None})
def do_GET(self) -> None: # noqa: N802 # pylint: disable=invalid-name
parsed = urlparse(self.path)
if parsed.path == "/healthz":
self._send(200, {"ok": True})
return
if parsed.path == "/provenance":
self._provenance(parse_qs(parsed.query))
return
self._send(404, {"error": "not found"})
def _provenance(self, query: dict[str, list[str]]) -> None:
try:
owner = query["owner"][0]
repo = query["repo"][0]
issue = int(query["issue"][0])
except (KeyError, IndexError, ValueError):
self._send(400, {"error": "owner, repo, issue required"})
return
record = self.server.store.get(owner, repo, issue)
if record is None:
self._send(404, {"error": "no such run"})
return
reader = self.server.op_log_reader
ops = ops_from_log(reader(record) if reader is not None else [])
prov = build_provenance(
record,
ops=ops,
started_at="",
finished_at=record.last_checkin_at,
exit_code=None,
watchdog_fired=False,
)
self._send(200, provenance_to_dict(prov))
+45 -20
View File
@@ -2,10 +2,11 @@
The supervise plane is the per-bottle MCP sidecar plus its host-side
queue/audit support. The sidecar (bot_bottle.supervise_server)
sits on the bottle's internal network and exposes MCP tools the agent
calls when it needs an operator-reviewed egress change:
sits on the bottle's internal network and exposes three MCP tools the
agent calls when it hits a stuck-recovery category:
* egress-block / allow agent proposes a new routes.yaml
* capability-block agent proposes a new agent Dockerfile
Each tool call: the agent passes the full proposed file plus a
justification text. The sidecar validates the proposal syntactically,
@@ -47,18 +48,16 @@ from pathlib import Path
SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100
TOOL_CAPABILITY_BLOCK = "capability-block"
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_ALLOW = "allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
# Written directly by the egress addon (not an agent-facing MCP tool) when an
# outbound DLP token block is routed to the operator for override (PRD 0062).
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
TOOLS: tuple[str, ...] = (
TOOL_EGRESS_ALLOW,
TOOL_ALLOW,
TOOL_CAPABILITY_BLOCK,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
@@ -72,8 +71,12 @@ TOOLS: tuple[str, ...] = (
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
# capability-block has no on-disk config the operator edits in place
# (the Dockerfile is rebuilt, not patched), so it has no audit log
# here — those changes are captured by git history + the rebuild record
# laid down in PRD 0016.
COMPONENT_FOR_TOOL: dict[str, str] = {
TOOL_EGRESS_ALLOW: "egress",
TOOL_ALLOW: "egress",
TOOL_EGRESS_BLOCK: "egress",
}
@@ -87,6 +90,8 @@ STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
ACTION_OPERATOR_EDIT = "operator-edit"
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
CURRENT_CONFIG_DIR_IN_AGENT = "/etc/bot-bottle/current-config"
DEFAULT_POLL_INTERVAL_SEC = 0.5
@@ -429,39 +434,59 @@ def sha256_hex(content: str) -> str:
# --- Sidecar plan + abstract lifecycle -------------------------------------
# Filename of the staged Dockerfile inside the agent's read-only
# current-config mount. The capability-block tool's description
# points the agent at this exact path so it can read the current
# Dockerfile and propose modifications.
#
# routes.yaml + allowlist used to live here too; PRD 0017 chunk 3
# moved them behind the `list-egress-routes` MCP tool (live state
# from egress's introspection endpoint) so the agent always sees
# current data rather than a launch-time snapshot.
CURRENT_CONFIG_DOCKERFILE = "Dockerfile"
@dataclass(frozen=True)
class SupervisePlan:
"""Output of Supervise.prepare; consumed by .start.
`queue_dir` is the host directory bind-mounted into the sidecar
at /run/supervise/queue. `internal_network` is empty at prepare
time; the backend's launch step fills it via dataclasses.replace
before calling .start."""
at /run/supervise/queue. `current_config_dir` is the host
directory bind-mounted (read-only) into the *agent* container
at /etc/bot-bottle/current-config currently holds only the
Dockerfile snapshot (routes.yaml + allowlist moved to the
`list-egress-routes` MCP tool). `internal_network` is
empty at prepare time; the backend's launch step fills it via
dataclasses.replace before calling .start."""
slug: str
queue_dir: Path
current_config_dir: Path
internal_network: str = ""
class Supervise(ABC):
"""Per-bottle supervise sidecar. Encapsulates the host-side
prepare (queue dir staging); the sidecar's start/stop lifecycle
is backend-specific."""
prepare (queue dir + current-config staging); the sidecar's
start/stop lifecycle is backend-specific."""
def prepare(
self,
slug: str,
stage_dir: Path,
) -> SupervisePlan:
"""Stage the per-bottle queue dir on the host. Returns the
plan; `internal_network` must be set by the launch step before
"""Stage the per-bottle queue dir on the host and the
current-config dir under `stage_dir`. Returns the plan;
`internal_network` must be set by the launch step before
.start runs."""
del stage_dir
queue_dir = queue_dir_for_slug(slug)
queue_dir.mkdir(parents=True, exist_ok=True)
current_config_dir = stage_dir / "current-config"
current_config_dir.mkdir(parents=True, exist_ok=True)
return SupervisePlan(
slug=slug,
queue_dir=queue_dir,
current_config_dir=current_config_dir,
)
# --- Helpers ---------------------------------------------------------------
@@ -512,6 +537,8 @@ __all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"COMPONENT_FOR_TOOL",
"CURRENT_CONFIG_DIR_IN_AGENT",
"CURRENT_CONFIG_DOCKERFILE",
"DEFAULT_POLL_INTERVAL_SEC",
"Proposal",
"QUEUE_DIR_IN_CONTAINER",
@@ -527,10 +554,8 @@ __all__ = [
"TOOLS",
"EGRESS_FORWARD_PROXY",
"EGRESS_INTROSPECT_URL",
"TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK",
"TOOL_CAPABILITY_BLOCK",
"TOOL_GITLEAKS_ALLOW",
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"archive_proposal",
"audit_dir",
+127 -106
View File
@@ -1,8 +1,8 @@
"""Supervise sidecar HTTP server (PRD 0013).
Per-bottle MCP server exposing tools the agent calls to propose egress
config changes when stuck. The tools are `egress-allow`,
`egress-block`, and `list-egress-routes`.
Per-bottle MCP server exposing tools the agent calls to propose config
changes when stuck. The tools are `allow`, `egress-block`,
`capability-block`, and `list-egress-routes`.
Each queued tool call:
@@ -47,11 +47,11 @@ from pathlib import Path
try:
# Same-directory imports inside the bundle container; these files are
# COPYed flat under /app by Dockerfile.sidecars.
from egress_addon_core import LOG_OFF, load_config
from egress_addon_core import load_routes
import supervise as _sv
except ModuleNotFoundError:
# Package imports for host-side tests and tooling.
from .egress_addon_core import LOG_OFF, load_config
from .egress_addon_core import load_routes
from . import supervise as _sv
@@ -90,19 +90,19 @@ def parse_jsonrpc(body: bytes) -> JsonRpcRequest:
try:
raw = json.loads(body)
except json.JSONDecodeError as e:
raise _RpcClientError(ERR_PARSE, f"parse error: {e}") from e
raise _RpcError(ERR_PARSE, f"parse error: {e}") from e
if not isinstance(raw, dict):
raise _RpcClientError(ERR_INVALID_REQUEST, "request must be a JSON object")
raise _RpcError(ERR_INVALID_REQUEST, "request must be a JSON object")
if raw.get("jsonrpc") != JSONRPC_VERSION:
raise _RpcClientError(ERR_INVALID_REQUEST, "jsonrpc field must be '2.0'")
raise _RpcError(ERR_INVALID_REQUEST, "jsonrpc field must be '2.0'")
method = raw.get("method")
if not isinstance(method, str):
raise _RpcClientError(ERR_INVALID_REQUEST, "method must be a string")
raise _RpcError(ERR_INVALID_REQUEST, "method must be a string")
params = raw.get("params", {})
if params is None:
params = {}
if not isinstance(params, dict):
raise _RpcClientError(ERR_INVALID_PARAMS, "params must be an object")
raise _RpcError(ERR_INVALID_PARAMS, "params must be an object")
rpc_id = raw.get("id", _NO_ID)
is_notification = rpc_id is _NO_ID
return JsonRpcRequest(
@@ -117,23 +117,12 @@ _NO_ID = object()
class _RpcError(Exception):
"""Base class for all typed RPC errors that surface as JSON-RPC error responses."""
def __init__(self, code: int, message: str):
super().__init__(message)
self.code = code
self.message = message
class _RpcClientError(_RpcError):
"""Caller sent a bad request; returned verbatim, no server-side logging."""
class _RpcInternalError(_RpcError):
"""Server-side fault; logged at ERROR with cause, always returns ERR_INTERNAL."""
def __init__(self, message: str) -> None:
super().__init__(ERR_INTERNAL, message)
def jsonrpc_result(request_id: object, result: object) -> bytes:
payload = {"jsonrpc": JSONRPC_VERSION, "id": request_id, "result": result}
return (json.dumps(payload) + "\n").encode("utf-8")
@@ -151,10 +140,39 @@ def jsonrpc_error(request_id: object, code: int, message: str) -> bytes:
# --- Tool definitions ------------------------------------------------------
# Shared by both proposal tools (egress-allow / egress-block): they take the
# same arguments and differ only in their top-level tool description. Kept as a
# single source of truth so the schema can't drift between the two tools.
_ROUTES_YAML_DESCRIPTION = (
TOOL_DEFINITIONS: list[dict[str, object]] = [
{
"name": _sv.TOOL_LIST_EGRESS_ROUTES,
"description": (
"List the current egress route table — the bottle's "
"allowlist. Returns JSON with one entry per allowed host, "
"each carrying its matches rules (if any) and whether "
"the proxy injects Authorization for the route. Use this "
"before composing an `allow` or `egress-block` proposal so "
"the new routes file extends the live one rather than "
"replacing it."
),
"inputSchema": {
"type": "object",
"properties": {},
"additionalProperties": False,
},
},
{
"name": _sv.TOOL_ALLOW,
"description": (
"Request operator approval to change the bottle's egress "
"allowlist. Pass the full proposed routes.yaml content, not "
"just the new host, plus a justification. Use "
"`list-egress-routes` first so the proposal preserves existing "
"routes."
),
"inputSchema": {
"type": "object",
"properties": {
"routes_yaml": {
"type": "string",
"description": (
"Full proposed /etc/egress/routes.yaml content. "
"Each route entry accepts these keys:\n"
" host: <hostname> (required)\n"
@@ -169,21 +187,9 @@ _ROUTES_YAML_DESCRIPTION = (
" dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format."
)
def _proposal_input_schema() -> dict[str, object]:
"""Build a fresh input schema for a routes.yaml proposal tool. Returns a
new dict per call so the two tool definitions don't alias one object."""
return {
"type": "object",
"properties": {
"routes_yaml": {
"type": "string",
"description": _ROUTES_YAML_DESCRIPTION,
),
},
"justification": {
"type": "string",
@@ -191,38 +197,8 @@ def _proposal_input_schema() -> dict[str, object]:
},
},
"required": ["routes_yaml", "justification"],
}
TOOL_DEFINITIONS: list[dict[str, object]] = [
{
"name": _sv.TOOL_LIST_EGRESS_ROUTES,
"description": (
"List the current egress route table — the bottle's "
"allowlist. Returns JSON with one entry per allowed host, "
"each carrying its matches rules (if any) and whether "
"the proxy injects Authorization for the route. Use this "
"before composing an `egress-allow` or `egress-block` proposal so "
"the new routes file extends the live one rather than "
"replacing it."
),
"inputSchema": {
"type": "object",
"properties": {},
"additionalProperties": False,
},
},
{
"name": _sv.TOOL_EGRESS_ALLOW,
"description": (
"Request operator approval to change the bottle's egress "
"allowlist. Pass the full proposed routes.yaml content, not "
"just the new host, plus a justification. Use "
"`list-egress-routes` first so the proposal preserves existing "
"routes."
),
"inputSchema": _proposal_input_schema(),
},
{
"name": _sv.TOOL_EGRESS_BLOCK,
"description": (
@@ -232,7 +208,65 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
"`list-egress-routes` first so the proposal preserves existing "
"routes."
),
"inputSchema": _proposal_input_schema(),
"inputSchema": {
"type": "object",
"properties": {
"routes_yaml": {
"type": "string",
"description": (
"Full proposed /etc/egress/routes.yaml content. "
"Each route entry accepts these keys:\n"
" host: <hostname> (required)\n"
" auth_scheme: Bearer|token (must pair with token_env)\n"
" token_env: <ENV_VAR_NAME> (must pair with auth_scheme)\n"
" matches: (optional list of match entries)\n"
" - paths: [{type: prefix|exact|regex, value: /...}]\n"
" methods: [GET, POST, ...]\n"
" headers: [{name: X-Hdr, value: val, type: exact|regex}]\n"
" git: (optional; omit to block git clone/fetch)\n"
" fetch: true\n"
" dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n"
"Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format."
),
},
"justification": {
"type": "string",
"description": "Why this egress route is needed.",
},
},
"required": ["routes_yaml", "justification"],
},
},
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"description": (
"Call when the bottle is missing a tool, skill, permission, "
"or env var you need — something that lives in the agent "
"Dockerfile rather than in the egress routes. "
"Read the current Dockerfile from "
"/etc/bot-bottle/current-config/Dockerfile, compose a "
"modified version, and pass the full new file plus a "
"justification. On approval the supervisor rebuilds the "
"bottle from the new Dockerfile and starts a replacement on "
"the same branch (wired in PRD 0016; v1 acknowledges only)."
),
"inputSchema": {
"type": "object",
"properties": {
"dockerfile": {
"type": "string",
"description": "Full proposed Dockerfile content.",
},
"justification": {
"type": "string",
"description": "Why this capability is needed.",
},
},
"required": ["dockerfile", "justification"],
},
},
]
@@ -240,7 +274,8 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
# Map each proposal tool to the input field that carries the agent's
# payload (stored in Proposal.proposed_file).
PROPOSED_FILE_FIELD: dict[str, str] = {
_sv.TOOL_EGRESS_ALLOW: "routes_yaml",
_sv.TOOL_ALLOW: "routes_yaml",
_sv.TOOL_CAPABILITY_BLOCK: "dockerfile",
_sv.TOOL_EGRESS_BLOCK: "routes_yaml",
}
@@ -253,22 +288,21 @@ def validate_proposed_file(tool: str, content: str) -> None:
catches obvious paste-errors / wrong-tool selections before they
enter the queue."""
if not content.strip():
raise _RpcClientError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
if tool in (_sv.TOOL_EGRESS_ALLOW, _sv.TOOL_EGRESS_BLOCK):
raise _RpcError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
if tool == _sv.TOOL_CAPABILITY_BLOCK:
# Dockerfiles are too varied to validate syntactically beyond
# non-empty. The operator reads the diff in the TUI.
pass
elif tool in (_sv.TOOL_ALLOW, _sv.TOOL_EGRESS_BLOCK):
try:
config = load_config(content)
load_routes(content)
except ValueError as e:
raise _RpcClientError(
raise _RpcError(
ERR_INVALID_PARAMS,
f"{tool}: proposed routes.yaml is not valid: {e}",
) from e
if config.log != LOG_OFF:
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{tool}: proposed routes.yaml must not change egress logging",
)
else:
raise _RpcClientError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
# --- MCP handlers ----------------------------------------------------------
@@ -341,17 +375,17 @@ def handle_tools_call(
doesn't need operator approval."""
name = params.get("name")
if not isinstance(name, str):
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
raise _RpcError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
if name == _sv.TOOL_LIST_EGRESS_ROUTES:
return handle_list_egress_routes(typing.cast(dict[str, object], params.get("arguments", {})), config)
args_raw = params.get("arguments", {})
if not isinstance(args_raw, dict):
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
raise _RpcError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
justification = args_raw.get("justification")
if not isinstance(justification, str) or not justification.strip():
raise _RpcClientError(
raise _RpcError(
ERR_INVALID_PARAMS,
f"{name}: 'justification' is required and must be a non-empty string",
)
@@ -360,13 +394,13 @@ def handle_tools_call(
file_field = PROPOSED_FILE_FIELD[name]
proposed_file = args_raw.get(file_field)
if not isinstance(proposed_file, str):
raise _RpcClientError(
raise _RpcError(
ERR_INVALID_PARAMS,
f"{name}: '{file_field}' is required and must be a string",
)
validate_proposed_file(name, proposed_file)
else:
raise _RpcClientError(ERR_INVALID_PARAMS, f"unknown tool {name!r}")
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {name!r}")
proposal = _sv.Proposal.new(
bottle_slug=config.bottle_slug,
@@ -375,10 +409,7 @@ def handle_tools_call(
justification=justification,
current_file_hash=_sv.sha256_hex(proposed_file),
)
try:
_sv.write_proposal(config.queue_dir, proposal)
except OSError as e:
raise _RpcInternalError(f"failed to write proposal to queue: {e}") from e
sys.stderr.write(
f"supervise: queued proposal {proposal.id} ({name}) "
f"for bottle {config.bottle_slug}; waiting for operator...\n"
@@ -398,10 +429,7 @@ def handle_tools_call(
"content": [{"type": "text", "text": text}],
"isError": False,
}
try:
_sv.archive_proposal(config.queue_dir, proposal.id)
except OSError as e:
raise _RpcInternalError(f"failed to archive proposal: {e}") from e
text = format_response_text(response)
return {
@@ -435,8 +463,9 @@ def format_pending_response_text(timeout_seconds: float) -> str:
# --- HTTP transport --------------------------------------------------------
# Max request body the server accepts. 1 MB is well above any realistic
# routes.yaml proposal.
# Max request body the server accepts. Generous because Dockerfile
# proposals can be a few KB; routes.json is small. 1 MB is well above
# any realistic config file.
MAX_BODY_BYTES = 1 * 1024 * 1024
@@ -476,7 +505,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
req = parse_jsonrpc(body)
except _RpcClientError as e:
except _RpcError as e:
self._write_jsonrpc(jsonrpc_error(None, e.code, e.message))
return
@@ -484,19 +513,11 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
result = self._dispatch(req, config)
except _RpcClientError as e:
except _RpcError as e:
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
return
except _RpcInternalError as e:
cause = e.__cause__
detail = f": {cause}" if cause else ""
sys.stderr.write(f"supervise: internal error: {e.message}{detail}\n")
sys.stderr.flush()
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
return
except Exception as e: # noqa: W0718 — unexpected errors
sys.stderr.write(f"supervise: unexpected error: {type(e).__name__}: {e}\n")
sys.stderr.flush()
except Exception as e: # noqa: W0718 — catch-all for RPC dispatch errors
sys.stderr.write(f"supervise: internal error: {e}\n")
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
return
@@ -515,7 +536,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
return handle_tools_list(req.params)
if method == "tools/call":
return handle_tools_call(req.params, config)
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
raise _RpcError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def _write_jsonrpc(self, body: bytes) -> None:
self.send_response(200)
-96
View File
@@ -1,96 +0,0 @@
# ADR 0004: Risk-weighted coverage, not a single global target
- **Status:** Accepted
- **Date:** 2026-06-25
- **Deciders:** didericis
## Context
bot-bottle is a security tool: it sandboxes agents, scans egress for
secret exfiltration, strips credentials, and gates git pushes. A latent
bug in that logic is expensive, so test coverage there genuinely
matters. But the repo also contains code where coverage is a poor
signal:
- **Interactive entry-point shells**`cli/init.py` (a `read_tty_line()`
prompt loop) and `cli/tui.py` (a curses picker). Their bodies are I/O;
a unit test has to fake the entire terminal conversation, so it
inflates the number without asserting behaviour that would otherwise
go unchecked.
- **Subprocess / backend orchestration** — the docker / smolmachines /
macos-container backends shell out to `docker`, `container`, `smolvm`.
Mock-heavy unit tests here mostly re-assert the argv you already
wrote (the test passes whether or not the real teardown works), while
many of the missed *branches* are failure paths you cannot provoke
against a real daemon on cue.
Chasing a single global percentage (e.g. 90%) pushes the most test
effort onto the least safety-relevant code — exactly backwards — and
invites performative tests written to colour a line rather than to catch
a regression (Goodhart's law).
## Decision
Coverage is **risk-weighted**, measured over the **combined unit +
integration** suites, with three rules:
1. **Critical modules target ≥ 90%.** The security/logic core —
`egress_addon{,_core}.py`, `dlp_detectors.py`, `egress.py`,
`manifest*.py`, `git_gate.py`, `git_http_backend.py`, `supervise.py`,
`yaml_subset.py`, `bottle_state.py` — is Docker-independent and
unit-testable, so it carries the high bar. We ratchet toward 90% as
these modules are touched; new gaps in them are not acceptable.
2. **Subprocess/backend orchestration is covered by the integration
suite, not omitted.** `scripts/coverage.sh` runs unit + integration
under one coverage measurement so these modules are scored where they
are actually exercised. They stay *visible* — hiding the code that
tears down sandboxes and wires networks is the one place we will not
omit.
3. **Interactive entry-point shells are omitted** (`.coveragerc`), with a
rationale comment. This is the only sanctioned use of `omit` besides
`tests/*`.
The forward-looking guard is a **diff-coverage gate**
(`scripts/diff_coverage.py`): new/changed executable lines on a branch
must be ≥ 90% covered. This catches regressions where they are
introduced without forcing a back-fill crusade through legacy glue. The
gate skips lines in omitted files (there is no coverage data for them),
so the omit list cannot launder *new* logic into the dark: anything that
needs real testing must live outside the interactive shells to be
scored at all.
The **global percentage is informational**, not a CI gate — it would
otherwise be hostage to the CI runner's Docker availability and to the
omit list.
## Consequences
- The number we report (`scripts/coverage.sh`) means "coverage of the
code we consider testable, across both suites" — a dip is a real
regression in code we control, not noise from added CLI glue.
- No incentive to write mock-the-mock tests for orchestration to defend
a global figure.
- The omit list needs governance: an entry must be a genuinely
interactive shell, justified in the `.coveragerc` comment and here.
`cli/init.py` and `cli/tui.py` qualify; backend orchestration does
not.
- CI must run the integration suite under coverage to score the
orchestration modules; where the runner lacks Docker those tests skip
and their modules read low — accepted, because the *enforced* gates
(critical-module standard + diff coverage) are Docker-independent.
- "We're at N%" is now a curated figure; outsiders should read the
policy, not just the badge.
## Links
- PRs #290 (cover the egress adapter), and the coverage-policy PR that
introduces this record.
- `.coveragerc`, `scripts/coverage.sh`, `scripts/diff_coverage.py`.
- `scripts/critical-modules.txt` — the single source of truth for the
core-module list; read by both `scripts/coverage.sh` and the
`update-badges.yml` "core coverage" badge so they cannot drift.
- The README carries a `core coverage` badge (auto-updated from that
list) — the headline number, distinct from the informational global
`coverage` badge.
@@ -1,210 +0,0 @@
# PRD 0062: Supervisor override for egress token blocks
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-24
- **Issue:** #261
## Summary
Give each egress route a policy for what happens when an outbound DLP detector
matches a token, via `dlp.outbound_on_match: block | redact | supervise`
(default `supervise`):
- **`supervise`** (default) — route the block through the existing supervisor
approval queue instead of returning `403` immediately. The proxy holds the
request open until the operator approves or rejects it. On approval the
matched token is added to an in-memory "safe tokens" set so the request — and
any later request carrying the same token — flows through without
re-prompting.
- **`redact`** — scrub the matched value(s) from the request and forward it,
no operator in the loop. For routes where a token-shaped value is noise the
upstream doesn't need (telemetry/log sinks). Fails closed if a match lands on
a surface redaction can't rewrite (the hostname).
- **`block`** — the original hard `403`; never overridable. For routes where a
detected token must always stop.
The motivating goal is reducing friction from false positives without weakening
the default-deny posture: supervise keeps a human in the loop, redact is an
explicit per-route opt-in, and block stays available for sensitive routes.
## Problem
The outbound DLP detectors (`token_patterns`, `known_secrets`) are
deliberately aggressive: any string that looks like a credential is blocked
before it leaves the bottle. That is the right default, but it produces false
positives — a token-shaped value that is not actually a secret, or a credential
the agent legitimately needs to send to a declared host. Today the only
recovery is for the operator to notice the `egress DLP` 403 in the logs and
hand-edit the route's `dlp.outbound_detectors`, which disables the detector for
the whole route rather than allowing the one value.
The operator has no in-the-loop signal that a token block happened and no
fine-grained way to say "this specific value is fine."
## Goals / Success Criteria
1. An outbound DLP **token** block (a `ScanResult` carrying a matched secret
value) creates a supervisor proposal instead of an immediate `403`.
2. The egress proxy holds the blocked request open, polling for the operator's
response up to a bounded timeout.
3. The proposal shows the operator the host, method, path, the detector reason,
and a **redacted** context snippet — never the raw token value.
4. On `approved`/`modified`, the matched token value is added to an in-memory
safe-tokens set and the request proceeds normally; later requests carrying
the same value skip the block.
5. On `rejected`, timeout, malformed response, or missing supervisor wiring,
the request fails closed with the same `403` as today.
6. Structural blocks that carry no token value (CRLF injection) and the
route-not-allowlisted / git blocks are unchanged — they stay hard `403`s and
keep their existing agent-driven `allow` / `egress-block` MCP path.
7. The proxy event loop is not stalled while waiting: the wait is asynchronous,
so other flows keep being served.
## Non-goals
- Persisting the safe-tokens set across egress restarts. It lives in process
memory only; a restart re-prompts. (The issue explicitly defers persistence.)
- Supervising inbound (prompt-injection) blocks or WebSocket frame blocks.
WebSocket frames still honour the safe-tokens set for already-approved values
but cannot wait for approval (there is no response surface after upgrade).
- Generalising an approved secret across encodings. The safe-tokens set matches
the exact value the detector found.
- Replacing the per-route `dlp.outbound_detectors` override. That remains the
way to turn a detector off wholesale.
- Making `redact` the default. Silent redaction of a true false positive
corrupts legitimate data, so it is opt-in per route; `supervise` (human in
the loop) stays the default.
## Scope
### In scope
The minimum cut that ships, in build order:
1. **Core**`ScanResult.matched`; thread `safe_tokens` through
`scan_outbound` / the token detectors; `build_token_allow_payload`.
2. **Supervise + TUI**`TOOL_EGRESS_TOKEN_ALLOW`; TUI suffix, modify guard,
required approval reason.
3. **Addon glue** — async `request`, safe-tokens set, proposal write + async
poll, allow/block decision; pass `safe_tokens` into the WebSocket path.
4. **On-match policy**`dlp.outbound_on_match` through manifest → render →
addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in
the addon's outbound handler.
5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README
egress + supervisor notes.
### Out of scope
The deferrals enumerated under **Non-goals** — restart persistence, inbound /
WebSocket-frame supervision, cross-encoding generalisation, replacing
`dlp.outbound_detectors`, and making `redact` the default.
## Proposed Design
### New services / components
A new proposal tool constant `egress-token-allow` (`TOOL_EGRESS_TOKEN_ALLOW`)
is added to `supervise.TOOLS`, and the egress addon gains an in-memory
safe-tokens set plus the policy-dispatch path that drives it.
On an outbound block the addon dispatches on the resolved policy:
- **Structural blocks always 403.** A `ScanResult` with no `matched` value
(CRLF injection) is a hard `403` regardless of policy — there is nothing to
redact or safelist.
- **`redact`** runs `redact_tokens` over the body, non-`host` header values,
and path/query, then re-scans. If the re-scan is clean the (rewritten)
request is forwarded; if a block-severity match remains (e.g. in the
hostname, or a unicode-evasion token redaction can't reach) it fails closed
with a `403`.
- **`block`** writes the `403` immediately.
- **`supervise`** runs the queue-and-wait loop, falling back to `block` when
supervise isn't wired for the bottle.
For `supervise`, the addon writes the proposal directly to
`SUPERVISE_QUEUE_DIR` (the queue is bind-mounted into the sidecar bundle and
shared by every daemon, exactly as git-gate's `gitleaks-allow` proposal in PRD
0061 does). The proposal's `proposed_file` is a human-readable text payload
built by `build_token_allow_payload`:
```
egress blocked an outbound request carrying a detected token
host: api.example.com
method: POST
path: /v1/ingest
detector: OpenAI API key found in body
context: ...before ******** after...
```
The justification tells the operator to approve only if the value is a false
positive or a credential the request legitimately needs. The addon then polls
`<proposal-id>.response.json` for `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default
300). `approved`/`modified` allow the request and add the value to the
safe-tokens set; `rejected`, malformed responses, and timeout fail the request
closed. The proposal + response are archived to `processed/` after a decision.
Because the wait happens inside mitmproxy's asyncio loop, the addon's `request`
hook is async and polls with `asyncio.sleep`, so concurrent flows are
unaffected.
### Existing code touched
- **Policy threading.** `dlp.outbound_on_match` is a per-route enum threaded
from the bottle manifest (`manifest_egress`) through the resolved route
(`egress.EgressRoute`), the rendered `routes.yaml` (`egress_render_routes`),
and the addon's `Route` (`egress_addon_core`). Unset renders nothing and
resolves to `supervise` at request time. The `list-egress-routes`
introspection endpoint round-trips it so the agent's proposals preserve it.
- **Provider-route default.** Agent-provider routes (the agent talking to its
own LLM API — `api.anthropic.com`, the Codex backend, etc.) are the worst
source of token-shaped false positives because the whole conversation payload
flows through them. `egress_routes_for_bottle` fills `outbound_on_match=redact`
on any provider route that doesn't set it explicitly; a provider that sets the
policy keeps its choice, and manifest routes are unaffected (they default to
`supervise`).
- **Scanners.** `scan_outbound` (and the token detectors `scan_token_patterns`
/ `scan_known_secrets` it calls) accept a `safe_tokens` set. A match whose
value is in `safe_tokens` is skipped, so an approved token no longer blocks;
the scanners keep searching past a safelisted match so a second, un-approved
secret in the same request is still caught. The WebSocket path is passed the
same `safe_tokens` set.
- **Supervisor UI.** `cli/supervise.py` renders `egress-token-allow` like
`gitleaks-allow`: the text payload is shown, modify is unavailable (there is
no file patch to edit), and approval prompts for a non-empty reason recorded
in the response notes. There is no on-disk config diff, so — like
`gitleaks-allow` and `capability-block` — it writes no egress audit-log entry.
- **Failure handling.** If `SUPERVISE_QUEUE_DIR` / `SUPERVISE_BOTTLE_SLUG` are
unset (supervise disabled for the bottle), the addon skips the queue and
returns the existing `403`. Any error writing the proposal or reading the
response also fails closed.
### Data model changes
- New per-route manifest field `dlp.outbound_on_match: block | redact |
supervise`, rendered into `routes.yaml` (omitted when unset).
- `ScanResult` gains a `matched: str = ""` field carrying the raw substring the
detector matched. The token detectors populate it; the structural CRLF
detector leaves it empty. The value stays inside the egress sidecar process —
never written to a log line (logs use the redacted `context`) nor to the
proposal file.
- Proposal text payload (above) plus `<proposal-id>.response.json` in
`SUPERVISE_QUEUE_DIR`, archived to `processed/` after a decision.
- New env var `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default 300).
### External dependencies
None. Reuses the existing supervisor queue (`SUPERVISE_QUEUE_DIR`) and the
mitmproxy addon framework already in the egress sidecar.
## Open questions
- Should `known_secrets` (provisioned `EGRESS_TOKEN_*` exfiltration) be
override-able at all, or only `token_patterns`? This PRD allows both —
approval is an explicit operator decision and the safe-tokens set matches the
exact found value — but a future revision could restrict `known_secrets` to
reject-only.
## References
- Issue #261
- PRD 0061 — `gitleaks-allow` supervisor proposal pattern this reuses.
@@ -1,189 +0,0 @@
# PRD 0063: Strengthen outbound exfiltration detection
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-25
- **Issue:** #259
## Summary
The current DLP system has two detection tiers: weak (vendor token regexes)
and strong (exact matching of provisioned `EGRESS_TOKEN_*` values in multiple
encodings). This PRD adds four complementary hardening layers in ranked order
of impact:
1. **Canary tokens** — inject a per-session fake secret into the agent's
environment; any outbound appearance is a zero-false-positive exfil signal.
2. **Broadened known-value scanning** — extend the strong tier beyond
`EGRESS_TOKEN_*` via a `BOT_BOTTLE_SENSITIVE_PREFIXES` sidecar env var.
3. **Fragmentation-resistant matching** — catch separator injection and
long-substring leaks via alnum projection and sliding-window substring
search.
4. **Entropy scoring** — warn-only flag on high-entropy blobs as a
complementary signal for encrypted exfil.
## Problem
After PRD 0056 the outbound scan covers all HTTP request surfaces (host, path,
query, headers, body). But the content-based strong tier only matches
`EGRESS_TOKEN_*` values and their encodings. Three gaps remain:
| Gap | How an agent exploits it |
|-----|--------------------------|
| No canary | There is no "can't be false-positive" signal |
| Narrow prefix filter | Sensitive values provisioned under other env prefixes (MCP keys, API keys injected via the agent provider) are invisible to `scan_known_secrets` |
| Fragmentation blindness | Inserting `-`, space, or `\n` between secret characters turns an exact-match into a miss |
## Goals / Success Criteria
1. Each launched bottle has a unique canary token in the agent's environment
under a randomized `WORD_WORD_SECRET` env var name. The egress sidecar gets
the same env var and registers that exact name through
`BOT_BOTTLE_SENSITIVE_PREFIXES`. Any outbound appearance of the canary
blocks the request as a known-secret match.
2. `scan_known_secrets` accepts a `sensitive_prefixes` parameter (default:
`("EGRESS_TOKEN_",)`). `scan_outbound` reads
`BOT_BOTTLE_SENSITIVE_PREFIXES` from `environ` and merges those prefixes
in, so operators can mark additional env vars as scanned values without
changing the manifest schema.
3. For every secret that passes exact-match, a secondary alnum-projection pass
checks for the secret with all non-alphanumeric characters stripped. This
catches separator-injection evasion (`MY-SECRET` → body contains
`MY SECRET`).
4. A sliding-window partial-match pass checks for long-enough contiguous
substrings of the secret's alnum projection in the text's alnum projection.
Any match ≥ `PARTIAL_MATCH_MIN_LEN` (12 chars) blocks with reason
`"partial match"`.
5. A new `scan_entropy` detector flags outbound text windows with Shannon
entropy ≥ `ENTROPY_BLOCK_THRESHOLD` (5.5 bits/char) at **warn** severity
only. It is registered under the new detector name `"entropy"` in
`OUTBOUND_DETECTOR_NAMES` and disabled by default (routes must opt in).
6. Binary request bodies are decoded via `latin-1` instead of
`utf-8 errors="replace"`, preserving every byte value and allowing
ASCII-range secrets to be found within binary payloads.
7. All new behaviour is unit-tested; existing tests pass unchanged.
## Non-goals
- Rolling per-host buffer for split-across-requests detection (state in the
stateless addon is complex; deferred).
- Additional vendor regexes.
- ML / embedding-based detection.
- Entropy-based hard blocks (warn only per the issue).
## Design
### Canary token flow
```
Egress.prepare()
canary = secrets.token_urlsafe(32)
canary_env = <random WORD_WORD_SECRET>
EgressPlan(canary=canary, canary_env=canary_env, ...)
Docker compose render:
sidecar env: <canary_env>=<canary>
sidecar env: BOT_BOTTLE_SENSITIVE_PREFIXES=<canary_env>
agent env: <canary_env>=<canary> ← visible to agent as a "secret"
macos-container launch: same literals added to sidecar + agent env entries
```
The sidecar uses `BOT_BOTTLE_SENSITIVE_PREFIXES` to make the random canary env
name part of the existing `scan_known_secrets` detector without adding a
manifest schema field.
### Broadened known-value scanning
`scan_known_secrets` gains a `sensitive_prefixes` parameter:
```python
def scan_known_secrets(
text: str,
*,
location: str = "body",
env: Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> ScanResult | None:
```
`scan_outbound` reads `BOT_BOTTLE_SENSITIVE_PREFIXES` (comma-separated list
of additional prefixes) from `environ` and appends them:
```python
extra = tuple(
p for p in environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "").split(",") if p
)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
```
`redact_tokens` receives the same treatment for consistent redaction.
### Fragmentation-resistant matching
A new helper `_alnum_projection(text)` strips all non-alphanumeric characters.
`scan_known_secrets` runs two passes per secret:
1. **Exact pass** — existing encoded-variant loop (unchanged).
2. **Alnum-projection pass** — if the secret's alnum projection has ≥ 8 chars,
check if it appears in the text's alnum projection. Match → block with
`"fragmented match (separator injection)"` reason.
3. **Partial-substring pass** — if the secret's alnum projection has ≥
`PARTIAL_MATCH_MIN_LEN` chars (12), slide a window of that length across the
secret's projection and look for each window in the text's alnum projection.
First match → block with `"partial match"` reason.
All three passes run only for the `"known_secrets"` detector; the token-pattern
and entropy detectors are unchanged.
### Entropy scoring
New public function:
```python
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW, # 64
threshold: float = ENTROPY_BLOCK_THRESHOLD, # 5.5
) -> ScanResult | None:
```
Slides a window of `window` characters across `text` in steps of `window // 2`.
If any window's Shannon entropy exceeds `threshold`, returns a **warn**-severity
`ScanResult`. Never blocks.
`OUTBOUND_DETECTOR_NAMES` gains `"entropy"`. Routes opt in via their `dlp`
block; entropy scanning is **off by default** to avoid false-positive noise on
legitimate binary payloads.
### Binary body handling
In `scan_outbound`, the bytes → str decoding changes from:
```python
body.decode("utf-8", errors="replace")
```
to:
```python
body.decode("utf-8") if body is str else body.decode("latin-1")
```
`latin-1` is a bijective byte↔codepoint mapping; every byte value is preserved
as its corresponding Latin-1 code point, so ASCII-range secret strings remain
intact and `str.find` / regex still locate them correctly. The fallback from
strict UTF-8 is tried first so valid UTF-8 bodies are decoded faithfully.
## Implementation
Delivered in three commits on the same branch:
1. **DLP detector changes**`_alnum_projection`, fragmentation passes,
`scan_entropy`, broadened `scan_known_secrets`, updated `scan_outbound` and
`redact_tokens`; all accompanying unit tests.
2. **Canary injection**`EgressPlan.canary`, `Egress.prepare()`,
Docker compose + macos-container backend injection.
3. **PRD flip**`Status: Draft → Active`.
@@ -1,85 +0,0 @@
# PRD 0064: LOG_FULL egress logging credential redaction
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-25
- **Issue:** #257
## Summary
The `LOG_FULL` egress logging path (`_log_request` and `_log_response` in `egress_addon.py`) writes request/response headers and bodies to stderr without redaction and includes the sidecar-injected upstream `Authorization` header verbatim. This PR applies `redact_tokens` to header values and bodies in both log functions and strips the injected `Authorization` header from request logs entirely.
## Problem
`LOG_FULL` (log level 2) is intended for debugging egress traffic. When active it calls `_log_request` and `_log_response`. Both functions have two related bugs:
1. **Injected `Authorization` header exposure.** `_log_request` is called *after* the sidecar injects upstream credentials (`flow.request.headers["authorization"] = decision.inject_authorization`). The full header dict — including the live credential — is serialized to stderr. Any log collector that ingests the egress container's stderr will receive the upstream bearer token in plaintext.
2. **Unredacted bodies and header values.** Neither `_log_request` nor `_log_response` passes body or header values through `redact_tokens`. By contrast, `_req_ctx` (used for block/warn events) already calls `redact_tokens` on path and host. Any provisioned secret or recognized token pattern that appears in a request body, response body, or non-Authorization header value will be logged verbatim under `LOG_FULL`.
These two bugs compose: an agent that enables `LOG_FULL` and simultaneously triggers a request that carries a known token gains a write path from credentials → egress logs.
## Goals / Success Criteria
- `_log_request` never logs the `authorization` header in any form.
- `_log_request` applies `redact_tokens(value, env=os.environ)` to every other header value before serializing.
- `_log_request` applies `redact_tokens(body, env=os.environ)` to the request body before logging.
- `_log_response` applies `redact_tokens(value, env=os.environ)` to every response header value before logging.
- `_log_response` applies `redact_tokens(body, env=os.environ)` to the response body before logging.
- Unit tests cover each of the five cases above.
## Non-goals
- Redacting host or path in the full-log path (already covered by `_req_ctx` for block/warn events; `_log_request` already calls `redact_tokens` on host and path).
- Suppressing `LOG_FULL` or adding a new log level.
- Changing the outbound DLP scan logic.
## Design
### `_log_request`
```python
def _log_request(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.request.headers.items()
if k.lower() != "authorization"
}
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": headers,
"body": body,
})
+ "\n"
)
```
The `authorization` key is excluded because by the time `_log_request` is called the sidecar has already injected the upstream credential (`decision.inject_authorization`). Logging it would write a live bearer token to stderr on every allowed request. There is no safe subset to log — the value is always a live credential or empty.
### `_log_response`
```python
def _log_response(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.response.headers.items()
}
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": headers,
"body": body,
})
+ "\n"
)
```
Response headers don't carry injected credentials, so no header name is suppressed — only the values are scrubbed by `redact_tokens`.
-166
View File
@@ -1,166 +0,0 @@
# PRD 0065: Multi-parent `extends:` for bottles
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-25
- **Issue:** #268
- **Extends:** PRD 0025 (`0025-bottle-extends.md`)
## Summary
Allow a bottle's `extends:` field to accept either a single bottle name (existing
behavior) or a list of bottle names (new). Multiple parents are resolved
independently and folded left-to-right into a single effective parent before the
child is merged on top. This lets orthogonal concerns (base env, networking/egress,
agent provider) live in separate bottles and be composed without forcing them into a
linear chain.
## Problem
PRD 0025 shipped single-parent `extends:` and listed "No multi-parent inheritance"
as a non-goal. In practice, users want to compose multiple orthogonal bottles — a
base environment, a networking profile, and an agent-provider override — without
creating a three-level linear chain that couples unrelated parents to each other.
The linear chain workaround has two problems:
1. **Ordering constraint.** `networking extends base` works, but then
`agent extends networking` can't also pick up `base` without going through
`networking`, coupling two unrelated concerns.
2. **Quadratic duplication.** N orthogonal bottles require O(N²) chain variants
(one chain per permutation of applied concerns).
Multi-parent `extends:` removes both constraints: each orthogonal concern stays in
its own bottle, and the child bottle is the only place that names the combination.
## Goals / Success Criteria
- `extends:` accepts a list of strings in addition to a plain string.
- Backward compat: existing single-string `extends:` is unchanged.
- Parents are resolved left-to-right; later entries win on conflict.
- Child wins over all parents (unchanged from PRD 0025).
- Cycle detection covers multi-parent graphs, not just linear chains.
- Diamond inheritance: a shared ancestor is resolved once (via the existing cache).
- Invalid list entries (non-string, undefined bottle, self-reference) die at parse
with clear messages.
- `manifest_loader.py`'s `load_bottle_chain_from_dir` enqueues all parents from a
list `extends:` so the resolver sees every bottle in the graph.
## Non-goals
- No change to the agent-vs-bottle trust boundary (PRD 0025 "Alternatives
considered" option 2 stays rejected).
- No MRO / C3 linearization. Left-to-right fold is sufficient for the expected use
cases.
- No preflight display of per-field provenance across multiple parents (same open
question as PRD 0025; remains a follow-up).
## Design
### Schema
`extends:` now accepts either form:
```yaml
# single parent (unchanged)
extends: base
# multiple parents (new)
extends: [base, networking]
```
Both forms are normalized to a list internally. A list with one element behaves
identically to the string form.
### Merge rules for multi-parent fold
Parents are folded pairwise left-to-right before the child merge. For each step in
the fold, the "earlier" bottle is the running accumulator and the "later" bottle is
the next parent. Rules per field:
| Field | Fold rule |
|--------------------|--------------------------------------------------------------|
| `env` | dict merge; later wins on key collision |
| `git-gate.user` | per-field overlay; later's non-empty fields win |
| `git-gate.repos` | union by name; for same-name entries, later wins per-field |
| `egress.routes` | concatenate (earlier first, later appended) |
| `egress.log` | later wins (last-wins) |
| `agent_provider` | later wins (last-wins) |
| `supervise` | later wins (last-wins) |
After the fold, the combined parent is merged against the child using the existing
PRD 0025 rules (child always wins). The child's `egress.routes` appends to the
combined parent's concatenated routes; `validate_egress_routes` runs once on the
final merged set and catches duplicate hosts.
### Algorithm
```
extends: [p1, p2, p3]
fold:
combined = resolve(p1)
combined = fold_two(combined, resolve(p2))
combined = fold_two(combined, resolve(p3))
merge:
result = _merge_bottles(combined, child_raw, name)
```
`fold_two(earlier, later)` applies the rules in the table above. Cycle detection
(the `seen` tuple) is passed to each parent resolution call unchanged — if any
parent's chain circles back to the current bottle, it is caught. The `cache` dict
ensures a shared ancestor is only resolved once across all parents.
### Error cases
| Condition | Error message shape |
|----------------------------------------|------------------------------------------------------------------|
| `extends` is not a string or list | `extends must be a string or list of strings (was <type>)` |
| A list entry is not a string | `extends[<i>] must be a string (was <type>)` |
| A list entry names an undefined bottle | `extends '<name>' which is not defined. Available bottles: ...` |
| A list entry is the bottle itself | `extends itself; remove the self-reference` |
| Cycle through any parent edge | `is in an extends cycle: <chain>` |
## Implementation
### `bot_bottle/manifest_extends.py`
- `_resolve_one_bottle`: accept `str | list[str]` for `extends`; normalize to list;
validate each entry; for a single-entry list fall through to the existing
single-parent path; for multiple entries call `_fold_parents` then
`_merge_bottles`.
- `_fold_parents(parent_names, raws, cache, repos_cache, seen)`: resolve each
parent and fold pairwise left-to-right; return `(effective_bottle,
effective_repos_raw)`.
- `_fold_two_bottles(earlier, earlier_repos_raw, later, later_repos_raw)`: apply
the fold rules above; return `(folded_bottle, folded_repos_raw)`.
### `bot_bottle/manifest_loader.py`
- `load_bottle_chain_from_dir`: when `extends` is a list, enqueue all parent names
for loading (previously only `isinstance(parent, str)` was handled).
### `tests/unit/test_manifest_extends.py`
- `TestExtendsErrors.test_non_string_extends_dies`: update to use an integer
`extends` value (a list is now valid).
- New class `TestExtendsMultiParent` covering all cases listed in the issue.
## Testing strategy
Unit tests via `ManifestIndex.from_json_obj` (same resolver surface used by all
paths). No integration test changes needed — downstream code consumes the already-
merged bottle and is unchanged.
Test cases:
- Two-parent list: env union, egress routes concat, git repos union
- Last-parent-wins on scalar (supervise, agent_provider)
- Child wins over all parents on conflict
- Diamond: two parents share an ancestor; ancestor resolved once
- Single-element list: identical to string form
- Non-string extends value → ManifestError
- Non-string list entry → ManifestError
- Undefined bottle in list → ManifestError
- Self-reference in list → ManifestError
- Cycle through multi-parent edge → ManifestError
@@ -1,216 +0,0 @@
# PRD 0066: Separate agent and bottle selection
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-25
- **Issue:** #269
## Summary
Agents and bottles are two separate concerns: agents carry a system prompt and
skills; bottles carry infrastructure configuration (egress, git-gate, env,
agent provider). Today an agent's manifest file hard-codes a single `bottle:`
reference, which prevents the same agent prompt from being reused across
projects that need different bottle configurations. This PRD decouples them: at
launch time, after choosing the agent, the operator picks an ordered list of
bottles via a multi-select picker. The selected bottles are merged in order
(later entries override earlier ones) to produce the effective bottle for the
session.
## Problem
The current `bottle: <name>` field on an agent manifest file binds the agent
permanently to one bottle. To use the same system prompt with a different bottle
(e.g. `claude-implementer` at home vs. at a client site that needs a different
egress policy), the operator must duplicate the agent file and change the
`bottle:` field. Duplicate agent files drift out of sync.
## Goals / Success Criteria
1. `bottle:` in an agent's frontmatter becomes optional. Existing manifests with
`bottle:` continue to work unchanged (backward compat).
2. After selecting an agent (via the existing single-select picker), a new
multi-select bottle picker appears showing all available bottles.
3. The multi-select picker pre-populates with the agent's `bottle:` value when
present.
4. Confirming with one or more bottles selected uses those bottles, merged in
selection order, as the effective bottle for the session.
5. Confirming with an empty selection falls back to the agent's `bottle:` field.
If neither is set, a ManifestError is raised pointing the operator at the fix.
6. The ordered bottle list is stored in launch metadata so `./cli.py resume`
uses the same bottles.
7. The preflight summary (`y/N` screen) shows the effective bottle name(s).
8. The multi-select picker supports incremental filtering, Space/Enter to toggle
selection, an ordered "Selected: ..." summary line, Ctrl-D to confirm, and
Esc/q to cancel the whole start operation.
9. Unit tests cover: multi-select widget (filter, toggle, confirm, cancel),
the `cmd_start` bottle-picker step, and the manifest `load_for_agent`
runtime-bottle-merge path.
## Non-goals
- Reordering the selection list from within the picker (order = insertion order;
drag-and-drop is out of scope).
- Storing bottle selection history / MRU.
- Changes to `./cli.py edit`, `./cli.py list`, or `./cli.py info`.
- Removing the `bottle:` key from the agent schema (it stays, now optional).
## Design
### `bot_bottle/cli/tui.py``filter_multiselect`
```python
def filter_multiselect(
items: list[str],
*,
title: str = "",
initial: list[str] | None = None,
tty_path: str = "/dev/tty",
) -> list[str] | None:
"""Multi-select variant of filter_select.
Returns the ordered list of selected items, or None on cancel.
Press Space/Enter to toggle the item under the cursor.
Press Ctrl-D to confirm. Press Esc/q to cancel.
"""
```
Layout:
```
Select bottles
Filter: _
─────────────────────────────────────────
> [*] claude
[ ] dev
[ ] codex
─────────────────────────────────────────
Selected (in order): claude
─────────────────────────────────────────
[↑↓/jk] move [Space] toggle [Ctrl-D] done [Esc] cancel
```
`initial` pre-populates the ordered selection. `None` means no pre-selection.
Items added are appended in insertion order; items removed leave the remaining
order unchanged.
### `bot_bottle/manifest_schema.py` — optional `bottle:`
`bottle` moves from `AGENT_KEYS_REQUIRED` to `AGENT_KEYS_OPTIONAL`.
### `bot_bottle/manifest_agent.py` — optional `bottle:`
`ManifestAgent.bottle` changes from `str` (required) to `str = ""`.
`from_dict` no longer requires the key to be present; the bottle-exists
validation is skipped when the key is absent.
### `bot_bottle/manifest_loader.py``scan_bottle_names`
```python
def scan_bottle_names(bottles_dir: Path) -> list[str]:
"""Scan <bottles_dir>/*.md and return sorted bottle names."""
```
### `bot_bottle/manifest.py``ManifestIndex` changes
**`all_bottle_names` property** — analogous to `all_agent_names`; scans
`home_md / "bottles"` in lazy mode, returns `sorted(self.bottles.keys())` in
eager mode.
**`load_for_agent(agent_name, bottle_names: tuple[str, ...] = ())`** — new
`bottle_names` parameter. When non-empty, the listed bottles are resolved and
merged in order (index 0 is the base; each subsequent bottle is applied on top
using the same field-merge rules as `extends:`). The result replaces the bottle
that `agent.bottle` would have provided. When empty, falls back to `agent.bottle`.
Raises ManifestError if neither `bottle_names` nor `agent.bottle` is set.
### `bot_bottle/manifest_extends.py``merge_bottles_runtime`
```python
def merge_bottles_runtime(bottles: list[ManifestBottle]) -> ManifestBottle:
"""Merge an ordered list of pre-resolved ManifestBottle objects.
Index 0 is the base; each subsequent entry overrides the previous using
the same rules as the file-based extends machinery:
- env: dict merge, later wins
- git_user: per-field overlay, later wins on non-empty
- git (repos): union by name, later wins per-name
- egress.routes: concatenate
- agent_provider, supervise: later bottle's value replaces earlier
"""
```
This function operates on already-parsed `ManifestBottle` objects, so it does
not need to touch the raw-dict path.
### `bot_bottle/backend/__init__.py``BottleSpec` + `_validate`
`BottleSpec` gains `bottle_names: tuple[str, ...] = ()`.
`BottleBackend._validate` passes `spec.bottle_names` to `load_for_agent`:
```python
manifest = spec.manifest.load_for_agent(spec.agent_name, spec.bottle_names)
```
The preflight print updates `info(f"bottle: {agent.bottle}")` to display the
effective bottle name(s). When `spec.bottle_names` is non-empty those are
shown; when empty and `agent.bottle` is set, the agent's `bottle:` is shown.
### `bot_bottle/bottle_state.py` — persist bottle names
`BottleMetadata` gains `bottle_names: tuple[str, ...] = ()`. `read_metadata`
reads this from JSON (default `()`). `write_launch_metadata` passes
`spec.bottle_names` through.
### `bot_bottle/cli/start.py` — bottle multiselect step
After agent selection, before the name/color modal:
```python
available_bottle_names = manifest.all_bottle_names
# Peek at agent's bottle default for pre-population
initial_bottle = _peek_agent_bottle(manifest, agent_name)
initial = [initial_bottle] if initial_bottle else []
bottle_names_list = tui.filter_multiselect(
available_bottle_names,
title="Select bottles",
initial=initial,
)
if bottle_names_list is None:
return 0 # user cancelled
bottle_names = tuple(bottle_names_list)
```
`_peek_agent_bottle` reads the agent file's frontmatter without full parsing,
returning the `bottle:` value or `""` when absent.
`BottleSpec` is built with `bottle_names=bottle_names`.
### `bot_bottle/cli/resume.py` — bottle names from metadata
```python
spec = BottleSpec(
...
bottle_names=tuple(metadata.bottle_names),
)
```
## Implementation chunks
1. **Schema + model**`manifest_schema.py`, `manifest_agent.py` (optional
`bottle:`), `manifest_loader.py` (`scan_bottle_names`), `manifest.py`
(`all_bottle_names`, `load_for_agent` signature), `manifest_extends.py`
(`merge_bottles_runtime`), `bottle_state.py` (`bottle_names` field),
`resolve_common.py` (thread through).
2. **Backend**`BottleSpec.bottle_names`, `_validate`, preflight print.
3. **TUI**`filter_multiselect` in `tui.py` + unit tests.
4. **CLI wiring**`start.py` bottle picker step, `resume.py` metadata load.
5. **Tests**`test_cli_start_selector.py` bottle-picker cases,
`test_manifest_agent.py` optional-bottle cases, new
`test_manifest_bottle_merge.py` for `merge_bottles_runtime`.
## Open questions
None.
@@ -1,132 +0,0 @@
# PRD prd-new: Fold bot-bottle-orchestrator into this repo
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-07-01
- **Issue:** #321
## Summary
Move the `bot-bottle-orchestrator` binary into `bot_bottle/orchestrator/` as a
first-class subpackage. `pip install bot-bottle` gets you everything; the
orchestrator's entry point becomes `python -m bot_bottle.orchestrator run`. The
cross-repo CLI contract becomes an internal boundary, and the forge integration
layer (`GiteaClient`, `ScopedForge`, `SqliteForgeStateStore`) is promoted to
`bot_bottle/contrib/` where it belongs.
## Problem
The orchestrator and bot-bottle are tightly coupled:
- It always deploys on the same host.
- It imports from `bot_bottle` for the forge/state layer.
- Its runner shims (`start --headless`, `commit`, `resume`) map 1:1 to CLI
commands in `cli.py` — a breaking CLI change silently breaks the orchestrator
with no CI signal.
- Two repos means two version pins, two CI pipelines, and two install steps
every time the deploy environment is rebuilt.
## Goals / Success Criteria
- All orchestrator modules live under `bot_bottle/orchestrator/` and the package
is importable as `from bot_bottle.orchestrator import ...`.
- `python -m bot_bottle.orchestrator run` starts the webhook server.
- `python -m bot_bottle.orchestrator status` prints tracked runs.
- The forge integration layer (`GiteaClient`, `GiteaForge`, `ScopedForge`,
`ForgeState`, `SqliteForgeStateStore`) lives in `bot_bottle/contrib/` and is
covered by tests in `tests/unit/orchestrator/`.
- All orchestrator unit tests pass under bot-bottle's existing CI
(`python -m unittest discover -s tests/unit`).
- No functional change to the orchestrator's external behaviour: same
HTTP surface, same webhook protocol, same env-var config, same CLI flags.
## Non-goals
- Replacing `SubprocessBottleRunner` with a direct programmatic runner — the
subprocess shim stays; the `BottleRunner` protocol remains the internal
abstraction point.
- Merging the orchestrator's SQLite DB with any other bot-bottle state store.
- Archiving `bot-bottle-orchestrator` (that happens after this ships and the
deploy is updated; out of scope for this PR).
## Design
### Package layout
```
bot_bottle/
orchestrator/
__init__.py
__main__.py # python -m bot_bottle.orchestrator
bootstrap.py # wires contrib modules → orchestrator core
config.py
events.py
lifecycle.py
model.py
provenance.py
runner.py
sidecar.py
store.py
targeting.py
watchdog.py
webhook.py
contrib/
forge/
__init__.py
base.py # ScopedForge: read-anywhere / write-scoped wrapper
gitea/
client.py # GiteaClient (urllib.request), GiteaForge
forge_state.py # ForgeState dataclass + SqliteForgeStateStore
tests/unit/orchestrator/
__init__.py
_fakes.py
test_config.py
test_events.py
test_lifecycle.py
test_provenance.py
test_runner.py
test_sidecar.py
test_store.py
test_targeting.py
test_watchdog.py
test_webhook.py
```
### Module moves
Every `orchestrator/` source file moves verbatim into `bot_bottle/orchestrator/`.
Internal imports are already relative (`from .config import Config`) so no
changes are needed inside the orchestrator modules themselves.
`bootstrap.py` is the only file that changes meaningfully: the lazy `bot_bottle`
imports become direct relative imports (`from ..contrib.gitea.client import …`),
and the `_require_bot_bottle()` guard is removed since the package is always
present.
### New contrib modules
**`bot_bottle/contrib/forge/base.py``ScopedForge`**
Wraps any forge object and enforces read-anywhere / write-scoped access: reads
pass through unconditionally; `post_comment` and `update_description` raise
`PermissionError` for issue/PR numbers outside the assigned set.
**`bot_bottle/contrib/gitea/client.py``GiteaClient`, `GiteaForge`**
`GiteaClient` is a thin `urllib.request`-only HTTP wrapper (no new Python
dependencies). `GiteaForge` composes a client and exposes the forge protocol:
`is_org_member`, `read_issue`, `read_pr`, `read_comments`, `post_comment`,
`update_description`.
**`bot_bottle/contrib/gitea/forge_state.py``ForgeState`, `SqliteForgeStateStore`**
`ForgeState` is a dataclass mirroring `RunRecord` field-for-field. `SqliteForgeStateStore`
backs it with SQLite (stdlib `sqlite3`): a single `forge_state` table with one
row per (owner, repo, issue\_number).
### Test migration
All orchestrator test files move to `tests/unit/orchestrator/` with absolute
imports updated from `orchestrator.X` to `bot_bottle.orchestrator.X`. The unit
discovery command (`-s tests/unit`) picks them up automatically — no CI changes
required.
+75
View File
@@ -0,0 +1,75 @@
# PRD prd-new: Install script
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-06
- **Issue:** #197
## Summary
Add a proper Python package distribution and a thin `install.sh` bootstrapper so users can install bot-bottle with a single command without cloning the repo.
## Problem
There is currently no install path for new users. The only way to run bot-bottle is to clone the repo and invoke `cli.py` directly. This blocks any HN-style public demo: readers want `curl | sh` or `pipx install`, not a manual clone-and-configure flow.
## Goals / Success Criteria
- `curl -fsSL <url>/install.sh | sh` (or equivalent) leaves a working `bot-bottle` command on PATH.
- Python-native users can install with `pipx install bot-bottle` or `uv tool install bot-bottle`.
- `install.sh` validates prerequisites (Python ≥ 3.11, Docker) and exits with a clear message if they are missing. It does not silently install Docker.
- `install.sh` runs `bot-bottle doctor` (or equivalent diagnostic) after install to confirm the environment is ready.
- The package has no runtime pip dependencies (stdlib-only, matching the existing constraint).
## Non-goals
- Bundling a Python runtime or producing a standalone binary.
- Automatic Docker installation.
- Plugin architecture changes (out of scope; see issue #197 for future direction).
- Publishing to PyPI in this PR — the package structure is the deliverable; publishing is a separate step.
## Design
### Package structure
Add a minimal `pyproject.toml` at the repo root:
```toml
[project]
name = "bot-bottle"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = []
[project.scripts]
bot-bottle = "bot_bottle.cli:main"
```
The existing `bot_bottle/` package and `cli.py` entry point already contain the logic; this just wires up the standard entry point. `cli.py` may need a small refactor to expose a `main()` callable if it uses `if __name__ == "__main__"` only.
### `install.sh`
A thin bootstrapper that:
1. Checks `python3 --version` ≥ 3.11; exits with instructions if not met.
2. Checks `docker info` exits 0; exits with instructions if Docker is not running.
3. Installs via `pipx` if available, otherwise falls back to `pip install --user`.
4. Runs `bot-bottle doctor` to verify the install.
The script must be idempotent (safe to re-run) and must not require `sudo`.
### `bot-bottle doctor`
A new subcommand that checks and reports:
- Python version.
- Docker daemon reachability.
- Whether `~/.bot-bottle/` config directory exists.
Exits 0 if all checks pass, non-zero otherwise.
## Decisions
- `install.sh` is hosted from the repo's raw Gitea URL for now:
`https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh`.
- Should `version` in `pyproject.toml` be driven by a git tag at build time (e.g. via `hatch-vcs`) or kept as a static string? Static is simpler for now.
@@ -22,7 +22,7 @@ escapes**, and **whether credentials are short-lived and scoped**.
- Outbound: Docker containers have full internet access by default; no egress monitoring on most home networks
- Lateral movement: compromised container can reach the LAN — NAS, other machines, internal services
- Notable: CVE-2025-59536 (CVSS 8.7, Feb 2026) — a poisoned `.claude/settings.json` in a repo gives RCE when Claude Code opens it. `--dangerously-skip-permissions` removes the last gate.
- Supply chain: MCP servers, skills, and npm packages pulled during agent execution. A Jan 2026 large-scale empirical study of a 98,380-skill snapshot confirmed 157 malicious skills, ~71% of them credential harvesters. Exfiltration was overwhelmingly naive — plaintext HTTP to hardcoded endpoints; under 10% used any code obfuscation, and concealment was mostly at the documentation level, not the code level. ([Malicious Agent Skills in the Wild](https://arxiv.org/html/2602.06547v1), arXiv:2602.06547)
- Supply chain: MCP servers, skills, and npm packages pulled during agent execution. ~20% of ClawHub skills were found malicious in early 2026.
**What local topology protects:**
- No inbound attack surface — nothing listening on a public port
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/bin/sh
set -eu
PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}"
MIN_PYTHON="3.11"
say() {
printf 'bot-bottle install: %s\n' "$*" >&2
}
die() {
say "error: $*"
exit 1
}
command -v python3 >/dev/null 2>&1 || die "python3 is required (version ${MIN_PYTHON} or newer)"
python3 - <<'PY' || die "python3 3.11 or newer is required"
import sys
raise SystemExit(0 if sys.version_info >= (3, 11) else 1)
PY
command -v docker >/dev/null 2>&1 || die "Docker is required; install Docker and start the daemon, then re-run this script"
docker info >/dev/null 2>&1 || die "Docker is installed but the daemon is not reachable; start Docker and re-run this script"
mkdir -p \
"${HOME}/.bot-bottle/agents" \
"${HOME}/.bot-bottle/bottles" \
"${HOME}/.bot-bottle/contrib"
if command -v pipx >/dev/null 2>&1; then
say "installing with pipx"
pipx install --force "${PACKAGE_SPEC}"
else
say "pipx not found; installing with python3 -m pip --user"
python3 -m pip install --user --upgrade "${PACKAGE_SPEC}"
fi
if command -v bot-bottle >/dev/null 2>&1; then
BOT_BOTTLE_BIN="bot-bottle"
elif [ -x "${HOME}/.local/bin/bot-bottle" ]; then
BOT_BOTTLE_BIN="${HOME}/.local/bin/bot-bottle"
say "using ${BOT_BOTTLE_BIN}; add ${HOME}/.local/bin to PATH for future shells"
else
die "bot-bottle was installed but is not on PATH"
fi
say "running bot-bottle doctor"
"${BOT_BOTTLE_BIN}" doctor
+27
View File
@@ -0,0 +1,27 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "bot-bottle"
version = "0.1.0"
description = "Self-hosted sandbox for AI coding agents with egress controls"
readme = "README.md"
requires-python = ">=3.11"
license = { text = "Apache-2.0" }
dependencies = []
[project.scripts]
bot-bottle = "bot_bottle.cli:main"
[tool.setuptools.packages.find]
include = ["bot_bottle*"]
[tool.setuptools.package-data]
bot_bottle = [
"Dockerfile.sidecars",
"egress_entrypoint.sh",
"contrib/claude/Dockerfile",
"contrib/codex/Dockerfile",
"contrib/pi/Dockerfile",
]
-1
View File
@@ -4,4 +4,3 @@
pylint>=3.0.0
pyright>=1.1.300
coverage>=7.0.0
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# Combined unit + integration coverage (see docs/decisions/0004-coverage-policy.md).
#
# Runs the unit suite, then appends the integration suite (which skips
# cleanly when Docker / the backend CLIs are unavailable), and prints one
# combined report. The integration suite is what scores the subprocess /
# backend orchestration modules, so the number here is the policy's
# yardstick — not the unit-only badge.
#
# Usage:
# scripts/coverage.sh # combined report
# scripts/coverage.sh critical # also report just the critical modules
set -euo pipefail
cd "$(dirname "$0")/.."
PY="${PYTHON:-python3}"
# Critical security/logic core held to the high bar by ADR 0004. The list
# lives in one place (scripts/critical-modules.txt) so this report and the
# README "core coverage" badge can't drift; comma-join it for --include.
CRITICAL=$(grep -vE '^[[:space:]]*(#|$)' scripts/critical-modules.txt | paste -sd, -)
rm -f .coverage
echo "== unit ==" >&2
"$PY" -m coverage run -m unittest discover -t . -s tests/unit
echo "== integration (skips without Docker) ==" >&2
"$PY" -m coverage run --append -m unittest discover -t . -s tests/integration
echo "== combined report ==" >&2
"$PY" -m coverage report -m
if [ "${1:-}" = "critical" ]; then
echo "== critical modules (ADR 0004 target: 90%) ==" >&2
"$PY" -m coverage report --include="$CRITICAL"
fi
-25
View File
@@ -1,25 +0,0 @@
# Critical security/logic core held to the >=90% coverage bar by
# docs/decisions/0004-coverage-policy.md.
#
# SINGLE SOURCE OF TRUTH: scripts/coverage.sh (the `critical` report) and
# .gitea/workflows/update-badges.yml (the "core coverage" badge) both read
# this file. Add a module here when it becomes part of the core; a coverage
# number that silently stops measuring a module is worse than no badge.
#
# One module path per line, relative to the repo root. Blank lines and
# `#` comments are ignored.
bot_bottle/egress_addon.py
bot_bottle/egress_addon_core.py
bot_bottle/dlp_detectors.py
bot_bottle/egress.py
bot_bottle/manifest.py
bot_bottle/manifest_egress.py
bot_bottle/manifest_agent.py
bot_bottle/manifest_schema.py
bot_bottle/git_gate.py
bot_bottle/git_gate_render.py
bot_bottle/git_gate_provision.py
bot_bottle/git_http_backend.py
bot_bottle/supervise.py
bot_bottle/yaml_subset.py
bot_bottle/bottle_state.py
-126
View File
@@ -1,126 +0,0 @@
#!/usr/bin/env python3
"""Diff-coverage gate (see docs/decisions/0004-coverage-policy.md).
Fails if too few of the *added/changed* executable lines on this branch
are covered. Stdlib-only by design the project carries no runtime deps
and we are not adding `diff-cover` to satisfy a check.
Reads coverage data already produced by a `coverage run` (e.g. via
`scripts/coverage.sh`): it shells out to `coverage json` for per-line
data and to `git diff` for the changed lines. Lines in omitted files
(the interactive shells) have no coverage data and are skipped, by
policy.
Usage:
scripts/coverage.sh # produce .coverage first
python3 scripts/diff_coverage.py # gate against origin/main, min 90%
python3 scripts/diff_coverage.py --base main --min 85
"""
from __future__ import annotations
import argparse
import json
import re
import subprocess
import sys
import tempfile
from pathlib import Path
_HUNK_RE = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
def _run(cmd: list[str]) -> str:
return subprocess.run(
cmd, check=True, capture_output=True, text=True,
).stdout
def added_lines_by_file(base: str) -> dict[str, set[int]]:
"""Map each changed .py file to the set of line numbers added/changed
relative to `base`, parsed from a zero-context unified diff."""
diff = _run(["git", "diff", "--unified=0", f"{base}...HEAD", "--", "*.py"])
out: dict[str, set[int]] = {}
current: str | None = None
new_line = 0
for line in diff.splitlines():
if line.startswith("+++ b/"):
current = line[6:]
out.setdefault(current, set())
continue
hunk = _HUNK_RE.match(line)
if hunk:
new_line = int(hunk.group(1))
continue
if current is None:
continue
if line.startswith("+") and not line.startswith("+++"):
out[current].add(new_line)
new_line += 1
elif line.startswith("-") and not line.startswith("---"):
# Deletion: does not advance the new-file cursor.
continue
return out
def coverage_json() -> dict[str, object]:
"""Render the existing .coverage data to JSON and load it."""
with tempfile.NamedTemporaryFile("r", suffix=".json", delete=True) as fh:
_run([sys.executable, "-m", "coverage", "json", "-o", fh.name])
return json.load(open(fh.name, encoding="utf-8"))
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--base", default="origin/main",
help="git ref to diff against (default: origin/main)")
ap.add_argument("--min", type=float, default=90.0,
help="minimum %% of changed executable lines covered")
args = ap.parse_args()
if not Path(".coverage").exists():
print("diff-coverage: no .coverage data; run scripts/coverage.sh first",
file=sys.stderr)
return 2
added = added_lines_by_file(args.base)
files = coverage_json().get("files", {})
if not isinstance(files, dict):
files = {}
total = 0
covered = 0
misses: list[str] = []
for path, lines in sorted(added.items()):
info = files.get(path)
if not isinstance(info, dict):
# Omitted file or not measured (e.g. a test file) — skip by policy.
continue
executed = set(info.get("executed_lines", []))
missing = set(info.get("missing_lines", []))
executable = lines & (executed | missing)
for ln in sorted(executable):
total += 1
if ln in executed:
covered += 1
else:
misses.append(f"{path}:{ln}")
if total == 0:
print("diff-coverage: no measured changed lines to check — pass")
return 0
pct = 100.0 * covered / total
print(f"diff-coverage: {covered}/{total} changed lines covered ({pct:.1f}%)")
if misses:
print("uncovered changed lines:", file=sys.stderr)
for m in misses:
print(f" {m}", file=sys.stderr)
if pct + 1e-9 < args.min:
print(f"diff-coverage: below {args.min:.0f}% threshold", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+4 -7
View File
@@ -92,9 +92,9 @@ class TestSandboxEscape(unittest.TestCase):
"on PATH: curl -sSL https://smolmachines.com/install.sh | sh"
)
# Throwaway static key for the git-gate fixture. It need not
# be a real SSH key: test 5 reaches gitleaks before any SSH
# attempt anyway.
# Throwaway "identity file" for the git-gate's `identity` field.
# It need not be a real SSH key: test 5 reaches gitleaks before
# any SSH attempt anyway.
fd, kp = tempfile.mkstemp(prefix="sandbox-test-key.")
os.close(fd)
cls._key_path = Path(kp)
@@ -123,10 +123,7 @@ class TestSandboxEscape(unittest.TestCase):
"git-gate": {"repos": {
"throwaway": {
"url": "ssh://git@unreachable.invalid:22/throwaway.git",
"key": {
"provider": "static",
"path": str(cls._key_path),
},
"identity": str(cls._key_path),
},
}},
},
@@ -198,7 +198,6 @@ class TestSmolmachinesLaunch(unittest.TestCase):
# connect fails, which is the property chunk 3 will
# preserve once egress is actually running.
r = self.bottle.exec(
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
f"curl -s --show-error --max-time 3 http://{self.plan.bundle_ip}:9099 "
"2>&1 || true"
)
-37
View File
@@ -1,37 +0,0 @@
"""Unit-test package init.
Isolates ``HOME`` to a throwaway directory for the entire unit suite so
no test ever reads or writes the real ``~/.bot-bottle`` (state, queue,
and audit dirs all derive from ``supervise.bot_bottle_root()``
``Path.home()``). Without this, a test that takes a ``flock`` on the
real audit log can **block indefinitely** when a live bottle's supervise
sidecar holds that lock observed as a hung ``coverage run`` at 0% CPU
and unisolated tests otherwise pollute the developer's home dir.
Individual tests that need their own ``HOME`` still override
``os.environ['HOME']`` and restore it; they now restore to this isolated
dir rather than the real one, so isolation holds either way. Tests that
patch ``supervise.bot_bottle_root`` directly are unaffected.
"""
from __future__ import annotations
import atexit
import os
import shutil
import tempfile
_real_home = os.environ.get("HOME")
_tmp_home = tempfile.mkdtemp(prefix="bot-bottle-unit-home.")
os.environ["HOME"] = _tmp_home
def _restore_home() -> None:
if _real_home is None:
os.environ.pop("HOME", None)
else:
os.environ["HOME"] = _real_home
shutil.rmtree(_tmp_home, ignore_errors=True)
atexit.register(_restore_home)
View File
-66
View File
@@ -1,66 +0,0 @@
"""Shared test doubles: a duck-typed forge and bottle runner."""
# Test doubles mirror an API shape; some params are intentionally unused.
# pylint: disable=unused-argument
from __future__ import annotations
from collections.abc import Sequence
from bot_bottle.orchestrator.runner import slugify
class FakeForge:
def __init__(self, members: tuple[str, ...] = ()) -> None:
self.members = set(members)
self.comments: list[tuple[int, str]] = []
self.descriptions: list[tuple[int, str]] = []
self.scope_denied: set[int] = set()
def is_org_member(self, org: str, username: str) -> bool:
return username in self.members
def read_issue(self, number: int) -> dict[str, object]:
return {"number": number, "kind": "issue"}
def read_pr(self, number: int) -> dict[str, object]:
return {"number": number, "merged": False}
def read_comments(self, number: int) -> list[dict[str, object]]:
return [{"id": 1, "user": "alice", "body": "hi"}]
def post_comment(self, number: int, body: str) -> None:
if number in self.scope_denied:
raise PermissionError(f"write to #{number} denied")
self.comments.append((number, body))
def update_description(self, number: int, body: str) -> None:
if number in self.scope_denied:
raise PermissionError(f"write to #{number} denied")
self.descriptions.append((number, body))
class FakeRunner:
def __init__(self) -> None:
self.calls: list[tuple[object, ...]] = []
def start(
self,
*,
agent: str,
bottles: Sequence[str],
label: str,
prompt: str,
forge_env: dict[str, str],
) -> str:
self.calls.append(("start", agent, tuple(bottles), label, prompt, dict(forge_env)))
return slugify(label)
def freeze(self, slug: str) -> None:
self.calls.append(("freeze", slug))
def resume(self, slug: str, prompt: str) -> None:
self.calls.append(("resume", slug, prompt))
def destroy(self, slug: str) -> None:
self.calls.append(("destroy", slug))
-178
View File
@@ -1,178 +0,0 @@
"""Unit: BotBottleStateStore, _token, conversions, make_forge/make_sidecar, build."""
from __future__ import annotations
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle.orchestrator.bootstrap import (
BotBottleStateStore,
_to_forge_state,
_to_record,
_token,
build,
make_forge,
make_sidecar,
)
from bot_bottle.orchestrator.config import Config
from bot_bottle.orchestrator.model import RunRecord
def _config(tmp: str) -> Config:
return Config(
forge_org="org",
gitea_api="http://g/api/v1",
watchdog_timeout_secs=1800,
webhook_host="127.0.0.1",
webhook_port=0,
queue_dir=Path(tmp) / "q",
sidecar_socket=Path(tmp) / "s.sock",
db_path=None,
)
def _record(**kw: object) -> RunRecord:
defaults: dict[str, object] = {
"owner": "o", "repo": "r", "issue_number": 1, "slug": "s1", "agent_name": "a",
"bottle_names": ["claude"], "backend_name": "docker", "agent_git_user": "bot",
"pr_number": 5, "status": "running", "last_checkin_at": "2026-01-01T00:00:00+00:00",
}
defaults.update(kw)
return RunRecord(**defaults) # type: ignore[arg-type]
class TokenTest(unittest.TestCase):
def test_gitea_token_env(self):
with patch.dict(os.environ, {"GITEA_TOKEN": "tok123"}):
self.assertEqual("tok123", _token())
def test_forge_gitea_token_fallback(self):
clean = {k: v for k, v in os.environ.items()
if k not in ("GITEA_TOKEN", "FORGE_GITEA_TOKEN")}
with patch.dict(os.environ, {**clean, "FORGE_GITEA_TOKEN": "tok456"}, clear=True):
self.assertEqual("tok456", _token())
def test_missing_token_raises(self):
clean = {k: v for k, v in os.environ.items()
if k not in ("GITEA_TOKEN", "FORGE_GITEA_TOKEN")}
with patch.dict(os.environ, clean, clear=True):
with self.assertRaises(RuntimeError):
_token()
class ConversionRoundTripTest(unittest.TestCase):
def test_record_survives_forge_state_roundtrip(self):
rec = _record()
result = _to_record(_to_forge_state(rec))
self.assertEqual(rec.owner, result.owner)
self.assertEqual(rec.repo, result.repo)
self.assertEqual(rec.issue_number, result.issue_number)
self.assertEqual(rec.slug, result.slug)
self.assertEqual(rec.agent_name, result.agent_name)
self.assertEqual(rec.bottle_names, result.bottle_names)
self.assertEqual(rec.backend_name, result.backend_name)
self.assertEqual(rec.agent_git_user, result.agent_git_user)
self.assertEqual(rec.pr_number, result.pr_number)
self.assertEqual(rec.status, result.status)
self.assertEqual(rec.last_checkin_at, result.last_checkin_at)
def test_none_pr_number_preserved(self):
rec = _record(pr_number=None)
result = _to_record(_to_forge_state(rec))
self.assertIsNone(result.pr_number)
class BotBottleStateStoreTest(unittest.TestCase):
def setUp(self):
self.store = BotBottleStateStore(None)
def test_upsert_and_get(self):
self.store.upsert(_record())
got = self.store.get("o", "r", 1)
assert got is not None
self.assertEqual("s1", got.slug)
def test_get_missing(self):
self.assertIsNone(self.store.get("o", "r", 99))
def test_upsert_replaces(self):
self.store.upsert(_record())
self.store.upsert(_record(slug="new-slug"))
got = self.store.get("o", "r", 1)
assert got is not None
self.assertEqual("new-slug", got.slug)
def test_delete(self):
self.store.upsert(_record())
self.store.delete("o", "r", 1)
self.assertIsNone(self.store.get("o", "r", 1))
def test_all_returns_all_records(self):
self.store.upsert(_record(issue_number=1, slug="s1"))
self.store.upsert(_record(issue_number=2, slug="s2"))
recs = self.store.all()
self.assertEqual(2, len(recs))
slugs = {r.slug for r in recs}
self.assertEqual({"s1", "s2"}, slugs)
def test_all_empty(self):
self.assertEqual([], self.store.all())
def test_bottle_names_preserved(self):
self.store.upsert(_record(bottle_names=["claude", "dev"]))
got = self.store.get("o", "r", 1)
assert got is not None
self.assertEqual(["claude", "dev"], got.bottle_names)
class MakeForgeTest(unittest.TestCase):
def test_returns_gitea_forge(self):
with tempfile.TemporaryDirectory() as tmp:
config = _config(tmp)
with patch.dict(os.environ, {"GITEA_TOKEN": "tok"}):
forge = make_forge(config, "owner", "repo")
from bot_bottle.contrib.gitea.client import GiteaForge
self.assertIsInstance(forge, GiteaForge)
class MakeSidecarTest(unittest.TestCase):
def test_returns_forge_sidecar(self):
with tempfile.TemporaryDirectory() as tmp:
config = _config(tmp)
with patch.dict(os.environ, {"GITEA_TOKEN": "tok"}):
sidecar = make_sidecar(config, "owner", "repo", 1, [])
from bot_bottle.orchestrator.sidecar import ForgeSidecar
self.assertIsInstance(sidecar, ForgeSidecar)
class BuildTest(unittest.TestCase):
def test_returns_server_watchdog_orchestrator(self):
with tempfile.TemporaryDirectory() as tmp:
config = _config(tmp)
with patch.dict(os.environ, {"GITEA_TOKEN": "tok"}):
server, watchdog, orch = build(config)
server.server_close()
from bot_bottle.orchestrator.lifecycle import Orchestrator
from bot_bottle.orchestrator.watchdog import Watchdog
from bot_bottle.orchestrator.webhook import WebhookServer
self.assertIsInstance(server, WebhookServer)
self.assertIsInstance(watchdog, Watchdog)
self.assertIsInstance(orch, Orchestrator)
def test_server_binds_to_configured_host(self):
with tempfile.TemporaryDirectory() as tmp:
config = _config(tmp)
with patch.dict(os.environ, {"GITEA_TOKEN": "tok"}):
server, _, _ = build(config)
addr = server.server_address
server.server_close()
self.assertEqual("127.0.0.1", addr[0])
self.assertGreater(addr[1], 0)
if __name__ == "__main__":
unittest.main()
-38
View File
@@ -1,38 +0,0 @@
"""Unit: Config.from_env."""
from __future__ import annotations
import unittest
from pathlib import Path
from bot_bottle.orchestrator.config import Config
class ConfigTest(unittest.TestCase):
def test_defaults(self):
c = Config.from_env({"HOME": "/home/x"})
self.assertEqual("bot-bottle", c.forge_org)
self.assertEqual(1800, c.watchdog_timeout_secs)
self.assertEqual("127.0.0.1", c.webhook_host)
self.assertEqual(8477, c.webhook_port)
self.assertEqual(Path("/home/x/.bot-bottle/forge-queue"), c.queue_dir)
self.assertIsNone(c.db_path)
def test_overrides(self):
c = Config.from_env({
"HOME": "/home/x",
"FORGE_ORG": "agents",
"FORGE_WATCHDOG_TIMEOUT": "60",
"FORGE_GITEA_API": "https://g.example/api/v1",
"FORGE_WEBHOOK_PORT": "9000",
"FORGE_DB_PATH": "/data/bb.db",
})
self.assertEqual("agents", c.forge_org)
self.assertEqual(60, c.watchdog_timeout_secs)
self.assertEqual("https://g.example/api/v1", c.gitea_api)
self.assertEqual(9000, c.webhook_port)
self.assertEqual(Path("/data/bb.db"), c.db_path)
if __name__ == "__main__":
unittest.main()
-68
View File
@@ -1,68 +0,0 @@
"""Unit: webhook payload parsing."""
from __future__ import annotations
import unittest
from bot_bottle.orchestrator.events import parse_event
from bot_bottle.orchestrator.model import CommentCreated, IssueAssigned, PullRequestClosed
_REPO = {"repository": {"name": "bot-bottle", "owner": {"login": "didericis"}}}
class ParseEventTest(unittest.TestCase):
def test_issue_assigned(self):
payload = {
**_REPO,
"action": "assigned",
"issue": {
"number": 17,
"title": "Fix it",
"body": "please",
"assignees": [{"login": "agent-bot"}],
"labels": [{"name": "bot-bottle:implementer"}],
},
}
ev = parse_event("issues", payload)
self.assertIsInstance(ev, IssueAssigned)
assert isinstance(ev, IssueAssigned)
self.assertEqual(("didericis", "bot-bottle", 17), (ev.owner, ev.repo, ev.issue_number))
self.assertEqual(("agent-bot",), ev.assignees)
self.assertEqual(("bot-bottle:implementer",), ev.labels)
def test_issue_non_assigned_ignored(self):
self.assertIsNone(parse_event("issues", {**_REPO, "action": "opened", "issue": {}}))
def test_comment_created(self):
payload = {
**_REPO,
"action": "created",
"issue": {"number": 42, "pull_request": {"x": 1}},
"comment": {"id": 5, "user": {"login": "reviewer"}, "body": "redo"},
}
ev = parse_event("issue_comment", payload)
assert isinstance(ev, CommentCreated)
self.assertEqual(42, ev.issue_number)
self.assertEqual("reviewer", ev.author)
self.assertTrue(ev.is_pull)
def test_pull_request_closed(self):
payload = {**_REPO, "action": "closed", "pull_request": {"number": 8, "merged": True}}
ev = parse_event("pull_request", payload)
assert isinstance(ev, PullRequestClosed)
self.assertEqual(8, ev.pr_number)
self.assertTrue(ev.merged)
def test_pull_request_non_closed_ignored(self):
self.assertIsNone(parse_event("pull_request", {**_REPO, "action": "opened"}))
def test_comment_non_created_action_ignored(self):
payload = {**_REPO, "action": "edited", "issue": {}, "comment": {}}
self.assertIsNone(parse_event("issue_comment", payload))
def test_unknown_kind_ignored(self):
self.assertIsNone(parse_event("push", {**_REPO}))
if __name__ == "__main__":
unittest.main()
@@ -1,75 +0,0 @@
"""Unit: ForgeState + SqliteForgeStateStore."""
from __future__ import annotations
import unittest
from bot_bottle.contrib.gitea.forge_state import ForgeState, SqliteForgeStateStore
def _state(**kw: object) -> ForgeState:
defaults: dict[str, object] = dict(
owner="alice", repo="myrepo", issue_number=1,
slug="impl-alice-myrepo-1", agent_name="impl",
)
defaults.update(kw)
return ForgeState(**defaults) # type: ignore[arg-type]
class ForgeStateStoreTest(unittest.TestCase):
def setUp(self):
self.store = SqliteForgeStateStore(None)
def test_upsert_and_get(self):
s = _state()
self.store.upsert(s)
got = self.store.get("alice", "myrepo", 1)
assert got is not None
self.assertEqual("impl-alice-myrepo-1", got.slug)
self.assertEqual("impl", got.agent_name)
def test_get_missing(self):
self.assertIsNone(self.store.get("alice", "myrepo", 99))
def test_upsert_replaces(self):
self.store.upsert(_state(status="running"))
self.store.upsert(_state(status="frozen"))
got = self.store.get("alice", "myrepo", 1)
assert got is not None
self.assertEqual("frozen", got.status)
def test_delete(self):
self.store.upsert(_state())
self.store.delete("alice", "myrepo", 1)
self.assertIsNone(self.store.get("alice", "myrepo", 1))
def test_delete_missing_no_error(self):
self.store.delete("alice", "myrepo", 99)
def test_all_sorted(self):
self.store.upsert(_state(owner="z", issue_number=2))
self.store.upsert(_state(owner="a", issue_number=1))
rows = self.store.all()
self.assertEqual(("a", "z"), (rows[0].owner, rows[1].owner))
def test_bottle_names_roundtrip(self):
self.store.upsert(_state(bottle_names=["claude", "dev"]))
got = self.store.get("alice", "myrepo", 1)
assert got is not None
self.assertEqual(["claude", "dev"], got.bottle_names)
def test_pr_number_none_roundtrip(self):
self.store.upsert(_state(pr_number=None))
got = self.store.get("alice", "myrepo", 1)
assert got is not None
self.assertIsNone(got.pr_number)
def test_pr_number_int_roundtrip(self):
self.store.upsert(_state(pr_number=42))
got = self.store.get("alice", "myrepo", 1)
assert got is not None
self.assertEqual(42, got.pr_number)
if __name__ == "__main__":
unittest.main()
-163
View File
@@ -1,163 +0,0 @@
"""Unit: the orchestration lifecycle."""
from __future__ import annotations
import unittest
from typing import cast
from bot_bottle.orchestrator.lifecycle import Orchestrator
from bot_bottle.orchestrator.model import (
STATUS_FROZEN,
STATUS_RUNNING,
CommentCreated,
IssueAssigned,
PullRequestClosed,
)
from bot_bottle.orchestrator.store import InMemoryStateStore
from ._fakes import FakeForge, FakeRunner
def _assigned(
labels: tuple[str, ...] = ("bot-bottle:impl",),
assignees: tuple[str, ...] = ("agent-bot",),
) -> IssueAssigned:
return IssueAssigned(
owner="didericis", repo="bot-bottle", issue_number=17,
title="t", body="the task", assignees=tuple(assignees), labels=tuple(labels),
)
class LifecycleTest(unittest.TestCase):
def setUp(self):
self.forge = FakeForge(members=("agent-bot",))
self.store = InMemoryStateStore()
self.runner = FakeRunner()
self.orch = Orchestrator(
forge=self.forge, store=self.store, runner=self.runner,
org="bot-bottle", gitea_api="https://g/api/v1",
now=lambda: "2026-07-01T00:00:00-04:00",
)
def _record(self):
return self.store.get("didericis", "bot-bottle", 17)
def test_assigned_targeted_launches(self):
self.orch.handle(_assigned())
rec = self._record()
assert rec is not None
self.assertEqual(STATUS_RUNNING, rec.status)
self.assertEqual("impl-didericis-bot-bottle-17", rec.slug)
self.assertEqual("start", self.runner.calls[0][0])
# forge context injected into the child env.
env = cast("dict[str, str]", self.runner.calls[0][5])
self.assertEqual("didericis", env["FORGE_OWNER"])
self.assertEqual("17", env["FORGE_ISSUE_NUMBER"])
def test_untargeted_ignored(self):
self.orch.handle(_assigned(labels=("bug",)))
self.assertIsNone(self._record())
self.assertEqual([], self.runner.calls)
def test_assigned_is_idempotent(self):
self.orch.handle(_assigned())
self.orch.handle(_assigned()) # redelivery
starts = [c for c in self.runner.calls if c[0] == "start"]
self.assertEqual(1, len(starts))
def test_done_signal_freezes(self):
self.orch.handle(_assigned())
self.orch.on_done_signal("didericis", "bot-bottle", 17, "success", "done")
rec = self._record()
assert rec is not None
self.assertEqual(STATUS_FROZEN, rec.status)
self.assertIn(("freeze", "impl-didericis-bot-bottle-17"), self.runner.calls)
def test_done_signal_ignored_when_not_running(self):
# No record yet -> no freeze.
self.orch.on_done_signal("didericis", "bot-bottle", 17, "s", "")
self.assertEqual([], self.runner.calls)
def test_comment_on_frozen_resumes(self):
self.orch.handle(_assigned())
self.orch.on_done_signal("didericis", "bot-bottle", 17, "s", "")
self.orch.handle(CommentCreated(
owner="didericis", repo="bot-bottle", issue_number=17,
comment_id=1, author="reviewer", body="please redo", is_pull=False,
))
rec = self._record()
assert rec is not None
self.assertEqual(STATUS_RUNNING, rec.status)
self.assertIn(("resume", "impl-didericis-bot-bottle-17", "please redo"),
self.runner.calls)
def test_comment_echo_guard(self):
self.orch.handle(_assigned())
self.orch.on_done_signal("didericis", "bot-bottle", 17, "s", "")
rec = self._record()
assert rec is not None
rec.agent_git_user = "agent-bot"
self.store.upsert(rec)
self.orch.handle(CommentCreated(
owner="didericis", repo="bot-bottle", issue_number=17,
comment_id=2, author="agent-bot", body="I finished", is_pull=False,
))
# Still frozen, no resume triggered by the agent's own comment.
self.assertEqual(STATUS_FROZEN, self._record().status) # type: ignore[union-attr]
self.assertNotIn("resume", [c[0] for c in self.runner.calls])
def test_comment_on_running_ignored(self):
self.orch.handle(_assigned()) # running
self.orch.handle(CommentCreated(
owner="didericis", repo="bot-bottle", issue_number=17,
comment_id=1, author="reviewer", body="hi", is_pull=False,
))
self.assertNotIn("resume", [c[0] for c in self.runner.calls])
def test_pr_comment_routes_via_link(self):
self.orch.handle(_assigned())
self.orch.on_done_signal("didericis", "bot-bottle", 17, "s", "")
self.orch.link_pr("didericis", "bot-bottle", 17, 42)
# Comment arrives on PR #42 (issue_number == PR number in Gitea).
self.orch.handle(CommentCreated(
owner="didericis", repo="bot-bottle", issue_number=42,
comment_id=9, author="reviewer", body="fix", is_pull=True,
))
self.assertIn(("resume", "impl-didericis-bot-bottle-17", "fix"),
self.runner.calls)
def test_pr_closed_destroys_and_removes(self):
self.orch.handle(_assigned())
self.orch.link_pr("didericis", "bot-bottle", 17, 42)
self.orch.handle(PullRequestClosed(
owner="didericis", repo="bot-bottle", pr_number=42, merged=True,
))
self.assertIn(("destroy", "impl-didericis-bot-bottle-17"), self.runner.calls)
self.assertIsNone(self._record())
def test_comment_on_untracked_issue_ignored(self):
# No record in store and is_pull=False -> _route_comment returns None.
self.orch.handle(CommentCreated(
owner="didericis", repo="bot-bottle", issue_number=99,
comment_id=1, author="reviewer", body="hi", is_pull=False,
))
self.assertEqual([], self.runner.calls)
def test_pr_closed_untracked_pr_ignored(self):
# _find_by_pr finds nothing -> _on_pr_closed exits early.
self.orch.handle(PullRequestClosed(
owner="didericis", repo="bot-bottle", pr_number=999, merged=True,
))
self.assertEqual([], self.runner.calls)
class IsoNowTest(unittest.TestCase):
def test_returns_iso_string(self):
from bot_bottle.orchestrator.lifecycle import _iso_now
ts = _iso_now()
self.assertIsInstance(ts, str)
self.assertIn("T", ts)
if __name__ == "__main__":
unittest.main()
-88
View File
@@ -1,88 +0,0 @@
"""Unit: __main__ CLI entry points (run and status commands)."""
from __future__ import annotations
import io
import unittest
from unittest.mock import patch
from bot_bottle.orchestrator.__main__ import main
from bot_bottle.orchestrator.config import Config
from bot_bottle.orchestrator.model import RunRecord
def _config() -> Config:
return Config.from_env({"HOME": "/tmp"})
class MainRunTest(unittest.TestCase):
def test_run_delegates_to_bootstrap(self):
config = _config()
with patch.object(Config, "from_env", return_value=config), \
patch("bot_bottle.orchestrator.bootstrap.run") as mock_run:
rc = main(["run"])
self.assertEqual(0, rc)
mock_run.assert_called_once_with(config)
def test_run_prints_listen_address_to_stderr(self):
config = _config()
err = io.StringIO()
with patch.object(Config, "from_env", return_value=config), \
patch("bot_bottle.orchestrator.bootstrap.run"), \
patch("sys.stderr", err):
main(["run"])
self.assertIn(str(config.webhook_port), err.getvalue())
class MainStatusTest(unittest.TestCase):
def test_status_empty_store(self):
config = _config()
with patch.object(Config, "from_env", return_value=config), \
patch("bot_bottle.orchestrator.bootstrap.BotBottleStateStore") as MockStore:
MockStore.return_value.all.return_value = []
rc = main(["status"])
self.assertEqual(0, rc)
def test_status_prints_records(self):
config = _config()
rec = RunRecord(
owner="o", repo="r", issue_number=1, slug="my-slug",
agent_name="a", pr_number=7, status="frozen",
)
out = io.StringIO()
with patch.object(Config, "from_env", return_value=config), \
patch("bot_bottle.orchestrator.bootstrap.BotBottleStateStore") as MockStore, \
patch("sys.stdout", out):
MockStore.return_value.all.return_value = [rec]
rc = main(["status"])
self.assertEqual(0, rc)
self.assertIn("my-slug", out.getvalue())
self.assertIn("PR#7", out.getvalue())
def test_status_no_pr_prints_dash(self):
config = _config()
rec = RunRecord(
owner="o", repo="r", issue_number=2, slug="s2",
agent_name="a", pr_number=None, status="running",
)
out = io.StringIO()
with patch.object(Config, "from_env", return_value=config), \
patch("bot_bottle.orchestrator.bootstrap.BotBottleStateStore") as MockStore, \
patch("sys.stdout", out):
MockStore.return_value.all.return_value = [rec]
main(["status"])
self.assertIn("-", out.getvalue())
class MainArgparseTest(unittest.TestCase):
def test_no_command_exits(self):
with self.assertRaises(SystemExit):
main([])
def test_unknown_command_exits(self):
with self.assertRaises(SystemExit):
main(["bogus"])
if __name__ == "__main__":
unittest.main()
@@ -1,53 +0,0 @@
"""Unit: provenance assembly + serialization."""
from __future__ import annotations
import unittest
from bot_bottle.orchestrator.model import RunRecord
from bot_bottle.orchestrator.provenance import build_provenance, ops_from_log, provenance_to_dict
def _record() -> RunRecord:
return RunRecord(
owner="didericis", repo="bot-bottle", issue_number=17,
slug="impl-17", agent_name="impl", bottle_names=["claude"],
last_checkin_at="2026-07-01T00:05:00-04:00",
)
class ProvenanceTest(unittest.TestCase):
def test_ops_from_log(self):
ops = ops_from_log([
{"at": "T1", "op": "read_pr", "target": 5, "detail": "ok"},
{"at": "T2", "op": "signal_done", "target": None, "detail": "success: done"},
])
self.assertEqual(2, len(ops))
self.assertEqual("read_pr", ops[0].op)
self.assertIsNone(ops[1].target)
def test_build_and_serialize(self):
ops = ops_from_log([{"at": "T1", "op": "post_comment", "target": 17, "detail": "ok"}])
prov = build_provenance(
_record(), ops=ops, started_at="2026-07-01T00:00:00-04:00",
finished_at="2026-07-01T00:05:00-04:00", exit_code=0, watchdog_fired=False,
)
d = provenance_to_dict(prov)
self.assertEqual("impl-17", d["slug"])
self.assertEqual("didericis", d["owner"])
self.assertEqual(["claude"], d["bottles"])
self.assertEqual(0, d["exit_code"])
self.assertFalse(d["watchdog_fired"])
self.assertEqual(1, len(d["ops"]))
self.assertEqual("post_comment", d["ops"][0]["op"])
def test_watchdog_flag_serialized(self):
prov = build_provenance(
_record(), ops=(), started_at="", finished_at="",
exit_code=None, watchdog_fired=True,
)
self.assertTrue(provenance_to_dict(prov)["watchdog_fired"])
if __name__ == "__main__":
unittest.main()
-95
View File
@@ -1,95 +0,0 @@
"""Unit: ProgrammaticBottleRunner + slugify."""
from __future__ import annotations
import sys
import types
import unittest
from typing import Any
from unittest.mock import MagicMock
from bot_bottle.orchestrator.runner import ProgrammaticBottleRunner, slugify
class SlugifyTest(unittest.TestCase):
def test_basic(self):
self.assertEqual("impl-didericis-bot-bottle-17",
slugify("impl-didericis-bot-bottle-17"))
def test_collapses_and_strips(self):
self.assertEqual("a-b-c", slugify(" A_B/C!! "))
def _make_api_stub(**overrides: object) -> Any:
"""Return a mock bot_bottle.api module with sensible defaults."""
stub: Any = types.ModuleType("bot_bottle.api")
stub.start_headless = MagicMock(return_value="impl-r-17")
stub.freeze = MagicMock()
stub.resume_headless = MagicMock()
stub.destroy = MagicMock()
for k, v in overrides.items():
setattr(stub, k, v)
return stub
class ProgrammaticRunnerTest(unittest.TestCase):
def setUp(self) -> None:
self._api: Any = _make_api_stub()
sys.modules["bot_bottle.api"] = self._api
self.runner = ProgrammaticBottleRunner()
def tearDown(self) -> None:
sys.modules.pop("bot_bottle.api", None)
def test_start_returns_slug_from_api(self) -> None:
slug = self.runner.start(
agent="impl", bottles=["claude", "dev"], label="impl-r-17",
prompt="do it", forge_env={"FORGE_OWNER": "didericis"},
)
self.assertEqual("impl-r-17", slug)
def test_start_forwards_all_args(self) -> None:
self.runner.start(
agent="impl", bottles=["claude", "dev"], label="impl-r-17",
prompt="do it", forge_env={"FORGE_OWNER": "didericis"},
)
self._api.start_headless.assert_called_once_with(
"impl",
prompt="do it",
bottles=["claude", "dev"],
label="impl-r-17",
forge_env={"FORGE_OWNER": "didericis"},
)
def test_start_no_bottles_passes_none(self) -> None:
self.runner.start(agent="impl", bottles=[], label="l", prompt="p", forge_env={})
call_kwargs = self._api.start_headless.call_args[1]
self.assertIsNone(call_kwargs["bottles"])
def test_freeze_delegates_to_api(self) -> None:
self.runner.freeze("slug-1")
self._api.freeze.assert_called_once_with("slug-1")
def test_freeze_returns_none(self) -> None:
result = self.runner.freeze("slug-1")
self.assertIsNone(result)
def test_resume_delegates_to_api(self) -> None:
self.runner.resume("slug-1", "address review")
self._api.resume_headless.assert_called_once_with("slug-1", prompt="address review")
def test_resume_returns_none(self) -> None:
result = self.runner.resume("slug-1", "p")
self.assertIsNone(result)
def test_destroy_delegates_to_api(self) -> None:
self.runner.destroy("slug-7")
self._api.destroy.assert_called_once_with("slug-7")
def test_destroy_returns_none(self) -> None:
result = self.runner.destroy("slug-7")
self.assertIsNone(result)
if __name__ == "__main__":
unittest.main()
@@ -1,75 +0,0 @@
"""Unit: ScopedForge — read-anywhere / write-scoped access control."""
from __future__ import annotations
import unittest
from bot_bottle.contrib.forge.base import ScopedForge
from ._fakes import FakeForge
class ScopedForgeTest(unittest.TestCase):
def setUp(self):
self.inner = FakeForge()
self.scoped = ScopedForge(
self.inner, assigned_issue=10, assigned_prs=[20, 30]
)
# --- reads always pass through -----------------------------------------
def test_read_issue_allowed_anywhere(self):
for number in (10, 20, 99):
result = self.scoped.read_issue(number)
self.assertEqual(number, result["number"])
def test_read_pr_allowed_anywhere(self):
for number in (10, 20, 99):
result = self.scoped.read_pr(number)
self.assertEqual(number, result["number"])
def test_read_comments_allowed_anywhere(self):
comments = self.scoped.read_comments(99)
self.assertTrue(len(comments) > 0)
def test_is_org_member_passes_through(self):
inner = FakeForge(members=("alice",))
scoped = ScopedForge(inner, assigned_issue=1, assigned_prs=[])
self.assertTrue(scoped.is_org_member("org", "alice"))
self.assertFalse(scoped.is_org_member("org", "bob"))
# --- writes: assigned numbers allowed ----------------------------------
def test_post_comment_on_assigned_issue(self):
self.scoped.post_comment(10, "hi")
self.assertIn((10, "hi"), self.inner.comments)
def test_post_comment_on_assigned_pr(self):
self.scoped.post_comment(20, "lgtm")
self.assertIn((20, "lgtm"), self.inner.comments)
def test_update_description_on_assigned(self):
self.scoped.update_description(30, "updated")
self.assertIn((30, "updated"), self.inner.descriptions)
# --- writes: unassigned numbers denied ---------------------------------
def test_post_comment_denied_for_unassigned(self):
with self.assertRaises(PermissionError):
self.scoped.post_comment(99, "nope")
self.assertEqual([], self.inner.comments)
def test_update_description_denied_for_unassigned(self):
with self.assertRaises(PermissionError):
self.scoped.update_description(99, "nope")
self.assertEqual([], self.inner.descriptions)
def test_error_message_names_number(self):
try:
self.scoped.post_comment(99, "nope")
except PermissionError as exc:
self.assertIn("99", str(exc))
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More