Compare commits
109 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bba24d87f7 | |||
| efb3af4a93 | |||
| 65746af720 | |||
| d9e9d27e01 | |||
| 83351606c6 | |||
| d528f578aa | |||
| cf3310e818 | |||
| 74d6b25183 | |||
| dc837a5400 | |||
| 4eff49c9c5 | |||
| 965d5073c3 | |||
| e82bbb587f | |||
| c89a0d334a | |||
| ac9b6d593f | |||
| 8c0a9c5bc6 | |||
| 63a3b9b50a | |||
| 7e6e0b1f5a | |||
| ab528d9163 | |||
| 7967d32f12 | |||
| a7de3dbb9f | |||
| 0fbf2ab513 | |||
| 436f42c00c | |||
| 881869352d | |||
| 3f982009e2 | |||
| 52820278fd | |||
| abcb336e7c | |||
| 1c7812fa9f | |||
| 4c60779fac | |||
| 726713d081 | |||
| 5265e25f9b | |||
| 035ed430ba | |||
| f145203eee | |||
| eafd1c1fb2 | |||
| e6ad7ae10e | |||
| 05b12b41b6 | |||
| a59da9921e | |||
| bbd6ec85ac | |||
| ce8cb5f0f1 | |||
| 9eb5eef676 | |||
| c94a2542bd | |||
| e6b3cd1824 | |||
| 49f77f2d1e | |||
| d3c2d9e8f6 | |||
| f114c861b4 | |||
| 544a024e22 | |||
| 7f43f64c24 | |||
| 059bba8c4f | |||
| 82b8dffc54 | |||
| 8795616a99 | |||
| f548c30608 | |||
| 24c302ae0f | |||
| a5d08bd64e | |||
| e1ec0afd86 | |||
| b0679dc4c3 | |||
| 3afae56a35 | |||
| 2c18581e04 | |||
| 9800269d11 | |||
| a5078daf1c | |||
| 6316f8379f | |||
| dfe85a201d | |||
| 7c30cd2f52 | |||
| a0c6f938cb | |||
| a430bac1bf | |||
| 59b87bdaab | |||
| 0de3c93ad0 | |||
| 570cd42532 | |||
| 73a4fbe0a7 | |||
| b032ff746d | |||
| 873d75f852 | |||
| 1bd676de06 | |||
| 0bf1532557 | |||
| 58169e2ce9 | |||
| 86bb8e1908 | |||
| 0ca81b102c | |||
| 4e185fab6b | |||
| f665d62712 | |||
| 7b8f40a5f0 | |||
| 605a70408e | |||
| 832808ff9a | |||
| ea66f63d45 | |||
| 83db7336c8 | |||
| bcdffc8400 | |||
| f44751c4b8 | |||
| 3d557beeee | |||
| 44365ecf68 | |||
| 703b12ee9a | |||
| d1556f4659 | |||
| 06eed5b236 | |||
| 98e4e2b7dc | |||
| 9eca46b408 | |||
| 0efc07ba67 | |||
| f12b0f754e | |||
| a593b157d6 | |||
| 15b54cdff2 | |||
| d3bc463295 | |||
| 50ec920243 | |||
| 4372b8a6dd | |||
| 63a7e63ce9 | |||
| c0e1f5fd70 | |||
| 41570e04c0 | |||
| 6f0a42159f | |||
| 5c17f0de95 | |||
| 8a09e32fcc | |||
| 83463f1cc8 | |||
| 0b5d59cf9e | |||
| 464012d97c | |||
| b5f8a27c47 | |||
| f0ca4e3527 | |||
| ca6d257f30 |
@@ -1,6 +1,6 @@
|
|||||||
# Weekly canary suite. Catches upstream regressions (broken pipelock
|
# Weekly canary suite. Catches upstream regressions (broken pinned
|
||||||
# image packaging at the pinned digest, etc.) without coupling every
|
# digest, etc.) without coupling every dev push to upstream registry
|
||||||
# dev push to upstream registry availability.
|
# availability.
|
||||||
#
|
#
|
||||||
# Opt-in via CLAUDE_BOTTLE_RUN_CANARIES=1 so the same files can be run
|
# Opt-in via CLAUDE_BOTTLE_RUN_CANARIES=1 so the same files can be run
|
||||||
# locally with the same gating.
|
# locally with the same gating.
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
name: lint
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
paths:
|
||||||
|
- "**.py"
|
||||||
|
- ".pylintrc"
|
||||||
|
- ".gitea/workflows/lint.yml"
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
lint:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v3
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v4
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
|
||||||
|
- name: Install dev dependencies
|
||||||
|
run: |
|
||||||
|
python -m pip install --upgrade pip
|
||||||
|
pip install -r requirements-dev.txt
|
||||||
|
|
||||||
|
- name: Run pylint
|
||||||
|
run: |
|
||||||
|
# Run pylint on all Python files in the repo
|
||||||
|
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0 || true
|
||||||
|
|
||||||
|
- name: Run pyright
|
||||||
|
run: |
|
||||||
|
# Run pyright type checking
|
||||||
|
pyright .
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# Block PRs that add prd-new-*.md files directly to main.
|
||||||
|
#
|
||||||
|
# prd-new-*.md files are placeholders — they must go through a PR so
|
||||||
|
# the post-merge prd-number workflow can assign a sequential number and
|
||||||
|
# rename the file. A direct push or a PR that slips through without
|
||||||
|
# triggering the check would leave an un-numbered PRD on main.
|
||||||
|
|
||||||
|
name: prd-check
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
paths:
|
||||||
|
- 'docs/prds/prd-new-*.md'
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
no-prd-new-on-main:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
|
||||||
|
- name: Fail if prd-new-*.md files are present in the diff
|
||||||
|
run: |
|
||||||
|
base="${{ github.event.pull_request.base.sha }}"
|
||||||
|
head="${{ github.event.pull_request.head.sha }}"
|
||||||
|
new_prds=$(git diff --name-only --diff-filter=A "$base" "$head" \
|
||||||
|
| grep -E '^docs/prds/prd-new-.+\.md$' || true)
|
||||||
|
if [ -n "$new_prds" ]; then
|
||||||
|
echo "ERROR: PRs to main must not add prd-new-*.md files directly."
|
||||||
|
echo "These files must be merged via a feature branch so the"
|
||||||
|
echo "prd-number workflow can assign a sequential number on merge:"
|
||||||
|
echo "$new_prds"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "OK: no prd-new-*.md files added in this PR."
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
# Assign sequential numbers to prd-new-*.md files on merge to main.
|
||||||
|
#
|
||||||
|
# When a PR merges to main and includes prd-new-*.md files this workflow:
|
||||||
|
# 1. Finds the next available NNNN number by scanning existing PRDs.
|
||||||
|
# 2. Renames each prd-new-*.md to NNNN-<slug>.md.
|
||||||
|
# 3. Updates the title header (# PRD prd-new: → # PRD NNNN:).
|
||||||
|
# 4. Flips Status: Draft → Active when the merge commit also touched
|
||||||
|
# files outside docs/prds/ (i.e. the implementation shipped together
|
||||||
|
# with the PRD).
|
||||||
|
# 5. Commits the renaming back to main.
|
||||||
|
#
|
||||||
|
# No-op if the push contained no prd-new-*.md files.
|
||||||
|
|
||||||
|
name: prd-number
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
paths:
|
||||||
|
- 'docs/prds/prd-new-*.md'
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
assign-numbers:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
permissions:
|
||||||
|
contents: write
|
||||||
|
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 2
|
||||||
|
token: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
|
||||||
|
- name: Configure git
|
||||||
|
run: |
|
||||||
|
git config user.name "github-actions[bot]"
|
||||||
|
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||||
|
|
||||||
|
- name: Assign PRD numbers
|
||||||
|
run: |
|
||||||
|
python3 - <<'EOF'
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
prds_dir = Path("docs/prds")
|
||||||
|
|
||||||
|
# Files added in the latest commit (HEAD vs HEAD~1).
|
||||||
|
result = subprocess.run(
|
||||||
|
["git", "diff", "--name-only", "--diff-filter=A", "HEAD~1", "HEAD"],
|
||||||
|
capture_output=True, text=True, check=True,
|
||||||
|
)
|
||||||
|
added = [Path(p) for p in result.stdout.splitlines()]
|
||||||
|
new_prds = [p for p in added if p.parent == prds_dir
|
||||||
|
and re.match(r"prd-new-.+\.md$", p.name)]
|
||||||
|
|
||||||
|
if not new_prds:
|
||||||
|
print("No prd-new-*.md files added in this commit — nothing to do.")
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
|
# Determine whether non-PRD files were also changed (for Status flip).
|
||||||
|
all_changed = subprocess.run(
|
||||||
|
["git", "diff", "--name-only", "HEAD~1", "HEAD"],
|
||||||
|
capture_output=True, text=True, check=True,
|
||||||
|
).stdout.splitlines()
|
||||||
|
non_prd_changed = any(
|
||||||
|
not f.startswith("docs/prds/") for f in all_changed
|
||||||
|
)
|
||||||
|
|
||||||
|
# Find next available number.
|
||||||
|
existing = sorted(
|
||||||
|
int(m.group(1))
|
||||||
|
for p in prds_dir.glob("*.md")
|
||||||
|
if (m := re.match(r"^(\d{4})-", p.name))
|
||||||
|
)
|
||||||
|
next_num = (max(existing) + 1) if existing else 1
|
||||||
|
|
||||||
|
for prd_path in sorted(new_prds):
|
||||||
|
slug = re.sub(r"^prd-new-", "", prd_path.stem)
|
||||||
|
new_name = f"{next_num:04d}-{slug}.md"
|
||||||
|
new_path = prds_dir / new_name
|
||||||
|
print(f" {prd_path.name} → {new_name}")
|
||||||
|
|
||||||
|
content = prd_path.read_text()
|
||||||
|
|
||||||
|
# Update title header.
|
||||||
|
content = re.sub(
|
||||||
|
r"^(#\s+PRD\s+)prd-new(:)",
|
||||||
|
rf"\g<1>{next_num:04d}\2",
|
||||||
|
content,
|
||||||
|
count=1,
|
||||||
|
flags=re.MULTILINE,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Conditionally flip Status.
|
||||||
|
if non_prd_changed:
|
||||||
|
content = re.sub(
|
||||||
|
r"(\*\*Status:\*\*\s*)Draft",
|
||||||
|
r"\g<1>Active",
|
||||||
|
content,
|
||||||
|
count=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
new_path.write_text(content)
|
||||||
|
subprocess.run(["git", "rm", str(prd_path)], check=True)
|
||||||
|
subprocess.run(["git", "add", str(new_path)], check=True)
|
||||||
|
next_num += 1
|
||||||
|
|
||||||
|
subprocess.run(
|
||||||
|
["git", "commit", "-m", "ci(prd): assign sequential numbers to new PRDs"],
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
subprocess.run(["git", "push"], check=True)
|
||||||
|
EOF
|
||||||
@@ -21,7 +21,11 @@ on:
|
|||||||
push:
|
push:
|
||||||
branches:
|
branches:
|
||||||
- main
|
- main
|
||||||
|
paths:
|
||||||
|
- '**.py'
|
||||||
pull_request:
|
pull_request:
|
||||||
|
paths:
|
||||||
|
- '**.py'
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
unit:
|
unit:
|
||||||
|
|||||||
@@ -0,0 +1,79 @@
|
|||||||
|
name: Update Quality Badges
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
paths:
|
||||||
|
- '**.py'
|
||||||
|
- '.pylintrc'
|
||||||
|
- 'pyrightconfig.json'
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
update-badges:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v3
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
token: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v4
|
||||||
|
with:
|
||||||
|
python-version: '3.12'
|
||||||
|
|
||||||
|
- name: Install dev dependencies
|
||||||
|
run: |
|
||||||
|
python -m pip install --upgrade pip
|
||||||
|
pip install -r requirements-dev.txt
|
||||||
|
|
||||||
|
- name: Run pylint and extract score
|
||||||
|
id: pylint
|
||||||
|
run: |
|
||||||
|
PYLINT_OUTPUT=$(python -m pylint bot_bottle/ 2>&1) || true
|
||||||
|
SCORE=$(echo "$PYLINT_OUTPUT" | grep -oP '(?<=rated at )\d+\.\d+/10' | head -1)
|
||||||
|
echo "score=$SCORE" >> $GITHUB_OUTPUT
|
||||||
|
echo "Pylint score: $SCORE"
|
||||||
|
|
||||||
|
- name: Run pyright and check errors
|
||||||
|
id: pyright
|
||||||
|
run: |
|
||||||
|
PYRIGHT_OUTPUT=$(python -m pyright 2>&1) || true
|
||||||
|
ERRORS=$(echo "$PYRIGHT_OUTPUT" | grep -oP '\d+(?= error)' | head -1)
|
||||||
|
echo "errors=$ERRORS" >> $GITHUB_OUTPUT
|
||||||
|
echo "Pyright errors: $ERRORS"
|
||||||
|
|
||||||
|
- name: Update badges in README
|
||||||
|
run: |
|
||||||
|
PYLINT_SCORE="${{ steps.pylint.outputs.score }}"
|
||||||
|
PYRIGHT_ERRORS="${{ steps.pyright.outputs.errors }}"
|
||||||
|
|
||||||
|
PYLINT_SCORE_ENCODED=$(echo "$PYLINT_SCORE" | sed 's|/|%2F|g')
|
||||||
|
|
||||||
|
if [ -n "$PYLINT_SCORE_ENCODED" ]; then
|
||||||
|
sed -i "s|/badge/pylint-[^)]*|/badge/pylint-${PYLINT_SCORE_ENCODED}-brightgreen|" README.md
|
||||||
|
fi
|
||||||
|
if [ -n "$PYRIGHT_ERRORS" ]; then
|
||||||
|
sed -i "s|/badge/pyright-[^)]*|/badge/pyright-${PYRIGHT_ERRORS}%20errors-brightgreen|" README.md
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Updated badges:"
|
||||||
|
grep -E "pylint|pyright" README.md | head -2
|
||||||
|
|
||||||
|
- name: Commit and push badge updates
|
||||||
|
run: |
|
||||||
|
git config --local user.email "action@gitea.local"
|
||||||
|
git config --local user.name "Quality Badge Bot"
|
||||||
|
|
||||||
|
# Check if there are changes
|
||||||
|
if git diff --quiet README.md; then
|
||||||
|
echo "No badge changes needed"
|
||||||
|
else
|
||||||
|
echo "Badge changes detected, committing..."
|
||||||
|
git add README.md
|
||||||
|
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n\n'"[skip ci]"
|
||||||
|
git commit -m "$MSG"
|
||||||
|
git push
|
||||||
|
fi
|
||||||
@@ -0,0 +1,632 @@
|
|||||||
|
[MAIN]
|
||||||
|
|
||||||
|
# Analyse import fallback blocks. This can be used to support both Python 2 and
|
||||||
|
# 3 compatible code, which means that the block might have code that exists
|
||||||
|
# only in one or another interpreter, leading to false positives when analysed.
|
||||||
|
analyse-fallback-blocks=no
|
||||||
|
|
||||||
|
# Clear in-memory caches upon conclusion of linting. Useful if running pylint
|
||||||
|
# in a server-like mode.
|
||||||
|
clear-cache-post-run=no
|
||||||
|
|
||||||
|
# Load and enable all available extensions. Use --list-extensions to see a list
|
||||||
|
# all available extensions.
|
||||||
|
#enable-all-extensions=
|
||||||
|
|
||||||
|
# In error mode, messages with a category besides ERROR or FATAL are
|
||||||
|
# suppressed, and no reports are done by default. Error mode is compatible with
|
||||||
|
# disabling specific errors.
|
||||||
|
#errors-only=
|
||||||
|
|
||||||
|
# Always return a 0 (non-error) status code, even if lint errors are found.
|
||||||
|
# This is primarily useful in continuous integration scripts.
|
||||||
|
#exit-zero=
|
||||||
|
|
||||||
|
# A comma-separated list of package or module names from where C extensions may
|
||||||
|
# be loaded. Extensions are loading into the active Python interpreter and may
|
||||||
|
# run arbitrary code.
|
||||||
|
extension-pkg-allow-list=
|
||||||
|
|
||||||
|
# A comma-separated list of package or module names from where C extensions may
|
||||||
|
# be loaded. Extensions are loading into the active Python interpreter and may
|
||||||
|
# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
|
||||||
|
# for backward compatibility.)
|
||||||
|
extension-pkg-whitelist=
|
||||||
|
|
||||||
|
# Return non-zero exit code if any of these messages/categories are detected,
|
||||||
|
# even if score is above --fail-under value. Syntax same as enable. Messages
|
||||||
|
# specified are enabled, while categories only check already-enabled messages.
|
||||||
|
fail-on=
|
||||||
|
|
||||||
|
# Specify a score threshold under which the program will exit with error.
|
||||||
|
fail-under=10
|
||||||
|
|
||||||
|
# Interpret the stdin as a python script, whose filename needs to be passed as
|
||||||
|
# the module_or_package argument.
|
||||||
|
#from-stdin=
|
||||||
|
|
||||||
|
# Files or directories to be skipped. They should be base names, not paths.
|
||||||
|
ignore=CVS
|
||||||
|
|
||||||
|
# Add files or directories matching the regular expressions patterns to the
|
||||||
|
# ignore-list. The regex matches against paths and can be in Posix or Windows
|
||||||
|
# format. Because '\\' represents the directory delimiter on Windows systems,
|
||||||
|
# it can't be used as an escape character.
|
||||||
|
ignore-paths=
|
||||||
|
|
||||||
|
# Files or directories matching the regular expression patterns are skipped.
|
||||||
|
# The regex matches against base names, not paths. The default value ignores
|
||||||
|
# Emacs file locks
|
||||||
|
ignore-patterns=^\.#
|
||||||
|
|
||||||
|
# List of module names for which member attributes should not be checked and
|
||||||
|
# will not be imported (useful for modules/projects where namespaces are
|
||||||
|
# manipulated during runtime and thus existing member attributes cannot be
|
||||||
|
# deduced by static analysis). It supports qualified module names, as well as
|
||||||
|
# Unix pattern matching.
|
||||||
|
ignored-modules=
|
||||||
|
|
||||||
|
# Python code to execute, usually for sys.path manipulation such as
|
||||||
|
# pygtk.require().
|
||||||
|
#init-hook=
|
||||||
|
|
||||||
|
# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
|
||||||
|
# number of processors available to use, and will cap the count on Windows to
|
||||||
|
# avoid hangs.
|
||||||
|
jobs=1
|
||||||
|
|
||||||
|
# Control the amount of potential inferred values when inferring a single
|
||||||
|
# object. This can help the performance when dealing with large functions or
|
||||||
|
# complex, nested conditions.
|
||||||
|
limit-inference-results=100
|
||||||
|
|
||||||
|
# List of plugins (as comma separated values of python module names) to load,
|
||||||
|
# usually to register additional checkers.
|
||||||
|
load-plugins=
|
||||||
|
|
||||||
|
# Pickle collected data for later comparisons.
|
||||||
|
persistent=yes
|
||||||
|
|
||||||
|
# Resolve imports to .pyi stubs if available. May reduce no-member messages and
|
||||||
|
# increase not-an-iterable messages.
|
||||||
|
prefer-stubs=no
|
||||||
|
|
||||||
|
# Minimum Python version to use for version dependent checks. Will default to
|
||||||
|
# the version used to run pylint.
|
||||||
|
py-version=3.14
|
||||||
|
|
||||||
|
# Discover python modules and packages in the file system subtree.
|
||||||
|
recursive=no
|
||||||
|
|
||||||
|
# Add paths to the list of the source roots. Supports globbing patterns. The
|
||||||
|
# source root is an absolute path or a path relative to the current working
|
||||||
|
# directory used to determine a package namespace for modules located under the
|
||||||
|
# source root.
|
||||||
|
source-roots=
|
||||||
|
|
||||||
|
# Allow loading of arbitrary C extensions. Extensions are imported into the
|
||||||
|
# active Python interpreter and may run arbitrary code.
|
||||||
|
unsafe-load-any-extension=no
|
||||||
|
|
||||||
|
# In verbose mode, extra non-checker-related info will be displayed.
|
||||||
|
#verbose=
|
||||||
|
|
||||||
|
|
||||||
|
[BASIC]
|
||||||
|
|
||||||
|
# Naming style matching correct argument names.
|
||||||
|
argument-naming-style=snake_case
|
||||||
|
|
||||||
|
# Regular expression matching correct argument names. Overrides argument-
|
||||||
|
# naming-style. If left empty, argument names will be checked with the set
|
||||||
|
# naming style.
|
||||||
|
#argument-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct attribute names.
|
||||||
|
attr-naming-style=snake_case
|
||||||
|
|
||||||
|
# Regular expression matching correct attribute names. Overrides attr-naming-
|
||||||
|
# style. If left empty, attribute names will be checked with the set naming
|
||||||
|
# style.
|
||||||
|
#attr-rgx=
|
||||||
|
|
||||||
|
# Bad variable names which should always be refused, separated by a comma.
|
||||||
|
bad-names=foo,
|
||||||
|
bar,
|
||||||
|
baz,
|
||||||
|
toto,
|
||||||
|
tutu,
|
||||||
|
tata
|
||||||
|
|
||||||
|
# Bad variable names regexes, separated by a comma. If names match any regex,
|
||||||
|
# they will always be refused
|
||||||
|
bad-names-rgxs=
|
||||||
|
|
||||||
|
# Naming style matching correct class attribute names.
|
||||||
|
class-attribute-naming-style=any
|
||||||
|
|
||||||
|
# Regular expression matching correct class attribute names. Overrides class-
|
||||||
|
# attribute-naming-style. If left empty, class attribute names will be checked
|
||||||
|
# with the set naming style.
|
||||||
|
#class-attribute-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct class constant names.
|
||||||
|
class-const-naming-style=UPPER_CASE
|
||||||
|
|
||||||
|
# Regular expression matching correct class constant names. Overrides class-
|
||||||
|
# const-naming-style. If left empty, class constant names will be checked with
|
||||||
|
# the set naming style.
|
||||||
|
#class-const-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct class names.
|
||||||
|
class-naming-style=PascalCase
|
||||||
|
|
||||||
|
# Regular expression matching correct class names. Overrides class-naming-
|
||||||
|
# style. If left empty, class names will be checked with the set naming style.
|
||||||
|
#class-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct constant names.
|
||||||
|
const-naming-style=UPPER_CASE
|
||||||
|
|
||||||
|
# Regular expression matching correct constant names. Overrides const-naming-
|
||||||
|
# style. If left empty, constant names will be checked with the set naming
|
||||||
|
# style.
|
||||||
|
#const-rgx=
|
||||||
|
|
||||||
|
# Minimum line length for functions/classes that require docstrings, shorter
|
||||||
|
# ones are exempt.
|
||||||
|
docstring-min-length=-1
|
||||||
|
|
||||||
|
# Naming style matching correct function names.
|
||||||
|
function-naming-style=snake_case
|
||||||
|
|
||||||
|
# Regular expression matching correct function names. Overrides function-
|
||||||
|
# naming-style. If left empty, function names will be checked with the set
|
||||||
|
# naming style.
|
||||||
|
#function-rgx=
|
||||||
|
|
||||||
|
# Good variable names which should always be accepted, separated by a comma.
|
||||||
|
good-names=i,
|
||||||
|
j,
|
||||||
|
k,
|
||||||
|
ex,
|
||||||
|
Run,
|
||||||
|
_
|
||||||
|
|
||||||
|
# Good variable names regexes, separated by a comma. If names match any regex,
|
||||||
|
# they will always be accepted
|
||||||
|
good-names-rgxs=
|
||||||
|
|
||||||
|
# Include a hint for the correct naming format with invalid-name.
|
||||||
|
include-naming-hint=no
|
||||||
|
|
||||||
|
# Naming style matching correct inline iteration names.
|
||||||
|
inlinevar-naming-style=any
|
||||||
|
|
||||||
|
# Regular expression matching correct inline iteration names. Overrides
|
||||||
|
# inlinevar-naming-style. If left empty, inline iteration names will be checked
|
||||||
|
# with the set naming style.
|
||||||
|
#inlinevar-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct method names.
|
||||||
|
method-naming-style=snake_case
|
||||||
|
|
||||||
|
# Regular expression matching correct method names. Overrides method-naming-
|
||||||
|
# style. If left empty, method names will be checked with the set naming style.
|
||||||
|
#method-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct module names.
|
||||||
|
module-naming-style=snake_case
|
||||||
|
|
||||||
|
# Regular expression matching correct module names. Overrides module-naming-
|
||||||
|
# style. If left empty, module names will be checked with the set naming style.
|
||||||
|
#module-rgx=
|
||||||
|
|
||||||
|
# Colon-delimited sets of names that determine each other's naming style when
|
||||||
|
# the name regexes allow several styles.
|
||||||
|
name-group=
|
||||||
|
|
||||||
|
# Regular expression which should only match function or class names that do
|
||||||
|
# not require a docstring.
|
||||||
|
no-docstring-rgx=^_
|
||||||
|
|
||||||
|
# Regular expression matching correct parameter specification variable names.
|
||||||
|
# If left empty, parameter specification variable names will be checked with
|
||||||
|
# the set naming style.
|
||||||
|
#paramspec-rgx=
|
||||||
|
|
||||||
|
# List of decorators that produce properties, such as abc.abstractproperty. Add
|
||||||
|
# to this list to register other decorators that produce valid properties.
|
||||||
|
# These decorators are taken in consideration only for invalid-name.
|
||||||
|
property-classes=abc.abstractproperty
|
||||||
|
|
||||||
|
# Regular expression matching correct type alias names. If left empty, type
|
||||||
|
# alias names will be checked with the set naming style.
|
||||||
|
#typealias-rgx=
|
||||||
|
|
||||||
|
# Regular expression matching correct type variable names. If left empty, type
|
||||||
|
# variable names will be checked with the set naming style.
|
||||||
|
#typevar-rgx=
|
||||||
|
|
||||||
|
# Regular expression matching correct type variable tuple names. If left empty,
|
||||||
|
# type variable tuple names will be checked with the set naming style.
|
||||||
|
#typevartuple-rgx=
|
||||||
|
|
||||||
|
# Naming style matching correct variable names.
|
||||||
|
variable-naming-style=snake_case
|
||||||
|
|
||||||
|
# Regular expression matching correct variable names. Overrides variable-
|
||||||
|
# naming-style. If left empty, variable names will be checked with the set
|
||||||
|
# naming style.
|
||||||
|
#variable-rgx=
|
||||||
|
|
||||||
|
|
||||||
|
[CLASSES]
|
||||||
|
|
||||||
|
# Warn about protected attribute access inside special methods
|
||||||
|
check-protected-access-in-special-methods=no
|
||||||
|
|
||||||
|
# List of method names used to declare (i.e. assign) instance attributes.
|
||||||
|
defining-attr-methods=__init__,
|
||||||
|
__new__,
|
||||||
|
setUp,
|
||||||
|
asyncSetUp,
|
||||||
|
__post_init__
|
||||||
|
|
||||||
|
# List of member names, which should be excluded from the protected access
|
||||||
|
# warning.
|
||||||
|
exclude-protected=_asdict,_fields,_replace,_source,_make,os._exit
|
||||||
|
|
||||||
|
# List of valid names for the first argument in a class method.
|
||||||
|
valid-classmethod-first-arg=cls
|
||||||
|
|
||||||
|
# List of valid names for the first argument in a metaclass class method.
|
||||||
|
valid-metaclass-classmethod-first-arg=mcs
|
||||||
|
|
||||||
|
|
||||||
|
[DESIGN]
|
||||||
|
|
||||||
|
# List of regular expressions of class ancestor names to ignore when counting
|
||||||
|
# public methods (see R0903)
|
||||||
|
exclude-too-few-public-methods=
|
||||||
|
|
||||||
|
# List of qualified class names to ignore when counting class parents (see
|
||||||
|
# R0901)
|
||||||
|
ignored-parents=
|
||||||
|
|
||||||
|
# Maximum number of arguments for function / method.
|
||||||
|
max-args=5
|
||||||
|
|
||||||
|
# Maximum number of attributes for a class (see R0902).
|
||||||
|
max-attributes=7
|
||||||
|
|
||||||
|
# Maximum number of boolean expressions in an if statement (see R0916).
|
||||||
|
max-bool-expr=5
|
||||||
|
|
||||||
|
# Maximum number of branch for function / method body.
|
||||||
|
max-branches=12
|
||||||
|
|
||||||
|
# Maximum number of locals for function / method body.
|
||||||
|
max-locals=15
|
||||||
|
|
||||||
|
# Maximum number of parents for a class (see R0901).
|
||||||
|
max-parents=7
|
||||||
|
|
||||||
|
# Maximum number of positional arguments for function / method.
|
||||||
|
max-positional-arguments=5
|
||||||
|
|
||||||
|
# Maximum number of public methods for a class (see R0904).
|
||||||
|
max-public-methods=20
|
||||||
|
|
||||||
|
# Maximum number of return / yield for function / method body.
|
||||||
|
max-returns=6
|
||||||
|
|
||||||
|
# Maximum number of statements in function / method body.
|
||||||
|
max-statements=50
|
||||||
|
|
||||||
|
# Minimum number of public methods for a class (see R0903).
|
||||||
|
min-public-methods=2
|
||||||
|
|
||||||
|
|
||||||
|
[EXCEPTIONS]
|
||||||
|
|
||||||
|
# Exceptions that will emit a warning when caught.
|
||||||
|
overgeneral-exceptions=builtins.BaseException,builtins.Exception
|
||||||
|
|
||||||
|
|
||||||
|
[FORMAT]
|
||||||
|
|
||||||
|
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
|
||||||
|
expected-line-ending-format=
|
||||||
|
|
||||||
|
# Regexp for a line that is allowed to be longer than the limit.
|
||||||
|
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
|
||||||
|
|
||||||
|
# Number of spaces of indent required inside a hanging or continued line.
|
||||||
|
indent-after-paren=4
|
||||||
|
|
||||||
|
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
|
||||||
|
# tab).
|
||||||
|
indent-string=' '
|
||||||
|
|
||||||
|
# Maximum number of characters on a single line. Pylint's default of 100 is
|
||||||
|
# based on PEP 8's guidance that teams may choose line lengths up to 99
|
||||||
|
# characters.
|
||||||
|
max-line-length=100
|
||||||
|
|
||||||
|
# Maximum number of lines in a module.
|
||||||
|
max-module-lines=1000
|
||||||
|
|
||||||
|
# Allow the body of a class to be on the same line as the declaration if body
|
||||||
|
# contains single statement.
|
||||||
|
single-line-class-stmt=no
|
||||||
|
|
||||||
|
# Allow the body of an if to be on the same line as the test if there is no
|
||||||
|
# else.
|
||||||
|
single-line-if-stmt=no
|
||||||
|
|
||||||
|
|
||||||
|
[LOGGING]
|
||||||
|
|
||||||
|
# The type of string formatting that logging methods do. `old` means using %
|
||||||
|
# formatting, `new` is for `{}` formatting.
|
||||||
|
logging-format-style=old
|
||||||
|
|
||||||
|
# Logging modules to check that the string format arguments are in logging
|
||||||
|
# function parameter format.
|
||||||
|
logging-modules=logging
|
||||||
|
|
||||||
|
|
||||||
|
[MESSAGES CONTROL]
|
||||||
|
|
||||||
|
# Only show warnings with the listed confidence levels. Leave empty to show
|
||||||
|
# all. Valid levels: HIGH, CONTROL_FLOW, INFERENCE, INFERENCE_FAILURE,
|
||||||
|
# UNDEFINED.
|
||||||
|
confidence=HIGH,
|
||||||
|
CONTROL_FLOW,
|
||||||
|
INFERENCE,
|
||||||
|
INFERENCE_FAILURE,
|
||||||
|
UNDEFINED
|
||||||
|
|
||||||
|
# Disable the message, report, category or checker with the given id(s). You
|
||||||
|
# can either give multiple identifiers separated by comma (,) or put this
|
||||||
|
# option multiple times (only on the command line, not in the configuration
|
||||||
|
# file where it should appear only once). You can also use "--disable=all" to
|
||||||
|
# disable everything first and then re-enable specific checks. For example, if
|
||||||
|
# you want to run only the similarities checker, you can use "--disable=all
|
||||||
|
# --enable=similarities". If you want to run only the classes checker, but have
|
||||||
|
# no Warning level messages displayed, use "--disable=all --enable=classes
|
||||||
|
# --disable=W".
|
||||||
|
disable=raw-checker-failed,
|
||||||
|
bad-inline-option,
|
||||||
|
locally-disabled,
|
||||||
|
file-ignored,
|
||||||
|
suppressed-message,
|
||||||
|
useless-suppression,
|
||||||
|
deprecated-pragma,
|
||||||
|
use-symbolic-message-instead,
|
||||||
|
use-implicit-booleaness-not-comparison-to-string,
|
||||||
|
use-implicit-booleaness-not-comparison-to-zero,
|
||||||
|
missing-function-docstring,
|
||||||
|
missing-class-docstring,
|
||||||
|
missing-module-docstring,
|
||||||
|
invalid-name,
|
||||||
|
cyclic-import,
|
||||||
|
too-many-arguments,
|
||||||
|
too-many-locals,
|
||||||
|
too-many-branches,
|
||||||
|
too-many-statements,
|
||||||
|
too-many-instance-attributes,
|
||||||
|
duplicate-code,
|
||||||
|
import-outside-toplevel,
|
||||||
|
too-few-public-methods,
|
||||||
|
unnecessary-ellipsis
|
||||||
|
|
||||||
|
# Enable the message, report, category or checker with the given id(s). You can
|
||||||
|
# either give multiple identifier separated by comma (,) or put this option
|
||||||
|
# multiple time (only on the command line, not in the configuration file where
|
||||||
|
# it should appear only once). See also the "--disable" option for examples.
|
||||||
|
enable=
|
||||||
|
|
||||||
|
|
||||||
|
[METHOD_ARGS]
|
||||||
|
|
||||||
|
# List of qualified names (i.e., library.method) which require a timeout
|
||||||
|
# parameter e.g. 'requests.api.get,requests.api.post'
|
||||||
|
timeout-methods=requests.api.delete,requests.api.get,requests.api.head,requests.api.options,requests.api.patch,requests.api.post,requests.api.put,requests.api.request
|
||||||
|
|
||||||
|
|
||||||
|
[MISCELLANEOUS]
|
||||||
|
|
||||||
|
# Whether or not to search for fixme's in docstrings.
|
||||||
|
check-fixme-in-docstring=no
|
||||||
|
|
||||||
|
# List of note tags to take in consideration, separated by a comma.
|
||||||
|
notes=FIXME,
|
||||||
|
XXX,
|
||||||
|
TODO
|
||||||
|
|
||||||
|
# Regular expression of note tags to take in consideration.
|
||||||
|
notes-rgx=
|
||||||
|
|
||||||
|
|
||||||
|
[REFACTORING]
|
||||||
|
|
||||||
|
# Maximum number of nested blocks for function / method body
|
||||||
|
max-nested-blocks=5
|
||||||
|
|
||||||
|
# Complete name of functions that never returns. When checking for
|
||||||
|
# inconsistent-return-statements if a never returning function is called then
|
||||||
|
# it will be considered as an explicit return statement and no message will be
|
||||||
|
# printed.
|
||||||
|
never-returning-functions=sys.exit,argparse.parse_error
|
||||||
|
|
||||||
|
# Let 'consider-using-join' be raised when the separator to join on would be
|
||||||
|
# non-empty (resulting in expected fixes of the type: ``"- " + " -
|
||||||
|
# ".join(items)``)
|
||||||
|
suggest-join-with-non-empty-separator=yes
|
||||||
|
|
||||||
|
|
||||||
|
[REPORTS]
|
||||||
|
|
||||||
|
# Python expression which should return a score less than or equal to 10. You
|
||||||
|
# have access to the variables 'fatal', 'error', 'warning', 'refactor',
|
||||||
|
# 'convention', and 'info' which contain the number of messages in each
|
||||||
|
# category, as well as 'statement' which is the total number of statements
|
||||||
|
# analyzed. This score is used by the global evaluation report (RP0004).
|
||||||
|
evaluation=max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))
|
||||||
|
|
||||||
|
# Template used to display messages. This is a python new-style format string
|
||||||
|
# used to format the message information. See doc for all details.
|
||||||
|
msg-template=
|
||||||
|
|
||||||
|
# Set the output format. Available formats are: 'text', 'parseable',
|
||||||
|
# 'colorized', 'json2' (improved json format), 'json' (old json format), msvs
|
||||||
|
# (visual studio) and 'github' (GitHub actions). You can also give a reporter
|
||||||
|
# class, e.g. mypackage.mymodule.MyReporterClass.
|
||||||
|
#output-format=
|
||||||
|
|
||||||
|
# Tells whether to display a full report or only the messages.
|
||||||
|
reports=no
|
||||||
|
|
||||||
|
# Activate the evaluation score.
|
||||||
|
score=yes
|
||||||
|
|
||||||
|
|
||||||
|
[SIMILARITIES]
|
||||||
|
|
||||||
|
# Comments are removed from the similarity computation
|
||||||
|
ignore-comments=yes
|
||||||
|
|
||||||
|
# Docstrings are removed from the similarity computation
|
||||||
|
ignore-docstrings=yes
|
||||||
|
|
||||||
|
# Imports are removed from the similarity computation
|
||||||
|
ignore-imports=yes
|
||||||
|
|
||||||
|
# Signatures are removed from the similarity computation
|
||||||
|
ignore-signatures=yes
|
||||||
|
|
||||||
|
# Minimum lines number of a similarity.
|
||||||
|
min-similarity-lines=4
|
||||||
|
|
||||||
|
|
||||||
|
[SPELLING]
|
||||||
|
|
||||||
|
# Limits count of emitted suggestions for spelling mistakes.
|
||||||
|
max-spelling-suggestions=4
|
||||||
|
|
||||||
|
# Spelling dictionary name. No available dictionaries : You need to install
|
||||||
|
# both the python package and the system dependency for enchant to work.
|
||||||
|
spelling-dict=
|
||||||
|
|
||||||
|
# List of comma separated words that should be considered directives if they
|
||||||
|
# appear at the beginning of a comment and should not be checked.
|
||||||
|
spelling-ignore-comment-directives=fmt: on,fmt: off,noqa:,noqa,nosec,isort:skip,mypy:
|
||||||
|
|
||||||
|
# List of comma separated words that should not be checked.
|
||||||
|
spelling-ignore-words=
|
||||||
|
|
||||||
|
# A path to a file that contains the private dictionary; one word per line.
|
||||||
|
spelling-private-dict-file=
|
||||||
|
|
||||||
|
# Tells whether to store unknown words to the private dictionary (see the
|
||||||
|
# --spelling-private-dict-file option) instead of raising a message.
|
||||||
|
spelling-store-unknown-words=no
|
||||||
|
|
||||||
|
|
||||||
|
[STRING]
|
||||||
|
|
||||||
|
# This flag controls whether inconsistent-quotes generates a warning when the
|
||||||
|
# character used as a quote delimiter is used inconsistently within a module.
|
||||||
|
check-quote-consistency=no
|
||||||
|
|
||||||
|
# This flag controls whether the implicit-str-concat should generate a warning
|
||||||
|
# on implicit string concatenation in sequences defined over several lines.
|
||||||
|
check-str-concat-over-line-jumps=no
|
||||||
|
|
||||||
|
|
||||||
|
[TYPECHECK]
|
||||||
|
|
||||||
|
# List of decorators that produce context managers, such as
|
||||||
|
# contextlib.contextmanager. Add to this list to register other decorators that
|
||||||
|
# produce valid context managers.
|
||||||
|
contextmanager-decorators=contextlib.contextmanager
|
||||||
|
|
||||||
|
# List of members which are set dynamically and missed by pylint inference
|
||||||
|
# system, and so shouldn't trigger E1101 when accessed. Python regular
|
||||||
|
# expressions are accepted.
|
||||||
|
generated-members=
|
||||||
|
|
||||||
|
# Tells whether to warn about missing members when the owner of the attribute
|
||||||
|
# is inferred to be None.
|
||||||
|
ignore-none=yes
|
||||||
|
|
||||||
|
# This flag controls whether pylint should warn about no-member and similar
|
||||||
|
# checks whenever an opaque object is returned when inferring. The inference
|
||||||
|
# can return multiple potential results while evaluating a Python object, but
|
||||||
|
# some branches might not be evaluated, which results in partial inference. In
|
||||||
|
# that case, it might be useful to still emit no-member and other checks for
|
||||||
|
# the rest of the inferred objects.
|
||||||
|
ignore-on-opaque-inference=yes
|
||||||
|
|
||||||
|
# List of symbolic message names to ignore for Mixin members.
|
||||||
|
ignored-checks-for-mixins=no-member,
|
||||||
|
not-async-context-manager,
|
||||||
|
not-context-manager,
|
||||||
|
attribute-defined-outside-init
|
||||||
|
|
||||||
|
# List of class names for which member attributes should not be checked (useful
|
||||||
|
# for classes with dynamically set attributes). This supports the use of
|
||||||
|
# qualified names.
|
||||||
|
ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
|
||||||
|
|
||||||
|
# Show a hint with possible names when a member name was not found. The aspect
|
||||||
|
# of finding the hint is based on edit distance.
|
||||||
|
missing-member-hint=yes
|
||||||
|
|
||||||
|
# The maximum edit distance a name should have in order to be considered a
|
||||||
|
# similar match for a missing member name.
|
||||||
|
missing-member-hint-distance=1
|
||||||
|
|
||||||
|
# The total number of similar names that should be taken in consideration when
|
||||||
|
# showing a hint for a missing member.
|
||||||
|
missing-member-max-choices=1
|
||||||
|
|
||||||
|
# Regex pattern to define which classes are considered mixins.
|
||||||
|
mixin-class-rgx=.*[Mm]ixin
|
||||||
|
|
||||||
|
# List of decorators that change the signature of a decorated function.
|
||||||
|
signature-mutators=
|
||||||
|
|
||||||
|
|
||||||
|
[VARIABLES]
|
||||||
|
|
||||||
|
# List of additional names supposed to be defined in builtins. Remember that
|
||||||
|
# you should avoid defining new builtins when possible.
|
||||||
|
additional-builtins=
|
||||||
|
|
||||||
|
# Tells whether unused global variables should be treated as a violation.
|
||||||
|
allow-global-unused-variables=yes
|
||||||
|
|
||||||
|
# List of names allowed to shadow builtins
|
||||||
|
allowed-redefined-builtins=
|
||||||
|
|
||||||
|
# List of strings which can identify a callback function by name. A callback
|
||||||
|
# name must start or end with one of those strings.
|
||||||
|
callbacks=cb_,
|
||||||
|
_cb
|
||||||
|
|
||||||
|
# A regular expression matching the name of dummy variables (i.e. expected to
|
||||||
|
# not be used).
|
||||||
|
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
|
||||||
|
|
||||||
|
# Argument names that match this expression will be ignored.
|
||||||
|
ignored-argument-names=_.*|^ignored_|^unused_
|
||||||
|
|
||||||
|
# Tells whether we should check for unused import in __init__ files.
|
||||||
|
init-import=no
|
||||||
|
|
||||||
|
# List of qualified module names which can have objects that can redefine
|
||||||
|
# builtins.
|
||||||
|
redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io
|
||||||
@@ -25,9 +25,8 @@ the container lifecycle and the copying of skills and env vars into it.
|
|||||||
- `README.md` — short public-facing description.
|
- `README.md` — short public-facing description.
|
||||||
- `AGENTS.md` — this file, orientation for future agent sessions.
|
- `AGENTS.md` — this file, orientation for future agent sessions.
|
||||||
- `.gitignore` — OS junk.
|
- `.gitignore` — OS junk.
|
||||||
- `bot-bottle.json` — legacy manifest of named agents (env / skills / prompt
|
- `.bot-bottle/` — per-repo agent and bottle manifests (YAML markdown format).
|
||||||
per agent), consumed by `cli.py`. See "Manifest" under
|
- `examples/` — example bottles and agents showing the manifest format.
|
||||||
"Intended design".
|
|
||||||
- `docs/README.md` — docs overview; when to write which document.
|
- `docs/README.md` — docs overview; when to write which document.
|
||||||
- `docs/prds/` — product requirement docs (see `docs/prds/README.md` for format).
|
- `docs/prds/` — product requirement docs (see `docs/prds/README.md` for format).
|
||||||
- `docs/research/` — research notes (see `docs/research/README.md`).
|
- `docs/research/` — research notes (see `docs/research/README.md`).
|
||||||
@@ -37,10 +36,11 @@ the container lifecycle and the copying of skills and env vars into it.
|
|||||||
|
|
||||||
- Three kinds of doc, each with its own conventions in-folder; see
|
- Three kinds of doc, each with its own conventions in-folder; see
|
||||||
`docs/README.md` for when to write which:
|
`docs/README.md` for when to write which:
|
||||||
- **PRDs** (`docs/prds/`) — one feature per file, numbered
|
- **PRDs** (`docs/prds/`) — one feature per file. While a PR is open
|
||||||
`NNNN-kebab.md`. A `Status:` line tracks lifecycle: Draft → Active
|
the file is named `prd-new-<kebab>.md`; CI assigns a sequential
|
||||||
(shipped to `main`) → Superseded/Retargeted. Format in
|
number on merge to `main` and renames it. A `Status:` line tracks
|
||||||
`docs/prds/README.md`.
|
lifecycle: Draft → Active (shipped to `main`) →
|
||||||
|
Superseded/Retargeted. Format in `docs/prds/README.md`.
|
||||||
- **Research notes** (`docs/research/`) — opinionated investigations;
|
- **Research notes** (`docs/research/`) — opinionated investigations;
|
||||||
unnumbered kebab-case, freeform and verdict-first. See
|
unnumbered kebab-case, freeform and verdict-first. See
|
||||||
`docs/research/README.md`.
|
`docs/research/README.md`.
|
||||||
|
|||||||
+13
-7
@@ -16,14 +16,20 @@ FROM node:22-slim
|
|||||||
# features (status checks, commits, PR creation) — without git in the
|
# features (status checks, commits, PR creation) — without git in the
|
||||||
# image, those features fail in surprising ways once the user does any
|
# image, those features fail in surprising ways once the user does any
|
||||||
# real work. ca-certificates is already in the slim base; listed for
|
# real work. ca-certificates is already in the slim base; listed for
|
||||||
# clarity in case the base ever drops it. socat is the privileged
|
# clarity in case the base ever drops it. curl is here so any
|
||||||
# forwarder for the in-container ssh-agent (see bot_bottle/ssh.py): the agent
|
# HTTPS_PROXY-aware tool (curl itself, plus anything that shells out
|
||||||
# runs as root and rejects non-root connections, so socat sits between
|
# to it) works against egress's bumped TLS without the agent needing
|
||||||
# node and the agent socket. curl is here so any HTTPS_PROXY-aware
|
# local DNS.
|
||||||
# tool (curl itself, plus anything that shells out to it) works
|
|
||||||
# against pipelock's bumped TLS without the agent needing local DNS.
|
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
&& apt-get install -y --no-install-recommends git ca-certificates curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# App-specific deps. Python isn't required by claude-code itself
|
||||||
|
# (claude-code is a Node CLI), but is convenient for the agent to
|
||||||
|
# shell out to for ad-hoc scripts. Kept on its own layer so it can
|
||||||
|
# be moved to a downstream image if the base ever needs to shrink.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Install claude-code globally. Pinned to the version verified in the v1
|
# Install claude-code globally. Pinned to the version verified in the v1
|
||||||
|
|||||||
+9
-1
@@ -6,7 +6,15 @@
|
|||||||
FROM node:22-slim
|
FROM node:22-slim
|
||||||
|
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
&& apt-get install -y --no-install-recommends git ca-certificates curl \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# App-specific deps. Python isn't required by codex itself
|
||||||
|
# (codex is a Node CLI), but is convenient for the agent to shell
|
||||||
|
# out to for ad-hoc scripts. Kept on its own layer so it can be
|
||||||
|
# moved to a downstream image if the base ever needs to shrink.
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
|
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
|
||||||
|
|||||||
+12
-26
@@ -1,23 +1,18 @@
|
|||||||
# Per-bottle sidecar bundle image (PRD 0024).
|
# Per-bottle sidecar bundle image (PRD 0024).
|
||||||
#
|
#
|
||||||
# Collapses the four prior per-sidecar images (pipelock, egress,
|
# Collapses the prior per-sidecar images (egress, git-gate,
|
||||||
# git-gate, supervise) into one. A small stdlib-Python init
|
# supervise) into one. A small stdlib-Python init supervisor at
|
||||||
# supervisor at /app/sidecar_init.py spawns all four daemons,
|
# /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
|
||||||
# forwards SIGTERM, and propagates per-daemon stdout/stderr to the
|
# propagates per-daemon stdout/stderr to the container log with a
|
||||||
# container log with a `[name]` prefix. See PRD 0024 for the
|
# `[name]` prefix. See PRD 0024 for the rationale.
|
||||||
# rationale.
|
|
||||||
#
|
#
|
||||||
# Layout (preserved verbatim from the prior four Dockerfiles so the
|
# Layout:
|
||||||
# compose renderer's bind-mount paths and docker-cp targets keep
|
|
||||||
# working):
|
|
||||||
#
|
#
|
||||||
# /usr/local/bin/pipelock pipelock binary
|
|
||||||
# /usr/bin/gitleaks gitleaks binary
|
# /usr/bin/gitleaks gitleaks binary
|
||||||
# /app/egress_addon.py + siblings mitmproxy addon (egress)
|
# /app/egress_addon.py + siblings mitmproxy addon (egress)
|
||||||
# /app/egress-entrypoint.sh mitmdump launcher
|
# /app/egress-entrypoint.sh mitmdump launcher
|
||||||
# /app/supervise_server.py + .py supervise MCP server
|
# /app/supervise_server.py + .py supervise MCP server
|
||||||
# /app/sidecar_init.py PID 1 supervisor
|
# /app/sidecar_init.py PID 1 supervisor
|
||||||
# /etc/pipelock.yaml bind-mounted at run time
|
|
||||||
# /etc/egress/routes.yaml bind-mounted at run time
|
# /etc/egress/routes.yaml bind-mounted at run time
|
||||||
# /etc/git-gate/pre-receive docker-cp'd at start time
|
# /etc/git-gate/pre-receive docker-cp'd at start time
|
||||||
# /git-gate-entrypoint.sh docker-cp'd at start time
|
# /git-gate-entrypoint.sh docker-cp'd at start time
|
||||||
@@ -27,25 +22,17 @@
|
|||||||
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
|
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
|
||||||
#
|
#
|
||||||
# Exposed ports inside the container:
|
# Exposed ports inside the container:
|
||||||
# 8888 pipelock (HTTPS_PROXY)
|
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
|
||||||
# 9099 egress (mitmproxy, pipelock's upstream — not externally
|
|
||||||
# addressed by the agent)
|
|
||||||
# 9418 git-gate (git-daemon)
|
# 9418 git-gate (git-daemon)
|
||||||
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
||||||
# 9100 supervise (MCP HTTP)
|
# 9100 supervise (MCP HTTP)
|
||||||
|
|
||||||
# Stage 1: pipelock binary. The upstream pipelock image is a
|
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
|
||||||
# scratch image with the binary at /pipelock (entrypoint).
|
|
||||||
# Pinned by digest in lockstep with
|
|
||||||
# bot_bottle/backend/docker/pipelock.py:PIPELOCK_IMAGE.
|
|
||||||
FROM ghcr.io/luckypipewrench/pipelock@sha256:3b1a39417b98406ddc5dc2d8fcb42865ddc0c68a43d355db55f0f8cb06bc6de9 AS pipelock-src
|
|
||||||
|
|
||||||
# Stage 2: gitleaks binary. The upstream gitleaks image is alpine
|
|
||||||
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
|
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
|
||||||
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
|
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
|
||||||
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
|
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
|
||||||
|
|
||||||
# Stage 3: assembly. mitmproxy/mitmproxy is debian-slim-based with
|
# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
|
||||||
# Python + mitmdump pre-installed — heavier than the others, so
|
# Python + mitmdump pre-installed — heavier than the others, so
|
||||||
# this stage starts there and pulls the standalone binaries in.
|
# this stage starts there and pulls the standalone binaries in.
|
||||||
FROM mitmproxy/mitmproxy:11.1.3
|
FROM mitmproxy/mitmproxy:11.1.3
|
||||||
@@ -60,16 +47,14 @@ USER root
|
|||||||
# plus the core `git` binary the pre-receive hook invokes.
|
# plus the core `git` binary the pre-receive hook invokes.
|
||||||
# openssh-client supplies the upstream SSH transport the
|
# openssh-client supplies the upstream SSH transport the
|
||||||
# pre-receive hook uses to forward accepted refs.
|
# pre-receive hook uses to forward accepted refs.
|
||||||
# ca-certificates is needed for both pipelock and mitmdump
|
# ca-certificates is needed for mitmdump upstream TLS (the
|
||||||
# upstream TLS (the base image already has it; listed for
|
# base image already has it; listed for explicitness).
|
||||||
# explicitness).
|
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends \
|
&& apt-get install -y --no-install-recommends \
|
||||||
git openssh-client ca-certificates \
|
git openssh-client ca-certificates \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Pull the standalone binaries into the final image.
|
# Pull the standalone binaries into the final image.
|
||||||
COPY --from=pipelock-src /pipelock /usr/local/bin/pipelock
|
|
||||||
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
|
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
|
||||||
|
|
||||||
# Project Python: addon + server modules + the init supervisor.
|
# Project Python: addon + server modules + the init supervisor.
|
||||||
@@ -78,6 +63,7 @@ COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
|
|||||||
# Dockerfile.egress / Dockerfile.supervise layout.
|
# Dockerfile.egress / Dockerfile.supervise layout.
|
||||||
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
|
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
|
||||||
COPY bot_bottle/egress_addon.py /app/egress_addon.py
|
COPY bot_bottle/egress_addon.py /app/egress_addon.py
|
||||||
|
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
|
||||||
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
||||||
COPY bot_bottle/supervise.py /app/supervise.py
|
COPY bot_bottle/supervise.py /app/supervise.py
|
||||||
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
||||||
|
|||||||
@@ -5,97 +5,29 @@
|
|||||||
# bot-bottle
|
# bot-bottle
|
||||||
|
|
||||||
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
||||||
|
[](https://github.com/PyCQA/pylint)
|
||||||
|
[](https://github.com/microsoft/pyright)
|
||||||
|
|
||||||
Run multiple Claude Code agents on your own machine, each scoped to its own secrets, skills, and egress allowlist.
|
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
|
||||||
|
|
||||||

|
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
|
||||||
|
|
||||||
Four prompts to the agent inside a real bottle:
|
## Features
|
||||||
claude replies to `hello there` — proof api.anthropic.com routes
|
|
||||||
through pipelock's bumped TLS end-to-end;
|
|
||||||
asked to GET a non-allowlisted host, the agent's curl gets 403 back
|
|
||||||
from pipelock;
|
|
||||||
asked to POST a credential-shaped body to an allowlisted host, the
|
|
||||||
same 403 — pipelock's DLP body scanner caught it;
|
|
||||||
asked to commit and push an AKIA-shaped key, git-gate's gitleaks
|
|
||||||
pre-receive hook rejects the ref.
|
|
||||||
Run it yourself with `bash scripts/demo.sh`.
|
|
||||||
|
|
||||||
## Why "bot-bottle"?
|
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default.
|
||||||
|
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
||||||
Each container is a bottle; Claude is the genie inside. The genie's
|
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
||||||
powers are exactly what the manifest grants it — a specific set of
|
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
||||||
skills, a specific set of secrets, and a specific set of hosts it can
|
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
||||||
reach — nothing more. You uncork one bottle per agent
|
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
|
||||||
(`./cli.py start <agent>`), many bottles run in parallel, and each is
|
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other.
|
||||||
scoped to its task. When the session ends the bottle is destroyed and
|
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||||
the genie does not persist.
|
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||||
|
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker.
|
||||||
## Goals
|
|
||||||
|
|
||||||
- Scope each agent to the minimum credentials and network egress its task actually needs
|
|
||||||
- Run multiple agents in parallel, isolated from each other
|
|
||||||
- Keep code, credentials, and agent activity on infrastructure I control — no third-party agent runtime
|
|
||||||
|
|
||||||
## Project status
|
|
||||||
|
|
||||||
bot-bottle is a self-hosted secure runtime for AI coding agents.
|
|
||||||
Each agent runs in an isolated container or micro-VM-backed bottle with
|
|
||||||
scoped secrets, allowlisted egress, TLS-aware proxying, DLP checks, and
|
|
||||||
a git-gate that withholds upstream credentials and scans pushes before
|
|
||||||
forwarding. The project includes a documented threat model, PRD-driven
|
|
||||||
development history, Docker and smolmachines backends, dashboard and
|
|
||||||
remediation flows, and unit/integration tests covering exfiltration and
|
|
||||||
sandbox escape scenarios.
|
|
||||||
|
|
||||||
## Security model
|
|
||||||
|
|
||||||
Each agent runs in its own bottle: its own container, its own internal
|
|
||||||
Docker network, and its own pipelock sidecar. Bottles don't share
|
|
||||||
state, don't talk to each other, and only get the env vars, skills,
|
|
||||||
SSH identities, and egress hosts the manifest grants them — nothing
|
|
||||||
more. Any one agent only has the access it needs to do its job.
|
|
||||||
|
|
||||||
The bottle limits both what an agent can see and where it can send
|
|
||||||
it. Each bottle gets only the secrets and SSH identities the manifest
|
|
||||||
grants it — a Gitea token but not a GitHub token, a deploy key but
|
|
||||||
not a personal SSH key — so even a compromised or misbehaving agent
|
|
||||||
only handles credentials it was already trusted with for its job.
|
|
||||||
Egress flows through pipelock, which constrains where those
|
|
||||||
credentials can travel: an agent with a Gitea token can reach
|
|
||||||
`gitea.dideric.is`, not arbitrary attacker-controlled hosts. The same
|
|
||||||
constraint blocks DNS-over-HTTPS as an exfil channel — a DoH resolver
|
|
||||||
like `cloudflare-dns.com` would have to be on the allowlist for the
|
|
||||||
agent to reach it at all. The container itself adds a layer between
|
|
||||||
the agent and the host, but the v1 design leans more on secret
|
|
||||||
minimization and egress allowlisting than on the container as a
|
|
||||||
hardened boundary. On Linux hosts where [gVisor](https://gvisor.dev/)
|
|
||||||
is registered with Docker, bot-bottle auto-detects it and launches
|
|
||||||
every bottle under `runsc` for a userspace syscall barrier — no
|
|
||||||
manifest configuration required. The broader v2 discussion lives in
|
|
||||||
`docs/research/stronger-isolation-alternatives.md`.
|
|
||||||
|
|
||||||
The egress proxy and OAuth-token handling below are the load-bearing
|
|
||||||
pieces of v1.
|
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
A bottle is two containers per agent: an `agent` container, and a
|
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles egress + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||||
`sidecars` container that bundles pipelock + egress + git-gate +
|
|
||||||
supervise behind a Python init supervisor (PRD 0024). They share a
|
|
||||||
per-agent Docker `--internal` network; the agent has no default
|
|
||||||
route off-box. All HTTP and HTTPS egress funnels through pipelock,
|
|
||||||
where the egress allowlist, TLS interception, and request-body DLP
|
|
||||||
scanner enforce the manifest before any byte leaves the host. The
|
|
||||||
only egress that doesn't traverse pipelock is git-gate's SSH
|
|
||||||
push/fetch to `bottle.git` upstreams — pipelock can't proxy SSH,
|
|
||||||
so git-gate is its own L4-style egress path with gitleaks doing
|
|
||||||
the pre-receive scan.
|
|
||||||
|
|
||||||
The agent dials the bundle by the legacy short names (`pipelock`,
|
|
||||||
`egress`, `git-gate`, `supervise`); the renderer registers those as
|
|
||||||
docker-network aliases on the bundle so existing HTTPS_PROXY URLs
|
|
||||||
and MCP endpoints resolve without an agent-side change.
|
|
||||||
|
|
||||||
```
|
```
|
||||||
host ( ./cli.py )
|
host ( ./cli.py )
|
||||||
@@ -104,225 +36,47 @@ and MCP endpoints resolve without an agent-side change.
|
|||||||
▼
|
▼
|
||||||
┌─────────────────────────── bottle ──────────────────────────────────┐
|
┌─────────────────────────── bottle ──────────────────────────────────┐
|
||||||
│ │
|
│ │
|
||||||
│ ┌──────────────────┐ │
|
│ ┌──────────────────┐ ┌──────────────────────┐ │
|
||||||
│ │ agent image │ HTTPS_PROXY │
|
│ │ agent image │ HTTP(S) proxy │ egress image │ │
|
||||||
│ │ (claude-code, │ ────────────────────────┐ │
|
│ │ (claude-code, │ ─────────────────►│ (mitmproxy; TLS bump │ │ HTTPS to
|
||||||
│ │ built locally) │ │ │
|
│ │ codex, etc) │ │ DLP scan, path │───┼──► allowlisted
|
||||||
│ │ │ plain HTTP │ │
|
│ │ │ │ matching, auth │ │ hosts
|
||||||
│ │ skills, env, │ (token injection) ┌────▼─────────┐ │
|
│ │ environ: proxy │ │ injection) │ │
|
||||||
│ │ ~/.gitconfig, │ ──────────────────►│ cred-proxy │ │
|
│ │ URLs only, no │ └──────────────────────┘ │
|
||||||
│ │ ~/.npmrc, tea │ │ (strips/inj │ │
|
│ │ real tokens │ │
|
||||||
│ │ │ │ Authoriz.) │ │
|
│ │ │ git proxy ┌────────────────┐ │ SSH push/fetch
|
||||||
│ │ environ: URLs │ └─────┬────────┘ │
|
|
||||||
│ │ only, no real │ HTTPS_PROXY │ │
|
|
||||||
│ │ tokens │ ▼ │
|
|
||||||
│ │ │ ┌────────────────┐ │ HTTPS to
|
|
||||||
│ │ │ │ pipelock image │──────────┼──► allowlisted
|
|
||||||
│ │ │ │ (TLS bump, DLP │ │ hosts (incl.
|
|
||||||
│ │ │ │ body scan, │ │ cred-proxy
|
|
||||||
│ │ │ │ allowlist) │ │ upstreams)
|
|
||||||
│ │ │ └────────────────┘ │
|
|
||||||
│ │ │ │
|
|
||||||
│ │ │ git:// ┌────────────────┐ │ SSH push/fetch
|
|
||||||
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
|
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
|
||||||
│ │ │ │ (gitleaks + │ │ upstreams
|
│ │ │ │ (gitleaks + │ │ upstreams
|
||||||
│ └──────────────────┘ │ git daemon) │ │ (direct — not
|
│ └──────────────────┘ │ git daemon) │ │ (direct — not
|
||||||
│ └────────────────┘ │ via pipelock)
|
│ └────────────────┘ │ via egress)
|
||||||
│ │
|
│ │
|
||||||
│ agent on internal network (no default route); pipelock, │
|
│ agent on internal network (no default route); egress and │
|
||||||
│ cred-proxy, and git-gate straddle internal + egress networks. │
|
│ git-gate straddle internal + egress networks. │
|
||||||
│ pipelock is the single HTTP/HTTPS chokepoint — cred-proxy's │
|
│ egress is the single HTTP/HTTPS chokepoint — all agent HTTP/HTTPS │
|
||||||
│ outbound traverses it too. git-gate's SSH egress is direct │
|
│ traffic flows through it. git-gate's SSH egress is direct │
|
||||||
│ because pipelock is HTTP-only. │
|
│ because egress is HTTP-only. │
|
||||||
└─────────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
- **agent image** — built from the provider template Dockerfile
|
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
|
||||||
(`Dockerfile.claude` for Claude, `Dockerfile.codex` for Codex, or
|
|
||||||
`agent_provider.dockerfile`) on first run; runs the selected agent
|
|
||||||
CLI with the manifest-granted skills, env vars, and `~/.gitconfig`
|
|
||||||
(the latter for the git-gate's `insteadOf` rules when `bottle.git`
|
|
||||||
is set).
|
|
||||||
- **pipelock image** — per-agent sidecar. Terminates the agent's
|
|
||||||
outbound HTTP/HTTPS, enforces the resolved allowlist, runs DLP
|
|
||||||
scanning. Design in `docs/prds/0001-per-agent-egress-proxy-via-pipelock.md`
|
|
||||||
and `docs/prds/0006-pipelock-tls-interception.md`.
|
|
||||||
- **git-gate image** — per-agent sidecar built on `zricethezav/gitleaks`
|
|
||||||
(alpine + gitleaks + git-daemon + openssh-client). Runs
|
|
||||||
`git daemon` over `git://` as a bidirectional mirror of each
|
|
||||||
declared upstream. A pre-receive hook gitleaks-scans incoming
|
|
||||||
refs and forwards clean refs to the real upstream over SSH; an
|
|
||||||
access-hook runs `git fetch origin --prune` against the upstream
|
|
||||||
before every upload-pack so an agent fetch returns whatever the
|
|
||||||
upstream has *now* (fail-closed if unreachable). The agent's
|
|
||||||
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
|
|
||||||
so push, fetch, clone, and pull all route through. The agent
|
|
||||||
never sees the upstream credential. Brought up only when
|
|
||||||
`bottle.git` has entries. Design in `docs/prds/0008-git-gate.md`.
|
|
||||||
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
|
|
||||||
base, stdlib-only) that holds API tokens declared in
|
|
||||||
`bottle.cred_proxy.routes`. Each route names a `path`,
|
|
||||||
`upstream`, `auth_scheme`, and `token_ref` (host env var); the
|
|
||||||
agent dials `http://cred-proxy:9099<path>...` over plain HTTP
|
|
||||||
and the proxy strips any inbound `Authorization`, injects
|
|
||||||
`<auth_scheme> <token>` using the value held only in its own
|
|
||||||
container's environ, and forwards to the real upstream over
|
|
||||||
HTTPS. SSE responses stream back unbuffered. The cred-proxy's
|
|
||||||
outbound HTTPS routes through pipelock (it trusts pipelock's
|
|
||||||
per-bottle CA), so pipelock's egress allowlist + body scanner
|
|
||||||
apply to cred-proxy traffic the same way they apply to direct
|
|
||||||
agent traffic. Smart-HTTP push paths (`/git-receive-pack`,
|
|
||||||
`/info/refs?service=git-receive-pack`) are refused at the
|
|
||||||
proxy — push must go through `bottle.git` / git-gate where
|
|
||||||
gitleaks runs. Optional per-route `role` tags drive agent-side
|
|
||||||
rewrites: `anthropic-base-url`, `npm-registry`, `git-insteadof`,
|
|
||||||
`tea-login`. The agent's `printenv` shows only proxy URLs —
|
|
||||||
none of the real token values. Design in
|
|
||||||
`docs/prds/0010-cred-proxy.md`.
|
|
||||||
|
|
||||||
When the agent exits, `cli.py` tears down every sidecar that was
|
|
||||||
brought up and the two networks; nothing about a bottle persists
|
|
||||||
between runs.
|
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
Requires Docker on the host and a long-lived Claude Code OAuth token in
|
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||||
your shell env.
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||||
```
|
```
|
||||||
|
|
||||||
The container is removed automatically when the session ends. If the script
|
|
||||||
is killed with SIGKILL the exit trap won't fire and the container may be
|
|
||||||
left running; remove it with `docker rm -f <container-name>`.
|
|
||||||
|
|
||||||
### Smolmachines backend (experimental, macOS-only)
|
|
||||||
|
|
||||||
A second backend runs the agent in a smolvm micro-VM (libkrun) with the
|
|
||||||
sidecar bundle still in Docker. Selected via
|
|
||||||
`BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>`. Requires
|
|
||||||
`smolvm` on PATH (`curl -sSL https://smolmachines.com/install.sh | sh`).
|
|
||||||
|
|
||||||
The integration tests run against whichever backend the env var
|
|
||||||
selects and skip cleanly when its prerequisites are missing.
|
|
||||||
|
|
||||||
**One-time sudo on first launch (macOS):** smolmachines bottles
|
|
||||||
each reserve a loopback alias from a pool (`127.0.0.16` ..
|
|
||||||
`127.0.0.31`) and bind their bundle's port-forwards to it; the
|
|
||||||
first `./cli.py start` after each reboot prompts for sudo to add
|
|
||||||
missing aliases via `ifconfig lo0 alias`. Aliases persist until
|
|
||||||
reboot; subsequent launches don't prompt. The agent's TSI
|
|
||||||
allowlist is the alias's `/32`, so each bottle can only reach
|
|
||||||
its own bundle's published ports — not other bottles' ports,
|
|
||||||
not other host loopback services (postgres, dev servers, etc.).
|
|
||||||
|
|
||||||
This enforcement requires a workaround for a smolvm 0.8.0 bug:
|
|
||||||
the CLI's `--allow-cidr` flag is silently dropped when combined
|
|
||||||
with `--from <smolmachine>`. The launcher patches smolvm's
|
|
||||||
persistent state DB
|
|
||||||
(`~/Library/Application Support/smolvm/server/smolvm.db`)
|
|
||||||
directly between `machine create` and `machine start` to set
|
|
||||||
the allowlist. The hack falls away automatically when smolvm
|
|
||||||
honors the flag upstream — see the `loopback_alias` module's
|
|
||||||
docstring for the investigation trail.
|
|
||||||
|
|
||||||
## Manifest
|
## Manifest
|
||||||
|
|
||||||
Bottles and agents live as Markdown files with YAML frontmatter under
|
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Bottles live in `~/.bot-bottle/bottles/`; agents may also be shipped by a repo at `<repo>/.bot-bottle/agents/<name>.md`.
|
||||||
`~/.bot-bottle/`. Each bottle is one file in `bottles/`, each agent
|
|
||||||
is one file in `agents/`:
|
|
||||||
|
|
||||||
```
|
**Bottle** (`~/.bot-bottle/bottles/gitea-dev.md`):
|
||||||
~/.bot-bottle/
|
|
||||||
├── bottles/
|
|
||||||
│ ├── dev.md
|
|
||||||
│ └── gitea-dev.md
|
|
||||||
└── agents/
|
|
||||||
├── implementer.md
|
|
||||||
└── researcher.md
|
|
||||||
```
|
|
||||||
|
|
||||||
The filename (without `.md`) is the entity's name. Filenames must
|
|
||||||
match `[a-z][a-z0-9-]*`; files that don't are skipped with a warning.
|
|
||||||
|
|
||||||
A repo can ship its own agent files alongside its code at
|
|
||||||
`<repo>/.bot-bottle/agents/<name>.md`. Those agents reference
|
|
||||||
bottles defined in `~/.bot-bottle/bottles/` (the only place
|
|
||||||
bottles can come from); a `bottles/` subdir in a repo is ignored
|
|
||||||
with a warning. **This is the trust boundary**: bottle infrastructure
|
|
||||||
— credentials, egress allowlists, git remotes — comes from your home
|
|
||||||
directory only. A cloned repo cannot redirect a host env var to an
|
|
||||||
attacker-named upstream because it has no way to declare a bottle.
|
|
||||||
|
|
||||||
### Bottle composition with `extends:`
|
|
||||||
|
|
||||||
A bottle can inherit from another via `extends: <bottle-name>` so
|
|
||||||
operators don't have to duplicate a whole bottle file to vary one
|
|
||||||
field (PRD 0025). The parent's resolved config is the base; the
|
|
||||||
child's declared fields overlay. Merge rules:
|
|
||||||
|
|
||||||
- `env:` — dict merge, child wins on key collision.
|
|
||||||
- `git.user:` — per-field overlay (child's non-empty `name` /
|
|
||||||
`email` wins; empty falls through to parent).
|
|
||||||
- `git.remotes:` — dict merge by host, child wins on host collision.
|
|
||||||
An explicit `git.remotes: {}` clears the parent's remotes; omitting
|
|
||||||
`git.remotes` inherits the parent's remotes.
|
|
||||||
- `agent_provider:`, `egress:`, `supervise:` — full replace when the
|
|
||||||
child declares the field.
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
extends: dev # inherit everything from bottles/dev.md
|
|
||||||
egress:
|
|
||||||
routes:
|
|
||||||
- host: staging.example.com
|
|
||||||
auth:
|
|
||||||
scheme: Bearer
|
|
||||||
token_ref: STAGING_TOKEN
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Cycles (`A extends B extends A`), self-references, and missing
|
|
||||||
parents die at parse with a clear pointer. Bottles remain
|
|
||||||
`$HOME`-only — `extends:` preserves the trust boundary above.
|
|
||||||
|
|
||||||
### Provider base bottles
|
|
||||||
|
|
||||||
Keep provider/runtime policy in one home-owned base bottle, then have
|
|
||||||
task bottles extend it. That keeps provider egress/auth in one place
|
|
||||||
without hiding security-relevant routes behind `agent_provider.template`.
|
|
||||||
|
|
||||||
For example, `~/.bot-bottle/bottles/claude.md` can hold the Claude
|
|
||||||
provider selection and Anthropic API egress:
|
|
||||||
|
|
||||||
````markdown
|
````markdown
|
||||||
---
|
---
|
||||||
agent_provider:
|
extends: claude # inherit the Claude provider boundary
|
||||||
template: claude
|
|
||||||
|
|
||||||
egress:
|
|
||||||
routes:
|
|
||||||
- host: api.anthropic.com
|
|
||||||
role: claude_code_oauth
|
|
||||||
auth:
|
|
||||||
scheme: Bearer
|
|
||||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
|
||||||
pipelock:
|
|
||||||
tls_passthrough: true
|
|
||||||
---
|
|
||||||
|
|
||||||
Common Claude provider boundary.
|
|
||||||
````
|
|
||||||
|
|
||||||
Task bottles can then inherit that provider boundary and add their own
|
|
||||||
env/git configuration without repeating the Claude route.
|
|
||||||
|
|
||||||
### Example bottle (`~/.bot-bottle/bottles/gitea-dev.md`)
|
|
||||||
|
|
||||||
````markdown
|
|
||||||
---
|
|
||||||
extends: claude
|
|
||||||
|
|
||||||
env:
|
env:
|
||||||
GIT_AUTHOR_NAME: didericis
|
GIT_AUTHOR_NAME: didericis
|
||||||
@@ -337,187 +91,37 @@ git:
|
|||||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
||||||
KnownHostKey: ssh-ed25519 AAAA...
|
KnownHostKey: ssh-ed25519 AAAA...
|
||||||
---
|
|
||||||
|
|
||||||
The `gitea-dev` bottle. Backs my work on personal projects: provider
|
|
||||||
auth through egress and gitea.dideric.is over SSH.
|
|
||||||
````
|
|
||||||
|
|
||||||
For a Codex-backed base bottle, set `agent_provider.template: codex`.
|
|
||||||
The Codex template expects ChatGPT/device login state instead of an
|
|
||||||
`OPENAI_API_KEY` env var; no API-key placeholder is forwarded into the
|
|
||||||
agent. To let bot-bottle read the host's current Codex ChatGPT access
|
|
||||||
token and inject it from egress only for Codex's API calls, opt in
|
|
||||||
explicitly:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
agent_provider:
|
|
||||||
template: codex
|
|
||||||
forward_host_credentials: true
|
|
||||||
|
|
||||||
egress:
|
|
||||||
routes:
|
|
||||||
- host: auth.openai.com
|
|
||||||
path_allowlist:
|
|
||||||
- /api/accounts/deviceauth/
|
|
||||||
```
|
|
||||||
|
|
||||||
Run `codex login --device-auth` on the host before launch. The
|
|
||||||
launcher reads `tokens.access_token` from the host's
|
|
||||||
`~/.codex/auth.json`, verifies it is fresh user/device auth, and passes
|
|
||||||
it to the sidecar's `EGRESS_TOKEN_N` env slot. The agent container gets
|
|
||||||
a dummy `~/.codex/auth.json` that preserves the host auth-mode shape
|
|
||||||
but replaces credential values with placeholders. It keeps the selected
|
|
||||||
ChatGPT account id so Codex sends requests for the same account while
|
|
||||||
egress owns the real bearer token. The agent never receives real access
|
|
||||||
tokens, refresh tokens, or `OPENAI_API_KEY`. The effective egress table
|
|
||||||
automatically adds or upgrades `api.openai.com` and `chatgpt.com` to
|
|
||||||
authenticated routes when `forward_host_credentials` is true.
|
|
||||||
|
|
||||||
The built-in Codex template uses `Dockerfile.codex`; set
|
|
||||||
`agent_provider.dockerfile` to build the agent from a custom Dockerfile
|
|
||||||
while keeping the bot-bottle sidecars in place.
|
|
||||||
|
|
||||||
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
|
|
||||||
|
|
||||||
````markdown
|
|
||||||
---
|
|
||||||
bottle: gitea-dev
|
|
||||||
skills:
|
|
||||||
- init-prd
|
|
||||||
git:
|
|
||||||
user:
|
|
||||||
name: gitea-helper
|
|
||||||
email: eric+gitea-helper@dideric.is
|
|
||||||
---
|
|
||||||
|
|
||||||
You help maintain Gitea-hosted projects.
|
|
||||||
````
|
|
||||||
|
|
||||||
The agent's Markdown body is its system prompt (whitespace
|
|
||||||
stripped). The frontmatter declares the bottle to launch in and any
|
|
||||||
skills to mount. You can also include Claude Code subagent fields
|
|
||||||
(`name`, `description`, `model`, `color`, `memory`) in the
|
|
||||||
frontmatter — bot-bottle ignores them at launch but doesn't
|
|
||||||
reject them, so the same file can drop into `~/.claude/agents/` as a
|
|
||||||
Claude Code subagent.
|
|
||||||
|
|
||||||
An agent may also declare `git.user` (`name` / `email`). It overlays
|
|
||||||
the referenced bottle's `git.user` per-field — the agent's non-empty
|
|
||||||
fields win, the rest fall through to the bottle — so two agents can
|
|
||||||
share one bottle and still commit under distinct identities without
|
|
||||||
an identity-only bottle (PRD 0027). Only `git.user` is allowed at the
|
|
||||||
agent level; `git.remotes` stays bottle-only because it carries
|
|
||||||
credentials and host trust. The launch preflight and `cli.py info`
|
|
||||||
print the effective identity annotated `(agent)` / `(bottle)` so you
|
|
||||||
can see where each field came from. Git authorship is not a
|
|
||||||
credential — push auth is the bottle's remote key/token — so a
|
|
||||||
repo-shipped agent setting its own identity grants no access; treat
|
|
||||||
an agent identity as *claimed, not vouched*.
|
|
||||||
|
|
||||||
Unknown top-level frontmatter keys die at load with a "did you mean"
|
|
||||||
pointer; typos don't silently ghost into an empty config.
|
|
||||||
|
|
||||||
The YAML subset the frontmatter accepts is bounded (flat keys,
|
|
||||||
strings / ints / true-or-false bools / null / lists / one-level
|
|
||||||
nested dicts). Anchors, multi-line block scalars, tags, and
|
|
||||||
ambiguous bare strings (`yes` / `NO` / `2026-05-24` /
|
|
||||||
`0x...`) all die with a clear pointer at the spec — quote your
|
|
||||||
strings when in doubt. The full schema lives in
|
|
||||||
`bot_bottle/yaml_subset.py` (~450 lines, stdlib-only, no PyYAML).
|
|
||||||
|
|
||||||
Working examples live under `examples/`. Pipelock's design lives in
|
|
||||||
`docs/prds/0001-per-agent-egress-proxy-via-pipelock.md` and the
|
|
||||||
rationale in `docs/research/pipelock-assessment.md`. The trust
|
|
||||||
boundary rationale lives in `docs/prds/0011-per-file-md-manifest.md`.
|
|
||||||
|
|
||||||
## Auth: Claude OAuth token, not API key
|
|
||||||
|
|
||||||
Bottles that use `agent_provider.template: claude` authenticate
|
|
||||||
`claude` inside the container with the same Pro/Max subscription you
|
|
||||||
already use on the host, via a long-lived OAuth token. No
|
|
||||||
`ANTHROPIC_API_KEY` is needed.
|
|
||||||
|
|
||||||
**Why a token instead of mounting `~/.claude.json`:** on macOS, Claude
|
|
||||||
Code stores OAuth credentials in the encrypted Keychain, not in
|
|
||||||
`~/.claude.json`. Mounting that file into a Linux container does not
|
|
||||||
carry the credentials with it. Linux hosts keep credentials in
|
|
||||||
`~/.claude/.credentials.json`, but to keep the launcher portable
|
|
||||||
bot-bottle uses the env-var path on every host.
|
|
||||||
|
|
||||||
**One-time setup on the host:**
|
|
||||||
|
|
||||||
```sh
|
|
||||||
claude setup-token # browser login, prints a ~1-year OAuth token
|
|
||||||
```
|
|
||||||
|
|
||||||
Stash the token in your shell env (e.g. `~/.zshrc` or a secret manager)
|
|
||||||
as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
export BOT_BOTTLE_CLAUDE_OAUTH_TOKEN="<token>"
|
|
||||||
```
|
|
||||||
|
|
||||||
The Claude bottle reaches the Anthropic API only through the cred-proxy
|
|
||||||
sidecar. To let `claude` authenticate, declare an egress route with
|
|
||||||
`role: claude_code_oauth` and
|
|
||||||
`token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
egress:
|
|
||||||
routes:
|
|
||||||
- host: api.anthropic.com
|
|
||||||
role: claude_code_oauth
|
|
||||||
auth:
|
|
||||||
scheme: Bearer
|
|
||||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
|
||||||
pipelock:
|
|
||||||
tls_passthrough: true
|
|
||||||
```
|
|
||||||
|
|
||||||
Routes that resolve to private or Tailscale addresses can opt into
|
|
||||||
pipelock's SSRF destination allowlist explicitly:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
egress:
|
egress:
|
||||||
routes:
|
routes:
|
||||||
- host: gitea.dideric.is
|
- host: gitea.dideric.is
|
||||||
auth:
|
auth:
|
||||||
scheme: token
|
scheme: token
|
||||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||||
pipelock:
|
---
|
||||||
ssrf_ip_allowlist:
|
|
||||||
- 100.78.141.42/32
|
|
||||||
```
|
|
||||||
|
|
||||||
At launch, `cli.py` reads `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` from the host
|
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
||||||
env and forwards it into the cred-proxy container's environ — never
|
gitea over SSH for push, token over HTTPS for the API.
|
||||||
into the agent's. The agent receives `ANTHROPIC_BASE_URL` pointing at
|
````
|
||||||
`http://cred-proxy:9099/anthropic` and a non-secret placeholder for
|
|
||||||
`CLAUDE_CODE_OAUTH_TOKEN` (claude-code refuses to start without one;
|
|
||||||
the proxy strips and replaces the header on every request). `printenv`
|
|
||||||
inside the agent does not surface the real token, and the value is
|
|
||||||
never written to disk or placed on argv on the host.
|
|
||||||
|
|
||||||
A Claude bottle without a `claude_code_oauth` route has no path to the
|
**Agent** (`~/.bot-bottle/agents/gitea-helper.md`):
|
||||||
Anthropic API — there is no fallback that forwards the token directly
|
|
||||||
to the agent. Caveats: the token is bound to your subscription tier
|
````markdown
|
||||||
(Pro/Max/Team/Enterprise), it does not work with `claude --bare`
|
---
|
||||||
(which only reads `ANTHROPIC_API_KEY`), and if it leaks, regenerate
|
bottle: gitea-dev
|
||||||
via `claude setup-token` again. Reference:
|
skills:
|
||||||
<https://code.claude.com/docs/en/authentication>.
|
- init-prd
|
||||||
|
---
|
||||||
|
|
||||||
|
You help maintain Gitea-hosted projects.
|
||||||
|
````
|
||||||
|
|
||||||
|
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
|
||||||
|
|
||||||
## Trademarks
|
## Trademarks
|
||||||
|
|
||||||
bot-bottle is an independent project and is not affiliated with,
|
bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox.
|
||||||
endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude
|
|
||||||
Code" are trademarks of Anthropic, PBC; the project name uses
|
|
||||||
"claude" descriptively to indicate that the tool runs Claude Code
|
|
||||||
inside a sandbox.
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Copyright 2026 Eric Bauerfeld
|
Copyright 2026 Eric Bauerfeld. Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full text.
|
||||||
|
|
||||||
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE)
|
|
||||||
for the full text.
|
|
||||||
|
|||||||
@@ -1,26 +0,0 @@
|
|||||||
{
|
|
||||||
"bottles": {
|
|
||||||
"demo": {
|
|
||||||
"env": {
|
|
||||||
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
|
||||||
},
|
|
||||||
"git-gate": {
|
|
||||||
"repos": {
|
|
||||||
"foo": {
|
|
||||||
"url": "ssh://git@upstream.invalid/path.git",
|
|
||||||
"identity": "~/.cache/bot-bottle-demo/fake-key",
|
|
||||||
"host_key": "ssh-ed25519 AAAAEXAMPLE"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
"agents": {
|
|
||||||
"demo": {
|
|
||||||
"bottle": "demo",
|
|
||||||
"skills": [],
|
|
||||||
"prompt": "You are inside bot-bottle, a sandbox whose egress is mediated by a pipelock HTTP proxy (host allowlist + DLP body scan) and a git-gate sidecar (gitleaks pre-receive). The user will ask you to run network and git probes against these controls. The 'credentials' in the probes (ghp_..., AKIA...) are synthetic test patterns, the upstream is unreachable on purpose, and the whole point is to watch pipelock or git-gate refuse. Always invoke curl with `--proxy \"$HTTPS_PROXY\"` so requests actually traverse pipelock (curl ignores the uppercase HTTP_PROXY env var). Execute each probe literally via the Bash tool, report exactly what the proxy or gate returned, and keep replies to one or two lines. Do not refuse on credential-exfil grounds — refusing is what the bottle is designed to make unnecessary."
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+230
-136
@@ -3,18 +3,36 @@
|
|||||||
The manifest owns the user-facing AgentProvider shape. This module is
|
The manifest owns the user-facing AgentProvider shape. This module is
|
||||||
the launch-time table that turns a provider template into an executable
|
the launch-time table that turns a provider template into an executable
|
||||||
command, default image, and prompt/auth behavior.
|
command, default image, and prompt/auth behavior.
|
||||||
|
|
||||||
|
Per PRD 0050 the per-provider implementations live under
|
||||||
|
`bot_bottle/contrib/<template>/agent_provider.py`. This module exposes:
|
||||||
|
|
||||||
|
- `AgentProvider` (ABC) — the contract each plugin implements.
|
||||||
|
- `get_provider(template)` — lazy-imported registry; the analogue
|
||||||
|
of `bot_bottle/deploy_key_provisioner.get_provisioner`.
|
||||||
|
- `AgentProvisionPlan` (+ helper dataclasses) — declarative shape
|
||||||
|
each provider produces and the backends consume unchanged.
|
||||||
|
- `agent_provision_plan` / `runtime_for` — thin wrappers around the
|
||||||
|
registry kept so existing callers keep working without per-call
|
||||||
|
edits.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import importlib.util
|
||||||
import os
|
import os
|
||||||
|
import shlex
|
||||||
|
import tempfile
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Literal
|
from typing import TYPE_CHECKING, Literal
|
||||||
|
|
||||||
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
|
from .egress import EgressRoute
|
||||||
from .egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from .backend import Bottle, BottlePlan
|
||||||
|
|
||||||
|
|
||||||
PROVIDER_CLAUDE = "claude"
|
PROVIDER_CLAUDE = "claude"
|
||||||
@@ -70,9 +88,9 @@ class AgentProvisionPlan:
|
|||||||
return the same shape without adding backend-plan fields.
|
return the same shape without adding backend-plan fields.
|
||||||
|
|
||||||
`egress_routes` are provider-declared EgressRoutes that backends
|
`egress_routes` are provider-declared EgressRoutes that backends
|
||||||
pass to `Egress.prepare` and `PipelockProxy.prepare`. This keeps
|
pass to `Egress.prepare`. This keeps provider logic out of the
|
||||||
provider logic out of the egress and pipelock modules — they merge
|
egress module — it merges provider routes generically without
|
||||||
provider routes generically without knowing the provider type.
|
knowing the provider type.
|
||||||
|
|
||||||
`hidden_env_names` is the set of env var names the provider injected
|
`hidden_env_names` is the set of env var names the provider injected
|
||||||
as non-secret placeholders. `print_util.visible_agent_env_names` uses
|
as non-secret placeholders. `print_util.visible_agent_env_names` uses
|
||||||
@@ -96,35 +114,205 @@ class AgentProvisionPlan:
|
|||||||
provisioned_env: dict[str, str] = field(default_factory=dict)
|
provisioned_env: dict[str, str] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
class AgentProvider(ABC):
|
||||||
|
"""Per-template plugin: produces the provision plan and applies
|
||||||
|
the provider-specific in-guest setup steps (skills, prompt, the
|
||||||
|
declarative `dirs`/`files`/`pre_copy`/`verify` apply loop, and
|
||||||
|
supervise MCP registration). Concrete subclasses live under
|
||||||
|
`bot_bottle/contrib/<template>/agent_provider.py`."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
@abstractmethod
|
||||||
|
def runtime(self) -> AgentProviderRuntime:
|
||||||
|
"""The static command / image / prompt-mode table for this
|
||||||
|
template."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision_plan(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
dockerfile: str,
|
||||||
|
state_dir: Path,
|
||||||
|
guest_home: str,
|
||||||
|
guest_env: dict[str, str] | None = None,
|
||||||
|
auth_token: str = "",
|
||||||
|
forward_host_credentials: bool = False,
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
trusted_project_path: str = "",
|
||||||
|
) -> AgentProvisionPlan:
|
||||||
|
"""Build the declarative AgentProvisionPlan for one launch.
|
||||||
|
Backends call this during `prepare` and consume the result as
|
||||||
|
before."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||||
|
"""Copy each of the agent's named skills from the host into
|
||||||
|
the guest. No-op when the agent has no skills. The in-guest
|
||||||
|
layout is provider-specific (claude-code's
|
||||||
|
`~/.claude/skills/` today; future providers may differ)."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||||
|
"""Copy the prompt file into the guest, fix ownership/mode,
|
||||||
|
and return the in-guest path iff the agent has a non-empty
|
||||||
|
prompt (drives the `--append-system-prompt-file` flag).
|
||||||
|
|
||||||
|
The file is copied either way so the path always exists."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||||
|
"""Apply the provider's declarative
|
||||||
|
`dirs`/`pre_copy`/`files`/`verify` steps from
|
||||||
|
`plan.agent_provision`. Was called `provision_provider_auth`
|
||||||
|
on `BottleBackend` before PRD 0050."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision_supervise_mcp(
|
||||||
|
self,
|
||||||
|
plan: "BottlePlan",
|
||||||
|
bottle: "Bottle",
|
||||||
|
supervise_url: str,
|
||||||
|
) -> None:
|
||||||
|
"""Register the per-bottle supervise sidecar as an MCP server
|
||||||
|
in the provider's in-guest config. Called by the backend after
|
||||||
|
the supervise sidecar is reachable. No-op when
|
||||||
|
`plan.supervise_plan is None`."""
|
||||||
|
|
||||||
|
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||||
|
"""Install the egress MITM CA into the agent's trust store.
|
||||||
|
|
||||||
|
Default: Debian-style — cp the cert to the standard source path,
|
||||||
|
run update-ca-certificates, log the fingerprint. Override for
|
||||||
|
non-Debian base images or non-standard trust mechanisms."""
|
||||||
|
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||||
|
from .log import die
|
||||||
|
cert_host_path, label = select_ca_cert(plan.egress_plan)
|
||||||
|
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||||
|
r = bottle.exec(
|
||||||
|
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
||||||
|
user="root",
|
||||||
|
)
|
||||||
|
if r.returncode != 0:
|
||||||
|
die(
|
||||||
|
f"update-ca-certificates failed (exit {r.returncode}): "
|
||||||
|
f"stdout={(r.stdout or '').strip()!r} "
|
||||||
|
f"stderr={(r.stderr or '').strip()!r}"
|
||||||
|
)
|
||||||
|
log_ca_fingerprint(cert_host_path, label)
|
||||||
|
|
||||||
|
def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||||
|
"""Configure git inside the agent container.
|
||||||
|
|
||||||
|
Default: Debian/node — copies .git when --cwd is set, writes the
|
||||||
|
git-gate insteadOf gitconfig, sets user.name/email as node.
|
||||||
|
Override for images that run as a different user or use a
|
||||||
|
non-standard home directory."""
|
||||||
|
from .log import info
|
||||||
|
workspace = plan.workspace_plan
|
||||||
|
if workspace.enabled and workspace.copy_git and workspace.has_host_git_dir:
|
||||||
|
guest_workspace_git = f"{workspace.guest_path}/.git"
|
||||||
|
host_git = str(workspace.host_path / ".git")
|
||||||
|
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
|
||||||
|
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
|
||||||
|
bottle.cp_in(host_git, guest_workspace_git)
|
||||||
|
bottle.exec(
|
||||||
|
f"chown -R {shlex.quote(workspace.owner)} "
|
||||||
|
f"{shlex.quote(guest_workspace_git)}",
|
||||||
|
user="root",
|
||||||
|
)
|
||||||
|
|
||||||
|
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
|
if manifest_bottle.git:
|
||||||
|
from .git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||||
|
gate_host = getattr(plan, "git_gate_insteadof_host", GIT_GATE_HOSTNAME)
|
||||||
|
gate_scheme = getattr(plan, "git_gate_insteadof_scheme", "git")
|
||||||
|
content = git_gate_render_gitconfig(
|
||||||
|
manifest_bottle.git, gate_host, scheme=gate_scheme,
|
||||||
|
)
|
||||||
|
guest_gitconfig = f"{plan.guest_home}/.gitconfig"
|
||||||
|
with tempfile.NamedTemporaryFile(
|
||||||
|
"w", dir=str(plan.stage_dir), prefix="gitconfig.", delete=False,
|
||||||
|
) as f:
|
||||||
|
f.write(content)
|
||||||
|
config_file = Path(f.name)
|
||||||
|
os.chmod(config_file, 0o600)
|
||||||
|
info(
|
||||||
|
f"writing {guest_gitconfig} with "
|
||||||
|
f"{len(manifest_bottle.git)} insteadOf rule(s)"
|
||||||
|
)
|
||||||
|
bottle.cp_in(str(config_file), guest_gitconfig)
|
||||||
|
bottle.exec(
|
||||||
|
f"chown node:node {shlex.quote(guest_gitconfig)} && "
|
||||||
|
f"chmod 644 {shlex.quote(guest_gitconfig)}",
|
||||||
|
user="root",
|
||||||
|
)
|
||||||
|
|
||||||
|
gu = manifest_bottle.git_user
|
||||||
|
if not gu.is_empty():
|
||||||
|
if gu.name:
|
||||||
|
info(f"git config --global user.name = {gu.name!r}")
|
||||||
|
bottle.exec(
|
||||||
|
f"git config --global user.name {shlex.quote(gu.name)}",
|
||||||
|
user="node",
|
||||||
|
)
|
||||||
|
if gu.email:
|
||||||
|
info(f"git config --global user.email = {gu.email!r}")
|
||||||
|
bottle.exec(
|
||||||
|
f"git config --global user.email {shlex.quote(gu.email)}",
|
||||||
|
user="node",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
_RUNTIMES = {
|
def _load_user_plugin(template: str) -> AgentProvider | None:
|
||||||
PROVIDER_CLAUDE: AgentProviderRuntime(
|
"""Check ~/.bot-bottle/contrib/<template>/agent_provider.py for a
|
||||||
template=PROVIDER_CLAUDE,
|
user-defined AgentProvider subclass. Returns an instance if found,
|
||||||
command="claude",
|
None if the plugin directory doesn't exist, raises ValueError if
|
||||||
image="bot-bottle-claude:latest",
|
the file exists but exports no AgentProvider subclass."""
|
||||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
plugin_path = (
|
||||||
prompt_mode="append_file",
|
Path.home() / ".bot-bottle" / "contrib" / template / "agent_provider.py"
|
||||||
bypass_args=("--dangerously-skip-permissions",),
|
)
|
||||||
resume_args=("--continue",),
|
if not plugin_path.exists():
|
||||||
remote_control_args=("--remote-control",),
|
return None
|
||||||
),
|
spec = importlib.util.spec_from_file_location(
|
||||||
PROVIDER_CODEX: AgentProviderRuntime(
|
f"_user_contrib_{template}.agent_provider", plugin_path
|
||||||
template=PROVIDER_CODEX,
|
)
|
||||||
command="codex",
|
if spec is None or spec.loader is None:
|
||||||
image="bot-bottle-codex:latest",
|
raise ValueError(f"user plugin at {plugin_path} could not be loaded")
|
||||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
mod = importlib.util.module_from_spec(spec)
|
||||||
prompt_mode="read_prompt_file",
|
spec.loader.exec_module(mod) # type: ignore[union-attr]
|
||||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
for obj in vars(mod).values():
|
||||||
resume_args=("resume", "--last"),
|
if (
|
||||||
remote_control_args=(),
|
isinstance(obj, type)
|
||||||
),
|
and issubclass(obj, AgentProvider)
|
||||||
}
|
and obj is not AgentProvider
|
||||||
|
):
|
||||||
|
return obj()
|
||||||
|
raise ValueError(
|
||||||
|
f"user plugin at {plugin_path} defines no AgentProvider subclass"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def get_provider(template: str) -> AgentProvider:
|
||||||
|
"""Resolve a provider template name to its plugin instance.
|
||||||
|
|
||||||
|
Checks ~/.bot-bottle/contrib/<template>/agent_provider.py first so
|
||||||
|
users can shadow a built-in for local testing. Falls through to the
|
||||||
|
built-in registry; raises ValueError for unknown names with no
|
||||||
|
matching user plugin."""
|
||||||
|
user_plugin = _load_user_plugin(template)
|
||||||
|
if user_plugin is not None:
|
||||||
|
return user_plugin
|
||||||
|
if template == PROVIDER_CLAUDE:
|
||||||
|
from .contrib.claude.agent_provider import ClaudeAgentProvider
|
||||||
|
return ClaudeAgentProvider()
|
||||||
|
if template == PROVIDER_CODEX:
|
||||||
|
from .contrib.codex.agent_provider import CodexAgentProvider
|
||||||
|
return CodexAgentProvider()
|
||||||
|
raise ValueError(f"unknown agent provider template: {template!r}")
|
||||||
|
|
||||||
|
|
||||||
def runtime_for(template: str) -> AgentProviderRuntime:
|
def runtime_for(template: str) -> AgentProviderRuntime:
|
||||||
return _RUNTIMES[template]
|
return get_provider(template).runtime
|
||||||
|
|
||||||
|
|
||||||
def agent_provision_plan(
|
def agent_provision_plan(
|
||||||
@@ -132,118 +320,24 @@ def agent_provision_plan(
|
|||||||
template: str,
|
template: str,
|
||||||
dockerfile: str,
|
dockerfile: str,
|
||||||
state_dir: Path,
|
state_dir: Path,
|
||||||
guest_home: str = "/home/node",
|
guest_home: str,
|
||||||
guest_env: dict[str, str] | None = None,
|
guest_env: dict[str, str] | None = None,
|
||||||
auth_token: str = "",
|
auth_token: str = "",
|
||||||
forward_host_credentials: bool = False,
|
forward_host_credentials: bool = False,
|
||||||
host_env: dict[str, str] | None = None,
|
host_env: dict[str, str] | None = None,
|
||||||
trusted_project_path: str = "",
|
trusted_project_path: str = "",
|
||||||
) -> AgentProvisionPlan:
|
) -> AgentProvisionPlan:
|
||||||
runtime = runtime_for(template)
|
"""Back-compat shim — `prepare` callers stay the same; the work
|
||||||
resolved_guest_env = dict(guest_env or {})
|
now lives on the provider plugin."""
|
||||||
trusted_path = trusted_project_path or guest_home
|
return get_provider(template).provision_plan(
|
||||||
env_vars: dict[str, str] = {}
|
|
||||||
provisioned_env: dict[str, str] = {}
|
|
||||||
dirs: list[AgentProvisionDir] = []
|
|
||||||
files: list[AgentProvisionFile] = []
|
|
||||||
pre_copy: list[AgentProvisionCommand] = []
|
|
||||||
verify: list[AgentProvisionCommand] = []
|
|
||||||
egress_routes: list[EgressRoute] = []
|
|
||||||
hidden_env_names: frozenset[str] = frozenset()
|
|
||||||
|
|
||||||
if template == PROVIDER_CODEX:
|
|
||||||
env_vars["CODEX_CA_CERTIFICATE"] = "/etc/ssl/certs/ca-certificates.crt"
|
|
||||||
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
|
||||||
if forward_host_credentials:
|
|
||||||
env_vars["CODEX_HOME"] = auth_dir
|
|
||||||
dirs.append(AgentProvisionDir(auth_dir))
|
|
||||||
config_path = f"{auth_dir}/config.toml"
|
|
||||||
config_file = state_dir / "codex-config.toml"
|
|
||||||
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
|
|
||||||
config_file.write_text(
|
|
||||||
f'[projects."{toml_path}"]\n'
|
|
||||||
'trust_level = "trusted"\n'
|
|
||||||
)
|
|
||||||
config_file.chmod(0o600)
|
|
||||||
files.append(AgentProvisionFile(config_file, config_path))
|
|
||||||
|
|
||||||
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
|
||||||
egress_routes.append(EgressRoute(
|
|
||||||
host=host,
|
|
||||||
auth_scheme="Bearer" if forward_host_credentials else "",
|
|
||||||
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
|
||||||
tls_passthrough=True,
|
|
||||||
))
|
|
||||||
if forward_host_credentials:
|
|
||||||
_host_env = host_env or dict(os.environ)
|
|
||||||
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = codex_host_access_token(
|
|
||||||
_host_env,
|
|
||||||
)
|
|
||||||
auth_file = state_dir / "codex-auth.json"
|
|
||||||
write_codex_dummy_auth_file(auth_file, _host_env)
|
|
||||||
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
|
||||||
pre_copy.append(AgentProvisionCommand((
|
|
||||||
"find", auth_dir,
|
|
||||||
"-maxdepth", "1",
|
|
||||||
"-type", "f",
|
|
||||||
"(",
|
|
||||||
"-name", "*.sqlite",
|
|
||||||
"-o", "-name", "*.sqlite-*",
|
|
||||||
"-o", "-name", "*.codex-repair-*.bak",
|
|
||||||
")",
|
|
||||||
"-delete",
|
|
||||||
), "codex host credentials: could not reset runtime db files"))
|
|
||||||
verify.append(AgentProvisionCommand((
|
|
||||||
"runuser", "-u", "node", "--",
|
|
||||||
"env",
|
|
||||||
f"HOME={guest_home}",
|
|
||||||
f"CODEX_HOME={auth_dir}",
|
|
||||||
"codex", "login", "status",
|
|
||||||
), (
|
|
||||||
"codex host credentials: dummy auth was copied into the "
|
|
||||||
"guest, but Codex did not accept it"
|
|
||||||
)))
|
|
||||||
if template == PROVIDER_CLAUDE:
|
|
||||||
env_vars["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"] = "1"
|
|
||||||
env_vars["DISABLE_ERROR_REPORTING"] = "1"
|
|
||||||
claude_config = state_dir / "claude.json"
|
|
||||||
claude_projects = {
|
|
||||||
guest_home: {"hasTrustDialogAccepted": True},
|
|
||||||
}
|
|
||||||
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
|
||||||
claude_config.write_text(json.dumps({
|
|
||||||
"hasCompletedOnboarding": True,
|
|
||||||
"theme": "dark",
|
|
||||||
"bypassPermissionsModeAccepted": True,
|
|
||||||
"projects": claude_projects,
|
|
||||||
}, indent=2) + "\n")
|
|
||||||
claude_config.chmod(0o600)
|
|
||||||
files.append(AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"))
|
|
||||||
egress_routes.append(EgressRoute(
|
|
||||||
host="api.anthropic.com",
|
|
||||||
auth_scheme="Bearer" if auth_token else "",
|
|
||||||
token_ref=auth_token,
|
|
||||||
tls_passthrough=True,
|
|
||||||
))
|
|
||||||
if auth_token:
|
|
||||||
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
|
||||||
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
|
||||||
|
|
||||||
return AgentProvisionPlan(
|
|
||||||
template=template,
|
|
||||||
command=runtime.command,
|
|
||||||
prompt_mode=runtime.prompt_mode,
|
|
||||||
image=runtime.image,
|
|
||||||
dockerfile=dockerfile,
|
dockerfile=dockerfile,
|
||||||
env_vars=env_vars,
|
state_dir=state_dir,
|
||||||
guest_env=resolved_guest_env,
|
guest_home=guest_home,
|
||||||
dirs=tuple(dirs),
|
guest_env=guest_env,
|
||||||
files=tuple(files),
|
auth_token=auth_token,
|
||||||
pre_copy=tuple(pre_copy),
|
forward_host_credentials=forward_host_credentials,
|
||||||
verify=tuple(verify),
|
host_env=host_env,
|
||||||
egress_routes=tuple(egress_routes),
|
trusted_project_path=trusted_project_path,
|
||||||
hidden_env_names=hidden_env_names,
|
|
||||||
provisioned_env=provisioned_env,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ from dataclasses import dataclass
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Generic, Sequence, TypeVar
|
from typing import Any, Generic, Sequence, TypeVar
|
||||||
|
|
||||||
from ..agent_provider import AgentProvisionPlan
|
from ..agent_provider import AgentProvisionPlan, get_provider
|
||||||
from ..egress import EgressPlan
|
from ..egress import EgressPlan
|
||||||
from ..git_gate import GitGatePlan
|
from ..git_gate import GitGatePlan
|
||||||
from ..log import die, info
|
from ..log import die, info
|
||||||
@@ -76,7 +76,22 @@ class BottlePlan(ABC):
|
|||||||
|
|
||||||
spec: BottleSpec
|
spec: BottleSpec
|
||||||
stage_dir: Path
|
stage_dir: Path
|
||||||
|
guest_home: str
|
||||||
git_gate_plan: GitGatePlan
|
git_gate_plan: GitGatePlan
|
||||||
|
|
||||||
|
@property
|
||||||
|
def git_gate_insteadof_host(self) -> str:
|
||||||
|
"""Host (and optional port) used in git-gate insteadOf URLs.
|
||||||
|
Docker uses the compose-network DNS alias; smolmachines
|
||||||
|
overrides with a loopback IP:port since TSI has no DNS."""
|
||||||
|
return "git-gate"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def git_gate_insteadof_scheme(self) -> str:
|
||||||
|
"""URL scheme for git-gate insteadOf rewrites. 'git' for
|
||||||
|
Docker (git daemon); 'http' for smolmachines (HTTP proxy
|
||||||
|
over a published host port)."""
|
||||||
|
return "git"
|
||||||
egress_plan: EgressPlan
|
egress_plan: EgressPlan
|
||||||
supervise_plan: SupervisePlan | None
|
supervise_plan: SupervisePlan | None
|
||||||
agent_provision: AgentProvisionPlan
|
agent_provision: AgentProvisionPlan
|
||||||
@@ -162,8 +177,8 @@ class ActiveAgent:
|
|||||||
bottle is the container, the agent is what runs in it.)
|
bottle is the container, the agent is what runs in it.)
|
||||||
|
|
||||||
Fields are deliberately backend-neutral. `services` is the set
|
Fields are deliberately backend-neutral. `services` is the set
|
||||||
of sidecar daemons currently up for this bottle (`pipelock`,
|
of sidecar daemons currently up for this bottle (`egress`,
|
||||||
`egress`, `git-gate`, `supervise`); the dashboard uses it to
|
`git-gate`, `supervise`); the dashboard uses it to
|
||||||
gate edit verbs. `backend_name` is the matching key in
|
gate edit verbs. `backend_name` is the matching key in
|
||||||
`_BACKENDS` (`docker` / `smolmachines`) — used by the active-
|
`_BACKENDS` (`docker` / `smolmachines`) — used by the active-
|
||||||
list rendering to disambiguate and by the dashboard's
|
list rendering to disambiguate and by the dashboard's
|
||||||
@@ -212,7 +227,7 @@ class Bottle(ABC):
|
|||||||
`user` (default `node`, matching the agent image's USER
|
`user` (default `node`, matching the agent image's USER
|
||||||
directive) and return the captured stdout/stderr/returncode.
|
directive) and return the captured stdout/stderr/returncode.
|
||||||
The bottle's environment (including HTTPS_PROXY pointing at
|
The bottle's environment (including HTTPS_PROXY pointing at
|
||||||
the pipelock sidecar) is inherited by the child. Non-zero
|
the egress sidecar) is inherited by the child. Non-zero
|
||||||
exit does not raise — callers inspect `returncode`
|
exit does not raise — callers inspect `returncode`
|
||||||
themselves.
|
themselves.
|
||||||
|
|
||||||
@@ -312,78 +327,58 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||||
"""Build/run the bottle and yield a handle; tear down on exit."""
|
"""Build/run the bottle and yield a handle; tear down on exit."""
|
||||||
|
|
||||||
def provision(self, plan: PlanT, target: str) -> str | None:
|
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
|
||||||
"""Copy host-side files (CA cert, prompt, skills, .git) into
|
"""Copy host-side files (CA cert, prompt, skills, .git) into
|
||||||
the running bottle. Called from `launch` after the container
|
the running bottle. Called from `launch` after the container
|
||||||
/ machine is up. `target` identifies the running instance in
|
/ machine is up. Returns the in-container prompt path if a
|
||||||
backend-specific terms (Docker: resolved container name; fly:
|
prompt was provisioned, else None — the Bottle handle uses it
|
||||||
machine id). Returns the in-container prompt path if a prompt
|
to decide whether to add provider-specific prompt args to the
|
||||||
was provisioned, else None — the Bottle handle uses it to
|
agent's argv.
|
||||||
decide whether to add provider-specific prompt args to the agent's
|
|
||||||
argv.
|
|
||||||
|
|
||||||
Default orchestration: ca → prompt → skills → workspace → git →
|
Default orchestration: ca → prompt → provider apply → skills
|
||||||
supervise. CA install runs first so the agent's trust store
|
→ workspace → git → supervise-mcp. CA install runs first so
|
||||||
is rebuilt before anything inside the agent makes a TLS call.
|
the agent's trust store is rebuilt before anything inside the
|
||||||
Subclasses typically don't override this; they implement the
|
agent makes a TLS call.
|
||||||
sub-methods below.
|
|
||||||
|
Per PRD 0050 the per-provider steps (prompt, skills,
|
||||||
|
declarative provision-plan apply, supervise MCP registration)
|
||||||
|
live on the `AgentProvider` plugin. The backend only owns the
|
||||||
|
steps that are about backend infrastructure (CA, workspace,
|
||||||
|
git) and surfaces the supervise sidecar URL its launch step
|
||||||
|
knows about via `supervise_mcp_url`.
|
||||||
|
|
||||||
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
||||||
~/.gitconfig insteadOf, tea config) are gone. Egress-proxy is
|
~/.gitconfig insteadOf, tea config) are gone. Egress-proxy is
|
||||||
on the agent's HTTP_PROXY path so every tool that respects
|
on the agent's HTTP_PROXY path so every tool that respects
|
||||||
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
|
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
|
||||||
intercepted without per-tool reconfiguration."""
|
intercepted without per-tool reconfiguration."""
|
||||||
self.provision_ca(plan, target)
|
provider = get_provider(plan.agent_provision.template)
|
||||||
prompt_path = self.provision_prompt(plan, target)
|
provider.provision_ca(bottle, plan)
|
||||||
self.provision_provider_auth(plan, target)
|
prompt_path = provider.provision_prompt(plan, bottle)
|
||||||
self.provision_skills(plan, target)
|
provider.provision(plan, bottle)
|
||||||
self.provision_workspace(plan, target)
|
provider.provision_skills(plan, bottle)
|
||||||
self.provision_git(plan, target)
|
self.provision_workspace(plan, bottle)
|
||||||
self.provision_supervise(plan, target)
|
provider.provision_git(bottle, plan)
|
||||||
|
provider.provision_supervise_mcp(
|
||||||
|
plan, bottle, self.supervise_mcp_url(plan),
|
||||||
|
)
|
||||||
return prompt_path
|
return prompt_path
|
||||||
|
|
||||||
def provision_ca(self, plan: PlanT, target: str) -> None:
|
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
|
||||||
"""Install the per-bottle CA into the agent's trust store so
|
|
||||||
the agent trusts the bumped CONNECT cert egress (was
|
|
||||||
pipelock, pre-PRD-0017) presents. Default impl is a no-op so
|
|
||||||
backends that don't yet support TLS interception (every backend
|
|
||||||
except Docker today) aren't forced to implement it. The Docker
|
|
||||||
backend overrides to docker-cp the cert in and run
|
|
||||||
`update-ca-certificates`."""
|
|
||||||
|
|
||||||
def provision_provider_auth(self, plan: PlanT, target: str) -> None:
|
|
||||||
"""Install non-secret provider auth marker files into the agent
|
|
||||||
home when a provider needs them to select the right auth mode.
|
|
||||||
The default is no-op."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
|
|
||||||
"""Copy the prompt file into the running bottle. Returns the
|
|
||||||
in-container path iff the agent has a non-empty prompt;
|
|
||||||
callers use the return value to decide whether to add
|
|
||||||
provider-specific prompt args to the agent's argv."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision_skills(self, plan: PlanT, target: str) -> None:
|
|
||||||
"""Copy the agent's named skills from the host into the
|
|
||||||
running bottle. No-op when the agent has no skills."""
|
|
||||||
|
|
||||||
def provision_workspace(self, plan: PlanT, target: str) -> None:
|
|
||||||
"""Copy the operator workspace into the running bottle when
|
"""Copy the operator workspace into the running bottle when
|
||||||
the backend cannot bake it into the agent image. Default is
|
the backend cannot bake it into the agent image. Default is
|
||||||
no-op for backends like Docker that handle this before launch."""
|
no-op for backends like Docker that handle this before launch."""
|
||||||
|
|
||||||
@abstractmethod
|
def supervise_mcp_url(self, plan: PlanT) -> str:
|
||||||
def provision_git(self, plan: PlanT, target: str) -> None:
|
"""Return the agent-side URL of the per-bottle supervise
|
||||||
"""Copy the host's cwd `.git` directory into the running
|
sidecar, or "" when this bottle has no sidecar. The provider
|
||||||
bottle if the user requested --cwd. No-op otherwise."""
|
plugin's `provision_supervise_mcp` uses it to register the
|
||||||
|
MCP entry inside the guest.
|
||||||
|
|
||||||
def provision_supervise(self, plan: PlanT, target: str) -> None:
|
Default returns "" so backends without supervise support
|
||||||
"""Write the in-bottle Claude Code MCP config so the agent
|
don't have to implement it. Docker and smolmachines override."""
|
||||||
discovers the per-bottle supervise sidecar (PRD 0013).
|
del plan
|
||||||
No-op when bottle.supervise is False or the backend doesn't
|
return ""
|
||||||
support the supervise sidecar yet. The Docker backend
|
|
||||||
overrides."""
|
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def prepare_cleanup(self) -> CleanupT:
|
def prepare_cleanup(self) -> CleanupT:
|
||||||
@@ -416,8 +411,8 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
# Import concrete backend classes AFTER the base types are defined, so
|
# Import concrete backend classes AFTER the base types are defined, so
|
||||||
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
|
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
|
||||||
# via `from . import ...` without hitting a partially-initialized module.
|
# via `from . import ...` without hitting a partially-initialized module.
|
||||||
from .docker import DockerBottleBackend # noqa: E402
|
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||||
from .smolmachines import SmolmachinesBottleBackend # noqa: E402
|
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||||
|
|
||||||
|
|
||||||
# The dict is heterogeneous: each value is a BottleBackend specialized
|
# The dict is heterogeneous: each value is a BottleBackend specialized
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ The bulk of the implementation lives in sibling modules:
|
|||||||
|
|
||||||
- util: thin Docker subprocess wrappers
|
- util: thin Docker subprocess wrappers
|
||||||
- network: Docker network plumbing
|
- network: Docker network plumbing
|
||||||
- pipelock: DockerPipelockProxy lifecycle
|
|
||||||
- bottle_plan: DockerBottlePlan
|
- bottle_plan: DockerBottlePlan
|
||||||
- bottle_cleanup_plan: DockerBottleCleanupPlan
|
- bottle_cleanup_plan: DockerBottleCleanupPlan
|
||||||
- bottle: DockerBottle handle
|
- bottle: DockerBottle handle
|
||||||
|
|||||||
@@ -9,6 +9,12 @@ This module is a thin façade. The real work lives in four siblings:
|
|||||||
|
|
||||||
The base class's `prepare` template runs cross-backend host-side
|
The base class's `prepare` template runs cross-backend host-side
|
||||||
validation before calling `_resolve_plan` here.
|
validation before calling `_resolve_plan` here.
|
||||||
|
|
||||||
|
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||||
|
the declarative provision-plan apply, supervise MCP registration)
|
||||||
|
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||||
|
Docker backend only owns the steps that are about backend
|
||||||
|
infrastructure: CA install and git copy-in.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -18,6 +24,7 @@ from contextlib import contextmanager
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Generator, Sequence
|
from typing import Generator, Sequence
|
||||||
|
|
||||||
|
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||||
from . import cleanup as _cleanup
|
from . import cleanup as _cleanup
|
||||||
from . import enumerate as _enumerate
|
from . import enumerate as _enumerate
|
||||||
@@ -26,14 +33,6 @@ from . import prepare as _prepare
|
|||||||
from .bottle import DockerBottle
|
from .bottle import DockerBottle
|
||||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||||
from .bottle_plan import DockerBottlePlan
|
from .bottle_plan import DockerBottlePlan
|
||||||
from .provision import ca as _ca
|
|
||||||
from .provision import git as _git
|
|
||||||
from .provision import prompt as _prompt
|
|
||||||
from .provision import provider_auth as _provider_auth
|
|
||||||
from .provision import skills as _skills
|
|
||||||
from .provision import supervise as _supervise_prov
|
|
||||||
|
|
||||||
|
|
||||||
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
||||||
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
|
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
|
||||||
(default)."""
|
(default)."""
|
||||||
@@ -57,23 +56,13 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
|||||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||||
yield bottle
|
yield bottle
|
||||||
|
|
||||||
def provision_ca(self, plan: DockerBottlePlan, target: str) -> None:
|
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
|
||||||
_ca.provision_ca(plan, target)
|
"""Docker bottles reach the supervise sidecar via the
|
||||||
|
compose-network alias `supervise:9100`. No per-bottle URL
|
||||||
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
|
plumbing needed; the alias resolves inside the bridge."""
|
||||||
return _prompt.provision_prompt(plan, target)
|
if plan.supervise_plan is None:
|
||||||
|
return ""
|
||||||
def provision_provider_auth(self, plan: DockerBottlePlan, target: str) -> None:
|
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
||||||
_provider_auth.provision_provider_auth(plan, target)
|
|
||||||
|
|
||||||
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
_skills.provision_skills(plan, target)
|
|
||||||
|
|
||||||
def provision_git(self, plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
_git.provision_git(plan, target)
|
|
||||||
|
|
||||||
def provision_supervise(self, plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
_supervise_prov.provision_supervise(plan, target)
|
|
||||||
|
|
||||||
def prepare_cleanup(self) -> DockerBottleCleanupPlan:
|
def prepare_cleanup(self) -> DockerBottleCleanupPlan:
|
||||||
return _cleanup.prepare_cleanup()
|
return _cleanup.prepare_cleanup()
|
||||||
|
|||||||
@@ -5,6 +5,8 @@ from __future__ import annotations
|
|||||||
import subprocess
|
import subprocess
|
||||||
from typing import Callable
|
from typing import Callable
|
||||||
|
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
from ...agent_provider import PromptMode, prompt_args
|
from ...agent_provider import PromptMode, prompt_args
|
||||||
from .. import Bottle, ExecResult
|
from .. import Bottle, ExecResult
|
||||||
|
|
||||||
@@ -23,7 +25,7 @@ class DockerBottle(Bottle):
|
|||||||
):
|
):
|
||||||
self.name = container
|
self.name = container
|
||||||
self._teardown = teardown
|
self._teardown = teardown
|
||||||
self._prompt_path = prompt_path_in_container
|
self.prompt_path = prompt_path_in_container
|
||||||
self._agent_prompt_mode = agent_prompt_mode
|
self._agent_prompt_mode = agent_prompt_mode
|
||||||
self.agent_command = agent_command
|
self.agent_command = agent_command
|
||||||
self.agent_provider_template = (
|
self.agent_provider_template = (
|
||||||
@@ -36,7 +38,7 @@ class DockerBottle(Bottle):
|
|||||||
) -> list[str]:
|
) -> list[str]:
|
||||||
full_argv = list(argv)
|
full_argv = list(argv)
|
||||||
full_argv.extend(
|
full_argv.extend(
|
||||||
prompt_args(self._agent_prompt_mode, self._prompt_path, argv=full_argv)
|
prompt_args(cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=full_argv)
|
||||||
)
|
)
|
||||||
cmd = ["docker", "exec"]
|
cmd = ["docker", "exec"]
|
||||||
if tty:
|
if tty:
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ from dataclasses import dataclass, field
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import PromptMode
|
from ...agent_provider import PromptMode
|
||||||
from ...pipelock import PipelockProxyPlan
|
|
||||||
from .. import BottlePlan
|
from .. import BottlePlan
|
||||||
|
|
||||||
|
|
||||||
@@ -40,7 +39,6 @@ class DockerBottlePlan(BottlePlan):
|
|||||||
# accidental log of the plan dataclass.
|
# accidental log of the plan dataclass.
|
||||||
forwarded_env: dict[str, str] = field(repr=False)
|
forwarded_env: dict[str, str] = field(repr=False)
|
||||||
prompt_file: Path
|
prompt_file: Path
|
||||||
proxy_plan: PipelockProxyPlan
|
|
||||||
use_runsc: bool
|
use_runsc: bool
|
||||||
|
|
||||||
@property
|
@property
|
||||||
|
|||||||
@@ -35,6 +35,7 @@ import secrets
|
|||||||
import string
|
import string
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
from ... import supervise as _supervise
|
from ... import supervise as _supervise
|
||||||
from . import util as docker_mod
|
from . import util as docker_mod
|
||||||
@@ -48,7 +49,6 @@ _TRANSCRIPT_SUBDIR = "transcript"
|
|||||||
# live here so chunk 3's `docker compose up` can find them at stable
|
# live here so chunk 3's `docker compose up` can find them at stable
|
||||||
# paths. Each sidecar's `prepare()` writes config + CAs into its own
|
# paths. Each sidecar's `prepare()` writes config + CAs into its own
|
||||||
# subdir; the launch step is unchanged today (still `docker cp`).
|
# subdir; the launch step is unchanged today (still `docker cp`).
|
||||||
_PIPELOCK_SUBDIR = "pipelock"
|
|
||||||
_EGRESS_SUBDIR = "egress"
|
_EGRESS_SUBDIR = "egress"
|
||||||
_GIT_GATE_SUBDIR = "git-gate"
|
_GIT_GATE_SUBDIR = "git-gate"
|
||||||
_SUPERVISE_SUBDIR = "supervise"
|
_SUPERVISE_SUBDIR = "supervise"
|
||||||
@@ -56,8 +56,8 @@ _AGENT_SUBDIR = "agent"
|
|||||||
_METADATA_NAME = "metadata.json"
|
_METADATA_NAME = "metadata.json"
|
||||||
# Live-config dir bind-mounted into the supervise sidecar (read-only).
|
# Live-config dir bind-mounted into the supervise sidecar (read-only).
|
||||||
# Host's apply paths keep these files fresh so supervise's
|
# Host's apply paths keep these files fresh so supervise's
|
||||||
# `list-pipelock-allowlist` / `list-egress-routes` MCP tools
|
# `list-egress-routes` MCP tool returns the current state —
|
||||||
# return the current state — not a snapshot from launch time.
|
# not a snapshot from launch time.
|
||||||
_LIVE_CONFIG_SUBDIR = "live-config"
|
_LIVE_CONFIG_SUBDIR = "live-config"
|
||||||
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
|
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
|
||||||
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
|
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
|
||||||
@@ -135,14 +135,15 @@ def read_metadata(identity: str) -> BottleMetadata | None:
|
|||||||
raw = json.loads(path.read_text())
|
raw = json.loads(path.read_text())
|
||||||
if not isinstance(raw, dict):
|
if not isinstance(raw, dict):
|
||||||
return None
|
return None
|
||||||
|
raw_typed = cast(dict[str, object], raw)
|
||||||
return BottleMetadata(
|
return BottleMetadata(
|
||||||
identity=str(raw.get("identity", identity)),
|
identity=str(raw_typed.get("identity", identity)),
|
||||||
agent_name=str(raw.get("agent_name", "")),
|
agent_name=str(raw_typed.get("agent_name", "")),
|
||||||
cwd=str(raw.get("cwd", "")),
|
cwd=str(raw_typed.get("cwd", "")),
|
||||||
copy_cwd=bool(raw.get("copy_cwd", False)),
|
copy_cwd=bool(raw_typed.get("copy_cwd", False)),
|
||||||
started_at=str(raw.get("started_at", "")),
|
started_at=str(raw_typed.get("started_at", "")),
|
||||||
compose_project=str(raw.get("compose_project", "")),
|
compose_project=str(raw_typed.get("compose_project", "")),
|
||||||
backend=str(raw.get("backend", "")),
|
backend=str(raw_typed.get("backend", "")),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -232,12 +233,6 @@ def transcript_snapshot_dir(identity: str) -> Path:
|
|||||||
# nothing requested preservation.
|
# nothing requested preservation.
|
||||||
|
|
||||||
|
|
||||||
def pipelock_state_dir(identity: str) -> Path:
|
|
||||||
"""State subdir for the pipelock sidecar: pipelock.yaml + the
|
|
||||||
per-bottle CA cert/key. Bind-mount source from chunk 3 onward."""
|
|
||||||
return bottle_state_dir(identity) / _PIPELOCK_SUBDIR
|
|
||||||
|
|
||||||
|
|
||||||
def egress_state_dir(identity: str) -> Path:
|
def egress_state_dir(identity: str) -> Path:
|
||||||
"""State subdir for the egress sidecar: routes.yaml + the
|
"""State subdir for the egress sidecar: routes.yaml + the
|
||||||
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
|
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
|
||||||
@@ -323,7 +318,6 @@ __all__ = [
|
|||||||
"per_bottle_dockerfile",
|
"per_bottle_dockerfile",
|
||||||
"per_bottle_dockerfile_path",
|
"per_bottle_dockerfile_path",
|
||||||
"per_bottle_image_tag",
|
"per_bottle_image_tag",
|
||||||
"pipelock_state_dir",
|
|
||||||
"preserve_marker_path",
|
"preserve_marker_path",
|
||||||
"read_metadata",
|
"read_metadata",
|
||||||
"supervise_state_dir",
|
"supervise_state_dir",
|
||||||
|
|||||||
@@ -30,7 +30,6 @@ semantics open question.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -39,7 +38,6 @@ from ...log import info, warn
|
|||||||
from .bottle_state import (
|
from .bottle_state import (
|
||||||
mark_preserved,
|
mark_preserved,
|
||||||
per_bottle_dockerfile,
|
per_bottle_dockerfile,
|
||||||
per_bottle_dockerfile_path,
|
|
||||||
transcript_snapshot_dir,
|
transcript_snapshot_dir,
|
||||||
write_per_bottle_dockerfile,
|
write_per_bottle_dockerfile,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -7,34 +7,14 @@ two networks, no named volumes.
|
|||||||
|
|
||||||
Pure function. No I/O, no subprocess. Expects every launch-time
|
Pure function. No I/O, no subprocess. Expects every launch-time
|
||||||
field (network names, CA host paths, etc.) on the plan's inner
|
field (network names, CA host paths, etc.) on the plan's inner
|
||||||
plans to be populated; chunks 2+3 own that ordering. Chunk 1 just
|
plans to be populated; chunks 2+3 own that ordering.
|
||||||
encodes the translation so it can be unit-tested in isolation.
|
|
||||||
|
|
||||||
Conditional services follow the plan content (matches the
|
Conditional services follow the plan content:
|
||||||
SDK-call branching in `launch.py` today):
|
|
||||||
|
|
||||||
- pipelock + agent: always.
|
- agent + sidecars bundle: always.
|
||||||
- git-gate: iff plan.git_gate_plan.upstreams.
|
- git-gate: iff plan.git_gate_plan.upstreams.
|
||||||
- egress: iff plan.egress_plan.routes.
|
- egress: iff plan.egress_plan.routes.
|
||||||
- supervise: iff plan.supervise_plan is not None.
|
- supervise: iff plan.supervise_plan is not None.
|
||||||
|
|
||||||
Naming:
|
|
||||||
|
|
||||||
- Compose project: `bot-bottle-<slug>`.
|
|
||||||
- Service names (inside the file): `agent`, `pipelock`,
|
|
||||||
`egress`, `git-gate`, `supervise`.
|
|
||||||
- `container_name:` matches today's pattern
|
|
||||||
(`bot-bottle-<service>-<slug>`) so dashboard/cleanup discovery
|
|
||||||
via the prefix scan keeps working through the transition.
|
|
||||||
- Network aliases preserve the current dial-by-shortname pattern
|
|
||||||
for `egress` / `supervise`, and add the long container-name as
|
|
||||||
an internal-network alias for `pipelock` / `git-gate` so any
|
|
||||||
caller still referencing the long name resolves.
|
|
||||||
|
|
||||||
Sidecars that are built (egress, git-gate, supervise) get a
|
|
||||||
compose `build:` block pointing at the repo Dockerfile; the
|
|
||||||
`image:` tag is set explicitly so cached images on the daemon
|
|
||||||
aren't rebuilt on every up.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -51,7 +31,6 @@ from ...egress import (
|
|||||||
)
|
)
|
||||||
from ...git_gate import GIT_GATE_HOSTNAME
|
from ...git_gate import GIT_GATE_HOSTNAME
|
||||||
from ...log import die, warn
|
from ...log import die, warn
|
||||||
from ...pipelock import PIPELOCK_HOSTNAME
|
|
||||||
from ...supervise import (
|
from ...supervise import (
|
||||||
CURRENT_CONFIG_DIR_IN_AGENT,
|
CURRENT_CONFIG_DIR_IN_AGENT,
|
||||||
QUEUE_DIR_IN_CONTAINER,
|
QUEUE_DIR_IN_CONTAINER,
|
||||||
@@ -63,7 +42,7 @@ from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
|||||||
from .bottle_plan import DockerBottlePlan
|
from .bottle_plan import DockerBottlePlan
|
||||||
from .egress import (
|
from .egress import (
|
||||||
EGRESS_CA_IN_CONTAINER,
|
EGRESS_CA_IN_CONTAINER,
|
||||||
EGRESS_PIPELOCK_CA_IN_CONTAINER,
|
EGRESS_PORT,
|
||||||
)
|
)
|
||||||
from .git_gate import (
|
from .git_gate import (
|
||||||
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
||||||
@@ -71,11 +50,7 @@ from .git_gate import (
|
|||||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||||
GIT_GATE_HOOK_IN_CONTAINER,
|
GIT_GATE_HOOK_IN_CONTAINER,
|
||||||
)
|
)
|
||||||
from .pipelock import (
|
from . import network as network_mod
|
||||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
|
||||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
|
||||||
PIPELOCK_PORT,
|
|
||||||
)
|
|
||||||
from .sidecar_bundle import (
|
from .sidecar_bundle import (
|
||||||
SIDECAR_BUNDLE_DOCKERFILE,
|
SIDECAR_BUNDLE_DOCKERFILE,
|
||||||
SIDECAR_BUNDLE_IMAGE,
|
SIDECAR_BUNDLE_IMAGE,
|
||||||
@@ -91,12 +66,11 @@ def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
"""Render a Compose v2 spec dict from a fully-resolved
|
"""Render a Compose v2 spec dict from a fully-resolved
|
||||||
DockerBottlePlan.
|
DockerBottlePlan.
|
||||||
|
|
||||||
The plan must have its inner plans (`proxy_plan`,
|
The plan must have its inner plans (`git_gate_plan`,
|
||||||
`git_gate_plan`, `egress_plan`, `supervise_plan`) populated
|
`egress_plan`, `supervise_plan`) populated with launch-time
|
||||||
with launch-time fields — network names, CA host paths,
|
fields — network names, CA host paths. The renderer doesn't
|
||||||
pipelock_proxy_url. The renderer doesn't validate; callers
|
validate; callers feed it a fully-resolved plan or get an
|
||||||
feed it a fully-resolved plan or get an incomplete compose
|
incomplete compose spec back.
|
||||||
spec back.
|
|
||||||
"""
|
"""
|
||||||
project = f"bot-bottle-{plan.slug}"
|
project = f"bot-bottle-{plan.slug}"
|
||||||
services: dict[str, Any] = {
|
services: dict[str, Any] = {
|
||||||
@@ -118,11 +92,11 @@ def _networks(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
bridge."""
|
bridge."""
|
||||||
return {
|
return {
|
||||||
"internal": {
|
"internal": {
|
||||||
"name": plan.proxy_plan.internal_network,
|
"name": network_mod.network_name_for_slug(plan.slug),
|
||||||
"internal": True,
|
"internal": True,
|
||||||
},
|
},
|
||||||
"egress": {
|
"egress": {
|
||||||
"name": plan.proxy_plan.egress_network,
|
"name": network_mod.network_egress_name_for_slug(plan.slug),
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -142,29 +116,12 @@ def _bind(host: str | Path, target: str, *, read_only: bool = True) -> dict[str,
|
|||||||
|
|
||||||
def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||||
"""The `sidecars` service: one container per bottle, bundle
|
"""The `sidecars` service: one container per bottle, bundle
|
||||||
image, all four daemons under a Python init supervisor.
|
image, all daemons under a Python init supervisor.
|
||||||
|
|
||||||
Mechanics:
|
Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS` env.
|
||||||
|
egress is always present; git-gate / supervise are conditional.
|
||||||
- Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS`
|
|
||||||
env. pipelock is always present; egress / git-gate /
|
|
||||||
supervise are conditional on the plan.
|
|
||||||
- Volumes are the union of the four daemons' bind-mounts,
|
|
||||||
preserving the same in-container paths so each daemon
|
|
||||||
finds its config / hooks / CA where it expects.
|
|
||||||
- Environment is the union of *daemon-private* env vars
|
|
||||||
(EGRESS_UPSTREAM_PROXY, SUPERVISE_BOTTLE_SLUG, etc).
|
|
||||||
HTTPS_PROXY is NOT propagated here — see the comment in
|
|
||||||
egress_entrypoint.sh; setting it at the container level
|
|
||||||
would route git-gate's git fetches through pipelock,
|
|
||||||
which is wrong.
|
|
||||||
- Network aliases register every legacy short/long
|
|
||||||
hostname (pipelock, egress, git-gate, supervise plus
|
|
||||||
their `bot-bottle-<service>-<slug>` long forms) so
|
|
||||||
the agent's HTTPS_PROXY URL and any other inter-service
|
|
||||||
reference resolves to the bundle.
|
|
||||||
"""
|
"""
|
||||||
daemons: list[str] = ["egress", "pipelock"]
|
daemons: list[str] = ["egress"]
|
||||||
if plan.git_gate_plan.upstreams:
|
if plan.git_gate_plan.upstreams:
|
||||||
daemons.append("git-gate")
|
daemons.append("git-gate")
|
||||||
if plan.supervise_plan is not None:
|
if plan.supervise_plan is not None:
|
||||||
@@ -173,31 +130,15 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
env: list[str] = [f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(daemons)}"]
|
env: list[str] = [f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(daemons)}"]
|
||||||
volumes: list[dict[str, Any]] = []
|
volumes: list[dict[str, Any]] = []
|
||||||
|
|
||||||
# --- pipelock ----------------------------------------------------
|
# --- egress -------------------------------------------------------
|
||||||
pp = plan.proxy_plan
|
|
||||||
volumes += [
|
|
||||||
_bind(pp.yaml_path, "/etc/pipelock.yaml"),
|
|
||||||
_bind(pp.ca_cert_host_path, PIPELOCK_CA_CERT_IN_CONTAINER),
|
|
||||||
_bind(pp.ca_key_host_path, PIPELOCK_CA_KEY_IN_CONTAINER),
|
|
||||||
]
|
|
||||||
|
|
||||||
# --- egress (always part of the bundle; the EGRESS_UPSTREAM_*
|
|
||||||
# env vars + ca bind-mounts are needed iff routes exist; when
|
|
||||||
# the bottle has no routes the egress daemon falls back to its
|
|
||||||
# `regular@9099` mode and is unused) -----------------------------
|
|
||||||
ep = plan.egress_plan
|
ep = plan.egress_plan
|
||||||
|
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
|
||||||
if ep.routes:
|
if ep.routes:
|
||||||
env.append(f"EGRESS_UPSTREAM_PROXY={ep.pipelock_proxy_url}")
|
volumes.append(_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER))
|
||||||
env.append(f"EGRESS_UPSTREAM_CA={EGRESS_PIPELOCK_CA_IN_CONTAINER}")
|
|
||||||
volumes += [
|
|
||||||
_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER),
|
|
||||||
_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER),
|
|
||||||
_bind(ep.pipelock_ca_host_path, EGRESS_PIPELOCK_CA_IN_CONTAINER),
|
|
||||||
]
|
|
||||||
for token_env in sorted(ep.token_env_map.keys()):
|
for token_env in sorted(ep.token_env_map.keys()):
|
||||||
env.append(token_env)
|
env.append(token_env)
|
||||||
|
|
||||||
# --- git-gate ----------------------------------------------------
|
# --- git-gate -----------------------------------------------------
|
||||||
gp = plan.git_gate_plan
|
gp = plan.git_gate_plan
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
volumes += [
|
volumes += [
|
||||||
@@ -217,7 +158,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
||||||
))
|
))
|
||||||
|
|
||||||
# --- supervise ---------------------------------------------------
|
# --- supervise ----------------------------------------------------
|
||||||
sp = plan.supervise_plan
|
sp = plan.supervise_plan
|
||||||
if sp is not None:
|
if sp is not None:
|
||||||
env += [
|
env += [
|
||||||
@@ -232,13 +173,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
"read_only": False,
|
"read_only": False,
|
||||||
})
|
})
|
||||||
|
|
||||||
# Internal-network aliases: the agent reaches each daemon through
|
internal_aliases = [EGRESS_HOSTNAME]
|
||||||
# its short name (pipelock / egress / git-gate / supervise) which
|
|
||||||
# the bundle answers as if it were the daemon itself.
|
|
||||||
internal_aliases = [
|
|
||||||
PIPELOCK_HOSTNAME,
|
|
||||||
EGRESS_HOSTNAME,
|
|
||||||
]
|
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
internal_aliases.append(GIT_GATE_HOSTNAME)
|
internal_aliases.append(GIT_GATE_HOSTNAME)
|
||||||
if sp is not None:
|
if sp is not None:
|
||||||
@@ -263,11 +198,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
|
|
||||||
def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||||
"""Agent container. Runs `sleep infinity`; claude is `docker
|
"""Agent container. Runs `sleep infinity`; claude is `docker
|
||||||
exec -it`'d into it later. No TTY at the container level —
|
exec -it`'d into it later. HTTP_PROXY/HTTPS_PROXY point at the
|
||||||
interactivity is per-exec. HTTP_PROXY/HTTPS_PROXY point at the
|
egress sidecar."""
|
||||||
egress short-alias when an egress is declared, otherwise
|
|
||||||
straight at pipelock's container name. CA trust trio matches
|
|
||||||
the existing launch.py wiring."""
|
|
||||||
proxy_url = _agent_proxy_url(plan)
|
proxy_url = _agent_proxy_url(plan)
|
||||||
no_proxy = _agent_no_proxy(plan)
|
no_proxy = _agent_no_proxy(plan)
|
||||||
env: list[str] = [
|
env: list[str] = [
|
||||||
@@ -319,21 +251,14 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
|
|
||||||
|
|
||||||
def _agent_proxy_url(plan: DockerBottlePlan) -> str:
|
def _agent_proxy_url(plan: DockerBottlePlan) -> str:
|
||||||
"""Pick the agent's HTTP_PROXY. With egress declared, the agent
|
"""Agent's HTTP_PROXY — always points at egress."""
|
||||||
goes through egress (which in turn HTTPS_PROXYs to pipelock on
|
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
|
||||||
its outbound leg). Without egress, the agent talks straight to
|
|
||||||
pipelock."""
|
|
||||||
if plan.egress_plan.routes:
|
|
||||||
from .egress import EGRESS_PORT
|
|
||||||
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
|
|
||||||
return f"http://{PIPELOCK_HOSTNAME}:{PIPELOCK_PORT}"
|
|
||||||
|
|
||||||
|
|
||||||
def _agent_no_proxy(plan: DockerBottlePlan) -> str:
|
def _agent_no_proxy(plan: DockerBottlePlan) -> str:
|
||||||
"""NO_PROXY for the agent. Matches the launch.py rules:
|
"""NO_PROXY for the agent: loopback always; supervise hostname
|
||||||
loopback always, supervise hostname when the supervise sidecar
|
when the supervise sidecar is up (MCP long-poll must bypass
|
||||||
is up (the MCP long-poll pattern needs to bypass pipelock's
|
the egress proxy)."""
|
||||||
idle timeout)."""
|
|
||||||
hosts = ["localhost", "127.0.0.1"]
|
hosts = ["localhost", "127.0.0.1"]
|
||||||
if plan.supervise_plan is not None:
|
if plan.supervise_plan is not None:
|
||||||
hosts.append(SUPERVISE_HOSTNAME)
|
hosts.append(SUPERVISE_HOSTNAME)
|
||||||
|
|||||||
@@ -22,14 +22,8 @@ from ...log import die
|
|||||||
EGRESS_PORT = int(os.environ.get("BOT_BOTTLE_EGRESS_PORT", "9099"))
|
EGRESS_PORT = int(os.environ.get("BOT_BOTTLE_EGRESS_PORT", "9099"))
|
||||||
|
|
||||||
# In-container path for mitmproxy's CA. The format is a single PEM
|
# In-container path for mitmproxy's CA. The format is a single PEM
|
||||||
# file holding BOTH the cert and the private key, concatenated. The
|
# file holding BOTH the cert and the private key, concatenated.
|
||||||
# upstream-trust CA (pipelock's, so egress trusts the upstream
|
|
||||||
# leg) is a separate file because pipelock keeps a different CA on
|
|
||||||
# its end.
|
|
||||||
EGRESS_CA_IN_CONTAINER = "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem"
|
EGRESS_CA_IN_CONTAINER = "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem"
|
||||||
EGRESS_PIPELOCK_CA_IN_CONTAINER = (
|
|
||||||
"/home/mitmproxy/.mitmproxy/pipelock-ca.pem"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
||||||
@@ -42,16 +36,8 @@ def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
|||||||
trust store by `provision_ca` so the agent trusts the bumped
|
trust store by `provision_ca` so the agent trusts the bumped
|
||||||
CONNECT cert egress presents.
|
CONNECT cert egress presents.
|
||||||
|
|
||||||
Why openssl req (not the pipelock binary's `tls init`):
|
openssl req's `subjectKeyIdentifier=hash` extension uses
|
||||||
pipelock's CA generator stamps a non-standard `Subject Key
|
SHA-1(pubkey), matching mitmproxy's AKI computation on leaves.
|
||||||
Identifier` on the CA (random rather than SHA-1 of the pubkey).
|
|
||||||
mitmproxy computes the `Authority Key Identifier` on each leaf
|
|
||||||
it mints as SHA-1(issuer's pubkey). openssl's chain validator
|
|
||||||
uses the leaf's AKI to find the issuer cert by SKI; pipelock's
|
|
||||||
SKI doesn't match → openssl reports "unable to get local issuer
|
|
||||||
certificate" even though the CA is right there in the trust
|
|
||||||
store. openssl req's `subjectKeyIdentifier=hash` extension uses
|
|
||||||
SHA-1(pubkey), matching mitmproxy's computation.
|
|
||||||
|
|
||||||
Both files live under `<stage_dir>/egress-ca/` (mode 644 —
|
Both files live under `<stage_dir>/egress-ca/` (mode 644 —
|
||||||
`docker cp` preserves the mode into the container, where the
|
`docker cp` preserves the mode into the container, where the
|
||||||
|
|||||||
@@ -1,86 +1,25 @@
|
|||||||
"""Host-side helper to apply a routes.yaml change to a running
|
"""Host-side helper for egress sidecar inspection (issue #198).
|
||||||
egress sidecar (PRD 0014 retargeted by PRD 0017 chunk 3).
|
|
||||||
|
|
||||||
Used by the supervise dashboard when the operator approves an
|
`_merge_single_route`, `add_route`, and `apply_routes_change` were
|
||||||
egress-block proposal (or runs the operator-initiated
|
removed when the egress-block MCP tool was dropped. The remaining
|
||||||
`routes edit <bottle>` verb). Fetches the current routes.yaml via
|
helpers support runtime inspection and validation of the routes file
|
||||||
`docker exec cat`, validates the new content, writes it into the
|
without modifying it at runtime.
|
||||||
sidecar via `docker cp`, then `docker kill --signal HUP` to make
|
|
||||||
the addon reload without dropping connections.
|
|
||||||
|
|
||||||
Also mirrors the new route hosts into pipelock's hostname allowlist
|
|
||||||
so the downstream leg lets them through — egress enforces
|
|
||||||
the path-aware allowlist on the agent leg, pipelock enforces the
|
|
||||||
hostname allowlist + DLP body scan on the upstream leg, and a
|
|
||||||
host added to one must be in the other or the request 403s
|
|
||||||
somewhere along the chain.
|
|
||||||
|
|
||||||
Raises EgressApplyError on any failure — the dashboard
|
|
||||||
surfaces the message and keeps the proposal pending so the
|
|
||||||
operator can retry.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import subprocess
|
import subprocess
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER
|
from ...egress import EGRESS_ROUTES_IN_CONTAINER
|
||||||
from ...egress_addon_core import load_routes
|
from ...egress_addon_core import load_routes
|
||||||
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
|
|
||||||
from .bottle_state import egress_state_dir
|
|
||||||
from .sidecar_bundle import sidecar_bundle_container_name
|
from .sidecar_bundle import sidecar_bundle_container_name
|
||||||
from .pipelock_apply import (
|
|
||||||
PipelockApplyError,
|
|
||||||
apply_allowlist_change,
|
|
||||||
fetch_current_allowlist,
|
|
||||||
parse_allowlist_content,
|
|
||||||
render_allowlist_content,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _render_routes_payload(routes_list: list[dict[str, object]]) -> str:
|
|
||||||
"""Render a list-of-dicts routes payload as YAML matching the
|
|
||||||
shape `egress_render_routes` produces. The apply path
|
|
||||||
round-trips current routes.yaml through this so the file the
|
|
||||||
sidecar sees stays in the YAML format the addon expects."""
|
|
||||||
if not routes_list:
|
|
||||||
return "routes: []\n"
|
|
||||||
lines: list[str] = ["routes:"]
|
|
||||||
for entry in routes_list:
|
|
||||||
host = str(entry.get("host", ""))
|
|
||||||
lines.append(f' - host: "{host}"')
|
|
||||||
auth_scheme = entry.get("auth_scheme")
|
|
||||||
token_env = entry.get("token_env")
|
|
||||||
if auth_scheme and token_env:
|
|
||||||
lines.append(f' auth_scheme: "{auth_scheme}"')
|
|
||||||
lines.append(f' token_env: "{token_env}"')
|
|
||||||
paths = entry.get("path_allowlist") or []
|
|
||||||
if paths:
|
|
||||||
lines.append(" path_allowlist:")
|
|
||||||
for p in paths:
|
|
||||||
lines.append(f' - "{p}"')
|
|
||||||
return "\n".join(lines) + "\n"
|
|
||||||
|
|
||||||
|
|
||||||
def _egress_routes_host_path(slug: str) -> Path:
|
|
||||||
"""The bind-mount source for the egress sidecar's routes.yaml.
|
|
||||||
Must match what egress.prepare wrote at chunk-2 paths."""
|
|
||||||
return egress_state_dir(slug) / "egress_routes.yaml"
|
|
||||||
|
|
||||||
|
|
||||||
class EgressApplyError(RuntimeError):
|
class EgressApplyError(RuntimeError):
|
||||||
"""Raised when fetch / apply fails. Caller renders to the
|
pass
|
||||||
operator; does not crash the dashboard."""
|
|
||||||
|
|
||||||
|
|
||||||
def fetch_current_routes(slug: str) -> str:
|
def fetch_current_routes(slug: str) -> str:
|
||||||
"""Read the live routes.yaml from the running egress sidecar
|
|
||||||
for `slug`. Returns the file content as a string. Raises
|
|
||||||
EgressApplyError if the sidecar isn't reachable or the read
|
|
||||||
fails."""
|
|
||||||
container = sidecar_bundle_container_name(slug)
|
container = sidecar_bundle_container_name(slug)
|
||||||
r = subprocess.run(
|
r = subprocess.run(
|
||||||
["docker", "exec", container, "cat", EGRESS_ROUTES_IN_CONTAINER],
|
["docker", "exec", container, "cat", EGRESS_ROUTES_IN_CONTAINER],
|
||||||
@@ -95,9 +34,6 @@ def fetch_current_routes(slug: str) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def validate_routes_content(content: str) -> None:
|
def validate_routes_content(content: str) -> None:
|
||||||
"""Syntactic check before SIGHUP — the addon's reload also
|
|
||||||
validates, but failing here keeps the old routes live and gives
|
|
||||||
the operator a clearer error than the addon's stderr line."""
|
|
||||||
try:
|
try:
|
||||||
load_routes(content)
|
load_routes(content)
|
||||||
except ValueError as e:
|
except ValueError as e:
|
||||||
@@ -106,238 +42,8 @@ def validate_routes_content(content: str) -> None:
|
|||||||
) from e
|
) from e
|
||||||
|
|
||||||
|
|
||||||
def _hosts_in_routes(content: str) -> list[str]:
|
|
||||||
"""Extract the host list from a routes.yaml content string.
|
|
||||||
Uses the addon's own parser so any host the addon will match on
|
|
||||||
also lands in pipelock's allowlist. Returns sorted+deduped."""
|
|
||||||
try:
|
|
||||||
routes = load_routes(content)
|
|
||||||
except ValueError as e:
|
|
||||||
raise EgressApplyError(
|
|
||||||
f"proposed routes.yaml is not valid: {e}"
|
|
||||||
) from e
|
|
||||||
return sorted({r.host for r in routes if r.host})
|
|
||||||
|
|
||||||
|
|
||||||
# Pipelock's allowlist parser accepts only literal hostnames:
|
|
||||||
# `[A-Za-z0-9_.-]+`. Anything else (wildcards, IPv6 literals,
|
|
||||||
# stray characters) is silently dropped from the mirror so the
|
|
||||||
# pipelock apply doesn't fail parse before the new yaml is even
|
|
||||||
# written. The dropped hosts stay on egress's route table —
|
|
||||||
# but the addon does exact-host match only, so they'll never
|
|
||||||
# match anything either. (Wildcard host matching was removed —
|
|
||||||
# see `match_route` in egress_addon_core for the rationale.)
|
|
||||||
_PIPELOCK_HOST_RE = re.compile(r"^[A-Za-z0-9_.-]+$")
|
|
||||||
|
|
||||||
|
|
||||||
def _pipelock_safe_hosts(hosts: list[str]) -> list[str]:
|
|
||||||
"""Drop any host pipelock's allowlist parser would reject.
|
|
||||||
Order preserved."""
|
|
||||||
return [h for h in hosts if _PIPELOCK_HOST_RE.match(h)]
|
|
||||||
|
|
||||||
|
|
||||||
def _mirror_hosts_to_pipelock(slug: str, hosts: list[str]) -> None:
|
|
||||||
"""Ensure every pipelock-compatible `hosts` entry is on
|
|
||||||
pipelock's allowlist. Fetches pipelock's current allowlist,
|
|
||||||
merges, re-applies. Hosts pipelock can't represent (wildcards,
|
|
||||||
etc.) are silently skipped — they stay live on egress
|
|
||||||
but aren't enforced at pipelock. No-op if every host is already
|
|
||||||
present (apply still restarts pipelock if any host is new).
|
|
||||||
Raises EgressApplyError on pipelock failures so the
|
|
||||||
caller's diff/audit reflects the half-state."""
|
|
||||||
safe_hosts = _pipelock_safe_hosts(hosts)
|
|
||||||
try:
|
|
||||||
current = fetch_current_allowlist(slug)
|
|
||||||
existing = parse_allowlist_content(current)
|
|
||||||
merged = sorted(set(existing) | set(safe_hosts))
|
|
||||||
if merged == sorted(existing):
|
|
||||||
return # nothing to add
|
|
||||||
apply_allowlist_change(slug, render_allowlist_content(merged))
|
|
||||||
except PipelockApplyError as e:
|
|
||||||
# Mirror runs BEFORE the egress write, so egress
|
|
||||||
# is unchanged on this failure path. Report it as a
|
|
||||||
# pipelock-side problem so the operator looks in the right
|
|
||||||
# place; their `pipelock edit` flow can repair manually.
|
|
||||||
raise EgressApplyError(
|
|
||||||
f"pipelock allowlist mirror failed (egress NOT "
|
|
||||||
f"updated): {e}. Fix pipelock's allowlist manually with "
|
|
||||||
f"`pipelock edit <bottle>` then retry the proposal."
|
|
||||||
) from e
|
|
||||||
|
|
||||||
|
|
||||||
def apply_routes_change(slug: str, new_content: str) -> tuple[str, str]:
|
|
||||||
"""Apply `new_content` to the egress sidecar for `slug`:
|
|
||||||
1. Fetch current routes.yaml (for the before-diff).
|
|
||||||
2. Validate the new content via the addon's own parser.
|
|
||||||
3. Mirror the route hosts onto pipelock's allowlist (so the
|
|
||||||
downstream hostname gate lets them through).
|
|
||||||
4. Write to a temp file, `docker cp` into the egress
|
|
||||||
sidecar.
|
|
||||||
5. `docker kill --signal HUP` so the addon reloads.
|
|
||||||
|
|
||||||
Order matters: pipelock first, then egress. If the
|
|
||||||
pipelock step fails, egress hasn't been touched and the
|
|
||||||
old routes stay live. If the egress step fails after
|
|
||||||
pipelock succeeded, pipelock has the host in its allowlist but
|
|
||||||
egress doesn't enforce it yet — harmless extra-permissive
|
|
||||||
state at pipelock, and a re-approval will land the egress
|
|
||||||
side.
|
|
||||||
|
|
||||||
Returns (before, after) where `after` == `new_content`. Raises
|
|
||||||
EgressApplyError on any step."""
|
|
||||||
container = sidecar_bundle_container_name(slug)
|
|
||||||
before = fetch_current_routes(slug)
|
|
||||||
validate_routes_content(new_content)
|
|
||||||
|
|
||||||
# Pipelock mirror first — if it fails, egress stays intact
|
|
||||||
# and the operator gets a clear error about the half-state.
|
|
||||||
_mirror_hosts_to_pipelock(slug, _hosts_in_routes(new_content))
|
|
||||||
|
|
||||||
# routes.yaml is bind-mounted into the egress container as a
|
|
||||||
# SINGLE FILE. Docker single-file bind mounts pin the source
|
|
||||||
# inode at mount time; write-temp-then-rename swaps the inode
|
|
||||||
# on the host, which leaves the container's mount pointing at
|
|
||||||
# the now-orphaned old inode (so the SIGHUP'd reload re-reads
|
|
||||||
# unchanged content). Write in-place instead. Lose file-level
|
|
||||||
# atomicity, but the apply path issues SIGHUP only AFTER the
|
|
||||||
# write returns, and the addon's `load_routes` raises
|
|
||||||
# `ValueError` on a partial read and keeps the previous
|
|
||||||
# in-memory routes — so a SIGHUP that hypothetically raced an
|
|
||||||
# in-flight write is non-disruptive.
|
|
||||||
target = _egress_routes_host_path(slug)
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
target.write_text(new_content)
|
|
||||||
# mitmproxy in the container reads through the bind mount as
|
|
||||||
# uid 1000; the host file has to be world-readable for that
|
|
||||||
# read to succeed (parent dir at 0o700 still restricts who
|
|
||||||
# can reach the file on the host). Routes content is not
|
|
||||||
# secret — tokens live in the container's environ — so 0o644
|
|
||||||
# is the right trade-off.
|
|
||||||
target.chmod(0o644)
|
|
||||||
sig = subprocess.run(
|
|
||||||
["docker", "kill", "--signal", "HUP", container],
|
|
||||||
capture_output=True, text=True, check=False,
|
|
||||||
)
|
|
||||||
if sig.returncode != 0:
|
|
||||||
raise EgressApplyError(
|
|
||||||
f"failed to SIGHUP {container}: "
|
|
||||||
f"{(sig.stderr or '').strip()}"
|
|
||||||
)
|
|
||||||
|
|
||||||
return before, new_content
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_single_route(
|
|
||||||
current_yaml: str, new_route: dict[str, object],
|
|
||||||
) -> str:
|
|
||||||
"""Merge a single proposed route into the current routes.yaml
|
|
||||||
content, returning the merged YAML string.
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
- If `new_route['host']` is NOT in the current routes →
|
|
||||||
append the route.
|
|
||||||
- If the host IS already present → union the path_allowlist
|
|
||||||
entries (proposed ∪ existing). The existing `auth_scheme`
|
|
||||||
and `token_env` are preserved — agent-proposed auth changes
|
|
||||||
on an existing host are ignored, matching the tool's
|
|
||||||
documented semantics.
|
|
||||||
|
|
||||||
Round-trips the file through `yaml_subset` (the same parser
|
|
||||||
the addon uses), so the merged output is in the YAML format
|
|
||||||
the sidecar reads. Token VALUES never appear here; the routes
|
|
||||||
file carries only env-var slot NAMES."""
|
|
||||||
try:
|
|
||||||
cfg = parse_yaml_subset(current_yaml)
|
|
||||||
except YamlSubsetError as e:
|
|
||||||
raise EgressApplyError(
|
|
||||||
f"current routes.yaml is not valid YAML: {e}"
|
|
||||||
) from e
|
|
||||||
routes = cfg.get("routes")
|
|
||||||
if not isinstance(routes, list):
|
|
||||||
raise EgressApplyError(
|
|
||||||
"current routes.yaml: 'routes' is not a list"
|
|
||||||
)
|
|
||||||
|
|
||||||
new_host = str(new_route.get("host", "")).lower()
|
|
||||||
if not new_host:
|
|
||||||
raise EgressApplyError(
|
|
||||||
"proposed route is missing 'host'"
|
|
||||||
)
|
|
||||||
|
|
||||||
proposed_paths = list(new_route.get("path_allowlist") or [])
|
|
||||||
|
|
||||||
# Look for an existing entry with the same host (case-insensitive).
|
|
||||||
for entry in routes:
|
|
||||||
if not isinstance(entry, dict):
|
|
||||||
continue
|
|
||||||
if str(entry.get("host", "")).lower() == new_host:
|
|
||||||
# Merge path_allowlist: union proposed + existing, ordered
|
|
||||||
# by first-seen so existing paths stay in original order.
|
|
||||||
existing_paths: list[str] = list(entry.get("path_allowlist") or [])
|
|
||||||
seen = {p: None for p in existing_paths}
|
|
||||||
for p in proposed_paths:
|
|
||||||
seen.setdefault(p, None)
|
|
||||||
merged_paths = list(seen.keys())
|
|
||||||
if merged_paths:
|
|
||||||
entry["path_allowlist"] = merged_paths
|
|
||||||
# Preserve existing auth — tool description says agent-
|
|
||||||
# proposed auth on an existing host is ignored.
|
|
||||||
break
|
|
||||||
else:
|
|
||||||
# Host not present; build a new route entry from the
|
|
||||||
# proposed fields. Need to assign a token_env slot if
|
|
||||||
# `auth` was proposed (otherwise the addon's parser rejects
|
|
||||||
# a half-set auth pair). Slots: count existing slots, pick
|
|
||||||
# the next free index.
|
|
||||||
entry = {"host": new_route["host"]}
|
|
||||||
if proposed_paths:
|
|
||||||
entry["path_allowlist"] = proposed_paths
|
|
||||||
auth = new_route.get("auth")
|
|
||||||
if isinstance(auth, dict) and auth.get("scheme") and auth.get("token_ref"):
|
|
||||||
existing_slots = sorted({
|
|
||||||
str(r.get("token_env"))
|
|
||||||
for r in routes
|
|
||||||
if isinstance(r, dict) and r.get("token_env")
|
|
||||||
})
|
|
||||||
next_idx = len(existing_slots)
|
|
||||||
entry["auth_scheme"] = str(auth["scheme"])
|
|
||||||
entry["token_env"] = f"EGRESS_TOKEN_{next_idx}"
|
|
||||||
# NOTE: the addon reads token VALUES from its container's
|
|
||||||
# environ keyed by token_env. A newly-added auth route at
|
|
||||||
# runtime points at a slot that has no env value → the
|
|
||||||
# addon will 403 with "token env unset" until the operator
|
|
||||||
# arranges for the value to land in the container's env.
|
|
||||||
# Recording this here so the operator-facing diff carries
|
|
||||||
# the slot name they'll need to provision.
|
|
||||||
routes.append(entry)
|
|
||||||
|
|
||||||
return _render_routes_payload(routes)
|
|
||||||
|
|
||||||
|
|
||||||
def add_route(slug: str, proposed_route_json: str) -> tuple[str, str]:
|
|
||||||
"""Apply a single-route addition to the egress. Parses the
|
|
||||||
agent's proposed route, fetches the current routes file, merges,
|
|
||||||
and applies via `apply_routes_change`. Returns (before, after)
|
|
||||||
full-file content for the audit log."""
|
|
||||||
try:
|
|
||||||
proposed = json.loads(proposed_route_json)
|
|
||||||
except json.JSONDecodeError as e:
|
|
||||||
raise EgressApplyError(
|
|
||||||
f"proposed route is not valid JSON: {e}"
|
|
||||||
) from e
|
|
||||||
if not isinstance(proposed, dict):
|
|
||||||
raise EgressApplyError(
|
|
||||||
"proposed route must be a JSON object"
|
|
||||||
)
|
|
||||||
current = fetch_current_routes(slug)
|
|
||||||
merged = _merge_single_route(current, proposed)
|
|
||||||
return apply_routes_change(slug, merged)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"EgressApplyError",
|
"EgressApplyError",
|
||||||
"add_route",
|
|
||||||
"apply_routes_change",
|
|
||||||
"fetch_current_routes",
|
"fetch_current_routes",
|
||||||
"validate_routes_content",
|
"validate_routes_content",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -6,16 +6,10 @@ The flow is:
|
|||||||
|
|
||||||
1. Build the agent's base + derived image (compose builds the
|
1. Build the agent's base + derived image (compose builds the
|
||||||
sidecar images via the `build:` directive on first up).
|
sidecar images via the `build:` directive on first up).
|
||||||
2. Pre-create the per-bottle networks. We do this outside compose
|
2. Mint the per-bottle egress CA (chunk 2 writes it under
|
||||||
so we can inspect the assigned internal CIDR and embed it in
|
state/<slug>/egress/).
|
||||||
pipelock's yaml (compose's `external: true` lets the compose
|
3. Populate the inner plans with launch-time fields so the
|
||||||
file reference these pre-existing networks).
|
renderer can read network names, CA paths.
|
||||||
3. Mint the per-bottle CAs (chunk 2 writes them under
|
|
||||||
state/<slug>/{pipelock,egress}/).
|
|
||||||
4. Re-render pipelock yaml with the now-known internal CIDR so
|
|
||||||
the SSRF allowlist exempts the bottle's own subnet.
|
|
||||||
5. Populate the inner plans with launch-time fields so the
|
|
||||||
renderer can read network names, CA paths, pipelock URL.
|
|
||||||
6. Render the compose spec, write it to
|
6. Render the compose spec, write it to
|
||||||
state/<slug>/docker-compose.yml, write metadata.json.
|
state/<slug>/docker-compose.yml, write metadata.json.
|
||||||
7. `docker compose up -d` (token + OAuth values flow into the
|
7. `docker compose up -d` (token + OAuth values flow into the
|
||||||
@@ -43,6 +37,7 @@ from pathlib import Path
|
|||||||
from typing import Callable, Generator
|
from typing import Callable, Generator
|
||||||
|
|
||||||
from ...egress import egress_resolve_token_values
|
from ...egress import egress_resolve_token_values
|
||||||
|
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||||
from ...log import info, warn
|
from ...log import info, warn
|
||||||
from . import network as network_mod
|
from . import network as network_mod
|
||||||
from . import util as docker_mod
|
from . import util as docker_mod
|
||||||
@@ -51,7 +46,7 @@ from .bottle_plan import DockerBottlePlan
|
|||||||
from .bottle_state import (
|
from .bottle_state import (
|
||||||
bottle_state_dir,
|
bottle_state_dir,
|
||||||
egress_state_dir,
|
egress_state_dir,
|
||||||
pipelock_state_dir,
|
git_gate_state_dir,
|
||||||
)
|
)
|
||||||
from .compose import (
|
from .compose import (
|
||||||
bottle_plan_to_compose,
|
bottle_plan_to_compose,
|
||||||
@@ -64,10 +59,6 @@ from .compose import (
|
|||||||
write_compose_file,
|
write_compose_file,
|
||||||
)
|
)
|
||||||
from .egress import egress_tls_init
|
from .egress import egress_tls_init
|
||||||
from .pipelock import (
|
|
||||||
BUNDLE_LOCAL_PIPELOCK_URL,
|
|
||||||
pipelock_tls_init,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# Where the repo root lives, for `docker build` context. Computed once.
|
# Where the repo root lives, for `docker build` context. Computed once.
|
||||||
@@ -78,20 +69,26 @@ _REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
|||||||
def launch(
|
def launch(
|
||||||
plan: DockerBottlePlan,
|
plan: DockerBottlePlan,
|
||||||
*,
|
*,
|
||||||
provision: Callable[[DockerBottlePlan, str], str | None],
|
provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None],
|
||||||
) -> Generator[DockerBottle, None, None]:
|
) -> Generator[DockerBottle, None, None]:
|
||||||
"""Build, launch, and provision a Docker bottle via compose.
|
"""Build, launch, and provision a Docker bottle via compose.
|
||||||
Teardown on exit."""
|
Teardown on exit."""
|
||||||
stack = ExitStack()
|
stack = ExitStack()
|
||||||
|
|
||||||
|
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
|
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||||
|
|
||||||
def teardown() -> None:
|
def teardown() -> None:
|
||||||
try:
|
try:
|
||||||
stack.close()
|
stack.close()
|
||||||
except BaseException as exc:
|
except BaseException as exc: # noqa: W0718 — teardown must not fail
|
||||||
warn(
|
warn(
|
||||||
f"teardown failed for container {plan.container_name}"
|
f"teardown failed for container {plan.container_name}"
|
||||||
f" (compose-down): {exc!r}"
|
f" (compose-down): {exc!r}"
|
||||||
)
|
)
|
||||||
|
revoke_git_gate_provisioned_keys(
|
||||||
|
_bottle_for_revoke, _git_gate_dir_for_revoke
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Step 1: agent image build. Sidecar images get built lazily by
|
# Step 1: agent image build. Sidecar images get built lazily by
|
||||||
@@ -105,35 +102,13 @@ def launch(
|
|||||||
plan.derived_image, plan.image, plan.workspace_plan
|
plan.derived_image, plan.image, plan.workspace_plan
|
||||||
)
|
)
|
||||||
|
|
||||||
# Networks: compose-managed. The names are derived
|
|
||||||
# deterministically from the slug so the renderer can put
|
|
||||||
# them on the services and `compose up` creates them with
|
|
||||||
# those names. The empirical spike confirmed pipelock's
|
|
||||||
# SSRF guard only checks proxied-request destinations, not
|
|
||||||
# source IPs — so the bottle's own internal CIDR doesn't
|
|
||||||
# need to be in `ssrf.ip_allowlist`. Pre-create + CIDR
|
|
||||||
# introspection are gone; compose owns the network
|
|
||||||
# lifecycle.
|
|
||||||
internal_network = network_mod.network_name_for_slug(plan.slug)
|
internal_network = network_mod.network_name_for_slug(plan.slug)
|
||||||
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
|
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
|
||||||
|
|
||||||
# Mint per-bottle CAs into state/<slug>/{pipelock,egress}/.
|
|
||||||
ca_cert_host, ca_key_host = pipelock_tls_init(pipelock_state_dir(plan.slug))
|
|
||||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||||
egress_state_dir(plan.slug),
|
egress_state_dir(plan.slug),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Populate launch-time fields on every inner plan so the
|
|
||||||
# renderer reads concrete network names, CA paths, and
|
|
||||||
# pipelock URL.
|
|
||||||
proxy_plan = dataclasses.replace(
|
|
||||||
plan.proxy_plan,
|
|
||||||
internal_network=internal_network,
|
|
||||||
internal_network_cidr="",
|
|
||||||
egress_network=egress_network,
|
|
||||||
ca_cert_host_path=ca_cert_host,
|
|
||||||
ca_key_host_path=ca_key_host,
|
|
||||||
)
|
|
||||||
git_gate_plan = plan.git_gate_plan
|
git_gate_plan = plan.git_gate_plan
|
||||||
if git_gate_plan.upstreams:
|
if git_gate_plan.upstreams:
|
||||||
git_gate_plan = dataclasses.replace(
|
git_gate_plan = dataclasses.replace(
|
||||||
@@ -141,17 +116,13 @@ def launch(
|
|||||||
internal_network=internal_network,
|
internal_network=internal_network,
|
||||||
egress_network=egress_network,
|
egress_network=egress_network,
|
||||||
)
|
)
|
||||||
egress_plan = plan.egress_plan
|
egress_plan = dataclasses.replace(
|
||||||
if egress_plan.routes:
|
plan.egress_plan,
|
||||||
egress_plan = dataclasses.replace(
|
internal_network=internal_network,
|
||||||
egress_plan,
|
egress_network=egress_network,
|
||||||
internal_network=internal_network,
|
mitmproxy_ca_host_path=egress_ca_host,
|
||||||
egress_network=egress_network,
|
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||||
mitmproxy_ca_host_path=egress_ca_host,
|
)
|
||||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
|
||||||
pipelock_ca_host_path=ca_cert_host,
|
|
||||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
|
||||||
)
|
|
||||||
supervise_plan = plan.supervise_plan
|
supervise_plan = plan.supervise_plan
|
||||||
if supervise_plan is not None:
|
if supervise_plan is not None:
|
||||||
supervise_plan = dataclasses.replace(
|
supervise_plan = dataclasses.replace(
|
||||||
@@ -160,7 +131,6 @@ def launch(
|
|||||||
)
|
)
|
||||||
plan = dataclasses.replace(
|
plan = dataclasses.replace(
|
||||||
plan,
|
plan,
|
||||||
proxy_plan=proxy_plan,
|
|
||||||
git_gate_plan=git_gate_plan,
|
git_gate_plan=git_gate_plan,
|
||||||
egress_plan=egress_plan,
|
egress_plan=egress_plan,
|
||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
@@ -200,19 +170,21 @@ def launch(
|
|||||||
compose_dump_logs, project, compose_file, compose_log_path(state_dir),
|
compose_dump_logs, project, compose_file, compose_log_path(state_dir),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Step 8: provision. Unchanged — uses `docker exec` against
|
# Step 8: provision. Create the bottle first so provisioners
|
||||||
# the agent container by its known name.
|
# can use bottle.exec / bottle.cp_in; set the prompt path
|
||||||
prompt_path = provision(plan, plan.container_name)
|
# returned by provision_prompt after the fact.
|
||||||
|
bottle = DockerBottle(
|
||||||
|
plan.container_name,
|
||||||
|
teardown,
|
||||||
|
None,
|
||||||
|
agent_command=plan.agent_command,
|
||||||
|
agent_prompt_mode=plan.agent_prompt_mode,
|
||||||
|
)
|
||||||
|
bottle.prompt_path = provision(plan, bottle)
|
||||||
|
|
||||||
# Step 9: yield. exec_agent continues to use `docker exec -it`
|
# Step 9: yield. exec_agent continues to use `docker exec -it`
|
||||||
# — the agent runs `sleep infinity` per the renderer's
|
# — the agent runs `sleep infinity` per the renderer's
|
||||||
# service spec.
|
# service spec.
|
||||||
yield DockerBottle(
|
yield bottle
|
||||||
plan.container_name,
|
|
||||||
teardown,
|
|
||||||
prompt_path,
|
|
||||||
agent_command=plan.agent_command,
|
|
||||||
agent_prompt_mode=plan.agent_prompt_mode,
|
|
||||||
)
|
|
||||||
finally:
|
finally:
|
||||||
teardown()
|
teardown()
|
||||||
|
|||||||
@@ -1,11 +1,10 @@
|
|||||||
"""Docker network plumbing for the per-agent egress topology.
|
"""Docker network plumbing for the per-agent egress topology.
|
||||||
|
|
||||||
The agent container sits on a Docker `--internal` network (no default
|
The agent container sits on a Docker `--internal` network (no default
|
||||||
gateway). Pipelock straddles that network and a per-agent user-defined
|
gateway). Egress straddles that network and a per-agent user-defined
|
||||||
bridge for upstream egress. We deliberately do NOT use Docker's legacy
|
bridge for upstream traffic. We deliberately do NOT use Docker's legacy
|
||||||
`bridge` network because only user-defined bridges run Docker's
|
`bridge` network because only user-defined bridges run Docker's
|
||||||
embedded DNS resolver, which pipelock needs to resolve api.anthropic.com
|
embedded DNS resolver, which egress needs to resolve upstream hostnames.
|
||||||
and similar upstream hostnames.
|
|
||||||
|
|
||||||
Naming: bot-bottle-net-<slug> (internal),
|
Naming: bot-bottle-net-<slug> (internal),
|
||||||
bot-bottle-egress-<slug> (egress). Numeric suffix on conflict
|
bot-bottle-egress-<slug> (egress). Numeric suffix on conflict
|
||||||
@@ -77,20 +76,12 @@ def network_create_internal(slug: str) -> str:
|
|||||||
|
|
||||||
def network_create_egress(slug: str) -> str:
|
def network_create_egress(slug: str) -> str:
|
||||||
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`)
|
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`)
|
||||||
so the pipelock sidecar has working DNS for upstream hostnames."""
|
so the egress sidecar has working DNS for upstream hostnames."""
|
||||||
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
|
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
|
||||||
|
|
||||||
|
|
||||||
def network_inspect_cidr(name: str) -> str:
|
def network_inspect_cidr(name: str) -> str:
|
||||||
"""Return the IPv4 CIDR Docker assigned to a user-defined network.
|
"""Return the IPv4 CIDR Docker assigned to a user-defined network."""
|
||||||
|
|
||||||
Used by pipelock's SSRF guard exception: the bottle's internal
|
|
||||||
network sits in RFC1918 space, so pipelock's `internal:` list
|
|
||||||
would block any agent request whose destination resolves there
|
|
||||||
— including the cred-proxy sidecar's address. Adding the
|
|
||||||
network's CIDR to pipelock's `ssrf.ip_allowlist` lets traffic
|
|
||||||
targeted at the bottle's own sidecars through while pipelock
|
|
||||||
still body-scans and api_allowlist-gates as usual."""
|
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
["docker", "network", "inspect",
|
["docker", "network", "inspect",
|
||||||
"--format", "{{range .IPAM.Config}}{{.Subnet}}{{end}}", name],
|
"--format", "{{range .IPAM.Config}}{{.Subnet}}{{end}}", name],
|
||||||
|
|||||||
@@ -1,81 +0,0 @@
|
|||||||
"""Docker-side pipelock helpers: image pin, container naming, and
|
|
||||||
the one-shot `pipelock tls init` host-side CA mint. The
|
|
||||||
prepare-time YAML rendering itself lives on the platform-neutral
|
|
||||||
`PipelockProxy` ABC — backends instantiate it directly.
|
|
||||||
|
|
||||||
The per-container `.start()` / `.stop()` lifecycle was deleted in
|
|
||||||
PRD 0024 chunk 3; compose-up owns the container lifecycle (PRD
|
|
||||||
0018) and the bundle path (PRD 0024) collapses pipelock + egress
|
|
||||||
+ git-gate + supervise into one container."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ...log import die
|
|
||||||
# Re-exported for the compose renderer + smolmachines launch step
|
|
||||||
# (they used to import these from this module before they moved to
|
|
||||||
# the platform-neutral pipelock module).
|
|
||||||
from ...pipelock import ( # noqa: F401
|
|
||||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
|
||||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# Pipelock image, pinned by digest. The digest is the multi-arch image
|
|
||||||
# index for ghcr.io/luckypipewrench/pipelock:2.3.0.
|
|
||||||
PIPELOCK_IMAGE = os.environ.get(
|
|
||||||
"BOT_BOTTLE_PIPELOCK_IMAGE",
|
|
||||||
"ghcr.io/luckypipewrench/pipelock@sha256:3b1a39417b98406ddc5dc2d8fcb42865ddc0c68a43d355db55f0f8cb06bc6de9",
|
|
||||||
)
|
|
||||||
|
|
||||||
# Listening port for pipelock's forward proxy.
|
|
||||||
PIPELOCK_PORT = os.environ.get("BOT_BOTTLE_PIPELOCK_PORT", "8888")
|
|
||||||
|
|
||||||
|
|
||||||
# The URL egress dials for its upstream HTTPS_PROXY. egress and
|
|
||||||
# pipelock share the same container's network namespace inside the
|
|
||||||
# sidecar bundle, so loopback reaches pipelock directly — no docker
|
|
||||||
# DNS aliases involved.
|
|
||||||
BUNDLE_LOCAL_PIPELOCK_URL = f"http://127.0.0.1:{PIPELOCK_PORT}"
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
|
||||||
"""Generate a fresh per-bottle CA via a one-shot pipelock container.
|
|
||||||
|
|
||||||
Runs `pipelock tls init` against a host-mounted scratch dir, leaving
|
|
||||||
`ca.pem` (public cert, mode 600) and `ca-key.pem` (private key, mode
|
|
||||||
600) under `<stage_dir>/pipelock-ca/`. Returns the two host paths.
|
|
||||||
|
|
||||||
The image is pinned (same digest the running sidecar uses) so the
|
|
||||||
generated CA matches what the sidecar expects. Output is owned by
|
|
||||||
whatever UID the one-shot ran as; the compose renderer's
|
|
||||||
bind-mounts pin the files in place at runtime, so ownership
|
|
||||||
inside the running sidecar (root in pipelock's distroless image)
|
|
||||||
is independent."""
|
|
||||||
work = stage_dir / "pipelock-ca"
|
|
||||||
work.mkdir(exist_ok=True)
|
|
||||||
result = subprocess.run(
|
|
||||||
["docker", "run", "--rm",
|
|
||||||
"-v", f"{work}:/h",
|
|
||||||
"-e", "PIPELOCK_HOME=/h",
|
|
||||||
PIPELOCK_IMAGE, "tls", "init"],
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
die(f"pipelock tls init failed: {result.stderr.strip()}")
|
|
||||||
cert = work / "ca.pem"
|
|
||||||
key = work / "ca-key.pem"
|
|
||||||
if not cert.is_file() or not key.is_file():
|
|
||||||
die(f"pipelock tls init did not produce ca files in {work}")
|
|
||||||
# Explicit perms in case a future pipelock release changes
|
|
||||||
# defaults. Pipelock runs as root in its distroless image and
|
|
||||||
# bind-mounts work with 0o600 (root reads everything); the key
|
|
||||||
# has no reason to be readable to anyone else on the host.
|
|
||||||
key.chmod(0o600)
|
|
||||||
cert.chmod(0o644)
|
|
||||||
return (cert, key)
|
|
||||||
@@ -1,200 +0,0 @@
|
|||||||
"""pipelock_apply — host-side helper to apply an api_allowlist
|
|
||||||
change to a running pipelock sidecar (PRD 0015).
|
|
||||||
|
|
||||||
Used by the supervise dashboard when the operator approves a
|
|
||||||
pipelock-block proposal (or runs the operator-initiated `pipelock
|
|
||||||
edit <bottle>` verb). Fetches the current pipelock.yaml via `docker
|
|
||||||
exec`, parses it, swaps the api_allowlist with the proposed hosts,
|
|
||||||
re-renders, writes back via the bind-mount path, then signals the
|
|
||||||
bundle supervisor to restart the pipelock daemon (`docker kill
|
|
||||||
--signal USR1`) so
|
|
||||||
pipelock picks up the new config.
|
|
||||||
|
|
||||||
v1 uses restart, not SIGHUP — pipelock has no in-process reload
|
|
||||||
hook and adding one is the "SIGHUP reload for pipelock" open
|
|
||||||
question in PRD 0015. Restart drops in-flight outbound calls; the
|
|
||||||
agent's HTTP client retries pick up against the restarted proxy.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ...pipelock import pipelock_render_yaml
|
|
||||||
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
|
|
||||||
from .bottle_state import pipelock_state_dir
|
|
||||||
from .sidecar_bundle import sidecar_bundle_container_name
|
|
||||||
|
|
||||||
|
|
||||||
def _pipelock_yaml_host_path(slug: str) -> Path:
|
|
||||||
"""The bind-mount source for the pipelock sidecar's
|
|
||||||
pipelock.yaml — matches what pipelock.prepare wrote at chunk-2
|
|
||||||
paths."""
|
|
||||||
return pipelock_state_dir(slug) / "pipelock.yaml"
|
|
||||||
|
|
||||||
|
|
||||||
PIPELOCK_YAML_IN_CONTAINER = "/etc/pipelock.yaml"
|
|
||||||
|
|
||||||
# Allowlist proposals are one-hostname-per-line. Blank lines and
|
|
||||||
# `#`-prefixed comments are ignored. The character set matches the
|
|
||||||
# supervise sidecar's syntactic check on the agent's pipelock-block
|
|
||||||
# proposal (alphanumerics + dot/dash/underscore).
|
|
||||||
_HOST_OK = re.compile(r"^[A-Za-z0-9_.-]+$")
|
|
||||||
|
|
||||||
|
|
||||||
class PipelockApplyError(RuntimeError):
|
|
||||||
"""Raised when fetch / parse / apply fails. The dashboard renders
|
|
||||||
the message and keeps the proposal pending — never crashes."""
|
|
||||||
|
|
||||||
|
|
||||||
def parse_allowlist_content(content: str) -> list[str]:
|
|
||||||
"""One hostname per line. Blanks and `#` comments are ignored.
|
|
||||||
Raises PipelockApplyError if a line has a disallowed character."""
|
|
||||||
hosts: list[str] = []
|
|
||||||
for i, raw_line in enumerate(content.splitlines(), start=1):
|
|
||||||
line = raw_line.strip()
|
|
||||||
if not line or line.startswith("#"):
|
|
||||||
continue
|
|
||||||
if not _HOST_OK.match(line):
|
|
||||||
raise PipelockApplyError(
|
|
||||||
f"allowlist line {i}: {line!r} has disallowed characters"
|
|
||||||
)
|
|
||||||
hosts.append(line)
|
|
||||||
return hosts
|
|
||||||
|
|
||||||
|
|
||||||
def render_allowlist_content(hosts: list[str]) -> str:
|
|
||||||
"""Hosts → one-per-line string (the operator-facing format)."""
|
|
||||||
if not hosts:
|
|
||||||
return ""
|
|
||||||
return "\n".join(hosts) + "\n"
|
|
||||||
|
|
||||||
|
|
||||||
def fetch_current_yaml(slug: str) -> str:
|
|
||||||
"""Read the live /etc/pipelock.yaml from the sidecar bundle.
|
|
||||||
|
|
||||||
Uses `docker cp` because pipelock inside the bundle is the
|
|
||||||
distroless pipelock binary with no shell, and `docker cp` is a
|
|
||||||
daemon-API tarball copy that works regardless of what's
|
|
||||||
available inside the container.
|
|
||||||
|
|
||||||
Raises PipelockApplyError if the read fails."""
|
|
||||||
container = sidecar_bundle_container_name(slug)
|
|
||||||
fd, tmp_path = tempfile.mkstemp(prefix="cb-pipelock-fetch.", suffix=".yaml")
|
|
||||||
os.close(fd)
|
|
||||||
try:
|
|
||||||
r = subprocess.run(
|
|
||||||
[
|
|
||||||
"docker", "cp",
|
|
||||||
f"{container}:{PIPELOCK_YAML_IN_CONTAINER}", tmp_path,
|
|
||||||
],
|
|
||||||
capture_output=True, text=True, check=False,
|
|
||||||
)
|
|
||||||
if r.returncode != 0:
|
|
||||||
raise PipelockApplyError(
|
|
||||||
f"could not fetch pipelock.yaml from {container}: "
|
|
||||||
f"{(r.stderr or '').strip() or 'container not running?'}"
|
|
||||||
)
|
|
||||||
return Path(tmp_path).read_text()
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
Path(tmp_path).unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def fetch_current_allowlist(slug: str) -> str:
|
|
||||||
"""Fetch the live yaml, extract api_allowlist, render as one-per-
|
|
||||||
line — the operator-facing format for the TUI / agent's
|
|
||||||
current-config mount."""
|
|
||||||
yaml = fetch_current_yaml(slug)
|
|
||||||
try:
|
|
||||||
cfg = parse_yaml_subset(yaml)
|
|
||||||
except YamlSubsetError as e:
|
|
||||||
raise PipelockApplyError(f"running pipelock yaml: {e}") from e
|
|
||||||
hosts = cfg.get("api_allowlist", [])
|
|
||||||
if not isinstance(hosts, list):
|
|
||||||
raise PipelockApplyError(
|
|
||||||
"running pipelock yaml: api_allowlist is not a list"
|
|
||||||
)
|
|
||||||
return render_allowlist_content([str(h) for h in hosts])
|
|
||||||
|
|
||||||
|
|
||||||
def apply_allowlist_change(
|
|
||||||
slug: str, new_allowlist_content: str,
|
|
||||||
) -> tuple[str, str]:
|
|
||||||
"""Apply `new_allowlist_content` to the sidecar bundle:
|
|
||||||
1. Parse the proposed hosts (one per line).
|
|
||||||
2. Fetch + parse current pipelock.yaml.
|
|
||||||
3. Replace api_allowlist with the proposed hosts; re-render.
|
|
||||||
4. Write the new yaml to the bind-mount source.
|
|
||||||
5. `docker kill --signal USR1 <bundle>` so the supervisor
|
|
||||||
restarts the pipelock daemon in place (leaving egress,
|
|
||||||
git-gate, and supervise running). Pipelock has no
|
|
||||||
in-process reload; the supervisor's per-daemon restart
|
|
||||||
keeps the agent's MCP socket alive — a whole-bundle
|
|
||||||
`docker restart` would bounce supervise too.
|
|
||||||
|
|
||||||
Returns (before, after) where both are one-per-line allowlist
|
|
||||||
strings (operator-facing format). Raises PipelockApplyError on
|
|
||||||
any failure; the sidecar's existing config stays in place until
|
|
||||||
the host write succeeds, and the SIGUSR1 is what makes it
|
|
||||||
live."""
|
|
||||||
new_hosts = parse_allowlist_content(new_allowlist_content)
|
|
||||||
container = sidecar_bundle_container_name(slug)
|
|
||||||
current_yaml = fetch_current_yaml(slug)
|
|
||||||
try:
|
|
||||||
cfg = parse_yaml_subset(current_yaml)
|
|
||||||
except YamlSubsetError as e:
|
|
||||||
raise PipelockApplyError(f"running pipelock yaml: {e}") from e
|
|
||||||
current_hosts = cfg.get("api_allowlist", [])
|
|
||||||
if not isinstance(current_hosts, list):
|
|
||||||
raise PipelockApplyError(
|
|
||||||
"running pipelock yaml: api_allowlist is not a list"
|
|
||||||
)
|
|
||||||
|
|
||||||
before = render_allowlist_content([str(h) for h in current_hosts])
|
|
||||||
after = render_allowlist_content(new_hosts)
|
|
||||||
|
|
||||||
cfg["api_allowlist"] = new_hosts
|
|
||||||
rendered = pipelock_render_yaml(cfg)
|
|
||||||
|
|
||||||
# pipelock.yaml is bind-mounted into the container as a SINGLE
|
|
||||||
# FILE — same Docker single-file inode issue as egress_apply:
|
|
||||||
# write-temp-then-rename swaps the host inode and leaves the
|
|
||||||
# container's mount pointing at the orphaned old one. Write
|
|
||||||
# in-place. The SIGUSR1 below makes the new content live
|
|
||||||
# (pipelock has no in-process reload, so the supervisor
|
|
||||||
# restarts the pipelock daemon in response).
|
|
||||||
target = _pipelock_yaml_host_path(slug)
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
target.write_text(rendered)
|
|
||||||
# pipelock runs as root in its distroless image — any mode is
|
|
||||||
# fine — but 0o600 matches what prepare wrote.
|
|
||||||
target.chmod(0o600)
|
|
||||||
restart = subprocess.run(
|
|
||||||
["docker", "kill", "--signal", "USR1", container],
|
|
||||||
capture_output=True, text=True, check=False,
|
|
||||||
)
|
|
||||||
if restart.returncode != 0:
|
|
||||||
raise PipelockApplyError(
|
|
||||||
f"failed to signal {container} for pipelock restart: "
|
|
||||||
f"{(restart.stderr or '').strip()}"
|
|
||||||
)
|
|
||||||
|
|
||||||
return before, after
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"PIPELOCK_YAML_IN_CONTAINER",
|
|
||||||
"PipelockApplyError",
|
|
||||||
"apply_allowlist_change",
|
|
||||||
"fetch_current_allowlist",
|
|
||||||
"fetch_current_yaml",
|
|
||||||
"parse_allowlist_content",
|
|
||||||
"render_allowlist_content",
|
|
||||||
]
|
|
||||||
@@ -15,12 +15,11 @@ from datetime import datetime, timezone
|
|||||||
from dataclasses import replace
|
from dataclasses import replace
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import agent_provision_plan, runtime_for
|
from ...agent_provider import PROVIDER_TEMPLATES, agent_provision_plan, runtime_for
|
||||||
from ...egress import Egress
|
from ...egress import Egress
|
||||||
from ...env import ResolvedEnv, resolve_env
|
from ...env import ResolvedEnv, resolve_env
|
||||||
from ...git_gate import GitGate
|
from ...git_gate import GitGate
|
||||||
from ...log import die
|
from ...log import die
|
||||||
from ...pipelock import PipelockProxy
|
|
||||||
from ...supervise import Supervise
|
from ...supervise import Supervise
|
||||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||||
from .. import BottleSpec
|
from .. import BottleSpec
|
||||||
@@ -36,7 +35,6 @@ from .bottle_state import (
|
|||||||
per_bottle_dockerfile,
|
per_bottle_dockerfile,
|
||||||
per_bottle_dockerfile_path,
|
per_bottle_dockerfile_path,
|
||||||
per_bottle_image_tag,
|
per_bottle_image_tag,
|
||||||
pipelock_state_dir,
|
|
||||||
supervise_state_dir,
|
supervise_state_dir,
|
||||||
write_metadata,
|
write_metadata,
|
||||||
)
|
)
|
||||||
@@ -53,7 +51,6 @@ def resolve_plan(
|
|||||||
validation already ran in the base class."""
|
validation already ran in the base class."""
|
||||||
docker_mod.require_docker()
|
docker_mod.require_docker()
|
||||||
|
|
||||||
proxy = PipelockProxy()
|
|
||||||
git_gate = GitGate()
|
git_gate = GitGate()
|
||||||
egress = Egress()
|
egress = Egress()
|
||||||
supervise = Supervise()
|
supervise = Supervise()
|
||||||
@@ -63,7 +60,7 @@ def resolve_plan(
|
|||||||
bottle = manifest.bottle_for(spec.agent_name)
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
provider = bottle.agent_provider
|
provider = bottle.agent_provider
|
||||||
provider_runtime = runtime_for(provider.template)
|
provider_runtime = runtime_for(provider.template)
|
||||||
guest_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
guest_home = "/home/node"
|
||||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||||
|
|
||||||
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
||||||
@@ -103,6 +100,15 @@ def resolve_plan(
|
|||||||
elif provider_runtime.dockerfile:
|
elif provider_runtime.dockerfile:
|
||||||
image_default = provider_runtime.image
|
image_default = provider_runtime.image
|
||||||
dockerfile_path = provider_runtime.dockerfile
|
dockerfile_path = provider_runtime.dockerfile
|
||||||
|
elif provider.template not in PROVIDER_TEMPLATES:
|
||||||
|
user_dockerfile = (
|
||||||
|
Path.home() / ".bot-bottle" / "contrib" / provider.template / "Dockerfile"
|
||||||
|
)
|
||||||
|
if user_dockerfile.is_file():
|
||||||
|
image_default = f"bot-bottle-{provider.template}:{slug}"
|
||||||
|
dockerfile_path = str(user_dockerfile)
|
||||||
|
else:
|
||||||
|
image_default = provider_runtime.image
|
||||||
else:
|
else:
|
||||||
image_default = provider_runtime.image
|
image_default = provider_runtime.image
|
||||||
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||||
@@ -191,12 +197,6 @@ def resolve_plan(
|
|||||||
guest_env.setdefault(key, val)
|
guest_env.setdefault(key, val)
|
||||||
agent_provision = replace(agent_provision, guest_env=guest_env)
|
agent_provision = replace(agent_provision, guest_env=guest_env)
|
||||||
|
|
||||||
pipelock_dir = pipelock_state_dir(slug)
|
|
||||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
proxy_plan = proxy.prepare(
|
|
||||||
bottle, slug, pipelock_dir, agent_provision.egress_routes,
|
|
||||||
)
|
|
||||||
|
|
||||||
egress_dir = egress_state_dir(slug)
|
egress_dir = egress_state_dir(slug)
|
||||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||||
egress_plan = egress.prepare(
|
egress_plan = egress.prepare(
|
||||||
@@ -209,17 +209,16 @@ def resolve_plan(
|
|||||||
# root; for `--cwd` derived images the base Dockerfile is what
|
# root; for `--cwd` derived images the base Dockerfile is what
|
||||||
# the agent should propose changes against (the derived layer
|
# the agent should propose changes against (the derived layer
|
||||||
# is just a workspace copy).
|
# is just a workspace copy).
|
||||||
# (routes.yaml + pipelock allowlist used to land here too but
|
# (routes.yaml used to land here too but PRD 0017 chunk 3
|
||||||
# PRD 0017 chunk 3 moved them behind the
|
# moved it behind the `list-egress-routes` MCP tool so the
|
||||||
# `list-egress-routes` MCP tool so the agent gets live
|
# agent gets live state rather than a launch-time snapshot.)
|
||||||
# state rather than a launch-time snapshot.)
|
|
||||||
supervise_dockerfile_path = (
|
supervise_dockerfile_path = (
|
||||||
Path(dockerfile_path)
|
Path(dockerfile_path)
|
||||||
if dockerfile_path
|
if dockerfile_path
|
||||||
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
|
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
|
||||||
)
|
)
|
||||||
dockerfile_content = (
|
dockerfile_content = (
|
||||||
supervise_dockerfile_path.read_text()
|
supervise_dockerfile_path.read_text(encoding="utf-8")
|
||||||
if supervise_dockerfile_path.is_file()
|
if supervise_dockerfile_path.is_file()
|
||||||
else ""
|
else ""
|
||||||
)
|
)
|
||||||
@@ -233,6 +232,7 @@ def resolve_plan(
|
|||||||
return DockerBottlePlan(
|
return DockerBottlePlan(
|
||||||
spec=spec,
|
spec=spec,
|
||||||
stage_dir=stage_dir,
|
stage_dir=stage_dir,
|
||||||
|
guest_home=guest_home,
|
||||||
slug=slug,
|
slug=slug,
|
||||||
container_name=container_name,
|
container_name=container_name,
|
||||||
container_name_pinned=container_name_pinned,
|
container_name_pinned=container_name_pinned,
|
||||||
@@ -243,7 +243,6 @@ def resolve_plan(
|
|||||||
env_file=env_file,
|
env_file=env_file,
|
||||||
forwarded_env=forwarded_env,
|
forwarded_env=forwarded_env,
|
||||||
prompt_file=prompt_file,
|
prompt_file=prompt_file,
|
||||||
proxy_plan=proxy_plan,
|
|
||||||
git_gate_plan=git_gate_plan,
|
git_gate_plan=git_gate_plan,
|
||||||
egress_plan=egress_plan,
|
egress_plan=egress_plan,
|
||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
|
|||||||
@@ -1,8 +1,12 @@
|
|||||||
"""Per-provisioner modules for the Docker backend.
|
"""Backend-infrastructure provisioners for the Docker backend.
|
||||||
|
|
||||||
Each module exports one top-level function:
|
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||||
provision_<thing>(plan: DockerBottlePlan, target: str) -> ...
|
declarative provision-plan apply, supervise MCP registration) live on
|
||||||
|
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
|
||||||
|
provisioning also moved to the AgentProvider ABC (with Debian/node
|
||||||
|
defaults); user plugins override them for non-standard images.
|
||||||
|
|
||||||
`DockerBottleBackend.provision_*` methods delegate to these. The
|
No modules remain in this subpackage — the directory is kept so that
|
||||||
abstract `BottleBackend.provision_*` surface is unchanged; this
|
existing imports of `from .provision import ...` don't need updating
|
||||||
subpackage exists only to keep `backend.py` from being a god-file."""
|
if new backend-specific provisioners are added later.
|
||||||
|
"""
|
||||||
|
|||||||
@@ -1,63 +0,0 @@
|
|||||||
"""Install the per-bottle MITM CA into the agent container's trust
|
|
||||||
store.
|
|
||||||
|
|
||||||
Post-PRD-0017 the CA depends on the agent's HTTP_PROXY target:
|
|
||||||
|
|
||||||
- Bottle declares `egress.routes[]` → agent's HTTP_PROXY
|
|
||||||
points at egress; the cert the agent must trust is the
|
|
||||||
one egress mints leaf certs with (the egress CA).
|
|
||||||
- No egress routes → agent's HTTP_PROXY points straight at
|
|
||||||
pipelock; the cert the agent must trust is pipelock's CA (the
|
|
||||||
pre-cutover behavior).
|
|
||||||
|
|
||||||
By the time this provisioner runs, the corresponding `tls_init`
|
|
||||||
helper has generated the chosen CA under `plan.stage_dir`, and the
|
|
||||||
sidecar (pipelock or egress) is up referencing the
|
|
||||||
in-container CA paths.
|
|
||||||
|
|
||||||
Cert lands on Debian's standard source path
|
|
||||||
(`/usr/local/share/ca-certificates/`); `update-ca-certificates`
|
|
||||||
rebuilds `/etc/ssl/certs/ca-certificates.crt`, which is what curl,
|
|
||||||
Python `ssl`, and OpenSSL-based tools all read by default. The env
|
|
||||||
trio set on the agent's `docker run` covers Node
|
|
||||||
(`NODE_EXTRA_CA_CERTS`) and Python `requests` /
|
|
||||||
`SSL_CERT_FILE`-honoring libraries that don't load the system
|
|
||||||
bundle.
|
|
||||||
|
|
||||||
The fingerprint is computed via stdlib (`ssl.PEM_cert_to_DER_cert`
|
|
||||||
+ `hashlib.sha256`) and logged once to stderr. The private key
|
|
||||||
stays on the host (under `stage_dir`) until teardown wipes the
|
|
||||||
stage dir; nothing in the agent ever sees it."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
from ...util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
|
||||||
from ..bottle_plan import DockerBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_ca(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Copy the agent-facing CA cert into the agent, rebuild the
|
|
||||||
trust bundle, emit a one-line fingerprint log. Called from
|
|
||||||
`BottleBackend.provision` after the agent container is up."""
|
|
||||||
container = target
|
|
||||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
|
||||||
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "cp", str(cert_host_path), f"{container}:{AGENT_CA_PATH}"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "0", container, "chmod", "644", AGENT_CA_PATH],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "0", container, "update-ca-certificates"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
log_ca_fingerprint(cert_host_path, label)
|
|
||||||
@@ -1,123 +0,0 @@
|
|||||||
"""Git provisioning inside a running Docker bottle.
|
|
||||||
|
|
||||||
Three concerns, all about git in the agent:
|
|
||||||
|
|
||||||
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
|
||||||
into the planned guest workspace so the agent operates on the
|
|
||||||
user's repo.
|
|
||||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
|
||||||
~/.gitconfig with insteadOf rules so every git operation
|
|
||||||
against a declared upstream (push, fetch, clone, pull,
|
|
||||||
ls-remote) transparently hits the per-agent git-gate. The
|
|
||||||
gate mirrors the upstream in both directions, so URL
|
|
||||||
rewriting is symmetric.
|
|
||||||
3. If the bottle declares `git.user` (issue #86), set
|
|
||||||
`git config --global user.{name,email}` inside the bottle so
|
|
||||||
the agent's commits are attributed to that identity.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
|
||||||
from ....log import info
|
|
||||||
from .. import util as docker_mod
|
|
||||||
from ..bottle_plan import DockerBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_git(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Set up git inside the bottle. Runs all three subcases; each
|
|
||||||
no-ops when its condition isn't met."""
|
|
||||||
_provision_cwd_git(plan, target)
|
|
||||||
_provision_git_gate_config(plan, target)
|
|
||||||
_provision_git_user(plan, target)
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_cwd_git(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
|
||||||
it into /home/node/workspace/.git and fix ownership. No-op
|
|
||||||
otherwise."""
|
|
||||||
workspace = plan.workspace_plan
|
|
||||||
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
|
||||||
return
|
|
||||||
container = target
|
|
||||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
|
||||||
host_git = str(workspace.host_path / ".git")
|
|
||||||
info(f"copying {host_git} -> {container}:{guest_workspace_git}")
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "cp", host_git, f"{container}:{guest_workspace_git}"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
[
|
|
||||||
"docker", "exec", "-u", "0", container,
|
|
||||||
"chown", "-R", workspace.owner, guest_workspace_git,
|
|
||||||
],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_gate_config(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Write ~/.gitconfig in the bottle with the git-gate
|
|
||||||
insteadOf rules. No-op when the bottle has no `git` entries."""
|
|
||||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
|
||||||
if not bottle.git:
|
|
||||||
return
|
|
||||||
container = target
|
|
||||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
|
||||||
container_gitconfig = f"{container_home}/.gitconfig"
|
|
||||||
|
|
||||||
content = git_gate_render_gitconfig(bottle.git, GIT_GATE_HOSTNAME)
|
|
||||||
config_file = plan.stage_dir / "agent_gitconfig"
|
|
||||||
config_file.write_text(content)
|
|
||||||
config_file.chmod(0o600)
|
|
||||||
|
|
||||||
info(f"writing {container_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "cp", str(config_file), f"{container}:{container_gitconfig}"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
docker_mod.docker_exec_root(container, ["chown", "node:node", container_gitconfig])
|
|
||||||
docker_mod.docker_exec_root(container, ["chmod", "644", container_gitconfig])
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_user(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Apply `git config --global user.{name,email}` inside the
|
|
||||||
bottle so the agent's commits are attributed to the operator-
|
|
||||||
chosen identity instead of the agent image's default
|
|
||||||
(which is no user — git would refuse to commit at all
|
|
||||||
until the agent ran its own `git config`).
|
|
||||||
|
|
||||||
Runs as the `node` user so `--global` lands in
|
|
||||||
`/home/node/.gitconfig` (matching the existing
|
|
||||||
`_provision_git_gate_config` write location). No-op when the
|
|
||||||
bottle didn't declare `git.user`.
|
|
||||||
|
|
||||||
Each field set independently — name-only or email-only
|
|
||||||
configs only run the `git config` line for the field
|
|
||||||
present."""
|
|
||||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
|
||||||
gu = bottle.git_user
|
|
||||||
if gu.is_empty():
|
|
||||||
return
|
|
||||||
if gu.name:
|
|
||||||
info(f"git config --global user.name = {gu.name!r}")
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "node", target,
|
|
||||||
"git", "config", "--global", "user.name", gu.name],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
if gu.email:
|
|
||||||
info(f"git config --global user.email = {gu.email!r}")
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "node", target,
|
|
||||||
"git", "config", "--global", "user.email", gu.email],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
"""Copy the agent prompt into a running Docker bottle.
|
|
||||||
|
|
||||||
The prompt file is always copied (so the in-container path always
|
|
||||||
exists) but `--append-system-prompt-file` only fires when the agent
|
|
||||||
actually has a prompt — the return value signals which case."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
from ..bottle_plan import DockerBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_prompt(plan: DockerBottlePlan, target: str) -> str | None:
|
|
||||||
"""Copy the prompt file into the container, fix ownership/mode.
|
|
||||||
Returns the in-container path if the agent has a non-empty
|
|
||||||
prompt (drives --append-system-prompt-file), else None. The
|
|
||||||
file is copied either way so the path always exists."""
|
|
||||||
container = target
|
|
||||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
|
||||||
in_container_prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
|
|
||||||
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "cp", str(plan.prompt_file), f"{container}:{in_container_prompt_path}"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
# `docker cp` preserves host UID; re-own/mode as root so node
|
|
||||||
# can read its own mode-600 prompt regardless of host UID.
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "0", container, "chown", "node:node", in_container_prompt_path],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "0", container, "chmod", "600", in_container_prompt_path],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
return in_container_prompt_path if agent.prompt else None
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
"""Provision non-secret provider auth markers into a Docker bottle."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
from ..bottle_plan import DockerBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_provider_auth(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Apply provider-owned guest setup through Docker primitives."""
|
|
||||||
provision = plan.agent_provision
|
|
||||||
for d in provision.dirs:
|
|
||||||
_exec(target, ["mkdir", "-p", d.guest_path])
|
|
||||||
_exec(target, ["chown", d.owner, d.guest_path])
|
|
||||||
_exec(target, ["chmod", d.mode, d.guest_path])
|
|
||||||
for command in provision.pre_copy:
|
|
||||||
_exec(target, list(command.argv))
|
|
||||||
for f in provision.files:
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "cp", str(f.host_path), f"{target}:{f.guest_path}"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
_exec(target, ["chown", f.owner, f.guest_path])
|
|
||||||
_exec(target, ["chmod", f.mode, f.guest_path])
|
|
||||||
for command in provision.verify:
|
|
||||||
_exec(target, list(command.argv))
|
|
||||||
|
|
||||||
|
|
||||||
def _exec(target: str, argv: list[str]) -> None:
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", "-u", "0", target, *argv],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
"""Copy host-side skill directories into a running Docker bottle.
|
|
||||||
|
|
||||||
Skills are validated on the host before launch by the base class's
|
|
||||||
`BottleBackend._validate_skills` (called from `prepare`); this module
|
|
||||||
assumes that validation has already run. A skill disappearing between
|
|
||||||
validation and copy still dies loudly rather than silently producing
|
|
||||||
a partial container."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
from ....log import die, info
|
|
||||||
from ...util import host_skill_dir
|
|
||||||
from ..bottle_plan import DockerBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_skills(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Copy each of the agent's named skills from the host's
|
|
||||||
~/.claude/skills/<name>/ into the container's equivalent path.
|
|
||||||
For each skill: ensure parent dir, wipe any prior copy, then
|
|
||||||
`docker cp <host>/. <container>:<dst>/` so the contents are
|
|
||||||
copied into a freshly-created destination dir. No-op when the
|
|
||||||
agent has no skills."""
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
if not agent.skills:
|
|
||||||
return
|
|
||||||
|
|
||||||
container = target
|
|
||||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
|
||||||
skills_dir = os.environ.get(
|
|
||||||
"BOT_BOTTLE_CONTAINER_SKILLS_DIR", f"{container_home}/.claude/skills"
|
|
||||||
)
|
|
||||||
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", container, "mkdir", "-p", skills_dir],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
for n in agent.skills:
|
|
||||||
src = host_skill_dir(n)
|
|
||||||
if not os.path.isdir(src):
|
|
||||||
die(f"skill '{n}' disappeared from host between validation and copy at {src}.")
|
|
||||||
dst = f"{skills_dir}/{n}"
|
|
||||||
info(f"copying skill {n} into {container}:{dst}")
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", container, "rm", "-rf", dst],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "exec", container, "mkdir", "-p", dst],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "cp", f"{src}/.", f"{container}:{dst}/"],
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
"""Supervise sidecar provisioning inside a running Docker bottle
|
|
||||||
(PRD 0013).
|
|
||||||
|
|
||||||
Registers the per-bottle supervise sidecar as an HTTP MCP server in
|
|
||||||
the agent's claude-code config so the agent discovers the three
|
|
||||||
stuck-recovery MCP tools (cred-proxy-block, pipelock-block,
|
|
||||||
capability-block) at startup.
|
|
||||||
|
|
||||||
Uses `claude mcp add` rather than writing JSON directly. claude-code
|
|
||||||
owns the on-disk config format (`~/.claude.json` `mcpServers` shape,
|
|
||||||
field names, scope semantics) and changes it between versions; the
|
|
||||||
official command handles whatever the installed version expects.
|
|
||||||
|
|
||||||
No-op when bottle.supervise is False — bottles that haven't opted
|
|
||||||
into the supervise sidecar shouldn't get an MCP entry pointing at a
|
|
||||||
sidecar that isn't running.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
from ....log import info, warn
|
|
||||||
from ....supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
|
||||||
from ..bottle_plan import DockerBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
_SUPERVISE_MCP_NAME = "supervise"
|
|
||||||
|
|
||||||
|
|
||||||
def supervise_mcp_url() -> str:
|
|
||||||
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
|
||||||
|
|
||||||
|
|
||||||
def provision_supervise(plan: DockerBottlePlan, target: str) -> None:
|
|
||||||
"""Run `claude mcp add` inside the agent container to register
|
|
||||||
the supervise sidecar in claude-code's user config. No-op when
|
|
||||||
bottle.supervise is False.
|
|
||||||
|
|
||||||
Failure is logged but not fatal: the bottle still works (you
|
|
||||||
just can't call supervise tools from the agent until the entry
|
|
||||||
is added manually). The operator sees the warning at launch."""
|
|
||||||
if plan.supervise_plan is None:
|
|
||||||
return
|
|
||||||
url = supervise_mcp_url()
|
|
||||||
argv = [
|
|
||||||
"docker", "exec", "-u", "node", target,
|
|
||||||
"claude", "mcp", "add",
|
|
||||||
"--scope", "user",
|
|
||||||
"--transport", "http",
|
|
||||||
_SUPERVISE_MCP_NAME,
|
|
||||||
url,
|
|
||||||
]
|
|
||||||
info(f"registering supervise MCP server in agent claude config → {url}")
|
|
||||||
r = subprocess.run(argv, capture_output=True, text=True, check=False)
|
|
||||||
if r.returncode != 0:
|
|
||||||
warn(
|
|
||||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
|
||||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
|
||||||
f"register manually with: "
|
|
||||||
f"claude mcp add --scope user --transport http supervise {url}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["provision_supervise", "supervise_mcp_url"]
|
|
||||||
@@ -2,10 +2,10 @@
|
|||||||
(PRD 0024).
|
(PRD 0024).
|
||||||
|
|
||||||
The bundle image (built by Dockerfile.sidecars, PRD 0024 chunk 1)
|
The bundle image (built by Dockerfile.sidecars, PRD 0024 chunk 1)
|
||||||
runs pipelock + egress + git-gate + supervise as one container
|
runs egress + git-gate + supervise as one container per bottle
|
||||||
per bottle under a small Python init supervisor. As of chunk 5
|
under a small Python init supervisor. As of chunk 5 the bundle
|
||||||
the bundle is the only shape — the legacy four-sidecar topology
|
is the only shape — the legacy four-sidecar topology and its
|
||||||
and its `BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
|
`BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -14,8 +14,7 @@ import os
|
|||||||
|
|
||||||
# Bundle image. Defaults to a built-locally tag (built from the
|
# Bundle image. Defaults to a built-locally tag (built from the
|
||||||
# repo's Dockerfile.sidecars via compose `build:`). Operators
|
# repo's Dockerfile.sidecars via compose `build:`). Operators
|
||||||
# pinning to a published digest can override via env, matching
|
# pinning to a published digest can override via env.
|
||||||
# the existing `BOT_BOTTLE_PIPELOCK_IMAGE` shape.
|
|
||||||
SIDECAR_BUNDLE_IMAGE = os.environ.get(
|
SIDECAR_BUNDLE_IMAGE = os.environ.get(
|
||||||
"BOT_BOTTLE_SIDECAR_IMAGE",
|
"BOT_BOTTLE_SIDECAR_IMAGE",
|
||||||
"bot-bottle-sidecars:latest",
|
"bot-bottle-sidecars:latest",
|
||||||
|
|||||||
@@ -1,5 +1,11 @@
|
|||||||
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
||||||
BottleBackend (PRD 0023)."""
|
BottleBackend (PRD 0023).
|
||||||
|
|
||||||
|
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||||
|
the declarative provision-plan apply, supervise MCP registration)
|
||||||
|
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||||
|
smolmachines backend only owns the steps that are about backend
|
||||||
|
infrastructure: CA install (no-op for now), workspace, git copy-in."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -7,7 +13,7 @@ from contextlib import contextmanager
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Generator, Sequence
|
from typing import Generator, Sequence
|
||||||
|
|
||||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
|
||||||
from . import cleanup as _cleanup
|
from . import cleanup as _cleanup
|
||||||
from . import enumerate as _enumerate
|
from . import enumerate as _enumerate
|
||||||
from . import launch as _launch
|
from . import launch as _launch
|
||||||
@@ -16,12 +22,6 @@ from . import smolvm as _smolvm
|
|||||||
from .bottle import SmolmachinesBottle
|
from .bottle import SmolmachinesBottle
|
||||||
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
||||||
from .bottle_plan import SmolmachinesBottlePlan
|
from .bottle_plan import SmolmachinesBottlePlan
|
||||||
from .provision import ca as _ca
|
|
||||||
from .provision import git as _git
|
|
||||||
from .provision import prompt as _prompt
|
|
||||||
from .provision import provider_auth as _provider_auth
|
|
||||||
from .provision import skills as _skills
|
|
||||||
from .provision import supervise as _supervise
|
|
||||||
from .provision import workspace as _workspace
|
from .provision import workspace as _workspace
|
||||||
|
|
||||||
|
|
||||||
@@ -53,40 +53,17 @@ class SmolmachinesBottleBackend(
|
|||||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||||
yield bottle
|
yield bottle
|
||||||
|
|
||||||
def provision_ca(
|
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
|
||||||
) -> None:
|
|
||||||
_ca.provision_ca(plan, target)
|
|
||||||
|
|
||||||
def provision_prompt(
|
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
|
||||||
) -> str | None:
|
|
||||||
return _prompt.provision_prompt(plan, target)
|
|
||||||
|
|
||||||
def provision_provider_auth(
|
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
|
||||||
) -> None:
|
|
||||||
_provider_auth.provision_provider_auth(plan, target)
|
|
||||||
|
|
||||||
def provision_skills(
|
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
|
||||||
) -> None:
|
|
||||||
_skills.provision_skills(plan, target)
|
|
||||||
|
|
||||||
def provision_workspace(
|
def provision_workspace(
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
||||||
) -> None:
|
) -> None:
|
||||||
_workspace.provision_workspace(plan, target)
|
_workspace.provision_workspace(plan, bottle)
|
||||||
|
|
||||||
def provision_git(
|
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
"""The smolmachines guest reaches the supervise sidecar via a
|
||||||
) -> None:
|
host-published random port the launch step pinned earlier
|
||||||
_git.provision_git(plan, target)
|
(`http://<loopback_ip>:<random_port>/`). `agent_supervise_url`
|
||||||
|
on the plan is "" when the bottle has no sidecar."""
|
||||||
def provision_supervise(
|
return plan.agent_supervise_url
|
||||||
self, plan: SmolmachinesBottlePlan, target: str
|
|
||||||
) -> None:
|
|
||||||
_supervise.provision_supervise(plan, target)
|
|
||||||
|
|
||||||
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
||||||
return _cleanup.prepare_cleanup()
|
return _cleanup.prepare_cleanup()
|
||||||
|
|||||||
@@ -19,7 +19,8 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
from typing import Mapping
|
import time
|
||||||
|
from typing import Mapping, cast
|
||||||
|
|
||||||
from ...agent_provider import PromptMode, prompt_args
|
from ...agent_provider import PromptMode, prompt_args
|
||||||
from .. import Bottle, ExecResult
|
from .. import Bottle, ExecResult
|
||||||
@@ -72,7 +73,7 @@ class SmolmachinesBottle(Bottle):
|
|||||||
# In-VM path to the agent's prompt file. None when the
|
# In-VM path to the agent's prompt file. None when the
|
||||||
# agent declared no prompt (file still exists; we just
|
# agent declared no prompt (file still exists; we just
|
||||||
# don't pass --append-system-prompt-file).
|
# don't pass --append-system-prompt-file).
|
||||||
self._prompt_path = prompt_path
|
self.prompt_path = prompt_path
|
||||||
# Env vars the agent process needs (HTTPS_PROXY,
|
# Env vars the agent process needs (HTTPS_PROXY,
|
||||||
# CLAUDE_CODE_OAUTH_TOKEN, manifest-declared bottle env, …).
|
# CLAUDE_CODE_OAUTH_TOKEN, manifest-declared bottle env, …).
|
||||||
# Forwarded on every `smolvm machine exec` via `-e K=V`
|
# Forwarded on every `smolvm machine exec` via `-e K=V`
|
||||||
@@ -93,9 +94,9 @@ class SmolmachinesBottle(Bottle):
|
|||||||
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
|
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
|
||||||
self.agent_command]
|
self.agent_command]
|
||||||
provider_prompt_args = prompt_args(
|
provider_prompt_args = prompt_args(
|
||||||
self._agent_prompt_mode, self._prompt_path, argv=argv,
|
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
|
||||||
)
|
)
|
||||||
if self._agent_prompt_mode == "read_prompt_file":
|
if cast(PromptMode, self._agent_prompt_mode) == "read_prompt_file":
|
||||||
agent_tail += argv
|
agent_tail += argv
|
||||||
agent_tail += provider_prompt_args
|
agent_tail += provider_prompt_args
|
||||||
else:
|
else:
|
||||||
@@ -131,6 +132,11 @@ class SmolmachinesBottle(Bottle):
|
|||||||
self.agent_argv(argv, tty=tty), check=False,
|
self.agent_argv(argv, tty=tty), check=False,
|
||||||
).returncode
|
).returncode
|
||||||
|
|
||||||
|
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
|
||||||
|
# early-VM provisioning. Retry once after a short settle so
|
||||||
|
# callers (provision_ca, etc.) don't have to handle it themselves.
|
||||||
|
_SIGKILL_EXIT = 128 + 9
|
||||||
|
|
||||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||||
"""Run a POSIX shell script as `user` (default `node`) and
|
"""Run a POSIX shell script as `user` (default `node`) and
|
||||||
capture the result. Matches the docker backend's `exec`,
|
capture the result. Matches the docker backend's `exec`,
|
||||||
@@ -141,14 +147,22 @@ class SmolmachinesBottle(Bottle):
|
|||||||
|
|
||||||
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
|
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
|
||||||
without invoking a login shell, then sets HOME / USER and the
|
without invoking a login shell, then sets HOME / USER and the
|
||||||
bottle env in the child process."""
|
bottle env in the child process.
|
||||||
|
|
||||||
|
Retries once on SIGKILL (exit 137) — libkrun occasionally
|
||||||
|
kills short-lived execs during VM bring-up."""
|
||||||
|
r = self._exec_raw(script, user=user)
|
||||||
|
if r.returncode == self._SIGKILL_EXIT:
|
||||||
|
time.sleep(1.0)
|
||||||
|
r = self._exec_raw(script, user=user)
|
||||||
|
return r
|
||||||
|
|
||||||
|
def _exec_raw(self, script: str, *, user: str = "node") -> ExecResult:
|
||||||
argv = [
|
argv = [
|
||||||
"--", "runuser", "-u", user, "--",
|
"--", "runuser", "-u", user, "--",
|
||||||
"env", *_env_assignments_for(user, self._guest_env),
|
"env", *_env_assignments_for(user, self._guest_env),
|
||||||
"/bin/sh", "-c", script,
|
"/bin/sh", "-c", script,
|
||||||
]
|
]
|
||||||
# Call smolvm directly because this path needs the host-side
|
|
||||||
# subprocess capture shape used by the Docker backend.
|
|
||||||
r = subprocess.run(
|
r = subprocess.run(
|
||||||
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
||||||
capture_output=True, text=True, check=False,
|
capture_output=True, text=True, check=False,
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ from dataclasses import dataclass
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import PromptMode
|
from ...agent_provider import PromptMode
|
||||||
from ...pipelock import PipelockProxyPlan
|
|
||||||
from .. import BottlePlan
|
from .. import BottlePlan
|
||||||
|
|
||||||
|
|
||||||
@@ -71,7 +70,6 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
# docker's `--internal` + egress bridge topology; it's on a
|
# docker's `--internal` + egress bridge topology; it's on a
|
||||||
# per-bottle bridge with a pinned IP. The unused fields stay
|
# per-bottle bridge with a pinned IP. The unused fields stay
|
||||||
# at their dataclass defaults.
|
# at their dataclass defaults.
|
||||||
proxy_plan: PipelockProxyPlan
|
|
||||||
# Agent-side endpoints. On Docker Desktop the docker bridge
|
# Agent-side endpoints. On Docker Desktop the docker bridge
|
||||||
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
||||||
# networking; docker container IPs live in the daemon's VM),
|
# networking; docker container IPs live in the daemon's VM),
|
||||||
@@ -84,6 +82,14 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
agent_git_gate_host: str = ""
|
agent_git_gate_host: str = ""
|
||||||
agent_supervise_url: str = ""
|
agent_supervise_url: str = ""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def git_gate_insteadof_host(self) -> str:
|
||||||
|
return self.agent_git_gate_host
|
||||||
|
|
||||||
|
@property
|
||||||
|
def git_gate_insteadof_scheme(self) -> str:
|
||||||
|
return "http"
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def agent_command(self) -> str:
|
def agent_command(self) -> str:
|
||||||
return self.agent_provision.command
|
return self.agent_provision.command
|
||||||
|
|||||||
@@ -69,8 +69,8 @@ def enumerate_active() -> list[ActiveAgent]:
|
|||||||
|
|
||||||
|
|
||||||
def _query_bundle_services() -> dict[str, tuple[str, ...]]:
|
def _query_bundle_services() -> dict[str, tuple[str, ...]]:
|
||||||
"""`{slug: ('egress', 'pipelock', ...)}` from each running
|
"""`{slug: ('egress', ...)}` from each running bundle container's
|
||||||
bundle container's `BOT_BOTTLE_SIDECAR_DAEMONS` env var.
|
`BOT_BOTTLE_SIDECAR_DAEMONS` env var.
|
||||||
Smolmachines bundles all run the PRD-0024 image with the
|
Smolmachines bundles all run the PRD-0024 image with the
|
||||||
same daemon set declared via env, so one inspect per bundle
|
same daemon set declared via env, so one inspect per bundle
|
||||||
gets us the picture without exec'ing into the container.
|
gets us the picture without exec'ing into the container.
|
||||||
|
|||||||
@@ -9,13 +9,9 @@ guest pointed at the bundle's pinned IP via TSI's
|
|||||||
exit.
|
exit.
|
||||||
|
|
||||||
The bundle's daemons consume the inner Plans the docker backend
|
The bundle's daemons consume the inner Plans the docker backend
|
||||||
already produces: pipelock reads its yaml + CA from the
|
already produces: egress reads routes + CAs from the EgressPlan.
|
||||||
PipelockProxyPlan; egress reads routes + CAs from the EgressPlan
|
Git-gate + supervise plumb through the same plans the docker
|
||||||
+ EGRESS_UPSTREAM_PROXY pointing at `127.0.0.1:8888` (bundle
|
backend uses, minus the docker-network fields that don't apply here."""
|
||||||
local), since the agent dials pipelock first (not egress) on the
|
|
||||||
smolmachines path. Git-gate + supervise plumb through the same
|
|
||||||
plans the docker backend uses, minus the docker-network fields
|
|
||||||
that don't apply here."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -29,16 +25,11 @@ from ...egress import (
|
|||||||
EGRESS_ROUTES_IN_CONTAINER,
|
EGRESS_ROUTES_IN_CONTAINER,
|
||||||
egress_resolve_token_values,
|
egress_resolve_token_values,
|
||||||
)
|
)
|
||||||
from ...pipelock import (
|
|
||||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
|
||||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
|
||||||
)
|
|
||||||
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
||||||
from ...util import expand_tilde
|
from ...util import expand_tilde
|
||||||
from ..docker import util as docker_mod
|
from ..docker import util as docker_mod
|
||||||
from ..docker.egress import (
|
from ..docker.egress import (
|
||||||
EGRESS_CA_IN_CONTAINER,
|
EGRESS_CA_IN_CONTAINER,
|
||||||
EGRESS_PIPELOCK_CA_IN_CONTAINER,
|
|
||||||
EGRESS_PORT as _EGRESS_PORT,
|
EGRESS_PORT as _EGRESS_PORT,
|
||||||
egress_tls_init,
|
egress_tls_init,
|
||||||
)
|
)
|
||||||
@@ -48,11 +39,9 @@ from ..docker.git_gate import (
|
|||||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||||
GIT_GATE_HOOK_IN_CONTAINER,
|
GIT_GATE_HOOK_IN_CONTAINER,
|
||||||
)
|
)
|
||||||
from ..docker.pipelock import (
|
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||||
BUNDLE_LOCAL_PIPELOCK_URL,
|
from ...log import warn
|
||||||
PIPELOCK_PORT as _PIPELOCK_PORT_STR,
|
from ..docker.bottle_state import egress_state_dir, git_gate_state_dir
|
||||||
pipelock_tls_init,
|
|
||||||
)
|
|
||||||
from . import loopback_alias as _loopback
|
from . import loopback_alias as _loopback
|
||||||
from . import sidecar_bundle as _bundle
|
from . import sidecar_bundle as _bundle
|
||||||
from . import smolvm as _smolvm
|
from . import smolvm as _smolvm
|
||||||
@@ -75,9 +64,7 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
|
|||||||
# Container-internal listening ports for each bundle daemon. The
|
# Container-internal listening ports for each bundle daemon. The
|
||||||
# bundle publishes each one on a random host loopback port (see
|
# bundle publishes each one on a random host loopback port (see
|
||||||
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks
|
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks
|
||||||
# them up post-start. Pipelock's port is an env-overridable string
|
# them up post-start.
|
||||||
# in docker.pipelock; coerce to int here.
|
|
||||||
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
|
|
||||||
_GIT_HTTP_PORT = 9420
|
_GIT_HTTP_PORT = 9420
|
||||||
_SUPERVISE_PORT = SUPERVISE_PORT
|
_SUPERVISE_PORT = SUPERVISE_PORT
|
||||||
|
|
||||||
@@ -86,7 +73,7 @@ _SUPERVISE_PORT = SUPERVISE_PORT
|
|||||||
def launch(
|
def launch(
|
||||||
plan: SmolmachinesBottlePlan,
|
plan: SmolmachinesBottlePlan,
|
||||||
*,
|
*,
|
||||||
provision: Callable[[SmolmachinesBottlePlan, str], str | None],
|
provision: Callable[[SmolmachinesBottlePlan, "SmolmachinesBottle"], str | None],
|
||||||
) -> Generator[SmolmachinesBottle, None, None]:
|
) -> Generator[SmolmachinesBottle, None, None]:
|
||||||
"""Build + run the bottle and yield a handle; tear everything
|
"""Build + run the bottle and yield a handle; tear everything
|
||||||
down on exit. Errors during bringup unwind any partial state
|
down on exit. Errors during bringup unwind any partial state
|
||||||
@@ -110,17 +97,39 @@ def launch(
|
|||||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
||||||
_init_vm(plan)
|
_init_vm(plan)
|
||||||
|
|
||||||
prompt_path = provision(plan, plan.machine_name)
|
bottle = SmolmachinesBottle(
|
||||||
|
|
||||||
yield SmolmachinesBottle(
|
|
||||||
plan.machine_name,
|
plan.machine_name,
|
||||||
prompt_path=prompt_path,
|
prompt_path=None,
|
||||||
guest_env=plan.guest_env,
|
guest_env=plan.guest_env,
|
||||||
agent_command=plan.agent_command,
|
agent_command=plan.agent_command,
|
||||||
agent_prompt_mode=plan.agent_prompt_mode,
|
agent_prompt_mode=plan.agent_prompt_mode,
|
||||||
)
|
)
|
||||||
|
bottle.prompt_path = provision(plan, bottle)
|
||||||
|
|
||||||
|
yield bottle
|
||||||
finally:
|
finally:
|
||||||
|
_teardown_smolmachines(stack, plan)
|
||||||
|
|
||||||
|
|
||||||
|
def _teardown_smolmachines(
|
||||||
|
stack: ExitStack,
|
||||||
|
plan: SmolmachinesBottlePlan,
|
||||||
|
) -> None:
|
||||||
|
"""Unwind the ExitStack, then revoke any provisioned deploy keys.
|
||||||
|
|
||||||
|
ExitStack errors are caught and logged (non-fatal) so that key
|
||||||
|
revocation always runs. Revocation errors propagate — a stranded
|
||||||
|
deploy key is a security concern the operator must address."""
|
||||||
|
teardown_exc: BaseException | None = None
|
||||||
|
try:
|
||||||
stack.close()
|
stack.close()
|
||||||
|
except BaseException as exc: # noqa: W0718 — teardown must not fail
|
||||||
|
teardown_exc = exc
|
||||||
|
warn(f"smolmachines teardown failed: {exc!r}")
|
||||||
|
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
|
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
|
||||||
|
if teardown_exc is not None:
|
||||||
|
raise teardown_exc
|
||||||
|
|
||||||
|
|
||||||
def _allocate_resources(
|
def _allocate_resources(
|
||||||
@@ -142,33 +151,16 @@ def _allocate_resources(
|
|||||||
|
|
||||||
|
|
||||||
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
||||||
"""Mint per-bottle CAs and return the plan with CA paths filled.
|
"""Mint the egress MITM CA and return the plan with CA paths filled."""
|
||||||
|
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||||
Pipelock always runs in the bundle. Egress's CA is only minted
|
egress_state_dir(plan.slug),
|
||||||
when the bottle declares routes — otherwise egress runs idle
|
|
||||||
without MITM and the CA files would be unused."""
|
|
||||||
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
|
||||||
proxy_plan = dataclasses.replace(
|
|
||||||
plan.proxy_plan,
|
|
||||||
ca_cert_host_path=ca_cert_host,
|
|
||||||
ca_key_host_path=ca_key_host,
|
|
||||||
)
|
)
|
||||||
egress_plan = plan.egress_plan
|
egress_plan = dataclasses.replace(
|
||||||
if egress_plan.routes:
|
plan.egress_plan,
|
||||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
mitmproxy_ca_host_path=egress_ca_host,
|
||||||
plan.egress_plan.routes_path.parent,
|
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||||
)
|
)
|
||||||
egress_plan = dataclasses.replace(
|
return dataclasses.replace(plan, egress_plan=egress_plan)
|
||||||
egress_plan,
|
|
||||||
mitmproxy_ca_host_path=egress_ca_host,
|
|
||||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
|
||||||
pipelock_ca_host_path=ca_cert_host,
|
|
||||||
# On smolmachines, egress's upstream is pipelock on the
|
|
||||||
# bundle's localhost — they're in the same container's
|
|
||||||
# network namespace.
|
|
||||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
|
||||||
)
|
|
||||||
return dataclasses.replace(plan, proxy_plan=proxy_plan, egress_plan=egress_plan)
|
|
||||||
|
|
||||||
|
|
||||||
def _start_bundle(
|
def _start_bundle(
|
||||||
@@ -199,17 +191,10 @@ def _discover_urls(
|
|||||||
macOS networking, and macOS sees the daemon's bridge via the
|
macOS networking, and macOS sees the daemon's bridge via the
|
||||||
published-port loopback forward only.
|
published-port loopback forward only.
|
||||||
|
|
||||||
Proxy hop order: when the bottle declares egress routes, the
|
|
||||||
agent's first hop is egress (for token injection), then
|
|
||||||
pipelock. Without routes, the agent dials pipelock directly.
|
|
||||||
NO_PROXY includes the per-bottle loopback alias so the
|
NO_PROXY includes the per-bottle loopback alias so the
|
||||||
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
||||||
if plan.egress_plan.routes:
|
|
||||||
agent_facing_port = _EGRESS_PORT
|
|
||||||
else:
|
|
||||||
agent_facing_port = _PIPELOCK_PORT
|
|
||||||
agent_facing_host_port = _bundle.bundle_host_port(
|
agent_facing_host_port = _bundle.bundle_host_port(
|
||||||
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
plan.slug, _EGRESS_PORT, host_ip=loopback_ip,
|
||||||
)
|
)
|
||||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||||
|
|
||||||
@@ -303,8 +288,7 @@ def _bundle_launch_spec(
|
|||||||
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
||||||
|
|
||||||
Daemons in the CSV:
|
Daemons in the CSV:
|
||||||
- egress + pipelock are always present (pipelock is the
|
- egress is always present.
|
||||||
agent's first hop; egress is its upstream).
|
|
||||||
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
|
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
|
||||||
- supervise is conditional on plan.supervise_plan.
|
- supervise is conditional on plan.supervise_plan.
|
||||||
|
|
||||||
@@ -312,36 +296,15 @@ def _bundle_launch_spec(
|
|||||||
daemon-private values only (HTTPS_PROXY is scoped to the
|
daemon-private values only (HTTPS_PROXY is scoped to the
|
||||||
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
||||||
bind-address PR)."""
|
bind-address PR)."""
|
||||||
daemons: list[str] = ["egress", "pipelock"]
|
daemons: list[str] = ["egress"]
|
||||||
env: list[str] = []
|
env: list[str] = []
|
||||||
volumes: list[tuple[str, str, bool]] = []
|
volumes: list[tuple[str, str, bool]] = []
|
||||||
|
|
||||||
# In this Docker-Desktop-compatible topology, whichever daemon
|
|
||||||
# is "agent-facing" gets its port published on the host
|
|
||||||
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
|
||||||
# other stays bundle-internal. The bundle is NOT reachable by
|
|
||||||
# bridge IP from the smolvm guest on macOS — TSI uses macOS
|
|
||||||
# networking, and macOS sees the daemon's bridge via the
|
|
||||||
# published-port loopback forward only.
|
|
||||||
|
|
||||||
# --- pipelock ---------------------------------------------
|
|
||||||
pp = plan.proxy_plan
|
|
||||||
volumes += [
|
|
||||||
(str(pp.yaml_path), "/etc/pipelock.yaml", True),
|
|
||||||
(str(pp.ca_cert_host_path), PIPELOCK_CA_CERT_IN_CONTAINER, True),
|
|
||||||
(str(pp.ca_key_host_path), PIPELOCK_CA_KEY_IN_CONTAINER, True),
|
|
||||||
]
|
|
||||||
|
|
||||||
# --- egress -----------------------------------------------
|
# --- egress -----------------------------------------------
|
||||||
ep = plan.egress_plan
|
ep = plan.egress_plan
|
||||||
|
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
|
||||||
if ep.routes:
|
if ep.routes:
|
||||||
env.append(f"EGRESS_UPSTREAM_PROXY={ep.pipelock_proxy_url}")
|
volumes.append((str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True))
|
||||||
env.append(f"EGRESS_UPSTREAM_CA={EGRESS_PIPELOCK_CA_IN_CONTAINER}")
|
|
||||||
volumes += [
|
|
||||||
(str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True),
|
|
||||||
(str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True),
|
|
||||||
(str(ep.pipelock_ca_host_path), EGRESS_PIPELOCK_CA_IN_CONTAINER, True),
|
|
||||||
]
|
|
||||||
# Bare-name entries for upstream-token slots. Their values
|
# Bare-name entries for upstream-token slots. Their values
|
||||||
# come from the docker-run subprocess env (inherited from
|
# come from the docker-run subprocess env (inherited from
|
||||||
# the operator's shell), never landing on argv.
|
# the operator's shell), never landing on argv.
|
||||||
@@ -384,14 +347,8 @@ def _bundle_launch_spec(
|
|||||||
|
|
||||||
# Container ports the agent reaches from the smolvm guest —
|
# Container ports the agent reaches from the smolvm guest —
|
||||||
# published on host loopback so the guest can dial via TSI +
|
# published on host loopback so the guest can dial via TSI +
|
||||||
# macOS networking. The HTTP/HTTPS chokepoint is whichever
|
# macOS networking. Egress is always the agent's HTTP/HTTPS proxy.
|
||||||
# daemon's port we publish: egress when routes are declared
|
ports_to_publish: list[int] = [_EGRESS_PORT]
|
||||||
# (token injection first, then forwards to bundle-internal
|
|
||||||
# pipelock), pipelock otherwise.
|
|
||||||
if ep.routes:
|
|
||||||
ports_to_publish: list[int] = [_EGRESS_PORT]
|
|
||||||
else:
|
|
||||||
ports_to_publish = [_PIPELOCK_PORT]
|
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
ports_to_publish.append(_GIT_HTTP_PORT)
|
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||||
if sp is not None:
|
if sp is not None:
|
||||||
|
|||||||
@@ -42,13 +42,13 @@ import time
|
|||||||
import uuid
|
import uuid
|
||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Iterator
|
from typing import Generator
|
||||||
|
|
||||||
from ...log import die
|
from ...log import die
|
||||||
|
|
||||||
|
|
||||||
# registry:2.8.3, pinned by digest. Same env-override pattern as the
|
# registry:2.8.3, pinned by digest. Same env-override pattern as the
|
||||||
# pipelock image pin in bot_bottle/backend/docker/pipelock.py.
|
# sidecar image pin in bot_bottle/backend/docker/sidecar_bundle.py.
|
||||||
REGISTRY_IMAGE = os.environ.get(
|
REGISTRY_IMAGE = os.environ.get(
|
||||||
"BOT_BOTTLE_REGISTRY_IMAGE",
|
"BOT_BOTTLE_REGISTRY_IMAGE",
|
||||||
"registry@sha256:a3d8aaa63ed8681a604f1dea0aa03f100d5895b6a58ace528858a7b332415373",
|
"registry@sha256:a3d8aaa63ed8681a604f1dea0aa03f100d5895b6a58ace528858a7b332415373",
|
||||||
@@ -61,7 +61,10 @@ REGISTRY_IMAGE = os.environ.get(
|
|||||||
# narrow.
|
# narrow.
|
||||||
CRANE_IMAGE = os.environ.get(
|
CRANE_IMAGE = os.environ.get(
|
||||||
"BOT_BOTTLE_CRANE_IMAGE",
|
"BOT_BOTTLE_CRANE_IMAGE",
|
||||||
"gcr.io/go-containerregistry/crane@sha256:0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084",
|
(
|
||||||
|
"gcr.io/go-containerregistry/crane@sha256:"
|
||||||
|
"0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084"
|
||||||
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -95,7 +98,7 @@ class RegistryHandle:
|
|||||||
|
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def ephemeral_registry() -> Iterator[RegistryHandle]:
|
def ephemeral_registry() -> Generator[RegistryHandle, None, None]:
|
||||||
"""Bring up a per-session docker network + a `registry:2.8.3`
|
"""Bring up a per-session docker network + a `registry:2.8.3`
|
||||||
container on it (published on a random host port), yield a
|
container on it (published on a random host port), yield a
|
||||||
`RegistryHandle`, force-remove both on exit.
|
`RegistryHandle`, force-remove both on exit.
|
||||||
@@ -205,7 +208,6 @@ def _host_port(name: str) -> int:
|
|||||||
return int(port_str)
|
return int(port_str)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
die(f"unexpected `docker port` output: {line!r}")
|
die(f"unexpected `docker port` output: {line!r}")
|
||||||
return -1 # unreachable; die() never returns
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_ready(port: int) -> None:
|
def _wait_ready(port: int) -> None:
|
||||||
|
|||||||
@@ -47,7 +47,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import fcntl
|
import fcntl
|
||||||
import json
|
import json
|
||||||
import os
|
|
||||||
import platform
|
import platform
|
||||||
import re
|
import re
|
||||||
import sqlite3
|
import sqlite3
|
||||||
@@ -177,11 +176,11 @@ def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
|||||||
con.close()
|
con.close()
|
||||||
|
|
||||||
|
|
||||||
def allocate(slug: str) -> str:
|
def allocate(_slug: str) -> str:
|
||||||
"""Pick the lowest-numbered alias from the pool not already
|
"""Pick the lowest-numbered alias from the pool not already
|
||||||
in use by a running smolmachines bundle. Bails when the pool
|
in use by a running smolmachines bundle. Bails when the pool
|
||||||
is exhausted — the caller should report the limit to the
|
is exhausted — the caller should report the limit to the
|
||||||
operator. `slug` is logged for traceability; not otherwise
|
operator. `_slug` is logged for traceability; not otherwise
|
||||||
used (no on-disk reservation, allocation is purely
|
used (no on-disk reservation, allocation is purely
|
||||||
docker-state-driven).
|
docker-state-driven).
|
||||||
|
|
||||||
@@ -196,7 +195,7 @@ def allocate(slug: str) -> str:
|
|||||||
if not _is_macos():
|
if not _is_macos():
|
||||||
return "127.0.0.1"
|
return "127.0.0.1"
|
||||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
with open(_ALLOC_LOCK_PATH, "w") as lf:
|
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
|
||||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||||
return _allocate_locked()
|
return _allocate_locked()
|
||||||
|
|
||||||
@@ -212,7 +211,6 @@ def _allocate_locked() -> str:
|
|||||||
f"Stop a running bottle (`smolvm machine ls --json`) or "
|
f"Stop a running bottle (`smolvm machine ls --json`) or "
|
||||||
f"raise _POOL_END in loopback_alias.py."
|
f"raise _POOL_END in loopback_alias.py."
|
||||||
)
|
)
|
||||||
return "" # unreachable; die() never returns
|
|
||||||
|
|
||||||
|
|
||||||
def _alias_present(ip: str) -> bool:
|
def _alias_present(ip: str) -> bool:
|
||||||
|
|||||||
@@ -23,24 +23,21 @@ from ...backend.docker.bottle_state import (
|
|||||||
bottle_identity,
|
bottle_identity,
|
||||||
egress_state_dir,
|
egress_state_dir,
|
||||||
git_gate_state_dir,
|
git_gate_state_dir,
|
||||||
pipelock_state_dir,
|
|
||||||
supervise_state_dir,
|
supervise_state_dir,
|
||||||
write_metadata,
|
write_metadata,
|
||||||
)
|
)
|
||||||
from ...egress import Egress
|
from ...egress import Egress
|
||||||
from ...env import resolve_env
|
from ...env import resolve_env
|
||||||
from ...git_gate import GitGate
|
from ...git_gate import GitGate
|
||||||
from ...pipelock import PipelockProxy
|
|
||||||
from ...supervise import Supervise
|
from ...supervise import Supervise
|
||||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||||
from .bottle_plan import SmolmachinesBottlePlan
|
from .bottle_plan import SmolmachinesBottlePlan
|
||||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||||
|
|
||||||
|
|
||||||
# Gateway ports the bundle exposes inside its container — pipelock
|
# Gateway ports the bundle exposes inside its container — git-gate's
|
||||||
# HTTPS proxy, git-gate's git-daemon, supervise's MCP. The agent
|
# git-daemon, supervise's MCP. The agent inside the smolvm guest
|
||||||
# inside the smolvm guest dials these on the bundle's pinned IP.
|
# dials these on the bundle's pinned IP.
|
||||||
_BUNDLE_PIPELOCK_PORT = 8888
|
|
||||||
_BUNDLE_GIT_GATE_PORT = 9418
|
_BUNDLE_GIT_GATE_PORT = 9418
|
||||||
_BUNDLE_SUPERVISE_PORT = 9100
|
_BUNDLE_SUPERVISE_PORT = 9100
|
||||||
|
|
||||||
@@ -61,7 +58,7 @@ def resolve_plan(
|
|||||||
bottle = manifest.bottle_for(spec.agent_name)
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
provider = bottle.agent_provider
|
provider = bottle.agent_provider
|
||||||
provider_runtime = runtime_for(provider.template)
|
provider_runtime = runtime_for(provider.template)
|
||||||
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", "/home/node")
|
guest_home = "/home/node"
|
||||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||||
|
|
||||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||||
@@ -145,18 +142,6 @@ def resolve_plan(
|
|||||||
merged_guest_env.setdefault(key, val)
|
merged_guest_env.setdefault(key, val)
|
||||||
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
|
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
|
||||||
|
|
||||||
# Inner Plans for the four bundle daemons. The ABCs are
|
|
||||||
# platform-neutral — `.prepare()` writes config files + returns
|
|
||||||
# a Plan dataclass with no backend-specific assumptions. State
|
|
||||||
# dirs are still keyed by slug under the docker backend's
|
|
||||||
# bottle_state layout (shared on-host convention; not a docker
|
|
||||||
# dependency).
|
|
||||||
pipelock_dir = pipelock_state_dir(slug)
|
|
||||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
proxy_plan = PipelockProxy().prepare(
|
|
||||||
bottle, slug, pipelock_dir, agent_provision.egress_routes,
|
|
||||||
)
|
|
||||||
|
|
||||||
egress_dir = egress_state_dir(slug)
|
egress_dir = egress_state_dir(slug)
|
||||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||||
egress_plan = Egress().prepare(
|
egress_plan = Egress().prepare(
|
||||||
@@ -172,6 +157,7 @@ def resolve_plan(
|
|||||||
return SmolmachinesBottlePlan(
|
return SmolmachinesBottlePlan(
|
||||||
spec=spec,
|
spec=spec,
|
||||||
stage_dir=stage_dir,
|
stage_dir=stage_dir,
|
||||||
|
guest_home=guest_home,
|
||||||
slug=slug,
|
slug=slug,
|
||||||
bundle_subnet=subnet,
|
bundle_subnet=subnet,
|
||||||
bundle_gateway=gateway,
|
bundle_gateway=gateway,
|
||||||
@@ -180,7 +166,6 @@ def resolve_plan(
|
|||||||
agent_image_ref=agent_image_ref,
|
agent_image_ref=agent_image_ref,
|
||||||
guest_env=agent_provision.guest_env,
|
guest_env=agent_provision.guest_env,
|
||||||
prompt_file=prompt_file,
|
prompt_file=prompt_file,
|
||||||
proxy_plan=proxy_plan,
|
|
||||||
git_gate_plan=git_gate_plan,
|
git_gate_plan=git_gate_plan,
|
||||||
egress_plan=egress_plan,
|
egress_plan=egress_plan,
|
||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
|
|||||||
@@ -1,14 +1,13 @@
|
|||||||
"""Provisioning helpers for the smolmachines backend (PRD 0023
|
"""Backend-infrastructure provisioners for the smolmachines backend.
|
||||||
chunk 4).
|
|
||||||
|
|
||||||
Each method maps onto one of `BottleBackend`'s `provision_*`
|
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||||
overrides. They run after the VM is up + the bundle is reachable
|
declarative provision-plan apply, supervise MCP registration) live on
|
||||||
and copy host-side state (prompt, skills, .git, CA cert,
|
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
|
||||||
supervise MCP config) into the guest via `smolvm machine cp` /
|
provisioning also moved to the AgentProvider ABC (with Debian/node
|
||||||
`smolvm machine exec`.
|
defaults); user plugins override them for non-standard images.
|
||||||
|
|
||||||
Chunk 4a ships `provision_prompt` and `provision_skills` — the
|
The module left in this subpackage handles the remaining backend-
|
||||||
two that don't depend on agent-image tooling (claude-code,
|
specific step:
|
||||||
update-ca-certificates) beyond `cp` and `mkdir`. provision_ca /
|
|
||||||
provision_git / provision_supervise land once the agent-image
|
- workspace.py — copy the operator workspace into the guest
|
||||||
gap is solved."""
|
"""
|
||||||
|
|||||||
@@ -1,93 +0,0 @@
|
|||||||
"""Install the per-bottle MITM CA into the smolmachines guest's
|
|
||||||
trust store (PRD 0023 chunk 4d).
|
|
||||||
|
|
||||||
Mirrors `backend.docker.provision.ca`: select the right CA (egress
|
|
||||||
when the bottle has routes, else pipelock), `smolvm machine cp` it
|
|
||||||
to Debian's `/usr/local/share/ca-certificates/` path,
|
|
||||||
`update-ca-certificates` to rebuild the trust bundle, and log the
|
|
||||||
fingerprint once. The selected cert depends on the agent's
|
|
||||||
HTTP_PROXY target — same logic as the docker backend, since the
|
|
||||||
agent dials the same daemons through the same bundle.
|
|
||||||
|
|
||||||
`smolvm machine exec` runs commands as root in the VM (no `-u`
|
|
||||||
flag exists; the VM init is root), so we don't need the explicit
|
|
||||||
`-u 0` the docker backend uses on its `docker exec` calls."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
from ....log import die
|
|
||||||
from ...util import (
|
|
||||||
AGENT_CA_BUNDLE,
|
|
||||||
AGENT_CA_PATH,
|
|
||||||
log_ca_fingerprint,
|
|
||||||
select_ca_cert,
|
|
||||||
)
|
|
||||||
from .. import smolvm as _smolvm
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
_SIGKILL_EXIT = 128 + 9
|
|
||||||
|
|
||||||
|
|
||||||
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""Copy the agent-facing CA cert into the guest, rebuild the
|
|
||||||
trust bundle, emit a one-line fingerprint log. Called from
|
|
||||||
`BottleBackend.provision` after the smolvm guest is up."""
|
|
||||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
|
||||||
|
|
||||||
_smolvm.machine_cp(str(cert_host_path), f"{target}:{AGENT_CA_PATH}")
|
|
||||||
# Mode 0644 — readable to non-root tools in the guest.
|
|
||||||
# update-ca-certificates rebuilds the bundle at AGENT_CA_BUNDLE,
|
|
||||||
# which is what curl / Python ssl / OpenSSL-based tools read by
|
|
||||||
# default. The env trio (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE /
|
|
||||||
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
|
||||||
# `requests` / libraries that don't load the system bundle.
|
|
||||||
#
|
|
||||||
r = _install_ca(target)
|
|
||||||
if r.returncode == _SIGKILL_EXIT:
|
|
||||||
# smolvm/libkrun can SIGKILL an otherwise-normal exec
|
|
||||||
# during early-VM provisioning. `update-ca-certificates`
|
|
||||||
# is idempotent, so retry the same install once after a
|
|
||||||
# short settle delay before treating it as fatal.
|
|
||||||
time.sleep(1.0)
|
|
||||||
r = _install_ca(target)
|
|
||||||
|
|
||||||
if r.returncode != 0:
|
|
||||||
# update-ca-certificates not adding our cert is fatal —
|
|
||||||
# claude-code's TLS handshake against the egress-MITM'd
|
|
||||||
# api.anthropic.com would fail downstream. Bail early
|
|
||||||
# with what we can see (output is captured by smolvm so
|
|
||||||
# we can surface it).
|
|
||||||
die(
|
|
||||||
f"update-ca-certificates didn't add the agent CA "
|
|
||||||
f"(exit {r.returncode}): "
|
|
||||||
f"stdout={(r.stdout or '').strip()!r} "
|
|
||||||
f"stderr={(r.stderr or '').strip()!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
log_ca_fingerprint(cert_host_path, label)
|
|
||||||
|
|
||||||
|
|
||||||
def _install_ca(target: str) -> _smolvm.SmolvmRunResult:
|
|
||||||
# chown + chmod + update-ca-certificates + bundle
|
|
||||||
# verification run in one `sh -c` so we only pay one
|
|
||||||
# machine_exec round trip; the `&&` chaining surfaces the
|
|
||||||
# first failure as the return code. The verify check is more
|
|
||||||
# stable than requiring "1 added" in stdout: a retry after a
|
|
||||||
# partially-completed first run may legitimately report "0
|
|
||||||
# added" while the cert is already installed.
|
|
||||||
return _smolvm.machine_exec(target, [
|
|
||||||
"sh", "-c",
|
|
||||||
f"chown root:root {AGENT_CA_PATH} && "
|
|
||||||
f"chmod 644 {AGENT_CA_PATH} && "
|
|
||||||
f"update-ca-certificates && "
|
|
||||||
f"openssl verify -CAfile {AGENT_CA_BUNDLE} {AGENT_CA_PATH}",
|
|
||||||
])
|
|
||||||
|
|
||||||
|
|
||||||
# Re-exported for the launch/provision_ca caller + tests. The path
|
|
||||||
# constants live in the shared `backend.util` (Debian's
|
|
||||||
# `update-ca-certificates` layout is the same in both backends).
|
|
||||||
__all__ = ["AGENT_CA_BUNDLE", "AGENT_CA_PATH", "provision_ca"]
|
|
||||||
@@ -1,145 +0,0 @@
|
|||||||
"""Git provisioning inside a running smolmachines bottle
|
|
||||||
(PRD 0023 chunk 4d).
|
|
||||||
|
|
||||||
Three concerns, all about git in the agent:
|
|
||||||
|
|
||||||
1. If --cwd was passed AND the host cwd has a .git, copy that
|
|
||||||
.git into the planned guest workspace so the agent operates on
|
|
||||||
the user's repo.
|
|
||||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
|
||||||
~/.gitconfig with insteadOf rules so every git operation
|
|
||||||
against a declared upstream transparently hits the per-bottle
|
|
||||||
git-gate. The gate mirrors the upstream in both directions,
|
|
||||||
so URL rewriting is symmetric.
|
|
||||||
3. If the bottle declares `git.user` (issue #86), set
|
|
||||||
`git config --global user.{name,email}` inside the guest so
|
|
||||||
the agent's commits are attributed to that identity.
|
|
||||||
|
|
||||||
Differs from `backend.docker.provision.git` in one address detail:
|
|
||||||
the TSI-allowlisted guest can only reach the bundle's pinned IP
|
|
||||||
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
|
|
||||||
are `http://<bundle_ip>:<port>/<name>.git` rather than the
|
|
||||||
docker backend's `git://git-gate/<name>.git`. The render itself
|
|
||||||
is the shared `git_gate_render_gitconfig` on the platform-neutral
|
|
||||||
git_gate module."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import tempfile
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ....git_gate import git_gate_render_gitconfig
|
|
||||||
from ....log import info
|
|
||||||
from .. import smolvm as _smolvm
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
# `node` is the agent user from the repo Dockerfile. Override via
|
|
||||||
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
|
||||||
# BOT_BOTTLE_CONTAINER_HOME knob — same purpose, different
|
|
||||||
# transport.
|
|
||||||
_DEFAULT_GUEST_HOME = "/home/node"
|
|
||||||
|
|
||||||
|
|
||||||
def _guest_home() -> str:
|
|
||||||
return os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
|
||||||
|
|
||||||
|
|
||||||
def provision_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""Set up git inside the guest. Runs all three subcases; each
|
|
||||||
no-ops when its condition isn't met."""
|
|
||||||
_provision_cwd_git(plan, target)
|
|
||||||
_provision_git_gate_config(plan, target)
|
|
||||||
_provision_git_user(plan, target)
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_cwd_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
|
||||||
it into <guest_home>/workspace/.git and fix ownership. No-op
|
|
||||||
otherwise."""
|
|
||||||
workspace = plan.workspace_plan
|
|
||||||
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
|
||||||
return
|
|
||||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
|
||||||
host_git = str(workspace.host_path / ".git")
|
|
||||||
info(f"copying {host_git} -> {target}:{guest_workspace_git}")
|
|
||||||
# mkdir -p the workspace dir so `machine cp` lands the .git
|
|
||||||
# directly there even on first-time bottles.
|
|
||||||
_smolvm.machine_exec(target, ["mkdir", "-p", workspace.guest_path])
|
|
||||||
_smolvm.machine_cp(
|
|
||||||
host_git, f"{target}:{guest_workspace_git}",
|
|
||||||
)
|
|
||||||
# `machine cp` lands files as root; the agent runs as node so
|
|
||||||
# the workspace tree must be chowned over.
|
|
||||||
_smolvm.machine_exec(
|
|
||||||
target, ["chown", "-R", workspace.owner, guest_workspace_git],
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_gate_config(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""Write ~/.gitconfig in the guest with the git-gate insteadOf
|
|
||||||
rules. No-op when the bottle has no `git` entries."""
|
|
||||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
|
||||||
if not bottle.git:
|
|
||||||
return
|
|
||||||
|
|
||||||
# `<loopback alias>:<host port>` form: the bundle's git-gate
|
|
||||||
# HTTP port is published on host loopback at launch time so
|
|
||||||
# the smolvm guest (which can only reach macOS networking via
|
|
||||||
# TSI, not the docker bridge IP) can dial it. launch.py
|
|
||||||
# populates `plan.agent_git_gate_host` after bundle bringup.
|
|
||||||
content = git_gate_render_gitconfig(
|
|
||||||
bottle.git, plan.agent_git_gate_host, scheme="http",
|
|
||||||
)
|
|
||||||
|
|
||||||
guest_gitconfig = f"{_guest_home()}/.gitconfig"
|
|
||||||
# Stage the file under the plan's stage_dir so `machine cp`
|
|
||||||
# has a stable host path. The plan's stage_dir is cleaned up
|
|
||||||
# by start.py's session-end teardown.
|
|
||||||
with tempfile.NamedTemporaryFile(
|
|
||||||
"w", dir=str(plan.stage_dir), prefix="gitconfig.",
|
|
||||||
delete=False,
|
|
||||||
) as f:
|
|
||||||
f.write(content)
|
|
||||||
config_file = Path(f.name)
|
|
||||||
os.chmod(config_file, 0o600)
|
|
||||||
|
|
||||||
info(f"writing {guest_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
|
||||||
_smolvm.machine_cp(str(config_file), f"{target}:{guest_gitconfig}")
|
|
||||||
_smolvm.machine_exec(target, ["chown", "node:node", guest_gitconfig])
|
|
||||||
_smolvm.machine_exec(target, ["chmod", "644", guest_gitconfig])
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_user(
|
|
||||||
plan: SmolmachinesBottlePlan, target: str,
|
|
||||||
) -> None:
|
|
||||||
"""Apply `git config --global user.{name,email}` inside the
|
|
||||||
guest as the node user so --global lands in the same
|
|
||||||
`/home/node/.gitconfig` that `_provision_git_gate_config`
|
|
||||||
writes to. No-op when the bottle didn't declare `git.user`.
|
|
||||||
|
|
||||||
Runs via `runuser -u node --`; HOME is forced via smolvm's
|
|
||||||
`-e` flag because runuser (without -l) inherits root's
|
|
||||||
HOME=/root, which would put --global in the wrong file."""
|
|
||||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
|
||||||
gu = bottle.git_user
|
|
||||||
if gu.is_empty():
|
|
||||||
return
|
|
||||||
env = {"HOME": _guest_home(), "USER": "node"}
|
|
||||||
if gu.name:
|
|
||||||
info(f"git config --global user.name = {gu.name!r}")
|
|
||||||
_smolvm.machine_exec(
|
|
||||||
target,
|
|
||||||
["runuser", "-u", "node", "--",
|
|
||||||
"git", "config", "--global", "user.name", gu.name],
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
if gu.email:
|
|
||||||
info(f"git config --global user.email = {gu.email!r}")
|
|
||||||
_smolvm.machine_exec(
|
|
||||||
target,
|
|
||||||
["runuser", "-u", "node", "--",
|
|
||||||
"git", "config", "--global", "user.email", gu.email],
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
"""Copy the agent prompt into a running smolmachines bottle.
|
|
||||||
|
|
||||||
The prompt file is always copied (so the in-guest path always
|
|
||||||
exists) but `--append-system-prompt-file` only fires when the
|
|
||||||
agent actually has a prompt — the return value signals which
|
|
||||||
case, mirroring the docker backend's contract.
|
|
||||||
|
|
||||||
`smolvm machine cp` lands files as root inside the VM; the claude
|
|
||||||
process runs as `node`, so we chown + chmod the prompt after the
|
|
||||||
copy. Same flow as the docker backend's provision_prompt."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
|
|
||||||
from .. import smolvm as _smolvm
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
# `node` is the agent user from the repo Dockerfile.
|
|
||||||
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
|
||||||
# BOT_BOTTLE_CONTAINER_HOME knob.
|
|
||||||
_DEFAULT_GUEST_HOME = "/home/node"
|
|
||||||
|
|
||||||
|
|
||||||
def provision_prompt(plan: SmolmachinesBottlePlan, target: str) -> str | None:
|
|
||||||
"""Copy the prompt file into the running smolvm guest, fix
|
|
||||||
ownership/mode. Returns the in-guest path if the agent has a
|
|
||||||
non-empty prompt (drives --append-system-prompt-file), else
|
|
||||||
None. The file is copied either way so the path always
|
|
||||||
exists — mirrors the docker backend's behavior."""
|
|
||||||
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
|
||||||
in_guest_prompt_path = f"{guest_home}/.bot-bottle-prompt.txt"
|
|
||||||
|
|
||||||
_smolvm.machine_cp(str(plan.prompt_file), f"{target}:{in_guest_prompt_path}")
|
|
||||||
# machine cp lands as root, source's 0o600 mode is preserved —
|
|
||||||
# node can't read its own prompt without these two.
|
|
||||||
_smolvm.machine_exec(target, ["chown", "node:node", in_guest_prompt_path])
|
|
||||||
_smolvm.machine_exec(target, ["chmod", "600", in_guest_prompt_path])
|
|
||||||
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
return in_guest_prompt_path if agent.prompt else None
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
"""Provision non-secret provider auth markers into a smolmachines bottle."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from ....log import die
|
|
||||||
from .. import smolvm as _smolvm
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_provider_auth(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""Apply provider-owned guest setup through smolvm primitives."""
|
|
||||||
provision = plan.agent_provision
|
|
||||||
for d in provision.dirs:
|
|
||||||
_exec(target, ["mkdir", "-p", d.guest_path], f"could not create {d.guest_path}")
|
|
||||||
_exec(target, ["chown", d.owner, d.guest_path], f"could not chown {d.guest_path}")
|
|
||||||
_exec(target, ["chmod", d.mode, d.guest_path], f"could not chmod {d.guest_path}")
|
|
||||||
for command in provision.pre_copy:
|
|
||||||
_exec(target, list(command.argv), command.error)
|
|
||||||
for f in provision.files:
|
|
||||||
_smolvm.machine_cp(str(f.host_path), f"{target}:{f.guest_path}")
|
|
||||||
_exec(target, ["chown", f.owner, f.guest_path], f"could not chown {f.guest_path}")
|
|
||||||
_exec(target, ["chmod", f.mode, f.guest_path], f"could not chmod {f.guest_path}")
|
|
||||||
for command in provision.verify:
|
|
||||||
_exec(target, list(command.argv), command.error)
|
|
||||||
|
|
||||||
|
|
||||||
def _exec(target: str, argv: list[str], error: str) -> None:
|
|
||||||
result = _smolvm.machine_exec(target, argv)
|
|
||||||
if result.returncode != 0:
|
|
||||||
detail = (result.stderr or result.stdout).strip()
|
|
||||||
if detail:
|
|
||||||
detail = f": {detail}"
|
|
||||||
die(f"agent provider provisioning: {error}{detail}")
|
|
||||||
@@ -1,63 +0,0 @@
|
|||||||
"""Copy host-side skill directories into a running smolmachines
|
|
||||||
bottle.
|
|
||||||
|
|
||||||
Skills are validated on the host before launch by
|
|
||||||
`BottleBackend._validate_skills`; this module assumes that
|
|
||||||
validation has already run. A skill that disappears between
|
|
||||||
validation and copy still dies loudly rather than silently
|
|
||||||
producing a partial guest."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
|
|
||||||
from ....log import die, info
|
|
||||||
from ...util import host_skill_dir
|
|
||||||
from .. import smolvm as _smolvm
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
# In-guest path mirrors the docker backend's claude-skills
|
|
||||||
# convention (~/.claude/skills/<name>/) under the node user's
|
|
||||||
# home — same path as the real bot-bottle image's
|
|
||||||
# /home/node/.claude/skills (pre-created in the Dockerfile).
|
|
||||||
_DEFAULT_SKILLS_DIR = "/home/node/.claude/skills"
|
|
||||||
|
|
||||||
|
|
||||||
def provision_skills(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""Copy each of the agent's named skills from the host's
|
|
||||||
~/.claude/skills/<name>/ into the guest's equivalent path.
|
|
||||||
For each skill: `mkdir -p` the destination, `smolvm machine cp`
|
|
||||||
the host source dir over, then chown the result to node:node so
|
|
||||||
the agent can read it. No-op when the agent has no skills.
|
|
||||||
|
|
||||||
smolvm machine cp on a directory copies recursively (same
|
|
||||||
semantics as `cp -r`); unlike docker cp's trailing-slash
|
|
||||||
convention, smolvm doesn't need the `/.` suffix dance.
|
|
||||||
|
|
||||||
machine cp lands files as root inside the VM, so we chown each
|
|
||||||
skill tree over to node:node after the copy — same pattern as
|
|
||||||
the docker backend's provision_prompt."""
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
if not agent.skills:
|
|
||||||
return
|
|
||||||
|
|
||||||
skills_dir = os.environ.get(
|
|
||||||
"BOT_BOTTLE_GUEST_SKILLS_DIR", _DEFAULT_SKILLS_DIR,
|
|
||||||
)
|
|
||||||
|
|
||||||
_smolvm.machine_exec(target, ["mkdir", "-p", skills_dir])
|
|
||||||
|
|
||||||
for name in agent.skills:
|
|
||||||
src = host_skill_dir(name)
|
|
||||||
if not os.path.isdir(src):
|
|
||||||
die(
|
|
||||||
f"skill {name!r} disappeared from host between "
|
|
||||||
f"validation and copy at {src}."
|
|
||||||
)
|
|
||||||
dst = f"{skills_dir}/{name}"
|
|
||||||
info(f"copying skill {name} into {target}:{dst}")
|
|
||||||
# Wipe any prior copy so re-runs don't accumulate.
|
|
||||||
_smolvm.machine_exec(target, ["rm", "-rf", dst])
|
|
||||||
_smolvm.machine_cp(src, f"{target}:{dst}")
|
|
||||||
_smolvm.machine_exec(target, ["chown", "-R", "node:node", dst])
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
"""Supervise sidecar provisioning inside a running smolmachines
|
|
||||||
bottle (PRD 0023 chunk 4d; PRD 0013 supervise plane).
|
|
||||||
|
|
||||||
Registers the per-bottle supervise sidecar as an HTTP MCP server
|
|
||||||
in the agent's claude-code config so the agent discovers the
|
|
||||||
stuck-recovery MCP tools (pipelock-block, capability-block) at
|
|
||||||
startup.
|
|
||||||
|
|
||||||
Mirrors `backend.docker.provision.supervise` — same `claude mcp
|
|
||||||
add` call, just dispatched via `smolvm machine exec` instead of
|
|
||||||
`docker exec`, and against `<bundle_ip>:<port>` instead of the
|
|
||||||
short `supervise` alias (no DNS in the TSI-allowlisted guest)."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from ....log import info, warn
|
|
||||||
from .. import smolvm as _smolvm
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
_SUPERVISE_MCP_NAME = "supervise"
|
|
||||||
|
|
||||||
|
|
||||||
def provision_supervise(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|
||||||
"""Run `claude mcp add` inside the guest to register the
|
|
||||||
supervise sidecar in claude-code's user config. No-op when
|
|
||||||
bottle.supervise is False.
|
|
||||||
|
|
||||||
The URL is the agent-side endpoint launch.py populated after
|
|
||||||
bundle bringup — `http://127.0.0.1:<host port>/` rather than
|
|
||||||
the bundle's docker bridge IP, because that bridge isn't
|
|
||||||
reachable from the smolvm guest on macOS.
|
|
||||||
|
|
||||||
Failure is logged but not fatal: the bottle still works (you
|
|
||||||
just can't call supervise tools from the agent until the entry
|
|
||||||
is added manually). The operator sees the warning at launch."""
|
|
||||||
if plan.supervise_plan is None:
|
|
||||||
return
|
|
||||||
url = plan.agent_supervise_url
|
|
||||||
info(f"registering supervise MCP server in agent claude config → {url}")
|
|
||||||
# `claude mcp add --scope user` writes to ~/.claude.json. The
|
|
||||||
# agent is the `node` user; smolvm machine_exec runs as root
|
|
||||||
# by default, so we have to switch user explicitly and set
|
|
||||||
# HOME so the config lands in /home/node/.claude.json (where
|
|
||||||
# the agent's claude actually reads it from).
|
|
||||||
r = _smolvm.machine_exec(
|
|
||||||
target,
|
|
||||||
[
|
|
||||||
"runuser", "-u", "node", "--",
|
|
||||||
"env", "HOME=/home/node",
|
|
||||||
"claude", "mcp", "add",
|
|
||||||
"--scope", "user",
|
|
||||||
"--transport", "http",
|
|
||||||
_SUPERVISE_MCP_NAME,
|
|
||||||
url,
|
|
||||||
],
|
|
||||||
)
|
|
||||||
if r.returncode != 0:
|
|
||||||
warn(
|
|
||||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
|
||||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
|
||||||
f"register manually with: "
|
|
||||||
f"claude mcp add --scope user --transport http supervise {url}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["provision_supervise"]
|
|
||||||
@@ -5,11 +5,11 @@ from __future__ import annotations
|
|||||||
import shlex
|
import shlex
|
||||||
|
|
||||||
from ....log import info
|
from ....log import info
|
||||||
from .. import smolvm as _smolvm
|
from ... import Bottle
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
def provision_workspace(plan: SmolmachinesBottlePlan, target: str) -> None:
|
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||||
"""Copy host cwd contents to the planned guest workspace."""
|
"""Copy host cwd contents to the planned guest workspace."""
|
||||||
workspace = plan.workspace_plan
|
workspace = plan.workspace_plan
|
||||||
if not (workspace.enabled and workspace.copy_contents):
|
if not (workspace.enabled and workspace.copy_contents):
|
||||||
@@ -20,17 +20,13 @@ def provision_workspace(plan: SmolmachinesBottlePlan, target: str) -> None:
|
|||||||
guest_parent_q = shlex.quote(guest_parent)
|
guest_parent_q = shlex.quote(guest_parent)
|
||||||
owner_q = shlex.quote(workspace.owner)
|
owner_q = shlex.quote(workspace.owner)
|
||||||
mode_q = shlex.quote(workspace.mode)
|
mode_q = shlex.quote(workspace.mode)
|
||||||
info(f"copying {workspace.host_path} -> {target}:{workspace.guest_path}")
|
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
|
||||||
_smolvm.machine_exec(
|
bottle.exec(
|
||||||
target,
|
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
|
||||||
["sh", "-c", f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}"],
|
user="root",
|
||||||
)
|
)
|
||||||
_smolvm.machine_cp(str(workspace.host_path), f"{target}:{workspace.guest_path}")
|
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
|
||||||
_smolvm.machine_exec(
|
bottle.exec(
|
||||||
target,
|
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
|
||||||
[
|
user="root",
|
||||||
"sh", "-c",
|
|
||||||
f"chown -R {owner_q} {guest_path_q} && "
|
|
||||||
f"chmod {mode_q} {guest_path_q}",
|
|
||||||
],
|
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -42,6 +42,7 @@ import subprocess
|
|||||||
import sys
|
import sys
|
||||||
import termios
|
import termios
|
||||||
import threading
|
import threading
|
||||||
|
from types import FrameType
|
||||||
|
|
||||||
|
|
||||||
# How long to wait after the main exec starts before pushing the
|
# How long to wait after the main exec starts before pushing the
|
||||||
@@ -67,8 +68,9 @@ def _read_winsize() -> tuple[int, int] | None:
|
|||||||
- tmux respawn-pane: tmux sets all three to the pane's PTY.
|
- tmux respawn-pane: tmux sets all three to the pane's PTY.
|
||||||
- non-TTY (someone piped stdin in tests): none are; the
|
- non-TTY (someone piped stdin in tests): none are; the
|
||||||
sync just no-ops, which is the right behavior."""
|
sync just no-ops, which is the right behavior."""
|
||||||
for fd in (sys.stdin.fileno(), sys.stdout.fileno(), sys.stderr.fileno()):
|
for stream in (sys.stdin, sys.stdout, sys.stderr):
|
||||||
try:
|
try:
|
||||||
|
fd = stream.fileno()
|
||||||
data = fcntl.ioctl(fd, termios.TIOCGWINSZ, b"\x00" * 8)
|
data = fcntl.ioctl(fd, termios.TIOCGWINSZ, b"\x00" * 8)
|
||||||
except OSError:
|
except OSError:
|
||||||
continue
|
continue
|
||||||
@@ -123,13 +125,13 @@ def main(argv: list[str]) -> int:
|
|||||||
machine = argv[0]
|
machine = argv[0]
|
||||||
inner = argv[2:]
|
inner = argv[2:]
|
||||||
|
|
||||||
def sync(*_args) -> None:
|
def sync(_signum: int | None = None, _frame: FrameType | None = None) -> None:
|
||||||
size = _read_winsize()
|
size = _read_winsize()
|
||||||
if size is None:
|
if size is None:
|
||||||
return
|
return
|
||||||
_push_size(machine, *size)
|
_push_size(machine, *size)
|
||||||
|
|
||||||
signal.signal(signal.SIGWINCH, sync)
|
signal.signal(signal.SIGWINCH, sync) # type: ignore[arg-type]
|
||||||
|
|
||||||
proc = subprocess.Popen(inner)
|
proc = subprocess.Popen(inner)
|
||||||
# Initial sync is deferred — see _STARTUP_SYNC_DELAY_SEC.
|
# Initial sync is deferred — see _STARTUP_SYNC_DELAY_SEC.
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ This module ships the lifecycle primitives only — create
|
|||||||
network, start bundle, stop bundle, remove network — wrapped
|
network, start bundle, stop bundle, remove network — wrapped
|
||||||
around `subprocess.run(["docker", ...])`. Wiring them into the
|
around `subprocess.run(["docker", ...])`. Wiring them into the
|
||||||
launch flow + populating the `BundleLaunchSpec` from the inner
|
launch flow + populating the `BundleLaunchSpec` from the inner
|
||||||
Plans (PipelockProxyPlan, EgressPlan, …) lands in chunk 2d."""
|
Plans (EgressPlan, …) lands in chunk 2d."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -69,7 +69,7 @@ class BundleLaunchSpec:
|
|||||||
# Daemon subset CSV for BOT_BOTTLE_SIDECAR_DAEMONS. The
|
# Daemon subset CSV for BOT_BOTTLE_SIDECAR_DAEMONS. The
|
||||||
# supervisor inside the bundle reads it to skip
|
# supervisor inside the bundle reads it to skip
|
||||||
# bottle-irrelevant daemons (e.g. supervise=False bottles).
|
# bottle-irrelevant daemons (e.g. supervise=False bottles).
|
||||||
daemons_csv: str = "egress,pipelock"
|
daemons_csv: str = "egress"
|
||||||
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name
|
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name
|
||||||
# form inherits the value from the docker-run subprocess env,
|
# form inherits the value from the docker-run subprocess env,
|
||||||
# matching the docker backend's compose-up secret-forwarding
|
# matching the docker backend's compose-up secret-forwarding
|
||||||
@@ -223,7 +223,6 @@ def bundle_host_port(
|
|||||||
f"no port mapping on {host_ip} for {container} "
|
f"no port mapping on {host_ip} for {container} "
|
||||||
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
|
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
|
||||||
)
|
)
|
||||||
return -1 # unreachable; die() never returns
|
|
||||||
|
|
||||||
|
|
||||||
def stop_bundle(slug: str) -> None:
|
def stop_bundle(slug: str) -> None:
|
||||||
|
|||||||
@@ -52,7 +52,7 @@ class SmolvmError(RuntimeError):
|
|||||||
pack failed, etc.). Carries the captured stderr for the
|
pack failed, etc.). Carries the captured stderr for the
|
||||||
operator-facing log line."""
|
operator-facing log line."""
|
||||||
|
|
||||||
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess):
|
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess[str]):
|
||||||
self.argv = list(argv)
|
self.argv = list(argv)
|
||||||
self.returncode = result.returncode
|
self.returncode = result.returncode
|
||||||
self.stdout = result.stdout
|
self.stdout = result.stdout
|
||||||
@@ -65,7 +65,7 @@ class SmolvmError(RuntimeError):
|
|||||||
|
|
||||||
|
|
||||||
def _smolvm(*args: str, env: Mapping[str, str] | None = None,
|
def _smolvm(*args: str, env: Mapping[str, str] | None = None,
|
||||||
check: bool = True) -> subprocess.CompletedProcess:
|
check: bool = True) -> subprocess.CompletedProcess[str]:
|
||||||
"""One subprocess call into the smolvm CLI. `check=True`
|
"""One subprocess call into the smolvm CLI. `check=True`
|
||||||
raises SmolvmError on non-zero; `check=False` returns the
|
raises SmolvmError on non-zero; `check=False` returns the
|
||||||
CompletedProcess for the caller to inspect."""
|
CompletedProcess for the caller to inspect."""
|
||||||
|
|||||||
+11
-27
@@ -14,7 +14,6 @@ from ..log import die, info
|
|||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from ..egress import EgressPlan
|
from ..egress import EgressPlan
|
||||||
from ..pipelock import PipelockProxyPlan
|
|
||||||
|
|
||||||
|
|
||||||
# Debian-family CA layout, shared by every backend (all guest images
|
# Debian-family CA layout, shared by every backend (all guest images
|
||||||
@@ -35,35 +34,20 @@ def host_skill_dir(name: str) -> str:
|
|||||||
return f"{home}/.claude/skills/{name}"
|
return f"{home}/.claude/skills/{name}"
|
||||||
|
|
||||||
|
|
||||||
def select_ca_cert(
|
def select_ca_cert(egress_plan: EgressPlan) -> tuple[Path, str]:
|
||||||
egress_plan: EgressPlan, proxy_plan: PipelockProxyPlan
|
"""Return the egress MITM CA cert path and label for provision_ca.
|
||||||
) -> tuple[Path, str]:
|
|
||||||
"""Pick the agent-facing CA cert (and a short label for the log
|
|
||||||
line) that matches the proxy the agent's HTTP_PROXY points at.
|
|
||||||
Egress wins when the bottle declares any routes (it sits in front
|
|
||||||
of pipelock); else pipelock.
|
|
||||||
|
|
||||||
Shared by every backend's `provision_ca`: launch mints the chosen
|
Launch always mints the CA and re-binds the host path into the
|
||||||
CA(s) and re-binds their host paths into these inner plans before
|
egress_plan before provision runs, so an empty/missing path here
|
||||||
provision runs, so an empty/missing path here means launch's
|
means launch's bringup is broken — fatal."""
|
||||||
bringup is broken — fatal."""
|
cert = egress_plan.mitmproxy_ca_cert_only_host_path
|
||||||
if egress_plan.routes:
|
if cert == Path() or not cert.is_file():
|
||||||
cert = egress_plan.mitmproxy_ca_cert_only_host_path
|
|
||||||
if cert == Path() or not cert.is_file():
|
|
||||||
die(
|
|
||||||
f"egress CA cert missing at {cert or '(empty)'}; "
|
|
||||||
f"launch must have called egress_tls_init and "
|
|
||||||
f"re-bound the plan before provision"
|
|
||||||
)
|
|
||||||
return cert, "egress"
|
|
||||||
cert = proxy_plan.ca_cert_host_path
|
|
||||||
if not cert or not cert.is_file():
|
|
||||||
die(
|
die(
|
||||||
f"pipelock CA cert missing at {cert or '(empty)'}; "
|
f"egress CA cert missing at {cert or '(empty)'}; "
|
||||||
f"launch must have called pipelock_tls_init and re-bound "
|
f"launch must have called egress_tls_init and "
|
||||||
f"the plan before provision"
|
f"re-bound the plan before provision"
|
||||||
)
|
)
|
||||||
return cert, "pipelock"
|
return cert, "egress"
|
||||||
|
|
||||||
|
|
||||||
def log_ca_fingerprint(cert_host_path: Path, label: str) -> None:
|
def log_ca_fingerprint(cert_host_path: Path, label: str) -> None:
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Main CLI dispatcher.
|
"""Main CLI dispatcher.
|
||||||
|
|
||||||
Commands: cleanup, dashboard, edit, info, init, list, resume, start
|
Commands: cleanup, edit, info, init, list, resume, start, supervise
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -12,24 +12,24 @@ from ..manifest import ManifestError
|
|||||||
from ._common import PROG
|
from ._common import PROG
|
||||||
from . import list as _list_mod
|
from . import list as _list_mod
|
||||||
from .cleanup import cmd_cleanup
|
from .cleanup import cmd_cleanup
|
||||||
from .dashboard import cmd_dashboard
|
|
||||||
from .edit import cmd_edit
|
from .edit import cmd_edit
|
||||||
from .info import cmd_info
|
from .info import cmd_info
|
||||||
from .init import cmd_init
|
from .init import cmd_init
|
||||||
from .resume import cmd_resume
|
from .resume import cmd_resume
|
||||||
from .start import cmd_start
|
from .start import cmd_start
|
||||||
|
from .supervise import cmd_supervise
|
||||||
|
|
||||||
cmd_list = _list_mod.cmd_list
|
cmd_list = _list_mod.cmd_list
|
||||||
|
|
||||||
COMMANDS = {
|
COMMANDS = {
|
||||||
"cleanup": cmd_cleanup,
|
"cleanup": cmd_cleanup,
|
||||||
"dashboard": cmd_dashboard,
|
|
||||||
"edit": cmd_edit,
|
"edit": cmd_edit,
|
||||||
"info": cmd_info,
|
"info": cmd_info,
|
||||||
"init": cmd_init,
|
"init": cmd_init,
|
||||||
"list": cmd_list,
|
"list": cmd_list,
|
||||||
"resume": cmd_resume,
|
"resume": cmd_resume,
|
||||||
"start": cmd_start,
|
"start": cmd_start,
|
||||||
|
"supervise": cmd_supervise,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -37,13 +37,22 @@ def usage() -> None:
|
|||||||
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
||||||
sys.stderr.write("Commands:\n")
|
sys.stderr.write("Commands:\n")
|
||||||
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
||||||
sys.stderr.write(" dashboard view + approve/modify/reject pending supervise proposals (PRD 0013)\n")
|
|
||||||
sys.stderr.write(" edit open an agent in vim for editing\n")
|
sys.stderr.write(" edit open an agent in vim for editing\n")
|
||||||
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
||||||
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
||||||
sys.stderr.write(" list list available agents or active containers\n")
|
sys.stderr.write(" list list available agents or active containers\n")
|
||||||
sys.stderr.write(" resume re-launch a bottle by its identity (continues state from PRD 0016)\n")
|
sys.stderr.write(
|
||||||
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n\n")
|
" resume re-launch a bottle by its identity "
|
||||||
|
"(continues state from PRD 0016)\n"
|
||||||
|
)
|
||||||
|
sys.stderr.write(
|
||||||
|
" start boot a container for a named agent and "
|
||||||
|
"attach an interactive session\n"
|
||||||
|
)
|
||||||
|
sys.stderr.write(
|
||||||
|
" supervise view + approve/modify/reject pending supervise "
|
||||||
|
"proposals (PRD 0013)\n\n"
|
||||||
|
)
|
||||||
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
|
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
|
|||||||
def read_tty_line() -> str:
|
def read_tty_line() -> str:
|
||||||
"""Mirror `IFS= read -r REPLY </dev/tty`. Falls back to stdin."""
|
"""Mirror `IFS= read -r REPLY </dev/tty`. Falls back to stdin."""
|
||||||
try:
|
try:
|
||||||
with open("/dev/tty", "r") as tty:
|
with open("/dev/tty", "r", encoding="utf-8") as tty:
|
||||||
return tty.readline().rstrip("\n")
|
return tty.readline().rstrip("\n")
|
||||||
except OSError:
|
except OSError:
|
||||||
return sys.stdin.readline().rstrip("\n")
|
return sys.stdin.readline().rstrip("\n")
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
+18
-5
@@ -51,7 +51,8 @@ def cmd_init(argv: list[str]) -> int:
|
|||||||
die(f"{target_file} exists but is not valid JSON; fix or remove it first")
|
die(f"{target_file} exists but is not valid JSON; fix or remove it first")
|
||||||
if agent_name in (existing.get("agents") or {}):
|
if agent_name in (existing.get("agents") or {}):
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
f'bot-bottle: agent "{agent_name}" already exists in {target_file}. Overwrite? [y/N] '
|
f'bot-bottle: agent "{agent_name}" already exists in '
|
||||||
|
f'{target_file}. Overwrite? [y/N] '
|
||||||
)
|
)
|
||||||
sys.stderr.flush()
|
sys.stderr.flush()
|
||||||
ow = read_tty_line()
|
ow = read_tty_line()
|
||||||
@@ -71,7 +72,10 @@ def cmd_init(argv: list[str]) -> int:
|
|||||||
|
|
||||||
# Prompt
|
# Prompt
|
||||||
print(file=sys.stderr)
|
print(file=sys.stderr)
|
||||||
info("System prompt — enter text, then a lone '.' on its own line to finish (just '.' to leave empty):")
|
info(
|
||||||
|
"System prompt — enter text, then a lone '.' on its own line to "
|
||||||
|
"finish (just '.' to leave empty):"
|
||||||
|
)
|
||||||
prompt_lines: list[str] = []
|
prompt_lines: list[str] = []
|
||||||
while True:
|
while True:
|
||||||
line = read_tty_line()
|
line = read_tty_line()
|
||||||
@@ -99,7 +103,10 @@ def cmd_init(argv: list[str]) -> int:
|
|||||||
|
|
||||||
if bottle_name in (existing.get("bottles") or {}):
|
if bottle_name in (existing.get("bottles") or {}):
|
||||||
bottle_exists_already = True
|
bottle_exists_already = True
|
||||||
info(f"Bottle '{bottle_name}' already exists in {target_file}; agent will reference it.")
|
info(
|
||||||
|
f"Bottle '{bottle_name}' already exists in {target_file}; "
|
||||||
|
f"agent will reference it."
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
info(f"Creating new bottle '{bottle_name}'.")
|
info(f"Creating new bottle '{bottle_name}'.")
|
||||||
bottle_env = _prompt_for_env_vars()
|
bottle_env = _prompt_for_env_vars()
|
||||||
@@ -131,8 +138,14 @@ def cmd_init(argv: list[str]) -> int:
|
|||||||
|
|
||||||
def _prompt_for_env_vars() -> dict[str, str]:
|
def _prompt_for_env_vars() -> dict[str, str]:
|
||||||
print(file=sys.stderr)
|
print(file=sys.stderr)
|
||||||
info("Env vars — enter each var name then its mode. Press Enter with no name to finish.")
|
info(
|
||||||
info(" Modes: secret (prompt at runtime) | interpolated (read from host env) | literal (hardcoded value)")
|
"Env vars — enter each var name then its mode. Press Enter with "
|
||||||
|
"no name to finish."
|
||||||
|
)
|
||||||
|
info(
|
||||||
|
" Modes: secret (prompt at runtime) | interpolated (read from "
|
||||||
|
"host env) | literal (hardcoded value)"
|
||||||
|
)
|
||||||
out: dict[str, str] = {}
|
out: dict[str, str] = {}
|
||||||
while True:
|
while True:
|
||||||
print(file=sys.stderr)
|
print(file=sys.stderr)
|
||||||
|
|||||||
+39
-28
@@ -2,10 +2,8 @@
|
|||||||
interactive claude-code session. The container is torn down when the
|
interactive claude-code session. The container is torn down when the
|
||||||
session ends.
|
session ends.
|
||||||
|
|
||||||
The launch core is shared with `cli.py resume <identity>` and (PRD
|
The launch core is shared with `cli.py resume <identity>` through
|
||||||
0020 chunk 1+) the dashboard's in-process start flow: see the
|
the private orchestrator `_launch_bottle`.
|
||||||
public helpers `prepare_with_preflight`, `attach_agent`, and the
|
|
||||||
private orchestrator `_launch_bottle`.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -35,6 +33,7 @@ from ..backend.docker.capability_apply import snapshot_transcript
|
|||||||
from ..log import info
|
from ..log import info
|
||||||
from ..manifest import Manifest
|
from ..manifest import Manifest
|
||||||
from ._common import PROG, USER_CWD, read_tty_line
|
from ._common import PROG, USER_CWD, read_tty_line
|
||||||
|
from . import tui
|
||||||
|
|
||||||
|
|
||||||
def cmd_start(argv: list[str]) -> int:
|
def cmd_start(argv: list[str]) -> int:
|
||||||
@@ -51,15 +50,39 @@ def cmd_start(argv: list[str]) -> int:
|
|||||||
"or 'docker'). Overrides the env var when set."
|
"or 'docker'). Overrides the env var when set."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
parser.add_argument("name", help="agent name defined in bot-bottle.json")
|
parser.add_argument(
|
||||||
|
"name",
|
||||||
|
nargs="?",
|
||||||
|
default=None,
|
||||||
|
help="agent name defined in bot-bottle.json (omit to pick interactively)",
|
||||||
|
)
|
||||||
args = parser.parse_args(argv)
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
|
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
|
||||||
|
|
||||||
manifest = Manifest.resolve(USER_CWD)
|
manifest = Manifest.resolve(USER_CWD)
|
||||||
|
|
||||||
|
agent_name: str | None = args.name
|
||||||
|
if agent_name is None:
|
||||||
|
agent_name = tui.filter_select(
|
||||||
|
sorted(manifest.agents.keys()),
|
||||||
|
title="Select agent",
|
||||||
|
)
|
||||||
|
if agent_name is None:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
backend_name: str | None = args.backend
|
||||||
|
if backend_name is None and "BOT_BOTTLE_BACKEND" not in os.environ:
|
||||||
|
backend_name = tui.filter_select(
|
||||||
|
list(known_backend_names()),
|
||||||
|
title="Select backend",
|
||||||
|
)
|
||||||
|
if backend_name is None:
|
||||||
|
return 0
|
||||||
|
|
||||||
spec = BottleSpec(
|
spec = BottleSpec(
|
||||||
manifest=manifest,
|
manifest=manifest,
|
||||||
agent_name=args.name,
|
agent_name=agent_name,
|
||||||
copy_cwd=args.cwd,
|
copy_cwd=args.cwd,
|
||||||
user_cwd=USER_CWD,
|
user_cwd=USER_CWD,
|
||||||
)
|
)
|
||||||
@@ -67,11 +90,11 @@ def cmd_start(argv: list[str]) -> int:
|
|||||||
spec,
|
spec,
|
||||||
dry_run=dry_run,
|
dry_run=dry_run,
|
||||||
remote_control=args.remote_control,
|
remote_control=args.remote_control,
|
||||||
backend_name=args.backend,
|
backend_name=backend_name,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# --- Public helpers shared with the dashboard (PRD 0020) -----------------
|
# --- Launch helpers ------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def prepare_with_preflight(
|
def prepare_with_preflight(
|
||||||
@@ -84,14 +107,11 @@ def prepare_with_preflight(
|
|||||||
backend_name: str | None = None,
|
backend_name: str | None = None,
|
||||||
) -> tuple[DockerBottlePlan | None, str]:
|
) -> tuple[DockerBottlePlan | None, str]:
|
||||||
"""Run `backend.prepare`, render the preflight summary via the
|
"""Run `backend.prepare`, render the preflight summary via the
|
||||||
injected callable, prompt y/N via the injected callable. The CLI
|
injected callable, prompt y/N via the injected callable.
|
||||||
binds these to stderr/stdin; the dashboard binds them to a
|
|
||||||
curses modal.
|
|
||||||
|
|
||||||
`backend_name` selects which backend prepares the plan
|
`backend_name` selects which backend prepares the plan
|
||||||
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). Dashboard
|
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). The CLI passes
|
||||||
passes the value from its new-agent backend-picker modal; the
|
whatever `--backend` resolved to.
|
||||||
CLI passes whatever `--backend` resolved to.
|
|
||||||
|
|
||||||
Returns `(plan, identity)`. `plan` is None on dry-run or
|
Returns `(plan, identity)`. `plan` is None on dry-run or
|
||||||
operator-N, but `identity` is set as soon as `backend.prepare`
|
operator-N, but `identity` is set as soon as `backend.prepare`
|
||||||
@@ -122,16 +142,10 @@ def attach_agent(
|
|||||||
agent process's exit code.
|
agent process's exit code.
|
||||||
|
|
||||||
`resume=True` adds `--continue` so claude picks up its most
|
`resume=True` adds `--continue` so claude picks up its most
|
||||||
recent session non-interactively (no session-picker prompt) —
|
recent session non-interactively (no session-picker prompt).
|
||||||
the right shape for the dashboard's Enter re-attach (PRD 0020
|
First-attach paths (`./cli.py start`) leave it False.
|
||||||
chunk 3), where a bottle typically has exactly one session.
|
|
||||||
First-attach paths (`./cli.py start`, the dashboard's new-agent
|
|
||||||
flow) leave it False.
|
|
||||||
|
|
||||||
Used as the inner step of `./cli.py start` (one-shot) and by the
|
Used as the inner step of `./cli.py start`."""
|
||||||
dashboard, which calls it from inside a `curses.endwin → … →
|
|
||||||
stdscr.refresh()` handoff so the curses surface gets out of the
|
|
||||||
terminal's way while the agent has it."""
|
|
||||||
runtime = runtime_for(agent_provider_template)
|
runtime = runtime_for(agent_provider_template)
|
||||||
info(
|
info(
|
||||||
f"attaching interactive {agent_provider_template} session "
|
f"attaching interactive {agent_provider_template} session "
|
||||||
@@ -148,8 +162,7 @@ def attach_agent(
|
|||||||
def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
||||||
"""Inside the launch context, while the container is still
|
"""Inside the launch context, while the container is still
|
||||||
alive: snapshot the transcript and mark for preservation if
|
alive: snapshot the transcript and mark for preservation if
|
||||||
claude crashed. Public for the dashboard's death-handling path
|
claude crashed."""
|
||||||
(PRD 0020 open question 3)."""
|
|
||||||
# FIXME: this captures Claude-specific session state. A follow-up
|
# FIXME: this captures Claude-specific session state. A follow-up
|
||||||
# spike should explore freezing provider-neutral container state
|
# spike should explore freezing provider-neutral container state
|
||||||
# instead of relying on each agent's transcript layout.
|
# instead of relying on each agent's transcript layout.
|
||||||
@@ -162,9 +175,7 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
|||||||
|
|
||||||
def settle_state(identity: str) -> None:
|
def settle_state(identity: str) -> None:
|
||||||
"""Post-teardown housekeeping: print the resume hint if the
|
"""Post-teardown housekeeping: print the resume hint if the
|
||||||
state was preserved, otherwise reap the per-bottle state dir.
|
state was preserved, otherwise reap the per-bottle state dir."""
|
||||||
Public so the dashboard's explicit-stop path calls the same
|
|
||||||
settlement the CLI uses on context exit."""
|
|
||||||
if not identity:
|
if not identity:
|
||||||
return
|
return
|
||||||
if is_preserved(identity):
|
if is_preserved(identity):
|
||||||
|
|||||||
@@ -0,0 +1,515 @@
|
|||||||
|
"""supervise: list pending supervise proposals across all bottles and
|
||||||
|
act on them (approve / modify / reject).
|
||||||
|
|
||||||
|
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
|
||||||
|
approval handler wires to PRD 0016 (capability-block), which rebuilds
|
||||||
|
the bottle Dockerfile. The egress-block tool was removed in issue #198.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import curses
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import traceback
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .. import supervise as _supervise
|
||||||
|
from ..backend.docker.bottle_state import read_metadata
|
||||||
|
from ..backend.docker.capability_apply import (
|
||||||
|
CapabilityApplyError,
|
||||||
|
apply_capability_change,
|
||||||
|
)
|
||||||
|
from ..log import Die, error, info
|
||||||
|
from ..supervise import (
|
||||||
|
COMPONENT_FOR_TOOL,
|
||||||
|
AuditEntry,
|
||||||
|
Proposal,
|
||||||
|
Response,
|
||||||
|
STATUS_APPROVED,
|
||||||
|
STATUS_MODIFIED,
|
||||||
|
STATUS_REJECTED,
|
||||||
|
TOOL_CAPABILITY_BLOCK,
|
||||||
|
archive_proposal,
|
||||||
|
list_pending_proposals,
|
||||||
|
render_diff,
|
||||||
|
write_audit_entry,
|
||||||
|
write_response,
|
||||||
|
)
|
||||||
|
from ._common import PROG
|
||||||
|
|
||||||
|
|
||||||
|
_REFRESH_INTERVAL_MS = 1000
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class QueuedProposal:
|
||||||
|
"""A pending proposal plus the queue dir it was found in."""
|
||||||
|
|
||||||
|
proposal: Proposal
|
||||||
|
queue_dir: Path
|
||||||
|
|
||||||
|
|
||||||
|
# Errors any remediation engine may raise. Caught by the TUI key
|
||||||
|
# handlers and surfaced in the status line so a failed apply keeps
|
||||||
|
# the proposal pending rather than crashing curses.
|
||||||
|
ApplyError = (CapabilityApplyError,)
|
||||||
|
|
||||||
|
|
||||||
|
def discover_pending() -> list[QueuedProposal]:
|
||||||
|
"""Walk ~/.bot-bottle/queue/* and collect pending proposals."""
|
||||||
|
queue_root = _supervise.bot_bottle_root() / "queue"
|
||||||
|
if not queue_root.is_dir():
|
||||||
|
return []
|
||||||
|
out: list[QueuedProposal] = []
|
||||||
|
for slug_dir in sorted(queue_root.iterdir()):
|
||||||
|
if not slug_dir.is_dir():
|
||||||
|
continue
|
||||||
|
for proposal in list_pending_proposals(slug_dir):
|
||||||
|
out.append(QueuedProposal(proposal=proposal, queue_dir=slug_dir))
|
||||||
|
out.sort(key=lambda q: q.proposal.arrival_timestamp)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _approval_status(qp: QueuedProposal, verb: str) -> str:
|
||||||
|
"""Status-line text after a successful approval."""
|
||||||
|
base = f"{verb} {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
||||||
|
return f"{base}; resume: ./cli.py resume {qp.proposal.bottle_slug}"
|
||||||
|
|
||||||
|
|
||||||
|
def _detail_lines(
|
||||||
|
qp: QueuedProposal,
|
||||||
|
*,
|
||||||
|
green_attr: int = 0,
|
||||||
|
) -> list[tuple[str, int]]:
|
||||||
|
"""Return the detail-view body as (text, curses-attr) tuples."""
|
||||||
|
p = qp.proposal
|
||||||
|
out: list[tuple[str, int]] = [
|
||||||
|
(f"bottle: {p.bottle_slug}", 0),
|
||||||
|
(f"tool: {p.tool}", 0),
|
||||||
|
(f"id: {p.id}", 0),
|
||||||
|
(f"arrived: {p.arrival_timestamp}", 0),
|
||||||
|
(f"queue: {qp.queue_dir}", 0),
|
||||||
|
("", 0),
|
||||||
|
("justification:", 0),
|
||||||
|
]
|
||||||
|
out.extend((" " + line, 0) for line in p.justification.splitlines() or [""])
|
||||||
|
out.extend([
|
||||||
|
("", 0),
|
||||||
|
("proposed file:", 0),
|
||||||
|
])
|
||||||
|
out.extend((line, 0) for line in p.proposed_file.splitlines() or [""])
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _suffix_for_tool(tool: str) -> str:
|
||||||
|
if tool == TOOL_CAPABILITY_BLOCK:
|
||||||
|
return ".dockerfile"
|
||||||
|
return ".txt"
|
||||||
|
|
||||||
|
|
||||||
|
# --- Operator actions ------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def approve(
|
||||||
|
qp: QueuedProposal,
|
||||||
|
*,
|
||||||
|
notes: str = "",
|
||||||
|
final_file: str | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Apply the proposal, write the waiting response, and audit it."""
|
||||||
|
status = STATUS_MODIFIED if final_file is not None else STATUS_APPROVED
|
||||||
|
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
|
||||||
|
|
||||||
|
diff_before, diff_after = "", ""
|
||||||
|
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||||
|
_meta = read_metadata(qp.proposal.bottle_slug)
|
||||||
|
if _meta is not None and not _meta.compose_project:
|
||||||
|
raise CapabilityApplyError(
|
||||||
|
"capability-block remediation is not supported for smolmachines "
|
||||||
|
"bottles. Reject this proposal or handle the capability change "
|
||||||
|
"manually, then restart the bottle."
|
||||||
|
)
|
||||||
|
diff_before, diff_after = apply_capability_change(
|
||||||
|
qp.proposal.bottle_slug, file_to_apply,
|
||||||
|
)
|
||||||
|
|
||||||
|
response = Response(
|
||||||
|
proposal_id=qp.proposal.id,
|
||||||
|
status=status,
|
||||||
|
notes=notes,
|
||||||
|
final_file=final_file,
|
||||||
|
)
|
||||||
|
write_response(qp.queue_dir, response)
|
||||||
|
_write_audit(
|
||||||
|
qp, action=status, notes=notes,
|
||||||
|
diff_before=diff_before, diff_after=diff_after,
|
||||||
|
)
|
||||||
|
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||||
|
archive_proposal(qp.queue_dir, qp.proposal.id)
|
||||||
|
|
||||||
|
|
||||||
|
def reject(qp: QueuedProposal, *, reason: str) -> None:
|
||||||
|
"""Write a rejection response and an audit entry."""
|
||||||
|
response = Response(
|
||||||
|
proposal_id=qp.proposal.id,
|
||||||
|
status=STATUS_REJECTED,
|
||||||
|
notes=reason,
|
||||||
|
final_file=None,
|
||||||
|
)
|
||||||
|
write_response(qp.queue_dir, response)
|
||||||
|
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
|
||||||
|
|
||||||
|
|
||||||
|
def _write_audit(
|
||||||
|
qp: QueuedProposal,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
notes: str,
|
||||||
|
diff_before: str,
|
||||||
|
diff_after: str,
|
||||||
|
) -> None:
|
||||||
|
"""Audit log for egress tool."""
|
||||||
|
component = COMPONENT_FOR_TOOL.get(qp.proposal.tool)
|
||||||
|
if component is None:
|
||||||
|
return
|
||||||
|
write_audit_entry(AuditEntry(
|
||||||
|
timestamp=datetime.now(timezone.utc).isoformat(),
|
||||||
|
bottle_slug=qp.proposal.bottle_slug,
|
||||||
|
component=component,
|
||||||
|
operator_action=action,
|
||||||
|
operator_notes=notes,
|
||||||
|
justification=qp.proposal.justification,
|
||||||
|
diff=render_diff(diff_before, diff_after, label=component),
|
||||||
|
))
|
||||||
|
|
||||||
|
|
||||||
|
# --- $EDITOR integration --------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def edit_in_editor(content: str, *, suffix: str = ".tmp") -> str | None:
|
||||||
|
"""Open `content` in $EDITOR and return edited content, if changed."""
|
||||||
|
editor = os.environ.get("EDITOR", "vim")
|
||||||
|
with tempfile.NamedTemporaryFile(
|
||||||
|
mode="w", suffix=suffix, delete=False, prefix="supervise-modify.",
|
||||||
|
) as f:
|
||||||
|
f.write(content)
|
||||||
|
path = f.name
|
||||||
|
try:
|
||||||
|
subprocess.run([editor, path], check=False)
|
||||||
|
with open(path, encoding="utf-8") as f:
|
||||||
|
edited = f.read()
|
||||||
|
return edited if edited != content else None
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
os.unlink(path)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# --- TUI -------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_supervise(argv: list[str]) -> int:
|
||||||
|
parser = argparse.ArgumentParser(prog=f"{PROG} supervise", add_help=True)
|
||||||
|
parser.add_argument(
|
||||||
|
"--once", action="store_true",
|
||||||
|
help="list pending proposals once and exit (no TUI)",
|
||||||
|
)
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
if args.once:
|
||||||
|
return _list_once()
|
||||||
|
try:
|
||||||
|
curses.wrapper(_main_loop)
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
return 130
|
||||||
|
except Die as e:
|
||||||
|
if e.message:
|
||||||
|
error(e.message)
|
||||||
|
else:
|
||||||
|
error("supervise exited on a fatal error (no detail captured).")
|
||||||
|
return e.code if isinstance(e.code, int) else 1
|
||||||
|
except Exception as e: # noqa: W0718 — catch supervise crash for logging
|
||||||
|
log_path = _write_crash_log(e)
|
||||||
|
error(f"supervise crashed: {type(e).__name__}: {e}")
|
||||||
|
error(f"full traceback written to {log_path}")
|
||||||
|
return 1
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _write_crash_log(exc: BaseException) -> Path:
|
||||||
|
"""Persist `exc`'s traceback to a stable file under ~/.bot-bottle/."""
|
||||||
|
stamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
body = "".join(
|
||||||
|
traceback.format_exception(type(exc), exc, exc.__traceback__)
|
||||||
|
)
|
||||||
|
entry = f"=== supervise crash {stamp} ===\n{body}\n"
|
||||||
|
try:
|
||||||
|
log_dir = _supervise.bot_bottle_root() / "logs"
|
||||||
|
log_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
path = log_dir / "supervise-crash.log"
|
||||||
|
with path.open("a", encoding="utf-8") as fh:
|
||||||
|
fh.write(entry)
|
||||||
|
return path
|
||||||
|
except OSError:
|
||||||
|
fd, tmp = tempfile.mkstemp(
|
||||||
|
prefix="bot-bottle-supervise-crash-", suffix=".log",
|
||||||
|
)
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(entry)
|
||||||
|
return Path(tmp)
|
||||||
|
|
||||||
|
|
||||||
|
def _list_once() -> int:
|
||||||
|
pending = discover_pending()
|
||||||
|
if not pending:
|
||||||
|
info("no pending proposals")
|
||||||
|
return 0
|
||||||
|
for qp in pending:
|
||||||
|
sys.stdout.write(
|
||||||
|
f"{qp.proposal.arrival_timestamp} "
|
||||||
|
f"[{qp.proposal.bottle_slug}] "
|
||||||
|
f"{qp.proposal.tool} "
|
||||||
|
f"{qp.proposal.id}\n"
|
||||||
|
)
|
||||||
|
sys.stdout.write(f" {qp.proposal.justification}\n")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _try_init_green() -> int:
|
||||||
|
"""Initialise a green color pair and return its attr, or 0."""
|
||||||
|
try:
|
||||||
|
curses.start_color()
|
||||||
|
curses.use_default_colors()
|
||||||
|
curses.init_pair(1, curses.COLOR_GREEN, -1)
|
||||||
|
return curses.color_pair(1)
|
||||||
|
except curses.error:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
|
||||||
|
curses.curs_set(0)
|
||||||
|
stdscr.timeout(_REFRESH_INTERVAL_MS)
|
||||||
|
green_attr = _try_init_green()
|
||||||
|
selected = 0
|
||||||
|
status_line = ""
|
||||||
|
seen_ids: set[str] = set()
|
||||||
|
|
||||||
|
while True:
|
||||||
|
pending = discover_pending()
|
||||||
|
if selected >= len(pending):
|
||||||
|
selected = max(0, len(pending) - 1)
|
||||||
|
|
||||||
|
live_ids = {qp.proposal.id for qp in pending}
|
||||||
|
newly_arrived = live_ids - seen_ids
|
||||||
|
if seen_ids and newly_arrived:
|
||||||
|
try:
|
||||||
|
curses.beep()
|
||||||
|
except curses.error:
|
||||||
|
pass
|
||||||
|
for i, qp in enumerate(pending):
|
||||||
|
if qp.proposal.id in newly_arrived:
|
||||||
|
selected = i
|
||||||
|
break
|
||||||
|
seen_ids = live_ids
|
||||||
|
|
||||||
|
_render(
|
||||||
|
stdscr, pending, selected, status_line,
|
||||||
|
green_attr=green_attr,
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
key = stdscr.getch()
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
return
|
||||||
|
|
||||||
|
if key == -1:
|
||||||
|
continue
|
||||||
|
|
||||||
|
status_line = ""
|
||||||
|
|
||||||
|
if key in (ord("q"), 27):
|
||||||
|
return
|
||||||
|
|
||||||
|
if not pending:
|
||||||
|
continue
|
||||||
|
qp = pending[selected]
|
||||||
|
|
||||||
|
if key in (curses.KEY_DOWN, ord("j")):
|
||||||
|
selected = min(selected + 1, len(pending) - 1)
|
||||||
|
elif key in (curses.KEY_UP, ord("k")):
|
||||||
|
selected = max(selected - 1, 0)
|
||||||
|
elif key in (curses.KEY_ENTER, 10, 13):
|
||||||
|
_detail_view(stdscr, qp, green_attr=green_attr)
|
||||||
|
elif key == ord("a"):
|
||||||
|
try:
|
||||||
|
approve(qp)
|
||||||
|
status_line = _approval_status(qp, "approved")
|
||||||
|
except ApplyError as e:
|
||||||
|
status_line = f"apply failed: {e}"
|
||||||
|
elif key == ord("m"):
|
||||||
|
edited = _modify(stdscr, qp)
|
||||||
|
if edited is None:
|
||||||
|
status_line = "modify aborted (no change)"
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||||
|
status_line = _approval_status(qp, "modified+approved")
|
||||||
|
except ApplyError as e:
|
||||||
|
status_line = f"apply failed: {e}"
|
||||||
|
elif key == ord("r"):
|
||||||
|
reason = _prompt(stdscr, "reject reason: ")
|
||||||
|
if reason:
|
||||||
|
reject(qp, reason=reason)
|
||||||
|
status_line = f"rejected {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
||||||
|
else:
|
||||||
|
status_line = "reject aborted (empty reason)"
|
||||||
|
|
||||||
|
|
||||||
|
def _render(
|
||||||
|
stdscr: "curses._CursesWindow", # type: ignore
|
||||||
|
pending: list[QueuedProposal],
|
||||||
|
selected: int,
|
||||||
|
status_line: str,
|
||||||
|
*,
|
||||||
|
green_attr: int = 0, # noqa: F841 — unused, but required by interface
|
||||||
|
) -> None:
|
||||||
|
stdscr.erase()
|
||||||
|
h, w = stdscr.getmaxyx()
|
||||||
|
header = f"bot-bottle supervise ({len(pending)} pending)"
|
||||||
|
stdscr.addnstr(0, 0, header, w - 1, curses.A_BOLD)
|
||||||
|
stdscr.hline(1, 0, curses.ACS_HLINE, w)
|
||||||
|
|
||||||
|
row = 2
|
||||||
|
if not pending:
|
||||||
|
stdscr.addnstr(
|
||||||
|
row, 2,
|
||||||
|
"no pending proposals; agents will queue here when they call a "
|
||||||
|
"supervise tool",
|
||||||
|
w - 4,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
for i, qp in enumerate(pending):
|
||||||
|
if row >= h - 3:
|
||||||
|
break
|
||||||
|
p = qp.proposal
|
||||||
|
ts_short = (
|
||||||
|
p.arrival_timestamp.split("T", 1)[1][:8]
|
||||||
|
if "T" in p.arrival_timestamp else p.arrival_timestamp
|
||||||
|
)
|
||||||
|
cursor = "> " if i == selected else " "
|
||||||
|
line = (
|
||||||
|
f"{cursor}{ts_short} "
|
||||||
|
f"[{p.bottle_slug}] {p.tool:<18} {p.id[:8]}"
|
||||||
|
)
|
||||||
|
attr = curses.A_REVERSE if i == selected else curses.A_NORMAL
|
||||||
|
stdscr.addnstr(row, 0, line, w - 1, attr)
|
||||||
|
row += 1
|
||||||
|
if row >= h - 3:
|
||||||
|
break
|
||||||
|
if p.justification:
|
||||||
|
stdscr.addnstr(row, 4, p.justification[: max(0, w - 5)], w - 5)
|
||||||
|
row += 1
|
||||||
|
|
||||||
|
footer = "[j/k] move [Enter] view [a] approve [m] modify [r] reject [q] quit"
|
||||||
|
stdscr.hline(h - 2, 0, curses.ACS_HLINE, w)
|
||||||
|
stdscr.addnstr(h - 1, 0, footer, w - 1, curses.A_DIM)
|
||||||
|
if status_line:
|
||||||
|
stdscr.addnstr(h - 3, 0, status_line, w - 1, curses.A_BOLD)
|
||||||
|
stdscr.refresh()
|
||||||
|
|
||||||
|
|
||||||
|
def _detail_view(
|
||||||
|
stdscr: "curses._CursesWindow", # type: ignore
|
||||||
|
qp: QueuedProposal,
|
||||||
|
*,
|
||||||
|
green_attr: int = 0,
|
||||||
|
) -> None:
|
||||||
|
"""Render the full proposal. Scrollable. Press q to return."""
|
||||||
|
lines = _detail_lines(qp, green_attr=green_attr)
|
||||||
|
offset = 0
|
||||||
|
while True:
|
||||||
|
stdscr.erase()
|
||||||
|
h, w = stdscr.getmaxyx()
|
||||||
|
for i, (text, attr) in enumerate(lines[offset:offset + h - 1]):
|
||||||
|
stdscr.addnstr(i, 0, text, w - 1, attr)
|
||||||
|
stdscr.addnstr(
|
||||||
|
h - 1, 0,
|
||||||
|
"[j/k] scroll [g/G] top/bottom [a] approve [m] modify [r] reject [q] back",
|
||||||
|
w - 1, curses.A_DIM,
|
||||||
|
)
|
||||||
|
stdscr.refresh()
|
||||||
|
key = stdscr.getch()
|
||||||
|
if key in (ord("q"), 27):
|
||||||
|
return
|
||||||
|
if key in (curses.KEY_DOWN, ord("j")):
|
||||||
|
offset = min(offset + 1, max(0, len(lines) - 1))
|
||||||
|
elif key in (curses.KEY_UP, ord("k")):
|
||||||
|
offset = max(offset - 1, 0)
|
||||||
|
elif key == ord("g"):
|
||||||
|
offset = 0
|
||||||
|
elif key == ord("G"):
|
||||||
|
offset = max(0, len(lines) - 1)
|
||||||
|
elif key == ord("a"):
|
||||||
|
try:
|
||||||
|
approve(qp)
|
||||||
|
except ApplyError:
|
||||||
|
pass
|
||||||
|
return
|
||||||
|
elif key == ord("m"):
|
||||||
|
edited = _modify(stdscr, qp)
|
||||||
|
if edited is not None:
|
||||||
|
try:
|
||||||
|
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||||
|
except ApplyError:
|
||||||
|
pass
|
||||||
|
return
|
||||||
|
elif key == ord("r"):
|
||||||
|
reason = _prompt(stdscr, "reject reason: ")
|
||||||
|
if reason:
|
||||||
|
reject(qp, reason=reason)
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore
|
||||||
|
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
|
||||||
|
suffix = _suffix_for_tool(qp.proposal.tool)
|
||||||
|
curses.endwin()
|
||||||
|
try:
|
||||||
|
edited = edit_in_editor(qp.proposal.proposed_file, suffix=suffix)
|
||||||
|
finally:
|
||||||
|
stdscr.refresh()
|
||||||
|
return edited
|
||||||
|
|
||||||
|
|
||||||
|
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore
|
||||||
|
"""One-line input at the bottom of the screen."""
|
||||||
|
curses.curs_set(1)
|
||||||
|
h, _ = stdscr.getmaxyx()
|
||||||
|
stdscr.move(h - 2, 0)
|
||||||
|
stdscr.clrtoeol()
|
||||||
|
stdscr.addstr(h - 2, 0, label)
|
||||||
|
stdscr.refresh()
|
||||||
|
curses.echo()
|
||||||
|
try:
|
||||||
|
raw = stdscr.getstr(h - 2, len(label), 200)
|
||||||
|
finally:
|
||||||
|
curses.noecho()
|
||||||
|
curses.curs_set(0)
|
||||||
|
return raw.decode("utf-8", errors="replace").strip()
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"QueuedProposal",
|
||||||
|
"approve",
|
||||||
|
"cmd_supervise",
|
||||||
|
"discover_pending",
|
||||||
|
"edit_in_editor",
|
||||||
|
"reject",
|
||||||
|
]
|
||||||
@@ -0,0 +1,220 @@
|
|||||||
|
"""tui.py — minimal curses filter-select picker for CLI prompts.
|
||||||
|
|
||||||
|
Exposed surface:
|
||||||
|
|
||||||
|
filter_select(items, *, title="", tty_path="/dev/tty") -> str | None
|
||||||
|
|
||||||
|
Opens /dev/tty directly so the picker works even when stdout/stdin are
|
||||||
|
redirected. Returns the selected item or None on cancel.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import curses
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
|
||||||
|
def filter_select(
|
||||||
|
items: list[str],
|
||||||
|
*,
|
||||||
|
title: str = "",
|
||||||
|
tty_path: str = "/dev/tty",
|
||||||
|
) -> Optional[str]:
|
||||||
|
"""Render a filter-select picker over *items*.
|
||||||
|
|
||||||
|
Returns the selected item string, or ``None`` if the user cancelled
|
||||||
|
(Esc / ``q`` / Ctrl-C / Ctrl-D) or if the terminal is too small.
|
||||||
|
|
||||||
|
The picker opens *tty_path* directly so it works even when
|
||||||
|
stdout/stdin are redirected.
|
||||||
|
"""
|
||||||
|
if not items:
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
tty_fd = open(tty_path, "r+b", buffering=0)
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Use os.dup() to duplicate the fd so the original file object
|
||||||
|
# and FileIO in _run_picker each manage independent copies,
|
||||||
|
# preventing double-close errors.
|
||||||
|
fd_dup = os.dup(tty_fd.fileno())
|
||||||
|
return _run_picker(items, title=title, tty_fd=fd_dup)
|
||||||
|
finally:
|
||||||
|
tty_fd.close()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Internal implementation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_KEY_ESC = 27
|
||||||
|
_KEY_CTRL_C = 3
|
||||||
|
_KEY_CTRL_D = 4
|
||||||
|
_KEY_BACKSPACE_WIN = 8
|
||||||
|
_KEY_ENTER_ALT = 10
|
||||||
|
|
||||||
|
_CANCEL_KEYS = frozenset([_KEY_ESC, _KEY_CTRL_C, _KEY_CTRL_D, ord("q")])
|
||||||
|
|
||||||
|
|
||||||
|
def _run_picker(items: list[str], *, title: str, tty_fd: int) -> Optional[str]:
|
||||||
|
"""Drive a curses session on *tty_fd* and return the picked item."""
|
||||||
|
# newterm lets us run curses on an arbitrary fd rather than the
|
||||||
|
# process's controlling tty / stdout — crucial when stdout is piped.
|
||||||
|
os.environ.setdefault("TERM", "xterm-256color")
|
||||||
|
|
||||||
|
# Save / restore the real stdin/stdout so curses newterm can use tty_fd.
|
||||||
|
orig_stdin = sys.__stdin__
|
||||||
|
orig_stdout = sys.__stdout__
|
||||||
|
|
||||||
|
try:
|
||||||
|
import io
|
||||||
|
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode='r+'), write_through=True)
|
||||||
|
sys.__stdin__ = tty_text # type: ignore[assignment]
|
||||||
|
sys.__stdout__ = tty_text # type: ignore[assignment]
|
||||||
|
|
||||||
|
# curses.wrapper calls initscr which honours sys.__stdin__ / __stdout__
|
||||||
|
# on some builds; use newterm where available.
|
||||||
|
screen = curses.initscr()
|
||||||
|
curses.noecho()
|
||||||
|
curses.cbreak()
|
||||||
|
screen.keypad(True)
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = _picker_loop(screen, items, title=title)
|
||||||
|
finally:
|
||||||
|
screen.keypad(False)
|
||||||
|
curses.nocbreak()
|
||||||
|
curses.echo()
|
||||||
|
curses.endwin()
|
||||||
|
except Exception: # noqa: W0718 — curses can raise many error types
|
||||||
|
return None
|
||||||
|
finally:
|
||||||
|
sys.__stdin__ = orig_stdin # type: ignore[assignment]
|
||||||
|
sys.__stdout__ = orig_stdout # type: ignore[assignment]
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _picker_loop(screen: Any, items: list[str], *, title: str) -> Optional[str]:
|
||||||
|
query = ""
|
||||||
|
cursor = 0
|
||||||
|
|
||||||
|
while True:
|
||||||
|
filtered = _filter_items(items, query)
|
||||||
|
|
||||||
|
# Clamp cursor into the visible list.
|
||||||
|
if not filtered:
|
||||||
|
cursor = 0
|
||||||
|
elif cursor >= len(filtered):
|
||||||
|
cursor = len(filtered) - 1
|
||||||
|
|
||||||
|
try:
|
||||||
|
_render(screen, filtered, cursor, query=query, title=title)
|
||||||
|
except curses.error:
|
||||||
|
# Terminal too small or write error — bail out.
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
key = screen.getch()
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if key in _CANCEL_KEYS:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
|
||||||
|
return filtered[cursor] if filtered else None
|
||||||
|
|
||||||
|
if key in (curses.KEY_UP, ord("k")):
|
||||||
|
if cursor > 0:
|
||||||
|
cursor -= 1
|
||||||
|
|
||||||
|
elif key in (curses.KEY_DOWN, ord("j")):
|
||||||
|
if cursor < len(filtered) - 1:
|
||||||
|
cursor += 1
|
||||||
|
|
||||||
|
elif key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
|
||||||
|
query = query[:-1]
|
||||||
|
# After narrowing the filter, keep cursor in range.
|
||||||
|
new_filtered = _filter_items(items, query)
|
||||||
|
if cursor >= len(new_filtered):
|
||||||
|
cursor = max(0, len(new_filtered) - 1)
|
||||||
|
|
||||||
|
elif 32 <= key <= 126:
|
||||||
|
# Printable ASCII — append to query and reset cursor so the
|
||||||
|
# top of the newly-filtered list is selected.
|
||||||
|
query += chr(key)
|
||||||
|
cursor = 0
|
||||||
|
|
||||||
|
|
||||||
|
def _filter_items(items: list[str], query: str) -> list[str]:
|
||||||
|
if not query:
|
||||||
|
return list(items)
|
||||||
|
q = query.lower()
|
||||||
|
return [i for i in items if q in i.lower()]
|
||||||
|
|
||||||
|
|
||||||
|
def _render(screen: Any, filtered: list[str], cursor: int, *, query: str, title: str) -> None:
|
||||||
|
screen.erase()
|
||||||
|
rows, cols = screen.getmaxyx()
|
||||||
|
min_rows = 5
|
||||||
|
|
||||||
|
if rows < min_rows:
|
||||||
|
raise curses.error("terminal too small")
|
||||||
|
|
||||||
|
row = 0
|
||||||
|
|
||||||
|
if title and row < rows - 1:
|
||||||
|
_addstr_safe(screen, row, 0, title[:cols - 1], curses.A_BOLD)
|
||||||
|
row += 1
|
||||||
|
|
||||||
|
filter_label = f"Filter: {query}"
|
||||||
|
if row < rows - 1:
|
||||||
|
_addstr_safe(screen, row, 0, filter_label[:cols - 1])
|
||||||
|
row += 1
|
||||||
|
|
||||||
|
sep = "─" * min(cols - 1, 40)
|
||||||
|
if row < rows - 1:
|
||||||
|
_addstr_safe(screen, row, 0, sep)
|
||||||
|
row += 1
|
||||||
|
|
||||||
|
list_start = row
|
||||||
|
# Reserve two rows for separator + help line at bottom.
|
||||||
|
list_rows = rows - list_start - 2
|
||||||
|
if list_rows < 1:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Scroll window: keep cursor visible.
|
||||||
|
scroll = max(0, cursor - list_rows + 1)
|
||||||
|
visible = filtered[scroll: scroll + list_rows]
|
||||||
|
|
||||||
|
for idx, item in enumerate(visible):
|
||||||
|
abs_idx = scroll + idx
|
||||||
|
attr = curses.A_REVERSE if abs_idx == cursor else curses.A_NORMAL
|
||||||
|
prefix = "> " if abs_idx == cursor else " "
|
||||||
|
line = (prefix + item)[:cols - 1]
|
||||||
|
if row < rows - 1:
|
||||||
|
_addstr_safe(screen, row, 0, line, attr)
|
||||||
|
row += 1
|
||||||
|
|
||||||
|
if row < rows - 1:
|
||||||
|
_addstr_safe(screen, row, 0, sep)
|
||||||
|
row += 1
|
||||||
|
|
||||||
|
help_line = "[↑↓/jk] move [Enter] select [Esc/q] cancel"
|
||||||
|
if row < rows:
|
||||||
|
_addstr_safe(screen, min(rows - 1, row), 0, help_line[:cols - 1])
|
||||||
|
|
||||||
|
screen.refresh()
|
||||||
|
|
||||||
|
|
||||||
|
def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.A_NORMAL) -> None:
|
||||||
|
try:
|
||||||
|
screen.addstr(row, col, text, attr)
|
||||||
|
except curses.error:
|
||||||
|
pass
|
||||||
@@ -0,0 +1,225 @@
|
|||||||
|
"""Claude agent provider plugin (PRD 0050, contrib).
|
||||||
|
|
||||||
|
The Claude-specific behavior previously inlined under
|
||||||
|
`agent_provider.agent_provision_plan` (claude.json trust marker,
|
||||||
|
api.anthropic.com egress route, OAuth-token placeholder), plus
|
||||||
|
the `claude mcp add` invocation that registers the supervise
|
||||||
|
sidecar in claude-code's user config (PRD 0013)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import shlex
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from ...agent_provider import (
|
||||||
|
AgentProvider,
|
||||||
|
AgentProviderRuntime,
|
||||||
|
AgentProvisionFile,
|
||||||
|
AgentProvisionPlan,
|
||||||
|
)
|
||||||
|
from ...egress import EgressRoute
|
||||||
|
from ...log import die, info, warn
|
||||||
|
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from ...backend import Bottle, BottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||||
|
|
||||||
|
_SUPERVISE_MCP_NAME = "supervise"
|
||||||
|
|
||||||
|
|
||||||
|
def _skills_dir(guest_home: str) -> str:
|
||||||
|
return f"{guest_home}/.claude/skills"
|
||||||
|
|
||||||
|
|
||||||
|
def _prompt_path(guest_home: str) -> str:
|
||||||
|
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||||
|
|
||||||
|
_RUNTIME = AgentProviderRuntime(
|
||||||
|
template="claude",
|
||||||
|
command="claude",
|
||||||
|
image="bot-bottle-claude:latest",
|
||||||
|
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||||
|
prompt_mode="append_file",
|
||||||
|
bypass_args=("--dangerously-skip-permissions",),
|
||||||
|
resume_args=("--continue",),
|
||||||
|
remote_control_args=("--remote-control",),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ClaudeAgentProvider(AgentProvider):
|
||||||
|
@property
|
||||||
|
def runtime(self) -> AgentProviderRuntime:
|
||||||
|
return _RUNTIME
|
||||||
|
|
||||||
|
def provision_plan(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
dockerfile: str,
|
||||||
|
state_dir: Path,
|
||||||
|
guest_home: str,
|
||||||
|
guest_env: dict[str, str] | None = None,
|
||||||
|
auth_token: str = "",
|
||||||
|
forward_host_credentials: bool = False,
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
trusted_project_path: str = "",
|
||||||
|
) -> AgentProvisionPlan:
|
||||||
|
del forward_host_credentials, host_env # Codex-only knobs
|
||||||
|
resolved_guest_env = dict(guest_env or {})
|
||||||
|
trusted_path = trusted_project_path or guest_home
|
||||||
|
|
||||||
|
env_vars: dict[str, str] = {
|
||||||
|
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
|
||||||
|
"DISABLE_ERROR_REPORTING": "1",
|
||||||
|
}
|
||||||
|
claude_config = state_dir / "claude.json"
|
||||||
|
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
|
||||||
|
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
||||||
|
claude_config.write_text(json.dumps({
|
||||||
|
"hasCompletedOnboarding": True,
|
||||||
|
"theme": "dark",
|
||||||
|
"bypassPermissionsModeAccepted": True,
|
||||||
|
"projects": claude_projects,
|
||||||
|
}, indent=2) + "\n")
|
||||||
|
claude_config.chmod(0o600)
|
||||||
|
files = (
|
||||||
|
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
|
||||||
|
)
|
||||||
|
egress_routes = (EgressRoute(
|
||||||
|
host="api.anthropic.com",
|
||||||
|
auth_scheme="Bearer" if auth_token else "",
|
||||||
|
token_ref=auth_token,
|
||||||
|
),)
|
||||||
|
hidden_env_names: frozenset[str] = frozenset()
|
||||||
|
if auth_token:
|
||||||
|
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
||||||
|
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
||||||
|
|
||||||
|
return AgentProvisionPlan(
|
||||||
|
template=_RUNTIME.template,
|
||||||
|
command=_RUNTIME.command,
|
||||||
|
prompt_mode=_RUNTIME.prompt_mode,
|
||||||
|
image=_RUNTIME.image,
|
||||||
|
dockerfile=dockerfile,
|
||||||
|
env_vars=env_vars,
|
||||||
|
guest_env=resolved_guest_env,
|
||||||
|
files=files,
|
||||||
|
egress_routes=egress_routes,
|
||||||
|
hidden_env_names=hidden_env_names,
|
||||||
|
)
|
||||||
|
|
||||||
|
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||||
|
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
||||||
|
on the host into the guest's claude-code skills dir. No-op
|
||||||
|
when the agent has no skills."""
|
||||||
|
from ...backend.util import host_skill_dir
|
||||||
|
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
if not agent.skills:
|
||||||
|
return
|
||||||
|
skills_dir = _skills_dir(plan.guest_home)
|
||||||
|
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||||
|
for name in agent.skills:
|
||||||
|
src = host_skill_dir(name)
|
||||||
|
if not os.path.isdir(src):
|
||||||
|
die(
|
||||||
|
f"skill {name!r} disappeared from host between "
|
||||||
|
f"validation and copy at {src}."
|
||||||
|
)
|
||||||
|
dst = f"{skills_dir}/{name}"
|
||||||
|
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||||
|
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||||
|
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||||
|
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||||
|
|
||||||
|
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||||
|
"""Copy the prompt file into the guest, fix ownership/mode.
|
||||||
|
Returns the in-guest path iff the agent has a non-empty
|
||||||
|
prompt (drives `--append-system-prompt-file`); the file is
|
||||||
|
copied either way so the path always exists."""
|
||||||
|
prompt_path = _prompt_path(plan.guest_home)
|
||||||
|
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
|
||||||
|
bottle.exec(
|
||||||
|
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||||
|
user="root",
|
||||||
|
)
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
return prompt_path if agent.prompt else None
|
||||||
|
|
||||||
|
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||||
|
"""Apply the claude-side declarative provision steps from
|
||||||
|
`plan.agent_provision` — today that's the `claude.json`
|
||||||
|
trust-marker file. Hot-replace this with a richer flow as
|
||||||
|
claude-code's harness shape evolves."""
|
||||||
|
provision = plan.agent_provision
|
||||||
|
for d in provision.dirs:
|
||||||
|
path = shlex.quote(d.guest_path)
|
||||||
|
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chown {shlex.quote(d.owner)} {path}",
|
||||||
|
f"could not chown {d.guest_path}",
|
||||||
|
)
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chmod {shlex.quote(d.mode)} {path}",
|
||||||
|
f"could not chmod {d.guest_path}",
|
||||||
|
)
|
||||||
|
for command in provision.pre_copy:
|
||||||
|
_exec(bottle, shlex.join(command.argv), command.error)
|
||||||
|
for f in provision.files:
|
||||||
|
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||||
|
path = shlex.quote(f.guest_path)
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chown {shlex.quote(f.owner)} {path}",
|
||||||
|
f"could not chown {f.guest_path}",
|
||||||
|
)
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chmod {shlex.quote(f.mode)} {path}",
|
||||||
|
f"could not chmod {f.guest_path}",
|
||||||
|
)
|
||||||
|
for command in provision.verify:
|
||||||
|
_exec(bottle, shlex.join(command.argv), command.error)
|
||||||
|
|
||||||
|
def provision_supervise_mcp(
|
||||||
|
self,
|
||||||
|
plan: "BottlePlan",
|
||||||
|
bottle: "Bottle",
|
||||||
|
supervise_url: str,
|
||||||
|
) -> None:
|
||||||
|
"""Run `claude mcp add` inside the agent guest to register the
|
||||||
|
supervise sidecar in claude-code's user config (~/.claude.json).
|
||||||
|
|
||||||
|
Failure is logged but not fatal — the bottle still works without
|
||||||
|
the entry; the operator can register it manually."""
|
||||||
|
if plan.supervise_plan is None:
|
||||||
|
return
|
||||||
|
info(f"registering supervise MCP server in agent claude config → {supervise_url}")
|
||||||
|
r = bottle.exec(
|
||||||
|
f"claude mcp add --scope user --transport http "
|
||||||
|
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||||
|
user="node",
|
||||||
|
)
|
||||||
|
if r.returncode != 0:
|
||||||
|
warn(
|
||||||
|
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||||
|
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||||
|
f"register manually with: "
|
||||||
|
f"claude mcp add --scope user --transport http supervise {supervise_url}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||||
|
result = bottle.exec(script, user="root")
|
||||||
|
if result.returncode != 0:
|
||||||
|
detail = (result.stderr or result.stdout).strip()
|
||||||
|
if detail:
|
||||||
|
detail = f": {detail}"
|
||||||
|
die(f"agent provider provisioning: {error}{detail}")
|
||||||
@@ -0,0 +1,270 @@
|
|||||||
|
"""Codex agent provider plugin (PRD 0050, contrib).
|
||||||
|
|
||||||
|
The Codex-specific behavior previously inlined under
|
||||||
|
`agent_provider.agent_provision_plan` (config.toml trust marker,
|
||||||
|
chatgpt.com / api.openai.com egress routes, optional host-credential
|
||||||
|
forwarding with dummy-auth.json + verify), plus the `codex mcp add`
|
||||||
|
invocation that registers the supervise sidecar in Codex's
|
||||||
|
~/.codex/config.toml (PRD 0050)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shlex
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from ...agent_provider import (
|
||||||
|
CODEX_HOST_CREDENTIAL_HOSTS,
|
||||||
|
AgentProvider,
|
||||||
|
AgentProviderRuntime,
|
||||||
|
AgentProvisionCommand,
|
||||||
|
AgentProvisionDir,
|
||||||
|
AgentProvisionFile,
|
||||||
|
AgentProvisionPlan,
|
||||||
|
)
|
||||||
|
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
|
||||||
|
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
||||||
|
from ...log import die, info, warn
|
||||||
|
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from ...backend import Bottle, BottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||||
|
|
||||||
|
_SUPERVISE_MCP_NAME = "supervise"
|
||||||
|
|
||||||
|
|
||||||
|
def _skills_dir(guest_home: str) -> str:
|
||||||
|
# Codex agents still read skills from the claude-code convention
|
||||||
|
# (~/.claude/skills/) — the bot-bottle-codex image follows the
|
||||||
|
# same layout. If Codex grows native skill discovery later,
|
||||||
|
# change here.
|
||||||
|
return f"{guest_home}/.claude/skills"
|
||||||
|
|
||||||
|
|
||||||
|
def _prompt_path(guest_home: str) -> str:
|
||||||
|
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||||
|
|
||||||
|
_RUNTIME = AgentProviderRuntime(
|
||||||
|
template="codex",
|
||||||
|
command="codex",
|
||||||
|
image="bot-bottle-codex:latest",
|
||||||
|
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||||
|
prompt_mode="read_prompt_file",
|
||||||
|
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||||
|
resume_args=("resume", "--last"),
|
||||||
|
remote_control_args=(),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class CodexAgentProvider(AgentProvider):
|
||||||
|
@property
|
||||||
|
def runtime(self) -> AgentProviderRuntime:
|
||||||
|
return _RUNTIME
|
||||||
|
|
||||||
|
def provision_plan(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
dockerfile: str,
|
||||||
|
state_dir: Path,
|
||||||
|
guest_home: str,
|
||||||
|
guest_env: dict[str, str] | None = None,
|
||||||
|
auth_token: str = "",
|
||||||
|
forward_host_credentials: bool = False,
|
||||||
|
host_env: dict[str, str] | None = None,
|
||||||
|
trusted_project_path: str = "",
|
||||||
|
) -> AgentProvisionPlan:
|
||||||
|
del auth_token # Claude-only knob
|
||||||
|
resolved_guest_env = dict(guest_env or {})
|
||||||
|
trusted_path = trusted_project_path or guest_home
|
||||||
|
|
||||||
|
env_vars: dict[str, str] = {
|
||||||
|
"CODEX_CA_CERTIFICATE": "/etc/ssl/certs/ca-certificates.crt",
|
||||||
|
}
|
||||||
|
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
||||||
|
if forward_host_credentials:
|
||||||
|
env_vars["CODEX_HOME"] = auth_dir
|
||||||
|
|
||||||
|
dirs = [AgentProvisionDir(auth_dir)]
|
||||||
|
files: list[AgentProvisionFile] = []
|
||||||
|
pre_copy: list[AgentProvisionCommand] = []
|
||||||
|
verify: list[AgentProvisionCommand] = []
|
||||||
|
provisioned_env: dict[str, str] = {}
|
||||||
|
|
||||||
|
config_path = f"{auth_dir}/config.toml"
|
||||||
|
config_file = state_dir / "codex-config.toml"
|
||||||
|
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
|
||||||
|
config_file.write_text(
|
||||||
|
f'[projects."{toml_path}"]\n'
|
||||||
|
'trust_level = "trusted"\n'
|
||||||
|
)
|
||||||
|
config_file.chmod(0o600)
|
||||||
|
files.append(AgentProvisionFile(config_file, config_path))
|
||||||
|
|
||||||
|
egress_routes: list[EgressRoute] = []
|
||||||
|
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
||||||
|
egress_routes.append(EgressRoute(
|
||||||
|
host=host,
|
||||||
|
auth_scheme="Bearer" if forward_host_credentials else "",
|
||||||
|
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
||||||
|
))
|
||||||
|
|
||||||
|
if forward_host_credentials:
|
||||||
|
_host_env = host_env or dict(os.environ)
|
||||||
|
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = (
|
||||||
|
codex_host_access_token(_host_env)
|
||||||
|
)
|
||||||
|
auth_file = state_dir / "codex-auth.json"
|
||||||
|
write_codex_dummy_auth_file(auth_file, _host_env)
|
||||||
|
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
||||||
|
pre_copy.append(AgentProvisionCommand((
|
||||||
|
"find", auth_dir,
|
||||||
|
"-maxdepth", "1",
|
||||||
|
"-type", "f",
|
||||||
|
"(",
|
||||||
|
"-name", "*.sqlite",
|
||||||
|
"-o", "-name", "*.sqlite-*",
|
||||||
|
"-o", "-name", "*.codex-repair-*.bak",
|
||||||
|
")",
|
||||||
|
"-delete",
|
||||||
|
), "codex host credentials: could not reset runtime db files"))
|
||||||
|
verify.append(AgentProvisionCommand((
|
||||||
|
"runuser", "-u", "node", "--",
|
||||||
|
"env",
|
||||||
|
f"HOME={guest_home}",
|
||||||
|
f"CODEX_HOME={auth_dir}",
|
||||||
|
"codex", "login", "status",
|
||||||
|
), (
|
||||||
|
"codex host credentials: dummy auth was copied into the "
|
||||||
|
"guest, but Codex did not accept it"
|
||||||
|
)))
|
||||||
|
|
||||||
|
return AgentProvisionPlan(
|
||||||
|
template=_RUNTIME.template,
|
||||||
|
command=_RUNTIME.command,
|
||||||
|
prompt_mode=_RUNTIME.prompt_mode,
|
||||||
|
image=_RUNTIME.image,
|
||||||
|
dockerfile=dockerfile,
|
||||||
|
env_vars=env_vars,
|
||||||
|
guest_env=resolved_guest_env,
|
||||||
|
dirs=tuple(dirs),
|
||||||
|
files=tuple(files),
|
||||||
|
pre_copy=tuple(pre_copy),
|
||||||
|
verify=tuple(verify),
|
||||||
|
egress_routes=tuple(egress_routes),
|
||||||
|
provisioned_env=provisioned_env,
|
||||||
|
)
|
||||||
|
|
||||||
|
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||||
|
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
||||||
|
on the host into the guest. No-op when the agent has no
|
||||||
|
skills."""
|
||||||
|
from ...backend.util import host_skill_dir
|
||||||
|
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
if not agent.skills:
|
||||||
|
return
|
||||||
|
skills_dir = _skills_dir(plan.guest_home)
|
||||||
|
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||||
|
for name in agent.skills:
|
||||||
|
src = host_skill_dir(name)
|
||||||
|
if not os.path.isdir(src):
|
||||||
|
die(
|
||||||
|
f"skill {name!r} disappeared from host between "
|
||||||
|
f"validation and copy at {src}."
|
||||||
|
)
|
||||||
|
dst = f"{skills_dir}/{name}"
|
||||||
|
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||||
|
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||||
|
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||||
|
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||||
|
|
||||||
|
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||||
|
"""Copy the prompt file into the guest, fix ownership/mode.
|
||||||
|
Codex reads it via the agent's `Read and follow the
|
||||||
|
instructions in <path>.` bootstrap (see `prompt_args`); the
|
||||||
|
file is copied either way so the path always exists."""
|
||||||
|
prompt_path = _prompt_path(plan.guest_home)
|
||||||
|
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
|
||||||
|
bottle.exec(
|
||||||
|
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||||
|
user="root",
|
||||||
|
)
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
return prompt_path if agent.prompt else None
|
||||||
|
|
||||||
|
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||||
|
"""Apply the codex-side declarative provision steps from
|
||||||
|
`plan.agent_provision`: the `~/.codex/` dir + config.toml
|
||||||
|
trust marker, plus the dummy-auth.json drop + `codex login
|
||||||
|
status` verify when host-credential forwarding is on."""
|
||||||
|
provision = plan.agent_provision
|
||||||
|
for d in provision.dirs:
|
||||||
|
path = shlex.quote(d.guest_path)
|
||||||
|
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chown {shlex.quote(d.owner)} {path}",
|
||||||
|
f"could not chown {d.guest_path}",
|
||||||
|
)
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chmod {shlex.quote(d.mode)} {path}",
|
||||||
|
f"could not chmod {d.guest_path}",
|
||||||
|
)
|
||||||
|
for command in provision.pre_copy:
|
||||||
|
_exec(bottle, shlex.join(command.argv), command.error)
|
||||||
|
for f in provision.files:
|
||||||
|
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||||
|
path = shlex.quote(f.guest_path)
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chown {shlex.quote(f.owner)} {path}",
|
||||||
|
f"could not chown {f.guest_path}",
|
||||||
|
)
|
||||||
|
_exec(
|
||||||
|
bottle,
|
||||||
|
f"chmod {shlex.quote(f.mode)} {path}",
|
||||||
|
f"could not chmod {f.guest_path}",
|
||||||
|
)
|
||||||
|
for command in provision.verify:
|
||||||
|
_exec(bottle, shlex.join(command.argv), command.error)
|
||||||
|
|
||||||
|
def provision_supervise_mcp(
|
||||||
|
self,
|
||||||
|
plan: "BottlePlan",
|
||||||
|
bottle: "Bottle",
|
||||||
|
supervise_url: str,
|
||||||
|
) -> None:
|
||||||
|
"""Run `codex mcp add` inside the agent guest to register the
|
||||||
|
supervise sidecar in Codex's user config (~/.codex/config.toml).
|
||||||
|
|
||||||
|
Mirrors the Claude provider's `claude mcp add` flow — failure
|
||||||
|
is logged but not fatal."""
|
||||||
|
if plan.supervise_plan is None:
|
||||||
|
return
|
||||||
|
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
||||||
|
r = bottle.exec(
|
||||||
|
f"codex mcp add --transport http "
|
||||||
|
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||||
|
user="node",
|
||||||
|
)
|
||||||
|
if r.returncode != 0:
|
||||||
|
warn(
|
||||||
|
f"`codex mcp add supervise` failed (exit {r.returncode}): "
|
||||||
|
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||||
|
f"register manually with: "
|
||||||
|
f"codex mcp add --transport http supervise {supervise_url}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||||
|
result = bottle.exec(script, user="root")
|
||||||
|
if result.returncode != 0:
|
||||||
|
detail = (result.stderr or result.stdout).strip()
|
||||||
|
if detail:
|
||||||
|
detail = f": {detail}"
|
||||||
|
die(f"agent provider provisioning: {error}{detail}")
|
||||||
@@ -13,9 +13,10 @@ import os
|
|||||||
from copy import deepcopy
|
from copy import deepcopy
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
from .log import die
|
from ...log import die
|
||||||
from .util import expand_tilde
|
from ...util import expand_tilde
|
||||||
|
|
||||||
|
|
||||||
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
|
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
|
||||||
@@ -50,7 +51,8 @@ def codex_host_access_token(
|
|||||||
tokens = raw.get("tokens")
|
tokens = raw.get("tokens")
|
||||||
if not isinstance(tokens, dict):
|
if not isinstance(tokens, dict):
|
||||||
die(f"codex host credentials: {path} is missing tokens")
|
die(f"codex host credentials: {path} is missing tokens")
|
||||||
access = tokens.get("access_token")
|
tokens_typed = cast(dict[str, object], tokens)
|
||||||
|
access = tokens_typed.get("access_token")
|
||||||
if not isinstance(access, str) or not access:
|
if not isinstance(access, str) or not access:
|
||||||
die(
|
die(
|
||||||
f"codex host credentials: {path} is missing tokens.access_token. "
|
f"codex host credentials: {path} is missing tokens.access_token. "
|
||||||
@@ -105,14 +107,14 @@ def write_codex_dummy_auth_file(
|
|||||||
path.chmod(0o600)
|
path.chmod(0o600)
|
||||||
|
|
||||||
|
|
||||||
def _read_auth_object(path: Path) -> dict:
|
def _read_auth_object(path: Path) -> dict[str, object]:
|
||||||
try:
|
try:
|
||||||
raw = json.loads(path.read_text())
|
raw = json.loads(path.read_text())
|
||||||
except (OSError, json.JSONDecodeError) as e:
|
except (OSError, json.JSONDecodeError) as e:
|
||||||
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
|
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
|
||||||
if not isinstance(raw, dict):
|
if not isinstance(raw, dict):
|
||||||
die(f"codex host credentials: {path} must contain a JSON object")
|
die(f"codex host credentials: {path} must contain a JSON object")
|
||||||
return raw
|
return cast(dict[str, object], raw)
|
||||||
|
|
||||||
|
|
||||||
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
|
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
|
||||||
@@ -151,11 +153,11 @@ def _dummy_jwt_from_host(
|
|||||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||||
if not isinstance(payload, dict):
|
if not isinstance(payload, dict):
|
||||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||||
return _encode_dummy_jwt(_redact_jwt_payload(payload, now=now, exp_ts=exp_ts))
|
return _encode_dummy_jwt(_redact_jwt_payload(cast(dict[str, object], payload), now=now, exp_ts=exp_ts))
|
||||||
|
|
||||||
|
|
||||||
def _encode_dummy_jwt(payload: dict) -> str:
|
def _encode_dummy_jwt(payload: dict[str, object]) -> str:
|
||||||
def enc(obj: dict) -> str:
|
def enc(obj: dict[str, object]) -> str:
|
||||||
raw = json.dumps(obj, separators=(",", ":")).encode()
|
raw = json.dumps(obj, separators=(",", ":")).encode()
|
||||||
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||||
|
|
||||||
@@ -163,23 +165,24 @@ def _encode_dummy_jwt(payload: dict) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def _redact_jwt_payload(
|
def _redact_jwt_payload(
|
||||||
payload: dict,
|
payload: dict[str, object],
|
||||||
*,
|
*,
|
||||||
now: datetime | None = None,
|
now: datetime | None = None,
|
||||||
exp_ts: int | None = None,
|
exp_ts: int | None = None,
|
||||||
) -> dict:
|
) -> dict[str, object]:
|
||||||
out = _redact_claims(payload)
|
out = _redact_claims(payload)
|
||||||
if not isinstance(out, dict):
|
if not isinstance(out, dict):
|
||||||
out = {}
|
out = {}
|
||||||
out["exp"] = _dummy_exp(now, exp_ts)
|
out_typed: dict[str, object] = cast(dict[str, object], out)
|
||||||
out.setdefault("sub", "bot-bottle-placeholder")
|
out_typed["exp"] = _dummy_exp(now, exp_ts)
|
||||||
return out
|
out_typed.setdefault("sub", "bot-bottle-placeholder")
|
||||||
|
return out_typed
|
||||||
|
|
||||||
|
|
||||||
def _redact_claims(value: object) -> object:
|
def _redact_claims(value: object) -> object:
|
||||||
if isinstance(value, dict):
|
if isinstance(value, dict):
|
||||||
out: dict[str, object] = {}
|
out: dict[str, object] = {}
|
||||||
for key, inner in value.items():
|
for key, inner in cast(dict[str, object], value).items():
|
||||||
lower = key.lower()
|
lower = key.lower()
|
||||||
if key == "https://api.openai.com/profile":
|
if key == "https://api.openai.com/profile":
|
||||||
out[key] = _redact_profile_claim(inner)
|
out[key] = _redact_profile_claim(inner)
|
||||||
@@ -207,16 +210,16 @@ def _redact_claims(value: object) -> object:
|
|||||||
return "bot-bottle-placeholder"
|
return "bot-bottle-placeholder"
|
||||||
|
|
||||||
|
|
||||||
def _redact_profile_claim(value: object) -> dict:
|
def _redact_profile_claim(value: object) -> dict[str, object]:
|
||||||
profile = value if isinstance(value, dict) else {}
|
profile = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||||
return {
|
return {
|
||||||
"email": "bot-bottle@example.invalid",
|
"email": "bot-bottle@example.invalid",
|
||||||
"email_verified": bool(profile.get("email_verified", True)),
|
"email_verified": bool(profile.get("email_verified", True)),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def _redact_auth_claim(value: object) -> dict:
|
def _redact_auth_claim(value: object) -> dict[str, object]:
|
||||||
auth = value if isinstance(value, dict) else {}
|
auth = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||||
out: dict[str, object] = {}
|
out: dict[str, object] = {}
|
||||||
for key, inner in auth.items():
|
for key, inner in auth.items():
|
||||||
lower = key.lower()
|
lower = key.lower()
|
||||||
@@ -247,7 +250,7 @@ def _redact_auth_claim(value: object) -> dict:
|
|||||||
def _redact_codex_auth(
|
def _redact_codex_auth(
|
||||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||||
) -> object:
|
) -> object:
|
||||||
auth = value if isinstance(value, dict) else {}
|
auth = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||||
out: dict[str, object] = {}
|
out: dict[str, object] = {}
|
||||||
for key, inner in auth.items():
|
for key, inner in auth.items():
|
||||||
lower = key.lower()
|
lower = key.lower()
|
||||||
@@ -269,7 +272,7 @@ def _redact_codex_auth(
|
|||||||
def _redact_token_block(
|
def _redact_token_block(
|
||||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||||
) -> dict[str, object]:
|
) -> dict[str, object]:
|
||||||
tokens = value if isinstance(value, dict) else {}
|
tokens = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||||
out: dict[str, object] = {}
|
out: dict[str, object] = {}
|
||||||
for key, inner in tokens.items():
|
for key, inner in tokens.items():
|
||||||
lower = key.lower()
|
lower = key.lower()
|
||||||
@@ -306,7 +309,7 @@ def _jwt_exp(token: str) -> datetime | None:
|
|||||||
return None
|
return None
|
||||||
if not isinstance(payload, dict):
|
if not isinstance(payload, dict):
|
||||||
return None
|
return None
|
||||||
exp = payload.get("exp")
|
exp = cast(dict[str, object], payload).get("exp")
|
||||||
if not isinstance(exp, (int, float)):
|
if not isinstance(exp, (int, float)):
|
||||||
return None
|
return None
|
||||||
return datetime.fromtimestamp(exp, timezone.utc)
|
return datetime.fromtimestamp(exp, timezone.utc)
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
"""Gitea deploy-key provisioner (PRD 0048, contrib).
|
||||||
|
|
||||||
|
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
|
||||||
|
them using the Gitea deploy-key HTTP API. No new Python dependencies —
|
||||||
|
only stdlib `urllib.request` and `subprocess`."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from ...deploy_key_provisioner import DeployKeyProvisioner
|
||||||
|
|
||||||
|
|
||||||
|
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
|
||||||
|
"""Manages deploy keys on a Gitea instance."""
|
||||||
|
|
||||||
|
def __init__(self, *, token: str, api_url: str) -> None:
|
||||||
|
self._token = token
|
||||||
|
self._api_url = api_url.rstrip("/")
|
||||||
|
|
||||||
|
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||||
|
"""Generate an ed25519 keypair, register the public half as a
|
||||||
|
repo deploy key, and return `(key_id, private_key_bytes)`.
|
||||||
|
|
||||||
|
The key is registered with `read_only=False` because git-gate
|
||||||
|
needs push access to forward gitleaks-scanned refs upstream."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
key_path = Path(tmpdir) / "key"
|
||||||
|
subprocess.run(
|
||||||
|
[
|
||||||
|
"ssh-keygen", "-t", "ed25519",
|
||||||
|
"-f", str(key_path),
|
||||||
|
"-N", "",
|
||||||
|
],
|
||||||
|
check=True,
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
stderr=subprocess.DEVNULL,
|
||||||
|
)
|
||||||
|
private_key = key_path.read_bytes()
|
||||||
|
public_key = key_path.with_suffix(".pub").read_text().strip()
|
||||||
|
|
||||||
|
owner, repo = _split_owner_repo(owner_repo)
|
||||||
|
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys"
|
||||||
|
payload = json.dumps({
|
||||||
|
"key": public_key,
|
||||||
|
"read_only": False,
|
||||||
|
"title": title,
|
||||||
|
}).encode()
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
data=payload,
|
||||||
|
headers={
|
||||||
|
"Authorization": f"token {self._token}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req) as resp:
|
||||||
|
body = json.loads(resp.read())
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
_body = _read_error_body(exc)
|
||||||
|
raise RuntimeError(
|
||||||
|
f"failed to create deploy key for {owner_repo}: "
|
||||||
|
f"HTTP {exc.code} — {_body}"
|
||||||
|
) from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"failed to create deploy key for {owner_repo}: {exc.reason}"
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
return str(body["id"]), private_key
|
||||||
|
|
||||||
|
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||||
|
"""Delete the deploy key. HTTP 404 (already gone) is success.
|
||||||
|
All other errors raise RuntimeError so teardown halts."""
|
||||||
|
owner, repo = _split_owner_repo(owner_repo)
|
||||||
|
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys/{key_id}"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
headers={"Authorization": f"token {self._token}"},
|
||||||
|
method="DELETE",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req):
|
||||||
|
pass
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
if exc.code == 404:
|
||||||
|
return
|
||||||
|
_body = _read_error_body(exc)
|
||||||
|
raise RuntimeError(
|
||||||
|
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
||||||
|
f"HTTP {exc.code} — {_body}"
|
||||||
|
) from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
||||||
|
f"{exc.reason}"
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _split_owner_repo(owner_repo: str) -> tuple[str, str]:
|
||||||
|
"""Split `'owner/repo'` into `('owner', 'repo')`."""
|
||||||
|
parts = owner_repo.split("/", 1)
|
||||||
|
if len(parts) != 2 or not all(parts):
|
||||||
|
raise ValueError(
|
||||||
|
f"expected 'owner/repo' format, got {owner_repo!r}"
|
||||||
|
)
|
||||||
|
return parts[0], parts[1]
|
||||||
|
|
||||||
|
|
||||||
|
def _read_error_body(exc: urllib.error.HTTPError) -> str:
|
||||||
|
try:
|
||||||
|
return exc.read().decode("utf-8", errors="replace")
|
||||||
|
except Exception: # noqa: broad-exception-caught — safely fallback to empty error message
|
||||||
|
return ""
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
"""Deploy-key provisioner interface and factory (PRD 0048).
|
||||||
|
|
||||||
|
The core defines the abstract contract; concrete implementations live
|
||||||
|
in `bot_bottle/contrib/<provider>/deploy_key_provisioner.py`. The
|
||||||
|
factory `get_provisioner` imports contrib modules lazily so that a
|
||||||
|
missing optional dependency in one provider doesn't break unrelated
|
||||||
|
features."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
|
|
||||||
|
class DeployKeyProvisioner(ABC):
|
||||||
|
"""Manages a single deploy-key lifecycle on a remote forge."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||||
|
"""Generate a keypair and register the public half as a
|
||||||
|
deploy key on the forge.
|
||||||
|
|
||||||
|
`owner_repo` is the `<owner>/<repo>` path (no `.git` suffix).
|
||||||
|
`title` is the human-readable label shown in the forge UI.
|
||||||
|
|
||||||
|
Returns `(key_id, private_key_bytes)` where `key_id` is opaque
|
||||||
|
to the caller and is only ever passed back to `delete`."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||||
|
"""Delete the registered deploy key.
|
||||||
|
|
||||||
|
Must not raise if the key is already absent (HTTP 404 is
|
||||||
|
success). Must raise for all other failures so teardown halts."""
|
||||||
|
|
||||||
|
|
||||||
|
def get_provisioner(
|
||||||
|
provider: str, token: str, api_url: str
|
||||||
|
) -> DeployKeyProvisioner:
|
||||||
|
"""Instantiate the contrib provisioner for `provider`.
|
||||||
|
|
||||||
|
Raises `ManifestError` for unknown providers so the error surfaces
|
||||||
|
at parse time rather than at runtime."""
|
||||||
|
if provider == "gitea":
|
||||||
|
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
|
||||||
|
GiteaDeployKeyProvisioner,
|
||||||
|
)
|
||||||
|
return GiteaDeployKeyProvisioner(token=token, api_url=api_url)
|
||||||
|
from .manifest_util import ManifestError
|
||||||
|
raise ManifestError(
|
||||||
|
f"unknown provisioned_key provider: {provider!r}; "
|
||||||
|
f"available: gitea"
|
||||||
|
)
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
"""DLP detectors for the egress proxy (PRD 0053).
|
||||||
|
|
||||||
|
Pure Python, no mitmproxy dependency. Each detector is a module-level
|
||||||
|
function returning `ScanResult | None`.
|
||||||
|
|
||||||
|
Ships flat into the sidecar bundle image alongside
|
||||||
|
`egress_addon_core.py` — both this file and the package source use
|
||||||
|
the same try/except import shim pattern.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import re
|
||||||
|
import typing
|
||||||
|
from urllib.parse import quote as url_quote
|
||||||
|
|
||||||
|
try:
|
||||||
|
from egress_addon_core import ScanResult # type: ignore[import-not-found]
|
||||||
|
except ImportError: # pragma: no cover - host-side path
|
||||||
|
from .egress_addon_core import ScanResult
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Token patterns detector (Phase 1a)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
|
||||||
|
("AWS access key", re.compile(r"AKIA[0-9A-Z]{16}")),
|
||||||
|
("GitHub token (classic)", re.compile(r"ghp_[A-Za-z0-9_]{36}")),
|
||||||
|
("GitHub fine-grained token", re.compile(r"github_pat_[A-Za-z0-9_]{82}")),
|
||||||
|
("Anthropic API key", re.compile(r"sk-ant-[A-Za-z0-9\-_]{93}")),
|
||||||
|
("OpenAI API key", re.compile(r"sk-[A-Za-z0-9]{48}")),
|
||||||
|
("Stripe live key", re.compile(r"sk_live_[A-Za-z0-9]{24}")),
|
||||||
|
("Generic Bearer JWT", re.compile(r"Bearer\s+[A-Za-z0-9._\-]{50,}")),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def scan_token_patterns(text: str) -> ScanResult | None:
|
||||||
|
for name, pattern in TOKEN_PATTERNS:
|
||||||
|
if pattern.search(text):
|
||||||
|
return ScanResult(
|
||||||
|
severity="block",
|
||||||
|
reason=f"outbound request contains {name}",
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Known secrets detector (Phase 1b)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _encoded_variants(secret: str) -> list[str]:
|
||||||
|
"""Return the secret plus base64, URL-encoded, and hex variants."""
|
||||||
|
variants = [secret]
|
||||||
|
secret_bytes = secret.encode("utf-8")
|
||||||
|
b64 = base64.b64encode(secret_bytes).decode("ascii")
|
||||||
|
if b64 != secret:
|
||||||
|
variants.append(b64)
|
||||||
|
url_enc = url_quote(secret, safe="")
|
||||||
|
if url_enc != secret:
|
||||||
|
variants.append(url_enc)
|
||||||
|
hex_enc = secret_bytes.hex()
|
||||||
|
if hex_enc != secret:
|
||||||
|
variants.append(hex_enc)
|
||||||
|
return variants
|
||||||
|
|
||||||
|
|
||||||
|
def scan_known_secrets(
|
||||||
|
text: str,
|
||||||
|
*,
|
||||||
|
env: typing.Mapping[str, str] | None = None,
|
||||||
|
) -> ScanResult | None:
|
||||||
|
if env is None:
|
||||||
|
return None
|
||||||
|
for key, value in env.items():
|
||||||
|
if not key.startswith("EGRESS_TOKEN_") or not value:
|
||||||
|
continue
|
||||||
|
for variant in _encoded_variants(value):
|
||||||
|
if variant in text:
|
||||||
|
return ScanResult(
|
||||||
|
severity="block",
|
||||||
|
reason=(
|
||||||
|
f"outbound request contains provisioned secret "
|
||||||
|
f"from {key}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Naive prompt injection detector (Phase 2)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
DISCLOSURE_PHRASES: tuple[re.Pattern[str], ...] = (
|
||||||
|
re.compile(r"(?i)system\s+prompt"),
|
||||||
|
re.compile(r"(?i)my\s+instructions\s+are"),
|
||||||
|
re.compile(r"(?i)original\s+instructions"),
|
||||||
|
re.compile(r"(?i)secret\s+instructions"),
|
||||||
|
re.compile(r"(?i)hidden\s+rules"),
|
||||||
|
)
|
||||||
|
|
||||||
|
JAILBREAK_PHRASES: tuple[re.Pattern[str], ...] = (
|
||||||
|
re.compile(r"(?i)ignore\s+previous"),
|
||||||
|
re.compile(r"(?i)forget\s+everything"),
|
||||||
|
re.compile(r"(?i)disregard\s+(?:all\s+)?(?:previous|prior)"),
|
||||||
|
re.compile(r"(?i)pretend\s+you\s+are"),
|
||||||
|
re.compile(r"(?i)act\s+as\s+(?:if|though)"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
PROXIMITY_CHARS = 500
|
||||||
|
|
||||||
|
|
||||||
|
def _min_distance(
|
||||||
|
a_matches: list[re.Match[str]],
|
||||||
|
b_matches: list[re.Match[str]],
|
||||||
|
) -> int | None:
|
||||||
|
"""Smallest char distance between any pair of matches."""
|
||||||
|
if not a_matches or not b_matches:
|
||||||
|
return None
|
||||||
|
best = None
|
||||||
|
for a in a_matches:
|
||||||
|
for b in b_matches:
|
||||||
|
gap = max(0, max(a.start(), b.start()) - min(a.end(), b.end()))
|
||||||
|
if best is None or gap < best:
|
||||||
|
best = gap
|
||||||
|
return best
|
||||||
|
|
||||||
|
|
||||||
|
def scan_naive_injection(text: str) -> ScanResult | None:
|
||||||
|
disclosure_hits = [m for p in DISCLOSURE_PHRASES for m in p.finditer(text)]
|
||||||
|
jailbreak_hits = [m for p in JAILBREAK_PHRASES for m in p.finditer(text)]
|
||||||
|
|
||||||
|
if disclosure_hits and jailbreak_hits:
|
||||||
|
dist = _min_distance(disclosure_hits, jailbreak_hits)
|
||||||
|
if dist is not None and dist <= PROXIMITY_CHARS:
|
||||||
|
return ScanResult(
|
||||||
|
severity="block",
|
||||||
|
reason=(
|
||||||
|
f"disclosure and jailbreak phrases within "
|
||||||
|
f"{dist} chars in response"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
if disclosure_hits:
|
||||||
|
return ScanResult(
|
||||||
|
severity="warn",
|
||||||
|
reason="prompt disclosure phrase detected in response",
|
||||||
|
)
|
||||||
|
|
||||||
|
if jailbreak_hits:
|
||||||
|
return ScanResult(
|
||||||
|
severity="warn",
|
||||||
|
reason="jailbreak phrase detected in response",
|
||||||
|
)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"TOKEN_PATTERNS",
|
||||||
|
"scan_known_secrets",
|
||||||
|
"scan_naive_injection",
|
||||||
|
"scan_token_patterns",
|
||||||
|
]
|
||||||
+112
-142
@@ -1,36 +1,26 @@
|
|||||||
"""Per-bottle egress proxy (PRD 0017).
|
"""Per-bottle egress proxy (PRD 0017, PRD 0053).
|
||||||
|
|
||||||
Replaces the cred-proxy sidecar (PRD 0010) with a mitmproxy-based
|
|
||||||
sidecar that becomes the agent's `HTTP_PROXY` / `HTTPS_PROXY`. It
|
|
||||||
owns three jobs:
|
|
||||||
|
|
||||||
1. MITM the agent's HTTPS with the per-bottle CA (moved from
|
|
||||||
pipelock).
|
|
||||||
2. Enforce manifest-declared `path_allowlist` per route.
|
|
||||||
3. Inject `Authorization` headers for routes that declare an
|
|
||||||
`auth` block, the same way cred-proxy does today.
|
|
||||||
|
|
||||||
This module defines the abstract proxy (`Egress`), its plan
|
This module defines the abstract proxy (`Egress`), its plan
|
||||||
dataclass (`EgressPlan`), and the resolved per-route shape
|
dataclass (`EgressPlan`), and the resolved per-route shape
|
||||||
(`EgressRoute`). The sidecar's start/stop lifecycle is backend-
|
(`EgressRoute`). The sidecar's start/stop lifecycle is backend-
|
||||||
specific and lives on concrete subclasses (see
|
specific and lives on concrete subclasses (see
|
||||||
`bot_bottle/backend/docker/egress.py`).
|
`bot_bottle/backend/docker/egress.py`).
|
||||||
|
|
||||||
Chunks 1+2 of the PRD: this module + the mitmproxy addon + the Docker
|
|
||||||
lifecycle are wired into the agent's `HTTP_PROXY` path; cred-proxy
|
|
||||||
has been removed. Chunk 3 retargets the cred-proxy-block remediation
|
|
||||||
flow (PRD 0014) at egress and renames the MCP tool.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import dataclasses
|
import dataclasses
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
from .egress_addon_core import Route
|
from .egress_addon_core import (
|
||||||
|
HeaderMatch as CoreHeaderMatch,
|
||||||
|
MatchEntry as CoreMatchEntry,
|
||||||
|
PathMatch as CorePathMatch,
|
||||||
|
Route,
|
||||||
|
)
|
||||||
from .log import die
|
from .log import die
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -38,19 +28,8 @@ if TYPE_CHECKING:
|
|||||||
|
|
||||||
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||||
|
|
||||||
|
|
||||||
# DNS name agents will dial for the per-bottle egress sidecar.
|
|
||||||
# Backend-agnostic by contract: every concrete backend (Docker today,
|
|
||||||
# others later) attaches this name to its sidecar on the bottle's
|
|
||||||
# internal network. The agent's `HTTP_PROXY` env var resolves to
|
|
||||||
# `http://egress:<port>` once chunk 2 cuts over.
|
|
||||||
EGRESS_HOSTNAME = "egress"
|
EGRESS_HOSTNAME = "egress"
|
||||||
|
|
||||||
# In-container path the addon reads. Pre-created in
|
|
||||||
# `Dockerfile.sidecars` so the host bind-mount can drop the file
|
|
||||||
# directly. Content is YAML (hand-rolled by `egress_render_routes`
|
|
||||||
# in the style of `pipelock_render_yaml`, parsed by `yaml_subset`
|
|
||||||
# inside the addon).
|
|
||||||
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
||||||
|
|
||||||
|
|
||||||
@@ -58,68 +37,23 @@ EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
|||||||
class EgressRoute(Route):
|
class EgressRoute(Route):
|
||||||
"""Host-side extension of the addon's `Route`.
|
"""Host-side extension of the addon's `Route`.
|
||||||
|
|
||||||
Inherits `host`, `path_allowlist`, `auth_scheme`, and `token_env`
|
Inherits `host`, `matches`, `auth_scheme`, and `token_env`
|
||||||
from `egress_addon_core.Route` — those are the fields that cross the
|
from `egress_addon_core.Route` — those are the fields that cross the
|
||||||
YAML wire into the sidecar. The three fields below are host-only and
|
YAML wire into the sidecar. The fields below are host-only and
|
||||||
are never serialised to the addon.
|
are never serialised to the addon.
|
||||||
|
|
||||||
`token_ref` is the host env var the CLI reads at launch and forwards
|
`token_ref` is the host env var the CLI reads at launch and forwards
|
||||||
into the container's environ under `token_env`. Routes that share a
|
into the container's environ under `token_env`.
|
||||||
`token_ref` coalesce to one `token_env` slot.
|
|
||||||
|
|
||||||
`roles` carries the manifest route's role tuple (reserved for
|
`roles` carries the manifest route's role tuple (reserved for
|
||||||
future use; always empty today).
|
future use; always empty today)."""
|
||||||
|
|
||||||
`tls_passthrough` signals that pipelock must not TLS-MITM this
|
|
||||||
host — either because the manifest declared `pipelock.tls_passthrough:
|
|
||||||
true` (lifted in `egress_manifest_routes`) or because a provider
|
|
||||||
route set it (e.g. egress injects its own Bearer on that host
|
|
||||||
after the agent boundary and pipelock's header DLP would block it)."""
|
|
||||||
|
|
||||||
token_ref: str = ""
|
token_ref: str = ""
|
||||||
roles: tuple[str, ...] = ()
|
roles: tuple[str, ...] = ()
|
||||||
tls_passthrough: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class EgressPlan:
|
class EgressPlan:
|
||||||
"""Output of Egress.prepare; consumed by .start.
|
|
||||||
|
|
||||||
The slug + routes_path + routes + token_env_map fields are
|
|
||||||
filled at prepare time (host-side, side-effect-free on docker).
|
|
||||||
The network + CA + pipelock fields are populated by the backend's
|
|
||||||
launch step via `dataclasses.replace` once those resources
|
|
||||||
exist. Empty defaults are sentinels meaning "not yet set";
|
|
||||||
`.start` validates that they are populated.
|
|
||||||
|
|
||||||
`token_env_map` is `{<token_env in container>: <token_ref on host>}`.
|
|
||||||
The backend's start step reads `os.environ[token_ref]` and
|
|
||||||
forwards the value into the egress container's environ
|
|
||||||
under `token_env`. The plan itself never holds token values —
|
|
||||||
secrets never land in a dataclass that might be logged.
|
|
||||||
|
|
||||||
`mitmproxy_ca_host_path` is the host path of the per-bottle
|
|
||||||
egress CA (single PEM with cert+key concatenated) minted
|
|
||||||
by `egress_tls_init`. `.start` docker-cps it into the
|
|
||||||
sidecar at `~/.mitmproxy/mitmproxy-ca.pem` — mitmproxy reads
|
|
||||||
that file at boot to mint per-host leaf certs.
|
|
||||||
|
|
||||||
`mitmproxy_ca_cert_only_host_path` is the cert-only PEM (no
|
|
||||||
key) for installing into the agent's trust store via
|
|
||||||
`provision_ca`. Separate file rather than re-parsing the
|
|
||||||
concat so secrets and trust artefacts stay on distinct paths.
|
|
||||||
|
|
||||||
`pipelock_ca_host_path` is the host path of the pipelock CA
|
|
||||||
(cert only). `.start` docker-cps it into the sidecar so the
|
|
||||||
proxy's outbound HTTPS client trusts pipelock's MITM on the
|
|
||||||
egress → upstream leg.
|
|
||||||
|
|
||||||
`pipelock_proxy_url` is the URL egress sets as `HTTPS_PROXY`
|
|
||||||
in its environ so outbound HTTPS traverses pipelock — keeping
|
|
||||||
pipelock's hostname allowlist + DLP body scanner on the
|
|
||||||
egress → upstream leg.
|
|
||||||
"""
|
|
||||||
|
|
||||||
slug: str
|
slug: str
|
||||||
routes_path: Path
|
routes_path: Path
|
||||||
routes: tuple[EgressRoute, ...]
|
routes: tuple[EgressRoute, ...]
|
||||||
@@ -128,26 +62,36 @@ class EgressPlan:
|
|||||||
egress_network: str = ""
|
egress_network: str = ""
|
||||||
mitmproxy_ca_host_path: Path = Path()
|
mitmproxy_ca_host_path: Path = Path()
|
||||||
mitmproxy_ca_cert_only_host_path: Path = Path()
|
mitmproxy_ca_cert_only_host_path: Path = Path()
|
||||||
pipelock_ca_host_path: Path = Path()
|
|
||||||
pipelock_proxy_url: str = ""
|
|
||||||
|
|
||||||
|
|
||||||
def egress_manifest_routes(
|
def egress_manifest_routes(
|
||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Lift each `bottle.egress.routes[]` manifest entry into an EgressRoute.
|
|
||||||
Order is preserved. Token slots are not assigned here — slot assignment
|
|
||||||
is a final step in `egress_routes_for_bottle` after provider and manifest
|
|
||||||
routes are merged."""
|
|
||||||
out: list[EgressRoute] = []
|
out: list[EgressRoute] = []
|
||||||
for r in bottle.egress.routes:
|
for r in bottle.egress.routes:
|
||||||
|
core_matches: list[CoreMatchEntry] = []
|
||||||
|
for m in r.Matches:
|
||||||
|
core_paths = tuple(
|
||||||
|
CorePathMatch(type=p.Type, value=p.Value)
|
||||||
|
for p in m.Paths
|
||||||
|
)
|
||||||
|
core_headers = tuple(
|
||||||
|
CoreHeaderMatch(name=h.Name, value=h.Value, type=h.Type)
|
||||||
|
for h in m.Headers
|
||||||
|
)
|
||||||
|
core_matches.append(CoreMatchEntry(
|
||||||
|
paths=core_paths,
|
||||||
|
methods=m.Methods,
|
||||||
|
headers=core_headers,
|
||||||
|
))
|
||||||
out.append(EgressRoute(
|
out.append(EgressRoute(
|
||||||
host=r.Host,
|
host=r.Host,
|
||||||
path_allowlist=r.PathAllowlist,
|
matches=tuple(core_matches),
|
||||||
auth_scheme=r.AuthScheme,
|
auth_scheme=r.AuthScheme,
|
||||||
token_ref=r.TokenRef,
|
token_ref=r.TokenRef,
|
||||||
roles=r.Role,
|
roles=r.Role,
|
||||||
tls_passthrough=r.Pipelock.TlsPassthrough,
|
outbound_detectors=r.OutboundDetectors,
|
||||||
|
inbound_detectors=r.InboundDetectors,
|
||||||
))
|
))
|
||||||
return tuple(out)
|
return tuple(out)
|
||||||
|
|
||||||
@@ -156,12 +100,6 @@ def egress_routes_for_bottle(
|
|||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Effective egress routes for the agent.
|
|
||||||
|
|
||||||
Provider routes own their hosts outright; manifest routes for hosts
|
|
||||||
not claimed by any provider are appended. Token slots are assigned
|
|
||||||
in a final pass over the merged list in order, so provisioned routes
|
|
||||||
get the lower slot numbers."""
|
|
||||||
manifest = egress_manifest_routes(bottle)
|
manifest = egress_manifest_routes(bottle)
|
||||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||||
merged = list(provider_routes) + [
|
merged = list(provider_routes) + [
|
||||||
@@ -173,10 +111,6 @@ def egress_routes_for_bottle(
|
|||||||
def _assign_token_slots(
|
def _assign_token_slots(
|
||||||
routes: list[EgressRoute],
|
routes: list[EgressRoute],
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Assign EGRESS_TOKEN_N slots to authenticated routes in order.
|
|
||||||
|
|
||||||
Routes sharing a token_ref share a slot. Unauthenticated routes
|
|
||||||
(no auth_scheme / token_ref) keep token_env empty."""
|
|
||||||
slot_for_ref: dict[str, str] = {}
|
slot_for_ref: dict[str, str] = {}
|
||||||
out: list[EgressRoute] = []
|
out: list[EgressRoute] = []
|
||||||
for r in routes:
|
for r in routes:
|
||||||
@@ -194,13 +128,6 @@ def _assign_token_slots(
|
|||||||
def egress_token_env_map(
|
def egress_token_env_map(
|
||||||
routes: tuple[EgressRoute, ...],
|
routes: tuple[EgressRoute, ...],
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
"""Collapse the route list into `{token_env: token_ref}` for the
|
|
||||||
authenticated routes. Routes without `auth` contribute no entry.
|
|
||||||
|
|
||||||
Conflict detection: two routes that share a `token_env` slot but
|
|
||||||
name different `token_ref` host vars is a programming error in
|
|
||||||
`egress_routes_for_bottle`; surface it as a die rather than
|
|
||||||
silently picking one."""
|
|
||||||
out: dict[str, str] = {}
|
out: dict[str, str] = {}
|
||||||
for r in routes:
|
for r in routes:
|
||||||
if not (r.auth_scheme and r.token_ref and r.token_env):
|
if not (r.auth_scheme and r.token_ref and r.token_env):
|
||||||
@@ -216,30 +143,54 @@ def egress_token_env_map(
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _route_to_yaml_fields(r: Route) -> dict:
|
def _route_to_yaml_fields(r: Route) -> dict[str, object]:
|
||||||
"""Return the addon-visible fields for one route.
|
fields: dict[str, object] = {"host": r.host}
|
||||||
|
|
||||||
Single authoritative mapping between EgressRoute (host-side) and
|
|
||||||
egress_addon_core.Route (sidecar-side). When a field is added to
|
|
||||||
the addon's Route that must appear in the YAML, add it here and
|
|
||||||
in egress_addon_core._parse_one together."""
|
|
||||||
fields: dict = {"host": r.host}
|
|
||||||
if r.auth_scheme and r.token_env:
|
if r.auth_scheme and r.token_env:
|
||||||
fields["auth_scheme"] = r.auth_scheme
|
fields["auth_scheme"] = r.auth_scheme
|
||||||
fields["token_env"] = r.token_env
|
fields["token_env"] = r.token_env
|
||||||
if r.path_allowlist:
|
if r.matches:
|
||||||
fields["path_allowlist"] = list(r.path_allowlist)
|
matches_data: list[dict[str, object]] = []
|
||||||
|
for entry in r.matches:
|
||||||
|
entry_data: dict[str, object] = {}
|
||||||
|
if entry.paths:
|
||||||
|
paths_data: list[dict[str, str]] = []
|
||||||
|
for pm in entry.paths:
|
||||||
|
pd: dict[str, str] = {"value": pm.value}
|
||||||
|
if pm.type != "prefix":
|
||||||
|
pd["type"] = pm.type
|
||||||
|
paths_data.append(pd)
|
||||||
|
entry_data["paths"] = paths_data
|
||||||
|
if entry.methods:
|
||||||
|
entry_data["methods"] = list(entry.methods)
|
||||||
|
if entry.headers:
|
||||||
|
headers_data: list[dict[str, str]] = []
|
||||||
|
for hm in entry.headers:
|
||||||
|
hd: dict[str, str] = {"name": hm.name, "value": hm.value}
|
||||||
|
if hm.type != "exact":
|
||||||
|
hd["type"] = hm.type
|
||||||
|
headers_data.append(hd)
|
||||||
|
entry_data["headers"] = headers_data
|
||||||
|
matches_data.append(entry_data)
|
||||||
|
fields["matches"] = matches_data
|
||||||
|
if r.outbound_detectors is not None or r.inbound_detectors is not None:
|
||||||
|
dlp: dict[str, object] = {}
|
||||||
|
if r.outbound_detectors is not None:
|
||||||
|
dlp["outbound_detectors"] = (
|
||||||
|
False if not r.outbound_detectors
|
||||||
|
else list(r.outbound_detectors)
|
||||||
|
)
|
||||||
|
if r.inbound_detectors is not None:
|
||||||
|
dlp["inbound_detectors"] = (
|
||||||
|
False if not r.inbound_detectors
|
||||||
|
else list(r.inbound_detectors)
|
||||||
|
)
|
||||||
|
fields["dlp"] = dlp
|
||||||
return fields
|
return fields
|
||||||
|
|
||||||
|
|
||||||
def egress_render_routes(
|
def egress_render_routes(
|
||||||
routes: tuple[EgressRoute, ...],
|
routes: tuple[EgressRoute, ...],
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Serialize the route table for the addon to read.
|
|
||||||
|
|
||||||
YAML content — no token values, no host env-var names. Fields are
|
|
||||||
determined by `_route_to_yaml_fields`, which is the single point of
|
|
||||||
truth for the EgressRoute → egress_addon_core.Route mapping."""
|
|
||||||
lines: list[str] = ["routes:"]
|
lines: list[str] = ["routes:"]
|
||||||
if not routes:
|
if not routes:
|
||||||
lines[0] = "routes: []"
|
lines[0] = "routes: []"
|
||||||
@@ -250,10 +201,49 @@ def egress_render_routes(
|
|||||||
if "auth_scheme" in f:
|
if "auth_scheme" in f:
|
||||||
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
||||||
lines.append(f' token_env: "{f["token_env"]}"')
|
lines.append(f' token_env: "{f["token_env"]}"')
|
||||||
if "path_allowlist" in f:
|
if "matches" in f:
|
||||||
lines.append(" path_allowlist:")
|
lines.append(" matches:")
|
||||||
for p in f["path_allowlist"]:
|
for entry in f["matches"]: # type: ignore
|
||||||
lines.append(f' - "{p}"')
|
entry_dict: dict[str, object] = entry # type: ignore
|
||||||
|
first_key = True
|
||||||
|
if "paths" in entry_dict:
|
||||||
|
lines.append(" - paths:")
|
||||||
|
first_key = False
|
||||||
|
for pd in entry_dict["paths"]: # type: ignore
|
||||||
|
pd_dict: dict[str, str] = pd # type: ignore
|
||||||
|
if "type" in pd_dict:
|
||||||
|
lines.append(f' - type: "{pd_dict["type"]}"')
|
||||||
|
lines.append(f' value: "{pd_dict["value"]}"')
|
||||||
|
else:
|
||||||
|
lines.append(f' - value: "{pd_dict["value"]}"')
|
||||||
|
if "methods" in entry_dict:
|
||||||
|
methods_str = ", ".join(
|
||||||
|
f'"{m}"' for m in entry_dict["methods"] # type: ignore
|
||||||
|
)
|
||||||
|
prefix = " - " if first_key else " "
|
||||||
|
lines.append(f'{prefix}methods: [{methods_str}]')
|
||||||
|
first_key = False
|
||||||
|
if "headers" in entry_dict:
|
||||||
|
prefix = " - " if first_key else " "
|
||||||
|
lines.append(f"{prefix}headers:")
|
||||||
|
first_key = False
|
||||||
|
for hd in entry_dict["headers"]: # type: ignore
|
||||||
|
hd_dict: dict[str, str] = hd # type: ignore
|
||||||
|
lines.append(f' - name: "{hd_dict["name"]}"')
|
||||||
|
lines.append(f' value: "{hd_dict["value"]}"')
|
||||||
|
if "type" in hd_dict:
|
||||||
|
lines.append(f' type: "{hd_dict["type"]}"')
|
||||||
|
if first_key:
|
||||||
|
lines.append(" - {}")
|
||||||
|
if "dlp" in f:
|
||||||
|
dlp_dict: dict[str, object] = f["dlp"] # type: ignore
|
||||||
|
lines.append(" dlp:")
|
||||||
|
for dk, dv in dlp_dict.items():
|
||||||
|
if dv is False:
|
||||||
|
lines.append(f" {dk}: false")
|
||||||
|
elif isinstance(dv, list):
|
||||||
|
items_str = ", ".join(f'"{x}"' for x in dv)
|
||||||
|
lines.append(f" {dk}: [{items_str}]")
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
@@ -261,12 +251,6 @@ def egress_resolve_token_values(
|
|||||||
token_env_map: dict[str, str],
|
token_env_map: dict[str, str],
|
||||||
host_env: dict[str, str],
|
host_env: dict[str, str],
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
"""Read `host_env[TokenRef]` for each entry in `token_env_map` and
|
|
||||||
return `{token_env: <value>}`. Dies (with a pointer at the missing
|
|
||||||
var name) if any TokenRef is unset.
|
|
||||||
|
|
||||||
Pure function: takes the host env as an argument so tests can pass
|
|
||||||
a sealed mapping without touching `os.environ`."""
|
|
||||||
out: dict[str, str] = {}
|
out: dict[str, str] = {}
|
||||||
for token_env, token_ref in token_env_map.items():
|
for token_env, token_ref in token_env_map.items():
|
||||||
value = host_env.get(token_ref)
|
value = host_env.get(token_ref)
|
||||||
@@ -287,11 +271,6 @@ def egress_resolve_token_values(
|
|||||||
|
|
||||||
|
|
||||||
class Egress(ABC):
|
class Egress(ABC):
|
||||||
"""The per-bottle egress proxy. Encapsulates the host-side prepare
|
|
||||||
(route lift + routes.yaml render + token-env-map derivation); the
|
|
||||||
sidecar's start/stop lifecycle is backend-specific and lives on
|
|
||||||
concrete subclasses."""
|
|
||||||
|
|
||||||
def prepare(
|
def prepare(
|
||||||
self,
|
self,
|
||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
@@ -299,15 +278,6 @@ class Egress(ABC):
|
|||||||
stage_dir: Path,
|
stage_dir: Path,
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
) -> EgressPlan:
|
) -> EgressPlan:
|
||||||
"""Lift `bottle.egress.routes` + `provider_routes` into resolved
|
|
||||||
routes, render the routes file (mode 600) under `stage_dir`, and
|
|
||||||
return the plan. Pure host-side, no docker subprocess. The
|
|
||||||
token-env map records the mapping the launch step uses to
|
|
||||||
forward values from the host's environ into the sidecar's environ.
|
|
||||||
|
|
||||||
Returned plan is incomplete: the launch step must fill
|
|
||||||
`internal_network` / `egress_network` / `pipelock_proxy_url`
|
|
||||||
via `dataclasses.replace` before passing it to `.start`."""
|
|
||||||
routes = egress_routes_for_bottle(bottle, provider_routes)
|
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||||
routes_path = stage_dir / "egress_routes.yaml"
|
routes_path = stage_dir / "egress_routes.yaml"
|
||||||
routes_path.write_text(egress_render_routes(routes))
|
routes_path.write_text(egress_render_routes(routes))
|
||||||
|
|||||||
+63
-77
@@ -1,28 +1,7 @@
|
|||||||
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017).
|
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
|
||||||
|
|
||||||
Loaded by `mitmdump -s /app/egress_addon.py` inside the
|
Loaded by `mitmdump -s /app/egress_addon.py` inside the
|
||||||
egress container. Wraps the pure logic from
|
egress container."""
|
||||||
`egress_addon_core` with mitmproxy's HTTPFlow API:
|
|
||||||
|
|
||||||
- At startup, read `EGRESS_ROUTES` (default
|
|
||||||
`/etc/egress/routes.yaml`, JSON content) → routes table.
|
|
||||||
- SIGHUP re-reads the file and atomically swaps the in-memory
|
|
||||||
table. A parse error keeps the old table in place — better to
|
|
||||||
keep serving the old config than to leave the proxy with no
|
|
||||||
routes after a typo.
|
|
||||||
- On each `request`: strip the inbound Authorization header, then
|
|
||||||
consult `decide()` for forward / block / inject-auth and apply
|
|
||||||
the decision to the flow.
|
|
||||||
|
|
||||||
This file imports `mitmproxy` and is never imported on the host —
|
|
||||||
mitmproxy is a container-only dependency. The host's tests target
|
|
||||||
`egress_addon_core`.
|
|
||||||
|
|
||||||
Dockerfile.sidecars copies both this file and
|
|
||||||
`egress_addon_core.py` flat into `/app/`; the absolute import
|
|
||||||
below works because mitmdump runs with `/app` on its sys.path. The
|
|
||||||
parallel file in the package source tree (bot_bottle/) is the
|
|
||||||
build input — not a module the host imports."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -35,30 +14,23 @@ from pathlib import Path
|
|||||||
|
|
||||||
from mitmproxy import http # type: ignore[import-not-found]
|
from mitmproxy import http # type: ignore[import-not-found]
|
||||||
|
|
||||||
# Absolute import (NOT `from .egress_addon_core`) — the
|
from egress_addon_core import ( # type: ignore[import-not-found]
|
||||||
# container drops both files flat into /app/ so they are sibling
|
Route,
|
||||||
# top-level modules to mitmdump's loader, not a package.
|
decide,
|
||||||
from egress_addon_core import Route, decide, is_git_push_request, load_routes # type: ignore[import-not-found]
|
is_git_push_request,
|
||||||
|
load_routes,
|
||||||
|
match_route,
|
||||||
|
scan_inbound,
|
||||||
|
scan_outbound,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
|
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
|
||||||
|
|
||||||
# Magic hostname the addon recognises as an introspection target.
|
|
||||||
# Requests through the proxy for `_egress.local/<path>` are
|
|
||||||
# intercepted and answered with synthetic responses (the addon's
|
|
||||||
# `request` hook sets `flow.response` before any upstream connection).
|
|
||||||
# The hostname is not in DNS — only clients dialing through this
|
|
||||||
# specific egress can reach it, and only via HTTP (no TLS).
|
|
||||||
# Used by the supervise sidecar's `list-egress-routes` MCP
|
|
||||||
# tool to surface the live route table to the agent.
|
|
||||||
INTROSPECT_HOST = "_egress.local"
|
INTROSPECT_HOST = "_egress.local"
|
||||||
|
|
||||||
|
|
||||||
class EgressAddon:
|
class EgressAddon:
|
||||||
"""The mitmproxy addon. One instance per `mitmdump` process; the
|
|
||||||
request hook is invoked on every CONNECT-decapsulated HTTP/HTTPS
|
|
||||||
request the agent makes."""
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
|
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
|
||||||
self.routes: tuple[Route, ...] = ()
|
self.routes: tuple[Route, ...] = ()
|
||||||
@@ -75,9 +47,6 @@ class EgressAddon:
|
|||||||
f"egress: {tag} load failed: {e}\n"
|
f"egress: {tag} load failed: {e}\n"
|
||||||
)
|
)
|
||||||
if initial:
|
if initial:
|
||||||
# No baseline to fall back on; serve nothing rather
|
|
||||||
# than masquerade as a proxy with a route table the
|
|
||||||
# operator never declared.
|
|
||||||
self.routes = ()
|
self.routes = ()
|
||||||
return
|
return
|
||||||
self.routes = new_routes
|
self.routes = new_routes
|
||||||
@@ -97,11 +66,6 @@ class EgressAddon:
|
|||||||
signal.signal(signal.SIGHUP, handler)
|
signal.signal(signal.SIGHUP, handler)
|
||||||
|
|
||||||
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
|
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
|
||||||
"""Synthesize a response for `_egress.local` requests.
|
|
||||||
Currently supports `/allowlist` which returns the in-memory
|
|
||||||
route table as JSON (host, path_allowlist, auth_scheme,
|
|
||||||
token_env per route — no token VALUES, those live in the
|
|
||||||
container's environ)."""
|
|
||||||
if path == "/allowlist":
|
if path == "/allowlist":
|
||||||
payload = json.dumps(
|
payload = json.dumps(
|
||||||
{"routes": [dataclasses.asdict(r) for r in self.routes]},
|
{"routes": [dataclasses.asdict(r) for r in self.routes]},
|
||||||
@@ -118,61 +82,83 @@ class EgressAddon:
|
|||||||
{"Content-Type": "text/plain; charset=utf-8"},
|
{"Content-Type": "text/plain; charset=utf-8"},
|
||||||
)
|
)
|
||||||
|
|
||||||
# mitmproxy's addon API: this method name + signature is how
|
def _block(self, flow: http.HTTPFlow, reason: str) -> None:
|
||||||
# mitmdump discovers the request hook.
|
sys.stderr.write(f"{reason}\n")
|
||||||
|
flow.response = http.Response.make(
|
||||||
|
403,
|
||||||
|
reason.encode("utf-8"),
|
||||||
|
{"Content-Type": "text/plain; charset=utf-8"},
|
||||||
|
)
|
||||||
|
|
||||||
def request(self, flow: http.HTTPFlow) -> None:
|
def request(self, flow: http.HTTPFlow) -> None:
|
||||||
request_path, _, query = flow.request.path.partition("?")
|
request_path, _, query = flow.request.path.partition("?")
|
||||||
|
|
||||||
# Introspection: requests to the magic `_egress.local`
|
|
||||||
# host are answered locally with a synthetic response. Check
|
|
||||||
# before the strip-auth + route logic — these requests aren't
|
|
||||||
# real upstream traffic, the agent isn't injecting auth, and
|
|
||||||
# the addon's own decide() would 403 the magic host (it's
|
|
||||||
# never in the routes table).
|
|
||||||
if flow.request.pretty_host == INTROSPECT_HOST:
|
if flow.request.pretty_host == INTROSPECT_HOST:
|
||||||
self._serve_introspection(flow, request_path)
|
self._serve_introspection(flow, request_path)
|
||||||
return
|
return
|
||||||
|
|
||||||
# Inbound Authorization is always stripped — the agent cannot
|
# DLP outbound scan BEFORE stripping auth — catches tokens the
|
||||||
# smuggle a stolen token through the proxy. If the matched
|
# agent tried to smuggle in the Authorization header.
|
||||||
# route declares an auth pair, a fresh header is injected
|
route = match_route(self.routes, flow.request.pretty_host)
|
||||||
# below.
|
if route is not None:
|
||||||
|
body = flow.request.get_text(strict=False) or ""
|
||||||
|
auth_header = flow.request.headers.get("authorization", "")
|
||||||
|
scan_text = body
|
||||||
|
if auth_header:
|
||||||
|
scan_text = auth_header + "\n" + body
|
||||||
|
dlp_result = scan_outbound(route, scan_text, os.environ)
|
||||||
|
if dlp_result is not None and dlp_result.severity == "block":
|
||||||
|
self._block(flow, f"egress DLP: {dlp_result.reason}")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Strip inbound Authorization — agent cannot smuggle tokens.
|
||||||
flow.request.headers.pop("authorization", None)
|
flow.request.headers.pop("authorization", None)
|
||||||
|
|
||||||
# Universal HTTPS git-push block. Defense-in-depth: git-gate
|
|
||||||
# (PRD 0008) is the only sanctioned outbound path for git
|
|
||||||
# writes — its pre-receive runs gitleaks. Letting HTTPS push
|
|
||||||
# through egress + auth injection would route around
|
|
||||||
# that scan, so we 403 before any route logic.
|
|
||||||
if is_git_push_request(request_path, query):
|
if is_git_push_request(request_path, query):
|
||||||
flow.response = http.Response.make(
|
self._block(
|
||||||
403,
|
flow,
|
||||||
(
|
"egress: git push over HTTPS is not supported; "
|
||||||
b"egress: git push over HTTPS is not supported; "
|
"use the bottle.git SSH path (gitleaks-scanned by "
|
||||||
b"use the bottle.git SSH path (gitleaks-scanned by "
|
"git-gate's pre-receive hook).",
|
||||||
b"git-gate's pre-receive hook)."
|
|
||||||
),
|
|
||||||
{"Content-Type": "text/plain; charset=utf-8"},
|
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
# Build headers mapping for match evaluation
|
||||||
|
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
|
||||||
|
|
||||||
decision = decide(
|
decision = decide(
|
||||||
self.routes,
|
self.routes,
|
||||||
flow.request.pretty_host,
|
flow.request.pretty_host,
|
||||||
request_path,
|
request_path,
|
||||||
os.environ,
|
os.environ,
|
||||||
|
request_method=flow.request.method,
|
||||||
|
request_headers=req_headers,
|
||||||
)
|
)
|
||||||
|
|
||||||
if decision.action == "block":
|
if decision.action == "block":
|
||||||
flow.response = http.Response.make(
|
self._block(flow, decision.reason)
|
||||||
403,
|
|
||||||
decision.reason.encode("utf-8"),
|
|
||||||
{"Content-Type": "text/plain; charset=utf-8"},
|
|
||||||
)
|
|
||||||
return
|
return
|
||||||
|
|
||||||
if decision.inject_authorization is not None:
|
if decision.inject_authorization is not None:
|
||||||
flow.request.headers["authorization"] = decision.inject_authorization
|
flow.request.headers["authorization"] = decision.inject_authorization
|
||||||
|
|
||||||
|
def response(self, flow: http.HTTPFlow) -> None:
|
||||||
|
"""DLP inbound scan on response bodies (PRD 0053)."""
|
||||||
|
route = match_route(self.routes, flow.request.pretty_host)
|
||||||
|
if route is None:
|
||||||
|
return
|
||||||
|
if flow.response is None:
|
||||||
|
return
|
||||||
|
body = flow.response.get_text(strict=False) or ""
|
||||||
|
if not body:
|
||||||
|
return
|
||||||
|
result = scan_inbound(route, body)
|
||||||
|
if result is None:
|
||||||
|
return
|
||||||
|
if result.severity == "block":
|
||||||
|
self._block(flow, f"egress DLP: {result.reason}")
|
||||||
|
elif result.severity == "warn":
|
||||||
|
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
|
||||||
|
|
||||||
|
|
||||||
addons = [EgressAddon()]
|
addons = [EgressAddon()]
|
||||||
|
|||||||
+406
-123
@@ -1,4 +1,4 @@
|
|||||||
"""Pure logic for the egress mitmproxy addon (PRD 0017).
|
"""Pure logic for the egress mitmproxy addon (PRD 0017, PRD 0053).
|
||||||
|
|
||||||
Split out of `egress_addon.py` so the host's unit tests can
|
Split out of `egress_addon.py` so the host's unit tests can
|
||||||
exercise the parse + decision functions without depending on the
|
exercise the parse + decision functions without depending on the
|
||||||
@@ -8,81 +8,263 @@ container.
|
|||||||
|
|
||||||
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
||||||
ships flat into the sidecar bundle image alongside this file —
|
ships flat into the sidecar bundle image alongside this file —
|
||||||
see `Dockerfile.sidecars`).
|
see `Dockerfile.sidecars`)."""
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
import typing
|
import typing
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
|
||||||
# Absolute import — `yaml_subset.py` is copied flat into the bundle
|
|
||||||
# image's `/app/` next to this file (via `Dockerfile.sidecars`).
|
|
||||||
# The host-side unit tests run with the repo on sys.path, where the
|
|
||||||
# import resolves under the `bot_bottle` package. The try/except
|
|
||||||
# shim picks whichever import works.
|
|
||||||
try:
|
try:
|
||||||
from yaml_subset import YamlSubsetError, parse_yaml_subset # type: ignore[import-not-found]
|
from yaml_subset import YamlSubsetError, parse_yaml_subset # type: ignore[import-not-found]
|
||||||
except ImportError: # pragma: no cover - host-side path
|
except ImportError: # pragma: no cover - host-side path
|
||||||
from .yaml_subset import YamlSubsetError, parse_yaml_subset
|
from .yaml_subset import YamlSubsetError, parse_yaml_subset
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Match types (Gateway API HTTPRoute vocabulary, PRD 0053)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PATH_MATCH_TYPES = ("exact", "prefix", "regex")
|
||||||
|
HEADER_MATCH_TYPES = ("exact", "regex")
|
||||||
|
|
||||||
|
VALID_METHODS = frozenset({
|
||||||
|
"GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "TRACE",
|
||||||
|
"CONNECT",
|
||||||
|
})
|
||||||
|
|
||||||
|
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
|
||||||
|
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PathMatch:
|
||||||
|
type: str # "exact" | "prefix" | "regex"
|
||||||
|
value: str
|
||||||
|
compiled: re.Pattern[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HeaderMatch:
|
||||||
|
name: str
|
||||||
|
value: str
|
||||||
|
type: str = "exact" # "exact" | "regex"
|
||||||
|
compiled: re.Pattern[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class MatchEntry:
|
||||||
|
paths: tuple[PathMatch, ...] = ()
|
||||||
|
methods: tuple[str, ...] = ()
|
||||||
|
headers: tuple[HeaderMatch, ...] = ()
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class Route:
|
class Route:
|
||||||
"""One row of the egress route table.
|
|
||||||
|
|
||||||
`host` is the request's `Host` header (or SNI hostname) to match
|
|
||||||
against. `path_allowlist` is an optional tuple of absolute path
|
|
||||||
prefixes the request path must start with; empty tuple means no
|
|
||||||
path constraint. `auth_scheme` and `token_env` together form the
|
|
||||||
credential-injection pair (both set or both empty); a non-empty
|
|
||||||
pair tells the addon to overwrite the inbound Authorization with
|
|
||||||
`<auth_scheme> <value-of-environ[token_env]>`.
|
|
||||||
"""
|
|
||||||
|
|
||||||
host: str
|
host: str
|
||||||
path_allowlist: tuple[str, ...] = ()
|
matches: tuple[MatchEntry, ...] = ()
|
||||||
auth_scheme: str = ""
|
auth_scheme: str = ""
|
||||||
token_env: str = ""
|
token_env: str = ""
|
||||||
|
outbound_detectors: tuple[str, ...] | None = None
|
||||||
|
inbound_detectors: tuple[str, ...] | None = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class Decision:
|
class Decision:
|
||||||
"""The result of `decide()`. Either forward (with optional
|
|
||||||
`inject_authorization` header) or block (with a `reason` to surface
|
|
||||||
to the agent)."""
|
|
||||||
|
|
||||||
action: str # "forward" or "block"
|
action: str # "forward" or "block"
|
||||||
reason: str = ""
|
reason: str = ""
|
||||||
inject_authorization: str | None = None
|
inject_authorization: str | None = None
|
||||||
|
|
||||||
|
|
||||||
def parse_routes(payload: object) -> tuple[Route, ...]:
|
@dataclass(frozen=True)
|
||||||
"""Parse the routes-file payload (already JSON-decoded) into a
|
class ScanResult:
|
||||||
tuple of `Route`s. Raises `ValueError` on any malformed entry —
|
severity: str # "block" or "warn"
|
||||||
the caller decides whether to keep the old table or refuse to
|
reason: str
|
||||||
start.
|
|
||||||
|
|
||||||
Schema:
|
|
||||||
{
|
# ---------------------------------------------------------------------------
|
||||||
"routes": [
|
# Parsing
|
||||||
{
|
# ---------------------------------------------------------------------------
|
||||||
"host": "api.github.com",
|
|
||||||
"path_allowlist": ["/repos/x/", "/users/x"], # optional
|
def _parse_path_match(idx: int, j: int, raw: object) -> PathMatch:
|
||||||
"auth_scheme": "Bearer", # optional
|
label = f"route[{idx}] matches paths[{j}]"
|
||||||
"token_env": "EGRESS_TOKEN_0" # optional
|
if not isinstance(raw, dict):
|
||||||
},
|
raise ValueError(f"{label}: must be an object")
|
||||||
...
|
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||||
]
|
ptype = raw_dict.get("type", "prefix")
|
||||||
}
|
if not isinstance(ptype, str) or ptype not in PATH_MATCH_TYPES:
|
||||||
"""
|
raise ValueError(
|
||||||
|
f"{label}: 'type' must be one of {', '.join(PATH_MATCH_TYPES)} "
|
||||||
|
f"(got {ptype!r})"
|
||||||
|
)
|
||||||
|
value = raw_dict.get("value")
|
||||||
|
if not isinstance(value, str) or not value:
|
||||||
|
raise ValueError(f"{label}: 'value' must be a non-empty string")
|
||||||
|
if ptype in ("exact", "prefix") and not value.startswith("/"):
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: value {value!r} must start with '/' for "
|
||||||
|
f"type {ptype!r}"
|
||||||
|
)
|
||||||
|
compiled: re.Pattern[str] | None = None
|
||||||
|
if ptype == "regex":
|
||||||
|
try:
|
||||||
|
compiled = re.compile(value)
|
||||||
|
except re.error as e:
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: regex {value!r} failed to compile: {e}"
|
||||||
|
) from e
|
||||||
|
for k in raw_dict:
|
||||||
|
if k not in ("type", "value"):
|
||||||
|
raise ValueError(f"{label}: unknown key {k!r}")
|
||||||
|
return PathMatch(type=ptype, value=value, compiled=compiled)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_header_match(idx: int, j: int, raw: object) -> HeaderMatch:
|
||||||
|
label = f"route[{idx}] matches headers[{j}]"
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
raise ValueError(f"{label}: must be an object")
|
||||||
|
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||||
|
name = raw_dict.get("name")
|
||||||
|
if not isinstance(name, str) or not name:
|
||||||
|
raise ValueError(f"{label}: 'name' must be a non-empty string")
|
||||||
|
value = raw_dict.get("value")
|
||||||
|
if not isinstance(value, str):
|
||||||
|
raise ValueError(f"{label}: 'value' must be a string")
|
||||||
|
htype = raw_dict.get("type", "exact")
|
||||||
|
if not isinstance(htype, str) or htype not in HEADER_MATCH_TYPES:
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: 'type' must be one of {', '.join(HEADER_MATCH_TYPES)} "
|
||||||
|
f"(got {htype!r})"
|
||||||
|
)
|
||||||
|
compiled: re.Pattern[str] | None = None
|
||||||
|
if htype == "regex":
|
||||||
|
try:
|
||||||
|
compiled = re.compile(value)
|
||||||
|
except re.error as e:
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: regex {value!r} failed to compile: {e}"
|
||||||
|
) from e
|
||||||
|
for k in raw_dict:
|
||||||
|
if k not in ("name", "value", "type"):
|
||||||
|
raise ValueError(f"{label}: unknown key {k!r}")
|
||||||
|
return HeaderMatch(name=name, value=value, type=htype, compiled=compiled)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_match_entry(idx: int, k: int, raw: object) -> MatchEntry:
|
||||||
|
label = f"route[{idx}] matches[{k}]"
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
raise ValueError(f"{label}: must be an object")
|
||||||
|
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||||
|
|
||||||
|
paths: tuple[PathMatch, ...] = ()
|
||||||
|
paths_raw = raw_dict.get("paths")
|
||||||
|
if paths_raw is not None:
|
||||||
|
if not isinstance(paths_raw, list):
|
||||||
|
raise ValueError(f"{label}: 'paths' must be a list")
|
||||||
|
paths_list = typing.cast(list[object], paths_raw)
|
||||||
|
paths = tuple(_parse_path_match(idx, j, p) for j, p in enumerate(paths_list))
|
||||||
|
|
||||||
|
methods: tuple[str, ...] = ()
|
||||||
|
methods_raw = raw_dict.get("methods")
|
||||||
|
if methods_raw is not None:
|
||||||
|
if not isinstance(methods_raw, list):
|
||||||
|
raise ValueError(f"{label}: 'methods' must be a list")
|
||||||
|
methods_list = typing.cast(list[object], methods_raw)
|
||||||
|
normalised: list[str] = []
|
||||||
|
for j, m in enumerate(methods_list):
|
||||||
|
if not isinstance(m, str):
|
||||||
|
raise ValueError(f"{label}: methods[{j}] must be a string")
|
||||||
|
upper = m.upper()
|
||||||
|
if upper not in VALID_METHODS:
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: methods[{j}] {m!r} is not a valid HTTP method"
|
||||||
|
)
|
||||||
|
normalised.append(upper)
|
||||||
|
methods = tuple(normalised)
|
||||||
|
|
||||||
|
headers: tuple[HeaderMatch, ...] = ()
|
||||||
|
headers_raw = raw_dict.get("headers")
|
||||||
|
if headers_raw is not None:
|
||||||
|
if not isinstance(headers_raw, list):
|
||||||
|
raise ValueError(f"{label}: 'headers' must be a list")
|
||||||
|
headers_list = typing.cast(list[object], headers_raw)
|
||||||
|
headers = tuple(
|
||||||
|
_parse_header_match(idx, j, h) for j, h in enumerate(headers_list)
|
||||||
|
)
|
||||||
|
|
||||||
|
for key in raw_dict:
|
||||||
|
if key not in ("paths", "methods", "headers"):
|
||||||
|
raise ValueError(f"{label}: unknown key {key!r}")
|
||||||
|
|
||||||
|
return MatchEntry(paths=paths, methods=methods, headers=headers)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_detectors(
|
||||||
|
idx: int,
|
||||||
|
host: str,
|
||||||
|
raw_dict: dict[str, object],
|
||||||
|
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
|
||||||
|
"""Parse the optional `dlp` block on a route, returning
|
||||||
|
(outbound_detectors, inbound_detectors)."""
|
||||||
|
dlp_raw = raw_dict.get("dlp")
|
||||||
|
if dlp_raw is None:
|
||||||
|
return None, None
|
||||||
|
label = f"route[{idx}] ({host})"
|
||||||
|
if not isinstance(dlp_raw, dict):
|
||||||
|
raise ValueError(f"{label}: 'dlp' must be an object")
|
||||||
|
dlp = typing.cast(dict[str, object], dlp_raw)
|
||||||
|
|
||||||
|
def _parse_detector_field(
|
||||||
|
field: str,
|
||||||
|
valid_names: frozenset[str],
|
||||||
|
) -> tuple[str, ...] | None:
|
||||||
|
val = dlp.get(field)
|
||||||
|
if val is None:
|
||||||
|
return None
|
||||||
|
if val is False:
|
||||||
|
return ()
|
||||||
|
if not isinstance(val, list):
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: dlp.{field} must be false, a list, or omitted"
|
||||||
|
)
|
||||||
|
items = typing.cast(list[object], val)
|
||||||
|
names: list[str] = []
|
||||||
|
for j, item in enumerate(items):
|
||||||
|
if not isinstance(item, str):
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: dlp.{field}[{j}] must be a string"
|
||||||
|
)
|
||||||
|
if item not in valid_names:
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: dlp.{field}[{j}] {item!r} is not a valid "
|
||||||
|
f"detector name; valid names: {', '.join(sorted(valid_names))}"
|
||||||
|
)
|
||||||
|
names.append(item)
|
||||||
|
return tuple(names)
|
||||||
|
|
||||||
|
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
|
||||||
|
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
|
||||||
|
|
||||||
|
for k in dlp:
|
||||||
|
if k not in ("outbound_detectors", "inbound_detectors"):
|
||||||
|
raise ValueError(
|
||||||
|
f"{label}: dlp has unknown key {k!r}; accepted keys "
|
||||||
|
f"are 'outbound_detectors', 'inbound_detectors'"
|
||||||
|
)
|
||||||
|
return outbound, inbound
|
||||||
|
|
||||||
|
|
||||||
|
def parse_routes(payload: object) -> tuple[Route, ...]:
|
||||||
if not isinstance(payload, dict):
|
if not isinstance(payload, dict):
|
||||||
raise ValueError("routes payload: top-level must be an object")
|
raise ValueError("routes payload: top-level must be an object")
|
||||||
raw = payload.get("routes")
|
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
|
||||||
|
raw: object = payload_dict.get("routes")
|
||||||
if not isinstance(raw, list):
|
if not isinstance(raw, list):
|
||||||
raise ValueError("routes payload: 'routes' must be a list")
|
raise ValueError("routes payload: 'routes' must be a list")
|
||||||
|
raw_list: list[object] = typing.cast(list[object], raw)
|
||||||
out: list[Route] = []
|
out: list[Route] = []
|
||||||
for i, r in enumerate(raw):
|
for i, r in enumerate(raw_list):
|
||||||
out.append(_parse_one(i, r))
|
out.append(_parse_one(i, r))
|
||||||
return tuple(out)
|
return tuple(out)
|
||||||
|
|
||||||
@@ -91,35 +273,29 @@ def _parse_one(idx: int, raw: object) -> Route:
|
|||||||
label = f"route[{idx}]"
|
label = f"route[{idx}]"
|
||||||
if not isinstance(raw, dict):
|
if not isinstance(raw, dict):
|
||||||
raise ValueError(f"{label}: must be an object (got {type(raw).__name__})")
|
raise ValueError(f"{label}: must be an object (got {type(raw).__name__})")
|
||||||
host = raw.get("host")
|
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||||
|
host: object = raw_dict.get("host")
|
||||||
if not isinstance(host, str) or not host:
|
if not isinstance(host, str) or not host:
|
||||||
raise ValueError(f"{label}: 'host' must be a non-empty string")
|
raise ValueError(f"{label}: 'host' must be a non-empty string")
|
||||||
|
|
||||||
path_allow_raw = raw.get("path_allowlist", [])
|
# matches
|
||||||
if not isinstance(path_allow_raw, list):
|
matches: tuple[MatchEntry, ...] = ()
|
||||||
raise ValueError(f"{label} ({host}): 'path_allowlist' must be a list")
|
matches_raw = raw_dict.get("matches")
|
||||||
prefixes: list[str] = []
|
if matches_raw is not None:
|
||||||
for j, p in enumerate(path_allow_raw):
|
if not isinstance(matches_raw, list):
|
||||||
if not isinstance(p, str):
|
raise ValueError(f"{label} ({host}): 'matches' must be a list")
|
||||||
raise ValueError(
|
matches_list = typing.cast(list[object], matches_raw)
|
||||||
f"{label} ({host}): path_allowlist[{j}] must be a string"
|
matches = tuple(
|
||||||
)
|
_parse_match_entry(idx, k, m) for k, m in enumerate(matches_list)
|
||||||
if not p.startswith("/"):
|
)
|
||||||
raise ValueError(
|
|
||||||
f"{label} ({host}): path_allowlist[{j}] {p!r} must be an "
|
|
||||||
f"absolute path prefix starting with '/'"
|
|
||||||
)
|
|
||||||
prefixes.append(p)
|
|
||||||
|
|
||||||
auth_scheme = raw.get("auth_scheme", "")
|
# auth (unchanged wire format)
|
||||||
token_env = raw.get("token_env", "")
|
auth_scheme: object = raw_dict.get("auth_scheme", "")
|
||||||
|
token_env: object = raw_dict.get("token_env", "")
|
||||||
if not isinstance(auth_scheme, str):
|
if not isinstance(auth_scheme, str):
|
||||||
raise ValueError(f"{label} ({host}): 'auth_scheme' must be a string")
|
raise ValueError(f"{label} ({host}): 'auth_scheme' must be a string")
|
||||||
if not isinstance(token_env, str):
|
if not isinstance(token_env, str):
|
||||||
raise ValueError(f"{label} ({host}): 'token_env' must be a string")
|
raise ValueError(f"{label} ({host}): 'token_env' must be a string")
|
||||||
# Both-or-neither: 'auth' on the manifest side renders to this
|
|
||||||
# pair atomically. A partial pair here means the renderer or a
|
|
||||||
# hand-edited file is broken.
|
|
||||||
if bool(auth_scheme) != bool(token_env):
|
if bool(auth_scheme) != bool(token_env):
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"{label} ({host}): 'auth_scheme' and 'token_env' must be both "
|
f"{label} ({host}): 'auth_scheme' and 'token_env' must be both "
|
||||||
@@ -127,19 +303,30 @@ def _parse_one(idx: int, raw: object) -> Route:
|
|||||||
f"token_env={token_env!r})"
|
f"token_env={token_env!r})"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# dlp detectors
|
||||||
|
outbound_detectors, inbound_detectors = _parse_detectors(
|
||||||
|
idx, host, raw_dict,
|
||||||
|
)
|
||||||
|
|
||||||
|
for k in raw_dict:
|
||||||
|
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp"):
|
||||||
|
raise ValueError(
|
||||||
|
f"{label} ({host}): unknown key {k!r}; accepted keys "
|
||||||
|
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp'"
|
||||||
|
)
|
||||||
|
|
||||||
return Route(
|
return Route(
|
||||||
host=host,
|
host=host,
|
||||||
path_allowlist=tuple(prefixes),
|
matches=matches,
|
||||||
auth_scheme=auth_scheme,
|
auth_scheme=auth_scheme,
|
||||||
token_env=token_env,
|
token_env=token_env,
|
||||||
|
outbound_detectors=outbound_detectors,
|
||||||
|
inbound_detectors=inbound_detectors,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def load_routes(text: str) -> tuple[Route, ...]:
|
def load_routes(text: str) -> tuple[Route, ...]:
|
||||||
"""Parse YAML text → routes. Raises `ValueError` for both
|
"""Parse YAML text → routes."""
|
||||||
decode and shape errors so callers handle them uniformly.
|
|
||||||
`YamlSubsetError` from the parser is a `ValueError` subclass so
|
|
||||||
it already satisfies the same surface; we let it propagate."""
|
|
||||||
try:
|
try:
|
||||||
payload = parse_yaml_subset(text)
|
payload = parse_yaml_subset(text)
|
||||||
except YamlSubsetError as e:
|
except YamlSubsetError as e:
|
||||||
@@ -147,29 +334,76 @@ def load_routes(text: str) -> tuple[Route, ...]:
|
|||||||
return parse_routes(payload)
|
return parse_routes(payload)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Match evaluation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _path_matches(pm: PathMatch, request_path: str) -> bool:
|
||||||
|
if pm.type == "exact":
|
||||||
|
return request_path == pm.value
|
||||||
|
if pm.type == "prefix":
|
||||||
|
if request_path == pm.value:
|
||||||
|
return True
|
||||||
|
if not pm.value.endswith("/"):
|
||||||
|
return request_path.startswith(pm.value + "/")
|
||||||
|
return request_path.startswith(pm.value)
|
||||||
|
if pm.type == "regex" and pm.compiled is not None:
|
||||||
|
return pm.compiled.search(request_path) is not None
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _entry_matches(
|
||||||
|
entry: MatchEntry,
|
||||||
|
request_path: str,
|
||||||
|
request_method: str,
|
||||||
|
request_headers: typing.Mapping[str, str],
|
||||||
|
) -> bool:
|
||||||
|
"""All predicates within a MatchEntry are ANDed."""
|
||||||
|
if entry.paths:
|
||||||
|
if not any(_path_matches(pm, request_path) for pm in entry.paths):
|
||||||
|
return False
|
||||||
|
if entry.methods:
|
||||||
|
if request_method.upper() not in entry.methods:
|
||||||
|
return False
|
||||||
|
if entry.headers:
|
||||||
|
for hm in entry.headers:
|
||||||
|
header_val = request_headers.get(hm.name.lower())
|
||||||
|
if header_val is None:
|
||||||
|
return False
|
||||||
|
if hm.type == "exact":
|
||||||
|
if header_val != hm.value:
|
||||||
|
return False
|
||||||
|
elif hm.type == "regex" and hm.compiled is not None:
|
||||||
|
if not hm.compiled.search(header_val):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_matches(
|
||||||
|
route: Route,
|
||||||
|
request_path: str,
|
||||||
|
request_method: str = "GET",
|
||||||
|
request_headers: typing.Mapping[str, str] | None = None,
|
||||||
|
) -> bool:
|
||||||
|
"""Return True if the request matches this route's match entries.
|
||||||
|
Empty matches tuple means all requests match (bare-pass route)."""
|
||||||
|
if not route.matches:
|
||||||
|
return True
|
||||||
|
hdrs: typing.Mapping[str, str] = request_headers or {}
|
||||||
|
return any(
|
||||||
|
_entry_matches(entry, request_path, request_method, hdrs)
|
||||||
|
for entry in route.matches
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Git push detection (unchanged)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
def is_git_push_request(path: str, query: str) -> bool:
|
def is_git_push_request(path: str, query: str) -> bool:
|
||||||
"""Return True if the request is a git smart-HTTP push.
|
|
||||||
|
|
||||||
git push over HTTPS hits two endpoints:
|
|
||||||
GET <repo>/info/refs?service=git-receive-pack (capabilities)
|
|
||||||
POST <repo>/git-receive-pack (the push)
|
|
||||||
|
|
||||||
Fetches use `service=git-upload-pack` / `/git-upload-pack` and
|
|
||||||
are unaffected. Egress-proxy refuses HTTPS push because git-gate's
|
|
||||||
pre-receive gitleaks scan is the gate for outbound git data;
|
|
||||||
routing push through egress would bypass that. Use the
|
|
||||||
bottle.git SSH path if you need to push.
|
|
||||||
|
|
||||||
Universal across routes — the block fires even when no
|
|
||||||
egress route matches the host. A bare-pass route (host with
|
|
||||||
no auth, no path_allowlist) would otherwise let push through to
|
|
||||||
pipelock + upstream untouched.
|
|
||||||
"""
|
|
||||||
if path.endswith("/git-receive-pack"):
|
if path.endswith("/git-receive-pack"):
|
||||||
return True
|
return True
|
||||||
if path.endswith("/info/refs"):
|
if path.endswith("/info/refs"):
|
||||||
# Query string is parsed leniently — `service=git-receive-pack`
|
|
||||||
# may appear with other params in any order.
|
|
||||||
for pair in query.split("&"):
|
for pair in query.split("&"):
|
||||||
k, _, v = pair.partition("=")
|
k, _, v = pair.partition("=")
|
||||||
if k == "service" and v == "git-receive-pack":
|
if k == "service" and v == "git-receive-pack":
|
||||||
@@ -177,18 +411,14 @@ def is_git_push_request(path: str, query: str) -> bool:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Route lookup + decision
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
def match_route(
|
def match_route(
|
||||||
routes: typing.Sequence[Route],
|
routes: typing.Sequence[Route],
|
||||||
request_host: str,
|
request_host: str,
|
||||||
) -> Route | None:
|
) -> Route | None:
|
||||||
"""Return the first route whose `host` matches `request_host`
|
|
||||||
exactly (case-insensitive). DNS names are case-insensitive.
|
|
||||||
|
|
||||||
Wildcard hosts (`*.foo.com`) are NOT supported — they caused
|
|
||||||
too many edge cases (apex match? cert validation? pipelock
|
|
||||||
mirror mismatch?) for too little payoff. Operators that need
|
|
||||||
multiple subdomains declare them individually (or one common
|
|
||||||
parent host as a bare-pass route)."""
|
|
||||||
target = request_host.lower()
|
target = request_host.lower()
|
||||||
for r in routes:
|
for r in routes:
|
||||||
if r.host.lower() == target:
|
if r.host.lower() == target:
|
||||||
@@ -201,24 +431,9 @@ def decide(
|
|||||||
request_host: str,
|
request_host: str,
|
||||||
request_path: str,
|
request_path: str,
|
||||||
environ: typing.Mapping[str, str],
|
environ: typing.Mapping[str, str],
|
||||||
|
request_method: str = "GET",
|
||||||
|
request_headers: typing.Mapping[str, str] | None = None,
|
||||||
) -> Decision:
|
) -> Decision:
|
||||||
"""Pure decision: given a route table + request host + path + env,
|
|
||||||
return what the addon should do with the request.
|
|
||||||
|
|
||||||
- No matching route → BLOCK. The route table is the bottle's
|
|
||||||
egress allowlist; defense-in-depth complements pipelock's
|
|
||||||
hostname gate on the downstream leg. A bottle that wants a
|
|
||||||
host reachable from the agent must declare a route for it
|
|
||||||
(bare-pass route — no `auth`, no `path_allowlist` — is fine
|
|
||||||
for hosts that just need passthrough).
|
|
||||||
- Matching route with `path_allowlist` set, request path doesn't
|
|
||||||
start with any of the allowed prefixes → block with a clear
|
|
||||||
reason.
|
|
||||||
- Matching route with an auth pair → forward + inject
|
|
||||||
Authorization. Token comes from `environ[route.token_env]`;
|
|
||||||
missing/empty values block (route declared auth but the secret
|
|
||||||
isn't here — operator misconfig).
|
|
||||||
"""
|
|
||||||
route = match_route(routes, request_host)
|
route = match_route(routes, request_host)
|
||||||
if route is None:
|
if route is None:
|
||||||
return Decision(
|
return Decision(
|
||||||
@@ -230,15 +445,15 @@ def decide(
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
if route.path_allowlist:
|
if not evaluate_matches(route, request_path, request_method, request_headers):
|
||||||
if not any(request_path.startswith(p) for p in route.path_allowlist):
|
return Decision(
|
||||||
return Decision(
|
action="block",
|
||||||
action="block",
|
reason=(
|
||||||
reason=(
|
f"egress: request {request_method} {request_path!r} "
|
||||||
f"egress: path {request_path!r} not in "
|
f"does not match any entry in matches for "
|
||||||
f"path_allowlist for {route.host!r}"
|
f"{route.host!r}"
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
if route.auth_scheme and route.token_env:
|
if route.auth_scheme and route.token_env:
|
||||||
token = environ.get(route.token_env, "")
|
token = environ.get(route.token_env, "")
|
||||||
@@ -258,12 +473,80 @@ def decide(
|
|||||||
return Decision(action="forward")
|
return Decision(action="forward")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# DLP scan dispatch (PRD 0053)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _detector_enabled(
|
||||||
|
configured: tuple[str, ...] | None,
|
||||||
|
name: str,
|
||||||
|
) -> bool:
|
||||||
|
"""Check if a named detector is enabled for a route direction.
|
||||||
|
None means all enabled; empty tuple means all disabled."""
|
||||||
|
if configured is None:
|
||||||
|
return True
|
||||||
|
return name in configured
|
||||||
|
|
||||||
|
|
||||||
|
def scan_outbound(
|
||||||
|
route: Route,
|
||||||
|
body: str | bytes,
|
||||||
|
environ: typing.Mapping[str, str],
|
||||||
|
) -> ScanResult | None:
|
||||||
|
# Lazy import to avoid circular deps and keep dlp_detectors optional
|
||||||
|
# at import time (the sidecar copies it flat alongside this file).
|
||||||
|
try:
|
||||||
|
from dlp_detectors import scan_token_patterns, scan_known_secrets # type: ignore[import-not-found]
|
||||||
|
except ImportError: # pragma: no cover - host-side path
|
||||||
|
from .dlp_detectors import scan_token_patterns, scan_known_secrets # type: ignore[import-not-found]
|
||||||
|
|
||||||
|
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
|
||||||
|
|
||||||
|
if _detector_enabled(route.outbound_detectors, "token_patterns"):
|
||||||
|
result = scan_token_patterns(text)
|
||||||
|
if result is not None:
|
||||||
|
return result
|
||||||
|
|
||||||
|
if _detector_enabled(route.outbound_detectors, "known_secrets"):
|
||||||
|
result = scan_known_secrets(text, env=environ)
|
||||||
|
if result is not None:
|
||||||
|
return result
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def scan_inbound(
|
||||||
|
route: Route,
|
||||||
|
body: str | bytes,
|
||||||
|
) -> ScanResult | None:
|
||||||
|
try:
|
||||||
|
from dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
|
||||||
|
except ImportError: # pragma: no cover - host-side path
|
||||||
|
from .dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
|
||||||
|
|
||||||
|
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
|
||||||
|
|
||||||
|
if _detector_enabled(route.inbound_detectors, "naive_injection_detection"):
|
||||||
|
result = scan_naive_injection(text)
|
||||||
|
if result is not None:
|
||||||
|
return result
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"Decision",
|
"Decision",
|
||||||
|
"HeaderMatch",
|
||||||
|
"MatchEntry",
|
||||||
|
"PathMatch",
|
||||||
"Route",
|
"Route",
|
||||||
|
"ScanResult",
|
||||||
"decide",
|
"decide",
|
||||||
|
"evaluate_matches",
|
||||||
"is_git_push_request",
|
"is_git_push_request",
|
||||||
"load_routes",
|
"load_routes",
|
||||||
"match_route",
|
"match_route",
|
||||||
"parse_routes",
|
"parse_routes",
|
||||||
|
"scan_inbound",
|
||||||
|
"scan_outbound",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -6,15 +6,15 @@
|
|||||||
# call it as a normal child. Behavior is unchanged:
|
# call it as a normal child. Behavior is unchanged:
|
||||||
#
|
#
|
||||||
# * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch
|
# * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch
|
||||||
# to `--mode upstream:URL` to forward all post-MITM traffic
|
# to `--mode upstream:URL` to chain through an upstream proxy.
|
||||||
# through pipelock. mitmproxy does NOT honor HTTPS_PROXY on
|
# mitmproxy does NOT honor HTTPS_PROXY on its outbound side,
|
||||||
# its outbound side, so the upstream wiring has to be the
|
# so the upstream wiring has to be the mitmproxy mode flag,
|
||||||
# mitmproxy mode flag, not env.
|
# not env.
|
||||||
# * Upstream trust: when EGRESS_UPSTREAM_CA is set, build a
|
# * Upstream trust: when EGRESS_UPSTREAM_CA is set, build a
|
||||||
# combined trust bundle (system roots + pipelock CA) and point
|
# combined trust bundle (system roots + upstream CA) and point
|
||||||
# mitmproxy at it. The option REPLACES mitmproxy's default
|
# mitmproxy at it. The option REPLACES mitmproxy's default
|
||||||
# trust store, so passing pipelock's CA alone would break
|
# trust store, so passing the upstream CA alone would break
|
||||||
# route-configured pipelock passthrough hosts.
|
# non-chained hosts.
|
||||||
# * `-s /app/egress_addon.py` loads the addon that reads
|
# * `-s /app/egress_addon.py` loads the addon that reads
|
||||||
# /etc/egress/routes.yaml.
|
# /etc/egress/routes.yaml.
|
||||||
|
|
||||||
@@ -38,11 +38,7 @@ fi
|
|||||||
|
|
||||||
# Bind address. Docker backend wants `0.0.0.0` (agent dials egress
|
# Bind address. Docker backend wants `0.0.0.0` (agent dials egress
|
||||||
# directly via the docker network alias). Smolmachines backend
|
# directly via the docker network alias). Smolmachines backend
|
||||||
# wants `127.0.0.1` because the agent dials pipelock — not egress
|
# uses EGRESS_LISTEN_HOST when a non-default binding is needed.
|
||||||
# — and egress is pipelock's localhost-only upstream inside the
|
|
||||||
# bundle. TSI's IP-only allowlist would otherwise let the agent
|
|
||||||
# reach `<bundle-ip>:9099` and bypass pipelock's DLP; binding
|
|
||||||
# 127.0.0.1 inside the bundle closes that gap (PRD 0023 chunk 3).
|
|
||||||
LISTEN_HOST_FLAG=""
|
LISTEN_HOST_FLAG=""
|
||||||
if [ -n "$EGRESS_LISTEN_HOST" ]; then
|
if [ -n "$EGRESS_LISTEN_HOST" ]; then
|
||||||
LISTEN_HOST_FLAG="--listen-host $EGRESS_LISTEN_HOST"
|
LISTEN_HOST_FLAG="--listen-host $EGRESS_LISTEN_HOST"
|
||||||
@@ -56,13 +52,10 @@ if [ -n "$EGRESS_UPSTREAM_CA" ] && [ -f "$EGRESS_UPSTREAM_CA" ]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# Scope the proxy env to this process tree only. In the bundle
|
# Scope the proxy env to this process tree only. In the bundle
|
||||||
# image (PRD 0024) the four daemons share one container — setting
|
# image (PRD 0024) multiple daemons share one container — setting
|
||||||
# HTTPS_PROXY at the container level would route git-gate's git
|
# HTTPS_PROXY at the container level would route git-gate's git
|
||||||
# pushes through pipelock, which is wrong (pipelock doesn't proxy
|
# pushes through an upstream proxy unintentionally. Setting them
|
||||||
# SSH and would block public git repos). Setting them here means
|
# here means only mitmdump's subprocess inherits them.
|
||||||
# only mitmdump's subprocess inherits them. In the legacy
|
|
||||||
# four-sidecar setup these env vars are also set in compose; here
|
|
||||||
# they're additionally defensive.
|
|
||||||
if [ -n "$EGRESS_UPSTREAM_PROXY" ]; then
|
if [ -n "$EGRESS_UPSTREAM_PROXY" ]; then
|
||||||
export HTTPS_PROXY="$EGRESS_UPSTREAM_PROXY"
|
export HTTPS_PROXY="$EGRESS_UPSTREAM_PROXY"
|
||||||
export HTTP_PROXY="$EGRESS_UPSTREAM_PROXY"
|
export HTTP_PROXY="$EGRESS_UPSTREAM_PROXY"
|
||||||
|
|||||||
+1
-1
@@ -89,7 +89,7 @@ def _read_secret_silent(name: str, prompt_body: str) -> str:
|
|||||||
if not (sys.stdin.isatty() or sys.stderr.isatty()):
|
if not (sys.stdin.isatty() or sys.stderr.isatty()):
|
||||||
# Fall back to /dev/tty so this still works when stdin is a pipe.
|
# Fall back to /dev/tty so this still works when stdin is a pipe.
|
||||||
try:
|
try:
|
||||||
tty = open("/dev/tty", "r+")
|
tty = open("/dev/tty", "r+", encoding="utf-8")
|
||||||
except OSError:
|
except OSError:
|
||||||
die(
|
die(
|
||||||
f"cannot prompt for secret '{name}': no tty available. "
|
f"cannot prompt for secret '{name}': no tty available. "
|
||||||
|
|||||||
+92
-4
@@ -15,9 +15,9 @@ a bare repo on the gate; `git daemon` serves the bare repos over
|
|||||||
|
|
||||||
The agent never sees the upstream credential under either path.
|
The agent never sees the upstream credential under either path.
|
||||||
|
|
||||||
Why a third sidecar (not folded into pipelock or ssh-gate): the
|
Why a separate sidecar (not folded into egress or ssh-gate): the
|
||||||
gate is the only one of the three that holds upstream push
|
gate is the only one of the three that holds upstream push
|
||||||
credentials. Mixing it with pipelock would put push creds in the
|
credentials. Mixing it with egress would put push creds in the
|
||||||
same blast radius as internet-facing TLS interception; mixing it
|
same blast radius as internet-facing TLS interception; mixing it
|
||||||
with ssh-gate would force ssh-gate above L4 and into git-protocol
|
with ssh-gate would force ssh-gate above L4 and into git-protocol
|
||||||
land. See `docs/prds/0008-git-gate.md`.
|
land. See `docs/prds/0008-git-gate.md`.
|
||||||
@@ -29,11 +29,14 @@ backend-specific and lives on concrete subclasses (see
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
import os
|
||||||
import shlex
|
import shlex
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .log import info
|
||||||
from .manifest import Bottle, GitEntry
|
from .manifest import Bottle, GitEntry
|
||||||
|
|
||||||
|
|
||||||
@@ -357,6 +360,80 @@ exit 0
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def _provision_dynamic_key(
|
||||||
|
entry: GitEntry,
|
||||||
|
slug: str,
|
||||||
|
stage_dir: Path,
|
||||||
|
) -> str:
|
||||||
|
"""Generate a fresh ed25519 keypair, register the public half with
|
||||||
|
the forge, and persist the private key + key ID under `stage_dir`.
|
||||||
|
|
||||||
|
Returns the host-side path to the private key file so the caller
|
||||||
|
can inject it into the GitGateUpstream as `identity_file`."""
|
||||||
|
from .deploy_key_provisioner import get_provisioner
|
||||||
|
pk = entry.ProvisionedKey
|
||||||
|
assert pk is not None
|
||||||
|
token = os.environ.get(pk.token_env)
|
||||||
|
if token is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
||||||
|
f" = {pk.token_env!r}: env var is not set"
|
||||||
|
)
|
||||||
|
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||||
|
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||||
|
|
||||||
|
owner_repo = entry.UpstreamPath
|
||||||
|
if owner_repo.endswith(".git"):
|
||||||
|
owner_repo = owner_repo[:-4]
|
||||||
|
title = f"bot-bottle:{slug}:{entry.Name}"
|
||||||
|
|
||||||
|
info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]")
|
||||||
|
key_id, private_key_bytes = provisioner.create(owner_repo, title)
|
||||||
|
|
||||||
|
key_file = stage_dir / f"{entry.Name}-key"
|
||||||
|
key_file.write_bytes(private_key_bytes)
|
||||||
|
key_file.chmod(0o600)
|
||||||
|
|
||||||
|
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
||||||
|
id_file.write_text(key_id)
|
||||||
|
id_file.chmod(0o600)
|
||||||
|
|
||||||
|
info(f"provisioned deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||||
|
return str(key_file)
|
||||||
|
|
||||||
|
|
||||||
|
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
|
||||||
|
"""Revoke all deploy keys provisioned for `bottle` during prepare.
|
||||||
|
|
||||||
|
Called at teardown after containers stop. Raises if any revocation
|
||||||
|
fails — a stranded key is a security concern that the operator must
|
||||||
|
address manually."""
|
||||||
|
from .deploy_key_provisioner import get_provisioner
|
||||||
|
for entry in bottle.git:
|
||||||
|
if entry.ProvisionedKey is None:
|
||||||
|
continue
|
||||||
|
pk = entry.ProvisionedKey
|
||||||
|
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
||||||
|
if not id_file.exists():
|
||||||
|
continue
|
||||||
|
key_id = id_file.read_text().strip()
|
||||||
|
token = os.environ.get(pk.token_env)
|
||||||
|
if token is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
||||||
|
f" = {pk.token_env!r}: env var is not set;"
|
||||||
|
f" cannot revoke deploy key {key_id}"
|
||||||
|
)
|
||||||
|
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||||
|
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||||
|
owner_repo = entry.UpstreamPath
|
||||||
|
if owner_repo.endswith(".git"):
|
||||||
|
owner_repo = owner_repo[:-4]
|
||||||
|
info(f"revoking deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||||
|
provisioner.delete(owner_repo, key_id)
|
||||||
|
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||||
|
|
||||||
|
|
||||||
class GitGate(ABC):
|
class GitGate(ABC):
|
||||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||||
(upstream lift + entrypoint/hook render); the sidecar's
|
(upstream lift + entrypoint/hook render); the sidecar's
|
||||||
@@ -368,10 +445,21 @@ class GitGate(ABC):
|
|||||||
entrypoint, pre-receive hook, and access-hook scripts (mode
|
entrypoint, pre-receive hook, and access-hook scripts (mode
|
||||||
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
||||||
|
|
||||||
|
For `provisioned_key` entries, also generates and registers
|
||||||
|
a fresh deploy key via the forge API and writes the private key
|
||||||
|
+ key ID to `stage_dir`.
|
||||||
|
|
||||||
Returned plan is incomplete: the launch step must fill
|
Returned plan is incomplete: the launch step must fill
|
||||||
`internal_network` / `egress_network` via `dataclasses.replace`
|
`internal_network` / `egress_network` via `dataclasses.replace`
|
||||||
before passing the plan to `.start`."""
|
before passing the plan to `.start`."""
|
||||||
upstreams = git_gate_upstreams_for_bottle(bottle)
|
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
|
||||||
|
for i, entry in enumerate(bottle.git):
|
||||||
|
if entry.ProvisionedKey is not None:
|
||||||
|
key_file = _provision_dynamic_key(entry, slug, stage_dir)
|
||||||
|
upstreams_list[i] = dataclasses.replace(
|
||||||
|
upstreams_list[i], identity_file=key_file
|
||||||
|
)
|
||||||
|
upstreams = tuple(upstreams_list)
|
||||||
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
||||||
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
||||||
entrypoint.chmod(0o600)
|
entrypoint.chmod(0o600)
|
||||||
|
|||||||
@@ -78,8 +78,8 @@ class GitHttpHandler(BaseHTTPRequestHandler):
|
|||||||
"REMOTE_ADDR": self.client_address[0],
|
"REMOTE_ADDR": self.client_address[0],
|
||||||
"REMOTE_PORT": str(self.client_address[1]),
|
"REMOTE_PORT": str(self.client_address[1]),
|
||||||
"REMOTE_USER": "",
|
"REMOTE_USER": "",
|
||||||
"SERVER_NAME": self.server.server_name,
|
"SERVER_NAME": self.server.server_name, # type: ignore
|
||||||
"SERVER_PORT": str(self.server.server_port),
|
"SERVER_PORT": str(self.server.server_port), # type: ignore
|
||||||
"SERVER_PROTOCOL": self.request_version,
|
"SERVER_PROTOCOL": self.request_version,
|
||||||
})
|
})
|
||||||
for header, variable in (
|
for header, variable in (
|
||||||
@@ -157,8 +157,8 @@ class GitHttpHandler(BaseHTTPRequestHandler):
|
|||||||
self.end_headers()
|
self.end_headers()
|
||||||
self.wfile.write(body)
|
self.wfile.write(body)
|
||||||
|
|
||||||
def log_message(self, fmt: str, *args: object) -> None:
|
def log_message(self, format: str, *args: object) -> None: # type: ignore # noqa: A002
|
||||||
sys.stdout.write(fmt % args + "\n")
|
sys.stdout.write(format % args + "\n")
|
||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+11
-13
@@ -18,8 +18,7 @@ Bottle schema (frontmatter):
|
|||||||
user: { name: <str>, email: <str> } # optional
|
user: { name: <str>, email: <str> } # optional
|
||||||
repos: { <name>: <git-gate-entry>, ... } # optional
|
repos: { <name>: <git-gate-entry>, ... } # optional
|
||||||
egress: { routes: [ <egress-route>, ... ] }
|
egress: { routes: [ <egress-route>, ... ] }
|
||||||
# route keys: host, path_allowlist, auth, role, pipelock
|
# route keys: host, matches, auth, role, dlp
|
||||||
# pipelock: { tls_passthrough: <bool>, ssrf_ip_allowlist: [<cidr>, ...] }
|
|
||||||
supervise: <bool> # optional
|
supervise: <bool> # optional
|
||||||
|
|
||||||
Agent schema (frontmatter):
|
Agent schema (frontmatter):
|
||||||
@@ -56,8 +55,6 @@ from .manifest_egress import (
|
|||||||
EGRESS_AUTH_SCHEMES,
|
EGRESS_AUTH_SCHEMES,
|
||||||
EgressConfig,
|
EgressConfig,
|
||||||
EgressRoute,
|
EgressRoute,
|
||||||
PipelockRoutePolicy,
|
|
||||||
validate_egress_routes,
|
|
||||||
)
|
)
|
||||||
from .manifest_git import GitEntry, GitUser, parse_git_gate_config
|
from .manifest_git import GitEntry, GitUser, parse_git_gate_config
|
||||||
from .manifest_schema import BOTTLE_KEYS
|
from .manifest_schema import BOTTLE_KEYS
|
||||||
@@ -69,7 +66,6 @@ __all__ = [
|
|||||||
"GitUser",
|
"GitUser",
|
||||||
"AgentProvider",
|
"AgentProvider",
|
||||||
"EGRESS_AUTH_SCHEMES",
|
"EGRESS_AUTH_SCHEMES",
|
||||||
"PipelockRoutePolicy",
|
|
||||||
"EgressRoute",
|
"EgressRoute",
|
||||||
"EgressConfig",
|
"EgressConfig",
|
||||||
"Agent",
|
"Agent",
|
||||||
@@ -101,12 +97,11 @@ class Bottle:
|
|||||||
git_user: GitUser = field(default_factory=GitUser)
|
git_user: GitUser = field(default_factory=GitUser)
|
||||||
egress: EgressConfig = field(default_factory=EgressConfig)
|
egress: EgressConfig = field(default_factory=EgressConfig)
|
||||||
# Opt-in per-bottle stuck-recovery sidecar (PRD 0013). When true,
|
# Opt-in per-bottle stuck-recovery sidecar (PRD 0013). When true,
|
||||||
# the launch step brings up a supervise sidecar that exposes three
|
# the launch step brings up a supervise sidecar that exposes MCP
|
||||||
# MCP tools to the agent (cred-proxy-block, pipelock-block,
|
# tools to the agent (egress-block, capability-block) plus mounts
|
||||||
# capability-block; the cred-proxy-block tool is renamed and
|
# the current-config dir read-only into the agent at
|
||||||
# retargeted at egress in PRD 0017 chunk 3) plus mounts the
|
# /etc/bot-bottle/current-config. False (the default) skips the
|
||||||
# current-config dir read-only into the agent at /etc/bot-bottle/
|
# sidecar and mount.
|
||||||
# current-config. False (the default) skips the sidecar and mount.
|
|
||||||
supervise: bool = False
|
supervise: bool = False
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -323,8 +318,11 @@ class Manifest:
|
|||||||
return
|
return
|
||||||
available = ", ".join(self.agents.keys())
|
available = ", ".join(self.agents.keys())
|
||||||
if available:
|
if available:
|
||||||
raise ManifestError(f"agent '{name}' not defined in bot-bottle.json. Available: {available}")
|
msg = f"agent '{name}' not defined in bot-bottle.json. Available: {available}"
|
||||||
raise ManifestError(f"agent '{name}' not defined in bot-bottle.json (manifest is empty).")
|
raise ManifestError(msg)
|
||||||
|
raise ManifestError(
|
||||||
|
f"agent '{name}' not defined in bot-bottle.json (manifest is empty)."
|
||||||
|
)
|
||||||
|
|
||||||
def has_bottle(self, name: str) -> bool:
|
def has_bottle(self, name: str) -> bool:
|
||||||
return name in self.bottles
|
return name in self.bottles
|
||||||
|
|||||||
@@ -49,11 +49,6 @@ class AgentProvider:
|
|||||||
f"bottle '{bottle_name}' agent_provider.template must be a "
|
f"bottle '{bottle_name}' agent_provider.template must be a "
|
||||||
f"non-empty string"
|
f"non-empty string"
|
||||||
)
|
)
|
||||||
if template not in PROVIDER_TEMPLATES:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.template {template!r} "
|
|
||||||
f"is not one of {', '.join(sorted(PROVIDER_TEMPLATES))}"
|
|
||||||
)
|
|
||||||
dockerfile = d.get("dockerfile", "")
|
dockerfile = d.get("dockerfile", "")
|
||||||
if not isinstance(dockerfile, str):
|
if not isinstance(dockerfile, str):
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
@@ -66,6 +61,12 @@ class AgentProvider:
|
|||||||
f"bottle '{bottle_name}' agent_provider.auth_token must be a "
|
f"bottle '{bottle_name}' agent_provider.auth_token must be a "
|
||||||
f"string (was {type(auth_token).__name__})"
|
f"string (was {type(auth_token).__name__})"
|
||||||
)
|
)
|
||||||
|
if auth_token and template not in PROVIDER_TEMPLATES:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.auth_token is only "
|
||||||
|
f"supported for built-in templates "
|
||||||
|
f"({', '.join(sorted(PROVIDER_TEMPLATES))})"
|
||||||
|
)
|
||||||
if auth_token and template != "claude":
|
if auth_token and template != "claude":
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' agent_provider.auth_token is only "
|
f"bottle '{bottle_name}' agent_provider.auth_token is only "
|
||||||
@@ -77,6 +78,12 @@ class AgentProvider:
|
|||||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||||
f"must be a boolean (was {type(forward_host_credentials).__name__})"
|
f"must be a boolean (was {type(forward_host_credentials).__name__})"
|
||||||
)
|
)
|
||||||
|
if forward_host_credentials and template not in PROVIDER_TEMPLATES:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||||
|
f"is only supported for built-in templates "
|
||||||
|
f"({', '.join(sorted(PROVIDER_TEMPLATES))})"
|
||||||
|
)
|
||||||
if forward_host_credentials and template != "codex":
|
if forward_host_credentials and template != "codex":
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||||
@@ -114,7 +121,10 @@ class Agent:
|
|||||||
|
|
||||||
bottle = d.get("bottle")
|
bottle = d.get("bottle")
|
||||||
if not isinstance(bottle, str) or not bottle:
|
if not isinstance(bottle, str) or not bottle:
|
||||||
raise ManifestError(f"agent '{name}' must declare a 'bottle' field naming a defined bottle")
|
raise ManifestError(
|
||||||
|
f"agent '{name}' must declare a 'bottle' field naming a "
|
||||||
|
f"defined bottle"
|
||||||
|
)
|
||||||
if bottle not in bottle_names:
|
if bottle not in bottle_names:
|
||||||
available = ", ".join(sorted(bottle_names)) or "(none defined)"
|
available = ", ".join(sorted(bottle_names)) or "(none defined)"
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
@@ -126,7 +136,10 @@ class Agent:
|
|||||||
skills_raw = d.get("skills")
|
skills_raw = d.get("skills")
|
||||||
if skills_raw is not None:
|
if skills_raw is not None:
|
||||||
if not isinstance(skills_raw, list):
|
if not isinstance(skills_raw, list):
|
||||||
raise ManifestError(f"agent '{name}' skills must be an array (was {type(skills_raw).__name__})")
|
raise ManifestError(
|
||||||
|
f"agent '{name}' skills must be an array "
|
||||||
|
f"(was {type(skills_raw).__name__})"
|
||||||
|
)
|
||||||
collected: list[str] = []
|
collected: list[str] = []
|
||||||
skills_list = cast(list[object], skills_raw)
|
skills_list = cast(list[object], skills_raw)
|
||||||
for i, skill in enumerate(skills_list):
|
for i, skill in enumerate(skills_list):
|
||||||
@@ -144,7 +157,10 @@ class Agent:
|
|||||||
elif isinstance(prompt_raw, str):
|
elif isinstance(prompt_raw, str):
|
||||||
prompt = prompt_raw
|
prompt = prompt_raw
|
||||||
else:
|
else:
|
||||||
raise ManifestError(f"agent '{name}' prompt must be a string (was {type(prompt_raw).__name__})")
|
raise ManifestError(
|
||||||
|
f"agent '{name}' prompt must be a string "
|
||||||
|
f"(was {type(prompt_raw).__name__})"
|
||||||
|
)
|
||||||
|
|
||||||
# git-gate: agents may declare only `git-gate.user` (name/email).
|
# git-gate: agents may declare only `git-gate.user` (name/email).
|
||||||
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
||||||
@@ -152,7 +168,7 @@ class Agent:
|
|||||||
git_raw = d.get("git-gate")
|
git_raw = d.get("git-gate")
|
||||||
if git_raw is not None:
|
if git_raw is not None:
|
||||||
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
||||||
for k in gd.keys():
|
for k in gd:
|
||||||
if k != "user":
|
if k != "user":
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"agent '{name}' git-gate.{k} is not allowed at the "
|
f"agent '{name}' git-gate.{k} is not allowed at the "
|
||||||
|
|||||||
+223
-133
@@ -1,33 +1,31 @@
|
|||||||
"""Egress routing manifest dataclasses and helpers."""
|
"""Egress routing manifest dataclasses and helpers (PRD 0017, PRD 0053)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import ipaddress
|
import re
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from typing import cast
|
from typing import cast
|
||||||
|
|
||||||
from .manifest_util import ManifestError, as_json_object
|
from .manifest_util import ManifestError, as_json_object
|
||||||
|
|
||||||
|
|
||||||
# Auth schemes for the egress route's optional `auth` block.
|
|
||||||
# Same values cred-proxy accepts today; `token` sidesteps the Gitea
|
|
||||||
# token-not-Bearer quirk (go-gitea/gitea#16734).
|
|
||||||
EGRESS_AUTH_SCHEMES = ("Bearer", "token")
|
EGRESS_AUTH_SCHEMES = ("Bearer", "token")
|
||||||
|
|
||||||
|
PATH_MATCH_TYPES = ("exact", "prefix", "regex")
|
||||||
|
HEADER_MATCH_TYPES = ("exact", "regex")
|
||||||
|
|
||||||
|
VALID_METHODS = frozenset({
|
||||||
|
"GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "TRACE",
|
||||||
|
"CONNECT",
|
||||||
|
})
|
||||||
|
|
||||||
|
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
|
||||||
|
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
|
||||||
|
|
||||||
|
|
||||||
def validate_egress_routes(
|
def validate_egress_routes(
|
||||||
bottle_name: str,
|
bottle_name: str,
|
||||||
routes: tuple[EgressRoute, ...],
|
routes: tuple[EgressRoute, ...],
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Cross-validation for `bottle.egress.routes`: hosts must be unique.
|
|
||||||
|
|
||||||
The proxy matches by exact-host (v1); duplicate hosts leave the
|
|
||||||
route choice ambiguous so we reject them up front.
|
|
||||||
|
|
||||||
No cross-validation against `bottle.git-gate.repos` is performed.
|
|
||||||
git-gate (SSH push/fetch) and egress (HTTPS) broker different
|
|
||||||
protocols; declaring both for the same host is a legitimate dev
|
|
||||||
setup."""
|
|
||||||
seen_hosts: dict[str, None] = {}
|
seen_hosts: dict[str, None] = {}
|
||||||
for r in routes:
|
for r in routes:
|
||||||
key = r.Host.lower()
|
key = r.Host.lower()
|
||||||
@@ -40,99 +38,34 @@ def validate_egress_routes(
|
|||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class PipelockRoutePolicy:
|
class PathMatch:
|
||||||
"""Per-route pipelock policy overrides.
|
Type: str = "prefix"
|
||||||
|
Value: str = ""
|
||||||
|
|
||||||
`TlsPassthrough` adds the route host to pipelock's
|
|
||||||
`tls_interception.passthrough_domains`, so pipelock still enforces
|
|
||||||
the hostname allowlist but does not MITM/decrypt request bodies or
|
|
||||||
headers for that host.
|
|
||||||
|
|
||||||
`SsrfIpAllowlist` adds explicit IPs/CIDRs to pipelock's SSRF
|
@dataclass(frozen=True)
|
||||||
allowlist for private/internal destinations behind this route.
|
class HeaderMatch:
|
||||||
"""
|
Name: str = ""
|
||||||
|
Value: str = ""
|
||||||
|
Type: str = "exact"
|
||||||
|
|
||||||
TlsPassthrough: bool = False
|
|
||||||
SsrfIpAllowlist: tuple[str, ...] = ()
|
|
||||||
|
|
||||||
@classmethod
|
@dataclass(frozen=True)
|
||||||
def from_dict(
|
class MatchEntry:
|
||||||
cls, bottle_name: str, idx: int, raw: object,
|
Paths: tuple[PathMatch, ...] = ()
|
||||||
) -> "PipelockRoutePolicy":
|
Methods: tuple[str, ...] = ()
|
||||||
label = f"bottle '{bottle_name}' egress.routes[{idx}] pipelock"
|
Headers: tuple[HeaderMatch, ...] = ()
|
||||||
d = as_json_object(raw, label)
|
|
||||||
for k in d:
|
|
||||||
if k not in ("tls_passthrough", "ssrf_ip_allowlist"):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} has unknown key {k!r}; "
|
|
||||||
f"only 'tls_passthrough' and 'ssrf_ip_allowlist' "
|
|
||||||
f"are accepted"
|
|
||||||
)
|
|
||||||
tls_passthrough_raw = d.get("tls_passthrough", False)
|
|
||||||
if not isinstance(tls_passthrough_raw, bool):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.tls_passthrough must be a boolean "
|
|
||||||
f"(was {type(tls_passthrough_raw).__name__})"
|
|
||||||
)
|
|
||||||
ssrf_raw = d.get("ssrf_ip_allowlist", [])
|
|
||||||
if not isinstance(ssrf_raw, list):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.ssrf_ip_allowlist must be an array "
|
|
||||||
f"(was {type(ssrf_raw).__name__})"
|
|
||||||
)
|
|
||||||
ssrf_ip_allowlist: list[str] = []
|
|
||||||
for j, item in enumerate(ssrf_raw):
|
|
||||||
if not isinstance(item, str) or not item:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.ssrf_ip_allowlist[{j}] must be a non-empty "
|
|
||||||
f"string (was {type(item).__name__})"
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
ipaddress.ip_network(item, strict=False)
|
|
||||||
except ValueError as e:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.ssrf_ip_allowlist[{j}] must be an IP address "
|
|
||||||
f"or CIDR (was {item!r}): {e}"
|
|
||||||
)
|
|
||||||
ssrf_ip_allowlist.append(item)
|
|
||||||
return cls(
|
|
||||||
TlsPassthrough=tls_passthrough_raw,
|
|
||||||
SsrfIpAllowlist=tuple(ssrf_ip_allowlist),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class EgressRoute:
|
class EgressRoute:
|
||||||
"""One route on the per-bottle egress sidecar (PRD 0017).
|
|
||||||
|
|
||||||
`Host` matches the request's hostname (case-insensitive). The
|
|
||||||
optional `PathAllowlist` constrains the URL path to a set of
|
|
||||||
prefixes; empty tuple means no path-level filtering. The optional
|
|
||||||
`AuthScheme` / `TokenRef` pair drives credential injection:
|
|
||||||
when set, the proxy strips any inbound Authorization and injects
|
|
||||||
`<AuthScheme> <value-of-host-env-named-by-TokenRef>`. When the
|
|
||||||
manifest's `auth` block is omitted both fields are empty strings —
|
|
||||||
no Authorization is written, no token forwarded.
|
|
||||||
|
|
||||||
`Role` is reserved for future use; all role strings are currently
|
|
||||||
rejected by the validator.
|
|
||||||
|
|
||||||
Validation rules (enforced in `from_dict`):
|
|
||||||
- `host` required, non-empty.
|
|
||||||
- `path_allowlist` optional, list of absolute path prefixes.
|
|
||||||
- `auth` optional. If present, MUST carry both `scheme` and
|
|
||||||
`token_ref` as non-empty strings; an empty `auth: {}` is an
|
|
||||||
error rather than a synonym for "no auth" (omit `auth` for
|
|
||||||
that case).
|
|
||||||
- `role` optional, reserved — any non-empty value is rejected.
|
|
||||||
"""
|
|
||||||
|
|
||||||
Host: str
|
Host: str
|
||||||
PathAllowlist: tuple[str, ...] = ()
|
Matches: tuple[MatchEntry, ...] = ()
|
||||||
AuthScheme: str = ""
|
AuthScheme: str = ""
|
||||||
TokenRef: str = ""
|
TokenRef: str = ""
|
||||||
Role: tuple[str, ...] = ()
|
Role: tuple[str, ...] = ()
|
||||||
Pipelock: PipelockRoutePolicy = field(default_factory=PipelockRoutePolicy)
|
OutboundDetectors: tuple[str, ...] | None = None
|
||||||
|
InboundDetectors: tuple[str, ...] | None = None
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
|
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
|
||||||
@@ -142,30 +75,24 @@ class EgressRoute:
|
|||||||
if not isinstance(host, str) or not host:
|
if not isinstance(host, str) or not host:
|
||||||
raise ManifestError(f"{label} missing required string field 'host'")
|
raise ManifestError(f"{label} missing required string field 'host'")
|
||||||
|
|
||||||
path_allow_raw = d.get("path_allowlist")
|
# --- matches ---
|
||||||
prefixes: tuple[str, ...] = ()
|
matches: tuple[MatchEntry, ...] = ()
|
||||||
if path_allow_raw is not None:
|
matches_raw = d.get("matches")
|
||||||
if not isinstance(path_allow_raw, list):
|
if matches_raw is not None:
|
||||||
|
if not isinstance(matches_raw, list):
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"{label} path_allowlist must be an array "
|
f"{label} matches must be an array "
|
||||||
f"(was {type(path_allow_raw).__name__})"
|
f"(was {type(matches_raw).__name__})"
|
||||||
)
|
)
|
||||||
path_list = cast(list[object], path_allow_raw)
|
matches_list = cast(list[object], matches_raw)
|
||||||
collected: list[str] = []
|
entries: list[MatchEntry] = []
|
||||||
for j, p in enumerate(path_list):
|
for k, entry_raw in enumerate(matches_list):
|
||||||
if not isinstance(p, str):
|
entries.append(
|
||||||
raise ManifestError(
|
_parse_match_entry(label, k, entry_raw)
|
||||||
f"{label} path_allowlist[{j}] must be a string "
|
)
|
||||||
f"(was {type(p).__name__})"
|
matches = tuple(entries)
|
||||||
)
|
|
||||||
if not p.startswith("/"):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} path_allowlist[{j}] {p!r} must be an "
|
|
||||||
f"absolute path prefix starting with '/'"
|
|
||||||
)
|
|
||||||
collected.append(p)
|
|
||||||
prefixes = tuple(collected)
|
|
||||||
|
|
||||||
|
# --- auth ---
|
||||||
auth_scheme = ""
|
auth_scheme = ""
|
||||||
token_ref = ""
|
token_ref = ""
|
||||||
if "auth" in d:
|
if "auth" in d:
|
||||||
@@ -203,6 +130,7 @@ class EgressRoute:
|
|||||||
auth_scheme = auth_scheme_raw
|
auth_scheme = auth_scheme_raw
|
||||||
token_ref = token_ref_raw
|
token_ref = token_ref_raw
|
||||||
|
|
||||||
|
# --- role (reserved) ---
|
||||||
role_raw = d.get("role")
|
role_raw = d.get("role")
|
||||||
roles: tuple[str, ...] = ()
|
roles: tuple[str, ...] = ()
|
||||||
if role_raw is None:
|
if role_raw is None:
|
||||||
@@ -214,7 +142,8 @@ class EgressRoute:
|
|||||||
collected_roles: list[str] = []
|
collected_roles: list[str] = []
|
||||||
for r in role_list:
|
for r in role_list:
|
||||||
if not isinstance(r, str):
|
if not isinstance(r, str):
|
||||||
raise ManifestError(f"{label} role items must be strings (got {type(r).__name__})")
|
msg = f"{label} role items must be strings (got {type(r).__name__})"
|
||||||
|
raise ManifestError(msg)
|
||||||
collected_roles.append(r)
|
collected_roles.append(r)
|
||||||
roles = tuple(collected_roles)
|
roles = tuple(collected_roles)
|
||||||
else:
|
else:
|
||||||
@@ -228,36 +157,197 @@ class EgressRoute:
|
|||||||
f"the 'role' field is reserved for future use"
|
f"the 'role' field is reserved for future use"
|
||||||
)
|
)
|
||||||
|
|
||||||
pipelock = (
|
# --- dlp ---
|
||||||
PipelockRoutePolicy.from_dict(bottle_name, idx, d["pipelock"])
|
outbound_detectors: tuple[str, ...] | None = None
|
||||||
if "pipelock" in d
|
inbound_detectors: tuple[str, ...] | None = None
|
||||||
else PipelockRoutePolicy()
|
if "dlp" in d:
|
||||||
)
|
outbound_detectors, inbound_detectors = _parse_dlp_block(
|
||||||
|
label, d.get("dlp"),
|
||||||
|
)
|
||||||
|
|
||||||
for k in d:
|
for k in d:
|
||||||
if k not in ("host", "path_allowlist", "auth", "role", "pipelock"):
|
if k not in ("host", "matches", "auth", "role", "dlp"):
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"{label} has unknown key {k!r}; accepted keys are "
|
f"{label} has unknown key {k!r}; accepted keys are "
|
||||||
f"'host', 'path_allowlist', 'auth', 'role', 'pipelock'"
|
f"'host', 'matches', 'auth', 'role', 'dlp'"
|
||||||
)
|
)
|
||||||
|
|
||||||
return cls(
|
return cls(
|
||||||
Host=host,
|
Host=host,
|
||||||
PathAllowlist=prefixes,
|
Matches=matches,
|
||||||
AuthScheme=auth_scheme,
|
AuthScheme=auth_scheme,
|
||||||
TokenRef=token_ref,
|
TokenRef=token_ref,
|
||||||
Role=roles,
|
Role=roles,
|
||||||
Pipelock=pipelock,
|
OutboundDetectors=outbound_detectors,
|
||||||
|
InboundDetectors=inbound_detectors,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_match_entry(
|
||||||
|
route_label: str, k: int, raw: object,
|
||||||
|
) -> MatchEntry:
|
||||||
|
label = f"{route_label} matches[{k}]"
|
||||||
|
d = as_json_object(raw, label)
|
||||||
|
|
||||||
|
paths: tuple[PathMatch, ...] = ()
|
||||||
|
paths_raw = d.get("paths")
|
||||||
|
if paths_raw is not None:
|
||||||
|
if not isinstance(paths_raw, list):
|
||||||
|
raise ManifestError(f"{label} paths must be an array")
|
||||||
|
paths_list = cast(list[object], paths_raw)
|
||||||
|
parsed_paths: list[PathMatch] = []
|
||||||
|
for j, p_raw in enumerate(paths_list):
|
||||||
|
parsed_paths.append(_parse_path_match(label, j, p_raw))
|
||||||
|
paths = tuple(parsed_paths)
|
||||||
|
|
||||||
|
methods: tuple[str, ...] = ()
|
||||||
|
methods_raw = d.get("methods")
|
||||||
|
if methods_raw is not None:
|
||||||
|
if not isinstance(methods_raw, list):
|
||||||
|
raise ManifestError(f"{label} methods must be an array")
|
||||||
|
methods_list = cast(list[object], methods_raw)
|
||||||
|
normalised: list[str] = []
|
||||||
|
for j, m in enumerate(methods_list):
|
||||||
|
if not isinstance(m, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} methods[{j}] must be a string"
|
||||||
|
)
|
||||||
|
upper = m.upper()
|
||||||
|
if upper not in VALID_METHODS:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} methods[{j}] {m!r} is not a valid HTTP method"
|
||||||
|
)
|
||||||
|
normalised.append(upper)
|
||||||
|
methods = tuple(normalised)
|
||||||
|
|
||||||
|
headers: tuple[HeaderMatch, ...] = ()
|
||||||
|
headers_raw = d.get("headers")
|
||||||
|
if headers_raw is not None:
|
||||||
|
if not isinstance(headers_raw, list):
|
||||||
|
raise ManifestError(f"{label} headers must be an array")
|
||||||
|
headers_list = cast(list[object], headers_raw)
|
||||||
|
parsed_headers: list[HeaderMatch] = []
|
||||||
|
for j, h_raw in enumerate(headers_list):
|
||||||
|
parsed_headers.append(_parse_header_match(label, j, h_raw))
|
||||||
|
headers = tuple(parsed_headers)
|
||||||
|
|
||||||
|
for key in d:
|
||||||
|
if key not in ("paths", "methods", "headers"):
|
||||||
|
raise ManifestError(f"{label} has unknown key {key!r}")
|
||||||
|
|
||||||
|
return MatchEntry(Paths=paths, Methods=methods, Headers=headers)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_path_match(
|
||||||
|
entry_label: str, j: int, raw: object,
|
||||||
|
) -> PathMatch:
|
||||||
|
label = f"{entry_label} paths[{j}]"
|
||||||
|
d = as_json_object(raw, label)
|
||||||
|
ptype = d.get("type", "prefix")
|
||||||
|
if not isinstance(ptype, str) or ptype not in PATH_MATCH_TYPES:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} type must be one of {', '.join(PATH_MATCH_TYPES)} "
|
||||||
|
f"(got {ptype!r})"
|
||||||
|
)
|
||||||
|
value = d.get("value")
|
||||||
|
if not isinstance(value, str) or not value:
|
||||||
|
raise ManifestError(f"{label} value must be a non-empty string")
|
||||||
|
if ptype in ("exact", "prefix") and not value.startswith("/"):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} value {value!r} must start with '/' for type {ptype!r}"
|
||||||
|
)
|
||||||
|
if ptype == "regex":
|
||||||
|
try:
|
||||||
|
re.compile(value)
|
||||||
|
except re.error as e:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} regex {value!r} failed to compile: {e}"
|
||||||
|
) from e
|
||||||
|
for k in d:
|
||||||
|
if k not in ("type", "value"):
|
||||||
|
raise ManifestError(f"{label} has unknown key {k!r}")
|
||||||
|
return PathMatch(Type=ptype, Value=value)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_header_match(
|
||||||
|
entry_label: str, j: int, raw: object,
|
||||||
|
) -> HeaderMatch:
|
||||||
|
label = f"{entry_label} headers[{j}]"
|
||||||
|
d = as_json_object(raw, label)
|
||||||
|
name = d.get("name")
|
||||||
|
if not isinstance(name, str) or not name:
|
||||||
|
raise ManifestError(f"{label} name must be a non-empty string")
|
||||||
|
value = d.get("value")
|
||||||
|
if not isinstance(value, str):
|
||||||
|
raise ManifestError(f"{label} value must be a string")
|
||||||
|
htype = d.get("type", "exact")
|
||||||
|
if not isinstance(htype, str) or htype not in HEADER_MATCH_TYPES:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} type must be one of {', '.join(HEADER_MATCH_TYPES)} "
|
||||||
|
f"(got {htype!r})"
|
||||||
|
)
|
||||||
|
if htype == "regex":
|
||||||
|
try:
|
||||||
|
re.compile(value)
|
||||||
|
except re.error as e:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} regex {value!r} failed to compile: {e}"
|
||||||
|
) from e
|
||||||
|
for k in d:
|
||||||
|
if k not in ("name", "value", "type"):
|
||||||
|
raise ManifestError(f"{label} has unknown key {k!r}")
|
||||||
|
return HeaderMatch(Name=name, Value=value, Type=htype)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_dlp_block(
|
||||||
|
route_label: str,
|
||||||
|
raw: object,
|
||||||
|
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
|
||||||
|
label = f"{route_label} dlp"
|
||||||
|
d = as_json_object(raw, label)
|
||||||
|
|
||||||
|
def _parse_field(
|
||||||
|
field: str,
|
||||||
|
valid_names: frozenset[str],
|
||||||
|
) -> tuple[str, ...] | None:
|
||||||
|
val = d.get(field)
|
||||||
|
if val is None:
|
||||||
|
return None
|
||||||
|
if val is False:
|
||||||
|
return ()
|
||||||
|
if not isinstance(val, list):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} {field} must be false, a list, or omitted"
|
||||||
|
)
|
||||||
|
items = cast(list[object], val)
|
||||||
|
names: list[str] = []
|
||||||
|
for j, item in enumerate(items):
|
||||||
|
if not isinstance(item, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} {field}[{j}] must be a string"
|
||||||
|
)
|
||||||
|
if item not in valid_names:
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} {field}[{j}] {item!r} is not a valid "
|
||||||
|
f"detector; valid: {', '.join(sorted(valid_names))}"
|
||||||
|
)
|
||||||
|
names.append(item)
|
||||||
|
return tuple(names)
|
||||||
|
|
||||||
|
outbound = _parse_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
|
||||||
|
inbound = _parse_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
|
||||||
|
|
||||||
|
for k in d:
|
||||||
|
if k not in ("outbound_detectors", "inbound_detectors"):
|
||||||
|
raise ManifestError(
|
||||||
|
f"{label} has unknown key {k!r}; accepted keys are "
|
||||||
|
f"'outbound_detectors', 'inbound_detectors'"
|
||||||
|
)
|
||||||
|
return outbound, inbound
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class EgressConfig:
|
class EgressConfig:
|
||||||
"""Per-bottle egress configuration. Today this is just the
|
|
||||||
route table; the nesting under `egress:` leaves room for
|
|
||||||
per-bottle proxy settings (port override, log level, etc.) in
|
|
||||||
follow-ups."""
|
|
||||||
|
|
||||||
routes: tuple[EgressRoute, ...] = ()
|
routes: tuple[EgressRoute, ...] = ()
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
|
|||||||
+100
-15
@@ -4,6 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import re
|
import re
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
from .manifest_util import ManifestError, as_json_object
|
from .manifest_util import ManifestError, as_json_object
|
||||||
|
|
||||||
@@ -29,12 +30,18 @@ def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
|
|||||||
raise ManifestError(f"{label} must be an ssh:// URL (was {url!r})")
|
raise ManifestError(f"{label} must be an ssh:// URL (was {url!r})")
|
||||||
rest = url[len("ssh://"):]
|
rest = url[len("ssh://"):]
|
||||||
if "@" not in rest:
|
if "@" not in rest:
|
||||||
raise ManifestError(f"{label} must include a user (e.g. ssh://git@host/path.git); was {url!r}")
|
raise ManifestError(
|
||||||
|
f"{label} must include a user (e.g. ssh://git@host/path.git); "
|
||||||
|
f"was {url!r}"
|
||||||
|
)
|
||||||
user, _, hostpart = rest.partition("@")
|
user, _, hostpart = rest.partition("@")
|
||||||
if not user:
|
if not user:
|
||||||
raise ManifestError(f"{label} user is empty in {url!r}")
|
raise ManifestError(f"{label} user is empty in {url!r}")
|
||||||
if "/" not in hostpart:
|
if "/" not in hostpart:
|
||||||
raise ManifestError(f"{label} must include a path (e.g. ssh://git@host/path.git); was {url!r}")
|
raise ManifestError(
|
||||||
|
f"{label} must include a path (e.g. ssh://git@host/path.git); "
|
||||||
|
f"was {url!r}"
|
||||||
|
)
|
||||||
hostport, _, path = hostpart.partition("/")
|
hostport, _, path = hostpart.partition("/")
|
||||||
if not path:
|
if not path:
|
||||||
raise ManifestError(f"{label} path is empty in {url!r}")
|
raise ManifestError(f"{label} path is empty in {url!r}")
|
||||||
@@ -61,6 +68,24 @@ def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> No
|
|||||||
seen[g.Name] = None
|
seen[g.Name] = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProvisionedKeyConfig:
|
||||||
|
"""Configuration for automatic deploy-key lifecycle management
|
||||||
|
(PRD 0048). Used when a git-gate.repos entry opts out of a
|
||||||
|
static identity file and instead wants a fresh SSH keypair
|
||||||
|
generated at spin-up and revoked at teardown.
|
||||||
|
|
||||||
|
`provider` names the contrib sub-package to load (e.g. `gitea`).
|
||||||
|
`token_env` is the name of a host-side env var carrying the API
|
||||||
|
token; the value is read at provision time, never stored on the
|
||||||
|
plan. `api_url` is the forge's HTTP API root; if empty, it is
|
||||||
|
derived from the upstream URL's host at provision time."""
|
||||||
|
|
||||||
|
provider: str
|
||||||
|
token_env: str
|
||||||
|
api_url: str = ""
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class GitEntry:
|
class GitEntry:
|
||||||
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
|
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
|
||||||
@@ -74,14 +99,15 @@ class GitEntry:
|
|||||||
stashed in the `Upstream*` fields so the git-gate render step
|
stashed in the `Upstream*` fields so the git-gate render step
|
||||||
doesn't have to re-parse.
|
doesn't have to re-parse.
|
||||||
|
|
||||||
Manifest source: `git-gate.repos.<Name>` (PRD 0047). The YAML keys
|
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). Exactly
|
||||||
are `url`, `identity`, and `host_key`; the internal field names are
|
one of `identity` (static key path) or `provisioned_key` (automatic
|
||||||
stable across that rename."""
|
lifecycle) must be present. The internal field names are stable."""
|
||||||
|
|
||||||
Name: str
|
Name: str
|
||||||
Upstream: str
|
Upstream: str
|
||||||
IdentityFile: str
|
IdentityFile: str = ""
|
||||||
KnownHostKey: str = ""
|
KnownHostKey: str = ""
|
||||||
|
ProvisionedKey: Optional[ProvisionedKeyConfig] = None
|
||||||
RemoteKey: str = ""
|
RemoteKey: str = ""
|
||||||
UpstreamUser: str = ""
|
UpstreamUser: str = ""
|
||||||
UpstreamHost: str = ""
|
UpstreamHost: str = ""
|
||||||
@@ -94,8 +120,9 @@ class GitEntry:
|
|||||||
) -> "GitEntry":
|
) -> "GitEntry":
|
||||||
"""Parse one entry from `git-gate.repos.<repo_name>`.
|
"""Parse one entry from `git-gate.repos.<repo_name>`.
|
||||||
|
|
||||||
YAML keys: `url` (required), `identity` (required),
|
YAML keys: `url` (required), exactly one of `identity` or
|
||||||
`host_key` (optional). The repo_name becomes `Name`."""
|
`provisioned_key` (required), `host_key` (optional).
|
||||||
|
The repo_name becomes `Name`."""
|
||||||
if not repo_name:
|
if not repo_name:
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' git-gate.repos has an empty key"
|
f"bottle '{bottle_name}' git-gate.repos has an empty key"
|
||||||
@@ -108,21 +135,44 @@ class GitEntry:
|
|||||||
label = f"git-gate.repos[{repo_name!r}]"
|
label = f"git-gate.repos[{repo_name!r}]"
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
||||||
for k in d:
|
for k in d:
|
||||||
if k not in {"url", "identity", "host_key"}:
|
if k not in {"url", "identity", "provisioned_key", "host_key"}:
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
||||||
f"allowed: url, identity, host_key"
|
f"allowed: url, identity, provisioned_key, host_key"
|
||||||
)
|
)
|
||||||
upstream = d.get("url")
|
upstream = d.get("url")
|
||||||
if not isinstance(upstream, str) or not upstream:
|
if not isinstance(upstream, str) or not upstream:
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
||||||
)
|
)
|
||||||
ident = d.get("identity")
|
|
||||||
if not isinstance(ident, str) or not ident:
|
has_identity = "identity" in d
|
||||||
|
has_provisioned = "provisioned_key" in d
|
||||||
|
if has_identity and has_provisioned:
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' {label} missing required string field 'identity'"
|
f"bottle '{bottle_name}' {label} must set exactly one of "
|
||||||
|
f"'identity' or 'provisioned_key'; got both."
|
||||||
)
|
)
|
||||||
|
if not has_identity and not has_provisioned:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label} must set exactly one of "
|
||||||
|
f"'identity' or 'provisioned_key'; got neither."
|
||||||
|
)
|
||||||
|
|
||||||
|
ident = ""
|
||||||
|
provisioned_key: Optional[ProvisionedKeyConfig] = None
|
||||||
|
if has_identity:
|
||||||
|
raw_ident = d.get("identity")
|
||||||
|
if not isinstance(raw_ident, str) or not raw_ident:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label} 'identity' must be a non-empty string"
|
||||||
|
)
|
||||||
|
ident = raw_ident
|
||||||
|
else:
|
||||||
|
provisioned_key = _parse_provisioned_key_config(
|
||||||
|
bottle_name, label, d["provisioned_key"]
|
||||||
|
)
|
||||||
|
|
||||||
khk = _opt_str(
|
khk = _opt_str(
|
||||||
d.get("host_key"),
|
d.get("host_key"),
|
||||||
f"bottle '{bottle_name}' {label} host_key",
|
f"bottle '{bottle_name}' {label} host_key",
|
||||||
@@ -135,6 +185,7 @@ class GitEntry:
|
|||||||
Upstream=upstream,
|
Upstream=upstream,
|
||||||
IdentityFile=ident,
|
IdentityFile=ident,
|
||||||
KnownHostKey=khk,
|
KnownHostKey=khk,
|
||||||
|
ProvisionedKey=provisioned_key,
|
||||||
RemoteKey=host,
|
RemoteKey=host,
|
||||||
UpstreamUser=user,
|
UpstreamUser=user,
|
||||||
UpstreamHost=host,
|
UpstreamHost=host,
|
||||||
@@ -143,6 +194,40 @@ class GitEntry:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_provisioned_key_config(
|
||||||
|
bottle_name: str, label: str, raw: object
|
||||||
|
) -> ProvisionedKeyConfig:
|
||||||
|
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.provisioned_key")
|
||||||
|
for k in d:
|
||||||
|
if k not in {"provider", "token_env", "api_url"}:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label}.provisioned_key has unknown key {k!r}; "
|
||||||
|
f"allowed: provider, token_env, api_url"
|
||||||
|
)
|
||||||
|
provider = d.get("provider")
|
||||||
|
if not isinstance(provider, str) or not provider:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
||||||
|
f"string field 'provider'"
|
||||||
|
)
|
||||||
|
token_env = d.get("token_env")
|
||||||
|
if not isinstance(token_env, str) or not token_env:
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
||||||
|
f"string field 'token_env'"
|
||||||
|
)
|
||||||
|
api_url_raw = d.get("api_url", "")
|
||||||
|
if not isinstance(api_url_raw, str):
|
||||||
|
raise ManifestError(
|
||||||
|
f"bottle '{bottle_name}' {label}.provisioned_key 'api_url' must be a string"
|
||||||
|
)
|
||||||
|
return ProvisionedKeyConfig(
|
||||||
|
provider=provider,
|
||||||
|
token_env=token_env,
|
||||||
|
api_url=api_url_raw,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class GitUser:
|
class GitUser:
|
||||||
"""Per-bottle `git config --global user.name` / `user.email`
|
"""Per-bottle `git config --global user.name` / `user.email`
|
||||||
@@ -161,7 +246,7 @@ class GitUser:
|
|||||||
@classmethod
|
@classmethod
|
||||||
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
|
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
|
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
|
||||||
for k in d.keys():
|
for k in d:
|
||||||
if k not in {"name", "email"}:
|
if k not in {"name", "email"}:
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' git-gate.user has unknown key {k!r}; "
|
f"bottle '{bottle_name}' git-gate.user has unknown key {k!r}; "
|
||||||
@@ -196,7 +281,7 @@ def parse_git_gate_config(
|
|||||||
raw: object,
|
raw: object,
|
||||||
) -> tuple[tuple[GitEntry, ...], GitUser]:
|
) -> tuple[tuple[GitEntry, ...], GitUser]:
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
||||||
for k in d.keys():
|
for k in d:
|
||||||
if k not in {"user", "repos"}:
|
if k not in {"user", "repos"}:
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
||||||
|
|||||||
@@ -54,9 +54,9 @@ def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
|
|||||||
try:
|
try:
|
||||||
fm, _body = parse_frontmatter(path.read_text())
|
fm, _body = parse_frontmatter(path.read_text())
|
||||||
except OSError as e:
|
except OSError as e:
|
||||||
raise ManifestError(f"could not read {path}: {e}")
|
raise ManifestError(f"could not read {path}: {e}") from e
|
||||||
except YamlSubsetError as e:
|
except YamlSubsetError as e:
|
||||||
raise ManifestError(f"{path}: {e}")
|
raise ManifestError(f"{path}: {e}") from e
|
||||||
validate_bottle_frontmatter_keys(path, fm.keys())
|
validate_bottle_frontmatter_keys(path, fm.keys())
|
||||||
raws[name] = fm
|
raws[name] = fm
|
||||||
return resolve_bottles(raws)
|
return resolve_bottles(raws)
|
||||||
@@ -66,7 +66,7 @@ def load_agents_from_dir(
|
|||||||
agents_dir: Path,
|
agents_dir: Path,
|
||||||
bottle_names: set[str],
|
bottle_names: set[str],
|
||||||
*,
|
*,
|
||||||
source: str,
|
source: str, # noqa: F841 — unused, but required by interface
|
||||||
) -> dict[str, Agent]:
|
) -> dict[str, Agent]:
|
||||||
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
|
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
|
||||||
`{name: Agent}`. The Markdown body becomes the agent's prompt.
|
`{name: Agent}`. The Markdown body becomes the agent's prompt.
|
||||||
@@ -87,9 +87,9 @@ def load_agents_from_dir(
|
|||||||
try:
|
try:
|
||||||
fm, body = parse_frontmatter(path.read_text())
|
fm, body = parse_frontmatter(path.read_text())
|
||||||
except OSError as e:
|
except OSError as e:
|
||||||
raise ManifestError(f"could not read {path}: {e}")
|
raise ManifestError(f"could not read {path}: {e}") from e
|
||||||
except YamlSubsetError as e:
|
except YamlSubsetError as e:
|
||||||
raise ManifestError(f"{path}: {e}")
|
raise ManifestError(f"{path}: {e}") from e
|
||||||
validate_agent_frontmatter_keys(path, fm.keys())
|
validate_agent_frontmatter_keys(path, fm.keys())
|
||||||
# Build the dict Agent.from_dict expects. The body becomes
|
# Build the dict Agent.from_dict expects. The body becomes
|
||||||
# prompt; Claude Code passthrough fields stay in fm and get
|
# prompt; Claude Code passthrough fields stay in fm and get
|
||||||
|
|||||||
@@ -60,11 +60,11 @@ def _validate_frontmatter_keys(
|
|||||||
) -> None:
|
) -> None:
|
||||||
from .manifest_util import ManifestError
|
from .manifest_util import ManifestError
|
||||||
|
|
||||||
key_set = set(keys)
|
key_set = set(keys) # type: ignore
|
||||||
unknown = key_set - allowed_keys
|
unknown = key_set - allowed_keys # type: ignore
|
||||||
if unknown:
|
if unknown:
|
||||||
allowed = ", ".join(sorted(allowed_keys))
|
allowed = ", ".join(sorted(allowed_keys))
|
||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"{kind} file {path}: unknown frontmatter key(s) "
|
f"{kind} file {path}: unknown frontmatter key(s) "
|
||||||
f"{sorted(unknown)}; allowed keys are {allowed}."
|
f"{sorted(unknown)}; allowed keys are {allowed}." # type: ignore
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -1,546 +0,0 @@
|
|||||||
"""Pipelock sidecar lifecycle for the per-agent egress topology.
|
|
||||||
|
|
||||||
Pipelock (https://github.com/luckyPipewrench/pipelock) is an HTTP
|
|
||||||
forward proxy with hostname allowlisting + DLP scanning + URL-entropy
|
|
||||||
checks. One sidecar per agent, attached to the agent's --internal
|
|
||||||
network and a per-agent user-defined egress bridge.
|
|
||||||
|
|
||||||
Post-PRD-0017 topology: the agent's HTTP_PROXY points at egress
|
|
||||||
(not pipelock); egress sets `HTTPS_PROXY=pipelock` on its
|
|
||||||
outbound leg. So pipelock no longer sees the agent's connections
|
|
||||||
directly — it sees the egress → upstream leg, applies the
|
|
||||||
hostname allowlist + DLP body scan there, and forwards to the real
|
|
||||||
upstream.
|
|
||||||
|
|
||||||
Image pin: ghcr.io/luckypipewrench/pipelock@sha256:<digest> for tag 2.3.0.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from .egress import EGRESS_HOSTNAME, EgressRoute, egress_routes_for_bottle
|
|
||||||
from .supervise import SUPERVISE_HOSTNAME
|
|
||||||
from .manifest import Bottle
|
|
||||||
|
|
||||||
# Hosts pipelock should NOT TLS-MITM, even when tls_interception is
|
|
||||||
# enabled. This is now route-owned manifest policy via
|
|
||||||
# `egress.routes[].pipelock.tls_passthrough`; no provider hosts are
|
|
||||||
# injected implicitly.
|
|
||||||
DEFAULT_TLS_PASSTHROUGH: tuple[str, ...] = ()
|
|
||||||
|
|
||||||
|
|
||||||
# In-container paths the rendered pipelock YAML references under
|
|
||||||
# `tls_interception`. The pipelock binary expects the per-bottle CA
|
|
||||||
# cert + key at these exact paths inside its container — independent
|
|
||||||
# of how the daemon is wrapped (own container, sidecar bundle, etc.),
|
|
||||||
# which is why they live in the platform-neutral module.
|
|
||||||
PIPELOCK_CA_CERT_IN_CONTAINER = "/etc/pipelock-ca.pem"
|
|
||||||
PIPELOCK_CA_KEY_IN_CONTAINER = "/etc/pipelock-ca-key.pem"
|
|
||||||
|
|
||||||
|
|
||||||
# Short network alias for pipelock inside the sidecar bundle. The
|
|
||||||
# agent's HTTP_PROXY (when no egress is declared) and any in-bundle
|
|
||||||
# consumer's URL both reference this name.
|
|
||||||
PIPELOCK_HOSTNAME = "pipelock"
|
|
||||||
|
|
||||||
|
|
||||||
# --- Allowlist resolution --------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_effective_allowlist(
|
|
||||||
bottle: Bottle,
|
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
|
||||||
) -> list[str]:
|
|
||||||
"""Hostnames pipelock allows. Sorted for stability.
|
|
||||||
|
|
||||||
Always mirrors `egress_routes_for_bottle(bottle, provider_routes)` —
|
|
||||||
egress is the single allowlist surface, and pipelock's allowlist is
|
|
||||||
the downstream copy for defense-in-depth + DLP body scanning. For
|
|
||||||
bottles without any `egress.routes[]` declared, this is empty except
|
|
||||||
for supervise sidecar traffic when `supervise: true`.
|
|
||||||
|
|
||||||
The supervise sidecar's hostname is auto-added when supervise
|
|
||||||
is enabled (sibling-sidecar traffic that flows through pipelock
|
|
||||||
would otherwise be 403'd). Git upstreams declared in
|
|
||||||
`bottle.git` do NOT contribute here — git traffic flows
|
|
||||||
through git-gate (PRD 0008), not pipelock."""
|
|
||||||
seen: dict[str, None] = {}
|
|
||||||
for r in egress_routes_for_bottle(bottle, provider_routes):
|
|
||||||
if r.host:
|
|
||||||
seen.setdefault(r.host, None)
|
|
||||||
if bottle.supervise:
|
|
||||||
seen.setdefault(SUPERVISE_HOSTNAME, None)
|
|
||||||
return sorted(seen.keys())
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_seed_phrase_detection_enabled(bottle: Bottle) -> bool:
|
|
||||||
"""Whether pipelock's BIP-39 seed-phrase detector stays on.
|
|
||||||
|
|
||||||
LLM conversation bodies legitimately trip the detector — any 12+
|
|
||||||
English words that pass the BIP-39 checksum match — so agents can
|
|
||||||
get blocked on ordinary prompts/responses regardless of provider
|
|
||||||
(Claude, Codex/OpenAI, or future harnesses). We tried two narrower
|
|
||||||
knobs first:
|
|
||||||
|
|
||||||
- `suppress: [{rule, path}]` — pipelock accepts the schema
|
|
||||||
but the entry only silences the alert; the body_dlp block
|
|
||||||
still fires.
|
|
||||||
- `rules.disabled: ["dlp:BIP-39 Seed Phrase"]` — same shape,
|
|
||||||
same outcome: 403 still returned.
|
|
||||||
|
|
||||||
Empirically only `seed_phrase_detection.enabled: false`
|
|
||||||
actually stops the block (verified by sending a 12-word BIP-39
|
|
||||||
body through three pipelock instances). It is a global toggle —
|
|
||||||
no per-path / per-host knob in pipelock 2.3.0 — so we turn off
|
|
||||||
only this detector for every bottle. The rest of pipelock's DLP
|
|
||||||
defaults and request-body/header scanning remain enabled."""
|
|
||||||
del bottle # kept for call-site stability and future policy knobs.
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_effective_tls_passthrough(
|
|
||||||
bottle: Bottle,
|
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
|
||||||
) -> list[str]:
|
|
||||||
"""Hostnames pipelock should pass through (no TLS MITM).
|
|
||||||
|
|
||||||
A manifest route opts in with `pipelock.tls_passthrough: true`
|
|
||||||
(lifted into `EgressRoute.tls_passthrough` in `egress_manifest_routes`).
|
|
||||||
Provider routes that set `tls_passthrough=True` (e.g. Codex credential
|
|
||||||
routes where egress injects the host bearer after the agent boundary)
|
|
||||||
are also included. Both arrive via `egress_routes_for_bottle` — no
|
|
||||||
provider-specific branching needed here.
|
|
||||||
"""
|
|
||||||
seen: dict[str, None] = {host: None for host in DEFAULT_TLS_PASSTHROUGH}
|
|
||||||
for route in egress_routes_for_bottle(bottle, provider_routes):
|
|
||||||
if route.tls_passthrough:
|
|
||||||
seen.setdefault(route.host, None)
|
|
||||||
return sorted(seen.keys())
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_effective_ssrf_ip_allowlist(
|
|
||||||
bottle: Bottle,
|
|
||||||
extra: tuple[str, ...] = (),
|
|
||||||
) -> list[str]:
|
|
||||||
"""IP/CIDR entries that bypass pipelock's SSRF destination guard.
|
|
||||||
|
|
||||||
Launch code can pass backend-owned entries through `extra`, while
|
|
||||||
route-owned entries come from `pipelock.ssrf_ip_allowlist`.
|
|
||||||
"""
|
|
||||||
seen: dict[str, None] = {ip: None for ip in extra}
|
|
||||||
for route in bottle.egress.routes:
|
|
||||||
for ip in route.Pipelock.SsrfIpAllowlist:
|
|
||||||
seen.setdefault(ip, None)
|
|
||||||
return sorted(seen.keys())
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# --- Config build + YAML render --------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_build_config(
|
|
||||||
bottle: Bottle,
|
|
||||||
*,
|
|
||||||
ca_cert_path: str = "",
|
|
||||||
ca_key_path: str = "",
|
|
||||||
ssrf_ip_allowlist: tuple[str, ...] = (),
|
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
|
||||||
) -> dict[str, object]:
|
|
||||||
"""Build the structured pipelock config dict the sidecar will load.
|
|
||||||
|
|
||||||
Deliberately carries no env values, no secrets, no per-agent
|
|
||||||
customization beyond the resolved hostname list. The shape mirrors
|
|
||||||
the YAML pipelock expects on disk; `pipelock_render_yaml` serializes
|
|
||||||
it. Tests assert on this dict; production code renders it.
|
|
||||||
|
|
||||||
`ca_cert_path` / `ca_key_path` are the **in-container** paths the
|
|
||||||
pipelock sidecar will read its CA from at runtime (they're
|
|
||||||
populated into the container at start time via `docker cp`).
|
|
||||||
Pass both or neither: both → emit `tls_interception` block with
|
|
||||||
`enabled: true`; neither → omit the block entirely (pipelock
|
|
||||||
falls back to its built-in default of `enabled: false`). Used
|
|
||||||
by PRD 0006 to turn on pipelock's native TLS interception.
|
|
||||||
|
|
||||||
`ssrf_ip_allowlist` is the list of IPs / CIDRs that bypass
|
|
||||||
pipelock's SSRF guard. Pipelock blocks RFC1918-resolved
|
|
||||||
destinations by default, which would catch sibling-sidecar
|
|
||||||
traffic on the bottle's internal Docker network in 172.x space
|
|
||||||
(e.g. egress → pipelock on the upstream leg). Pass the
|
|
||||||
bottle's internal network CIDR here so internal-network requests
|
|
||||||
pass through pipelock while api_allowlist + body-scanning still
|
|
||||||
apply. Empty by default; omitted from the rendered yaml when
|
|
||||||
empty so pipelock keeps its built-in SSRF defaults."""
|
|
||||||
cfg: dict[str, object] = {
|
|
||||||
"version": 1,
|
|
||||||
"mode": "strict",
|
|
||||||
"enforce": True,
|
|
||||||
"api_allowlist": pipelock_effective_allowlist(bottle, provider_routes),
|
|
||||||
"forward_proxy": {"enabled": True},
|
|
||||||
}
|
|
||||||
if not pipelock_seed_phrase_detection_enabled(bottle):
|
|
||||||
cfg["seed_phrase_detection"] = {"enabled": False}
|
|
||||||
cfg["dlp"] = {"include_defaults": True, "scan_env": True}
|
|
||||||
# Body-scan enforcement is a separate pipelock section (each DLP
|
|
||||||
# "surface" — body, MCP, response — has its own action). Pipelock's
|
|
||||||
# built-in default for request_body_scanning is "warn" (forward
|
|
||||||
# with a log line); bot-bottle hard-codes "block" so a hit
|
|
||||||
# actually stops the request from leaving the egress network.
|
|
||||||
#
|
|
||||||
# `scan_headers: true` + `header_mode: all` extends the scan to
|
|
||||||
# every request header — pipelock's default `header_mode:
|
|
||||||
# sensitive` only checks Authorization / Cookie / X-Api-Key /
|
|
||||||
# X-Token / Proxy-Authorization / X-Goog-Api-Key, which an
|
|
||||||
# agent attempting to exfil could trivially avoid by picking
|
|
||||||
# a non-sensitive header name. "all" closes the gap; pipelock
|
|
||||||
# caps it at the same max_body_bytes the body scan uses.
|
|
||||||
cfg["request_body_scanning"] = {
|
|
||||||
"action": "block",
|
|
||||||
"scan_headers": True,
|
|
||||||
"header_mode": "all",
|
|
||||||
}
|
|
||||||
if ca_cert_path or ca_key_path:
|
|
||||||
if not (ca_cert_path and ca_key_path):
|
|
||||||
raise ValueError(
|
|
||||||
"pipelock_build_config: pass both ca_cert_path and ca_key_path "
|
|
||||||
"to enable tls_interception, or neither to leave it off"
|
|
||||||
)
|
|
||||||
cfg["tls_interception"] = {
|
|
||||||
"enabled": True,
|
|
||||||
"ca_cert": ca_cert_path,
|
|
||||||
"ca_key": ca_key_path,
|
|
||||||
"passthrough_domains": pipelock_effective_tls_passthrough(bottle, provider_routes),
|
|
||||||
}
|
|
||||||
effective_ssrf_ip_allowlist = pipelock_effective_ssrf_ip_allowlist(
|
|
||||||
bottle, ssrf_ip_allowlist,
|
|
||||||
)
|
|
||||||
if effective_ssrf_ip_allowlist:
|
|
||||||
cfg["ssrf"] = {"ip_allowlist": effective_ssrf_ip_allowlist}
|
|
||||||
return cfg
|
|
||||||
|
|
||||||
|
|
||||||
_PIPELOCK_TOP_LEVEL_KEYS = {
|
|
||||||
"version",
|
|
||||||
"mode",
|
|
||||||
"enforce",
|
|
||||||
"api_allowlist",
|
|
||||||
"seed_phrase_detection",
|
|
||||||
"forward_proxy",
|
|
||||||
"dlp",
|
|
||||||
"request_body_scanning",
|
|
||||||
"tls_interception",
|
|
||||||
"ssrf",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _pipelock_render_error(section: str, key: str, expected: str) -> ValueError:
|
|
||||||
return ValueError(
|
|
||||||
f"pipelock_render_yaml: {section}.{key} must be {expected}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _reject_unknown_keys(
|
|
||||||
section: str,
|
|
||||||
obj: dict[str, object],
|
|
||||||
allowed: set[str],
|
|
||||||
) -> None:
|
|
||||||
for key in sorted(set(obj) - allowed):
|
|
||||||
raise ValueError(f"pipelock_render_yaml: {section}.{key} is unsupported")
|
|
||||||
|
|
||||||
|
|
||||||
def _required_dict(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> dict[str, object]:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise _pipelock_render_error(section, key, "a mapping")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_bool(obj: dict[str, object], section: str, key: str) -> bool:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, bool):
|
|
||||||
raise _pipelock_render_error(section, key, "a boolean")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_int(obj: dict[str, object], section: str, key: str) -> int:
|
|
||||||
value = obj.get(key)
|
|
||||||
if isinstance(value, bool) or not isinstance(value, int):
|
|
||||||
raise _pipelock_render_error(section, key, "an integer")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_str(obj: dict[str, object], section: str, key: str) -> str:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, str):
|
|
||||||
raise _pipelock_render_error(section, key, "a string")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_str_list(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> list[str]:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, list) or not all(isinstance(v, str) for v in value):
|
|
||||||
raise _pipelock_render_error(section, key, "a list of strings")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _optional_str_list(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> list[str]:
|
|
||||||
if key not in obj:
|
|
||||||
return []
|
|
||||||
return _required_str_list(obj, section, key)
|
|
||||||
|
|
||||||
|
|
||||||
def _optional_bool(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> bool | None:
|
|
||||||
if key not in obj:
|
|
||||||
return None
|
|
||||||
return _required_bool(obj, section, key)
|
|
||||||
|
|
||||||
|
|
||||||
def _optional_str(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> str | None:
|
|
||||||
if key not in obj:
|
|
||||||
return None
|
|
||||||
return _required_str(obj, section, key)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_pipelock_render_config(cfg: dict[str, object]) -> dict[str, object]:
|
|
||||||
_reject_unknown_keys("config", cfg, _PIPELOCK_TOP_LEVEL_KEYS)
|
|
||||||
normalized: dict[str, object] = {
|
|
||||||
"version": _required_int(cfg, "config", "version"),
|
|
||||||
"mode": _required_str(cfg, "config", "mode"),
|
|
||||||
"enforce": _required_bool(cfg, "config", "enforce"),
|
|
||||||
"api_allowlist": _required_str_list(cfg, "config", "api_allowlist"),
|
|
||||||
}
|
|
||||||
|
|
||||||
if "seed_phrase_detection" in cfg:
|
|
||||||
spd = _required_dict(cfg, "config", "seed_phrase_detection")
|
|
||||||
_reject_unknown_keys("seed_phrase_detection", spd, {"enabled"})
|
|
||||||
normalized["seed_phrase_detection"] = {
|
|
||||||
"enabled": _required_bool(spd, "seed_phrase_detection", "enabled"),
|
|
||||||
}
|
|
||||||
|
|
||||||
fp = _required_dict(cfg, "config", "forward_proxy")
|
|
||||||
_reject_unknown_keys("forward_proxy", fp, {"enabled"})
|
|
||||||
normalized["forward_proxy"] = {
|
|
||||||
"enabled": _required_bool(fp, "forward_proxy", "enabled"),
|
|
||||||
}
|
|
||||||
|
|
||||||
dlp = _required_dict(cfg, "config", "dlp")
|
|
||||||
_reject_unknown_keys("dlp", dlp, {"include_defaults", "scan_env"})
|
|
||||||
normalized["dlp"] = {
|
|
||||||
"include_defaults": _required_bool(dlp, "dlp", "include_defaults"),
|
|
||||||
"scan_env": _required_bool(dlp, "dlp", "scan_env"),
|
|
||||||
}
|
|
||||||
|
|
||||||
rbs = _required_dict(cfg, "config", "request_body_scanning")
|
|
||||||
_reject_unknown_keys(
|
|
||||||
"request_body_scanning",
|
|
||||||
rbs,
|
|
||||||
{"action", "scan_headers", "header_mode"},
|
|
||||||
)
|
|
||||||
normalized_rbs: dict[str, object] = {
|
|
||||||
"action": _required_str(rbs, "request_body_scanning", "action"),
|
|
||||||
}
|
|
||||||
scan_headers = _optional_bool(rbs, "request_body_scanning", "scan_headers")
|
|
||||||
if scan_headers is not None:
|
|
||||||
normalized_rbs["scan_headers"] = scan_headers
|
|
||||||
header_mode = _optional_str(rbs, "request_body_scanning", "header_mode")
|
|
||||||
if header_mode is not None:
|
|
||||||
normalized_rbs["header_mode"] = header_mode
|
|
||||||
normalized["request_body_scanning"] = normalized_rbs
|
|
||||||
|
|
||||||
if "tls_interception" in cfg:
|
|
||||||
tls = _required_dict(cfg, "config", "tls_interception")
|
|
||||||
_reject_unknown_keys(
|
|
||||||
"tls_interception",
|
|
||||||
tls,
|
|
||||||
{"enabled", "ca_cert", "ca_key", "passthrough_domains"},
|
|
||||||
)
|
|
||||||
normalized["tls_interception"] = {
|
|
||||||
"enabled": _required_bool(tls, "tls_interception", "enabled"),
|
|
||||||
"ca_cert": _required_str(tls, "tls_interception", "ca_cert"),
|
|
||||||
"ca_key": _required_str(tls, "tls_interception", "ca_key"),
|
|
||||||
"passthrough_domains": _optional_str_list(
|
|
||||||
tls, "tls_interception", "passthrough_domains",
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
if "ssrf" in cfg:
|
|
||||||
ssrf = _required_dict(cfg, "config", "ssrf")
|
|
||||||
_reject_unknown_keys("ssrf", ssrf, {"ip_allowlist"})
|
|
||||||
normalized["ssrf"] = {
|
|
||||||
"ip_allowlist": _required_str_list(ssrf, "ssrf", "ip_allowlist"),
|
|
||||||
}
|
|
||||||
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|
||||||
"""Render a pipelock config dict (as produced by
|
|
||||||
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
|
|
||||||
YAML-parser dependency for a fixed, narrow shape."""
|
|
||||||
def _bool(b: object) -> str:
|
|
||||||
return "true" if b else "false"
|
|
||||||
|
|
||||||
cfg = _validate_pipelock_render_config(cfg)
|
|
||||||
lines: list[str] = []
|
|
||||||
lines.append(f"version: {cfg['version']}")
|
|
||||||
lines.append(f"mode: {cfg['mode']}")
|
|
||||||
lines.append(f"enforce: {_bool(cfg['enforce'])}")
|
|
||||||
lines.append("")
|
|
||||||
lines.append("api_allowlist:")
|
|
||||||
api_allowlist = cfg["api_allowlist"]
|
|
||||||
assert isinstance(api_allowlist, list)
|
|
||||||
for h in api_allowlist:
|
|
||||||
lines.append(f' - "{h}"')
|
|
||||||
lines.append("")
|
|
||||||
if "seed_phrase_detection" in cfg:
|
|
||||||
lines.append("seed_phrase_detection:")
|
|
||||||
spd = cfg["seed_phrase_detection"]
|
|
||||||
assert isinstance(spd, dict)
|
|
||||||
lines.append(f" enabled: {_bool(spd['enabled'])}")
|
|
||||||
lines.append("")
|
|
||||||
lines.append("forward_proxy:")
|
|
||||||
fp = cfg["forward_proxy"]
|
|
||||||
assert isinstance(fp, dict)
|
|
||||||
lines.append(f" enabled: {_bool(fp['enabled'])}")
|
|
||||||
lines.append("")
|
|
||||||
lines.append("dlp:")
|
|
||||||
dlp = cfg["dlp"]
|
|
||||||
assert isinstance(dlp, dict)
|
|
||||||
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
|
|
||||||
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
|
|
||||||
lines.append("")
|
|
||||||
lines.append("request_body_scanning:")
|
|
||||||
rbs = cfg["request_body_scanning"]
|
|
||||||
assert isinstance(rbs, dict)
|
|
||||||
lines.append(f' action: "{rbs["action"]}"')
|
|
||||||
if "scan_headers" in rbs:
|
|
||||||
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
|
|
||||||
if "header_mode" in rbs:
|
|
||||||
lines.append(f' header_mode: "{rbs["header_mode"]}"')
|
|
||||||
if "tls_interception" in cfg:
|
|
||||||
lines.append("")
|
|
||||||
lines.append("tls_interception:")
|
|
||||||
tls = cfg["tls_interception"]
|
|
||||||
assert isinstance(tls, dict)
|
|
||||||
lines.append(f" enabled: {_bool(tls['enabled'])}")
|
|
||||||
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
|
|
||||||
lines.append(f' ca_key: "{tls["ca_key"]}"')
|
|
||||||
passthrough = tls["passthrough_domains"]
|
|
||||||
assert isinstance(passthrough, list)
|
|
||||||
if passthrough:
|
|
||||||
lines.append(" passthrough_domains:")
|
|
||||||
for d in passthrough:
|
|
||||||
lines.append(f' - "{d}"')
|
|
||||||
if "ssrf" in cfg:
|
|
||||||
lines.append("")
|
|
||||||
lines.append("ssrf:")
|
|
||||||
ssrf = cfg["ssrf"]
|
|
||||||
assert isinstance(ssrf, dict)
|
|
||||||
lines.append(" ip_allowlist:")
|
|
||||||
ip_allowlist = ssrf["ip_allowlist"]
|
|
||||||
assert isinstance(ip_allowlist, list)
|
|
||||||
for ip in ip_allowlist:
|
|
||||||
lines.append(f' - "{ip}"')
|
|
||||||
return "\n".join(lines) + "\n"
|
|
||||||
|
|
||||||
|
|
||||||
# --- Proxy class -----------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class PipelockProxyPlan:
|
|
||||||
"""Output of PipelockProxy.prepare; consumed by .start when the
|
|
||||||
sidecar needs to be brought up.
|
|
||||||
|
|
||||||
yaml_path + slug are filled in at prepare time (host-side, side-
|
|
||||||
effect-free; the YAML references the in-container CA paths
|
|
||||||
already so it doesn't need the host paths to be valid). The
|
|
||||||
remaining fields are populated by the backend's launch step
|
|
||||||
via `dataclasses.replace`: internal/egress networks once
|
|
||||||
those networks exist, the CA host paths once the one-shot
|
|
||||||
`pipelock tls init` has run, and `internal_network_cidr` once
|
|
||||||
Docker has assigned a subnet to the internal network. Empty
|
|
||||||
defaults are sentinels meaning "not yet set"; `.start` validates
|
|
||||||
that they are populated.
|
|
||||||
|
|
||||||
`internal_network_cidr` ends up on pipelock's `ssrf.ip_allowlist`
|
|
||||||
so traffic from sibling sidecars (egress → pipelock on the
|
|
||||||
upstream leg, etc.) bypasses pipelock's RFC1918 SSRF guard while
|
|
||||||
api_allowlist and body-scanning still apply."""
|
|
||||||
|
|
||||||
yaml_path: Path
|
|
||||||
slug: str
|
|
||||||
internal_network: str = ""
|
|
||||||
internal_network_cidr: str = ""
|
|
||||||
egress_network: str = ""
|
|
||||||
ca_cert_host_path: Path = Path()
|
|
||||||
ca_key_host_path: Path = Path()
|
|
||||||
|
|
||||||
|
|
||||||
class PipelockProxy:
|
|
||||||
"""The pipelock egress proxy. Encapsulates the YAML-config
|
|
||||||
generation; the container lifecycle is owned by whatever
|
|
||||||
wraps the daemon (compose-managed pipelock container on docker,
|
|
||||||
sidecar-bundle PID 1 on smolmachines).
|
|
||||||
|
|
||||||
Backends instantiate the class directly — there are no
|
|
||||||
platform-specific subclasses; the in-container CA paths are
|
|
||||||
universal module-level constants
|
|
||||||
(`PIPELOCK_CA_CERT_IN_CONTAINER` / `PIPELOCK_CA_KEY_IN_CONTAINER`)."""
|
|
||||||
|
|
||||||
def prepare(
|
|
||||||
self,
|
|
||||||
bottle: Bottle,
|
|
||||||
slug: str,
|
|
||||||
stage_dir: Path,
|
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
|
||||||
) -> PipelockProxyPlan:
|
|
||||||
"""Write the pipelock yaml config (mode 600) under `stage_dir`
|
|
||||||
and return the plan for launch. Pure host-side, no docker
|
|
||||||
subprocess.
|
|
||||||
|
|
||||||
`slug` is the agent-derived identifier (lowercased,
|
|
||||||
hyphen-normalized) used as the suffix in every per-agent
|
|
||||||
resource name — the agent container, the sidecar bundle
|
|
||||||
container, the internal/egress networks. It's stored on the
|
|
||||||
returned plan so the backend's launch step can derive those
|
|
||||||
names.
|
|
||||||
|
|
||||||
The CA paths the YAML references are the module-level
|
|
||||||
in-container constants. The host-side counterparts are
|
|
||||||
generated by the launch step (not here, so prepare stays
|
|
||||||
side-effect-free on docker) and added to the plan via
|
|
||||||
`dataclasses.replace` before the daemon starts."""
|
|
||||||
yaml_path = stage_dir / "pipelock.yaml"
|
|
||||||
cfg = pipelock_build_config(
|
|
||||||
bottle,
|
|
||||||
ca_cert_path=PIPELOCK_CA_CERT_IN_CONTAINER,
|
|
||||||
ca_key_path=PIPELOCK_CA_KEY_IN_CONTAINER,
|
|
||||||
provider_routes=provider_routes,
|
|
||||||
)
|
|
||||||
yaml_path.write_text(pipelock_render_yaml(cfg))
|
|
||||||
yaml_path.chmod(0o600)
|
|
||||||
return PipelockProxyPlan(yaml_path=yaml_path, slug=slug)
|
|
||||||
+12
-42
@@ -1,7 +1,7 @@
|
|||||||
"""Per-bottle sidecar supervisor (PRD 0024 chunk 1).
|
"""Per-bottle sidecar supervisor (PRD 0024 chunk 1).
|
||||||
|
|
||||||
PID 1 inside the `bot-bottle-sidecars` bundle image. Spawns
|
PID 1 inside the `bot-bottle-sidecars` bundle image. Spawns
|
||||||
the configured daemons (egress, pipelock, git-gate, supervise),
|
the configured daemons (egress, git-gate, supervise),
|
||||||
forwards SIGTERM/SIGINT to each child, and propagates per-daemon
|
forwards SIGTERM/SIGINT to each child, and propagates per-daemon
|
||||||
stdout+stderr to the container log with a `[name] ` prefix.
|
stdout+stderr to the container log with a `[name] ` prefix.
|
||||||
|
|
||||||
@@ -19,7 +19,7 @@ PR; the interim policy is "don't take the bundle down for one
|
|||||||
sick daemon."
|
sick daemon."
|
||||||
|
|
||||||
Daemon subset is env-driven. The compose renderer narrows it via
|
Daemon subset is env-driven. The compose renderer narrows it via
|
||||||
`BOT_BOTTLE_SIDECAR_DAEMONS=egress,pipelock` for bottles that
|
`BOT_BOTTLE_SIDECAR_DAEMONS=egress` for bottles that
|
||||||
don't use git-gate or supervise. Default: all daemons.
|
don't use git-gate or supervise. Default: all daemons.
|
||||||
|
|
||||||
Stdlib-only by design — adding supervisord/s6/runit for four
|
Stdlib-only by design — adding supervisord/s6/runit for four
|
||||||
@@ -57,14 +57,7 @@ class _DaemonSpec:
|
|||||||
# Env-var name prefixes that carry egress-only credentials.
|
# Env-var name prefixes that carry egress-only credentials.
|
||||||
# `egress_apply.py` assigns `EGRESS_TOKEN_<n>` slots that egress
|
# `egress_apply.py` assigns `EGRESS_TOKEN_<n>` slots that egress
|
||||||
# reads to inject `Authorization` headers on configured routes;
|
# reads to inject `Authorization` headers on configured routes;
|
||||||
# every other daemon in the bundle (especially pipelock with
|
# no other daemon in the bundle should see these values.
|
||||||
# `scan_env: true`) MUST NOT see these values or it'll match the
|
|
||||||
# injected token in the request egress just sent and 403-block
|
|
||||||
# the legitimate traffic (issue #84). The agent itself runs in a
|
|
||||||
# different machine and never has access to these slots in the
|
|
||||||
# first place, so stripping them from non-egress daemons loses no
|
|
||||||
# DLP coverage — pipelock can't catch the exfil of a value the
|
|
||||||
# agent doesn't have.
|
|
||||||
_EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",)
|
_EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",)
|
||||||
|
|
||||||
|
|
||||||
@@ -81,22 +74,8 @@ def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
# Order matters only for first-launch race-window reasons: egress
|
|
||||||
# starts first so pipelock's upstream connect succeeds during
|
|
||||||
# pipelock's own startup. git-gate and supervise are independent.
|
|
||||||
# Pipelock binds 0.0.0.0:8888 explicitly. Without `--listen` it
|
|
||||||
# defaults to 127.0.0.1 which would be unreachable from sibling
|
|
||||||
# services on the docker network. The legacy four-sidecar
|
|
||||||
# compose renderer passed the same flag; the bundle keeps the
|
|
||||||
# explicit binding.
|
|
||||||
_DAEMONS: tuple[_DaemonSpec, ...] = (
|
_DAEMONS: tuple[_DaemonSpec, ...] = (
|
||||||
_DaemonSpec("egress", ("/bin/sh", "/app/egress-entrypoint.sh")),
|
_DaemonSpec("egress", ("/bin/sh", "/app/egress-entrypoint.sh")),
|
||||||
_DaemonSpec(
|
|
||||||
"pipelock",
|
|
||||||
("/usr/local/bin/pipelock", "run",
|
|
||||||
"--config", "/etc/pipelock.yaml",
|
|
||||||
"--listen", "0.0.0.0:8888"),
|
|
||||||
),
|
|
||||||
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
|
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
|
||||||
_DaemonSpec("git-http", ("python3", "/app/git_http_backend.py")),
|
_DaemonSpec("git-http", ("python3", "/app/git_http_backend.py")),
|
||||||
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")),
|
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")),
|
||||||
@@ -138,7 +117,7 @@ def _pump(name: str, stream: IO[bytes]) -> None:
|
|||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
def _spawn(spec: _DaemonSpec) -> subprocess.Popen:
|
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
|
||||||
proc = subprocess.Popen(
|
proc = subprocess.Popen(
|
||||||
list(spec.argv),
|
list(spec.argv),
|
||||||
stdout=subprocess.PIPE,
|
stdout=subprocess.PIPE,
|
||||||
@@ -158,7 +137,7 @@ class _Supervisor:
|
|||||||
|
|
||||||
def __init__(self, specs: Sequence[_DaemonSpec]):
|
def __init__(self, specs: Sequence[_DaemonSpec]):
|
||||||
self.specs = tuple(specs)
|
self.specs = tuple(specs)
|
||||||
self.procs: list[tuple[_DaemonSpec, subprocess.Popen]] = []
|
self.procs: list[tuple[_DaemonSpec, subprocess.Popen[bytes]]] = []
|
||||||
self.shutdown_at: float | None = None
|
self.shutdown_at: float | None = None
|
||||||
# Names of children that have been logged as having exited
|
# Names of children that have been logged as having exited
|
||||||
# so we only log each death once across watch-loop ticks.
|
# so we only log each death once across watch-loop ticks.
|
||||||
@@ -303,10 +282,8 @@ class _Supervisor:
|
|||||||
|
|
||||||
def restart_daemon(self, daemon_name: str, *, grace: float = 5.0) -> bool:
|
def restart_daemon(self, daemon_name: str, *, grace: float = 5.0) -> bool:
|
||||||
"""Terminate one named child and spawn a fresh one, leaving
|
"""Terminate one named child and spawn a fresh one, leaving
|
||||||
the other daemons running. Used by the pipelock-apply path:
|
the other daemons running. A daemon that has no in-process
|
||||||
pipelock has no in-process reload, so apply_allowlist_change
|
reload can be restarted this way after its config file changes.
|
||||||
runs `docker kill --signal USR1 <bundle>` after writing the
|
|
||||||
new yaml; the supervisor catches SIGUSR1 and calls this.
|
|
||||||
|
|
||||||
Behavior: SIGTERM → wait up to `grace` seconds → SIGKILL if
|
Behavior: SIGTERM → wait up to `grace` seconds → SIGKILL if
|
||||||
still alive → spawn a replacement under the same DaemonSpec.
|
still alive → spawn a replacement under the same DaemonSpec.
|
||||||
@@ -314,8 +291,8 @@ class _Supervisor:
|
|||||||
forward_signal / shutdown calls reach the new pid.
|
forward_signal / shutdown calls reach the new pid.
|
||||||
|
|
||||||
Returns True iff a daemon by that name was running and a
|
Returns True iff a daemon by that name was running and a
|
||||||
replacement spawned; False if no such daemon (the
|
replacement spawned; False if no such daemon (not wired
|
||||||
compose-renderer subset said this bottle doesn't run it)."""
|
for this bottle)."""
|
||||||
if self.shutdown_at is not None:
|
if self.shutdown_at is not None:
|
||||||
_log(f"restart {daemon_name} skipped; supervisor is shutting down")
|
_log(f"restart {daemon_name} skipped; supervisor is shutting down")
|
||||||
return False
|
return False
|
||||||
@@ -360,20 +337,13 @@ def main(argv: Sequence[str] | None = None) -> int:
|
|||||||
sup = _Supervisor(specs)
|
sup = _Supervisor(specs)
|
||||||
sup.start_all()
|
sup.start_all()
|
||||||
|
|
||||||
signal.signal(signal.SIGTERM, lambda *_: sup.request_shutdown("SIGTERM"))
|
signal.signal(signal.SIGTERM, lambda *_: sup.request_shutdown("SIGTERM")) # type: ignore
|
||||||
signal.signal(signal.SIGINT, lambda *_: sup.request_shutdown("SIGINT"))
|
signal.signal(signal.SIGINT, lambda *_: sup.request_shutdown("SIGINT")) # type: ignore
|
||||||
# SIGHUP reload path: egress_apply.py runs `docker kill
|
# SIGHUP reload path: egress_apply.py runs `docker kill
|
||||||
# --signal HUP <bundle>` after writing routes.yaml. The kernel
|
# --signal HUP <bundle>` after writing routes.yaml. The kernel
|
||||||
# delivers SIGHUP to PID 1 (this supervisor); forward it to
|
# delivers SIGHUP to PID 1 (this supervisor); forward it to
|
||||||
# mitmdump so it reloads its addon.
|
# mitmdump so it reloads its addon.
|
||||||
signal.signal(signal.SIGHUP, lambda *_: sup.forward_signal(signal.SIGHUP, "egress"))
|
signal.signal(signal.SIGHUP, lambda *_: sup.forward_signal(signal.SIGHUP, "egress")) # type: ignore
|
||||||
# SIGUSR1 pipelock-restart path: pipelock_apply.py runs
|
|
||||||
# `docker kill --signal USR1 <bundle>` after writing
|
|
||||||
# pipelock.yaml. Pipelock has no in-process reload, so the
|
|
||||||
# supervisor restarts the pipelock daemon in place (other
|
|
||||||
# daemons keep running — specifically supervise, whose MCP
|
|
||||||
# socket would drop on a whole-container `docker restart`).
|
|
||||||
signal.signal(signal.SIGUSR1, lambda *_: sup.request_restart("pipelock"))
|
|
||||||
|
|
||||||
while not sup.tick():
|
while not sup.tick():
|
||||||
time.sleep(_POLL_INTERVAL)
|
time.sleep(_POLL_INTERVAL)
|
||||||
|
|||||||
+11
-22
@@ -6,14 +6,13 @@ sits on the bottle's internal network and exposes three MCP tools the
|
|||||||
agent calls when it hits a stuck-recovery category:
|
agent calls when it hits a stuck-recovery category:
|
||||||
|
|
||||||
* egress-block — agent proposes a new routes.yaml
|
* egress-block — agent proposes a new routes.yaml
|
||||||
* pipelock-block — agent proposes a new pipelock allowlist
|
* capability-block — agent proposes a new agent Dockerfile
|
||||||
* capability-block — agent proposes a new agent Dockerfile
|
|
||||||
|
|
||||||
Each tool call: the agent passes the full proposed file plus a
|
Each tool call: the agent passes the full proposed file plus a
|
||||||
justification text. The sidecar validates the proposal syntactically,
|
justification text. The sidecar validates the proposal syntactically,
|
||||||
writes it to the host's per-bottle queue dir, and holds the tool-call
|
writes it to the host's per-bottle queue dir, and holds the tool-call
|
||||||
connection open. The operator's TUI dashboard
|
connection open. The operator's supervise TUI
|
||||||
(bot_bottle.cli.dashboard) sees the proposal, accepts
|
(bot_bottle.cli.supervise) sees the proposal, accepts
|
||||||
approve / modify / reject, and writes a response file alongside the
|
approve / modify / reject, and writes a response file alongside the
|
||||||
proposal. The sidecar sees the response and returns `{status, notes}`
|
proposal. The sidecar sees the response and returns `{status, notes}`
|
||||||
to the agent.
|
to the agent.
|
||||||
@@ -40,7 +39,7 @@ import json
|
|||||||
import os
|
import os
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -49,13 +48,9 @@ from pathlib import Path
|
|||||||
SUPERVISE_HOSTNAME = "supervise"
|
SUPERVISE_HOSTNAME = "supervise"
|
||||||
SUPERVISE_PORT = 9100
|
SUPERVISE_PORT = 9100
|
||||||
|
|
||||||
TOOL_EGRESS_BLOCK = "egress-block"
|
|
||||||
TOOL_PIPELOCK_BLOCK = "pipelock-block"
|
|
||||||
TOOL_CAPABILITY_BLOCK = "capability-block"
|
TOOL_CAPABILITY_BLOCK = "capability-block"
|
||||||
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
|
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
|
||||||
TOOLS: tuple[str, ...] = (
|
TOOLS: tuple[str, ...] = (
|
||||||
TOOL_EGRESS_BLOCK,
|
|
||||||
TOOL_PIPELOCK_BLOCK,
|
|
||||||
TOOL_CAPABILITY_BLOCK,
|
TOOL_CAPABILITY_BLOCK,
|
||||||
TOOL_LIST_EGRESS_ROUTES,
|
TOOL_LIST_EGRESS_ROUTES,
|
||||||
)
|
)
|
||||||
@@ -73,11 +68,8 @@ EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
|
|||||||
# capability-block has no on-disk config the operator edits in place
|
# capability-block has no on-disk config the operator edits in place
|
||||||
# (the Dockerfile is rebuilt, not patched), so it has no audit log
|
# (the Dockerfile is rebuilt, not patched), so it has no audit log
|
||||||
# here — those changes are captured by git history + the rebuild
|
# here — those changes are captured by git history + the rebuild
|
||||||
# record laid down in PRD 0016.
|
# record laid down in PRD 0016. egress-block was removed in issue #198.
|
||||||
COMPONENT_FOR_TOOL: dict[str, str] = {
|
COMPONENT_FOR_TOOL: dict[str, str] = {}
|
||||||
TOOL_EGRESS_BLOCK: "egress",
|
|
||||||
TOOL_PIPELOCK_BLOCK: "pipelock",
|
|
||||||
}
|
|
||||||
|
|
||||||
STATUS_APPROVED = "approved"
|
STATUS_APPROVED = "approved"
|
||||||
STATUS_MODIFIED = "modified"
|
STATUS_MODIFIED = "modified"
|
||||||
@@ -85,8 +77,7 @@ STATUS_REJECTED = "rejected"
|
|||||||
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
|
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
|
||||||
|
|
||||||
# Operator-initiated audit entries (no tool call). PRD 0014's
|
# Operator-initiated audit entries (no tool call). PRD 0014's
|
||||||
# `routes edit <bottle>` and PRD 0015's `pipelock edit <bottle>`
|
# `routes edit <bottle>` verb writes entries with this action.
|
||||||
# verbs write entries with this action.
|
|
||||||
ACTION_OPERATOR_EDIT = "operator-edit"
|
ACTION_OPERATOR_EDIT = "operator-edit"
|
||||||
|
|
||||||
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
|
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
|
||||||
@@ -519,22 +510,22 @@ def _atomic_write(path: Path, content: str, *, mode: int) -> None:
|
|||||||
try:
|
try:
|
||||||
import fcntl as _fcntl
|
import fcntl as _fcntl
|
||||||
|
|
||||||
def _try_flock(fd: int) -> None:
|
def _try_flock(fd: int) -> None: # type: ignore[reportRedeclaration]
|
||||||
try:
|
try:
|
||||||
_fcntl.flock(fd, _fcntl.LOCK_EX)
|
_fcntl.flock(fd, _fcntl.LOCK_EX)
|
||||||
except OSError:
|
except OSError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def _try_funlock(fd: int) -> None:
|
def _try_funlock(fd: int) -> None: # type: ignore[reportRedeclaration]
|
||||||
try:
|
try:
|
||||||
_fcntl.flock(fd, _fcntl.LOCK_UN)
|
_fcntl.flock(fd, _fcntl.LOCK_UN)
|
||||||
except OSError:
|
except OSError:
|
||||||
pass
|
pass
|
||||||
except ImportError: # pragma: no cover — Windows path
|
except ImportError: # pragma: no cover — Windows path
|
||||||
def _try_flock(fd: int) -> None:
|
def _try_flock(fd: int) -> None: # noqa: F841 — Windows fallback
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def _try_funlock(fd: int) -> None:
|
def _try_funlock(fd: int) -> None: # noqa: F841 — Windows fallback
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
@@ -560,9 +551,7 @@ __all__ = [
|
|||||||
"EGRESS_FORWARD_PROXY",
|
"EGRESS_FORWARD_PROXY",
|
||||||
"EGRESS_INTROSPECT_URL",
|
"EGRESS_INTROSPECT_URL",
|
||||||
"TOOL_CAPABILITY_BLOCK",
|
"TOOL_CAPABILITY_BLOCK",
|
||||||
"TOOL_EGRESS_BLOCK",
|
|
||||||
"TOOL_LIST_EGRESS_ROUTES",
|
"TOOL_LIST_EGRESS_ROUTES",
|
||||||
"TOOL_PIPELOCK_BLOCK",
|
|
||||||
"archive_proposal",
|
"archive_proposal",
|
||||||
"audit_dir",
|
"audit_dir",
|
||||||
"audit_log_path",
|
"audit_log_path",
|
||||||
|
|||||||
+20
-229
@@ -1,8 +1,10 @@
|
|||||||
"""Supervise sidecar HTTP server (PRD 0013).
|
"""Supervise sidecar HTTP server (PRD 0013).
|
||||||
|
|
||||||
Per-bottle MCP server exposing three tools — `egress-block`,
|
Per-bottle MCP server exposing tools the agent calls to propose config
|
||||||
`pipelock-block`, `capability-block` — that the agent calls to
|
changes when stuck. The egress-block tool was removed in issue #198;
|
||||||
propose config changes when stuck. Each tool call:
|
the remaining tools are `capability-block` and `list-egress-routes`.
|
||||||
|
|
||||||
|
Each queued tool call:
|
||||||
|
|
||||||
1. Validates the proposed file syntactically.
|
1. Validates the proposed file syntactically.
|
||||||
2. Writes a Proposal to /run/supervise/queue/ (bind-mounted from
|
2. Writes a Proposal to /run/supervise/queue/ (bind-mounted from
|
||||||
@@ -18,7 +20,7 @@ Speaks MCP over HTTP+JSON-RPC. Methods handled:
|
|||||||
|
|
||||||
* `initialize` — handshake; returns server info + caps.
|
* `initialize` — handshake; returns server info + caps.
|
||||||
* `notifications/initialized` — ack-only.
|
* `notifications/initialized` — ack-only.
|
||||||
* `tools/list` — returns the three tool definitions.
|
* `tools/list` — returns the tool definitions.
|
||||||
* `tools/call` — validates, queues, blocks, returns.
|
* `tools/call` — validates, queues, blocks, returns.
|
||||||
|
|
||||||
Everything else returns JSON-RPC error -32601 (method not found).
|
Everything else returns JSON-RPC error -32601 (method not found).
|
||||||
@@ -38,7 +40,6 @@ import sys
|
|||||||
import time
|
import time
|
||||||
import typing
|
import typing
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.parse
|
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -134,81 +135,15 @@ def jsonrpc_error(request_id: object, code: int, message: str) -> bytes:
|
|||||||
|
|
||||||
|
|
||||||
TOOL_DEFINITIONS: list[dict[str, object]] = [
|
TOOL_DEFINITIONS: list[dict[str, object]] = [
|
||||||
{
|
|
||||||
"name": _sv.TOOL_EGRESS_BLOCK,
|
|
||||||
"description": (
|
|
||||||
"Call when egress refused your HTTPS request — host "
|
|
||||||
"without a matching route, or a path outside the route's "
|
|
||||||
"path_allowlist (typically a 403 from the proxy). Propose "
|
|
||||||
"a SINGLE route to add: the host you need + (optionally) "
|
|
||||||
"a path_allowlist + (optionally) an auth block. The "
|
|
||||||
"supervisor merges the route into the live table at "
|
|
||||||
"approval time — you do NOT need to see or reproduce the "
|
|
||||||
"existing routes, and you do not pass a full routes file. "
|
|
||||||
"If the host already has a route, the proposed "
|
|
||||||
"path_allowlist entries are unioned with the existing "
|
|
||||||
"ones (host stays single-route). The operator approves "
|
|
||||||
"or rejects in the supervise TUI. On approval the "
|
|
||||||
"supervisor writes the merged routes.yaml, SIGHUPs "
|
|
||||||
"egress (atomic swap, no dropped connections), and "
|
|
||||||
"mirrors the host onto pipelock's allowlist for the "
|
|
||||||
"downstream gate."
|
|
||||||
),
|
|
||||||
"inputSchema": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"host": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "The hostname to allow (e.g. 'api.github.com'). Case-insensitive on match.",
|
|
||||||
},
|
|
||||||
"path_allowlist": {
|
|
||||||
"type": "array",
|
|
||||||
"items": {"type": "string"},
|
|
||||||
"description": (
|
|
||||||
"Optional URL path prefixes the route permits. "
|
|
||||||
"Each must start with '/'. Omit to allow all "
|
|
||||||
"paths under this host (bare-pass route)."
|
|
||||||
),
|
|
||||||
},
|
|
||||||
"auth": {
|
|
||||||
"type": "object",
|
|
||||||
"description": (
|
|
||||||
"Optional credential injection. {scheme, "
|
|
||||||
"token_ref}: scheme is 'Bearer' or 'token'; "
|
|
||||||
"token_ref names the host env var holding the "
|
|
||||||
"secret value. Omit to add a host without "
|
|
||||||
"credential injection. Ignored if the host "
|
|
||||||
"already has a route (operator decides auth "
|
|
||||||
"changes, not the agent)."
|
|
||||||
),
|
|
||||||
"properties": {
|
|
||||||
"scheme": {"type": "string"},
|
|
||||||
"token_ref": {"type": "string"},
|
|
||||||
},
|
|
||||||
"required": ["scheme", "token_ref"],
|
|
||||||
"additionalProperties": False,
|
|
||||||
},
|
|
||||||
"justification": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Why this host needs to be allowed.",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"required": ["host", "justification"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"name": _sv.TOOL_LIST_EGRESS_ROUTES,
|
"name": _sv.TOOL_LIST_EGRESS_ROUTES,
|
||||||
"description": (
|
"description": (
|
||||||
"List the current egress route table — the bottle's "
|
"List the current egress route table — the bottle's "
|
||||||
"primary egress allowlist. Returns JSON with one entry "
|
"allowlist. Returns JSON with one entry per allowed host, "
|
||||||
"per allowed host, each carrying its path_allowlist (if "
|
"each carrying its matches rules (if any) and whether "
|
||||||
"any) and whether the proxy injects Authorization for "
|
"the proxy injects Authorization for the route. Use this "
|
||||||
"the route. Use this before composing an "
|
"before composing an `egress-block` proposal so the new "
|
||||||
"`egress-block` proposal so the new routes file "
|
"routes file extends the live one rather than replacing it."
|
||||||
"extends the live one rather than replacing it. "
|
|
||||||
"Pipelock's allowlist is a mirror of this set — every "
|
|
||||||
"host listed here is also reachable through pipelock's "
|
|
||||||
"downstream hostname gate."
|
|
||||||
),
|
),
|
||||||
"inputSchema": {
|
"inputSchema": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
@@ -216,48 +151,12 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
|
|||||||
"additionalProperties": False,
|
"additionalProperties": False,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
|
||||||
"name": _sv.TOOL_PIPELOCK_BLOCK,
|
|
||||||
"description": (
|
|
||||||
"Call when pipelock refused your outbound request and "
|
|
||||||
"the failing host is genuinely missing from the bottle's "
|
|
||||||
"allowlist (vs. blocked for DLP reasons — those need a "
|
|
||||||
"different remediation). In practice pipelock's allowlist "
|
|
||||||
"is now a mirror of the egress routes set by "
|
|
||||||
"`egress-block`, so prefer that tool when you want "
|
|
||||||
"to add a host. This tool stays available for the rare "
|
|
||||||
"case where pipelock and egress have diverged. "
|
|
||||||
"Pass the full URL you tried to hit (scheme + host + "
|
|
||||||
"path); the supervisor extracts the hostname and merges "
|
|
||||||
"it into pipelock's allowlist. On approval the "
|
|
||||||
"supervisor restarts pipelock."
|
|
||||||
),
|
|
||||||
"inputSchema": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"failed_url": {
|
|
||||||
"type": "string",
|
|
||||||
"description": (
|
|
||||||
"The full URL pipelock blocked, e.g. "
|
|
||||||
"https://api.github.com/repos/foo/bar. Scheme "
|
|
||||||
"and hostname are required; path is recorded "
|
|
||||||
"as operator context."
|
|
||||||
),
|
|
||||||
},
|
|
||||||
"justification": {
|
|
||||||
"type": "string",
|
|
||||||
"description": "Why the new host should be allowed.",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"required": ["failed_url", "justification"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"name": _sv.TOOL_CAPABILITY_BLOCK,
|
"name": _sv.TOOL_CAPABILITY_BLOCK,
|
||||||
"description": (
|
"description": (
|
||||||
"Call when the bottle is missing a tool, skill, permission, "
|
"Call when the bottle is missing a tool, skill, permission, "
|
||||||
"or env var you need — something that lives in the agent "
|
"or env var you need — something that lives in the agent "
|
||||||
"Dockerfile rather than in routes or the pipelock allowlist. "
|
"Dockerfile rather than in the egress routes. "
|
||||||
"Read the current Dockerfile from "
|
"Read the current Dockerfile from "
|
||||||
"/etc/bot-bottle/current-config/Dockerfile, compose a "
|
"/etc/bot-bottle/current-config/Dockerfile, compose a "
|
||||||
"modified version, and pass the full new file plus a "
|
"modified version, and pass the full new file plus a "
|
||||||
@@ -283,27 +182,10 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
# Map each tool to the input field that carries the agent's
|
# Map each non-egress tool to the input field that carries the agent's
|
||||||
# tool-specific payload (stored in Proposal.proposed_file as
|
# payload (stored in Proposal.proposed_file). egress-block builds its
|
||||||
# free-form text the apply path interprets per tool).
|
# payload from structured input fields in `handle_egress_block`.
|
||||||
#
|
|
||||||
# egress-block: JSON object describing a SINGLE route to
|
|
||||||
# add — `{host, path_allowlist?, auth?}`. The
|
|
||||||
# supervisor merges this into the live routes
|
|
||||||
# file at approval time.
|
|
||||||
# pipelock-block: the full failed URL (scheme + host + path) —
|
|
||||||
# supervisor extracts the host, merges into the
|
|
||||||
# bottle's current allowlist; the path is shown
|
|
||||||
# to the operator for context (pipelock doesn't
|
|
||||||
# do path-level matching).
|
|
||||||
# capability-block: full proposed Dockerfile
|
|
||||||
#
|
|
||||||
# Egress-proxy-block doesn't use a single "field name" → the JSON
|
|
||||||
# payload is constructed from multiple structured input fields in
|
|
||||||
# `handle_egress_block`. The mapping stays one-entry-per-tool
|
|
||||||
# so the generic dispatch keeps working for the other two.
|
|
||||||
PROPOSED_FILE_FIELD: dict[str, str] = {
|
PROPOSED_FILE_FIELD: dict[str, str] = {
|
||||||
_sv.TOOL_PIPELOCK_BLOCK: "failed_url",
|
|
||||||
_sv.TOOL_CAPABILITY_BLOCK: "dockerfile",
|
_sv.TOOL_CAPABILITY_BLOCK: "dockerfile",
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -311,34 +193,13 @@ PROPOSED_FILE_FIELD: dict[str, str] = {
|
|||||||
# --- Validation ------------------------------------------------------------
|
# --- Validation ------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
# Auth schemes accepted on egress-block proposals — match the
|
|
||||||
# manifest-side EGRESS_AUTH_SCHEMES.
|
|
||||||
_AUTH_SCHEMES = ("Bearer", "token")
|
|
||||||
|
|
||||||
|
|
||||||
def validate_proposed_file(tool: str, content: str) -> None:
|
def validate_proposed_file(tool: str, content: str) -> None:
|
||||||
"""Syntactic validation. The operator is the real gate; this just
|
"""Syntactic validation. The operator is the real gate; this just
|
||||||
catches obvious paste-errors / wrong-tool selections before they
|
catches obvious paste-errors / wrong-tool selections before they
|
||||||
enter the queue."""
|
enter the queue."""
|
||||||
if not content.strip():
|
if not content.strip():
|
||||||
raise _RpcError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
|
raise _RpcError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
|
||||||
if tool == _sv.TOOL_PIPELOCK_BLOCK:
|
if tool == _sv.TOOL_CAPABILITY_BLOCK:
|
||||||
# `content` is the full failed URL. Require scheme + host so
|
|
||||||
# the supervisor can extract a hostname for the allowlist
|
|
||||||
# merge; the path is preserved for operator context.
|
|
||||||
parsed = urllib.parse.urlsplit(content.strip())
|
|
||||||
if parsed.scheme not in ("http", "https"):
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: failed_url must start with http:// or https:// "
|
|
||||||
f"(got {content!r})",
|
|
||||||
)
|
|
||||||
if not parsed.hostname:
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: failed_url is missing a hostname (got {content!r})",
|
|
||||||
)
|
|
||||||
elif tool == _sv.TOOL_CAPABILITY_BLOCK:
|
|
||||||
# Dockerfiles are too varied to validate syntactically beyond
|
# Dockerfiles are too varied to validate syntactically beyond
|
||||||
# non-empty. The operator reads the diff in the TUI.
|
# non-empty. The operator reads the diff in the TUI.
|
||||||
pass
|
pass
|
||||||
@@ -346,70 +207,6 @@ def validate_proposed_file(tool: str, content: str) -> None:
|
|||||||
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
|
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
|
||||||
|
|
||||||
|
|
||||||
def _validate_and_bundle_egress_route(
|
|
||||||
args: dict[str, object],
|
|
||||||
) -> str:
|
|
||||||
"""Validate egress-block input fields and bundle them into
|
|
||||||
a JSON string that becomes the Proposal.proposed_file. Raises
|
|
||||||
_RpcError on bad input — the agent retries with a fixed shape."""
|
|
||||||
tool = _sv.TOOL_EGRESS_BLOCK
|
|
||||||
host = args.get("host")
|
|
||||||
if not isinstance(host, str) or not host.strip():
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: 'host' is required and must be a non-empty string",
|
|
||||||
)
|
|
||||||
payload: dict[str, object] = {"host": host}
|
|
||||||
|
|
||||||
path_allow_raw = args.get("path_allowlist")
|
|
||||||
if path_allow_raw is not None:
|
|
||||||
if not isinstance(path_allow_raw, list):
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: 'path_allowlist' must be an array of strings",
|
|
||||||
)
|
|
||||||
prefixes: list[str] = []
|
|
||||||
for i, p in enumerate(path_allow_raw):
|
|
||||||
if not isinstance(p, str):
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: path_allowlist[{i}] must be a string",
|
|
||||||
)
|
|
||||||
if not p.startswith("/"):
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: path_allowlist[{i}] {p!r} must start with '/'",
|
|
||||||
)
|
|
||||||
prefixes.append(p)
|
|
||||||
if prefixes:
|
|
||||||
payload["path_allowlist"] = prefixes
|
|
||||||
|
|
||||||
auth_raw = args.get("auth")
|
|
||||||
if auth_raw is not None:
|
|
||||||
if not isinstance(auth_raw, dict):
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: 'auth' must be an object with 'scheme' and 'token_ref'",
|
|
||||||
)
|
|
||||||
scheme = auth_raw.get("scheme")
|
|
||||||
token_ref = auth_raw.get("token_ref")
|
|
||||||
if not isinstance(scheme, str) or scheme not in _AUTH_SCHEMES:
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: auth.scheme must be one of "
|
|
||||||
f"{', '.join(_AUTH_SCHEMES)} (got {scheme!r})",
|
|
||||||
)
|
|
||||||
if not isinstance(token_ref, str) or not token_ref:
|
|
||||||
raise _RpcError(
|
|
||||||
ERR_INVALID_PARAMS,
|
|
||||||
f"{tool}: auth.token_ref must be a non-empty string "
|
|
||||||
f"naming the host env var holding the token",
|
|
||||||
)
|
|
||||||
payload["auth"] = {"scheme": scheme, "token_ref": token_ref}
|
|
||||||
|
|
||||||
return json.dumps(payload, indent=2) + "\n"
|
|
||||||
|
|
||||||
|
|
||||||
# --- MCP handlers ----------------------------------------------------------
|
# --- MCP handlers ----------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@@ -482,7 +279,7 @@ def handle_tools_call(
|
|||||||
if not isinstance(name, str):
|
if not isinstance(name, str):
|
||||||
raise _RpcError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
|
raise _RpcError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
|
||||||
if name == _sv.TOOL_LIST_EGRESS_ROUTES:
|
if name == _sv.TOOL_LIST_EGRESS_ROUTES:
|
||||||
return handle_list_egress_routes(params.get("arguments", {}), config)
|
return handle_list_egress_routes(typing.cast(dict[str, object], params.get("arguments", {})), config)
|
||||||
|
|
||||||
args_raw = params.get("arguments", {})
|
args_raw = params.get("arguments", {})
|
||||||
if not isinstance(args_raw, dict):
|
if not isinstance(args_raw, dict):
|
||||||
@@ -495,13 +292,7 @@ def handle_tools_call(
|
|||||||
f"{name}: 'justification' is required and must be a non-empty string",
|
f"{name}: 'justification' is required and must be a non-empty string",
|
||||||
)
|
)
|
||||||
|
|
||||||
if name == _sv.TOOL_EGRESS_BLOCK:
|
if name in PROPOSED_FILE_FIELD:
|
||||||
# Structured input → JSON bundle on Proposal.proposed_file.
|
|
||||||
# The dashboard's apply step (egress_apply.add_route)
|
|
||||||
# parses this JSON, fetches the current routes, merges in
|
|
||||||
# the new one, and writes the merged file.
|
|
||||||
proposed_file = _validate_and_bundle_egress_route(args_raw)
|
|
||||||
elif name in PROPOSED_FILE_FIELD:
|
|
||||||
file_field = PROPOSED_FILE_FIELD[name]
|
file_field = PROPOSED_FILE_FIELD[name]
|
||||||
proposed_file = args_raw.get(file_field)
|
proposed_file = args_raw.get(file_field)
|
||||||
if not isinstance(proposed_file, str):
|
if not isinstance(proposed_file, str):
|
||||||
@@ -587,7 +378,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
|
|
||||||
server_version = f"{SERVER_NAME}/{SERVER_VERSION}"
|
server_version = f"{SERVER_NAME}/{SERVER_VERSION}"
|
||||||
|
|
||||||
def log_message(self, format: str, *args: typing.Any) -> None:
|
def log_message(self, format: str, *args: typing.Any) -> None: # noqa: A002
|
||||||
if os.environ.get("SUPERVISE_DEBUG"):
|
if os.environ.get("SUPERVISE_DEBUG"):
|
||||||
super().log_message(format, *args)
|
super().log_message(format, *args)
|
||||||
|
|
||||||
@@ -627,7 +418,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
|||||||
except _RpcError as e:
|
except _RpcError as e:
|
||||||
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
|
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
|
||||||
return
|
return
|
||||||
except Exception as e: # pragma: no cover — defensive
|
except Exception as e: # noqa: W0718 — catch-all for RPC dispatch errors
|
||||||
sys.stderr.write(f"supervise: internal error: {e}\n")
|
sys.stderr.write(f"supervise: internal error: {e}\n")
|
||||||
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
|
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -13,8 +13,15 @@ DEFAULT_WORKSPACE_MODE = "755"
|
|||||||
|
|
||||||
|
|
||||||
class WorkspaceSpec(Protocol):
|
class WorkspaceSpec(Protocol):
|
||||||
copy_cwd: bool
|
@property
|
||||||
user_cwd: str
|
def copy_cwd(self) -> bool:
|
||||||
|
"""Whether to copy the current working directory."""
|
||||||
|
...
|
||||||
|
|
||||||
|
@property
|
||||||
|
def user_cwd(self) -> str:
|
||||||
|
"""The user's current working directory."""
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
|
|||||||
@@ -58,11 +58,12 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import re
|
import re
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
|
|
||||||
class YamlSubsetError(ValueError):
|
class YamlSubsetError(ValueError):
|
||||||
"""Raised when input violates the YAML subset's rules. Callers
|
"""Raised when input violates the YAML subset's rules. Callers
|
||||||
that want fatal-exit semantics (manifest loader, pipelock-apply,
|
that want fatal-exit semantics (manifest loader, egress-apply,
|
||||||
etc.) catch this at their own boundary and forward to `die`;
|
etc.) catch this at their own boundary and forward to `die`;
|
||||||
callers running outside the bot-bottle CLI process (the
|
callers running outside the bot-bottle CLI process (the
|
||||||
egress sidecar's addon) handle it as a normal exception."""
|
egress sidecar's addon) handle it as a normal exception."""
|
||||||
@@ -283,7 +284,7 @@ def _split_flow(body: str, lineno: int, kind: str) -> list[str]:
|
|||||||
depth_c = 0
|
depth_c = 0
|
||||||
in_single = False
|
in_single = False
|
||||||
in_double = False
|
in_double = False
|
||||||
cur = []
|
cur: list[str] = []
|
||||||
for ch in body:
|
for ch in body:
|
||||||
if ch == "'" and not in_double:
|
if ch == "'" and not in_double:
|
||||||
in_single = not in_single
|
in_single = not in_single
|
||||||
@@ -330,6 +331,7 @@ def _split_key_value(content: str, lineno: int) -> tuple[str, str]:
|
|||||||
if i + 1 >= len(content) or content[i + 1] in (" ", "\t"):
|
if i + 1 >= len(content) or content[i + 1] in (" ", "\t"):
|
||||||
return content[:i].strip(), content[i + 1:].lstrip()
|
return content[:i].strip(), content[i + 1:].lstrip()
|
||||||
die(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
|
die(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
|
||||||
|
return "", "" # unreachable, but needed for type checker
|
||||||
|
|
||||||
|
|
||||||
def _parse_block(
|
def _parse_block(
|
||||||
@@ -536,7 +538,7 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
|
|||||||
)
|
)
|
||||||
if not isinstance(value, dict):
|
if not isinstance(value, dict):
|
||||||
die("yaml-subset: top-level value must be a mapping")
|
die("yaml-subset: top-level value must be a mapping")
|
||||||
return value
|
return cast(dict[str, object], value)
|
||||||
|
|
||||||
|
|
||||||
def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
|
def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
|
||||||
|
|||||||
+6
-4
@@ -22,7 +22,9 @@ mounted in. That topology breaks two assumptions those tests make:
|
|||||||
`http://127.0.0.1:<host_port>` from inside the job time out.
|
`http://127.0.0.1:<host_port>` from inside the job time out.
|
||||||
|
|
||||||
The affected tests (`test_orphan_cleanup.test_create_and_remove`,
|
The affected tests (`test_orphan_cleanup.test_create_and_remove`,
|
||||||
`test_pipelock_sidecar_smoke.test_smoke`) still run locally where the
|
`test_sidecar_bundle_image.TestSidecarBundleImage`,
|
||||||
test process and Docker daemon share a host. Making them work in CI
|
`test_sidecar_bundle_compose.TestSidecarBundleCompose`) still run
|
||||||
is a follow-up: either re-write them to discover container IPs via
|
locally where the test process and Docker daemon share a host.
|
||||||
`docker inspect`, or reconfigure the runner with host networking.
|
Making them work in CI is a follow-up: either re-write them to
|
||||||
|
discover container IPs via `docker inspect`, or reconfigure the
|
||||||
|
runner with host networking.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0019: Active agents in the dashboard, agent-scoped edit verbs
|
# PRD 0019: Active agents in the dashboard, agent-scoped edit verbs
|
||||||
|
|
||||||
- **Status:** Active
|
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||||
- **Author:** didericis
|
- **Author:** didericis
|
||||||
- **Created:** 2026-05-26
|
- **Created:** 2026-05-26
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0020: Start and attach to agents from inside the dashboard
|
# PRD 0020: Start and attach to agents from inside the dashboard
|
||||||
|
|
||||||
- **Status:** Active
|
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||||
- **Author:** didericis
|
- **Author:** didericis
|
||||||
- **Created:** 2026-05-26
|
- **Created:** 2026-05-26
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0021: Dashboard as left tmux pane, selected agent as right pane
|
# PRD 0021: Dashboard as left tmux pane, selected agent as right pane
|
||||||
|
|
||||||
- **Status:** Active
|
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||||
- **Author:** didericis
|
- **Author:** didericis
|
||||||
- **Created:** 2026-05-26
|
- **Created:** 2026-05-26
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,296 @@
|
|||||||
|
# PRD 0048: SSH Deploy-Key Provisioning
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis-claude
|
||||||
|
- **Created:** 2026-06-03
|
||||||
|
- **Issue:** #169
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Replace per-repo static SSH identity files with short-lived ed25519 deploy
|
||||||
|
keys that are generated at spin-up and revoked at teardown. Introduce
|
||||||
|
`bot_bottle/contrib/` as the package for platform-specific provisioners and
|
||||||
|
ship the first contrib sub-package: `bot_bottle/contrib/gitea/` with
|
||||||
|
`GiteaDeployKeyProvisioner`. A new `provisioned_key:` block in `git-gate.repos`
|
||||||
|
entries opts a repo into automatic key lifecycle management; `identity:` stays
|
||||||
|
valid for operators who supply their own key material.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The current `git-gate.repos` entries require an `identity:` field pointing to
|
||||||
|
a host-side SSH private key (PRD 0047). Keys are static: the operator generates
|
||||||
|
them once, registers them with the upstream forge, and the same key is reused
|
||||||
|
across every bottle spin-up. This has several consequences:
|
||||||
|
|
||||||
|
- **No automatic revocation.** If a bottle misbehaves or a key leaks, the
|
||||||
|
operator must notice and manually delete the key from the forge. There is no
|
||||||
|
teardown hook that does it.
|
||||||
|
- **Broad blast radius.** A forge deploy key typically grants write access for
|
||||||
|
the lifetime of the key. A static key that survives bottle teardown continues
|
||||||
|
to grant that access.
|
||||||
|
- **Manual rotation burden.** Operators must manage key files on disk, keeping
|
||||||
|
them secure, rotating them on a schedule, and distributing them across hosts
|
||||||
|
that run `./cli.py start`.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `git-gate.repos` entries accept `provisioned_key:` as an alternative to
|
||||||
|
`identity:`. The parser rejects entries that have both, or neither.
|
||||||
|
- `provisioned_key.provider: gitea` provisions and revokes deploy keys via the
|
||||||
|
Gitea HTTP API.
|
||||||
|
- At prepare time the provisioner generates a fresh ed25519 keypair, registers
|
||||||
|
the public half as a repo-scoped deploy key, and makes the private key
|
||||||
|
available to git-gate at the path it expects — the rest of the pipeline is
|
||||||
|
unchanged.
|
||||||
|
- At teardown the provisioner deletes the registered deploy key. Failure to
|
||||||
|
delete halts teardown and propagates the error loudly.
|
||||||
|
- `bot_bottle/contrib/` is introduced as the package for platform-specific
|
||||||
|
implementations; the core defines the abstract interface; contrib sub-packages
|
||||||
|
provide concrete implementations.
|
||||||
|
- Existing `identity:`-based repos continue to work without change.
|
||||||
|
- The unit test suite passes unchanged for `identity:` paths; new tests cover
|
||||||
|
`provisioned_key:` parse, validation, and provisioner dispatch.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- GitHub, GitLab, or other forge providers (a future contrib sub-package each).
|
||||||
|
- Dashboard UI for listing or revoking orphaned deploy keys.
|
||||||
|
- SSH CA certificate approach (rejected in the issue thread in favour of
|
||||||
|
per-repo deploy keys for simpler revocation, smaller blast radius, and forge
|
||||||
|
compatibility).
|
||||||
|
- Key rotation mid-session (keys live for exactly one spin-up / teardown cycle).
|
||||||
|
- Any change to how `identity:` repos are provisioned.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### Manifest changes (builds on PRD 0047)
|
||||||
|
|
||||||
|
`git-gate.repos.<name>` currently accepts exactly:
|
||||||
|
|
||||||
|
```
|
||||||
|
url (required string)
|
||||||
|
identity (required string)
|
||||||
|
host_key (optional string)
|
||||||
|
```
|
||||||
|
|
||||||
|
After this PRD:
|
||||||
|
|
||||||
|
```
|
||||||
|
url (required string)
|
||||||
|
identity (optional string — mutually exclusive with provisioned_key)
|
||||||
|
provisioned_key (optional object — mutually exclusive with identity)
|
||||||
|
host_key (optional string)
|
||||||
|
```
|
||||||
|
|
||||||
|
Exactly one of `identity` or `provisioned_key` must be present. The parser
|
||||||
|
emits a targeted error for each violation:
|
||||||
|
|
||||||
|
```
|
||||||
|
bottle 'dev' git-gate.repos['bot-bottle'] must set exactly one of
|
||||||
|
'identity' or 'provisioned_key'; got neither.
|
||||||
|
|
||||||
|
bottle 'dev' git-gate.repos['bot-bottle'] must set exactly one of
|
||||||
|
'identity' or 'provisioned_key'; got both.
|
||||||
|
```
|
||||||
|
|
||||||
|
`provisioned_key` object schema:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
provisioned_key:
|
||||||
|
provider: gitea # required; names the contrib module to load
|
||||||
|
token_env: GITEA_TOKEN # required; name of a host env var holding the API token
|
||||||
|
api_url: https://... # optional; defaults to https://<host from url>
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Type | Notes |
|
||||||
|
|-------|------|-------|
|
||||||
|
| `provider` | required string | Must match a sub-package under `bot_bottle/contrib/` |
|
||||||
|
| `token_env` | required string | Resolved at provision time via `os.environ`; never stored in plan |
|
||||||
|
| `api_url` | optional string | Override when the API endpoint differs from the git host |
|
||||||
|
|
||||||
|
**Example bottle manifest:**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
git-gate:
|
||||||
|
user:
|
||||||
|
name: implementer-bot
|
||||||
|
email: eric+implementer@dideric.is
|
||||||
|
repos:
|
||||||
|
bot-bottle:
|
||||||
|
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
|
provisioned_key:
|
||||||
|
provider: gitea
|
||||||
|
token_env: GITEA_DEPLOY_TOKEN
|
||||||
|
host_key: "ssh-rsa AAAA..."
|
||||||
|
```
|
||||||
|
|
||||||
|
### `contrib` package structure
|
||||||
|
|
||||||
|
```
|
||||||
|
bot_bottle/
|
||||||
|
contrib/
|
||||||
|
__init__.py # empty; no core symbols
|
||||||
|
gitea/
|
||||||
|
__init__.py # empty
|
||||||
|
deploy_key_provisioner.py
|
||||||
|
```
|
||||||
|
|
||||||
|
`contrib` is a flat namespace of forge/platform sub-packages. Each sub-package
|
||||||
|
is self-contained; the core imports from contrib lazily (inside factory
|
||||||
|
functions) so that missing optional dependencies in a contrib sub-package don't
|
||||||
|
break unrelated features.
|
||||||
|
|
||||||
|
### Core interface
|
||||||
|
|
||||||
|
New file: `bot_bottle/deploy_key_provisioner.py`
|
||||||
|
|
||||||
|
```python
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
|
class DeployKeyProvisioner(ABC):
|
||||||
|
@abstractmethod
|
||||||
|
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||||
|
"""Generate a keypair and register the public half.
|
||||||
|
|
||||||
|
owner_repo: '<owner>/<repo>' portion of the git upstream URL.
|
||||||
|
title: human-readable label shown in the forge key list.
|
||||||
|
|
||||||
|
Returns (key_id, private_key_pem) where key_id is opaque to
|
||||||
|
the caller and is only passed back to delete()."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||||
|
"""Delete the registered deploy key.
|
||||||
|
|
||||||
|
Must not raise if the key is already absent (HTTP 404 is success).
|
||||||
|
Must raise for all other failures so that teardown halts."""
|
||||||
|
|
||||||
|
|
||||||
|
def get_provisioner(provider: str, token: str, api_url: str) -> DeployKeyProvisioner:
|
||||||
|
"""Instantiate the named contrib provisioner.
|
||||||
|
|
||||||
|
Raises ManifestError for unknown providers so the error is caught
|
||||||
|
at parse time rather than at runtime."""
|
||||||
|
if provider == "gitea":
|
||||||
|
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
|
||||||
|
GiteaDeployKeyProvisioner,
|
||||||
|
)
|
||||||
|
return GiteaDeployKeyProvisioner(token=token, api_url=api_url)
|
||||||
|
from .manifest_util import ManifestError
|
||||||
|
raise ManifestError(f"unknown provisioned_key provider: {provider!r}")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Gitea contrib implementation
|
||||||
|
|
||||||
|
`bot_bottle/contrib/gitea/deploy_key_provisioner.py`:
|
||||||
|
|
||||||
|
`create(owner_repo, title)`:
|
||||||
|
1. Generate an ed25519 keypair via `ssh-keygen -t ed25519 -f <tmpfile> -N ''`
|
||||||
|
(uses the SSH tooling already required by git-gate; no new Python dependency).
|
||||||
|
2. Read the private key bytes and the `.pub` file.
|
||||||
|
3. `POST /api/v1/repos/{owner}/{repo}/keys` with the public key, `title`, and
|
||||||
|
`read_only: false` (deploy keys always need push access for git-gate).
|
||||||
|
4. Return `(str(response["id"]), private_key_bytes)`.
|
||||||
|
|
||||||
|
`delete(owner_repo, key_id)`:
|
||||||
|
1. `DELETE /api/v1/repos/{owner}/{repo}/keys/{id}`.
|
||||||
|
2. Treat HTTP 404 as success (key already gone).
|
||||||
|
3. Raise `RuntimeError` for any other non-2xx response or network error,
|
||||||
|
including the status code and response body in the message.
|
||||||
|
|
||||||
|
HTTP calls use `urllib.request` from the stdlib; no new runtime dependency.
|
||||||
|
|
||||||
|
### `GitEntry` dataclass changes
|
||||||
|
|
||||||
|
`bot_bottle/manifest_git.py`:
|
||||||
|
|
||||||
|
- Add `ProvisionedKeyConfig` dataclass:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProvisionedKeyConfig:
|
||||||
|
provider: str
|
||||||
|
token_env: str
|
||||||
|
api_url: str # empty string means "derive from UpstreamHost"
|
||||||
|
```
|
||||||
|
|
||||||
|
- `GitEntry`:
|
||||||
|
- `IdentityFile: str` unchanged internally; empty string when
|
||||||
|
`provisioned_key` is used; set at provision time, not parse time.
|
||||||
|
- New field: `ProvisionedKey: ProvisionedKeyConfig | None = None`
|
||||||
|
- `from_repos_entry` validates the mutually-exclusive constraint and parses
|
||||||
|
the `provisioned_key` block when present.
|
||||||
|
|
||||||
|
### `GitGateUpstream` / prepare-time changes
|
||||||
|
|
||||||
|
`bot_bottle/git_gate.py` and `bot_bottle/backend/docker/provision/git.py`:
|
||||||
|
|
||||||
|
The existing path writes the identity file path into `GitGateUpstream.IdentityFile`
|
||||||
|
and docker-cp's it into `/git-gate/creds/<name>-key`. That path stays unchanged
|
||||||
|
for `identity:` repos.
|
||||||
|
|
||||||
|
For `provisioned_key:` repos, a new helper `provision_deploy_key(entry,
|
||||||
|
stage_dir, bottle_name)` runs before the git-gate sidecar starts:
|
||||||
|
|
||||||
|
1. Resolve `token = os.environ[entry.ProvisionedKey.token_env]`. Missing key
|
||||||
|
raises `RuntimeError` with a clear message naming the env var.
|
||||||
|
2. Resolve `api_url = entry.ProvisionedKey.api_url or f"https://{entry.UpstreamHost}"`.
|
||||||
|
3. Instantiate `get_provisioner(entry.ProvisionedKey.provider, token, api_url)`.
|
||||||
|
4. Call `provisioner.create(entry.UpstreamPath.lstrip("/"), title)` where
|
||||||
|
`title = f"bot-bottle:{bottle_name}:{entry.Name}"`.
|
||||||
|
5. Write private key to `stage_dir / f"{entry.Name}-key"` (mode 0o600).
|
||||||
|
6. Write key ID to `stage_dir / f"{entry.Name}-deploy-key-id"` (plain text).
|
||||||
|
7. Return the key file path; caller sets `GitGateUpstream.IdentityFile` to it.
|
||||||
|
|
||||||
|
`owner_repo` is extracted from `entry.UpstreamPath` (the path component of the
|
||||||
|
`ssh://` URL, e.g. `/didericis/bot-bottle.git` → `didericis/bot-bottle`).
|
||||||
|
|
||||||
|
### Teardown changes
|
||||||
|
|
||||||
|
`bot_bottle/backend/docker/cleanup.py` (or the equivalent teardown path):
|
||||||
|
|
||||||
|
After the git-gate sidecar stops, for each `GitEntry` with `ProvisionedKey`
|
||||||
|
set:
|
||||||
|
|
||||||
|
1. Check that `stage_dir / f"{entry.Name}-deploy-key-id"` exists; skip if
|
||||||
|
absent (provision never ran or already cleaned up).
|
||||||
|
2. Resolve token and API URL as above.
|
||||||
|
3. Instantiate provisioner and call `provisioner.delete(owner_repo, key_id)`.
|
||||||
|
4. On success, log at INFO. On failure, allow the exception to propagate —
|
||||||
|
teardown halts and the error surfaces to the operator.
|
||||||
|
|
||||||
|
A stranded deploy key is a security concern: the operator must know about it
|
||||||
|
and address it manually. Silent continuation is not acceptable.
|
||||||
|
|
||||||
|
The private key file in `stage_dir` is cleaned up as part of normal stage-dir
|
||||||
|
teardown (no extra step needed).
|
||||||
|
|
||||||
|
## Testing strategy
|
||||||
|
|
||||||
|
```
|
||||||
|
python3 -m unittest discover -s tests/unit
|
||||||
|
```
|
||||||
|
|
||||||
|
New / modified test files:
|
||||||
|
|
||||||
|
- `tests/unit/test_manifest_git.py` — add cases for:
|
||||||
|
- `provisioned_key:` accepted with valid `provider`, `token_env`, optional `api_url`
|
||||||
|
- Both `identity` and `provisioned_key` present → `ManifestError`
|
||||||
|
- Neither `identity` nor `provisioned_key` present → `ManifestError`
|
||||||
|
- Unknown key inside `provisioned_key` block → `ManifestError`
|
||||||
|
- Missing `provider` or `token_env` inside `provisioned_key` → `ManifestError`
|
||||||
|
|
||||||
|
- `tests/unit/test_deploy_key_provisioner.py` — new:
|
||||||
|
- `get_provisioner("gitea", ...)` returns `GiteaDeployKeyProvisioner`
|
||||||
|
- `get_provisioner("unknown", ...)` raises `ManifestError`
|
||||||
|
|
||||||
|
- `tests/unit/test_contrib_gitea_deploy_key.py` — new (using `unittest.mock`
|
||||||
|
to stub `urllib.request.urlopen` and `subprocess.run`):
|
||||||
|
- `create()` calls `ssh-keygen`, POSTs to correct endpoint, returns key ID
|
||||||
|
- `delete()` DELETEs to correct endpoint
|
||||||
|
- `delete()` tolerates HTTP 404 (already-deleted key)
|
||||||
|
- `delete()` raises `RuntimeError` on non-404 HTTP error
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
None.
|
||||||
@@ -0,0 +1,343 @@
|
|||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** didericis
|
||||||
|
- **Created:** 2026-06-03
|
||||||
|
- **Issue:** #174
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
The `./cli.py dashboard` command has grown from its PRD 0013 roots
|
||||||
|
(triage supervise proposals) into a parallel-agent control surface
|
||||||
|
(PRDs 0019/0020/0021): an active-agents pane, agent picker + start,
|
||||||
|
re-attach, per-bottle stop, tmux split-pane handoff, operator-
|
||||||
|
initiated `routes`/`pipelock` edits. Each chunk is reasonable on its
|
||||||
|
own; together they make the dashboard the largest CLI file in the
|
||||||
|
repo and the thing most likely to break on a rough edge (curses /
|
||||||
|
tmux / docker-exec / metadata-discovery interactions).
|
||||||
|
|
||||||
|
This PRD reverses that scope creep. The dashboard is reduced to the
|
||||||
|
**supervise-plane triage TUI** it was in PRDs 0013–0016: list pending
|
||||||
|
proposals, approve / modify / reject each one, write audit entries,
|
||||||
|
deliver the response that unblocks the agent's tool call. Everything
|
||||||
|
that's about *starting / re-entering / stopping* bottles, or about
|
||||||
|
*operator-initiated* config edits, comes out. The command is renamed
|
||||||
|
`./cli.py supervise` so the name matches what it does after the cut.
|
||||||
|
|
||||||
|
Future agent-management UX is explicitly punted: if and when a
|
||||||
|
control surface for parallel agents resurfaces, the working
|
||||||
|
assumption (per the issue) is that a web GUI — usable from mobile
|
||||||
|
— is a better second pass than another round of curses iteration.
|
||||||
|
That decision is not in this PRD's scope; this PRD only removes the
|
||||||
|
half-built local-curses path so we stop maintaining it.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Three concrete pains, all downstream of the dashboard's growth:
|
||||||
|
|
||||||
|
1. **Surface area vs. polish.** `dashboard.py` is ~1740 lines;
|
||||||
|
`dashboard_model.py` adds another ~420. The interactions among
|
||||||
|
curses, modals, tmux split-pane, docker-exec handoff, agent
|
||||||
|
provider templates, metadata-driven re-attach, and
|
||||||
|
ExitStack-free bottle ownership are intricate enough that
|
||||||
|
shipping the next polish increment costs more than it returns.
|
||||||
|
2. **No clear ownership of "starts and stops bottles".** Today
|
||||||
|
that responsibility is split: `./cli.py start` owns one-shot
|
||||||
|
sessions; the dashboard owns multi-session bottles it started
|
||||||
|
itself; `./cli.py cleanup` owns everything else. The dashboard
|
||||||
|
tracking its own `bottles: dict[str, (cm, bottle, identity)]`
|
||||||
|
that doesn't survive a quit is a confusing third lane.
|
||||||
|
3. **Wrong target shape for a "manage many agents" UI.** The
|
||||||
|
parallel-agent experience the dashboard reaches for is mobile-
|
||||||
|
meaningful — checking in on agents from a phone is the high-
|
||||||
|
value case — and curses inside an SSH session is the wrong
|
||||||
|
tool for that. Continuing to polish a local-only TUI delays
|
||||||
|
the right next investment.
|
||||||
|
|
||||||
|
The triage half of the dashboard isn't suffering from any of these.
|
||||||
|
Pending proposals are a small, well-scoped, real workload, and the
|
||||||
|
PRD 0013–0016 surface for handling them is the right shape. The
|
||||||
|
problem is everything that got bolted onto that core after.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
1. The supervise TUI starts up, lists pending proposals across all
|
||||||
|
running bottles, and supports approve / modify / reject + the
|
||||||
|
`--once` non-interactive mode — exactly as PRDs 0013–0016
|
||||||
|
specified, minus everything 0019/0020/0021 added.
|
||||||
|
2. The CLI subcommand is renamed `supervise` (was `dashboard`). The
|
||||||
|
old name is not aliased — this PRD is intentionally a
|
||||||
|
compat/breaking change (the issue carries the
|
||||||
|
`Compat/Breaking` label).
|
||||||
|
3. `dashboard.py` shrinks to a single proposal-triage curses loop:
|
||||||
|
no agents pane, no Tab pane switching, no agent picker, no
|
||||||
|
start / re-attach / stop verbs, no tmux split-pane, no
|
||||||
|
`e`/`p` operator-edit verbs, no per-process `bottles` dict.
|
||||||
|
4. `dashboard_model.py` is collapsed into whatever
|
||||||
|
`supervise.py` (CLI) needs; the model module is removed if it
|
||||||
|
has no purpose after the cut.
|
||||||
|
5. The proposal-side apply paths in `bot_bottle/backend/docker/
|
||||||
|
egress_apply.py`, `pipelock_apply.py`, and `capability_apply.py`
|
||||||
|
are unchanged — they are still called by the approve path.
|
||||||
|
6. The supervise-sidecar / proposal-queue protocol (PRD 0013) is
|
||||||
|
unchanged: the agent's experience is identical.
|
||||||
|
7. The previously-active PRDs that this one undoes are marked
|
||||||
|
`Superseded by PRD 0049`:
|
||||||
|
- PRD 0019 — active-agents pane + agent-scoped edit verbs
|
||||||
|
- PRD 0020 — start / re-attach / stop from the dashboard
|
||||||
|
- PRD 0021 — tmux split-pane
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- **A web GUI for managing agents.** The issue floats this as a
|
||||||
|
second pass; this PRD does not design or commit to it. The cut
|
||||||
|
is "remove the path we no longer want to invest in", not
|
||||||
|
"build the replacement".
|
||||||
|
- **A separate CLI for operator-initiated routes / pipelock
|
||||||
|
edits.** Today those edits live as `e` / `p` keys inside the
|
||||||
|
dashboard. After this PRD they don't exist anywhere — operators
|
||||||
|
who need ad-hoc edits use the same path the agents do (call the
|
||||||
|
supervise tool from inside the bottle) or hand-edit the host-
|
||||||
|
side files and restart the sidecar. Adding a `./cli.py routes
|
||||||
|
edit <slug>` verb is a follow-up if the loss bites.
|
||||||
|
- **Removing `./cli.py start` or changing its semantics.** Start
|
||||||
|
remains the one-shot launch path. PRD 0020's bottle-outlives-
|
||||||
|
process model is removed; the only path to a long-running
|
||||||
|
bottle is `./cli.py start` (foreground) plus `cli.py cleanup`
|
||||||
|
for teardown.
|
||||||
|
- **Removing the supervise-sidecar protocol or any of the three
|
||||||
|
block-remediation engines.** PRDs 0013–0016 stay Active. The
|
||||||
|
agent's view of the world doesn't change.
|
||||||
|
- **Renaming `dashboard` anywhere other than the CLI entry
|
||||||
|
point.** The dashboard-related docs (PRDs, decision records,
|
||||||
|
research notes) keep their historical references — they
|
||||||
|
describe the state of the world at the time they were written,
|
||||||
|
and the Status: Superseded line is the marker that the world
|
||||||
|
has moved on.
|
||||||
|
- **Migrating the proposal-queue file layout.** The queue still
|
||||||
|
lives at `~/.bot-bottle/queue/<slug>/`; the audit log still
|
||||||
|
lives at `~/.bot-bottle/audit/<component>-<slug>.log`. The CLI
|
||||||
|
surface changes; the on-disk surface does not.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
### In scope
|
||||||
|
|
||||||
|
- **Rename the subcommand.** `./cli.py dashboard` becomes
|
||||||
|
`./cli.py supervise`. The module moves from `bot_bottle/cli/
|
||||||
|
dashboard.py` to `bot_bottle/cli/supervise.py`. The dispatcher
|
||||||
|
in `bot_bottle/cli/__init__.py` and the help text both update.
|
||||||
|
- **Strip the curses loop to proposal-only.** The remaining
|
||||||
|
surface is: list pending proposals (with the new-arrival bell
|
||||||
|
from PRD 0013), Enter for detail view,
|
||||||
|
`a`/`m`/`r` for approve / modify / reject, `q` to quit. No
|
||||||
|
agents pane, no Tab, no agent picker, no `n`/`x`/`e`/`p`, no
|
||||||
|
tmux dispatch, no `bottles` dict on the main loop.
|
||||||
|
- **Drop unused helpers.** `_picker_modal`, `_preflight_modal`,
|
||||||
|
`_backend_picker_modal`, `_new_agent_flow`, `_attach_to_bottle`,
|
||||||
|
`_attach_in_tmux`, `_attach_via_handoff`, `_tmux_*`,
|
||||||
|
`_ensure_right_pane`, `_redirect_stderr_to_file`,
|
||||||
|
`_route_op_to_right_pane`, `_stop_bottle_flow`,
|
||||||
|
`_operator_edit_*_flow`, `operator_edit_routes`,
|
||||||
|
`operator_edit_allowlist`, and their imports come out.
|
||||||
|
- **Collapse the model module.** `dashboard_model.py`'s
|
||||||
|
proposal-side helpers (`QueuedProposal`, `discover_pending`,
|
||||||
|
`_approval_status`, `_detail_lines`,
|
||||||
|
`_failed_url_host`, `_proposed_payload_label`,
|
||||||
|
`_suffix_for_tool`, `_REFRESH_INTERVAL_MS`) move back into
|
||||||
|
`supervise.py` (CLI) or into `bot_bottle/supervise.py`
|
||||||
|
(the daemon-side module) — wherever they fit. The agents /
|
||||||
|
picker / tmux helpers in that module (`PANE_*`,
|
||||||
|
`_filter_agents`, `_running_counts`, `_format_agent_row`,
|
||||||
|
`_selection_status`, `_selected_agent`, `_bottle_for_slug`,
|
||||||
|
`_pick_next_after_stop`, `_agent_runtime_args`,
|
||||||
|
`_build_resume_argv_with_fallback`, `_build_split_pane_argv`,
|
||||||
|
`_build_respawn_pane_argv`, `_in_tmux`,
|
||||||
|
`discover_active_agents`) are deleted.
|
||||||
|
- **Mark superseded PRDs.** The Status line on PRDs 0019, 0020,
|
||||||
|
and 0021 changes to `Superseded by [PRD 0049](0049-strip-
|
||||||
|
dashboard-to-supervisor-tui.md)`.
|
||||||
|
- **Test cleanup.** Any test that targets a removed surface (the
|
||||||
|
agent picker, the tmux split helpers, the start-from-dashboard
|
||||||
|
flow, the operator-edit flows, `discover_active_agents`)
|
||||||
|
comes out. Tests covering proposal triage stay.
|
||||||
|
- **Help / usage strings.** `bot_bottle/cli/__init__.py`'s usage
|
||||||
|
block updates the command name and one-liner.
|
||||||
|
|
||||||
|
### Out of scope
|
||||||
|
|
||||||
|
- Any new feature in the supervise TUI. The cut is purely
|
||||||
|
subtractive (except for the rename).
|
||||||
|
- Behavior changes in `./cli.py start`, `cli.py cleanup`,
|
||||||
|
`cli.py resume`, `cli.py list`, `cli.py info`, `cli.py edit`,
|
||||||
|
`cli.py init` — unchanged.
|
||||||
|
- Changes to the supervise sidecar (`supervise_server.py`,
|
||||||
|
`supervise.py` daemon module). The wire protocol stays.
|
||||||
|
- Changes to the routes / pipelock / capability apply engines.
|
||||||
|
- Migration helpers, deprecation warnings, or a transitional
|
||||||
|
`dashboard` alias for `supervise`. The label on the issue says
|
||||||
|
Compat/Breaking; the rename is a hard cutover.
|
||||||
|
|
||||||
|
## Proposed design
|
||||||
|
|
||||||
|
### Final shape of the TUI
|
||||||
|
|
||||||
|
After this PRD the `./cli.py supervise` curses surface is:
|
||||||
|
|
||||||
|
```
|
||||||
|
bot-bottle supervise (3 pending)
|
||||||
|
─────────────────────────────────────────────────────────
|
||||||
|
> 03:14:22 [implementer-cy7a6] egress-block abc123… add
|
||||||
|
github.com/foo
|
||||||
|
03:13:55 [researcher-9xqs1] pipelock-block def456… allow
|
||||||
|
registry.npmjs.org
|
||||||
|
03:13:10 [implementer-cy7a6] capability-block ghi789… install
|
||||||
|
ripgrep
|
||||||
|
|
||||||
|
─────────────────────────────────────────────────────────
|
||||||
|
[j/k] move [Enter] view [a] approve [m] modify [r] reject [q] quit
|
||||||
|
```
|
||||||
|
|
||||||
|
- One pane. No Tab. `j` / `k` / arrows move through the queue.
|
||||||
|
- Enter opens the existing detail view (justification +
|
||||||
|
proposed-file body + the green pipelock host-extraction hint).
|
||||||
|
`a` / `m` / `r` work from both the list view and the detail
|
||||||
|
view, same as today.
|
||||||
|
- `q` / Esc quits. There are no dashboard-owned bottles, so no
|
||||||
|
per-process teardown decision — `q` just exits.
|
||||||
|
- The new-arrival bell stays, because it is a real win for the
|
||||||
|
operator's "I was typing at claude and a proposal landed" case.
|
||||||
|
No tmux-specific focus management remains.
|
||||||
|
|
||||||
|
### Code organisation
|
||||||
|
|
||||||
|
After the cut, the CLI module looks roughly like:
|
||||||
|
|
||||||
|
```
|
||||||
|
bot_bottle/cli/supervise.py
|
||||||
|
- cmd_supervise(argv)
|
||||||
|
- _list_once() # --once mode
|
||||||
|
- _main_loop(stdscr) # proposal-only
|
||||||
|
- _render(stdscr, pending, ...)
|
||||||
|
- _detail_view(stdscr, qp, ...)
|
||||||
|
- _modify(stdscr, qp)
|
||||||
|
- _prompt(stdscr, label)
|
||||||
|
- _write_crash_log(exc)
|
||||||
|
- approve(qp, *, notes, final_file)
|
||||||
|
- reject(qp, *, reason)
|
||||||
|
- QueuedProposal, discover_pending
|
||||||
|
- _detail_lines, _approval_status,
|
||||||
|
_failed_url_host,
|
||||||
|
_proposed_payload_label,
|
||||||
|
_suffix_for_tool
|
||||||
|
```
|
||||||
|
|
||||||
|
`dashboard_model.py` has no purpose once the agents / picker /
|
||||||
|
tmux helpers are gone, so it is removed and the surviving
|
||||||
|
proposal-side helpers move into `supervise.py` directly. The
|
||||||
|
PRD-0013 refactor that split model out (`refactor: extract
|
||||||
|
dashboard state/model layer into dashboard_model.py`) was
|
||||||
|
load-bearing for the bigger dashboard surface; with the surface
|
||||||
|
shrunk back, the split is no longer justified.
|
||||||
|
|
||||||
|
### Removed PRDs: how to mark them
|
||||||
|
|
||||||
|
The three superseded PRDs keep their bodies intact. Only the
|
||||||
|
Status line at the top changes:
|
||||||
|
|
||||||
|
```
|
||||||
|
- **Status:** Superseded by [PRD
|
||||||
|
0049](0049-strip-dashboard-to-supervisor-tui.md)
|
||||||
|
```
|
||||||
|
|
||||||
|
The PRD's own Goals / Success Criteria are left as the historical
|
||||||
|
record of what the feature shipped — readers tracing back from the
|
||||||
|
code or the git log land in a PRD that explains what once was, with
|
||||||
|
a clear pointer forward. No PRD body is rewritten.
|
||||||
|
|
||||||
|
### Tests to keep, tests to remove
|
||||||
|
|
||||||
|
Keep:
|
||||||
|
- `tests/cli/test_dashboard*.py` cases that exercise
|
||||||
|
`discover_pending`, `approve`, `reject`, `_detail_lines`,
|
||||||
|
`_approval_status`, `_failed_url_host`,
|
||||||
|
`_proposed_payload_label`, `_suffix_for_tool`,
|
||||||
|
`_modify` / `edit_in_editor`.
|
||||||
|
- `tests/cli/test_dashboard_once.py` (or equivalent) — the
|
||||||
|
`--once` listing mode.
|
||||||
|
|
||||||
|
Remove:
|
||||||
|
- Any test of `_picker_modal`, `_preflight_modal`,
|
||||||
|
`_backend_picker_modal`, `_new_agent_flow`, `_attach_*`,
|
||||||
|
`_tmux_*`, `_route_op_to_right_pane`,
|
||||||
|
`_redirect_stderr_to_file`, `_stop_bottle_flow`,
|
||||||
|
`_operator_edit_*`, `_filter_agents`, `_running_counts`,
|
||||||
|
`_format_agent_row`, `_selection_status`,
|
||||||
|
`_selected_agent`, `_bottle_for_slug`,
|
||||||
|
`_pick_next_after_stop`, `_agent_runtime_args`,
|
||||||
|
`_build_*_argv`, `discover_active_agents`.
|
||||||
|
- The test files that exist solely to cover those (e.g.,
|
||||||
|
`test_dashboard_picker.py`, `test_dashboard_tmux.py`,
|
||||||
|
`test_dashboard_attach.py`, `test_dashboard_agents.py` —
|
||||||
|
whichever of these exist after the file walk).
|
||||||
|
|
||||||
|
Files are renamed `test_supervise_*.py` to mirror the module
|
||||||
|
rename. The rename is mechanical; no test logic changes.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
Sized for a single PR each.
|
||||||
|
|
||||||
|
1. **Strip + rename in one cut.** Move `bot_bottle/cli/
|
||||||
|
dashboard.py` to `bot_bottle/cli/supervise.py`, delete the
|
||||||
|
removed helpers, delete `dashboard_model.py`, inline the
|
||||||
|
surviving helpers, update the dispatcher + usage in
|
||||||
|
`bot_bottle/cli/__init__.py`, rename tests to match, mark
|
||||||
|
PRDs 0019/0020/0021 as superseded. One commit per logical
|
||||||
|
piece inside the PR (rename, strip, supersede notes,
|
||||||
|
tests).
|
||||||
|
2. **Activate PRD 0049.** Flip this PRD's Status line from
|
||||||
|
Draft to Active in the same PR as chunk 1 once the
|
||||||
|
implementation lands. (The repo convention is that a PRD's
|
||||||
|
shipping commit is also the Status flip — see the recent
|
||||||
|
`docs(prd): activate PRD 0048…` commit shape.)
|
||||||
|
|
||||||
|
The PR closes issue #174.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
1. **`e` / `p` operator-initiated edits — gone for good or
|
||||||
|
moved to a separate CLI verb?** The PRD removes them with no
|
||||||
|
replacement. The simplest replacement is `./cli.py routes
|
||||||
|
edit <slug>` and `./cli.py pipelock edit <slug>`, sharing
|
||||||
|
the existing `apply_routes_change` / `apply_allowlist_change`
|
||||||
|
engines. If the loss is felt within the first parallel
|
||||||
|
run after this lands, that follow-up is a small PR. Leaving
|
||||||
|
it for a separate PRD so this one stays subtractive.
|
||||||
|
|
||||||
|
2. **`--once` output shape.** The text listing today emits one
|
||||||
|
proposal per line. Worth keeping exactly as-is for
|
||||||
|
scripting consumers; this PRD does not change it. Flagging
|
||||||
|
only because the rename could tempt a tweak.
|
||||||
|
|
||||||
|
3. **Audit-log entry shape for an unprompted edit applied via
|
||||||
|
a future `routes edit` CLI verb.** Today's
|
||||||
|
`operator_edit_routes` writes an `ACTION_OPERATOR_EDIT`
|
||||||
|
audit entry. With those flows removed the constant has no
|
||||||
|
callers inside this PRD's scope. Keep the constant exported
|
||||||
|
from `supervise.py` (it's already an `__all__` member) so a
|
||||||
|
follow-up CLI verb can re-use the same audit shape without
|
||||||
|
re-introducing dead code first.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Issue
|
||||||
|
[#174](https://gitea.dideric.is/didericis/bot-bottle/issues/174)
|
||||||
|
— the request: "strip the dashboard down into just a TUI for
|
||||||
|
managing agent requests for new egress routes and new
|
||||||
|
capabilities."
|
||||||
|
- PRD 0013 — supervise plane foundation (the floor this PRD
|
||||||
|
reverts the dashboard to).
|
||||||
|
- PRDs 0014 / 0015 / 0016 — block-remediation engines that the
|
||||||
|
supervise TUI continues to drive on approve.
|
||||||
|
- PRDs 0019 / 0020 / 0021 — the bolted-on capabilities this PRD
|
||||||
|
removes.
|
||||||
@@ -0,0 +1,401 @@
|
|||||||
|
# PRD 0050: Move provider-specific agent logic into contrib
|
||||||
|
|
||||||
|
- **Status:** Active
|
||||||
|
- **Author:** claude
|
||||||
|
- **Created:** 2026-06-03
|
||||||
|
- **Issue:** #177
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
The agent provider module (`bot_bottle/agent_provider.py`) hard-codes
|
||||||
|
the Claude- and Codex-specific provisioning rules — auth file shapes,
|
||||||
|
trust-dialog markers, egress routes, dummy-auth dance, env vars — in a
|
||||||
|
single `if template == "codex": ... if template == "claude": ...`
|
||||||
|
chain (lines 154–230 today). Other pieces of provider behavior live in
|
||||||
|
each backend's `provision/` directory (`provision_skills`,
|
||||||
|
`provision_prompt`, `provision_provider_auth`, `provision_supervise`),
|
||||||
|
duplicated once per backend, even though almost none of what they do
|
||||||
|
is actually backend-specific.
|
||||||
|
|
||||||
|
This PRD reshapes the agent provider into a proper plugin boundary.
|
||||||
|
The two existing providers (Claude, Codex) move out of `agent_provider`
|
||||||
|
into `bot_bottle/contrib/claude/` and `bot_bottle/contrib/codex/` —
|
||||||
|
the same `contrib/` layout PRD 0048 established for the Gitea
|
||||||
|
deploy-key provisioner. The four provisioner methods backends
|
||||||
|
currently duplicate move into the provider plugin itself; the backend
|
||||||
|
keeps only the bottle-side primitives (`cp_in`, `exec`) the plugin
|
||||||
|
calls through. MCP server registration becomes a first-class part of
|
||||||
|
the provider contract so Codex finally gets the supervise sidecar
|
||||||
|
wired in alongside Claude.
|
||||||
|
|
||||||
|
The shipping artifact is two new provider plugins under `contrib/`, a
|
||||||
|
narrower `AgentProvider` ABC in `bot_bottle/agent_provider.py`, four
|
||||||
|
fewer provisioner hooks on `BottleBackend`, and a supervise-MCP entry
|
||||||
|
visible from the Codex agent at launch.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Three concrete pains, all downstream of the provider abstraction not
|
||||||
|
being where the work happens:
|
||||||
|
|
||||||
|
1. **Adding a third provider is a five-file edit.** A hypothetical
|
||||||
|
Gemini or Aider provider has to: (a) add a branch in
|
||||||
|
`agent_provision_plan`, (b) add a runtime entry in `_RUNTIMES`,
|
||||||
|
(c) thread a `prompt_mode` enum value, (d) potentially extend
|
||||||
|
`provision_provider_auth` per backend, (e) wire MCP registration
|
||||||
|
into both `backend/docker/provision/supervise.py` and
|
||||||
|
`backend/smolmachines/provision/supervise.py`. Nothing about that
|
||||||
|
spread is load-bearing; it's leftover from when there was one
|
||||||
|
provider.
|
||||||
|
|
||||||
|
2. **MCP server registration is Claude-only.** Both
|
||||||
|
`backend/docker/provision/supervise.py` and
|
||||||
|
`backend/smolmachines/provision/supervise.py` run `claude mcp add`
|
||||||
|
verbatim. Codex bottles silently get no MCP entry — the sidecar
|
||||||
|
is running, the routes are open, but the agent can't see the
|
||||||
|
tools because nothing wrote them into Codex's TOML config. Today
|
||||||
|
this is a latent gap. The provider plugin is the only layer that
|
||||||
|
knows how a given agent discovers MCP servers, so that's where
|
||||||
|
the registration belongs.
|
||||||
|
|
||||||
|
3. **`provision_skills` / `provision_prompt` / `provision_provider_auth`
|
||||||
|
are duplicated between backends.** Each backend has its own
|
||||||
|
~50-line copy. The differences are entirely about which path the
|
||||||
|
backend uses for `cp_in` and what user it `chown`s to. Same
|
||||||
|
business logic, two implementations, two test surfaces, two
|
||||||
|
places to update when the rules change.
|
||||||
|
|
||||||
|
The agent_provider module is the right home for all of this. It already
|
||||||
|
owns the `AgentProvisionPlan` (the declarative description of what
|
||||||
|
needs to land in the guest); extending it to own the imperative
|
||||||
|
"actually land it" step is the natural next move. Putting
|
||||||
|
provider-specific code under `contrib/` mirrors the convention PRD 0048
|
||||||
|
established and keeps the core package provider-agnostic.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
1. `bot_bottle/agent_provider.py` contains no Claude- or
|
||||||
|
Codex-specific branches. The Claude and Codex template strings
|
||||||
|
themselves still live in the core module (they're the public
|
||||||
|
manifest values), but everything keyed off them moves out.
|
||||||
|
2. `bot_bottle/contrib/claude/agent_provider.py` and
|
||||||
|
`bot_bottle/contrib/codex/agent_provider.py` exist and contain
|
||||||
|
the provider-specific behavior previously in lines 154–230 of
|
||||||
|
`agent_provider.py`. Each is reachable from the core registry via
|
||||||
|
a lazy import (the same pattern PRD 0048 used for
|
||||||
|
`GiteaDeployKeyProvisioner`).
|
||||||
|
3. `AgentProvider` is an ABC (or protocol) with at minimum:
|
||||||
|
- `provision_plan(...) -> AgentProvisionPlan` — what the existing
|
||||||
|
`agent_provision_plan` produces today, scoped to one provider.
|
||||||
|
- `provision_skills(bottle, plan)` — copy host skills into the guest.
|
||||||
|
- `provision_prompt(bottle, plan)` — copy the prompt file, return
|
||||||
|
the in-guest path (or None).
|
||||||
|
- `provision_supervise_mcp(bottle, plan, supervise_url)` — register
|
||||||
|
the supervise sidecar in the provider's MCP config. No-op when
|
||||||
|
the bottle has no supervise sidecar.
|
||||||
|
- The Claude implementation runs `claude mcp add`. The Codex
|
||||||
|
implementation writes the corresponding entry into
|
||||||
|
`~/.codex/config.toml`'s `[mcp_servers.supervise]` table.
|
||||||
|
4. `BottleBackend` loses the four abstract methods being moved
|
||||||
|
(`provision_skills`, `provision_prompt`, `provision_provider_auth`,
|
||||||
|
`provision_supervise`). `BottleBackend.provision_in_bottle` calls
|
||||||
|
the provider plugin directly via the bottle and plan it already
|
||||||
|
has. `provision_ca`, `provision_workspace`, and `provision_git`
|
||||||
|
stay on the backend — they're backend infrastructure, not
|
||||||
|
provider behavior.
|
||||||
|
5. `bot_bottle/backend/docker/provision/{skills,prompt,provider_auth,
|
||||||
|
supervise}.py` and `bot_bottle/backend/smolmachines/provision/{skills,
|
||||||
|
prompt,provider_auth,supervise}.py` are deleted. The
|
||||||
|
backend-specific provisioners that remain (`ca`, `git`,
|
||||||
|
`workspace`) stay.
|
||||||
|
6. A Codex bottle launched with `--supervise` shows the
|
||||||
|
supervise MCP server entry in its Codex config and can call
|
||||||
|
supervise tools from inside the bottle (egress-block,
|
||||||
|
pipelock-block, capability-block).
|
||||||
|
7. Existing tests for the moved logic move with the code:
|
||||||
|
provider-specific tests under `tests/unit/test_contrib_claude_*.py`
|
||||||
|
and `tests/unit/test_contrib_codex_*.py`, mirroring
|
||||||
|
`tests/unit/test_contrib_gitea_deploy_key.py`.
|
||||||
|
8. PRD 0050's Status flips Draft → Active in the same commit that
|
||||||
|
removes the last `if template == "claude"` branch from
|
||||||
|
`agent_provider.py`.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- **A third agent provider.** This PRD reshapes the boundary so a
|
||||||
|
third provider is cheap to add. It does not add one.
|
||||||
|
- **Changing the manifest surface.** The `agent.provider`
|
||||||
|
manifest field still takes `"claude"` or `"codex"`. The set of
|
||||||
|
valid strings is unchanged.
|
||||||
|
- **Changing `AgentProvisionPlan`'s shape.** The dataclasses
|
||||||
|
(`AgentProvisionDir`, `AgentProvisionFile`, `AgentProvisionCommand`,
|
||||||
|
`AgentProvisionPlan` itself) stay in the core module and keep their
|
||||||
|
current fields. Provider plugins produce the same plan shape; only
|
||||||
|
the producer moves.
|
||||||
|
- **Changing the supervise sidecar protocol or the supervise tool
|
||||||
|
surface.** PRDs 0013–0016 stay Active. What changes is how the
|
||||||
|
agent discovers the sidecar's MCP endpoint, not what it does once
|
||||||
|
connected.
|
||||||
|
- **Per-skill provider differences.** A Codex agent and a Claude
|
||||||
|
agent see the same `~/.claude/skills/<name>/` tree today (Codex
|
||||||
|
reads it via its own skills mechanism). This PRD does not change
|
||||||
|
that — `provision_skills` lands the same content for both.
|
||||||
|
- **Removing the `prompt_args` helper from `agent_provider.py`.** It
|
||||||
|
stays at module scope; it's already a pure dispatch on `prompt_mode`
|
||||||
|
and has no Claude/Codex `if` chain to extract.
|
||||||
|
- **`provision_provider_auth` migration.** The issue notes this method
|
||||||
|
is "probably not needed anymore" once each provider owns its own
|
||||||
|
provisioning. After the move, the work that
|
||||||
|
`provision_provider_auth` did (apply `dirs` / `files` / `pre_copy` /
|
||||||
|
`verify` from the plan) becomes a shared helper the per-provider
|
||||||
|
`provision_skills` / `provision_prompt` calls dispatch through —
|
||||||
|
or, more likely, a single `provision(bottle)` entry point on the
|
||||||
|
provider. The hook is removed from `BottleBackend`; whether the
|
||||||
|
underlying loop lives on `AgentProvider` as a default
|
||||||
|
implementation or as a free function in `contrib/_apply.py` is
|
||||||
|
decided at implementation time, not in this PRD.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
### In scope
|
||||||
|
|
||||||
|
- New `AgentProvider` ABC in `bot_bottle/agent_provider.py` with the
|
||||||
|
five methods listed under Goal 3. Existing `agent_provision_plan`
|
||||||
|
becomes `AgentProvider.provision_plan`.
|
||||||
|
- New `bot_bottle/contrib/claude/__init__.py`,
|
||||||
|
`bot_bottle/contrib/claude/agent_provider.py`,
|
||||||
|
`bot_bottle/contrib/codex/__init__.py`,
|
||||||
|
`bot_bottle/contrib/codex/agent_provider.py`. Each defines a
|
||||||
|
`ClaudeAgentProvider` / `CodexAgentProvider` class.
|
||||||
|
- A `get_provider(template) -> AgentProvider` registry in
|
||||||
|
`bot_bottle/agent_provider.py`, lazy-imported from `contrib/`,
|
||||||
|
mirroring `get_provisioner(provider, ...)` in
|
||||||
|
`bot_bottle/deploy_key_provisioner.py`.
|
||||||
|
- Backend changes:
|
||||||
|
- `BottleBackend.provision_in_bottle` resolves the provider once
|
||||||
|
and calls `provider.provision_skills(bottle, plan)`,
|
||||||
|
`provider.provision_prompt(bottle, plan)`, and
|
||||||
|
`provider.provision_supervise_mcp(bottle, plan, url)` in place
|
||||||
|
of the current four abstract hooks.
|
||||||
|
- `BottleBackend.provision_skills`, `provision_prompt`,
|
||||||
|
`provision_provider_auth`, `provision_supervise` are removed.
|
||||||
|
- Docker and smolmachines backends remove their corresponding
|
||||||
|
`provision_*` implementations and the
|
||||||
|
`backend/<name>/provision/{skills,prompt,provider_auth,
|
||||||
|
supervise}.py` modules.
|
||||||
|
- Codex MCP wiring: `CodexAgentProvider.provision_supervise_mcp`
|
||||||
|
writes a `[mcp_servers.supervise]` block into
|
||||||
|
`~/.codex/config.toml` pointing at the same agent-side supervise
|
||||||
|
URL the Claude provider uses. The file already exists from the
|
||||||
|
trust-dialog step; the MCP entry is appended (or the file is
|
||||||
|
rewritten in a single shot, whichever's simpler).
|
||||||
|
- Tests migrate. Backend tests that targeted the four moved
|
||||||
|
provisioners are rewritten against the provider plugin, with one
|
||||||
|
test file per provider mirroring `tests/unit/test_contrib_gitea_*.py`.
|
||||||
|
|
||||||
|
### Out of scope
|
||||||
|
|
||||||
|
- Adding a manifest field for "extra MCP servers the agent should
|
||||||
|
see". The supervise sidecar is the only MCP server provisioned
|
||||||
|
today, and the issue's "Add mcp server configuring into agent
|
||||||
|
provision" line is about the supervise sidecar specifically. A
|
||||||
|
general-purpose user-declared MCP list is a follow-up if and when
|
||||||
|
the need surfaces.
|
||||||
|
- Refactoring `AgentProvisionPlan`'s dataclasses. They stay byte-
|
||||||
|
for-byte the same so the diff is purely "who owns the producer".
|
||||||
|
- A `BottleBackend.provision_provider_auth` shim during transition.
|
||||||
|
The hook is removed in one cut; the only caller is the backend
|
||||||
|
itself, no manifest consumers reference it.
|
||||||
|
- Renaming `agent_provider.py` → `agent_providers/`. The module
|
||||||
|
still has core dataclasses + the ABC + the registry; it's a single
|
||||||
|
file's worth of code.
|
||||||
|
|
||||||
|
## Proposed design
|
||||||
|
|
||||||
|
### Module shape after the cut
|
||||||
|
|
||||||
|
```
|
||||||
|
bot_bottle/agent_provider.py
|
||||||
|
PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_TEMPLATES
|
||||||
|
PromptMode (Literal)
|
||||||
|
AgentProvisionDir, AgentProvisionFile, AgentProvisionCommand,
|
||||||
|
AgentProvisionPlan (dataclasses, unchanged)
|
||||||
|
AgentProviderRuntime (dataclass — template/command/image/etc.)
|
||||||
|
AgentProvider (ABC)
|
||||||
|
.runtime() -> AgentProviderRuntime
|
||||||
|
.provision_plan(state_dir, ..., trusted_project_path, ...) -> AgentProvisionPlan
|
||||||
|
.provision_skills(bottle, plan) -> None
|
||||||
|
.provision_prompt(bottle, plan) -> str | None
|
||||||
|
.provision_supervise_mcp(bottle, plan, supervise_url) -> None
|
||||||
|
get_provider(template: str) -> AgentProvider # lazy-imports contrib
|
||||||
|
prompt_args(prompt_mode, prompt_path, *, argv) # unchanged
|
||||||
|
|
||||||
|
bot_bottle/contrib/claude/agent_provider.py
|
||||||
|
ClaudeAgentProvider(AgentProvider)
|
||||||
|
_RUNTIME = AgentProviderRuntime(template="claude", ...)
|
||||||
|
.provision_plan(...) # owns the lines-204–230 chunk
|
||||||
|
.provision_skills(...) # was backend/<name>/provision/skills.py
|
||||||
|
.provision_prompt(...) # was backend/<name>/provision/prompt.py
|
||||||
|
.provision_supervise_mcp(...)# was backend/<name>/provision/supervise.py
|
||||||
|
|
||||||
|
bot_bottle/contrib/codex/agent_provider.py
|
||||||
|
CodexAgentProvider(AgentProvider)
|
||||||
|
_RUNTIME = AgentProviderRuntime(template="codex", ...)
|
||||||
|
.provision_plan(...) # owns the lines-154–204 chunk
|
||||||
|
.provision_skills(...) # same as Claude impl, factored to shared helper
|
||||||
|
.provision_prompt(...) # same as Claude impl, factored to shared helper
|
||||||
|
.provision_supervise_mcp(...)# writes [mcp_servers.supervise] to config.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
The skills / prompt / provider-auth-apply implementations are 99%
|
||||||
|
identical across providers — `cp_in` then `chown` / `chmod`. They are
|
||||||
|
extracted to small free functions in
|
||||||
|
`bot_bottle/contrib/_provision_apply.py` (or kept as default
|
||||||
|
implementations on `AgentProvider` if every concrete subclass would
|
||||||
|
just call them). Picked at implementation time; both options match
|
||||||
|
PRD 0048's contrib convention. The visible contract is that
|
||||||
|
provisioning lives on the provider plugin.
|
||||||
|
|
||||||
|
### MCP registration for Codex
|
||||||
|
|
||||||
|
Codex reads MCP servers from `~/.codex/config.toml` (or whatever
|
||||||
|
`CODEX_HOME/config.toml` resolves to). The provider already writes
|
||||||
|
this file once during `provision_plan` to set the project trust
|
||||||
|
level. `CodexAgentProvider.provision_supervise_mcp` extends the
|
||||||
|
existing write: same path, append a `[mcp_servers.supervise]` table
|
||||||
|
pointing at the agent-side supervise URL.
|
||||||
|
|
||||||
|
Two implementation routes worth flagging:
|
||||||
|
|
||||||
|
- **Option A:** Pre-bake the MCP entry in the same config-write that
|
||||||
|
happens during `provision_plan`, before bottle launch. Simpler;
|
||||||
|
the supervise URL has to be known at plan time, which means
|
||||||
|
`provision_plan` needs the supervise URL (or a sentinel that means
|
||||||
|
"fill this in"). The smolmachines backend already plumbs
|
||||||
|
`agent_supervise_url` through to its provision_supervise step, so
|
||||||
|
the value is available.
|
||||||
|
- **Option B:** Append at bottle-launch time via a `bottle.exec`
|
||||||
|
that writes to the file inside the guest, matching the
|
||||||
|
`claude mcp add` flow. Slower but uniform with how
|
||||||
|
`ClaudeAgentProvider.provision_supervise_mcp` works.
|
||||||
|
|
||||||
|
Option B is the symmetric choice and the one this PRD assumes.
|
||||||
|
The implementer can switch to A if Option B turns out to need a
|
||||||
|
TOML-merge primitive the codebase doesn't already have.
|
||||||
|
|
||||||
|
### Backend after the cut
|
||||||
|
|
||||||
|
```python
|
||||||
|
class BottleBackend:
|
||||||
|
def provision_in_bottle(self, plan, bottle, supervise_url):
|
||||||
|
provider = get_provider(plan.spec.manifest.agents[
|
||||||
|
plan.spec.agent_name].provider)
|
||||||
|
self.provision_ca(plan, bottle)
|
||||||
|
prompt_path = provider.provision_prompt(bottle, plan)
|
||||||
|
provider.provision_skills(bottle, plan)
|
||||||
|
self.provision_workspace(plan, bottle)
|
||||||
|
self.provision_git(plan, bottle)
|
||||||
|
provider.provision_supervise_mcp(bottle, plan, supervise_url)
|
||||||
|
return prompt_path
|
||||||
|
```
|
||||||
|
|
||||||
|
`supervise_url` is the existing per-backend "where does the agent
|
||||||
|
reach the sidecar from inside the guest" value. The Docker backend
|
||||||
|
passes `http://supervise:<port>/`; smolmachines passes the
|
||||||
|
`http://127.0.0.1:<port>/` it already computed. The backend's only
|
||||||
|
remaining provider-touching duty is "tell the provider what the
|
||||||
|
sidecar URL is".
|
||||||
|
|
||||||
|
### Registry
|
||||||
|
|
||||||
|
```python
|
||||||
|
# bot_bottle/agent_provider.py
|
||||||
|
def get_provider(template: str) -> AgentProvider:
|
||||||
|
if template == PROVIDER_CLAUDE:
|
||||||
|
from bot_bottle.contrib.claude.agent_provider import (
|
||||||
|
ClaudeAgentProvider,
|
||||||
|
)
|
||||||
|
return ClaudeAgentProvider()
|
||||||
|
if template == PROVIDER_CODEX:
|
||||||
|
from bot_bottle.contrib.codex.agent_provider import (
|
||||||
|
CodexAgentProvider,
|
||||||
|
)
|
||||||
|
return CodexAgentProvider()
|
||||||
|
raise ValueError(f"unknown agent provider template: {template!r}")
|
||||||
|
```
|
||||||
|
|
||||||
|
Lazy imports keep core import-time graph small and match PRD 0048.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
Each chunk is one commit on the PR; the PR ships as one cut.
|
||||||
|
|
||||||
|
1. **Lift `AgentProvider` ABC + registry.** Add the ABC and
|
||||||
|
`get_provider` next to the existing `agent_provision_plan`
|
||||||
|
function. Have `agent_provision_plan` delegate to
|
||||||
|
`get_provider(template).provision_plan(...)` so callers keep
|
||||||
|
working through the transition.
|
||||||
|
2. **Move provider-specific `provision_plan` content into
|
||||||
|
contrib.** Create `contrib/claude/` and `contrib/codex/`. The
|
||||||
|
Claude and Codex branches of `agent_provision_plan` move into
|
||||||
|
the respective provider classes. The shared scaffolding
|
||||||
|
(initial dict setup, final `AgentProvisionPlan(...)` return)
|
||||||
|
stays in the ABC as a template method or moves into each
|
||||||
|
subclass — whichever needs less indirection.
|
||||||
|
3. **Move backend provisioners onto the provider.** Add
|
||||||
|
`provision_skills`, `provision_prompt`, `provision_supervise_mcp`
|
||||||
|
to `AgentProvider` (with a shared apply helper for skills /
|
||||||
|
prompt). Update `BottleBackend.provision_in_bottle` to call them.
|
||||||
|
Delete the four backend hook methods and the eight
|
||||||
|
`backend/<name>/provision/{skills,prompt,provider_auth,supervise}.py`
|
||||||
|
modules.
|
||||||
|
4. **Add Codex MCP support.** Implement
|
||||||
|
`CodexAgentProvider.provision_supervise_mcp` against
|
||||||
|
`~/.codex/config.toml`. Add a unit test that runs the method
|
||||||
|
against an in-memory FakeBottle and asserts the
|
||||||
|
`[mcp_servers.supervise]` block is present.
|
||||||
|
5. **Migrate tests.** Per-backend tests for the moved
|
||||||
|
provisioners turn into per-provider tests under
|
||||||
|
`tests/unit/test_contrib_claude_*.py` and
|
||||||
|
`tests/unit/test_contrib_codex_*.py`. Keep one integration-style
|
||||||
|
test per backend that confirms `provision_in_bottle` still
|
||||||
|
reaches every step.
|
||||||
|
6. **Activate.** Flip Status: Draft → Active in this PRD; close
|
||||||
|
#177 on merge.
|
||||||
|
|
||||||
|
## Open questions (resolved)
|
||||||
|
|
||||||
|
1. **`codex mcp add` exists.** Implementation calls
|
||||||
|
`codex mcp add --transport http supervise <url>` as `node` —
|
||||||
|
symmetric with `claude mcp add` (no `--scope user`; Codex writes
|
||||||
|
`~/.codex/config.toml` by default). Failure logs a warning; the
|
||||||
|
bottle still works without the entry.
|
||||||
|
2. **Each provider owns its apply steps end-to-end.** The base
|
||||||
|
ABC declares `provision_skills` / `provision_prompt` /
|
||||||
|
`provision` as abstract; each concrete provider implements its
|
||||||
|
own copy loop. No shared `_provision_apply.py`. The apply
|
||||||
|
sequences look similar today, but Claude and Codex harnesses
|
||||||
|
diverge over time (codex already grew a dummy-auth dance + a
|
||||||
|
`codex login status` verify with no Claude analogue) and the
|
||||||
|
"shared because both happen to call cp_in then chown" coupling
|
||||||
|
would just rot. Duplication is intentional.
|
||||||
|
3. **Env knobs removed.** `BOT_BOTTLE_CONTAINER_HOME`,
|
||||||
|
`BOT_BOTTLE_GUEST_HOME`, `BOT_BOTTLE_CONTAINER_SKILLS_DIR`, and
|
||||||
|
`BOT_BOTTLE_GUEST_SKILLS_DIR` are gone; `/home/node` is hardcoded
|
||||||
|
everywhere it was read. The values were effectively constants;
|
||||||
|
the knobs added surface area for no real flexibility.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Issue
|
||||||
|
[#177](https://gitea.dideric.is/didericis/bot-bottle/issues/177)
|
||||||
|
— the request: move provider logic into contrib, add MCP
|
||||||
|
configuration to agent provision, rename provision_supervise →
|
||||||
|
provision_supervise_mcp, ensure Codex gets MCP provisioned.
|
||||||
|
- PRD 0013 — supervise plane foundation (defines the MCP-discoverable
|
||||||
|
block-remediation tools this PRD makes available to Codex).
|
||||||
|
- PRD 0048 — SSH deploy key provisioning (the `contrib/` convention
|
||||||
|
this PRD follows).
|
||||||
|
- Current source:
|
||||||
|
[agent_provider.py L154-L230](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/bot_bottle/agent_provider.py#L154-L230)
|
||||||
|
— the provider-specific block this PRD relocates to contrib.
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user