Compare commits
1 Commits
pr-211
..
ba7ebf2f20
| Author | SHA1 | Date | |
|---|---|---|---|
 didericis-claude
|
ba7ebf2f20 |
@@ -1,76 +0,0 @@
|
||||
---
|
||||
name: quality-eval
|
||||
description: Use when the user asks to objectively evaluate, score, rate, audit, or quality-gate code, codebases, files, pull requests, or snippets using a strict 5-dimension engineering rubric with scores and refactoring steps.
|
||||
metadata:
|
||||
short-description: Score code quality with a strict rubric
|
||||
---
|
||||
|
||||
# Quality Eval
|
||||
|
||||
## Role
|
||||
|
||||
Act as a Staff Software Engineer and automated quality gate. Evaluate code objectively against the rubric below, surface hidden anti-patterns, and provide a mathematical grade with atomic refactoring steps.
|
||||
|
||||
## Evaluation Rules
|
||||
|
||||
- Evaluate only against the five rubric dimensions.
|
||||
- Be candid. Do not inflate scores for politeness.
|
||||
- Avoid generic advice. Every recommendation must name a specific code location, behavior, or pattern and include a concrete improvement direction.
|
||||
- Inspect the code before scoring. For codebases, read enough representative files, tests, and architecture boundaries to justify the scope.
|
||||
- When exact line numbers are available, cite them.
|
||||
- Do not reveal private chain-of-thought. In the required `Chain of Thought Analysis` section, provide a concise, step-by-step audit rationale with observable findings and score justifications.
|
||||
|
||||
## Rubric
|
||||
|
||||
Score each dimension from 1 to 5 using these anchors:
|
||||
|
||||
| Dimension | Score 1 (Fail) | Score 3 (Pass) | Score 5 (Exemplary) |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Architecture** | Spaghettified; tight coupling; violated separation of concerns. | Modular but relies on leaky abstractions or mixed domains. | Strict domain isolation; follows SOLID; clear dependency inversion. |
|
||||
| **Readability** | Cryptic naming; deep nesting (>3 levels); widespread DRY violations. | Idiomatic but features over-complex functions or sparse documentation. | Self-documenting; expressive naming; high cohesion; flat structure. |
|
||||
| **Resilience** | Swallows errors blindly; lacks contextual logging; fragile to bad input. | Basic try/catch blocks present but lacks granular, typed error handling. | Explicit error boundaries; contextual logging; structured failure modes. |
|
||||
| **Testability** | Hardcoded dependencies make mocking or isolated testing impossible. | Pure functions are testable, but side-effect heavy logic lacks test hooks. | Decoupled IO; deterministic execution; structured for unit and integration tests. |
|
||||
| **SecOps** | Hardcoded secrets; O(n^2) bottlenecks; zero input sanitization. | Safe from obvious flaws but lacks deep defensive optimization. | Validated inputs; optimized algorithmic complexity; zero security debt. |
|
||||
|
||||
## Scoring Method
|
||||
|
||||
1. Determine the evaluated scope and primary language.
|
||||
2. Identify concrete evidence for each dimension.
|
||||
3. Assign integer dimension scores from 1 to 5.
|
||||
4. Compute `composite_score` as the arithmetic mean of the five dimension scores, rounded to one decimal place.
|
||||
5. Include code snippets only when they make a refactoring step more actionable.
|
||||
|
||||
## Required Output
|
||||
|
||||
Structure every response into exactly these three Markdown sections:
|
||||
|
||||
### 1. Chain of Thought Analysis
|
||||
|
||||
Provide a concise step-by-step audit rationale. Name specific files, functions, patterns, anti-patterns, and rubric anchors. Keep it evidence-based and do not include hidden private reasoning.
|
||||
|
||||
### 2. Normalized Score Report
|
||||
|
||||
```json
|
||||
{
|
||||
"evaluation_metadata": {
|
||||
"target_scope": "string",
|
||||
"primary_language": "string"
|
||||
},
|
||||
"metrics": {
|
||||
"architecture_and_modularity": 0,
|
||||
"readability_and_maintainability": 0,
|
||||
"error_handling_and_resilience": 0,
|
||||
"testability_and_mocking": 0,
|
||||
"security_and_performance": 0
|
||||
},
|
||||
"composite_score": 0.0
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Atomic Refactoring Playbook
|
||||
|
||||
* **High Priority (To lift Score 1/2 to 3):**
|
||||
- [ ] Actionable, specific refactoring step with file/line/context reference.
|
||||
* **Medium Priority (To lift Score 3 to 4/5):**
|
||||
- [ ] Optimization or architectural pattern implementation step.
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
display_name: Quality Eval
|
||||
short_description: Scores code quality with a strict five-dimension rubric and refactoring playbook.
|
||||
default_prompt: Evaluate this code objectively using the quality-eval rubric and return the three-section score report.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Weekly canary suite. Catches upstream regressions (broken pinned
|
||||
# digest, etc.) without coupling every dev push to upstream registry
|
||||
# availability.
|
||||
# Weekly canary suite. Catches upstream regressions (broken pipelock
|
||||
# image packaging at the pinned digest, etc.) without coupling every
|
||||
# dev push to upstream registry availability.
|
||||
#
|
||||
# Opt-in via CLAUDE_BOTTLE_RUN_CANARIES=1 so the same files can be run
|
||||
# locally with the same gating.
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
name: lint
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- "**.py"
|
||||
- ".pylintrc"
|
||||
- ".gitea/workflows/lint.yml"
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install dev dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements-dev.txt
|
||||
|
||||
- name: Run pylint
|
||||
run: |
|
||||
# Run pylint on all Python files in the repo
|
||||
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0
|
||||
|
||||
- name: Run pyright
|
||||
run: |
|
||||
# Run pyright type checking
|
||||
pyright .
|
||||
@@ -1,125 +0,0 @@
|
||||
# Assign sequential numbers to prd-new-*.md files on merge to main.
|
||||
#
|
||||
# When a PR merges to main and includes prd-new-*.md files this workflow:
|
||||
# 1. Finds the next available NNNN number by scanning existing PRDs.
|
||||
# 2. Renames each prd-new-*.md to NNNN-<slug>.md.
|
||||
# 3. Updates the title header (# PRD prd-new: → # PRD NNNN:).
|
||||
# 4. Flips Status: Draft → Active when the push touched files outside
|
||||
# docs/prds/ anywhere in its commit range (i.e. the implementation
|
||||
# shipped together with the PRD).
|
||||
# 5. Commits the renaming back to main.
|
||||
#
|
||||
# No-op if the working tree contains no prd-new-*.md files.
|
||||
#
|
||||
# NOTE: The workflow scans the working tree (not just HEAD~1..HEAD) because
|
||||
# PRs land as multi-commit pushes and the prd-new file is often added in an
|
||||
# earlier commit on the branch, not in the final squash/merge commit.
|
||||
|
||||
name: prd-number
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- 'docs/prds/prd-new-*.md'
|
||||
|
||||
jobs:
|
||||
assign-numbers:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Configure git
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Assign PRD numbers
|
||||
run: |
|
||||
python3 - <<'EOF'
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
prds_dir = Path("docs/prds")
|
||||
|
||||
# Scan the working tree — prd-new files may have landed in any
|
||||
# commit of a multi-commit push, not just HEAD.
|
||||
new_prds = sorted(prds_dir.glob("prd-new-*.md"))
|
||||
|
||||
if not new_prds:
|
||||
print("No prd-new-*.md files found — nothing to do.")
|
||||
sys.exit(0)
|
||||
|
||||
# Determine whether non-PRD files were also changed anywhere in
|
||||
# the push range (BEFORE_SHA → HEAD). Falls back to HEAD~1 when
|
||||
# the env var isn't set (e.g. local act runs).
|
||||
before_sha = os.environ.get("GITHUB_EVENT_BEFORE", "HEAD~1")
|
||||
all_changed = subprocess.run(
|
||||
["git", "diff", "--name-only", before_sha, "HEAD"],
|
||||
capture_output=True, text=True, check=True,
|
||||
).stdout.splitlines()
|
||||
non_prd_changed = any(
|
||||
not f.startswith("docs/prds/") for f in all_changed
|
||||
)
|
||||
|
||||
# Find next available number.
|
||||
existing = sorted(
|
||||
int(m.group(1))
|
||||
for p in prds_dir.glob("*.md")
|
||||
if (m := re.match(r"^(\d{4})-", p.name))
|
||||
)
|
||||
next_num = (max(existing) + 1) if existing else 1
|
||||
|
||||
for prd_path in sorted(new_prds):
|
||||
slug = re.sub(r"^prd-new-", "", prd_path.stem)
|
||||
new_name = f"{next_num:04d}-{slug}.md"
|
||||
new_path = prds_dir / new_name
|
||||
print(f" {prd_path.name} → {new_name}")
|
||||
|
||||
content = prd_path.read_text()
|
||||
|
||||
# Update title header.
|
||||
content = re.sub(
|
||||
r"^(#\s+PRD\s+)prd-new(:)",
|
||||
rf"\g<1>{next_num:04d}\2",
|
||||
content,
|
||||
count=1,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
|
||||
# Conditionally flip Status.
|
||||
if non_prd_changed:
|
||||
content = re.sub(
|
||||
r"(\*\*Status:\*\*\s*)Draft",
|
||||
r"\g<1>Active",
|
||||
content,
|
||||
count=1,
|
||||
)
|
||||
|
||||
new_path.write_text(content)
|
||||
subprocess.run(["git", "rm", str(prd_path)], check=True)
|
||||
subprocess.run(["git", "add", str(new_path)], check=True)
|
||||
next_num += 1
|
||||
|
||||
subprocess.run(
|
||||
["git", "commit", "-m", "ci(prd): assign sequential numbers to new PRDs"],
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(["git", "push"], check=True)
|
||||
EOF
|
||||
@@ -21,11 +21,7 @@ on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- '**.py'
|
||||
pull_request:
|
||||
paths:
|
||||
- '**.py'
|
||||
|
||||
jobs:
|
||||
unit:
|
||||
|
||||
@@ -1,79 +0,0 @@
|
||||
name: Update Quality Badges
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- '**.py'
|
||||
- '.pylintrc'
|
||||
- 'pyrightconfig.json'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
update-badges:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install dev dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements-dev.txt
|
||||
|
||||
- name: Run pylint and extract score
|
||||
id: pylint
|
||||
run: |
|
||||
PYLINT_OUTPUT=$(python -m pylint bot_bottle/ 2>&1) || true
|
||||
SCORE=$(echo "$PYLINT_OUTPUT" | grep -oP '(?<=rated at )\d+\.\d+/10' | head -1)
|
||||
echo "score=$SCORE" >> $GITHUB_OUTPUT
|
||||
echo "Pylint score: $SCORE"
|
||||
|
||||
- name: Run pyright and check errors
|
||||
id: pyright
|
||||
run: |
|
||||
PYRIGHT_OUTPUT=$(python -m pyright 2>&1) || true
|
||||
ERRORS=$(echo "$PYRIGHT_OUTPUT" | grep -oP '\d+(?= error)' | head -1)
|
||||
echo "errors=$ERRORS" >> $GITHUB_OUTPUT
|
||||
echo "Pyright errors: $ERRORS"
|
||||
|
||||
- name: Update badges in README
|
||||
run: |
|
||||
PYLINT_SCORE="${{ steps.pylint.outputs.score }}"
|
||||
PYRIGHT_ERRORS="${{ steps.pyright.outputs.errors }}"
|
||||
|
||||
PYLINT_SCORE_ENCODED=$(echo "$PYLINT_SCORE" | sed 's|/|%2F|g')
|
||||
|
||||
if [ -n "$PYLINT_SCORE_ENCODED" ]; then
|
||||
sed -i "s|/badge/pylint-[^)]*|/badge/pylint-${PYLINT_SCORE_ENCODED}-brightgreen|" README.md
|
||||
fi
|
||||
if [ -n "$PYRIGHT_ERRORS" ]; then
|
||||
sed -i "s|/badge/pyright-[^)]*|/badge/pyright-${PYRIGHT_ERRORS}%20errors-brightgreen|" README.md
|
||||
fi
|
||||
|
||||
echo "Updated badges:"
|
||||
grep -E "pylint|pyright" README.md | head -2
|
||||
|
||||
- name: Commit and push badge updates
|
||||
run: |
|
||||
git config --local user.email "action@gitea.local"
|
||||
git config --local user.name "Quality Badge Bot"
|
||||
|
||||
# Check if there are changes
|
||||
if git diff --quiet README.md; then
|
||||
echo "No badge changes needed"
|
||||
else
|
||||
echo "Badge changes detected, committing..."
|
||||
git add README.md
|
||||
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n\n'"[skip ci]"
|
||||
git commit -m "$MSG"
|
||||
git push
|
||||
fi
|
||||
@@ -1,632 +0,0 @@
|
||||
[MAIN]
|
||||
|
||||
# Analyse import fallback blocks. This can be used to support both Python 2 and
|
||||
# 3 compatible code, which means that the block might have code that exists
|
||||
# only in one or another interpreter, leading to false positives when analysed.
|
||||
analyse-fallback-blocks=no
|
||||
|
||||
# Clear in-memory caches upon conclusion of linting. Useful if running pylint
|
||||
# in a server-like mode.
|
||||
clear-cache-post-run=no
|
||||
|
||||
# Load and enable all available extensions. Use --list-extensions to see a list
|
||||
# all available extensions.
|
||||
#enable-all-extensions=
|
||||
|
||||
# In error mode, messages with a category besides ERROR or FATAL are
|
||||
# suppressed, and no reports are done by default. Error mode is compatible with
|
||||
# disabling specific errors.
|
||||
#errors-only=
|
||||
|
||||
# Always return a 0 (non-error) status code, even if lint errors are found.
|
||||
# This is primarily useful in continuous integration scripts.
|
||||
#exit-zero=
|
||||
|
||||
# A comma-separated list of package or module names from where C extensions may
|
||||
# be loaded. Extensions are loading into the active Python interpreter and may
|
||||
# run arbitrary code.
|
||||
extension-pkg-allow-list=
|
||||
|
||||
# A comma-separated list of package or module names from where C extensions may
|
||||
# be loaded. Extensions are loading into the active Python interpreter and may
|
||||
# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
|
||||
# for backward compatibility.)
|
||||
extension-pkg-whitelist=
|
||||
|
||||
# Return non-zero exit code if any of these messages/categories are detected,
|
||||
# even if score is above --fail-under value. Syntax same as enable. Messages
|
||||
# specified are enabled, while categories only check already-enabled messages.
|
||||
fail-on=
|
||||
|
||||
# Specify a score threshold under which the program will exit with error.
|
||||
fail-under=10
|
||||
|
||||
# Interpret the stdin as a python script, whose filename needs to be passed as
|
||||
# the module_or_package argument.
|
||||
#from-stdin=
|
||||
|
||||
# Files or directories to be skipped. They should be base names, not paths.
|
||||
ignore=CVS
|
||||
|
||||
# Add files or directories matching the regular expressions patterns to the
|
||||
# ignore-list. The regex matches against paths and can be in Posix or Windows
|
||||
# format. Because '\\' represents the directory delimiter on Windows systems,
|
||||
# it can't be used as an escape character.
|
||||
ignore-paths=
|
||||
|
||||
# Files or directories matching the regular expression patterns are skipped.
|
||||
# The regex matches against base names, not paths. The default value ignores
|
||||
# Emacs file locks
|
||||
ignore-patterns=^\.#
|
||||
|
||||
# List of module names for which member attributes should not be checked and
|
||||
# will not be imported (useful for modules/projects where namespaces are
|
||||
# manipulated during runtime and thus existing member attributes cannot be
|
||||
# deduced by static analysis). It supports qualified module names, as well as
|
||||
# Unix pattern matching.
|
||||
ignored-modules=
|
||||
|
||||
# Python code to execute, usually for sys.path manipulation such as
|
||||
# pygtk.require().
|
||||
#init-hook=
|
||||
|
||||
# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
|
||||
# number of processors available to use, and will cap the count on Windows to
|
||||
# avoid hangs.
|
||||
jobs=1
|
||||
|
||||
# Control the amount of potential inferred values when inferring a single
|
||||
# object. This can help the performance when dealing with large functions or
|
||||
# complex, nested conditions.
|
||||
limit-inference-results=100
|
||||
|
||||
# List of plugins (as comma separated values of python module names) to load,
|
||||
# usually to register additional checkers.
|
||||
load-plugins=
|
||||
|
||||
# Pickle collected data for later comparisons.
|
||||
persistent=yes
|
||||
|
||||
# Resolve imports to .pyi stubs if available. May reduce no-member messages and
|
||||
# increase not-an-iterable messages.
|
||||
prefer-stubs=no
|
||||
|
||||
# Minimum Python version to use for version dependent checks. Will default to
|
||||
# the version used to run pylint.
|
||||
py-version=3.14
|
||||
|
||||
# Discover python modules and packages in the file system subtree.
|
||||
recursive=no
|
||||
|
||||
# Add paths to the list of the source roots. Supports globbing patterns. The
|
||||
# source root is an absolute path or a path relative to the current working
|
||||
# directory used to determine a package namespace for modules located under the
|
||||
# source root.
|
||||
source-roots=
|
||||
|
||||
# Allow loading of arbitrary C extensions. Extensions are imported into the
|
||||
# active Python interpreter and may run arbitrary code.
|
||||
unsafe-load-any-extension=no
|
||||
|
||||
# In verbose mode, extra non-checker-related info will be displayed.
|
||||
#verbose=
|
||||
|
||||
|
||||
[BASIC]
|
||||
|
||||
# Naming style matching correct argument names.
|
||||
argument-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct argument names. Overrides argument-
|
||||
# naming-style. If left empty, argument names will be checked with the set
|
||||
# naming style.
|
||||
#argument-rgx=
|
||||
|
||||
# Naming style matching correct attribute names.
|
||||
attr-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct attribute names. Overrides attr-naming-
|
||||
# style. If left empty, attribute names will be checked with the set naming
|
||||
# style.
|
||||
#attr-rgx=
|
||||
|
||||
# Bad variable names which should always be refused, separated by a comma.
|
||||
bad-names=foo,
|
||||
bar,
|
||||
baz,
|
||||
toto,
|
||||
tutu,
|
||||
tata
|
||||
|
||||
# Bad variable names regexes, separated by a comma. If names match any regex,
|
||||
# they will always be refused
|
||||
bad-names-rgxs=
|
||||
|
||||
# Naming style matching correct class attribute names.
|
||||
class-attribute-naming-style=any
|
||||
|
||||
# Regular expression matching correct class attribute names. Overrides class-
|
||||
# attribute-naming-style. If left empty, class attribute names will be checked
|
||||
# with the set naming style.
|
||||
#class-attribute-rgx=
|
||||
|
||||
# Naming style matching correct class constant names.
|
||||
class-const-naming-style=UPPER_CASE
|
||||
|
||||
# Regular expression matching correct class constant names. Overrides class-
|
||||
# const-naming-style. If left empty, class constant names will be checked with
|
||||
# the set naming style.
|
||||
#class-const-rgx=
|
||||
|
||||
# Naming style matching correct class names.
|
||||
class-naming-style=PascalCase
|
||||
|
||||
# Regular expression matching correct class names. Overrides class-naming-
|
||||
# style. If left empty, class names will be checked with the set naming style.
|
||||
#class-rgx=
|
||||
|
||||
# Naming style matching correct constant names.
|
||||
const-naming-style=UPPER_CASE
|
||||
|
||||
# Regular expression matching correct constant names. Overrides const-naming-
|
||||
# style. If left empty, constant names will be checked with the set naming
|
||||
# style.
|
||||
#const-rgx=
|
||||
|
||||
# Minimum line length for functions/classes that require docstrings, shorter
|
||||
# ones are exempt.
|
||||
docstring-min-length=-1
|
||||
|
||||
# Naming style matching correct function names.
|
||||
function-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct function names. Overrides function-
|
||||
# naming-style. If left empty, function names will be checked with the set
|
||||
# naming style.
|
||||
#function-rgx=
|
||||
|
||||
# Good variable names which should always be accepted, separated by a comma.
|
||||
good-names=i,
|
||||
j,
|
||||
k,
|
||||
ex,
|
||||
Run,
|
||||
_
|
||||
|
||||
# Good variable names regexes, separated by a comma. If names match any regex,
|
||||
# they will always be accepted
|
||||
good-names-rgxs=
|
||||
|
||||
# Include a hint for the correct naming format with invalid-name.
|
||||
include-naming-hint=no
|
||||
|
||||
# Naming style matching correct inline iteration names.
|
||||
inlinevar-naming-style=any
|
||||
|
||||
# Regular expression matching correct inline iteration names. Overrides
|
||||
# inlinevar-naming-style. If left empty, inline iteration names will be checked
|
||||
# with the set naming style.
|
||||
#inlinevar-rgx=
|
||||
|
||||
# Naming style matching correct method names.
|
||||
method-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct method names. Overrides method-naming-
|
||||
# style. If left empty, method names will be checked with the set naming style.
|
||||
#method-rgx=
|
||||
|
||||
# Naming style matching correct module names.
|
||||
module-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct module names. Overrides module-naming-
|
||||
# style. If left empty, module names will be checked with the set naming style.
|
||||
#module-rgx=
|
||||
|
||||
# Colon-delimited sets of names that determine each other's naming style when
|
||||
# the name regexes allow several styles.
|
||||
name-group=
|
||||
|
||||
# Regular expression which should only match function or class names that do
|
||||
# not require a docstring.
|
||||
no-docstring-rgx=^_
|
||||
|
||||
# Regular expression matching correct parameter specification variable names.
|
||||
# If left empty, parameter specification variable names will be checked with
|
||||
# the set naming style.
|
||||
#paramspec-rgx=
|
||||
|
||||
# List of decorators that produce properties, such as abc.abstractproperty. Add
|
||||
# to this list to register other decorators that produce valid properties.
|
||||
# These decorators are taken in consideration only for invalid-name.
|
||||
property-classes=abc.abstractproperty
|
||||
|
||||
# Regular expression matching correct type alias names. If left empty, type
|
||||
# alias names will be checked with the set naming style.
|
||||
#typealias-rgx=
|
||||
|
||||
# Regular expression matching correct type variable names. If left empty, type
|
||||
# variable names will be checked with the set naming style.
|
||||
#typevar-rgx=
|
||||
|
||||
# Regular expression matching correct type variable tuple names. If left empty,
|
||||
# type variable tuple names will be checked with the set naming style.
|
||||
#typevartuple-rgx=
|
||||
|
||||
# Naming style matching correct variable names.
|
||||
variable-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct variable names. Overrides variable-
|
||||
# naming-style. If left empty, variable names will be checked with the set
|
||||
# naming style.
|
||||
#variable-rgx=
|
||||
|
||||
|
||||
[CLASSES]
|
||||
|
||||
# Warn about protected attribute access inside special methods
|
||||
check-protected-access-in-special-methods=no
|
||||
|
||||
# List of method names used to declare (i.e. assign) instance attributes.
|
||||
defining-attr-methods=__init__,
|
||||
__new__,
|
||||
setUp,
|
||||
asyncSetUp,
|
||||
__post_init__
|
||||
|
||||
# List of member names, which should be excluded from the protected access
|
||||
# warning.
|
||||
exclude-protected=_asdict,_fields,_replace,_source,_make,os._exit
|
||||
|
||||
# List of valid names for the first argument in a class method.
|
||||
valid-classmethod-first-arg=cls
|
||||
|
||||
# List of valid names for the first argument in a metaclass class method.
|
||||
valid-metaclass-classmethod-first-arg=mcs
|
||||
|
||||
|
||||
[DESIGN]
|
||||
|
||||
# List of regular expressions of class ancestor names to ignore when counting
|
||||
# public methods (see R0903)
|
||||
exclude-too-few-public-methods=
|
||||
|
||||
# List of qualified class names to ignore when counting class parents (see
|
||||
# R0901)
|
||||
ignored-parents=
|
||||
|
||||
# Maximum number of arguments for function / method.
|
||||
max-args=5
|
||||
|
||||
# Maximum number of attributes for a class (see R0902).
|
||||
max-attributes=7
|
||||
|
||||
# Maximum number of boolean expressions in an if statement (see R0916).
|
||||
max-bool-expr=5
|
||||
|
||||
# Maximum number of branch for function / method body.
|
||||
max-branches=12
|
||||
|
||||
# Maximum number of locals for function / method body.
|
||||
max-locals=15
|
||||
|
||||
# Maximum number of parents for a class (see R0901).
|
||||
max-parents=7
|
||||
|
||||
# Maximum number of positional arguments for function / method.
|
||||
max-positional-arguments=5
|
||||
|
||||
# Maximum number of public methods for a class (see R0904).
|
||||
max-public-methods=20
|
||||
|
||||
# Maximum number of return / yield for function / method body.
|
||||
max-returns=6
|
||||
|
||||
# Maximum number of statements in function / method body.
|
||||
max-statements=50
|
||||
|
||||
# Minimum number of public methods for a class (see R0903).
|
||||
min-public-methods=2
|
||||
|
||||
|
||||
[EXCEPTIONS]
|
||||
|
||||
# Exceptions that will emit a warning when caught.
|
||||
overgeneral-exceptions=builtins.BaseException,builtins.Exception
|
||||
|
||||
|
||||
[FORMAT]
|
||||
|
||||
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
|
||||
expected-line-ending-format=
|
||||
|
||||
# Regexp for a line that is allowed to be longer than the limit.
|
||||
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
|
||||
|
||||
# Number of spaces of indent required inside a hanging or continued line.
|
||||
indent-after-paren=4
|
||||
|
||||
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
|
||||
# tab).
|
||||
indent-string=' '
|
||||
|
||||
# Maximum number of characters on a single line. Pylint's default of 100 is
|
||||
# based on PEP 8's guidance that teams may choose line lengths up to 99
|
||||
# characters.
|
||||
max-line-length=100
|
||||
|
||||
# Maximum number of lines in a module.
|
||||
max-module-lines=1000
|
||||
|
||||
# Allow the body of a class to be on the same line as the declaration if body
|
||||
# contains single statement.
|
||||
single-line-class-stmt=no
|
||||
|
||||
# Allow the body of an if to be on the same line as the test if there is no
|
||||
# else.
|
||||
single-line-if-stmt=no
|
||||
|
||||
|
||||
[LOGGING]
|
||||
|
||||
# The type of string formatting that logging methods do. `old` means using %
|
||||
# formatting, `new` is for `{}` formatting.
|
||||
logging-format-style=old
|
||||
|
||||
# Logging modules to check that the string format arguments are in logging
|
||||
# function parameter format.
|
||||
logging-modules=logging
|
||||
|
||||
|
||||
[MESSAGES CONTROL]
|
||||
|
||||
# Only show warnings with the listed confidence levels. Leave empty to show
|
||||
# all. Valid levels: HIGH, CONTROL_FLOW, INFERENCE, INFERENCE_FAILURE,
|
||||
# UNDEFINED.
|
||||
confidence=HIGH,
|
||||
CONTROL_FLOW,
|
||||
INFERENCE,
|
||||
INFERENCE_FAILURE,
|
||||
UNDEFINED
|
||||
|
||||
# Disable the message, report, category or checker with the given id(s). You
|
||||
# can either give multiple identifiers separated by comma (,) or put this
|
||||
# option multiple times (only on the command line, not in the configuration
|
||||
# file where it should appear only once). You can also use "--disable=all" to
|
||||
# disable everything first and then re-enable specific checks. For example, if
|
||||
# you want to run only the similarities checker, you can use "--disable=all
|
||||
# --enable=similarities". If you want to run only the classes checker, but have
|
||||
# no Warning level messages displayed, use "--disable=all --enable=classes
|
||||
# --disable=W".
|
||||
disable=raw-checker-failed,
|
||||
bad-inline-option,
|
||||
locally-disabled,
|
||||
file-ignored,
|
||||
suppressed-message,
|
||||
useless-suppression,
|
||||
deprecated-pragma,
|
||||
use-symbolic-message-instead,
|
||||
use-implicit-booleaness-not-comparison-to-string,
|
||||
use-implicit-booleaness-not-comparison-to-zero,
|
||||
missing-function-docstring,
|
||||
missing-class-docstring,
|
||||
missing-module-docstring,
|
||||
invalid-name,
|
||||
cyclic-import,
|
||||
too-many-arguments,
|
||||
too-many-locals,
|
||||
too-many-branches,
|
||||
too-many-statements,
|
||||
too-many-instance-attributes,
|
||||
duplicate-code,
|
||||
import-outside-toplevel,
|
||||
too-few-public-methods,
|
||||
unnecessary-ellipsis
|
||||
|
||||
# Enable the message, report, category or checker with the given id(s). You can
|
||||
# either give multiple identifier separated by comma (,) or put this option
|
||||
# multiple time (only on the command line, not in the configuration file where
|
||||
# it should appear only once). See also the "--disable" option for examples.
|
||||
enable=
|
||||
|
||||
|
||||
[METHOD_ARGS]
|
||||
|
||||
# List of qualified names (i.e., library.method) which require a timeout
|
||||
# parameter e.g. 'requests.api.get,requests.api.post'
|
||||
timeout-methods=requests.api.delete,requests.api.get,requests.api.head,requests.api.options,requests.api.patch,requests.api.post,requests.api.put,requests.api.request
|
||||
|
||||
|
||||
[MISCELLANEOUS]
|
||||
|
||||
# Whether or not to search for fixme's in docstrings.
|
||||
check-fixme-in-docstring=no
|
||||
|
||||
# List of note tags to take in consideration, separated by a comma.
|
||||
notes=FIXME,
|
||||
XXX,
|
||||
TODO
|
||||
|
||||
# Regular expression of note tags to take in consideration.
|
||||
notes-rgx=
|
||||
|
||||
|
||||
[REFACTORING]
|
||||
|
||||
# Maximum number of nested blocks for function / method body
|
||||
max-nested-blocks=5
|
||||
|
||||
# Complete name of functions that never returns. When checking for
|
||||
# inconsistent-return-statements if a never returning function is called then
|
||||
# it will be considered as an explicit return statement and no message will be
|
||||
# printed.
|
||||
never-returning-functions=sys.exit,argparse.parse_error
|
||||
|
||||
# Let 'consider-using-join' be raised when the separator to join on would be
|
||||
# non-empty (resulting in expected fixes of the type: ``"- " + " -
|
||||
# ".join(items)``)
|
||||
suggest-join-with-non-empty-separator=yes
|
||||
|
||||
|
||||
[REPORTS]
|
||||
|
||||
# Python expression which should return a score less than or equal to 10. You
|
||||
# have access to the variables 'fatal', 'error', 'warning', 'refactor',
|
||||
# 'convention', and 'info' which contain the number of messages in each
|
||||
# category, as well as 'statement' which is the total number of statements
|
||||
# analyzed. This score is used by the global evaluation report (RP0004).
|
||||
evaluation=max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))
|
||||
|
||||
# Template used to display messages. This is a python new-style format string
|
||||
# used to format the message information. See doc for all details.
|
||||
msg-template=
|
||||
|
||||
# Set the output format. Available formats are: 'text', 'parseable',
|
||||
# 'colorized', 'json2' (improved json format), 'json' (old json format), msvs
|
||||
# (visual studio) and 'github' (GitHub actions). You can also give a reporter
|
||||
# class, e.g. mypackage.mymodule.MyReporterClass.
|
||||
#output-format=
|
||||
|
||||
# Tells whether to display a full report or only the messages.
|
||||
reports=no
|
||||
|
||||
# Activate the evaluation score.
|
||||
score=yes
|
||||
|
||||
|
||||
[SIMILARITIES]
|
||||
|
||||
# Comments are removed from the similarity computation
|
||||
ignore-comments=yes
|
||||
|
||||
# Docstrings are removed from the similarity computation
|
||||
ignore-docstrings=yes
|
||||
|
||||
# Imports are removed from the similarity computation
|
||||
ignore-imports=yes
|
||||
|
||||
# Signatures are removed from the similarity computation
|
||||
ignore-signatures=yes
|
||||
|
||||
# Minimum lines number of a similarity.
|
||||
min-similarity-lines=4
|
||||
|
||||
|
||||
[SPELLING]
|
||||
|
||||
# Limits count of emitted suggestions for spelling mistakes.
|
||||
max-spelling-suggestions=4
|
||||
|
||||
# Spelling dictionary name. No available dictionaries : You need to install
|
||||
# both the python package and the system dependency for enchant to work.
|
||||
spelling-dict=
|
||||
|
||||
# List of comma separated words that should be considered directives if they
|
||||
# appear at the beginning of a comment and should not be checked.
|
||||
spelling-ignore-comment-directives=fmt: on,fmt: off,noqa:,noqa,nosec,isort:skip,mypy:
|
||||
|
||||
# List of comma separated words that should not be checked.
|
||||
spelling-ignore-words=
|
||||
|
||||
# A path to a file that contains the private dictionary; one word per line.
|
||||
spelling-private-dict-file=
|
||||
|
||||
# Tells whether to store unknown words to the private dictionary (see the
|
||||
# --spelling-private-dict-file option) instead of raising a message.
|
||||
spelling-store-unknown-words=no
|
||||
|
||||
|
||||
[STRING]
|
||||
|
||||
# This flag controls whether inconsistent-quotes generates a warning when the
|
||||
# character used as a quote delimiter is used inconsistently within a module.
|
||||
check-quote-consistency=no
|
||||
|
||||
# This flag controls whether the implicit-str-concat should generate a warning
|
||||
# on implicit string concatenation in sequences defined over several lines.
|
||||
check-str-concat-over-line-jumps=no
|
||||
|
||||
|
||||
[TYPECHECK]
|
||||
|
||||
# List of decorators that produce context managers, such as
|
||||
# contextlib.contextmanager. Add to this list to register other decorators that
|
||||
# produce valid context managers.
|
||||
contextmanager-decorators=contextlib.contextmanager
|
||||
|
||||
# List of members which are set dynamically and missed by pylint inference
|
||||
# system, and so shouldn't trigger E1101 when accessed. Python regular
|
||||
# expressions are accepted.
|
||||
generated-members=
|
||||
|
||||
# Tells whether to warn about missing members when the owner of the attribute
|
||||
# is inferred to be None.
|
||||
ignore-none=yes
|
||||
|
||||
# This flag controls whether pylint should warn about no-member and similar
|
||||
# checks whenever an opaque object is returned when inferring. The inference
|
||||
# can return multiple potential results while evaluating a Python object, but
|
||||
# some branches might not be evaluated, which results in partial inference. In
|
||||
# that case, it might be useful to still emit no-member and other checks for
|
||||
# the rest of the inferred objects.
|
||||
ignore-on-opaque-inference=yes
|
||||
|
||||
# List of symbolic message names to ignore for Mixin members.
|
||||
ignored-checks-for-mixins=no-member,
|
||||
not-async-context-manager,
|
||||
not-context-manager,
|
||||
attribute-defined-outside-init
|
||||
|
||||
# List of class names for which member attributes should not be checked (useful
|
||||
# for classes with dynamically set attributes). This supports the use of
|
||||
# qualified names.
|
||||
ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
|
||||
|
||||
# Show a hint with possible names when a member name was not found. The aspect
|
||||
# of finding the hint is based on edit distance.
|
||||
missing-member-hint=yes
|
||||
|
||||
# The maximum edit distance a name should have in order to be considered a
|
||||
# similar match for a missing member name.
|
||||
missing-member-hint-distance=1
|
||||
|
||||
# The total number of similar names that should be taken in consideration when
|
||||
# showing a hint for a missing member.
|
||||
missing-member-max-choices=1
|
||||
|
||||
# Regex pattern to define which classes are considered mixins.
|
||||
mixin-class-rgx=.*[Mm]ixin
|
||||
|
||||
# List of decorators that change the signature of a decorated function.
|
||||
signature-mutators=
|
||||
|
||||
|
||||
[VARIABLES]
|
||||
|
||||
# List of additional names supposed to be defined in builtins. Remember that
|
||||
# you should avoid defining new builtins when possible.
|
||||
additional-builtins=
|
||||
|
||||
# Tells whether unused global variables should be treated as a violation.
|
||||
allow-global-unused-variables=yes
|
||||
|
||||
# List of names allowed to shadow builtins
|
||||
allowed-redefined-builtins=
|
||||
|
||||
# List of strings which can identify a callback function by name. A callback
|
||||
# name must start or end with one of those strings.
|
||||
callbacks=cb_,
|
||||
_cb
|
||||
|
||||
# A regular expression matching the name of dummy variables (i.e. expected to
|
||||
# not be used).
|
||||
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
|
||||
|
||||
# Argument names that match this expression will be ignored.
|
||||
ignored-argument-names=_.*|^ignored_|^unused_
|
||||
|
||||
# Tells whether we should check for unused import in __init__ files.
|
||||
init-import=no
|
||||
|
||||
# List of qualified module names which can have objects that can redefine
|
||||
# builtins.
|
||||
redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io
|
||||
@@ -2,18 +2,11 @@
|
||||
|
||||
## What this is
|
||||
|
||||
bot-bottle spins up an isolated backend runtime for running AI coding agents
|
||||
with a curated set of skills and env vars. The point is to run agents with
|
||||
broad permissions inside a sandbox, so a misbehaving agent cannot reach the
|
||||
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
||||
the runtime lifecycle and the copying of skills and env vars into it.
|
||||
The default backend on compatible macOS hosts is macos-container:
|
||||
agents and sidecar bundles run through Apple's `container` CLI without
|
||||
requiring Docker. The smolmachines backend remains available with
|
||||
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
|
||||
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
|
||||
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
|
||||
or `--backend=docker`.
|
||||
bot-bottle spins up an isolated container for running AI coding agents with a
|
||||
curated set of skills and env vars. The point is to run agents with broad
|
||||
permissions inside a sandbox, so a misbehaving agent cannot reach the host.
|
||||
A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
||||
the container lifecycle and the copying of skills and env vars into it.
|
||||
|
||||
## Goals
|
||||
|
||||
@@ -24,7 +17,7 @@ or `--backend=docker`.
|
||||
## Non-goals
|
||||
|
||||
- Communicating between agents directly
|
||||
- Removing the Docker backend
|
||||
- Self hosted VMs (v1 uses local Docker containers, not VMs)
|
||||
- Advanced agent auditing (lean on git history for auditing)
|
||||
|
||||
## Repository layout
|
||||
@@ -32,8 +25,9 @@ or `--backend=docker`.
|
||||
- `README.md` — short public-facing description.
|
||||
- `AGENTS.md` — this file, orientation for future agent sessions.
|
||||
- `.gitignore` — OS junk.
|
||||
- `.bot-bottle/` — per-repo agent and bottle manifests (YAML markdown format).
|
||||
- `examples/` — example bottles and agents showing the manifest format.
|
||||
- `bot-bottle.json` — legacy manifest of named agents (env / skills / prompt
|
||||
per agent), consumed by `cli.py`. See "Manifest" under
|
||||
"Intended design".
|
||||
- `docs/README.md` — docs overview; when to write which document.
|
||||
- `docs/prds/` — product requirement docs (see `docs/prds/README.md` for format).
|
||||
- `docs/research/` — research notes (see `docs/research/README.md`).
|
||||
@@ -43,11 +37,10 @@ or `--backend=docker`.
|
||||
|
||||
- Three kinds of doc, each with its own conventions in-folder; see
|
||||
`docs/README.md` for when to write which:
|
||||
- **PRDs** (`docs/prds/`) — one feature per file. While a PR is open
|
||||
the file is named `prd-new-<kebab>.md`; CI assigns a sequential
|
||||
number on merge to `main` and renames it. A `Status:` line tracks
|
||||
lifecycle: Draft → Active (shipped to `main`) →
|
||||
Superseded/Retargeted. Format in `docs/prds/README.md`.
|
||||
- **PRDs** (`docs/prds/`) — one feature per file, numbered
|
||||
`NNNN-kebab.md`. A `Status:` line tracks lifecycle: Draft → Active
|
||||
(shipped to `main`) → Superseded/Retargeted. Format in
|
||||
`docs/prds/README.md`.
|
||||
- **Research notes** (`docs/research/`) — opinionated investigations;
|
||||
unnumbered kebab-case, freeform and verdict-first. See
|
||||
`docs/research/README.md`.
|
||||
|
||||
@@ -16,27 +16,21 @@ FROM node:22-slim
|
||||
# features (status checks, commits, PR creation) — without git in the
|
||||
# image, those features fail in surprising ways once the user does any
|
||||
# real work. ca-certificates is already in the slim base; listed for
|
||||
# clarity in case the base ever drops it. curl is here so any
|
||||
# HTTPS_PROXY-aware tool (curl itself, plus anything that shells out
|
||||
# to it) works against egress's bumped TLS without the agent needing
|
||||
# local DNS.
|
||||
# clarity in case the base ever drops it. socat is the privileged
|
||||
# forwarder for the in-container ssh-agent (see bot_bottle/ssh.py): the agent
|
||||
# runs as root and rejects non-root connections, so socat sits between
|
||||
# node and the agent socket. curl is here so any HTTPS_PROXY-aware
|
||||
# tool (curl itself, plus anything that shells out to it) works
|
||||
# against pipelock's bumped TLS without the agent needing local DNS.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# App-specific deps. Python isn't required by claude-code itself
|
||||
# (claude-code is a Node CLI), but is convenient for the agent to
|
||||
# shell out to for ad-hoc scripts. Kept on its own layer so it can
|
||||
# be moved to a downstream image if the base ever needs to shrink.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install claude-code globally. Pinned to the version verified in the v1
|
||||
# build (`claude --version` returns 2.1.126). Bump deliberately when
|
||||
# rolling forward; an unpinned install would mean rebuilds silently pick
|
||||
# up new behavior.
|
||||
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
|
||||
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.126 \
|
||||
&& npm cache clean --force
|
||||
|
||||
# Run as a non-root user. The node image already provides a `node` user
|
||||
@@ -0,0 +1,20 @@
|
||||
# bot-bottle Codex provider image.
|
||||
#
|
||||
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
|
||||
# non-root node user, and the provider CLI installed globally.
|
||||
|
||||
FROM node:22-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.134.0 \
|
||||
&& npm cache clean --force
|
||||
|
||||
USER node
|
||||
WORKDIR /home/node
|
||||
|
||||
RUN mkdir -p /home/node/.codex
|
||||
|
||||
CMD ["codex"]
|
||||
+27
-15
@@ -1,18 +1,23 @@
|
||||
# Per-bottle sidecar bundle image (PRD 0024).
|
||||
#
|
||||
# Collapses the prior per-sidecar images (egress, git-gate,
|
||||
# supervise) into one. A small stdlib-Python init supervisor at
|
||||
# /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
|
||||
# propagates per-daemon stdout/stderr to the container log with a
|
||||
# `[name]` prefix. See PRD 0024 for the rationale.
|
||||
# Collapses the four prior per-sidecar images (pipelock, egress,
|
||||
# git-gate, supervise) into one. A small stdlib-Python init
|
||||
# supervisor at /app/sidecar_init.py spawns all four daemons,
|
||||
# forwards SIGTERM, and propagates per-daemon stdout/stderr to the
|
||||
# container log with a `[name]` prefix. See PRD 0024 for the
|
||||
# rationale.
|
||||
#
|
||||
# Layout:
|
||||
# Layout (preserved verbatim from the prior four Dockerfiles so the
|
||||
# compose renderer's bind-mount paths and docker-cp targets keep
|
||||
# working):
|
||||
#
|
||||
# /usr/local/bin/pipelock pipelock binary
|
||||
# /usr/bin/gitleaks gitleaks binary
|
||||
# /app/egress_addon.py + siblings mitmproxy addon (egress)
|
||||
# /app/egress-entrypoint.sh mitmdump launcher
|
||||
# /app/supervise_server.py + .py supervise MCP server
|
||||
# /app/sidecar_init.py PID 1 supervisor
|
||||
# /etc/pipelock.yaml bind-mounted at run time
|
||||
# /etc/egress/routes.yaml bind-mounted at run time
|
||||
# /etc/git-gate/pre-receive docker-cp'd at start time
|
||||
# /git-gate-entrypoint.sh docker-cp'd at start time
|
||||
@@ -22,17 +27,24 @@
|
||||
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
|
||||
#
|
||||
# Exposed ports inside the container:
|
||||
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
|
||||
# 8888 pipelock (HTTPS_PROXY)
|
||||
# 9099 egress (mitmproxy, pipelock's upstream — not externally
|
||||
# addressed by the agent)
|
||||
# 9418 git-gate (git-daemon)
|
||||
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
||||
# 9100 supervise (MCP HTTP)
|
||||
|
||||
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
|
||||
# Stage 1: pipelock binary. The upstream pipelock image is a
|
||||
# scratch image with the binary at /pipelock (entrypoint).
|
||||
# Pinned by digest in lockstep with
|
||||
# bot_bottle/backend/docker/pipelock.py:PIPELOCK_IMAGE.
|
||||
FROM ghcr.io/luckypipewrench/pipelock@sha256:3b1a39417b98406ddc5dc2d8fcb42865ddc0c68a43d355db55f0f8cb06bc6de9 AS pipelock-src
|
||||
|
||||
# Stage 2: gitleaks binary. The upstream gitleaks image is alpine
|
||||
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
|
||||
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
|
||||
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
|
||||
|
||||
# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
|
||||
# Stage 3: assembly. mitmproxy/mitmproxy is debian-slim-based with
|
||||
# Python + mitmdump pre-installed — heavier than the others, so
|
||||
# this stage starts there and pulls the standalone binaries in.
|
||||
FROM mitmproxy/mitmproxy:11.1.3
|
||||
@@ -47,14 +59,16 @@ USER root
|
||||
# plus the core `git` binary the pre-receive hook invokes.
|
||||
# openssh-client supplies the upstream SSH transport the
|
||||
# pre-receive hook uses to forward accepted refs.
|
||||
# ca-certificates is needed for mitmdump upstream TLS (the
|
||||
# base image already has it; listed for explicitness).
|
||||
# ca-certificates is needed for both pipelock and mitmdump
|
||||
# upstream TLS (the base image already has it; listed for
|
||||
# explicitness).
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
git openssh-client ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Pull the standalone binaries into the final image.
|
||||
COPY --from=pipelock-src /pipelock /usr/local/bin/pipelock
|
||||
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
|
||||
|
||||
# Project Python: addon + server modules + the init supervisor.
|
||||
@@ -63,12 +77,10 @@ COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
|
||||
# Dockerfile.egress / Dockerfile.supervise layout.
|
||||
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
|
||||
COPY bot_bottle/egress_addon.py /app/egress_addon.py
|
||||
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
|
||||
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
||||
COPY bot_bottle/supervise.py /app/supervise.py
|
||||
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
||||
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
|
||||
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
|
||||
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
|
||||
RUN chmod +x /app/egress-entrypoint.sh
|
||||
|
||||
@@ -85,7 +97,7 @@ RUN mkdir -p \
|
||||
|
||||
# Documentation only — the compose renderer publishes whichever
|
||||
# subset the bottle uses.
|
||||
EXPOSE 8888 9099 9418 9420 9100
|
||||
EXPOSE 8888 9099 9418 9100
|
||||
|
||||
# WORKDIR matches Dockerfile.supervise's prior layout so the
|
||||
# in-app same-dir import in supervise_server.py stays deterministic.
|
||||
|
||||
@@ -5,37 +5,97 @@
|
||||
# bot-bottle
|
||||
|
||||
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
||||
[](https://github.com/PyCQA/pylint)
|
||||
[](https://github.com/microsoft/pyright)
|
||||
|
||||
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
|
||||
Run multiple Claude Code agents on your own machine, each scoped to its own secrets, skills, and egress allowlist.
|
||||
|
||||
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
|
||||
|
||||
|
||||
## Features
|
||||
Four prompts to the agent inside a real bottle:
|
||||
claude replies to `hello there` — proof api.anthropic.com routes
|
||||
through pipelock's bumped TLS end-to-end;
|
||||
asked to GET a non-allowlisted host, the agent's curl gets 403 back
|
||||
from pipelock;
|
||||
asked to POST a credential-shaped body to an allowlisted host, the
|
||||
same 403 — pipelock's DLP body scanner caught it;
|
||||
asked to commit and push an AKIA-shaped key, git-gate's gitleaks
|
||||
pre-receive hook rejects the ref.
|
||||
Run it yourself with `bash scripts/demo.sh`.
|
||||
|
||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
|
||||
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
||||
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
||||
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
||||
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
||||
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
|
||||
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
|
||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
||||
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
|
||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||
## Why "bot-bottle"?
|
||||
|
||||
Each container is a bottle; Claude is the genie inside. The genie's
|
||||
powers are exactly what the manifest grants it — a specific set of
|
||||
skills, a specific set of secrets, and a specific set of hosts it can
|
||||
reach — nothing more. You uncork one bottle per agent
|
||||
(`./cli.py start <agent>`), many bottles run in parallel, and each is
|
||||
scoped to its task. When the session ends the bottle is destroyed and
|
||||
the genie does not persist.
|
||||
|
||||
## Goals
|
||||
|
||||
- Scope each agent to the minimum credentials and network egress its task actually needs
|
||||
- Run multiple agents in parallel, isolated from each other
|
||||
- Keep code, credentials, and agent activity on infrastructure I control — no third-party agent runtime
|
||||
|
||||
## Project status
|
||||
|
||||
bot-bottle is a self-hosted secure runtime for AI coding agents.
|
||||
Each agent runs in an isolated container or micro-VM-backed bottle with
|
||||
scoped secrets, allowlisted egress, TLS-aware proxying, DLP checks, and
|
||||
a git-gate that withholds upstream credentials and scans pushes before
|
||||
forwarding. The project includes a documented threat model, PRD-driven
|
||||
development history, Docker and smolmachines backends, dashboard and
|
||||
remediation flows, and unit/integration tests covering exfiltration and
|
||||
sandbox escape scenarios.
|
||||
|
||||
## Security model
|
||||
|
||||
Each agent runs in its own bottle: its own container, its own internal
|
||||
Docker network, and its own pipelock sidecar. Bottles don't share
|
||||
state, don't talk to each other, and only get the env vars, skills,
|
||||
SSH identities, and egress hosts the manifest grants them — nothing
|
||||
more. Any one agent only has the access it needs to do its job.
|
||||
|
||||
The bottle limits both what an agent can see and where it can send
|
||||
it. Each bottle gets only the secrets and SSH identities the manifest
|
||||
grants it — a Gitea token but not a GitHub token, a deploy key but
|
||||
not a personal SSH key — so even a compromised or misbehaving agent
|
||||
only handles credentials it was already trusted with for its job.
|
||||
Egress flows through pipelock, which constrains where those
|
||||
credentials can travel: an agent with a Gitea token can reach
|
||||
`gitea.dideric.is`, not arbitrary attacker-controlled hosts. The same
|
||||
constraint blocks DNS-over-HTTPS as an exfil channel — a DoH resolver
|
||||
like `cloudflare-dns.com` would have to be on the allowlist for the
|
||||
agent to reach it at all. The container itself adds a layer between
|
||||
the agent and the host, but the v1 design leans more on secret
|
||||
minimization and egress allowlisting than on the container as a
|
||||
hardened boundary. On Linux hosts where [gVisor](https://gvisor.dev/)
|
||||
is registered with Docker, bot-bottle auto-detects it and launches
|
||||
every bottle under `runsc` for a userspace syscall barrier — no
|
||||
manifest configuration required. The broader v2 discussion lives in
|
||||
`docs/research/stronger-isolation-alternatives.md`.
|
||||
|
||||
The egress proxy and OAuth-token handling below are the load-bearing
|
||||
pieces of v1.
|
||||
|
||||
## Architecture
|
||||
|
||||
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
|
||||
A bottle is two containers per agent: an `agent` container, and a
|
||||
`sidecars` container that bundles pipelock + egress + git-gate +
|
||||
supervise behind a Python init supervisor (PRD 0024). They share a
|
||||
per-agent Docker `--internal` network; the agent has no default
|
||||
route off-box. All HTTP and HTTPS egress funnels through pipelock,
|
||||
where the egress allowlist, TLS interception, and request-body DLP
|
||||
scanner enforce the manifest before any byte leaves the host. The
|
||||
only egress that doesn't traverse pipelock is git-gate's SSH
|
||||
push/fetch to `bottle.git` upstreams — pipelock can't proxy SSH,
|
||||
so git-gate is its own L4-style egress path with gitleaks doing
|
||||
the pre-receive scan.
|
||||
|
||||
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
|
||||
|
||||
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||
|
||||
The Docker topology looks like this:
|
||||
The agent dials the bundle by the legacy short names (`pipelock`,
|
||||
`egress`, `git-gate`, `supervise`); the renderer registers those as
|
||||
docker-network aliases on the bundle so existing HTTPS_PROXY URLs
|
||||
and MCP endpoints resolve without an agent-side change.
|
||||
|
||||
```
|
||||
host ( ./cli.py )
|
||||
@@ -44,49 +104,231 @@ The Docker topology looks like this:
|
||||
▼
|
||||
┌─────────────────────────── bottle ──────────────────────────────────┐
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────────┐ │
|
||||
│ │ agent image │ HTTP(S) proxy │ egress image │ │
|
||||
│ │ (claude-code, │ ─────────────────►│ (mitmproxy; TLS bump │ │ HTTPS to
|
||||
│ │ codex, etc) │ │ DLP scan, path │───┼──► allowlisted
|
||||
│ │ │ │ matching, auth │ │ hosts
|
||||
│ │ environ: proxy │ │ injection) │ │
|
||||
│ │ URLs only, no │ └──────────────────────┘ │
|
||||
│ │ real tokens │ │
|
||||
│ │ │ git proxy ┌────────────────┐ │ SSH push/fetch
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ agent image │ HTTPS_PROXY │
|
||||
│ │ (claude-code, │ ────────────────────────┐ │
|
||||
│ │ built locally) │ │ │
|
||||
│ │ │ plain HTTP │ │
|
||||
│ │ skills, env, │ (token injection) ┌────▼─────────┐ │
|
||||
│ │ ~/.gitconfig, │ ──────────────────►│ cred-proxy │ │
|
||||
│ │ ~/.npmrc, tea │ │ (strips/inj │ │
|
||||
│ │ │ │ Authoriz.) │ │
|
||||
│ │ environ: URLs │ └─────┬────────┘ │
|
||||
│ │ only, no real │ HTTPS_PROXY │ │
|
||||
│ │ tokens │ ▼ │
|
||||
│ │ │ ┌────────────────┐ │ HTTPS to
|
||||
│ │ │ │ pipelock image │──────────┼──► allowlisted
|
||||
│ │ │ │ (TLS bump, DLP │ │ hosts (incl.
|
||||
│ │ │ │ body scan, │ │ cred-proxy
|
||||
│ │ │ │ allowlist) │ │ upstreams)
|
||||
│ │ │ └────────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ │ git:// ┌────────────────┐ │ SSH push/fetch
|
||||
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
|
||||
│ │ │ │ (gitleaks + │ │ upstreams
|
||||
│ └──────────────────┘ │ git daemon) │ │ (direct — not
|
||||
│ └────────────────┘ │ via egress)
|
||||
│ └────────────────┘ │ via pipelock)
|
||||
│ │
|
||||
│ agent on internal network (no default route); egress and │
|
||||
│ git-gate straddle internal + egress networks. │
|
||||
│ egress is the single HTTP/HTTPS chokepoint — all agent HTTP/HTTPS │
|
||||
│ traffic flows through it. git-gate's SSH egress is direct │
|
||||
│ because egress is HTTP-only. │
|
||||
│ agent on internal network (no default route); pipelock, │
|
||||
│ cred-proxy, and git-gate straddle internal + egress networks. │
|
||||
│ pipelock is the single HTTP/HTTPS chokepoint — cred-proxy's │
|
||||
│ outbound traverses it too. git-gate's SSH egress is direct │
|
||||
│ because pipelock is HTTP-only. │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
|
||||
- **agent image** — built from the provider template Dockerfile
|
||||
(`Dockerfile.claude` for Claude, `Dockerfile.codex` for Codex, or
|
||||
`agent_provider.dockerfile`) on first run; runs the selected agent
|
||||
CLI with the manifest-granted skills, env vars, and `~/.gitconfig`
|
||||
(the latter for the git-gate's `insteadOf` rules when `bottle.git`
|
||||
is set).
|
||||
- **pipelock image** — per-agent sidecar. Terminates the agent's
|
||||
outbound HTTP/HTTPS, enforces the resolved allowlist, runs DLP
|
||||
scanning. Design in `docs/prds/0001-per-agent-egress-proxy-via-pipelock.md`
|
||||
and `docs/prds/0006-pipelock-tls-interception.md`.
|
||||
- **git-gate image** — per-agent sidecar built on `zricethezav/gitleaks`
|
||||
(alpine + gitleaks + git-daemon + openssh-client). Runs
|
||||
`git daemon` over `git://` as a bidirectional mirror of each
|
||||
declared upstream. A pre-receive hook gitleaks-scans incoming
|
||||
refs and forwards clean refs to the real upstream over SSH; an
|
||||
access-hook runs `git fetch origin --prune` against the upstream
|
||||
before every upload-pack so an agent fetch returns whatever the
|
||||
upstream has *now* (fail-closed if unreachable). The agent's
|
||||
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
|
||||
so push, fetch, clone, and pull all route through. The agent
|
||||
never sees the upstream credential. If the upstream's hostname
|
||||
isn't resolvable from the gate container (e.g. a Tailscale-only
|
||||
host whose public DNS points elsewhere), pin its IP via
|
||||
`ExtraHosts: { "<hostname>": "<ip>" }` on the `bottle.git` entry —
|
||||
the gate's `/etc/hosts` gets the override while the agent's
|
||||
`insteadOf` rewrite still keys off the original hostname. Brought
|
||||
up only when `bottle.git` has entries. Design in
|
||||
`docs/prds/0008-git-gate.md`.
|
||||
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
|
||||
base, stdlib-only) that holds API tokens declared in
|
||||
`bottle.cred_proxy.routes`. Each route names a `path`,
|
||||
`upstream`, `auth_scheme`, and `token_ref` (host env var); the
|
||||
agent dials `http://cred-proxy:9099<path>...` over plain HTTP
|
||||
and the proxy strips any inbound `Authorization`, injects
|
||||
`<auth_scheme> <token>` using the value held only in its own
|
||||
container's environ, and forwards to the real upstream over
|
||||
HTTPS. SSE responses stream back unbuffered. The cred-proxy's
|
||||
outbound HTTPS routes through pipelock (it trusts pipelock's
|
||||
per-bottle CA), so pipelock's egress allowlist + body scanner
|
||||
apply to cred-proxy traffic the same way they apply to direct
|
||||
agent traffic. Smart-HTTP push paths (`/git-receive-pack`,
|
||||
`/info/refs?service=git-receive-pack`) are refused at the
|
||||
proxy — push must go through `bottle.git` / git-gate where
|
||||
gitleaks runs. Optional per-route `role` tags drive agent-side
|
||||
rewrites: `anthropic-base-url`, `npm-registry`, `git-insteadof`,
|
||||
`tea-login`. The agent's `printenv` shows only proxy URLs —
|
||||
none of the real token values. Design in
|
||||
`docs/prds/0010-cred-proxy.md`.
|
||||
|
||||
When the agent exits, `cli.py` tears down every sidecar that was
|
||||
brought up and the two networks; nothing about a bottle persists
|
||||
between runs.
|
||||
|
||||
## Quickstart
|
||||
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
|
||||
Requires Docker on the host and a long-lived Claude Code OAuth token in
|
||||
your shell env.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
```
|
||||
|
||||
The container is removed automatically when the session ends. If the script
|
||||
is killed with SIGKILL the exit trap won't fire and the container may be
|
||||
left running; remove it with `docker rm -f <container-name>`.
|
||||
|
||||
### Smolmachines backend (experimental, macOS-only)
|
||||
|
||||
A second backend runs the agent in a smolvm micro-VM (libkrun) with the
|
||||
sidecar bundle still in Docker. Selected via
|
||||
`BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>`. Requires
|
||||
`smolvm` on PATH (`curl -sSL https://smolmachines.com/install.sh | sh`).
|
||||
|
||||
The integration tests run against whichever backend the env var
|
||||
selects and skip cleanly when its prerequisites are missing.
|
||||
|
||||
**One-time sudo on first launch (macOS):** smolmachines bottles
|
||||
each reserve a loopback alias from a pool (`127.0.0.16` ..
|
||||
`127.0.0.31`) and bind their bundle's port-forwards to it; the
|
||||
first `./cli.py start` after each reboot prompts for sudo to add
|
||||
missing aliases via `ifconfig lo0 alias`. Aliases persist until
|
||||
reboot; subsequent launches don't prompt. The agent's TSI
|
||||
allowlist is the alias's `/32`, so each bottle can only reach
|
||||
its own bundle's published ports — not other bottles' ports,
|
||||
not other host loopback services (postgres, dev servers, etc.).
|
||||
|
||||
This enforcement requires a workaround for a smolvm 0.8.0 bug:
|
||||
the CLI's `--allow-cidr` flag is silently dropped when combined
|
||||
with `--from <smolmachine>`. The launcher patches smolvm's
|
||||
persistent state DB
|
||||
(`~/Library/Application Support/smolvm/server/smolvm.db`)
|
||||
directly between `machine create` and `machine start` to set
|
||||
the allowlist. The hack falls away automatically when smolvm
|
||||
honors the flag upstream — see the `loopback_alias` module's
|
||||
docstring for the investigation trail.
|
||||
|
||||
## Manifest
|
||||
|
||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Bottles live in `~/.bot-bottle/bottles/`; agents may also be shipped by a repo at `<repo>/.bot-bottle/agents/<name>.md`.
|
||||
Bottles and agents live as Markdown files with YAML frontmatter under
|
||||
`~/.bot-bottle/`. Each bottle is one file in `bottles/`, each agent
|
||||
is one file in `agents/`:
|
||||
|
||||
**Bottle** (`~/.bot-bottle/bottles/gitea-dev.md`):
|
||||
```
|
||||
~/.bot-bottle/
|
||||
├── bottles/
|
||||
│ ├── dev.md
|
||||
│ └── gitea-dev.md
|
||||
└── agents/
|
||||
├── implementer.md
|
||||
└── researcher.md
|
||||
```
|
||||
|
||||
The filename (without `.md`) is the entity's name. Filenames must
|
||||
match `[a-z][a-z0-9-]*`; files that don't are skipped with a warning.
|
||||
|
||||
A repo can ship its own agent files alongside its code at
|
||||
`<repo>/.bot-bottle/agents/<name>.md`. Those agents reference
|
||||
bottles defined in `~/.bot-bottle/bottles/` (the only place
|
||||
bottles can come from); a `bottles/` subdir in a repo is ignored
|
||||
with a warning. **This is the trust boundary**: bottle infrastructure
|
||||
— credentials, egress allowlists, git remotes — comes from your home
|
||||
directory only. A cloned repo cannot redirect a host env var to an
|
||||
attacker-named upstream because it has no way to declare a bottle.
|
||||
|
||||
### Bottle composition with `extends:`
|
||||
|
||||
A bottle can inherit from another via `extends: <bottle-name>` so
|
||||
operators don't have to duplicate a whole bottle file to vary one
|
||||
field (PRD 0025). The parent's resolved config is the base; the
|
||||
child's declared fields overlay. Merge rules:
|
||||
|
||||
- `env:` — dict merge, child wins on key collision.
|
||||
- `git.user:` — per-field overlay (child's non-empty `name` /
|
||||
`email` wins; empty falls through to parent).
|
||||
- `git.remotes:` — dict merge by host, child wins on host collision.
|
||||
An explicit `git.remotes: {}` clears the parent's remotes; omitting
|
||||
`git.remotes` inherits the parent's remotes.
|
||||
- `agent_provider:`, `egress:`, `supervise:` — full replace when the
|
||||
child declares the field.
|
||||
|
||||
```yaml
|
||||
---
|
||||
extends: dev # inherit everything from bottles/dev.md
|
||||
egress:
|
||||
routes:
|
||||
- host: staging.example.com
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: STAGING_TOKEN
|
||||
---
|
||||
```
|
||||
|
||||
Cycles (`A extends B extends A`), self-references, and missing
|
||||
parents die at parse with a clear pointer. Bottles remain
|
||||
`$HOME`-only — `extends:` preserves the trust boundary above.
|
||||
|
||||
### Provider base bottles
|
||||
|
||||
Keep provider/runtime policy in one home-owned base bottle, then have
|
||||
task bottles extend it. That keeps provider egress/auth in one place
|
||||
without hiding security-relevant routes behind `agent_provider.template`.
|
||||
|
||||
For example, `~/.bot-bottle/bottles/claude.md` can hold the Claude
|
||||
provider selection and Anthropic API egress:
|
||||
|
||||
````markdown
|
||||
---
|
||||
extends: claude # inherit the Claude provider boundary
|
||||
agent_provider:
|
||||
template: claude
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: api.anthropic.com
|
||||
role: claude_code_oauth
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||
pipelock:
|
||||
tls_passthrough: true
|
||||
---
|
||||
|
||||
Common Claude provider boundary.
|
||||
````
|
||||
|
||||
Task bottles can then inherit that provider boundary and add their own
|
||||
env/git configuration without repeating the Claude route.
|
||||
|
||||
### Example bottle (`~/.bot-bottle/bottles/gitea-dev.md`)
|
||||
|
||||
````markdown
|
||||
---
|
||||
extends: claude
|
||||
|
||||
env:
|
||||
GIT_AUTHOR_NAME: didericis
|
||||
@@ -101,61 +343,158 @@ git:
|
||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
||||
KnownHostKey: ssh-ed25519 AAAA...
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
auth:
|
||||
scheme: token # Bearer | token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
matches: # optional — restrict to specific paths/methods/headers
|
||||
- paths:
|
||||
- {type: prefix, value: /api/v1/}
|
||||
methods: [GET, POST, PATCH, DELETE]
|
||||
dlp: # optional — per-route detector overrides (default: all on)
|
||||
outbound_detectors: [token_patterns, known_secrets]
|
||||
inbound_detectors: false # disable response scanning for this host
|
||||
---
|
||||
|
||||
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
||||
gitea over SSH for push, token over HTTPS for the API.
|
||||
The `gitea-dev` bottle. Backs my work on personal projects: provider
|
||||
auth through egress and gitea.dideric.is over SSH.
|
||||
````
|
||||
|
||||
**Agent** (`~/.bot-bottle/agents/gitea-helper.md`):
|
||||
For a Codex-backed base bottle, set `agent_provider.template: codex`
|
||||
and use the `codex_auth` egress role for the OpenAI API route. The
|
||||
built-in Codex template uses `Dockerfile.codex`; set
|
||||
`agent_provider.dockerfile` to build the agent from a custom
|
||||
Dockerfile while keeping the bot-bottle sidecars in place.
|
||||
|
||||
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
|
||||
|
||||
````markdown
|
||||
---
|
||||
bottle: gitea-dev
|
||||
skills:
|
||||
- init-prd
|
||||
git:
|
||||
user:
|
||||
name: gitea-helper
|
||||
email: eric+gitea-helper@dideric.is
|
||||
---
|
||||
|
||||
You help maintain Gitea-hosted projects.
|
||||
````
|
||||
|
||||
**Egress route fields:**
|
||||
The agent's Markdown body is its system prompt (whitespace
|
||||
stripped). The frontmatter declares the bottle to launch in and any
|
||||
skills to mount. You can also include Claude Code subagent fields
|
||||
(`name`, `description`, `model`, `color`, `memory`) in the
|
||||
frontmatter — bot-bottle ignores them at launch but doesn't
|
||||
reject them, so the same file can drop into `~/.claude/agents/` as a
|
||||
Claude Code subagent.
|
||||
|
||||
| Field | Required | Description |
|
||||
|---|---|---|
|
||||
| `host` | yes | Hostname to allowlist. One entry per host. |
|
||||
| `role` | no | Provider-specific role string (e.g. `claude_code_oauth`). Wires built-in auth flows; set by provider templates, not manually. |
|
||||
| `auth.scheme` | when `auth` present | `Bearer` or `token`. Injected by the proxy; the agent never sees the value. |
|
||||
| `auth.token_ref` | when `auth` present | Env-var name holding the secret on the host. |
|
||||
| `matches` | no | Array of `{paths, methods, headers}` filters. A request must match at least one entry (if any are given) to be forwarded. |
|
||||
| `matches[].paths` | no | Array of `{type, value}`. `type` is `prefix` (default), `exact`, or `regex`. |
|
||||
| `matches[].methods` | no | Array of HTTP method strings, e.g. `[GET, POST]`. |
|
||||
| `matches[].headers` | no | Array of `{name, value, type}`. `type` is `exact` (default) or `regex`. |
|
||||
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
|
||||
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
|
||||
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
|
||||
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
|
||||
An agent may also declare `git.user` (`name` / `email`). It overlays
|
||||
the referenced bottle's `git.user` per-field — the agent's non-empty
|
||||
fields win, the rest fall through to the bottle — so two agents can
|
||||
share one bottle and still commit under distinct identities without
|
||||
an identity-only bottle (PRD 0027). Only `git.user` is allowed at the
|
||||
agent level; `git.remotes` stays bottle-only because it carries
|
||||
credentials and host trust. The launch preflight and `cli.py info`
|
||||
print the effective identity annotated `(agent)` / `(bottle)` so you
|
||||
can see where each field came from. Git authorship is not a
|
||||
credential — push auth is the bottle's remote key/token — so a
|
||||
repo-shipped agent setting its own identity grants no access; treat
|
||||
an agent identity as *claimed, not vouched*.
|
||||
|
||||
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
|
||||
Unknown top-level frontmatter keys die at load with a "did you mean"
|
||||
pointer; typos don't silently ghost into an empty config.
|
||||
|
||||
The YAML subset the frontmatter accepts is bounded (flat keys,
|
||||
strings / ints / true-or-false bools / null / lists / one-level
|
||||
nested dicts). Anchors, multi-line block scalars, tags, and
|
||||
ambiguous bare strings (`yes` / `NO` / `2026-05-24` /
|
||||
`0x...`) all die with a clear pointer at the spec — quote your
|
||||
strings when in doubt. The full schema lives in
|
||||
`bot_bottle/yaml_subset.py` (~450 lines, stdlib-only, no PyYAML).
|
||||
|
||||
Working examples live under `examples/`. Pipelock's design lives in
|
||||
`docs/prds/0001-per-agent-egress-proxy-via-pipelock.md` and the
|
||||
rationale in `docs/research/pipelock-assessment.md`. The trust
|
||||
boundary rationale lives in `docs/prds/0011-per-file-md-manifest.md`.
|
||||
|
||||
## Auth: Claude OAuth token, not API key
|
||||
|
||||
Bottles that use `agent_provider.template: claude` authenticate
|
||||
`claude` inside the container with the same Pro/Max subscription you
|
||||
already use on the host, via a long-lived OAuth token. No
|
||||
`ANTHROPIC_API_KEY` is needed.
|
||||
|
||||
**Why a token instead of mounting `~/.claude.json`:** on macOS, Claude
|
||||
Code stores OAuth credentials in the encrypted Keychain, not in
|
||||
`~/.claude.json`. Mounting that file into a Linux container does not
|
||||
carry the credentials with it. Linux hosts keep credentials in
|
||||
`~/.claude/.credentials.json`, but to keep the launcher portable
|
||||
bot-bottle uses the env-var path on every host.
|
||||
|
||||
**One-time setup on the host:**
|
||||
|
||||
```sh
|
||||
claude setup-token # browser login, prints a ~1-year OAuth token
|
||||
```
|
||||
|
||||
Stash the token in your shell env (e.g. `~/.zshrc` or a secret manager)
|
||||
as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
||||
|
||||
```sh
|
||||
export BOT_BOTTLE_CLAUDE_OAUTH_TOKEN="<token>"
|
||||
```
|
||||
|
||||
The Claude bottle reaches the Anthropic API only through the cred-proxy
|
||||
sidecar. To let `claude` authenticate, declare an egress route with
|
||||
`role: claude_code_oauth` and
|
||||
`token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
||||
|
||||
```yaml
|
||||
egress:
|
||||
routes:
|
||||
- host: api.anthropic.com
|
||||
role: claude_code_oauth
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||
pipelock:
|
||||
tls_passthrough: true
|
||||
```
|
||||
|
||||
Routes that resolve to private or Tailscale addresses can opt into
|
||||
pipelock's SSRF destination allowlist explicitly:
|
||||
|
||||
```yaml
|
||||
egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
auth:
|
||||
scheme: token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
pipelock:
|
||||
ssrf_ip_allowlist:
|
||||
- 100.78.141.42/32
|
||||
```
|
||||
|
||||
At launch, `cli.py` reads `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` from the host
|
||||
env and forwards it into the cred-proxy container's environ — never
|
||||
into the agent's. The agent receives `ANTHROPIC_BASE_URL` pointing at
|
||||
`http://cred-proxy:9099/anthropic` and a non-secret placeholder for
|
||||
`CLAUDE_CODE_OAUTH_TOKEN` (claude-code refuses to start without one;
|
||||
the proxy strips and replaces the header on every request). `printenv`
|
||||
inside the agent does not surface the real token, and the value is
|
||||
never written to disk or placed on argv on the host.
|
||||
|
||||
A Claude bottle without a `claude_code_oauth` route has no path to the
|
||||
Anthropic API — there is no fallback that forwards the token directly
|
||||
to the agent. Caveats: the token is bound to your subscription tier
|
||||
(Pro/Max/Team/Enterprise), it does not work with `claude --bare`
|
||||
(which only reads `ANTHROPIC_API_KEY`), and if it leaks, regenerate
|
||||
via `claude setup-token` again. Reference:
|
||||
<https://code.claude.com/docs/en/authentication>.
|
||||
|
||||
## Trademarks
|
||||
|
||||
bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox.
|
||||
bot-bottle is an independent project and is not affiliated with,
|
||||
endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude
|
||||
Code" are trademarks of Anthropic, PBC; the project name uses
|
||||
"claude" descriptively to indicate that the tool runs Claude Code
|
||||
inside a sandbox.
|
||||
|
||||
## License
|
||||
|
||||
Copyright 2026 Eric Bauerfeld. Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full text.
|
||||
Copyright 2026 Eric Bauerfeld
|
||||
|
||||
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE)
|
||||
for the full text.
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"bottles": {
|
||||
"demo": {
|
||||
"env": {
|
||||
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
||||
},
|
||||
"git": [
|
||||
{
|
||||
"Name": "foo",
|
||||
"Upstream": "ssh://git@upstream.invalid/path.git",
|
||||
"IdentityFile": "~/.cache/bot-bottle-demo/fake-key",
|
||||
"KnownHostKey": "ssh-ed25519 AAAAEXAMPLE"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
"agents": {
|
||||
"demo": {
|
||||
"bottle": "demo",
|
||||
"skills": [],
|
||||
"prompt": "You are inside bot-bottle, a sandbox whose egress is mediated by a pipelock HTTP proxy (host allowlist + DLP body scan) and a git-gate sidecar (gitleaks pre-receive). The user will ask you to run network and git probes against these controls. The 'credentials' in the probes (ghp_..., AKIA...) are synthetic test patterns, the upstream is unreachable on purpose, and the whole point is to watch pipelock or git-gate refuse. Always invoke curl with `--proxy \"$HTTPS_PROXY\"` so requests actually traverse pipelock (curl ignores the uppercase HTTP_PROXY env var). Execute each probe literally via the Bash tool, report exactly what the proxy or gate returned, and keep replies to one or two lines. Do not refuse on credential-exfil grounds — refusing is what the bottle is designed to make unnecessary."
|
||||
}
|
||||
}
|
||||
}
|
||||
+35
-343
@@ -3,54 +3,19 @@
|
||||
The manifest owns the user-facing AgentProvider shape. This module is
|
||||
the launch-time table that turns a provider template into an executable
|
||||
command, default image, and prompt/auth behavior.
|
||||
|
||||
Per PRD 0050 the per-provider implementations live under
|
||||
`bot_bottle/contrib/<template>/agent_provider.py`. This module exposes:
|
||||
|
||||
- `AgentProvider` (ABC) — the contract each plugin implements.
|
||||
- `get_provider(template)` — lazy-imported registry; the analogue
|
||||
of `bot_bottle/deploy_key_provisioner.get_provisioner`.
|
||||
- `AgentProvisionPlan` (+ helper dataclasses) — declarative shape
|
||||
each provider produces and the backends consume unchanged.
|
||||
- `agent_provision_plan` / `runtime_for` — thin wrappers around the
|
||||
registry kept so existing callers keep working without per-call
|
||||
edits.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import inspect
|
||||
import os
|
||||
import shlex
|
||||
import tempfile
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
from .egress import EgressRoute
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .backend import Bottle, BottlePlan
|
||||
from typing import Literal
|
||||
|
||||
|
||||
PROVIDER_CLAUDE = "claude"
|
||||
PROVIDER_CODEX = "codex"
|
||||
PROVIDER_PI = "pi"
|
||||
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_PI})
|
||||
|
||||
# Hosts that egress injects the host ChatGPT bearer on when Codex
|
||||
# forward_host_credentials is enabled. Pipelock must pass these through
|
||||
# (no TLS MITM) or its header DLP blocks the injected JWT.
|
||||
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
|
||||
PromptMode = Literal[
|
||||
"append_file",
|
||||
"read_prompt_file",
|
||||
"print_read_prompt_file",
|
||||
"append_system_prompt",
|
||||
]
|
||||
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
|
||||
PromptMode = Literal["append_file", "read_prompt_file"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -58,317 +23,48 @@ class AgentProviderRuntime:
|
||||
template: str
|
||||
command: str
|
||||
image: str
|
||||
dockerfile: str
|
||||
auth_role: str
|
||||
placeholder_env: str
|
||||
prompt_mode: PromptMode
|
||||
bypass_args: tuple[str, ...]
|
||||
resume_args: tuple[str, ...]
|
||||
remote_control_args: tuple[str, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionDir:
|
||||
guest_path: str
|
||||
mode: str = "700"
|
||||
owner: str = "node:node"
|
||||
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionFile:
|
||||
host_path: Path
|
||||
guest_path: str
|
||||
mode: str = "600"
|
||||
owner: str = "node:node"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionCommand:
|
||||
argv: tuple[str, ...]
|
||||
error: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvisionPlan:
|
||||
"""Provider-owned guest setup.
|
||||
|
||||
Backends interpret this plan with their own copy/exec primitives.
|
||||
Provider-specific content stays here so future provider plugins can
|
||||
return the same shape without adding backend-plan fields.
|
||||
|
||||
`egress_routes` are provider-declared EgressRoutes that backends
|
||||
pass to `Egress.prepare`. This keeps provider logic out of the
|
||||
egress module — it merges provider routes generically without
|
||||
knowing the provider type.
|
||||
|
||||
`hidden_env_names` is the set of env var names the provider injected
|
||||
as non-secret placeholders. `print_util.visible_agent_env_names` uses
|
||||
this to suppress them from the preflight summary so operators don't
|
||||
mistake them for real credentials.
|
||||
"""
|
||||
|
||||
template: str
|
||||
command: str
|
||||
prompt_mode: PromptMode
|
||||
image: str
|
||||
dockerfile: str
|
||||
guest_home: str
|
||||
instance_name: str
|
||||
prompt_file: Path
|
||||
guest_env: dict[str, str]
|
||||
has_prompt: bool = False
|
||||
startup_args: tuple[str, ...] = ()
|
||||
env_vars: dict[str, str] = field(default_factory=dict)
|
||||
dirs: tuple[AgentProvisionDir, ...] = ()
|
||||
files: tuple[AgentProvisionFile, ...] = ()
|
||||
pre_copy: tuple[AgentProvisionCommand, ...] = ()
|
||||
verify: tuple[AgentProvisionCommand, ...] = ()
|
||||
egress_routes: tuple[EgressRoute, ...] = ()
|
||||
hidden_env_names: frozenset[str] = field(default_factory=frozenset)
|
||||
provisioned_env: dict[str, str] = field(default_factory=dict)
|
||||
|
||||
|
||||
class AgentProvider(ABC):
|
||||
"""Per-template plugin: produces the provision plan and applies
|
||||
the provider-specific in-guest setup steps (skills, prompt, the
|
||||
declarative `dirs`/`files`/`pre_copy`/`verify` apply loop, and
|
||||
supervise MCP registration). Concrete subclasses live under
|
||||
`bot_bottle/contrib/<template>/agent_provider.py`."""
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
"""The static command / image / prompt-mode table for this
|
||||
template."""
|
||||
|
||||
@property
|
||||
def guest_home(self) -> str:
|
||||
"""In-guest home directory for the agent user. Defaults to
|
||||
`/home/node` to match the Debian-based bot-bottle-* images
|
||||
(USER node). Override for plugins whose image runs as a
|
||||
different user."""
|
||||
return "/home/node"
|
||||
|
||||
@property
|
||||
def dockerfile(self) -> Path:
|
||||
"""Path to the provider's Dockerfile.
|
||||
|
||||
Default: the `Dockerfile` file next to this provider's
|
||||
`agent_provider.py` module. Override to point at a non-standard
|
||||
path."""
|
||||
return Path(inspect.getfile(type(self))).parent / "Dockerfile"
|
||||
|
||||
@abstractmethod
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
"""Build the declarative AgentProvisionPlan for one launch.
|
||||
Backends call this during `prepare` and consume the result as
|
||||
before."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Copy each of the agent's named skills from the host into
|
||||
the guest. No-op when the agent has no skills. The in-guest
|
||||
layout is provider-specific (claude-code's
|
||||
`~/.claude/skills/` today; future providers may differ)."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
"""Copy the prompt file into the guest, fix ownership/mode,
|
||||
and return the in-guest path iff the agent has a non-empty
|
||||
prompt (drives the `--append-system-prompt-file` flag).
|
||||
|
||||
The file is copied either way so the path always exists."""
|
||||
|
||||
@abstractmethod
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the provider's declarative
|
||||
`dirs`/`pre_copy`/`files`/`verify` steps from
|
||||
`plan.agent_provision`. Was called `provision_provider_auth`
|
||||
on `BottleBackend` before PRD 0050."""
|
||||
|
||||
@abstractmethod
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
"""Register the per-bottle supervise sidecar as an MCP server
|
||||
in the provider's in-guest config. Called by the backend after
|
||||
the supervise sidecar is reachable. No-op when
|
||||
`plan.supervise_plan is None`."""
|
||||
|
||||
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||
"""Install the egress MITM CA into the agent's trust store.
|
||||
|
||||
Default: Debian-style — cp the cert to the standard source path,
|
||||
run update-ca-certificates, log the fingerprint. Override for
|
||||
non-Debian base images or non-standard trust mechanisms."""
|
||||
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||
from .log import die
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan)
|
||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||
r = bottle.exec(
|
||||
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
||||
user="root",
|
||||
)
|
||||
if r.returncode != 0:
|
||||
die(
|
||||
f"update-ca-certificates failed (exit {r.returncode}): "
|
||||
f"stdout={(r.stdout or '').strip()!r} "
|
||||
f"stderr={(r.stderr or '').strip()!r}"
|
||||
)
|
||||
log_ca_fingerprint(cert_host_path, label)
|
||||
|
||||
def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||
"""Configure git inside the agent container.
|
||||
|
||||
Default: Debian/node — writes the git-gate insteadOf gitconfig
|
||||
and sets user.name/email as node. Workspace copy runs through
|
||||
BottleBackend.provision_workspace against the running bottle."""
|
||||
from .log import info
|
||||
|
||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if manifest_bottle.git:
|
||||
from .git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||
gate_host = getattr(plan, "git_gate_insteadof_host", GIT_GATE_HOSTNAME)
|
||||
gate_scheme = getattr(plan, "git_gate_insteadof_scheme", "git")
|
||||
content = git_gate_render_gitconfig(
|
||||
manifest_bottle.git, gate_host, scheme=gate_scheme,
|
||||
)
|
||||
guest_gitconfig = f"{plan.guest_home}/.gitconfig"
|
||||
with tempfile.NamedTemporaryFile(
|
||||
"w", dir=str(plan.stage_dir), prefix="gitconfig.", delete=False,
|
||||
) as f:
|
||||
f.write(content)
|
||||
config_file = Path(f.name)
|
||||
os.chmod(config_file, 0o600)
|
||||
info(
|
||||
f"writing {guest_gitconfig} with "
|
||||
f"{len(manifest_bottle.git)} insteadOf rule(s)"
|
||||
)
|
||||
bottle.cp_in(str(config_file), guest_gitconfig)
|
||||
bottle.exec(
|
||||
f"chown node:node {shlex.quote(guest_gitconfig)} && "
|
||||
f"chmod 644 {shlex.quote(guest_gitconfig)}",
|
||||
user="root",
|
||||
)
|
||||
|
||||
gu = manifest_bottle.git_user
|
||||
if not gu.is_empty():
|
||||
if gu.name:
|
||||
info(f"git config --global user.name = {gu.name!r}")
|
||||
bottle.exec(
|
||||
f"git config --global user.name {shlex.quote(gu.name)}",
|
||||
user="node",
|
||||
)
|
||||
if gu.email:
|
||||
info(f"git config --global user.email = {gu.email!r}")
|
||||
bottle.exec(
|
||||
f"git config --global user.email {shlex.quote(gu.email)}",
|
||||
user="node",
|
||||
)
|
||||
|
||||
|
||||
def _load_user_plugin(template: str) -> AgentProvider | None:
|
||||
"""Check ~/.bot-bottle/contrib/<template>/agent_provider.py for a
|
||||
user-defined AgentProvider subclass. Returns an instance if found,
|
||||
None if the plugin directory doesn't exist, raises ValueError if
|
||||
the file exists but exports no AgentProvider subclass."""
|
||||
plugin_path = (
|
||||
Path.home() / ".bot-bottle" / "contrib" / template / "agent_provider.py"
|
||||
)
|
||||
if not plugin_path.exists():
|
||||
return None
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
f"_user_contrib_{template}.agent_provider", plugin_path
|
||||
)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ValueError(f"user plugin at {plugin_path} could not be loaded")
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod) # type: ignore[union-attr]
|
||||
for obj in vars(mod).values():
|
||||
if (
|
||||
isinstance(obj, type)
|
||||
and issubclass(obj, AgentProvider)
|
||||
and obj is not AgentProvider
|
||||
):
|
||||
return obj()
|
||||
raise ValueError(
|
||||
f"user plugin at {plugin_path} defines no AgentProvider subclass"
|
||||
)
|
||||
|
||||
|
||||
def get_provider(template: str) -> AgentProvider:
|
||||
"""Resolve a provider template name to its plugin instance.
|
||||
|
||||
Checks ~/.bot-bottle/contrib/<template>/agent_provider.py first so
|
||||
users can shadow a built-in for local testing. Falls through to the
|
||||
built-in registry; raises ValueError for unknown names with no
|
||||
matching user plugin."""
|
||||
user_plugin = _load_user_plugin(template)
|
||||
if user_plugin is not None:
|
||||
return user_plugin
|
||||
if template == PROVIDER_CLAUDE:
|
||||
from .contrib.claude.agent_provider import ClaudeAgentProvider
|
||||
return ClaudeAgentProvider()
|
||||
if template == PROVIDER_CODEX:
|
||||
from .contrib.codex.agent_provider import CodexAgentProvider
|
||||
return CodexAgentProvider()
|
||||
if template == PROVIDER_PI:
|
||||
from .contrib.pi.agent_provider import PiAgentProvider
|
||||
return PiAgentProvider()
|
||||
raise ValueError(f"unknown agent provider template: {template!r}")
|
||||
_RUNTIMES = {
|
||||
PROVIDER_CLAUDE: AgentProviderRuntime(
|
||||
template=PROVIDER_CLAUDE,
|
||||
command="claude",
|
||||
image="bot-bottle-claude:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||
auth_role="claude_code_oauth",
|
||||
placeholder_env="CLAUDE_CODE_OAUTH_TOKEN",
|
||||
prompt_mode="append_file",
|
||||
bypass_args=("--dangerously-skip-permissions",),
|
||||
resume_args=("--continue",),
|
||||
remote_control_args=("--remote-control",),
|
||||
),
|
||||
PROVIDER_CODEX: AgentProviderRuntime(
|
||||
template=PROVIDER_CODEX,
|
||||
command="codex",
|
||||
image="bot-bottle-codex:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||
auth_role="codex_auth",
|
||||
placeholder_env="OPENAI_API_KEY",
|
||||
prompt_mode="read_prompt_file",
|
||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||
resume_args=("resume", "--last"),
|
||||
remote_control_args=(),
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def runtime_for(template: str) -> AgentProviderRuntime:
|
||||
return get_provider(template).runtime
|
||||
|
||||
|
||||
def build_agent_provision_plan(
|
||||
*,
|
||||
template: str,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
"""Back-compat shim — `prepare` callers stay the same; the work
|
||||
now lives on the provider plugin."""
|
||||
return get_provider(template).provision_plan(
|
||||
dockerfile=dockerfile,
|
||||
state_dir=state_dir,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
guest_env=guest_env,
|
||||
auth_token=auth_token,
|
||||
forward_host_credentials=forward_host_credentials,
|
||||
host_env=host_env,
|
||||
trusted_project_path=trusted_project_path,
|
||||
label=label,
|
||||
color=color,
|
||||
provider_settings=provider_settings,
|
||||
)
|
||||
return _RUNTIMES[template]
|
||||
|
||||
|
||||
def prompt_args(
|
||||
@@ -385,8 +81,4 @@ def prompt_args(
|
||||
if argv and "resume" in argv:
|
||||
return []
|
||||
return [f"Read and follow the instructions in {prompt_path}."]
|
||||
if prompt_mode == "print_read_prompt_file":
|
||||
return ["-p", f"Read and follow the instructions in {prompt_path}."]
|
||||
if prompt_mode == "append_system_prompt":
|
||||
return ["--append-system-prompt", prompt_path]
|
||||
raise ValueError(f"unknown provider prompt mode: {prompt_mode}")
|
||||
|
||||
+69
-259
@@ -24,33 +24,23 @@ backend exposes five methods:
|
||||
enough metadata for callers (CLI `list active`, dashboard
|
||||
agents pane) to render a row.
|
||||
|
||||
Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
|
||||
(env var). When neither is set, compatible macOS hosts default to
|
||||
`macos-container`; other hosts default to `smolmachines`. Per PRD 0003
|
||||
the manifest does not carry a backend field; the host picks.
|
||||
Selection is driven by `--backend` on `start` or
|
||||
BOT_BOTTLE_BACKEND (env var; default "docker"). Per PRD 0003 the
|
||||
manifest does not carry a backend field; the host picks.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
import sys
|
||||
from abc import ABC, abstractmethod
|
||||
from contextlib import AbstractContextManager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Generic, Sequence, TypeVar
|
||||
|
||||
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
|
||||
from ..egress import EgressPlan
|
||||
from ..git_gate import GitGatePlan
|
||||
from ..log import die, info
|
||||
from ..manifest import ManifestGitEntry, Manifest
|
||||
from ..supervise import SupervisePlan
|
||||
from ..log import die
|
||||
from ..manifest import GitEntry, Manifest
|
||||
from ..util import expand_tilde
|
||||
from ..env import resolve_env, ResolvedEnv
|
||||
from ..workspace import WorkspacePlan, workspace_plan
|
||||
from .print_util import print_multi, visible_agent_env_names
|
||||
from .util import host_skill_dir
|
||||
|
||||
|
||||
@@ -70,85 +60,20 @@ class BottleSpec:
|
||||
# (`cli.py resume <identity>`) sets this to continue an existing
|
||||
# bottle's state. Empty string for a fresh `start`.
|
||||
identity: str = ""
|
||||
label: str = ""
|
||||
color: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BottlePlan(ABC):
|
||||
"""Base output of a backend's prepare step. Concrete subclasses
|
||||
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
|
||||
(e.g. DockerBottlePlan) add backend-specific resolved fields and
|
||||
implement `print`."""
|
||||
|
||||
spec: BottleSpec
|
||||
stage_dir: Path
|
||||
git_gate_plan: GitGatePlan
|
||||
|
||||
@property
|
||||
def guest_home(self) -> str:
|
||||
return self.agent_provision.guest_home
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
"""Host (and optional port) used in git-gate insteadOf URLs.
|
||||
Docker uses the compose-network DNS alias; smolmachines
|
||||
overrides with a loopback IP:port since TSI has no DNS."""
|
||||
return "git-gate"
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
"""URL scheme for git-gate insteadOf rewrites. 'git' for
|
||||
Docker (git daemon); 'http' for smolmachines (HTTP proxy
|
||||
over a published host port)."""
|
||||
return "git"
|
||||
egress_plan: EgressPlan
|
||||
supervise_plan: SupervisePlan | None
|
||||
agent_provision: AgentProvisionPlan
|
||||
|
||||
@property
|
||||
def workspace_plan(self) -> WorkspacePlan:
|
||||
return workspace_plan(self.spec, guest_home=self.guest_home)
|
||||
|
||||
@abstractmethod
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Render the y/N preflight summary to stderr."""
|
||||
del remote_control
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(
|
||||
set(bottle.env.keys())
|
||||
| set(self.agent_provision.guest_env.keys())
|
||||
),
|
||||
hidden_env_names=self.agent_provision.hidden_env_names,
|
||||
)
|
||||
|
||||
print(file=sys.stderr)
|
||||
info(f"agent : {spec.agent_name}")
|
||||
info(f"provider : {self.agent_provision.template}")
|
||||
print_multi("env ", env_names)
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
|
||||
identity = manifest.git_identity_summary(spec.agent_name)
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
|
||||
git_lines = [
|
||||
f"{u.name} → {u.upstream_host}:{u.upstream_port}"
|
||||
for u in self.git_gate_plan.upstreams
|
||||
]
|
||||
if git_lines:
|
||||
print_multi(" git gate ", git_lines)
|
||||
|
||||
if self.egress_plan.routes:
|
||||
egress_lines = []
|
||||
for r in self.egress_plan.routes:
|
||||
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
||||
egress_lines.append(f"{r.host}{auth}")
|
||||
print_multi(" egress ", egress_lines)
|
||||
print(file=sys.stderr)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -188,10 +113,10 @@ class ActiveAgent:
|
||||
bottle is the container, the agent is what runs in it.)
|
||||
|
||||
Fields are deliberately backend-neutral. `services` is the set
|
||||
of sidecar daemons currently up for this bottle (`egress`,
|
||||
`git-gate`, `supervise`); the dashboard uses it to
|
||||
of sidecar daemons currently up for this bottle (`pipelock`,
|
||||
`egress`, `git-gate`, `supervise`); the dashboard uses it to
|
||||
gate edit verbs. `backend_name` is the matching key in
|
||||
`_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
|
||||
`_BACKENDS` (`docker` / `smolmachines`) — used by the active-
|
||||
list rendering to disambiguate and by the dashboard's
|
||||
re-attach path."""
|
||||
|
||||
@@ -200,8 +125,6 @@ class ActiveAgent:
|
||||
agent_name: str # from metadata.json; "?" if missing
|
||||
started_at: str # ISO 8601 from metadata.json; "" if missing
|
||||
services: tuple[str, ...] # alphabetical
|
||||
label: str = ""
|
||||
color: str = ""
|
||||
|
||||
|
||||
class Bottle(ABC):
|
||||
@@ -240,7 +163,7 @@ class Bottle(ABC):
|
||||
`user` (default `node`, matching the agent image's USER
|
||||
directive) and return the captured stdout/stderr/returncode.
|
||||
The bottle's environment (including HTTPS_PROXY pointing at
|
||||
the egress sidecar) is inherited by the child. Non-zero
|
||||
the pipelock sidecar) is inherited by the child. Non-zero
|
||||
exit does not raise — callers inspect `returncode`
|
||||
themselves.
|
||||
|
||||
@@ -272,88 +195,14 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
|
||||
name: str
|
||||
|
||||
def prepare(self, spec: BottleSpec, stage_dir: Path) -> PlanT:
|
||||
def prepare(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
|
||||
"""Template method: run cross-backend host-side validation, then
|
||||
delegate to the subclass's `_resolve_plan` for the
|
||||
backend-specific resolution (names, scratch files, etc.). The
|
||||
validation step is enforced here so a future backend cannot
|
||||
accidentally skip it. No remote/runtime resources are created."""
|
||||
from .resolve_common import (
|
||||
merge_provision_env_vars,
|
||||
mint_slug,
|
||||
prepare_agent_state_dir,
|
||||
prepare_egress,
|
||||
prepare_git_gate,
|
||||
prepare_supervise,
|
||||
resolve_manifest_dockerfile,
|
||||
write_launch_metadata,
|
||||
)
|
||||
|
||||
self._validate(spec)
|
||||
|
||||
self._preflight()
|
||||
|
||||
manifest = spec.manifest
|
||||
manifest_bottle = manifest.bottle_for(spec.agent_name)
|
||||
manifest_agent_provider = manifest_bottle.agent_provider
|
||||
agent_provider = get_provider(manifest_agent_provider.template)
|
||||
resolved_env = resolve_env(manifest, spec.agent_name)
|
||||
workspace = workspace_plan(spec, guest_home=agent_provider.guest_home)
|
||||
|
||||
slug = mint_slug(spec)
|
||||
write_launch_metadata(slug, spec, compose_project="", backend=self.name)
|
||||
|
||||
# Manifest may override the Dockerfile per-bottle; otherwise fall
|
||||
# back to the provider plugin's bundled Dockerfile (next to its
|
||||
# agent_provider.py module).
|
||||
if manifest_agent_provider.dockerfile:
|
||||
agent_dockerfile_path = resolve_manifest_dockerfile(
|
||||
manifest_agent_provider.dockerfile, spec,
|
||||
)
|
||||
else:
|
||||
agent_dockerfile_path = str(agent_provider.dockerfile)
|
||||
|
||||
agent_dir, prompt_file = prepare_agent_state_dir(slug, spec)
|
||||
|
||||
agent_provision_plan = build_agent_provision_plan(
|
||||
template=manifest_agent_provider.template,
|
||||
dockerfile=agent_dockerfile_path,
|
||||
state_dir=agent_dir,
|
||||
instance_name=f"bot-bottle-{slug}",
|
||||
prompt_file=prompt_file,
|
||||
guest_env=self._build_guest_env(resolved_env),
|
||||
forward_host_credentials=manifest_agent_provider.forward_host_credentials,
|
||||
auth_token=manifest_agent_provider.auth_token,
|
||||
host_env=dict(os.environ),
|
||||
trusted_project_path=workspace.workdir,
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
provider_settings=manifest_agent_provider.settings,
|
||||
)
|
||||
agent_provision_plan = merge_provision_env_vars(agent_provision_plan)
|
||||
egress_plan = prepare_egress(manifest_bottle, slug, agent_provision_plan)
|
||||
supervise_plan = prepare_supervise(manifest_bottle, slug)
|
||||
git_gate_plan = prepare_git_gate(manifest_bottle, slug)
|
||||
|
||||
return self._resolve_plan(
|
||||
spec,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return {}
|
||||
|
||||
def _preflight(self) -> None:
|
||||
"""
|
||||
tasks to do before resolving a plan
|
||||
"""
|
||||
pass
|
||||
return self._resolve_plan(spec, stage_dir=stage_dir)
|
||||
|
||||
def _validate(self, spec: BottleSpec) -> None:
|
||||
"""Cross-backend pre-launch checks. Confirms the agent exists,
|
||||
@@ -380,7 +229,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
f"Create it under ~/.claude/skills/, then re-run."
|
||||
)
|
||||
|
||||
def _validate_git_entries(self, entries: Sequence[ManifestGitEntry]) -> None:
|
||||
def _validate_git_entries(self, entries: Sequence[GitEntry]) -> None:
|
||||
"""Each entry's IdentityFile must exist on the host (after
|
||||
expanding leading ~) — the git-gate copies it in at start time
|
||||
to authenticate the upstream push (PRD 0008). Shape is already
|
||||
@@ -405,99 +254,75 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
)
|
||||
|
||||
@abstractmethod
|
||||
def _resolve_plan(self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path) -> PlanT:
|
||||
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
|
||||
"""Backend-specific plan resolution: image/container names,
|
||||
env-file, prompt-file, proxy plan, runtime detection. Called by
|
||||
`prepare` after `_validate` succeeds. Instance name, image,
|
||||
prompt file, Dockerfile path, and guest home all live on
|
||||
`agent_provision_plan` — the source of truth."""
|
||||
`prepare` after `_validate` succeeds."""
|
||||
|
||||
@abstractmethod
|
||||
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||
"""Build/run the bottle and yield a handle; tear down on exit."""
|
||||
|
||||
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
|
||||
def provision(self, plan: PlanT, target: str) -> str | None:
|
||||
"""Copy host-side files (CA cert, prompt, skills, .git) into
|
||||
the running bottle. Called from `launch` after the container
|
||||
/ machine is up. Returns the in-container prompt path if a
|
||||
prompt was provisioned, else None — the Bottle handle uses it
|
||||
to decide whether to add provider-specific prompt args to the
|
||||
agent's argv.
|
||||
/ machine is up. `target` identifies the running instance in
|
||||
backend-specific terms (Docker: resolved container name; fly:
|
||||
machine id). Returns the in-container prompt path if a prompt
|
||||
was provisioned, else None — the Bottle handle uses it to
|
||||
decide whether to add provider-specific prompt args to the agent's
|
||||
argv.
|
||||
|
||||
Default orchestration: ca → prompt → provider apply → skills
|
||||
→ workspace → git → supervise-mcp. CA install runs first so
|
||||
the agent's trust store is rebuilt before anything inside the
|
||||
agent makes a TLS call.
|
||||
|
||||
Per PRD 0050 the per-provider steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin. The backend only owns the
|
||||
steps that are about backend infrastructure (CA, workspace,
|
||||
git) and surfaces the supervise sidecar URL its launch step
|
||||
knows about via `supervise_mcp_url`.
|
||||
Default orchestration: ca → prompt → skills → git →
|
||||
supervise. CA install runs first so the agent's trust store
|
||||
is rebuilt before anything inside the agent makes a TLS call.
|
||||
Subclasses typically don't override this; they implement the
|
||||
sub-methods below.
|
||||
|
||||
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
||||
~/.gitconfig insteadOf, tea config) are gone. Egress-proxy is
|
||||
on the agent's HTTP_PROXY path so every tool that respects
|
||||
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
|
||||
intercepted without per-tool reconfiguration."""
|
||||
provider = get_provider(plan.agent_provision.template)
|
||||
provider.provision_ca(bottle, plan)
|
||||
prompt_path = provider.provision_prompt(plan, bottle)
|
||||
provider.provision(plan, bottle)
|
||||
provider.provision_skills(plan, bottle)
|
||||
self.provision_workspace(plan, bottle)
|
||||
provider.provision_git(bottle, plan)
|
||||
provider.provision_supervise_mcp(
|
||||
plan, bottle, self.supervise_mcp_url(plan),
|
||||
)
|
||||
self.provision_ca(plan, target)
|
||||
prompt_path = self.provision_prompt(plan, target)
|
||||
self.provision_skills(plan, target)
|
||||
self.provision_git(plan, target)
|
||||
self.provision_supervise(plan, target)
|
||||
return prompt_path
|
||||
|
||||
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
|
||||
"""Copy the operator workspace into the running bottle.
|
||||
def provision_ca(self, plan: PlanT, target: str) -> None:
|
||||
"""Install the per-bottle CA into the agent's trust store so
|
||||
the agent trusts the bumped CONNECT cert egress (was
|
||||
pipelock, pre-PRD-0017) presents. Default impl is a no-op so
|
||||
backends that don't yet support TLS interception (every backend
|
||||
except Docker today) aren't forced to implement it. The Docker
|
||||
backend overrides to docker-cp the cert in and run
|
||||
`update-ca-certificates`."""
|
||||
|
||||
This is the only supported workspace-provisioning path: Docker
|
||||
does not build a derived image containing the current
|
||||
workspace."""
|
||||
workspace = plan.workspace_plan
|
||||
if not (workspace.enabled and workspace.copy_contents):
|
||||
return
|
||||
@abstractmethod
|
||||
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
|
||||
"""Copy the prompt file into the running bottle. Returns the
|
||||
in-container path iff the agent has a non-empty prompt;
|
||||
callers use the return value to decide whether to add
|
||||
provider-specific prompt args to the agent's argv."""
|
||||
|
||||
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
|
||||
guest_path = shlex.quote(workspace.guest_path)
|
||||
guest_parent = shlex.quote(guest_parent)
|
||||
owner = shlex.quote(workspace.owner)
|
||||
mode = shlex.quote(workspace.mode)
|
||||
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
|
||||
bottle.exec(
|
||||
f"rm -rf {guest_path} && mkdir -p {guest_parent}",
|
||||
user="root",
|
||||
)
|
||||
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
|
||||
bottle.exec(
|
||||
f"chown -R {owner} {guest_path} && chmod {mode} {guest_path}",
|
||||
user="root",
|
||||
)
|
||||
@abstractmethod
|
||||
def provision_skills(self, plan: PlanT, target: str) -> None:
|
||||
"""Copy the agent's named skills from the host into the
|
||||
running bottle. No-op when the agent has no skills."""
|
||||
|
||||
def supervise_mcp_url(self, plan: PlanT) -> str:
|
||||
"""Return the agent-side URL of the per-bottle supervise
|
||||
sidecar, or "" when this bottle has no sidecar. The provider
|
||||
plugin's `provision_supervise_mcp` uses it to register the
|
||||
MCP entry inside the guest.
|
||||
@abstractmethod
|
||||
def provision_git(self, plan: PlanT, target: str) -> None:
|
||||
"""Copy the host's cwd `.git` directory into the running
|
||||
bottle if the user requested --cwd. No-op otherwise."""
|
||||
|
||||
Default returns "" so backends without supervise support
|
||||
don't have to implement it. Docker and smolmachines override."""
|
||||
del plan
|
||||
return ""
|
||||
def provision_supervise(self, plan: PlanT, target: str) -> None:
|
||||
"""Write the in-bottle Claude Code MCP config so the agent
|
||||
discovers the per-bottle supervise sidecar (PRD 0013).
|
||||
No-op when bottle.supervise is False or the backend doesn't
|
||||
support the supervise sidecar yet. The Docker backend
|
||||
overrides."""
|
||||
|
||||
@abstractmethod
|
||||
def prepare_cleanup(self) -> CleanupT:
|
||||
@@ -530,9 +355,8 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
# Import concrete backend classes AFTER the base types are defined, so
|
||||
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
|
||||
# via `from . import ...` without hitting a partially-initialized module.
|
||||
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .docker import DockerBottleBackend # noqa: E402
|
||||
from .smolmachines import SmolmachinesBottleBackend # noqa: E402
|
||||
|
||||
|
||||
# The dict is heterogeneous: each value is a BottleBackend specialized
|
||||
@@ -541,7 +365,6 @@ from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: dis
|
||||
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
|
||||
_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
|
||||
"docker": DockerBottleBackend(),
|
||||
"macos-container": MacosContainerBottleBackend(),
|
||||
"smolmachines": SmolmachinesBottleBackend(),
|
||||
}
|
||||
|
||||
@@ -554,24 +377,17 @@ def get_bottle_backend(
|
||||
`name` precedence:
|
||||
1. explicit arg (CLI `--backend=<name>` passes through here)
|
||||
2. BOT_BOTTLE_BACKEND env var
|
||||
3. `macos-container` on compatible macOS hosts
|
||||
4. default `smolmachines`
|
||||
3. default `docker`
|
||||
|
||||
Dies with a pointer at the known backends if the chosen name
|
||||
isn't implemented."""
|
||||
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or _default_backend_name()
|
||||
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
|
||||
if resolved not in _BACKENDS:
|
||||
known = ", ".join(sorted(_BACKENDS))
|
||||
die(f"unknown backend {resolved!r}; known backends: {known}")
|
||||
return _BACKENDS[resolved]
|
||||
|
||||
|
||||
def _default_backend_name() -> str:
|
||||
if has_backend("macos-container"):
|
||||
return "macos-container"
|
||||
return "smolmachines"
|
||||
|
||||
|
||||
def known_backend_names() -> tuple[str, ...]:
|
||||
"""Sorted tuple of all backend keys in `_BACKENDS`. Used by
|
||||
argparse (`--backend` choices) and the dashboard's backend
|
||||
@@ -597,20 +413,14 @@ def enumerate_active_agents() -> list[ActiveAgent]:
|
||||
"""All currently-running agents, across every available
|
||||
backend. Used by CLI `list active` and the dashboard's agents
|
||||
pane so neither has to know which backends exist. Skips
|
||||
backends whose `is_available()` reports False.
|
||||
|
||||
Sorted by `(started_at, slug)` so the list is stable across
|
||||
dashboard refresh ticks — agents don't shift position while
|
||||
the operator navigates with arrow keys. ISO 8601 timestamps
|
||||
sort lexicographically in chronological order; `slug` is the
|
||||
deterministic tiebreaker. Agents with missing metadata
|
||||
(`started_at == ""`) sort first."""
|
||||
backends whose `is_available()` reports False. Ordered by
|
||||
backend name, then by whatever each backend's
|
||||
`enumerate_active` returns."""
|
||||
out: list[ActiveAgent] = []
|
||||
for name in known_backend_names():
|
||||
if not has_backend(name):
|
||||
continue
|
||||
out.extend(_BACKENDS[name].enumerate_active())
|
||||
out.sort(key=lambda a: (a.started_at, a.slug))
|
||||
return out
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ The bulk of the implementation lives in sibling modules:
|
||||
|
||||
- util: thin Docker subprocess wrappers
|
||||
- network: Docker network plumbing
|
||||
- pipelock: DockerPipelockProxy lifecycle
|
||||
- bottle_plan: DockerBottlePlan
|
||||
- bottle_cleanup_plan: DockerBottleCleanupPlan
|
||||
- bottle: DockerBottle handle
|
||||
|
||||
@@ -2,19 +2,13 @@
|
||||
|
||||
This module is a thin façade. The real work lives in four siblings:
|
||||
|
||||
- resolve_plan.py — Docker-specific resolution into a DockerBottlePlan
|
||||
- launch.py — bring-up + teardown context manager
|
||||
- cleanup.py — orphan enumeration + removal
|
||||
- enumerate.py — active-agent listing
|
||||
- prepare.py — host-side resolution into a DockerBottlePlan
|
||||
- launch.py — bring-up + teardown context manager
|
||||
- cleanup.py — orphan enumeration + removal
|
||||
- enumerate.py — active-agent listing
|
||||
|
||||
The base class's `prepare` template runs cross-backend host-side
|
||||
validation before calling `_resolve_plan` here.
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
the declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||
Docker backend only owns the steps that are about backend
|
||||
infrastructure: CA install and git copy-in.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -24,23 +18,24 @@ from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from . import prepare as _prepare
|
||||
from .bottle import DockerBottle
|
||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .provision import ca as _ca
|
||||
from .provision import git as _git
|
||||
from .provision import prompt as _prompt
|
||||
from .provision import skills as _skills
|
||||
from .provision import supervise as _supervise_prov
|
||||
|
||||
|
||||
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
||||
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
|
||||
when set to `docker`; retained as a legacy/example backend."""
|
||||
(default)."""
|
||||
|
||||
name = "docker"
|
||||
|
||||
@@ -53,47 +48,28 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
||||
launch."""
|
||||
return shutil.which("docker") is not None
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return _resolve_plan.build_guest_env(resolved_env)
|
||||
|
||||
def _resolve_plan(
|
||||
self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
) -> DockerBottlePlan:
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> DockerBottlePlan:
|
||||
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
|
||||
|
||||
@contextmanager
|
||||
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
|
||||
"""Docker bottles reach the supervise sidecar via the
|
||||
compose-network alias `supervise:9100`. No per-bottle URL
|
||||
plumbing needed; the alias resolves inside the bridge."""
|
||||
if plan.supervise_plan is None:
|
||||
return ""
|
||||
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
||||
def provision_ca(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_ca.provision_ca(plan, target)
|
||||
|
||||
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
|
||||
return _prompt.provision_prompt(plan, target)
|
||||
|
||||
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_skills.provision_skills(plan, target)
|
||||
|
||||
def provision_git(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_git.provision_git(plan, target)
|
||||
|
||||
def provision_supervise(self, plan: DockerBottlePlan, target: str) -> None:
|
||||
_supervise_prov.provision_supervise(plan, target)
|
||||
|
||||
def prepare_cleanup(self) -> DockerBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
@@ -5,11 +5,8 @@ from __future__ import annotations
|
||||
import subprocess
|
||||
from typing import Callable
|
||||
|
||||
from typing import cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
|
||||
|
||||
class DockerBottle(Bottle):
|
||||
@@ -23,20 +20,15 @@ class DockerBottle(Bottle):
|
||||
*,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
):
|
||||
self.name = container
|
||||
self._teardown = teardown
|
||||
self.prompt_path = prompt_path_in_container
|
||||
self._prompt_path = prompt_path_in_container
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
self.agent_provider_template = (
|
||||
"codex" if agent_command == "codex" else "claude"
|
||||
)
|
||||
self._closed = False
|
||||
|
||||
def agent_argv(
|
||||
@@ -44,22 +36,18 @@ class DockerBottle(Bottle):
|
||||
) -> list[str]:
|
||||
full_argv = list(argv)
|
||||
full_argv.extend(
|
||||
prompt_args(cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=full_argv)
|
||||
prompt_args(self._agent_prompt_mode, self._prompt_path, argv=full_argv)
|
||||
)
|
||||
cmd = ["docker", "exec"]
|
||||
if tty:
|
||||
cmd.append("-it")
|
||||
if self.agent_workdir and self.agent_workdir != "/home/node":
|
||||
cmd.extend(["-w", self.agent_workdir])
|
||||
cmd.extend([self.name, self.agent_command, *full_argv])
|
||||
return cmd
|
||||
|
||||
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
return subprocess.run(["sh", "-lc", script], check=False).returncode
|
||||
return subprocess.run(
|
||||
self.agent_argv(argv, tty=tty), check=False,
|
||||
).returncode
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
# Pipe via stdin to `sh -s` so the caller never has to worry
|
||||
|
||||
@@ -2,60 +2,104 @@
|
||||
|
||||
Carries the Docker-specific resolved fields produced by
|
||||
DockerBottleBackend.prepare. The launch step consumes it without
|
||||
further resolution; preflight rendering is inherited from BottlePlan.
|
||||
further resolution; show_plan-style rendering is the `print` method.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from ...egress import EgressPlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...log import info
|
||||
from ...pipelock import PipelockProxyPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import BottlePlan
|
||||
from ..print_util import print_multi, visible_agent_env_names
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DockerBottlePlan(BottlePlan):
|
||||
"""Docker-specific resolved fields produced by
|
||||
DockerBottleBackend.prepare. Inherits `spec`, `stage_dir`,
|
||||
`git_gate_plan`, `egress_plan`, `supervise_plan`, and
|
||||
`agent_provision` from BottlePlan."""
|
||||
DockerBottleBackend.prepare. Inherits `spec` and `stage_dir` from
|
||||
BottlePlan."""
|
||||
|
||||
slug: str
|
||||
container_name: str
|
||||
container_name_pinned: bool
|
||||
image: str
|
||||
derived_image: str # "" -> no derived image
|
||||
runtime_image: str # image actually launched (derived or base)
|
||||
# Absolute path to the Dockerfile that builds `image`. Empty means
|
||||
# use the repo's default Dockerfile. Populated to a per-bottle
|
||||
# state file (~/.bot-bottle/state/<slug>/Dockerfile) after a
|
||||
# capability-block remediation (PRD 0016).
|
||||
dockerfile_path: str
|
||||
env_file: Path # docker --env-file: NAME=VALUE literals
|
||||
# name -> value for vars forwarded into the docker-run child process
|
||||
# via subprocess env (so values never land on argv or in a file).
|
||||
# repr=False keeps secret/interpolated/OAuth values out of any
|
||||
# accidental log of the plan dataclass.
|
||||
forwarded_env: dict[str, str] = field(repr=False)
|
||||
prompt_file: Path
|
||||
proxy_plan: PipelockProxyPlan
|
||||
git_gate_plan: GitGatePlan
|
||||
egress_plan: EgressPlan
|
||||
# None when bottle.supervise is False. PRD 0013 supervise sidecar
|
||||
# is opt-in via the manifest's bottle.supervise field.
|
||||
supervise_plan: SupervisePlan | None
|
||||
use_runsc: bool
|
||||
agent_command: str = "claude"
|
||||
agent_prompt_mode: PromptMode = "append_file"
|
||||
agent_provider_template: str = "claude"
|
||||
|
||||
@property
|
||||
def container_name(self) -> str:
|
||||
return self.agent_provision.instance_name
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Render the y/N preflight summary to stderr — compact form
|
||||
intended to fit on screen without scrolling. The full
|
||||
structured shape (image, container, runtime, etc.) lives on
|
||||
this dataclass for tooling that wants to introspect it."""
|
||||
del remote_control # not surfaced in the compact summary
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
# The agent sees the union of literal env names (rendered into
|
||||
# --env-file) and forwarded env names (`-e NAME` with the
|
||||
# value arriving via subprocess env). The forwarded set holds
|
||||
# the OAuth token (CLAUDE_CODE_OAUTH_TOKEN) and any host-env
|
||||
# interpolations from the manifest; egress holds
|
||||
# upstream tokens in its own environ, so no token forwarding
|
||||
# from the agent to the proxy is needed.
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(set(bottle.env.keys()) | set(self.forwarded_env.keys())),
|
||||
agent_provider_template=self.agent_provider_template,
|
||||
)
|
||||
|
||||
@property
|
||||
def image(self) -> str:
|
||||
return self.agent_provision.image
|
||||
print(file=sys.stderr)
|
||||
info(f"agent : {spec.agent_name}")
|
||||
info(f"provider : {self.agent_provider_template}")
|
||||
print_multi("env ", env_names)
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
|
||||
@property
|
||||
def dockerfile_path(self) -> str:
|
||||
"""Absolute path to the Dockerfile that builds `image`. Sourced
|
||||
from the agent provision plan — the manifest may override per
|
||||
bottle; otherwise the provider plugin's bundled Dockerfile."""
|
||||
return self.agent_provision.dockerfile
|
||||
identity = manifest.git_identity_summary(spec.agent_name)
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
return self.agent_provision.prompt_file
|
||||
git_lines = [
|
||||
f"{u.upstream_host}:{u.upstream_port}"
|
||||
for u in self.git_gate_plan.upstreams
|
||||
]
|
||||
if git_lines:
|
||||
print_multi(" git gate ", git_lines)
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
if self.egress_plan.routes:
|
||||
egress_lines = []
|
||||
for r in self.egress_plan.routes:
|
||||
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
||||
egress_lines.append(f"{r.host}{auth}")
|
||||
print_multi(" egress ", egress_lines)
|
||||
print(file=sys.stderr)
|
||||
|
||||
@@ -35,9 +35,9 @@ import secrets
|
||||
import string
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from . import supervise as _supervise
|
||||
from ... import supervise as _supervise
|
||||
from . import util as docker_mod
|
||||
|
||||
|
||||
# Directory layout: ~/.bot-bottle/state/<identity>/...
|
||||
@@ -48,6 +48,7 @@ _TRANSCRIPT_SUBDIR = "transcript"
|
||||
# live here so chunk 3's `docker compose up` can find them at stable
|
||||
# paths. Each sidecar's `prepare()` writes config + CAs into its own
|
||||
# subdir; the launch step is unchanged today (still `docker cp`).
|
||||
_PIPELOCK_SUBDIR = "pipelock"
|
||||
_EGRESS_SUBDIR = "egress"
|
||||
_GIT_GATE_SUBDIR = "git-gate"
|
||||
_SUPERVISE_SUBDIR = "supervise"
|
||||
@@ -55,8 +56,8 @@ _AGENT_SUBDIR = "agent"
|
||||
_METADATA_NAME = "metadata.json"
|
||||
# Live-config dir bind-mounted into the supervise sidecar (read-only).
|
||||
# Host's apply paths keep these files fresh so supervise's
|
||||
# `list-egress-routes` MCP tool returns the current state —
|
||||
# not a snapshot from launch time.
|
||||
# `list-pipelock-allowlist` / `list-egress-routes` MCP tools
|
||||
# return the current state — not a snapshot from launch time.
|
||||
_LIVE_CONFIG_SUBDIR = "live-config"
|
||||
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
|
||||
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
|
||||
@@ -81,7 +82,6 @@ def bottle_identity(agent_name: str) -> str:
|
||||
To continue an existing bottle's state, use the recorded
|
||||
identity from BottleMetadata via `cli.py resume <identity>`,
|
||||
not this function."""
|
||||
from .backend.docker import util as docker_mod
|
||||
slug = docker_mod.slugify(agent_name)
|
||||
suffix = "".join(secrets.choice(_SUFFIX_ALPHABET) for _ in range(_RANDOM_SUFFIX_LEN))
|
||||
return f"{slug}-{suffix}"
|
||||
@@ -105,12 +105,6 @@ class BottleMetadata:
|
||||
# written before chunk 3 (resume / inspect should fall back to
|
||||
# deriving from identity in that case).
|
||||
compose_project: str = ""
|
||||
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
|
||||
# for state dirs written before PRD 0040; callers default to "docker"
|
||||
# for backward compatibility.
|
||||
backend: str = ""
|
||||
label: str = ""
|
||||
color: str = ""
|
||||
|
||||
|
||||
def metadata_path(identity: str) -> Path:
|
||||
@@ -137,17 +131,13 @@ def read_metadata(identity: str) -> BottleMetadata | None:
|
||||
raw = json.loads(path.read_text())
|
||||
if not isinstance(raw, dict):
|
||||
return None
|
||||
raw_typed = cast(dict[str, object], raw)
|
||||
return BottleMetadata(
|
||||
identity=str(raw_typed.get("identity", identity)),
|
||||
agent_name=str(raw_typed.get("agent_name", "")),
|
||||
cwd=str(raw_typed.get("cwd", "")),
|
||||
copy_cwd=bool(raw_typed.get("copy_cwd", False)),
|
||||
started_at=str(raw_typed.get("started_at", "")),
|
||||
compose_project=str(raw_typed.get("compose_project", "")),
|
||||
backend=str(raw_typed.get("backend", "")),
|
||||
label=str(raw_typed.get("label", "")),
|
||||
color=str(raw_typed.get("color", "")),
|
||||
identity=str(raw.get("identity", identity)),
|
||||
agent_name=str(raw.get("agent_name", "")),
|
||||
cwd=str(raw.get("cwd", "")),
|
||||
copy_cwd=bool(raw.get("copy_cwd", False)),
|
||||
started_at=str(raw.get("started_at", "")),
|
||||
compose_project=str(raw.get("compose_project", "")),
|
||||
)
|
||||
|
||||
|
||||
@@ -237,6 +227,12 @@ def transcript_snapshot_dir(identity: str) -> Path:
|
||||
# nothing requested preservation.
|
||||
|
||||
|
||||
def pipelock_state_dir(identity: str) -> Path:
|
||||
"""State subdir for the pipelock sidecar: pipelock.yaml + the
|
||||
per-bottle CA cert/key. Bind-mount source from chunk 3 onward."""
|
||||
return bottle_state_dir(identity) / _PIPELOCK_SUBDIR
|
||||
|
||||
|
||||
def egress_state_dir(identity: str) -> Path:
|
||||
"""State subdir for the egress sidecar: routes.yaml + the
|
||||
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
|
||||
@@ -322,6 +318,7 @@ __all__ = [
|
||||
"per_bottle_dockerfile",
|
||||
"per_bottle_dockerfile_path",
|
||||
"per_bottle_image_tag",
|
||||
"pipelock_state_dir",
|
||||
"preserve_marker_path",
|
||||
"read_metadata",
|
||||
"supervise_state_dir",
|
||||
@@ -30,14 +30,16 @@ semantics open question.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import get_provider
|
||||
from ...log import info, warn
|
||||
from ...bottle_state import (
|
||||
from .bottle_state import (
|
||||
mark_preserved,
|
||||
per_bottle_dockerfile,
|
||||
per_bottle_dockerfile_path,
|
||||
transcript_snapshot_dir,
|
||||
write_per_bottle_dockerfile,
|
||||
)
|
||||
@@ -93,11 +95,11 @@ def fetch_current_dockerfile(slug: str) -> str:
|
||||
override = per_bottle_dockerfile(slug)
|
||||
if override is not None:
|
||||
return override
|
||||
repo_dockerfile = get_provider("claude").dockerfile
|
||||
repo_dockerfile = _repo_dockerfile_path()
|
||||
if repo_dockerfile.is_file():
|
||||
return repo_dockerfile.read_text()
|
||||
raise CapabilityApplyError(
|
||||
f"no per-bottle Dockerfile for {slug} and no provider Dockerfile at "
|
||||
f"no per-bottle Dockerfile for {slug} and no repo Dockerfile at "
|
||||
f"{repo_dockerfile}"
|
||||
)
|
||||
|
||||
@@ -125,6 +127,13 @@ def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
|
||||
# --- Internals -------------------------------------------------------------
|
||||
|
||||
|
||||
def _repo_dockerfile_path() -> Path:
|
||||
"""Path to the repo's Claude Dockerfile (one dir above this module's
|
||||
package root). Resolved at call time so the path is correct
|
||||
regardless of where this module is imported from."""
|
||||
# bot_bottle/backend/docker/capability_apply.py -> repo root
|
||||
return Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
|
||||
|
||||
|
||||
def snapshot_transcript(slug: str) -> None:
|
||||
"""`docker cp` /home/node/.claude out of the agent container into
|
||||
|
||||
@@ -31,7 +31,7 @@ from ... import supervise as _supervise
|
||||
from ...log import info, warn
|
||||
from . import util as docker_mod
|
||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from ...bottle_state import bottle_state_dir, is_preserved
|
||||
from .bottle_state import bottle_state_dir, is_preserved
|
||||
from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
|
||||
|
||||
|
||||
|
||||
@@ -7,14 +7,34 @@ two networks, no named volumes.
|
||||
|
||||
Pure function. No I/O, no subprocess. Expects every launch-time
|
||||
field (network names, CA host paths, etc.) on the plan's inner
|
||||
plans to be populated; chunks 2+3 own that ordering.
|
||||
plans to be populated; chunks 2+3 own that ordering. Chunk 1 just
|
||||
encodes the translation so it can be unit-tested in isolation.
|
||||
|
||||
Conditional services follow the plan content:
|
||||
Conditional services follow the plan content (matches the
|
||||
SDK-call branching in `launch.py` today):
|
||||
|
||||
- agent + sidecars bundle: always.
|
||||
- git-gate: iff plan.git_gate_plan.upstreams.
|
||||
- egress: iff plan.egress_plan.routes.
|
||||
- supervise: iff plan.supervise_plan is not None.
|
||||
- pipelock + agent: always.
|
||||
- git-gate: iff plan.git_gate_plan.upstreams.
|
||||
- egress: iff plan.egress_plan.routes.
|
||||
- supervise: iff plan.supervise_plan is not None.
|
||||
|
||||
Naming:
|
||||
|
||||
- Compose project: `bot-bottle-<slug>`.
|
||||
- Service names (inside the file): `agent`, `pipelock`,
|
||||
`egress`, `git-gate`, `supervise`.
|
||||
- `container_name:` matches today's pattern
|
||||
(`bot-bottle-<service>-<slug>`) so dashboard/cleanup discovery
|
||||
via the prefix scan keeps working through the transition.
|
||||
- Network aliases preserve the current dial-by-shortname pattern
|
||||
for `egress` / `supervise`, and add the long container-name as
|
||||
an internal-network alias for `pipelock` / `git-gate` so any
|
||||
caller still referencing the long name resolves.
|
||||
|
||||
Sidecars that are built (egress, git-gate, supervise) get a
|
||||
compose `build:` block pointing at the repo Dockerfile; the
|
||||
`image:` tag is set explicitly so cached images on the daemon
|
||||
aren't rebuilt on every up.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -29,8 +49,9 @@ from ...egress import (
|
||||
EGRESS_HOSTNAME,
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
)
|
||||
from ...git_gate import GIT_GATE_HOSTNAME
|
||||
from ...git_gate import GIT_GATE_HOSTNAME, git_gate_aggregate_extra_hosts
|
||||
from ...log import die, warn
|
||||
from ...pipelock import PIPELOCK_HOSTNAME
|
||||
from ...supervise import (
|
||||
CURRENT_CONFIG_DIR_IN_AGENT,
|
||||
QUEUE_DIR_IN_CONTAINER,
|
||||
@@ -42,7 +63,7 @@ from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .egress import (
|
||||
EGRESS_CA_IN_CONTAINER,
|
||||
EGRESS_PORT,
|
||||
EGRESS_PIPELOCK_CA_IN_CONTAINER,
|
||||
)
|
||||
from .git_gate import (
|
||||
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
||||
@@ -50,7 +71,11 @@ from .git_gate import (
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from . import network as network_mod
|
||||
from .pipelock import (
|
||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||
PIPELOCK_PORT,
|
||||
)
|
||||
from .sidecar_bundle import (
|
||||
SIDECAR_BUNDLE_DOCKERFILE,
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
@@ -66,11 +91,12 @@ def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"""Render a Compose v2 spec dict from a fully-resolved
|
||||
DockerBottlePlan.
|
||||
|
||||
The plan must have its inner plans (`git_gate_plan`,
|
||||
`egress_plan`, `supervise_plan`) populated with launch-time
|
||||
fields — network names, CA host paths. The renderer doesn't
|
||||
validate; callers feed it a fully-resolved plan or get an
|
||||
incomplete compose spec back.
|
||||
The plan must have its inner plans (`proxy_plan`,
|
||||
`git_gate_plan`, `egress_plan`, `supervise_plan`) populated
|
||||
with launch-time fields — network names, CA host paths,
|
||||
pipelock_proxy_url. The renderer doesn't validate; callers
|
||||
feed it a fully-resolved plan or get an incomplete compose
|
||||
spec back.
|
||||
"""
|
||||
project = f"bot-bottle-{plan.slug}"
|
||||
services: dict[str, Any] = {
|
||||
@@ -92,11 +118,11 @@ def _networks(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
bridge."""
|
||||
return {
|
||||
"internal": {
|
||||
"name": network_mod.network_name_for_slug(plan.slug),
|
||||
"name": plan.proxy_plan.internal_network,
|
||||
"internal": True,
|
||||
},
|
||||
"egress": {
|
||||
"name": network_mod.network_egress_name_for_slug(plan.slug),
|
||||
"name": plan.proxy_plan.egress_network,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -116,12 +142,29 @@ def _bind(host: str | Path, target: str, *, read_only: bool = True) -> dict[str,
|
||||
|
||||
def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"""The `sidecars` service: one container per bottle, bundle
|
||||
image, all daemons under a Python init supervisor.
|
||||
image, all four daemons under a Python init supervisor.
|
||||
|
||||
Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS` env.
|
||||
egress is always present; git-gate / supervise are conditional.
|
||||
Mechanics:
|
||||
|
||||
- Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS`
|
||||
env. pipelock is always present; egress / git-gate /
|
||||
supervise are conditional on the plan.
|
||||
- Volumes are the union of the four daemons' bind-mounts,
|
||||
preserving the same in-container paths so each daemon
|
||||
finds its config / hooks / CA where it expects.
|
||||
- Environment is the union of *daemon-private* env vars
|
||||
(EGRESS_UPSTREAM_PROXY, SUPERVISE_BOTTLE_SLUG, etc).
|
||||
HTTPS_PROXY is NOT propagated here — see the comment in
|
||||
egress_entrypoint.sh; setting it at the container level
|
||||
would route git-gate's git fetches through pipelock,
|
||||
which is wrong.
|
||||
- Network aliases register every legacy short/long
|
||||
hostname (pipelock, egress, git-gate, supervise plus
|
||||
their `bot-bottle-<service>-<slug>` long forms) so
|
||||
the agent's HTTPS_PROXY URL and any other inter-service
|
||||
reference resolves to the bundle.
|
||||
"""
|
||||
daemons: list[str] = ["egress"]
|
||||
daemons: list[str] = ["egress", "pipelock"]
|
||||
if plan.git_gate_plan.upstreams:
|
||||
daemons.append("git-gate")
|
||||
if plan.supervise_plan is not None:
|
||||
@@ -130,15 +173,32 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
env: list[str] = [f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(daemons)}"]
|
||||
volumes: list[dict[str, Any]] = []
|
||||
|
||||
# --- egress -------------------------------------------------------
|
||||
# --- pipelock ----------------------------------------------------
|
||||
pp = plan.proxy_plan
|
||||
volumes += [
|
||||
_bind(pp.yaml_path, "/etc/pipelock.yaml"),
|
||||
_bind(pp.ca_cert_host_path, PIPELOCK_CA_CERT_IN_CONTAINER),
|
||||
_bind(pp.ca_key_host_path, PIPELOCK_CA_KEY_IN_CONTAINER),
|
||||
]
|
||||
|
||||
# --- egress (always part of the bundle; the EGRESS_UPSTREAM_*
|
||||
# env vars + ca bind-mounts are needed iff routes exist; when
|
||||
# the bottle has no routes the egress daemon falls back to its
|
||||
# `regular@9099` mode and is unused) -----------------------------
|
||||
ep = plan.egress_plan
|
||||
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
|
||||
if ep.routes:
|
||||
volumes.append(_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER))
|
||||
env.append(f"EGRESS_UPSTREAM_PROXY={ep.pipelock_proxy_url}")
|
||||
env.append(f"EGRESS_UPSTREAM_CA={EGRESS_PIPELOCK_CA_IN_CONTAINER}")
|
||||
volumes += [
|
||||
_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER),
|
||||
_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER),
|
||||
_bind(ep.pipelock_ca_host_path, EGRESS_PIPELOCK_CA_IN_CONTAINER),
|
||||
]
|
||||
for token_env in sorted(ep.token_env_map.keys()):
|
||||
env.append(token_env)
|
||||
|
||||
# --- git-gate -----------------------------------------------------
|
||||
# --- git-gate ----------------------------------------------------
|
||||
extra_hosts: list[str] = []
|
||||
gp = plan.git_gate_plan
|
||||
if gp.upstreams:
|
||||
volumes += [
|
||||
@@ -157,8 +217,10 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
u.known_hosts_file,
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
||||
))
|
||||
extra_map = git_gate_aggregate_extra_hosts(gp.upstreams)
|
||||
extra_hosts = [f"{host}:{ip}" for host, ip in sorted(extra_map.items())]
|
||||
|
||||
# --- supervise ----------------------------------------------------
|
||||
# --- supervise ---------------------------------------------------
|
||||
sp = plan.supervise_plan
|
||||
if sp is not None:
|
||||
env += [
|
||||
@@ -173,7 +235,13 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"read_only": False,
|
||||
})
|
||||
|
||||
internal_aliases = [EGRESS_HOSTNAME]
|
||||
# Internal-network aliases: the agent reaches each daemon through
|
||||
# its short name (pipelock / egress / git-gate / supervise) which
|
||||
# the bundle answers as if it were the daemon itself.
|
||||
internal_aliases = [
|
||||
PIPELOCK_HOSTNAME,
|
||||
EGRESS_HOSTNAME,
|
||||
]
|
||||
if gp.upstreams:
|
||||
internal_aliases.append(GIT_GATE_HOSTNAME)
|
||||
if sp is not None:
|
||||
@@ -193,13 +261,18 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"environment": env,
|
||||
"volumes": volumes,
|
||||
}
|
||||
if extra_hosts:
|
||||
service["extra_hosts"] = extra_hosts
|
||||
return service
|
||||
|
||||
|
||||
def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"""Agent container. Runs `sleep infinity`; claude is `docker
|
||||
exec -it`'d into it later. HTTP_PROXY/HTTPS_PROXY point at the
|
||||
egress sidecar."""
|
||||
exec -it`'d into it later. No TTY at the container level —
|
||||
interactivity is per-exec. HTTP_PROXY/HTTPS_PROXY point at the
|
||||
egress short-alias when an egress is declared, otherwise
|
||||
straight at pipelock's container name. CA trust trio matches
|
||||
the existing launch.py wiring."""
|
||||
proxy_url = _agent_proxy_url(plan)
|
||||
no_proxy = _agent_no_proxy(plan)
|
||||
env: list[str] = [
|
||||
@@ -213,8 +286,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
|
||||
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
|
||||
]
|
||||
for name, value in sorted(plan.agent_provision.guest_env.items()):
|
||||
env.append(f"{name}={value}")
|
||||
# Forwarded vars (OAuth token, manifest host-interpolations):
|
||||
# bare name → inherits from compose-up process env, value
|
||||
# never lands on argv or in the compose file.
|
||||
@@ -222,7 +293,7 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
env.append(name)
|
||||
|
||||
service: dict[str, Any] = {
|
||||
"image": plan.image,
|
||||
"image": plan.runtime_image,
|
||||
"container_name": plan.container_name,
|
||||
"command": ["sleep", "infinity"],
|
||||
"networks": {"internal": None},
|
||||
@@ -230,6 +301,8 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
}
|
||||
if plan.use_runsc:
|
||||
service["runtime"] = "runsc"
|
||||
if plan.env_file and plan.env_file.exists() and plan.env_file.stat().st_size > 0:
|
||||
service["env_file"] = [str(plan.env_file)]
|
||||
|
||||
volumes: list[dict[str, Any]] = []
|
||||
if plan.supervise_plan is not None:
|
||||
@@ -249,14 +322,21 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
|
||||
|
||||
def _agent_proxy_url(plan: DockerBottlePlan) -> str:
|
||||
"""Agent's HTTP_PROXY — always points at egress."""
|
||||
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
|
||||
"""Pick the agent's HTTP_PROXY. With egress declared, the agent
|
||||
goes through egress (which in turn HTTPS_PROXYs to pipelock on
|
||||
its outbound leg). Without egress, the agent talks straight to
|
||||
pipelock."""
|
||||
if plan.egress_plan.routes:
|
||||
from .egress import EGRESS_PORT
|
||||
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
|
||||
return f"http://{PIPELOCK_HOSTNAME}:{PIPELOCK_PORT}"
|
||||
|
||||
|
||||
def _agent_no_proxy(plan: DockerBottlePlan) -> str:
|
||||
"""NO_PROXY for the agent: loopback always; supervise hostname
|
||||
when the supervise sidecar is up (MCP long-poll must bypass
|
||||
the egress proxy)."""
|
||||
"""NO_PROXY for the agent. Matches the launch.py rules:
|
||||
loopback always, supervise hostname when the supervise sidecar
|
||||
is up (the MCP long-poll pattern needs to bypass pipelock's
|
||||
idle timeout)."""
|
||||
hosts = ["localhost", "127.0.0.1"]
|
||||
if plan.supervise_plan is not None:
|
||||
hosts.append(SUPERVISE_HOSTNAME)
|
||||
|
||||
@@ -22,8 +22,14 @@ from ...log import die
|
||||
EGRESS_PORT = int(os.environ.get("BOT_BOTTLE_EGRESS_PORT", "9099"))
|
||||
|
||||
# In-container path for mitmproxy's CA. The format is a single PEM
|
||||
# file holding BOTH the cert and the private key, concatenated.
|
||||
# file holding BOTH the cert and the private key, concatenated. The
|
||||
# upstream-trust CA (pipelock's, so egress trusts the upstream
|
||||
# leg) is a separate file because pipelock keeps a different CA on
|
||||
# its end.
|
||||
EGRESS_CA_IN_CONTAINER = "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem"
|
||||
EGRESS_PIPELOCK_CA_IN_CONTAINER = (
|
||||
"/home/mitmproxy/.mitmproxy/pipelock-ca.pem"
|
||||
)
|
||||
|
||||
|
||||
def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
||||
@@ -36,8 +42,16 @@ def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
||||
trust store by `provision_ca` so the agent trusts the bumped
|
||||
CONNECT cert egress presents.
|
||||
|
||||
openssl req's `subjectKeyIdentifier=hash` extension uses
|
||||
SHA-1(pubkey), matching mitmproxy's AKI computation on leaves.
|
||||
Why openssl req (not the pipelock binary's `tls init`):
|
||||
pipelock's CA generator stamps a non-standard `Subject Key
|
||||
Identifier` on the CA (random rather than SHA-1 of the pubkey).
|
||||
mitmproxy computes the `Authority Key Identifier` on each leaf
|
||||
it mints as SHA-1(issuer's pubkey). openssl's chain validator
|
||||
uses the leaf's AKI to find the issuer cert by SKI; pipelock's
|
||||
SKI doesn't match → openssl reports "unable to get local issuer
|
||||
certificate" even though the CA is right there in the trust
|
||||
store. openssl req's `subjectKeyIdentifier=hash` extension uses
|
||||
SHA-1(pubkey), matching mitmproxy's computation.
|
||||
|
||||
Both files live under `<stage_dir>/egress-ca/` (mode 644 —
|
||||
`docker cp` preserves the mode into the container, where the
|
||||
|
||||
@@ -1,25 +1,86 @@
|
||||
"""Host-side helper for egress sidecar inspection (issue #198).
|
||||
"""Host-side helper to apply a routes.yaml change to a running
|
||||
egress sidecar (PRD 0014 retargeted by PRD 0017 chunk 3).
|
||||
|
||||
`_merge_single_route`, `add_route`, and `apply_routes_change` were
|
||||
removed when the egress-block MCP tool was dropped. The remaining
|
||||
helpers support runtime inspection and validation of the routes file
|
||||
without modifying it at runtime.
|
||||
Used by the supervise dashboard when the operator approves an
|
||||
egress-block proposal (or runs the operator-initiated
|
||||
`routes edit <bottle>` verb). Fetches the current routes.yaml via
|
||||
`docker exec cat`, validates the new content, writes it into the
|
||||
sidecar via `docker cp`, then `docker kill --signal HUP` to make
|
||||
the addon reload without dropping connections.
|
||||
|
||||
Also mirrors the new route hosts into pipelock's hostname allowlist
|
||||
so the downstream leg lets them through — egress enforces
|
||||
the path-aware allowlist on the agent leg, pipelock enforces the
|
||||
hostname allowlist + DLP body scan on the upstream leg, and a
|
||||
host added to one must be in the other or the request 403s
|
||||
somewhere along the chain.
|
||||
|
||||
Raises EgressApplyError on any failure — the dashboard
|
||||
surfaces the message and keeps the proposal pending so the
|
||||
operator can retry.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER
|
||||
from ...egress_addon_core import load_routes
|
||||
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
|
||||
from .bottle_state import egress_state_dir
|
||||
from .sidecar_bundle import sidecar_bundle_container_name
|
||||
from .pipelock_apply import (
|
||||
PipelockApplyError,
|
||||
apply_allowlist_change,
|
||||
fetch_current_allowlist,
|
||||
parse_allowlist_content,
|
||||
render_allowlist_content,
|
||||
)
|
||||
|
||||
|
||||
def _render_routes_payload(routes_list: list[dict[str, object]]) -> str:
|
||||
"""Render a list-of-dicts routes payload as YAML matching the
|
||||
shape `egress_render_routes` produces. The apply path
|
||||
round-trips current routes.yaml through this so the file the
|
||||
sidecar sees stays in the YAML format the addon expects."""
|
||||
if not routes_list:
|
||||
return "routes: []\n"
|
||||
lines: list[str] = ["routes:"]
|
||||
for entry in routes_list:
|
||||
host = str(entry.get("host", ""))
|
||||
lines.append(f' - host: "{host}"')
|
||||
auth_scheme = entry.get("auth_scheme")
|
||||
token_env = entry.get("token_env")
|
||||
if auth_scheme and token_env:
|
||||
lines.append(f' auth_scheme: "{auth_scheme}"')
|
||||
lines.append(f' token_env: "{token_env}"')
|
||||
paths = entry.get("path_allowlist") or []
|
||||
if paths:
|
||||
lines.append(" path_allowlist:")
|
||||
for p in paths:
|
||||
lines.append(f' - "{p}"')
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _egress_routes_host_path(slug: str) -> Path:
|
||||
"""The bind-mount source for the egress sidecar's routes.yaml.
|
||||
Must match what egress.prepare wrote at chunk-2 paths."""
|
||||
return egress_state_dir(slug) / "egress_routes.yaml"
|
||||
|
||||
|
||||
class EgressApplyError(RuntimeError):
|
||||
pass
|
||||
"""Raised when fetch / apply fails. Caller renders to the
|
||||
operator; does not crash the dashboard."""
|
||||
|
||||
|
||||
def fetch_current_routes(slug: str) -> str:
|
||||
"""Read the live routes.yaml from the running egress sidecar
|
||||
for `slug`. Returns the file content as a string. Raises
|
||||
EgressApplyError if the sidecar isn't reachable or the read
|
||||
fails."""
|
||||
container = sidecar_bundle_container_name(slug)
|
||||
r = subprocess.run(
|
||||
["docker", "exec", container, "cat", EGRESS_ROUTES_IN_CONTAINER],
|
||||
@@ -34,6 +95,9 @@ def fetch_current_routes(slug: str) -> str:
|
||||
|
||||
|
||||
def validate_routes_content(content: str) -> None:
|
||||
"""Syntactic check before SIGHUP — the addon's reload also
|
||||
validates, but failing here keeps the old routes live and gives
|
||||
the operator a clearer error than the addon's stderr line."""
|
||||
try:
|
||||
load_routes(content)
|
||||
except ValueError as e:
|
||||
@@ -42,8 +106,238 @@ def validate_routes_content(content: str) -> None:
|
||||
) from e
|
||||
|
||||
|
||||
def _hosts_in_routes(content: str) -> list[str]:
|
||||
"""Extract the host list from a routes.yaml content string.
|
||||
Uses the addon's own parser so any host the addon will match on
|
||||
also lands in pipelock's allowlist. Returns sorted+deduped."""
|
||||
try:
|
||||
routes = load_routes(content)
|
||||
except ValueError as e:
|
||||
raise EgressApplyError(
|
||||
f"proposed routes.yaml is not valid: {e}"
|
||||
) from e
|
||||
return sorted({r.host for r in routes if r.host})
|
||||
|
||||
|
||||
# Pipelock's allowlist parser accepts only literal hostnames:
|
||||
# `[A-Za-z0-9_.-]+`. Anything else (wildcards, IPv6 literals,
|
||||
# stray characters) is silently dropped from the mirror so the
|
||||
# pipelock apply doesn't fail parse before the new yaml is even
|
||||
# written. The dropped hosts stay on egress's route table —
|
||||
# but the addon does exact-host match only, so they'll never
|
||||
# match anything either. (Wildcard host matching was removed —
|
||||
# see `match_route` in egress_addon_core for the rationale.)
|
||||
_PIPELOCK_HOST_RE = re.compile(r"^[A-Za-z0-9_.-]+$")
|
||||
|
||||
|
||||
def _pipelock_safe_hosts(hosts: list[str]) -> list[str]:
|
||||
"""Drop any host pipelock's allowlist parser would reject.
|
||||
Order preserved."""
|
||||
return [h for h in hosts if _PIPELOCK_HOST_RE.match(h)]
|
||||
|
||||
|
||||
def _mirror_hosts_to_pipelock(slug: str, hosts: list[str]) -> None:
|
||||
"""Ensure every pipelock-compatible `hosts` entry is on
|
||||
pipelock's allowlist. Fetches pipelock's current allowlist,
|
||||
merges, re-applies. Hosts pipelock can't represent (wildcards,
|
||||
etc.) are silently skipped — they stay live on egress
|
||||
but aren't enforced at pipelock. No-op if every host is already
|
||||
present (apply still restarts pipelock if any host is new).
|
||||
Raises EgressApplyError on pipelock failures so the
|
||||
caller's diff/audit reflects the half-state."""
|
||||
safe_hosts = _pipelock_safe_hosts(hosts)
|
||||
try:
|
||||
current = fetch_current_allowlist(slug)
|
||||
existing = parse_allowlist_content(current)
|
||||
merged = sorted(set(existing) | set(safe_hosts))
|
||||
if merged == sorted(existing):
|
||||
return # nothing to add
|
||||
apply_allowlist_change(slug, render_allowlist_content(merged))
|
||||
except PipelockApplyError as e:
|
||||
# Mirror runs BEFORE the egress write, so egress
|
||||
# is unchanged on this failure path. Report it as a
|
||||
# pipelock-side problem so the operator looks in the right
|
||||
# place; their `pipelock edit` flow can repair manually.
|
||||
raise EgressApplyError(
|
||||
f"pipelock allowlist mirror failed (egress NOT "
|
||||
f"updated): {e}. Fix pipelock's allowlist manually with "
|
||||
f"`pipelock edit <bottle>` then retry the proposal."
|
||||
) from e
|
||||
|
||||
|
||||
def apply_routes_change(slug: str, new_content: str) -> tuple[str, str]:
|
||||
"""Apply `new_content` to the egress sidecar for `slug`:
|
||||
1. Fetch current routes.yaml (for the before-diff).
|
||||
2. Validate the new content via the addon's own parser.
|
||||
3. Mirror the route hosts onto pipelock's allowlist (so the
|
||||
downstream hostname gate lets them through).
|
||||
4. Write to a temp file, `docker cp` into the egress
|
||||
sidecar.
|
||||
5. `docker kill --signal HUP` so the addon reloads.
|
||||
|
||||
Order matters: pipelock first, then egress. If the
|
||||
pipelock step fails, egress hasn't been touched and the
|
||||
old routes stay live. If the egress step fails after
|
||||
pipelock succeeded, pipelock has the host in its allowlist but
|
||||
egress doesn't enforce it yet — harmless extra-permissive
|
||||
state at pipelock, and a re-approval will land the egress
|
||||
side.
|
||||
|
||||
Returns (before, after) where `after` == `new_content`. Raises
|
||||
EgressApplyError on any step."""
|
||||
container = sidecar_bundle_container_name(slug)
|
||||
before = fetch_current_routes(slug)
|
||||
validate_routes_content(new_content)
|
||||
|
||||
# Pipelock mirror first — if it fails, egress stays intact
|
||||
# and the operator gets a clear error about the half-state.
|
||||
_mirror_hosts_to_pipelock(slug, _hosts_in_routes(new_content))
|
||||
|
||||
# routes.yaml is bind-mounted into the egress container as a
|
||||
# SINGLE FILE. Docker single-file bind mounts pin the source
|
||||
# inode at mount time; write-temp-then-rename swaps the inode
|
||||
# on the host, which leaves the container's mount pointing at
|
||||
# the now-orphaned old inode (so the SIGHUP'd reload re-reads
|
||||
# unchanged content). Write in-place instead. Lose file-level
|
||||
# atomicity, but the apply path issues SIGHUP only AFTER the
|
||||
# write returns, and the addon's `load_routes` raises
|
||||
# `ValueError` on a partial read and keeps the previous
|
||||
# in-memory routes — so a SIGHUP that hypothetically raced an
|
||||
# in-flight write is non-disruptive.
|
||||
target = _egress_routes_host_path(slug)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(new_content)
|
||||
# mitmproxy in the container reads through the bind mount as
|
||||
# uid 1000; the host file has to be world-readable for that
|
||||
# read to succeed (parent dir at 0o700 still restricts who
|
||||
# can reach the file on the host). Routes content is not
|
||||
# secret — tokens live in the container's environ — so 0o644
|
||||
# is the right trade-off.
|
||||
target.chmod(0o644)
|
||||
sig = subprocess.run(
|
||||
["docker", "kill", "--signal", "HUP", container],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if sig.returncode != 0:
|
||||
raise EgressApplyError(
|
||||
f"failed to SIGHUP {container}: "
|
||||
f"{(sig.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
return before, new_content
|
||||
|
||||
|
||||
def _merge_single_route(
|
||||
current_yaml: str, new_route: dict[str, object],
|
||||
) -> str:
|
||||
"""Merge a single proposed route into the current routes.yaml
|
||||
content, returning the merged YAML string.
|
||||
|
||||
Behavior:
|
||||
- If `new_route['host']` is NOT in the current routes →
|
||||
append the route.
|
||||
- If the host IS already present → union the path_allowlist
|
||||
entries (proposed ∪ existing). The existing `auth_scheme`
|
||||
and `token_env` are preserved — agent-proposed auth changes
|
||||
on an existing host are ignored, matching the tool's
|
||||
documented semantics.
|
||||
|
||||
Round-trips the file through `yaml_subset` (the same parser
|
||||
the addon uses), so the merged output is in the YAML format
|
||||
the sidecar reads. Token VALUES never appear here; the routes
|
||||
file carries only env-var slot NAMES."""
|
||||
try:
|
||||
cfg = parse_yaml_subset(current_yaml)
|
||||
except YamlSubsetError as e:
|
||||
raise EgressApplyError(
|
||||
f"current routes.yaml is not valid YAML: {e}"
|
||||
) from e
|
||||
routes = cfg.get("routes")
|
||||
if not isinstance(routes, list):
|
||||
raise EgressApplyError(
|
||||
"current routes.yaml: 'routes' is not a list"
|
||||
)
|
||||
|
||||
new_host = str(new_route.get("host", "")).lower()
|
||||
if not new_host:
|
||||
raise EgressApplyError(
|
||||
"proposed route is missing 'host'"
|
||||
)
|
||||
|
||||
proposed_paths = list(new_route.get("path_allowlist") or [])
|
||||
|
||||
# Look for an existing entry with the same host (case-insensitive).
|
||||
for entry in routes:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
if str(entry.get("host", "")).lower() == new_host:
|
||||
# Merge path_allowlist: union proposed + existing, ordered
|
||||
# by first-seen so existing paths stay in original order.
|
||||
existing_paths: list[str] = list(entry.get("path_allowlist") or [])
|
||||
seen = {p: None for p in existing_paths}
|
||||
for p in proposed_paths:
|
||||
seen.setdefault(p, None)
|
||||
merged_paths = list(seen.keys())
|
||||
if merged_paths:
|
||||
entry["path_allowlist"] = merged_paths
|
||||
# Preserve existing auth — tool description says agent-
|
||||
# proposed auth on an existing host is ignored.
|
||||
break
|
||||
else:
|
||||
# Host not present; build a new route entry from the
|
||||
# proposed fields. Need to assign a token_env slot if
|
||||
# `auth` was proposed (otherwise the addon's parser rejects
|
||||
# a half-set auth pair). Slots: count existing slots, pick
|
||||
# the next free index.
|
||||
entry = {"host": new_route["host"]}
|
||||
if proposed_paths:
|
||||
entry["path_allowlist"] = proposed_paths
|
||||
auth = new_route.get("auth")
|
||||
if isinstance(auth, dict) and auth.get("scheme") and auth.get("token_ref"):
|
||||
existing_slots = sorted({
|
||||
str(r.get("token_env"))
|
||||
for r in routes
|
||||
if isinstance(r, dict) and r.get("token_env")
|
||||
})
|
||||
next_idx = len(existing_slots)
|
||||
entry["auth_scheme"] = str(auth["scheme"])
|
||||
entry["token_env"] = f"EGRESS_TOKEN_{next_idx}"
|
||||
# NOTE: the addon reads token VALUES from its container's
|
||||
# environ keyed by token_env. A newly-added auth route at
|
||||
# runtime points at a slot that has no env value → the
|
||||
# addon will 403 with "token env unset" until the operator
|
||||
# arranges for the value to land in the container's env.
|
||||
# Recording this here so the operator-facing diff carries
|
||||
# the slot name they'll need to provision.
|
||||
routes.append(entry)
|
||||
|
||||
return _render_routes_payload(routes)
|
||||
|
||||
|
||||
def add_route(slug: str, proposed_route_json: str) -> tuple[str, str]:
|
||||
"""Apply a single-route addition to the egress. Parses the
|
||||
agent's proposed route, fetches the current routes file, merges,
|
||||
and applies via `apply_routes_change`. Returns (before, after)
|
||||
full-file content for the audit log."""
|
||||
try:
|
||||
proposed = json.loads(proposed_route_json)
|
||||
except json.JSONDecodeError as e:
|
||||
raise EgressApplyError(
|
||||
f"proposed route is not valid JSON: {e}"
|
||||
) from e
|
||||
if not isinstance(proposed, dict):
|
||||
raise EgressApplyError(
|
||||
"proposed route must be a JSON object"
|
||||
)
|
||||
current = fetch_current_routes(slug)
|
||||
merged = _merge_single_route(current, proposed)
|
||||
return apply_routes_change(slug, merged)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"EgressApplyError",
|
||||
"add_route",
|
||||
"apply_routes_change",
|
||||
"fetch_current_routes",
|
||||
"validate_routes_content",
|
||||
]
|
||||
|
||||
@@ -15,7 +15,7 @@ from __future__ import annotations
|
||||
import subprocess
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ...bottle_state import read_metadata
|
||||
from .bottle_state import read_metadata
|
||||
from .compose import compose_project_name, list_active_slugs
|
||||
|
||||
|
||||
@@ -39,8 +39,6 @@ def enumerate_active() -> list[ActiveAgent]:
|
||||
agent_name=metadata.agent_name if metadata else "?",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
services=tuple(sorted(services)),
|
||||
label=metadata.label if metadata else "",
|
||||
color=metadata.color if metadata else "",
|
||||
))
|
||||
return out
|
||||
|
||||
|
||||
@@ -4,19 +4,25 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
|
||||
|
||||
The flow is:
|
||||
|
||||
1. Build the agent image from the provider Dockerfile (compose
|
||||
builds the sidecar images via the `build:` directive on first up).
|
||||
2. Mint the per-bottle egress CA (chunk 2 writes it under
|
||||
state/<slug>/egress/).
|
||||
3. Populate the inner plans with launch-time fields so the
|
||||
renderer can read network names, CA paths.
|
||||
1. Build the agent's base + derived image (compose builds the
|
||||
sidecar images via the `build:` directive on first up).
|
||||
2. Pre-create the per-bottle networks. We do this outside compose
|
||||
so we can inspect the assigned internal CIDR and embed it in
|
||||
pipelock's yaml (compose's `external: true` lets the compose
|
||||
file reference these pre-existing networks).
|
||||
3. Mint the per-bottle CAs (chunk 2 writes them under
|
||||
state/<slug>/{pipelock,egress}/).
|
||||
4. Re-render pipelock yaml with the now-known internal CIDR so
|
||||
the SSRF allowlist exempts the bottle's own subnet.
|
||||
5. Populate the inner plans with launch-time fields so the
|
||||
renderer can read network names, CA paths, pipelock URL.
|
||||
6. Render the compose spec, write it to
|
||||
state/<slug>/docker-compose.yml, write metadata.json.
|
||||
7. `docker compose up -d` (token + OAuth values flow into the
|
||||
compose subprocess env so `environment: [NAME]` bare-name
|
||||
entries inherit without rendering values into the file).
|
||||
8. Provision (CA install, prompt copy, skills, workspace, git,
|
||||
supervise config) — unchanged, uses `docker exec` / `docker cp`.
|
||||
8. Provision (CA install, prompt copy, skills, git, supervise
|
||||
config) — unchanged, uses `docker exec`.
|
||||
9. Yield a DockerBottle handle. `exec_agent` runs claude via
|
||||
`docker exec -it` exactly like the pre-compose world.
|
||||
|
||||
@@ -37,16 +43,15 @@ from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...egress import egress_resolve_token_values
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import info, warn
|
||||
from ...log import info
|
||||
from . import network as network_mod
|
||||
from . import util as docker_mod
|
||||
from .bottle import DockerBottle
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from ...bottle_state import (
|
||||
from .bottle_state import (
|
||||
bottle_state_dir,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
pipelock_state_dir,
|
||||
)
|
||||
from .compose import (
|
||||
bottle_plan_to_compose,
|
||||
@@ -59,6 +64,10 @@ from .compose import (
|
||||
write_compose_file,
|
||||
)
|
||||
from .egress import egress_tls_init
|
||||
from .pipelock import (
|
||||
BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
pipelock_tls_init,
|
||||
)
|
||||
|
||||
|
||||
# Where the repo root lives, for `docker build` context. Computed once.
|
||||
@@ -69,26 +78,19 @@ _REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
def launch(
|
||||
plan: DockerBottlePlan,
|
||||
*,
|
||||
provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None],
|
||||
provision: Callable[[DockerBottlePlan, str], str | None],
|
||||
) -> Generator[DockerBottle, None, None]:
|
||||
"""Build, launch, and provision a Docker bottle via compose.
|
||||
Teardown on exit."""
|
||||
stack = ExitStack()
|
||||
|
||||
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||
|
||||
def teardown() -> None:
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc: # noqa: W0718 — teardown must not fail
|
||||
warn(
|
||||
f"teardown failed for container {plan.container_name}"
|
||||
f" (compose-down): {exc!r}"
|
||||
)
|
||||
revoke_git_gate_provisioned_keys(
|
||||
_bottle_for_revoke, _git_gate_dir_for_revoke
|
||||
)
|
||||
except BaseException:
|
||||
# Teardown must not raise; swallow so the caller's
|
||||
# __exit__ path can still propagate the original error.
|
||||
pass
|
||||
|
||||
try:
|
||||
# Step 1: agent image build. Sidecar images get built lazily by
|
||||
@@ -97,14 +99,40 @@ def launch(
|
||||
plan.image, _REPO_DIR,
|
||||
dockerfile=plan.dockerfile_path,
|
||||
)
|
||||
if plan.derived_image:
|
||||
docker_mod.build_image_with_cwd(
|
||||
plan.derived_image, plan.image, plan.spec.user_cwd
|
||||
)
|
||||
|
||||
# Networks: compose-managed. The names are derived
|
||||
# deterministically from the slug so the renderer can put
|
||||
# them on the services and `compose up` creates them with
|
||||
# those names. The empirical spike confirmed pipelock's
|
||||
# SSRF guard only checks proxied-request destinations, not
|
||||
# source IPs — so the bottle's own internal CIDR doesn't
|
||||
# need to be in `ssrf.ip_allowlist`. Pre-create + CIDR
|
||||
# introspection are gone; compose owns the network
|
||||
# lifecycle.
|
||||
internal_network = network_mod.network_name_for_slug(plan.slug)
|
||||
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
|
||||
|
||||
# Mint per-bottle CAs into state/<slug>/{pipelock,egress}/.
|
||||
ca_cert_host, ca_key_host = pipelock_tls_init(pipelock_state_dir(plan.slug))
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
egress_state_dir(plan.slug),
|
||||
)
|
||||
|
||||
# Populate launch-time fields on every inner plan so the
|
||||
# renderer reads concrete network names, CA paths, and
|
||||
# pipelock URL.
|
||||
proxy_plan = dataclasses.replace(
|
||||
plan.proxy_plan,
|
||||
internal_network=internal_network,
|
||||
internal_network_cidr="",
|
||||
egress_network=egress_network,
|
||||
ca_cert_host_path=ca_cert_host,
|
||||
ca_key_host_path=ca_key_host,
|
||||
)
|
||||
git_gate_plan = plan.git_gate_plan
|
||||
if git_gate_plan.upstreams:
|
||||
git_gate_plan = dataclasses.replace(
|
||||
@@ -112,13 +140,17 @@ def launch(
|
||||
internal_network=internal_network,
|
||||
egress_network=egress_network,
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
plan.egress_plan,
|
||||
internal_network=internal_network,
|
||||
egress_network=egress_network,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
)
|
||||
egress_plan = plan.egress_plan
|
||||
if egress_plan.routes:
|
||||
egress_plan = dataclasses.replace(
|
||||
egress_plan,
|
||||
internal_network=internal_network,
|
||||
egress_network=egress_network,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
pipelock_ca_host_path=ca_cert_host,
|
||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
)
|
||||
supervise_plan = plan.supervise_plan
|
||||
if supervise_plan is not None:
|
||||
supervise_plan = dataclasses.replace(
|
||||
@@ -127,6 +159,7 @@ def launch(
|
||||
)
|
||||
plan = dataclasses.replace(
|
||||
plan,
|
||||
proxy_plan=proxy_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
@@ -143,10 +176,11 @@ def launch(
|
||||
# Step 7: compose up. Token values + the OAuth placeholder
|
||||
# flow through subprocess env; the compose file holds only
|
||||
# bare names for the secret-carrying entries.
|
||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, effective_env,
|
||||
)
|
||||
token_values: dict[str, str] = {}
|
||||
if plan.egress_plan.routes:
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, dict(os.environ),
|
||||
)
|
||||
compose_env: dict[str, str] = {
|
||||
**os.environ,
|
||||
**plan.forwarded_env,
|
||||
@@ -166,25 +200,19 @@ def launch(
|
||||
compose_dump_logs, project, compose_file, compose_log_path(state_dir),
|
||||
)
|
||||
|
||||
# Step 8: provision. Create the bottle first so provisioners
|
||||
# can use bottle.exec / bottle.cp_in; set the prompt path
|
||||
# returned by provision_prompt after the fact.
|
||||
bottle = DockerBottle(
|
||||
plan.container_name,
|
||||
teardown,
|
||||
None,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=plan.spec.label or plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
# Step 8: provision. Unchanged — uses `docker exec` against
|
||||
# the agent container by its known name.
|
||||
prompt_path = provision(plan, plan.container_name)
|
||||
|
||||
# Step 9: yield. exec_agent continues to use `docker exec -it`
|
||||
# — the agent runs `sleep infinity` per the renderer's
|
||||
# service spec.
|
||||
yield bottle
|
||||
yield DockerBottle(
|
||||
plan.container_name,
|
||||
teardown,
|
||||
prompt_path,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
)
|
||||
finally:
|
||||
teardown()
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
"""Docker network plumbing for the per-agent egress topology.
|
||||
|
||||
The agent container sits on a Docker `--internal` network (no default
|
||||
gateway). Egress straddles that network and a per-agent user-defined
|
||||
bridge for upstream traffic. We deliberately do NOT use Docker's legacy
|
||||
gateway). Pipelock straddles that network and a per-agent user-defined
|
||||
bridge for upstream egress. We deliberately do NOT use Docker's legacy
|
||||
`bridge` network because only user-defined bridges run Docker's
|
||||
embedded DNS resolver, which egress needs to resolve upstream hostnames.
|
||||
embedded DNS resolver, which pipelock needs to resolve api.anthropic.com
|
||||
and similar upstream hostnames.
|
||||
|
||||
Naming: bot-bottle-net-<slug> (internal),
|
||||
bot-bottle-egress-<slug> (egress). Numeric suffix on conflict
|
||||
@@ -76,12 +77,20 @@ def network_create_internal(slug: str) -> str:
|
||||
|
||||
def network_create_egress(slug: str) -> str:
|
||||
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`)
|
||||
so the egress sidecar has working DNS for upstream hostnames."""
|
||||
so the pipelock sidecar has working DNS for upstream hostnames."""
|
||||
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
|
||||
|
||||
|
||||
def network_inspect_cidr(name: str) -> str:
|
||||
"""Return the IPv4 CIDR Docker assigned to a user-defined network."""
|
||||
"""Return the IPv4 CIDR Docker assigned to a user-defined network.
|
||||
|
||||
Used by pipelock's SSRF guard exception: the bottle's internal
|
||||
network sits in RFC1918 space, so pipelock's `internal:` list
|
||||
would block any agent request whose destination resolves there
|
||||
— including the cred-proxy sidecar's address. Adding the
|
||||
network's CIDR to pipelock's `ssrf.ip_allowlist` lets traffic
|
||||
targeted at the bottle's own sidecars through while pipelock
|
||||
still body-scans and api_allowlist-gates as usual."""
|
||||
result = subprocess.run(
|
||||
["docker", "network", "inspect",
|
||||
"--format", "{{range .IPAM.Config}}{{.Subnet}}{{end}}", name],
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Docker-side pipelock helpers: image pin, container naming, and
|
||||
the one-shot `pipelock tls init` host-side CA mint. The
|
||||
prepare-time YAML rendering itself lives on the platform-neutral
|
||||
`PipelockProxy` ABC — backends instantiate it directly.
|
||||
|
||||
The per-container `.start()` / `.stop()` lifecycle was deleted in
|
||||
PRD 0024 chunk 3; compose-up owns the container lifecycle (PRD
|
||||
0018) and the bundle path (PRD 0024) collapses pipelock + egress
|
||||
+ git-gate + supervise into one container."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import die
|
||||
# Re-exported for the compose renderer + smolmachines launch step
|
||||
# (they used to import these from this module before they moved to
|
||||
# the platform-neutral pipelock module).
|
||||
from ...pipelock import ( # noqa: F401
|
||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||
)
|
||||
|
||||
|
||||
# Pipelock image, pinned by digest. The digest is the multi-arch image
|
||||
# index for ghcr.io/luckypipewrench/pipelock:2.3.0.
|
||||
PIPELOCK_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_PIPELOCK_IMAGE",
|
||||
"ghcr.io/luckypipewrench/pipelock@sha256:3b1a39417b98406ddc5dc2d8fcb42865ddc0c68a43d355db55f0f8cb06bc6de9",
|
||||
)
|
||||
|
||||
# Listening port for pipelock's forward proxy.
|
||||
PIPELOCK_PORT = os.environ.get("BOT_BOTTLE_PIPELOCK_PORT", "8888")
|
||||
|
||||
|
||||
# The URL egress dials for its upstream HTTPS_PROXY. egress and
|
||||
# pipelock share the same container's network namespace inside the
|
||||
# sidecar bundle, so loopback reaches pipelock directly — no docker
|
||||
# DNS aliases involved.
|
||||
BUNDLE_LOCAL_PIPELOCK_URL = f"http://127.0.0.1:{PIPELOCK_PORT}"
|
||||
|
||||
|
||||
def pipelock_tls_init(stage_dir: Path) -> tuple[Path, Path]:
|
||||
"""Generate a fresh per-bottle CA via a one-shot pipelock container.
|
||||
|
||||
Runs `pipelock tls init` against a host-mounted scratch dir, leaving
|
||||
`ca.pem` (public cert, mode 600) and `ca-key.pem` (private key, mode
|
||||
600) under `<stage_dir>/pipelock-ca/`. Returns the two host paths.
|
||||
|
||||
The image is pinned (same digest the running sidecar uses) so the
|
||||
generated CA matches what the sidecar expects. Output is owned by
|
||||
whatever UID the one-shot ran as; the compose renderer's
|
||||
bind-mounts pin the files in place at runtime, so ownership
|
||||
inside the running sidecar (root in pipelock's distroless image)
|
||||
is independent."""
|
||||
work = stage_dir / "pipelock-ca"
|
||||
work.mkdir(exist_ok=True)
|
||||
result = subprocess.run(
|
||||
["docker", "run", "--rm",
|
||||
"-v", f"{work}:/h",
|
||||
"-e", "PIPELOCK_HOME=/h",
|
||||
PIPELOCK_IMAGE, "tls", "init"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(f"pipelock tls init failed: {result.stderr.strip()}")
|
||||
cert = work / "ca.pem"
|
||||
key = work / "ca-key.pem"
|
||||
if not cert.is_file() or not key.is_file():
|
||||
die(f"pipelock tls init did not produce ca files in {work}")
|
||||
# Explicit perms in case a future pipelock release changes
|
||||
# defaults. Pipelock runs as root in its distroless image and
|
||||
# bind-mounts work with 0o600 (root reads everything); the key
|
||||
# has no reason to be readable to anyone else on the host.
|
||||
key.chmod(0o600)
|
||||
cert.chmod(0o644)
|
||||
return (cert, key)
|
||||
@@ -0,0 +1,200 @@
|
||||
"""pipelock_apply — host-side helper to apply an api_allowlist
|
||||
change to a running pipelock sidecar (PRD 0015).
|
||||
|
||||
Used by the supervise dashboard when the operator approves a
|
||||
pipelock-block proposal (or runs the operator-initiated `pipelock
|
||||
edit <bottle>` verb). Fetches the current pipelock.yaml via `docker
|
||||
exec`, parses it, swaps the api_allowlist with the proposed hosts,
|
||||
re-renders, writes back via the bind-mount path, then signals the
|
||||
bundle supervisor to restart the pipelock daemon (`docker kill
|
||||
--signal USR1`) so
|
||||
pipelock picks up the new config.
|
||||
|
||||
v1 uses restart, not SIGHUP — pipelock has no in-process reload
|
||||
hook and adding one is the "SIGHUP reload for pipelock" open
|
||||
question in PRD 0015. Restart drops in-flight outbound calls; the
|
||||
agent's HTTP client retries pick up against the restarted proxy.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from ...pipelock import pipelock_render_yaml
|
||||
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
|
||||
from .bottle_state import pipelock_state_dir
|
||||
from .sidecar_bundle import sidecar_bundle_container_name
|
||||
|
||||
|
||||
def _pipelock_yaml_host_path(slug: str) -> Path:
|
||||
"""The bind-mount source for the pipelock sidecar's
|
||||
pipelock.yaml — matches what pipelock.prepare wrote at chunk-2
|
||||
paths."""
|
||||
return pipelock_state_dir(slug) / "pipelock.yaml"
|
||||
|
||||
|
||||
PIPELOCK_YAML_IN_CONTAINER = "/etc/pipelock.yaml"
|
||||
|
||||
# Allowlist proposals are one-hostname-per-line. Blank lines and
|
||||
# `#`-prefixed comments are ignored. The character set matches the
|
||||
# supervise sidecar's syntactic check on the agent's pipelock-block
|
||||
# proposal (alphanumerics + dot/dash/underscore).
|
||||
_HOST_OK = re.compile(r"^[A-Za-z0-9_.-]+$")
|
||||
|
||||
|
||||
class PipelockApplyError(RuntimeError):
|
||||
"""Raised when fetch / parse / apply fails. The dashboard renders
|
||||
the message and keeps the proposal pending — never crashes."""
|
||||
|
||||
|
||||
def parse_allowlist_content(content: str) -> list[str]:
|
||||
"""One hostname per line. Blanks and `#` comments are ignored.
|
||||
Raises PipelockApplyError if a line has a disallowed character."""
|
||||
hosts: list[str] = []
|
||||
for i, raw_line in enumerate(content.splitlines(), start=1):
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
if not _HOST_OK.match(line):
|
||||
raise PipelockApplyError(
|
||||
f"allowlist line {i}: {line!r} has disallowed characters"
|
||||
)
|
||||
hosts.append(line)
|
||||
return hosts
|
||||
|
||||
|
||||
def render_allowlist_content(hosts: list[str]) -> str:
|
||||
"""Hosts → one-per-line string (the operator-facing format)."""
|
||||
if not hosts:
|
||||
return ""
|
||||
return "\n".join(hosts) + "\n"
|
||||
|
||||
|
||||
def fetch_current_yaml(slug: str) -> str:
|
||||
"""Read the live /etc/pipelock.yaml from the sidecar bundle.
|
||||
|
||||
Uses `docker cp` because pipelock inside the bundle is the
|
||||
distroless pipelock binary with no shell, and `docker cp` is a
|
||||
daemon-API tarball copy that works regardless of what's
|
||||
available inside the container.
|
||||
|
||||
Raises PipelockApplyError if the read fails."""
|
||||
container = sidecar_bundle_container_name(slug)
|
||||
fd, tmp_path = tempfile.mkstemp(prefix="cb-pipelock-fetch.", suffix=".yaml")
|
||||
os.close(fd)
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[
|
||||
"docker", "cp",
|
||||
f"{container}:{PIPELOCK_YAML_IN_CONTAINER}", tmp_path,
|
||||
],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
raise PipelockApplyError(
|
||||
f"could not fetch pipelock.yaml from {container}: "
|
||||
f"{(r.stderr or '').strip() or 'container not running?'}"
|
||||
)
|
||||
return Path(tmp_path).read_text()
|
||||
finally:
|
||||
try:
|
||||
Path(tmp_path).unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def fetch_current_allowlist(slug: str) -> str:
|
||||
"""Fetch the live yaml, extract api_allowlist, render as one-per-
|
||||
line — the operator-facing format for the TUI / agent's
|
||||
current-config mount."""
|
||||
yaml = fetch_current_yaml(slug)
|
||||
try:
|
||||
cfg = parse_yaml_subset(yaml)
|
||||
except YamlSubsetError as e:
|
||||
raise PipelockApplyError(f"running pipelock yaml: {e}") from e
|
||||
hosts = cfg.get("api_allowlist", [])
|
||||
if not isinstance(hosts, list):
|
||||
raise PipelockApplyError(
|
||||
"running pipelock yaml: api_allowlist is not a list"
|
||||
)
|
||||
return render_allowlist_content([str(h) for h in hosts])
|
||||
|
||||
|
||||
def apply_allowlist_change(
|
||||
slug: str, new_allowlist_content: str,
|
||||
) -> tuple[str, str]:
|
||||
"""Apply `new_allowlist_content` to the sidecar bundle:
|
||||
1. Parse the proposed hosts (one per line).
|
||||
2. Fetch + parse current pipelock.yaml.
|
||||
3. Replace api_allowlist with the proposed hosts; re-render.
|
||||
4. Write the new yaml to the bind-mount source.
|
||||
5. `docker kill --signal USR1 <bundle>` so the supervisor
|
||||
restarts the pipelock daemon in place (leaving egress,
|
||||
git-gate, and supervise running). Pipelock has no
|
||||
in-process reload; the supervisor's per-daemon restart
|
||||
keeps the agent's MCP socket alive — a whole-bundle
|
||||
`docker restart` would bounce supervise too.
|
||||
|
||||
Returns (before, after) where both are one-per-line allowlist
|
||||
strings (operator-facing format). Raises PipelockApplyError on
|
||||
any failure; the sidecar's existing config stays in place until
|
||||
the host write succeeds, and the SIGUSR1 is what makes it
|
||||
live."""
|
||||
new_hosts = parse_allowlist_content(new_allowlist_content)
|
||||
container = sidecar_bundle_container_name(slug)
|
||||
current_yaml = fetch_current_yaml(slug)
|
||||
try:
|
||||
cfg = parse_yaml_subset(current_yaml)
|
||||
except YamlSubsetError as e:
|
||||
raise PipelockApplyError(f"running pipelock yaml: {e}") from e
|
||||
current_hosts = cfg.get("api_allowlist", [])
|
||||
if not isinstance(current_hosts, list):
|
||||
raise PipelockApplyError(
|
||||
"running pipelock yaml: api_allowlist is not a list"
|
||||
)
|
||||
|
||||
before = render_allowlist_content([str(h) for h in current_hosts])
|
||||
after = render_allowlist_content(new_hosts)
|
||||
|
||||
cfg["api_allowlist"] = new_hosts
|
||||
rendered = pipelock_render_yaml(cfg)
|
||||
|
||||
# pipelock.yaml is bind-mounted into the container as a SINGLE
|
||||
# FILE — same Docker single-file inode issue as egress_apply:
|
||||
# write-temp-then-rename swaps the host inode and leaves the
|
||||
# container's mount pointing at the orphaned old one. Write
|
||||
# in-place. The SIGUSR1 below makes the new content live
|
||||
# (pipelock has no in-process reload, so the supervisor
|
||||
# restarts the pipelock daemon in response).
|
||||
target = _pipelock_yaml_host_path(slug)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(rendered)
|
||||
# pipelock runs as root in its distroless image — any mode is
|
||||
# fine — but 0o600 matches what prepare wrote.
|
||||
target.chmod(0o600)
|
||||
restart = subprocess.run(
|
||||
["docker", "kill", "--signal", "USR1", container],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if restart.returncode != 0:
|
||||
raise PipelockApplyError(
|
||||
f"failed to signal {container} for pipelock restart: "
|
||||
f"{(restart.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
return before, after
|
||||
|
||||
|
||||
__all__ = [
|
||||
"PIPELOCK_YAML_IN_CONTAINER",
|
||||
"PipelockApplyError",
|
||||
"apply_allowlist_change",
|
||||
"fetch_current_allowlist",
|
||||
"fetch_current_yaml",
|
||||
"parse_allowlist_content",
|
||||
"render_allowlist_content",
|
||||
]
|
||||
@@ -0,0 +1,272 @@
|
||||
"""Prepare step for the Docker bottle backend.
|
||||
|
||||
`resolve_plan` does all host-side resolution (image and container
|
||||
names, env-file, prompt-file, proxy plan, runtime detection) and
|
||||
returns a frozen DockerBottlePlan. No Docker resources are created;
|
||||
the only side effects are scratch files under `stage_dir` and a probe
|
||||
of `docker info`. Cross-backend host-side validation has already run
|
||||
via the base class's `prepare` template before this is called.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import runtime_for
|
||||
from ...egress import Egress
|
||||
from ...env import ResolvedEnv, resolve_env
|
||||
from ...git_gate import GitGate
|
||||
from ...log import die
|
||||
from ...pipelock import PipelockProxy
|
||||
from ...supervise import Supervise
|
||||
from .. import BottleSpec
|
||||
from . import util as docker_mod
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .bottle_state import (
|
||||
BottleMetadata,
|
||||
agent_state_dir,
|
||||
bottle_identity,
|
||||
clear_preserve_marker,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
per_bottle_dockerfile,
|
||||
per_bottle_dockerfile_path,
|
||||
per_bottle_image_tag,
|
||||
pipelock_state_dir,
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from .sidecar_bundle import sidecar_bundle_container_name
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
stage_dir: Path,
|
||||
) -> DockerBottlePlan:
|
||||
"""Resolve Docker-specific names and write scratch files. Trusts
|
||||
that the agent and its skills/git-gate keys are present —
|
||||
validation already ran in the base class."""
|
||||
docker_mod.require_docker()
|
||||
|
||||
proxy = PipelockProxy()
|
||||
git_gate = GitGate()
|
||||
egress = Egress()
|
||||
supervise = Supervise()
|
||||
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
provider = bottle.agent_provider
|
||||
provider_runtime = runtime_for(provider.template)
|
||||
|
||||
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
||||
# mints a random-suffixed identity (so parallel runs of the same
|
||||
# agent in the same cwd don't collide on container/network
|
||||
# names); a `resume` passes the recorded identity in via
|
||||
# spec.identity to continue an existing bottle's state.
|
||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||
# Record the launch metadata so `cli.py resume <identity>` can
|
||||
# reconstruct the spec. Idempotent — re-writes on resume with a
|
||||
# refreshed started_at.
|
||||
write_metadata(BottleMetadata(
|
||||
identity=slug,
|
||||
agent_name=spec.agent_name,
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
compose_project=f"bot-bottle-{slug}",
|
||||
))
|
||||
# Clear any leftover preserve marker from a prior capability-block
|
||||
# so this fresh launch can be cleaned up at session-end unless
|
||||
# the agent triggers another capability-block.
|
||||
clear_preserve_marker(slug)
|
||||
|
||||
# PRD 0016 capability-block: if a per-bottle Dockerfile has been
|
||||
# written (via apply_capability_change), the base image becomes
|
||||
# per_bottle_image_tag(slug) built from that file. --cwd still
|
||||
# layers a derived image on top.
|
||||
dockerfile_path = ""
|
||||
if per_bottle_dockerfile(slug) is not None:
|
||||
image_default = per_bottle_image_tag(slug)
|
||||
dockerfile_path = str(per_bottle_dockerfile_path(slug))
|
||||
elif provider.dockerfile:
|
||||
image_default = f"bot-bottle-{provider.template}:{slug}"
|
||||
dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
|
||||
elif provider_runtime.dockerfile:
|
||||
image_default = provider_runtime.image
|
||||
dockerfile_path = provider_runtime.dockerfile
|
||||
else:
|
||||
image_default = provider_runtime.image
|
||||
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||
derived_image = ""
|
||||
runtime_image = image
|
||||
if spec.copy_cwd:
|
||||
derived_image = os.environ.get(
|
||||
"BOT_BOTTLE_DERIVED_IMAGE", f"bot-bottle-cwd:{slug}"
|
||||
)
|
||||
runtime_image = derived_image
|
||||
|
||||
default_container = f"bot-bottle-{slug}"
|
||||
pinned_container = os.environ.get("BOT_BOTTLE_CONTAINER", "")
|
||||
container_name_pinned = bool(pinned_container)
|
||||
if container_name_pinned:
|
||||
container_name = pinned_container
|
||||
if docker_mod.container_exists(container_name):
|
||||
die(
|
||||
f"container '{container_name}' already exists "
|
||||
f"(pinned via BOT_BOTTLE_CONTAINER). "
|
||||
f"Remove it with 'docker rm -f {container_name}' or unset the override."
|
||||
)
|
||||
else:
|
||||
container_name = ""
|
||||
for candidate in docker_mod.container_name_candidates(default_container):
|
||||
if not docker_mod.container_exists(candidate):
|
||||
container_name = candidate
|
||||
break
|
||||
if not container_name:
|
||||
die(
|
||||
f"could not find a free container name after "
|
||||
f"{default_container}-{docker_mod.MAX_CONTAINER_SUFFIX}; "
|
||||
f"clean up old containers with 'docker rm -f <name>'"
|
||||
)
|
||||
|
||||
# Probe the sidecar-bundle container name for an orphan from a
|
||||
# previous run. Otherwise a stale bundle surfaces as a
|
||||
# docker-create conflict deep inside launch() with no actionable
|
||||
# hint; failing fast here points at the cleanup command.
|
||||
bundle_name = sidecar_bundle_container_name(slug)
|
||||
if docker_mod.container_exists(bundle_name):
|
||||
die(
|
||||
f"sidecar bundle container '{bundle_name}' already exists. "
|
||||
f"This is an orphan from a previous run; clean it up with "
|
||||
f"'./cli.py cleanup' (or 'docker rm -f {bundle_name}') and "
|
||||
f"retry."
|
||||
)
|
||||
|
||||
# PRD 0018 chunk 2: prepare-time scratch files live under
|
||||
# ~/.bot-bottle/state/<slug>/<service>/ so chunk 3's compose
|
||||
# bind-mounts can point at stable paths. The state subdirs are
|
||||
# cleaned up by start.py's session-end teardown unless something
|
||||
# explicitly preserves the state dir (capability-block, crash).
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
env_file = agent_dir / "agent.env"
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
prompt_file.write_text("")
|
||||
prompt_file.chmod(0o600)
|
||||
|
||||
pipelock_dir = pipelock_state_dir(slug)
|
||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||
proxy_plan = proxy.prepare(bottle, slug, pipelock_dir)
|
||||
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = egress.prepare(bottle, slug, egress_dir)
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
# Current Dockerfile for the agent image. Read from the repo
|
||||
# root; for `--cwd` derived images the base Dockerfile is what
|
||||
# the agent should propose changes against (the derived layer
|
||||
# is just a workspace copy).
|
||||
# (routes.yaml + pipelock allowlist used to land here too but
|
||||
# PRD 0017 chunk 3 moved them behind the
|
||||
# `list-egress-routes` MCP tool so the agent gets live
|
||||
# state rather than a launch-time snapshot.)
|
||||
supervise_dockerfile_path = (
|
||||
Path(dockerfile_path)
|
||||
if dockerfile_path
|
||||
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
|
||||
)
|
||||
dockerfile_content = (
|
||||
supervise_dockerfile_path.read_text()
|
||||
if supervise_dockerfile_path.is_file()
|
||||
else ""
|
||||
)
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
supervise_plan = supervise.prepare(
|
||||
slug, supervise_dir,
|
||||
dockerfile_content=dockerfile_content,
|
||||
)
|
||||
resolved = resolve_env(manifest, spec.agent_name)
|
||||
# Everything that should reach the bottle by-name (so its value
|
||||
# never lands on argv or in env_file) goes into one dict. Nothing
|
||||
# mutates the host os.environ.
|
||||
forwarded_env: dict[str, str] = dict(resolved.forwarded)
|
||||
# When the bottle declares an egress route with the
|
||||
# `claude_code_oauth` role marker, claude-code's outbound
|
||||
# Authorization gets stripped + re-injected by egress. The
|
||||
# agent's environ still needs *something* claude-code recognises
|
||||
# as a credential or it refuses to start; ship a non-secret
|
||||
# placeholder. The placeholder isn't any real token value, so
|
||||
# leaking it would tell an attacker only that egress is in
|
||||
# front. Manifest validation enforces singleton on this role.
|
||||
has_provider_auth = any(
|
||||
provider_runtime.auth_role in r.roles for r in egress_plan.routes
|
||||
)
|
||||
if has_provider_auth:
|
||||
forwarded_env[provider_runtime.placeholder_env] = "egress-placeholder"
|
||||
if provider.template == "claude" and has_provider_auth:
|
||||
# Belt-and-braces: turn off telemetry endpoints (statsig,
|
||||
# error reporting) that egress can't gate by auth.
|
||||
forwarded_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
|
||||
forwarded_env.setdefault("DISABLE_ERROR_REPORTING", "1")
|
||||
_write_env_file(resolved, env_file)
|
||||
prompt_file.write_text(agent.prompt)
|
||||
|
||||
use_runsc = docker_mod.runsc_available()
|
||||
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
container_name=container_name,
|
||||
container_name_pinned=container_name_pinned,
|
||||
image=image,
|
||||
derived_image=derived_image,
|
||||
runtime_image=runtime_image,
|
||||
dockerfile_path=dockerfile_path,
|
||||
env_file=env_file,
|
||||
forwarded_env=forwarded_env,
|
||||
prompt_file=prompt_file,
|
||||
proxy_plan=proxy_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
use_runsc=use_runsc,
|
||||
agent_command=provider_runtime.command,
|
||||
agent_prompt_mode=provider_runtime.prompt_mode,
|
||||
agent_provider_template=provider.template,
|
||||
)
|
||||
|
||||
|
||||
def _write_env_file(resolved: ResolvedEnv, env_file: Path) -> None:
|
||||
"""Serialize the literal portion of a ResolvedEnv into docker's
|
||||
`--env-file` syntax (NAME=VALUE per line, mode 600 since the file
|
||||
may carry verbatim values from the manifest). Forwarded names ride
|
||||
on the plan as a structured tuple instead."""
|
||||
env_lines: list[str] = []
|
||||
for name, value in resolved.literals.items():
|
||||
if "\n" in value:
|
||||
die(
|
||||
f"env entry {name} (literal) contains a newline; "
|
||||
f"docker --env-file cannot represent multi-line values."
|
||||
)
|
||||
env_lines.append(f"{name}={value}")
|
||||
env_file.write_text("\n".join(env_lines) + ("\n" if env_lines else ""))
|
||||
env_file.chmod(0o600)
|
||||
|
||||
|
||||
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
|
||||
path = Path(os.path.expanduser(path_value))
|
||||
if not path.is_absolute():
|
||||
path = Path(spec.user_cwd) / path
|
||||
return str(path)
|
||||
@@ -1,12 +1,8 @@
|
||||
"""Backend-infrastructure provisioners for the Docker backend.
|
||||
"""Per-provisioner modules for the Docker backend.
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration) live on
|
||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
|
||||
provisioning also moved to the AgentProvider ABC (with Debian/node
|
||||
defaults); user plugins override them for non-standard images.
|
||||
Each module exports one top-level function:
|
||||
provision_<thing>(plan: DockerBottlePlan, target: str) -> ...
|
||||
|
||||
No modules remain in this subpackage — the directory is kept so that
|
||||
existing imports of `from .provision import ...` don't need updating
|
||||
if new backend-specific provisioners are added later.
|
||||
"""
|
||||
`DockerBottleBackend.provision_*` methods delegate to these. The
|
||||
abstract `BottleBackend.provision_*` surface is unchanged; this
|
||||
subpackage exists only to keep `backend.py` from being a god-file."""
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Install the per-bottle MITM CA into the agent container's trust
|
||||
store.
|
||||
|
||||
Post-PRD-0017 the CA depends on the agent's HTTP_PROXY target:
|
||||
|
||||
- Bottle declares `egress.routes[]` → agent's HTTP_PROXY
|
||||
points at egress; the cert the agent must trust is the
|
||||
one egress mints leaf certs with (the egress CA).
|
||||
- No egress routes → agent's HTTP_PROXY points straight at
|
||||
pipelock; the cert the agent must trust is pipelock's CA (the
|
||||
pre-cutover behavior).
|
||||
|
||||
By the time this provisioner runs, the corresponding `tls_init`
|
||||
helper has generated the chosen CA under `plan.stage_dir`, and the
|
||||
sidecar (pipelock or egress) is up referencing the
|
||||
in-container CA paths.
|
||||
|
||||
Cert lands on Debian's standard source path
|
||||
(`/usr/local/share/ca-certificates/`); `update-ca-certificates`
|
||||
rebuilds `/etc/ssl/certs/ca-certificates.crt`, which is what curl,
|
||||
Python `ssl`, and OpenSSL-based tools all read by default. The env
|
||||
trio set on the agent's `docker run` covers Node
|
||||
(`NODE_EXTRA_CA_CERTS`) and Python `requests` /
|
||||
`SSL_CERT_FILE`-honoring libraries that don't load the system
|
||||
bundle.
|
||||
|
||||
The fingerprint is computed via stdlib (`ssl.PEM_cert_to_DER_cert`
|
||||
+ `hashlib.sha256`) and logged once to stderr. The private key
|
||||
stays on the host (under `stage_dir`) until teardown wipes the
|
||||
stage dir; nothing in the agent ever sees it."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_ca(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Copy the agent-facing CA cert into the agent, rebuild the
|
||||
trust bundle, emit a one-line fingerprint log. Called from
|
||||
`BottleBackend.provision` after the agent container is up."""
|
||||
container = target
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
||||
|
||||
subprocess.run(
|
||||
["docker", "cp", str(cert_host_path), f"{container}:{AGENT_CA_PATH}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "chmod", "644", AGENT_CA_PATH],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "update-ca-certificates"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
log_ca_fingerprint(cert_host_path, label)
|
||||
@@ -0,0 +1,121 @@
|
||||
"""Git provisioning inside a running Docker bottle.
|
||||
|
||||
Three concerns, all about git in the agent:
|
||||
|
||||
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
||||
into /home/node/workspace/.git so the agent operates on the
|
||||
user's repo.
|
||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||
~/.gitconfig with insteadOf rules so every git operation
|
||||
against a declared upstream (push, fetch, clone, pull,
|
||||
ls-remote) transparently hits the per-agent git-gate. The
|
||||
gate mirrors the upstream in both directions, so URL
|
||||
rewriting is symmetric.
|
||||
3. If the bottle declares `git.user` (issue #86), set
|
||||
`git config --global user.{name,email}` inside the bottle so
|
||||
the agent's commits are attributed to that identity.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||
from ....log import info
|
||||
from .. import util as docker_mod
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_git(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Set up git inside the bottle. Runs all three subcases; each
|
||||
no-ops when its condition isn't met."""
|
||||
_provision_cwd_git(plan, target)
|
||||
_provision_git_gate_config(plan, target)
|
||||
_provision_git_user(plan, target)
|
||||
|
||||
|
||||
def _provision_cwd_git(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||
it into /home/node/workspace/.git and fix ownership. No-op
|
||||
otherwise."""
|
||||
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
||||
return
|
||||
container = target
|
||||
info(f"copying {plan.spec.user_cwd}/.git -> {container}:/home/node/workspace/.git")
|
||||
subprocess.run(
|
||||
["docker", "cp", f"{plan.spec.user_cwd}/.git", f"{container}:/home/node/workspace/.git"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
[
|
||||
"docker", "exec", "-u", "0", container,
|
||||
"chown", "-R", "node:node", "/home/node/workspace/.git",
|
||||
],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def _provision_git_gate_config(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Write ~/.gitconfig in the bottle with the git-gate
|
||||
insteadOf rules. No-op when the bottle has no `git` entries."""
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if not bottle.git:
|
||||
return
|
||||
container = target
|
||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||
container_gitconfig = f"{container_home}/.gitconfig"
|
||||
|
||||
content = git_gate_render_gitconfig(bottle.git, GIT_GATE_HOSTNAME)
|
||||
config_file = plan.stage_dir / "agent_gitconfig"
|
||||
config_file.write_text(content)
|
||||
config_file.chmod(0o600)
|
||||
|
||||
info(f"writing {container_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
||||
subprocess.run(
|
||||
["docker", "cp", str(config_file), f"{container}:{container_gitconfig}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
docker_mod.docker_exec_root(container, ["chown", "node:node", container_gitconfig])
|
||||
docker_mod.docker_exec_root(container, ["chmod", "644", container_gitconfig])
|
||||
|
||||
|
||||
def _provision_git_user(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Apply `git config --global user.{name,email}` inside the
|
||||
bottle so the agent's commits are attributed to the operator-
|
||||
chosen identity instead of the agent image's default
|
||||
(which is no user — git would refuse to commit at all
|
||||
until the agent ran its own `git config`).
|
||||
|
||||
Runs as the `node` user so `--global` lands in
|
||||
`/home/node/.gitconfig` (matching the existing
|
||||
`_provision_git_gate_config` write location). No-op when the
|
||||
bottle didn't declare `git.user`.
|
||||
|
||||
Each field set independently — name-only or email-only
|
||||
configs only run the `git config` line for the field
|
||||
present."""
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
gu = bottle.git_user
|
||||
if gu.is_empty():
|
||||
return
|
||||
if gu.name:
|
||||
info(f"git config --global user.name = {gu.name!r}")
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "node", target,
|
||||
"git", "config", "--global", "user.name", gu.name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
if gu.email:
|
||||
info(f"git config --global user.email = {gu.email!r}")
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "node", target,
|
||||
"git", "config", "--global", "user.email", gu.email],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Copy the agent prompt into a running Docker bottle.
|
||||
|
||||
The prompt file is always copied (so the in-container path always
|
||||
exists) but `--append-system-prompt-file` only fires when the agent
|
||||
actually has a prompt — the return value signals which case."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_prompt(plan: DockerBottlePlan, target: str) -> str | None:
|
||||
"""Copy the prompt file into the container, fix ownership/mode.
|
||||
Returns the in-container path if the agent has a non-empty
|
||||
prompt (drives --append-system-prompt-file), else None. The
|
||||
file is copied either way so the path always exists."""
|
||||
container = target
|
||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||
in_container_prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
subprocess.run(
|
||||
["docker", "cp", str(plan.prompt_file), f"{container}:{in_container_prompt_path}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
# `docker cp` preserves host UID; re-own/mode as root so node
|
||||
# can read its own mode-600 prompt regardless of host UID.
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "chown", "node:node", in_container_prompt_path],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", "-u", "0", container, "chmod", "600", in_container_prompt_path],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return in_container_prompt_path if agent.prompt else None
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Copy host-side skill directories into a running Docker bottle.
|
||||
|
||||
Skills are validated on the host before launch by the base class's
|
||||
`BottleBackend._validate_skills` (called from `prepare`); this module
|
||||
assumes that validation has already run. A skill disappearing between
|
||||
validation and copy still dies loudly rather than silently producing
|
||||
a partial container."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from ....log import die, info
|
||||
from ...util import host_skill_dir
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
def provision_skills(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Copy each of the agent's named skills from the host's
|
||||
~/.claude/skills/<name>/ into the container's equivalent path.
|
||||
For each skill: ensure parent dir, wipe any prior copy, then
|
||||
`docker cp <host>/. <container>:<dst>/` so the contents are
|
||||
copied into a freshly-created destination dir. No-op when the
|
||||
agent has no skills."""
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
|
||||
container = target
|
||||
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||
skills_dir = os.environ.get(
|
||||
"BOT_BOTTLE_CONTAINER_SKILLS_DIR", f"{container_home}/.claude/skills"
|
||||
)
|
||||
|
||||
subprocess.run(
|
||||
["docker", "exec", container, "mkdir", "-p", skills_dir],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
for n in agent.skills:
|
||||
src = host_skill_dir(n)
|
||||
if not os.path.isdir(src):
|
||||
die(f"skill '{n}' disappeared from host between validation and copy at {src}.")
|
||||
dst = f"{skills_dir}/{n}"
|
||||
info(f"copying skill {n} into {container}:{dst}")
|
||||
subprocess.run(
|
||||
["docker", "exec", container, "rm", "-rf", dst],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "exec", container, "mkdir", "-p", dst],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "cp", f"{src}/.", f"{container}:{dst}/"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Supervise sidecar provisioning inside a running Docker bottle
|
||||
(PRD 0013).
|
||||
|
||||
Registers the per-bottle supervise sidecar as an HTTP MCP server in
|
||||
the agent's claude-code config so the agent discovers the three
|
||||
stuck-recovery MCP tools (cred-proxy-block, pipelock-block,
|
||||
capability-block) at startup.
|
||||
|
||||
Uses `claude mcp add` rather than writing JSON directly. claude-code
|
||||
owns the on-disk config format (`~/.claude.json` `mcpServers` shape,
|
||||
field names, scope semantics) and changes it between versions; the
|
||||
official command handles whatever the installed version expects.
|
||||
|
||||
No-op when bottle.supervise is False — bottles that haven't opted
|
||||
into the supervise sidecar shouldn't get an MCP entry pointing at a
|
||||
sidecar that isn't running.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ....log import info, warn
|
||||
from ....supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||
from ..bottle_plan import DockerBottlePlan
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def supervise_mcp_url() -> str:
|
||||
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
||||
|
||||
|
||||
def provision_supervise(plan: DockerBottlePlan, target: str) -> None:
|
||||
"""Run `claude mcp add` inside the agent container to register
|
||||
the supervise sidecar in claude-code's user config. No-op when
|
||||
bottle.supervise is False.
|
||||
|
||||
Failure is logged but not fatal: the bottle still works (you
|
||||
just can't call supervise tools from the agent until the entry
|
||||
is added manually). The operator sees the warning at launch."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
url = supervise_mcp_url()
|
||||
argv = [
|
||||
"docker", "exec", "-u", "node", target,
|
||||
"claude", "mcp", "add",
|
||||
"--scope", "user",
|
||||
"--transport", "http",
|
||||
_SUPERVISE_MCP_NAME,
|
||||
url,
|
||||
]
|
||||
info(f"registering supervise MCP server in agent claude config → {url}")
|
||||
r = subprocess.run(argv, capture_output=True, text=True, check=False)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"claude mcp add --scope user --transport http supervise {url}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["provision_supervise", "supervise_mcp_url"]
|
||||
@@ -1,59 +0,0 @@
|
||||
"""Prepare step for the Docker bottle backend.
|
||||
|
||||
`resolve_plan` does all host-side resolution (image and container
|
||||
names, prompt-file, proxy plan, runtime detection) and returns a
|
||||
frozen DockerBottlePlan. No Docker resources are created; the only
|
||||
side effects are scratch files under `stage_dir` and a probe of
|
||||
`docker info`. Cross-backend host-side validation has already run
|
||||
via the base class's `prepare` template before this is called.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from . import util as docker_mod
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .. import BottleSpec
|
||||
from ...env import ResolvedEnv
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...git_gate import GitGatePlan
|
||||
|
||||
def preflight() -> None:
|
||||
docker_mod.require_docker()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return dict(resolved_env.literals)
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> DockerBottlePlan:
|
||||
"""Resolve Docker-specific names and write scratch files. Trusts
|
||||
that the agent and its skills/git-gate keys are present —
|
||||
validation already ran in the base class."""
|
||||
|
||||
# ==== docker specific setup ====
|
||||
use_runsc = docker_mod.runsc_available()
|
||||
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
forwarded_env=dict(resolved_env.forwarded),
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
use_runsc=use_runsc,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -2,10 +2,10 @@
|
||||
(PRD 0024).
|
||||
|
||||
The bundle image (built by Dockerfile.sidecars, PRD 0024 chunk 1)
|
||||
runs egress + git-gate + supervise as one container per bottle
|
||||
under a small Python init supervisor. As of chunk 5 the bundle
|
||||
is the only shape — the legacy four-sidecar topology and its
|
||||
`BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
|
||||
runs pipelock + egress + git-gate + supervise as one container
|
||||
per bottle under a small Python init supervisor. As of chunk 5
|
||||
the bundle is the only shape — the legacy four-sidecar topology
|
||||
and its `BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -14,7 +14,8 @@ import os
|
||||
|
||||
# Bundle image. Defaults to a built-locally tag (built from the
|
||||
# repo's Dockerfile.sidecars via compose `build:`). Operators
|
||||
# pinning to a published digest can override via env.
|
||||
# pinning to a published digest can override via env, matching
|
||||
# the existing `BOT_BOTTLE_PIPELOCK_IMAGE` shape.
|
||||
SIDECAR_BUNDLE_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_SIDECAR_IMAGE",
|
||||
"bot-bottle-sidecars:latest",
|
||||
|
||||
@@ -10,7 +10,6 @@ import subprocess
|
||||
from typing import Iterable, Iterator
|
||||
|
||||
from ...log import die, info
|
||||
# from ...workspace import WorkspacePlan
|
||||
|
||||
|
||||
# Cap on the suffix the container-name conflict logic will try before
|
||||
@@ -117,39 +116,35 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
||||
subprocess.run(args, check=True)
|
||||
|
||||
|
||||
# def build_image_with_cwd(
|
||||
# derived: str,
|
||||
# base: str,
|
||||
# workspace: "WorkspacePlan",
|
||||
# ) -> None:
|
||||
# """Build a thin derived image that copies the workspace into
|
||||
# the plan's guest path and sets the plan's workdir."""
|
||||
# import os
|
||||
#
|
||||
# cwd = str(workspace.host_path)
|
||||
# if not os.path.isdir(cwd):
|
||||
# die(f"cwd not found at {cwd}")
|
||||
# info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
|
||||
# with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
|
||||
# context_dir = os.path.join(tmp, "context")
|
||||
# staged_workspace = os.path.join(context_dir, "workspace")
|
||||
# shutil.copytree(
|
||||
# cwd,
|
||||
# staged_workspace,
|
||||
# symlinks=True,
|
||||
# ignore=shutil.ignore_patterns(".git"),
|
||||
# )
|
||||
# dockerfile = (
|
||||
# f"FROM {base}\n"
|
||||
# f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
|
||||
# f"WORKDIR {workspace.workdir}\n"
|
||||
# )
|
||||
# subprocess.run(
|
||||
# ["docker", "build", "-t", derived, "-f", "-", context_dir],
|
||||
# input=dockerfile,
|
||||
# text=True,
|
||||
# check=True,
|
||||
# )
|
||||
_TRUST_DIALOG_NODE_SCRIPT = (
|
||||
'const fs=require("fs"),p=process.env.HOME+"/.claude.json",'
|
||||
'c=JSON.parse(fs.readFileSync(p,"utf8"));'
|
||||
'c.projects=c.projects||{};'
|
||||
'c.projects[process.env.HOME+"/workspace"]={hasTrustDialogAccepted:true};'
|
||||
'fs.writeFileSync(p,JSON.stringify(c,null,2));'
|
||||
)
|
||||
|
||||
|
||||
def build_image_with_cwd(derived: str, base: str, cwd: str) -> None:
|
||||
"""Build a thin derived image that copies <cwd> into
|
||||
/home/node/workspace and adds a trust-dialog entry for it."""
|
||||
import os
|
||||
|
||||
if not os.path.isdir(cwd):
|
||||
die(f"cwd not found at {cwd}")
|
||||
info(f"building image {derived} from {base} with {cwd} -> /home/node/workspace")
|
||||
dockerfile = (
|
||||
f"FROM {base}\n"
|
||||
f"COPY --chown=node:node . /home/node/workspace\n"
|
||||
f"RUN node -e '{_TRUST_DIALOG_NODE_SCRIPT}'\n"
|
||||
f"WORKDIR /home/node/workspace\n"
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "build", "-t", derived, "-f", "-", cwd],
|
||||
input=dockerfile,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
"""macOS Apple Container backend.
|
||||
|
||||
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
|
||||
the Apple `container` CLI integration; launch remains gated until the
|
||||
sidecar network enforcement shape is implemented.
|
||||
"""
|
||||
|
||||
from .backend import MacosContainerBottleBackend
|
||||
|
||||
__all__ = ["MacosContainerBottleBackend"]
|
||||
@@ -1,84 +0,0 @@
|
||||
"""MacosContainerBottleBackend — Apple Container implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from . import util as _container
|
||||
from .bottle import MacosContainerBottle
|
||||
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
|
||||
from .bottle_plan import MacosContainerBottlePlan
|
||||
|
||||
|
||||
class MacosContainerBottleBackend(
|
||||
BottleBackend["MacosContainerBottlePlan", "MacosContainerBottleCleanupPlan"]
|
||||
):
|
||||
"""Apple Container backend. Selected by
|
||||
`BOT_BOTTLE_BACKEND=macos-container` or
|
||||
`--backend=macos-container`."""
|
||||
|
||||
name = "macos-container"
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> bool:
|
||||
return _container.is_available()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return _resolve_plan.build_guest_env(resolved_env)
|
||||
|
||||
def _resolve_plan(
|
||||
self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
) -> MacosContainerBottlePlan:
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
self, plan: MacosContainerBottlePlan
|
||||
) -> Generator[MacosContainerBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
def cleanup(self, plan: MacosContainerBottleCleanupPlan) -> None:
|
||||
_cleanup.cleanup(plan)
|
||||
|
||||
def enumerate_active(self) -> Sequence[ActiveAgent]:
|
||||
return _enumerate.enumerate_active()
|
||||
|
||||
def supervise_mcp_url(self, plan: MacosContainerBottlePlan) -> str:
|
||||
return plan.agent_supervise_url
|
||||
@@ -1,91 +0,0 @@
|
||||
"""Bottle handle for Apple's `container` CLI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from typing import Callable, cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
|
||||
|
||||
class MacosContainerBottle(Bottle):
|
||||
def __init__(
|
||||
self,
|
||||
container: str,
|
||||
teardown: Callable[[], None],
|
||||
prompt_path_in_container: str | None,
|
||||
*,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
):
|
||||
self.name = container
|
||||
self._teardown = teardown
|
||||
self.prompt_path = prompt_path_in_container
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
self._closed = False
|
||||
|
||||
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
|
||||
full_argv = list(argv)
|
||||
full_argv.extend(
|
||||
prompt_args(
|
||||
cast(PromptMode, self._agent_prompt_mode),
|
||||
self.prompt_path,
|
||||
argv=full_argv,
|
||||
)
|
||||
)
|
||||
cmd = ["container", "exec"]
|
||||
if tty:
|
||||
cmd.extend(["--interactive", "--tty"])
|
||||
if self.agent_workdir and self.agent_workdir != "/home/node":
|
||||
cmd.extend(["--workdir", self.agent_workdir])
|
||||
cmd.extend([self.name, self.agent_command, *full_argv])
|
||||
return cmd
|
||||
|
||||
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = (
|
||||
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
|
||||
if tty else None
|
||||
)
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
return subprocess.run(["sh", "-lc", script], check=False).returncode
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
result = subprocess.run(
|
||||
["container", "exec", "--user", user, "--interactive",
|
||||
self.name, "sh", "-s"],
|
||||
input=script,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return ExecResult(
|
||||
returncode=result.returncode,
|
||||
stdout=result.stdout,
|
||||
stderr=result.stderr,
|
||||
)
|
||||
|
||||
def cp_in(self, host_path: str, container_path: str) -> None:
|
||||
subprocess.run(
|
||||
["container", "cp", host_path, f"{self.name}:{container_path}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
def close(self) -> None:
|
||||
if self._closed:
|
||||
return
|
||||
self._closed = True
|
||||
self._teardown()
|
||||
@@ -1,27 +0,0 @@
|
||||
"""Cleanup plan for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from ...log import info
|
||||
from .. import BottleCleanupPlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
|
||||
containers: tuple[str, ...] = ()
|
||||
networks: tuple[str, ...] = ()
|
||||
|
||||
def print(self) -> None:
|
||||
if not self.containers and not self.networks:
|
||||
info("macos-container cleanup: nothing to remove")
|
||||
return
|
||||
for name in self.containers:
|
||||
info(f"macos-container container: {name}")
|
||||
for name in self.networks:
|
||||
info(f"macos-container network: {name}")
|
||||
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not self.containers and not self.networks
|
||||
@@ -1,58 +0,0 @@
|
||||
"""Plan type for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from .. import BottlePlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MacosContainerBottlePlan(BottlePlan):
|
||||
slug: str
|
||||
forwarded_env: dict[str, str] = field(repr=False)
|
||||
agent_proxy_url: str = ""
|
||||
agent_git_gate_url: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
|
||||
@property
|
||||
def container_name(self) -> str:
|
||||
return self.agent_provision.instance_name
|
||||
|
||||
@property
|
||||
def image(self) -> str:
|
||||
return self.agent_provision.image
|
||||
|
||||
@property
|
||||
def dockerfile_path(self) -> str:
|
||||
return self.agent_provision.dockerfile
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
if self.agent_git_gate_url.startswith("http://"):
|
||||
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
|
||||
return super().git_gate_insteadof_host
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
if self.agent_git_gate_url.startswith("http://"):
|
||||
return "http"
|
||||
return super().git_gate_insteadof_scheme
|
||||
@@ -1,70 +0,0 @@
|
||||
"""Cleanup for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...log import info, warn
|
||||
from . import util as container_mod
|
||||
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
|
||||
|
||||
_PREFIX = "bot-bottle-"
|
||||
_BUNDLE_PREFIX = "bot-bottle-sidecars-"
|
||||
|
||||
|
||||
def _list_prefixed_containers() -> list[str]:
|
||||
result = subprocess.run(
|
||||
["container", "list", "--all", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
warn(f"container list failed: {result.stderr.strip()}")
|
||||
return []
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX) or name.startswith(_BUNDLE_PREFIX)
|
||||
)
|
||||
|
||||
|
||||
def _list_prefixed_networks() -> list[str]:
|
||||
result = subprocess.run(
|
||||
["container", "network", "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX)
|
||||
)
|
||||
|
||||
|
||||
def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
|
||||
container_mod.require_container()
|
||||
return MacosContainerBottleCleanupPlan(
|
||||
containers=tuple(_list_prefixed_containers()),
|
||||
networks=tuple(_list_prefixed_networks()),
|
||||
)
|
||||
|
||||
|
||||
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
|
||||
for name in plan.containers:
|
||||
info(f"container delete --force {name}")
|
||||
subprocess.run(
|
||||
["container", "delete", "--force", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
for name in plan.networks:
|
||||
info(f"container network delete {name}")
|
||||
subprocess.run(
|
||||
["container", "network", "delete", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
@@ -1,40 +0,0 @@
|
||||
"""Active-agent enumeration for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...bottle_state import read_metadata
|
||||
from .. import ActiveAgent
|
||||
|
||||
_PREFIX = "bot-bottle-"
|
||||
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
result = subprocess.run(
|
||||
["container", "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
out: list[ActiveAgent] = []
|
||||
for name in sorted(line.strip() for line in result.stdout.splitlines()):
|
||||
if not name.startswith(_PREFIX):
|
||||
continue
|
||||
if name.startswith(_SIDECAR_PREFIX):
|
||||
continue
|
||||
slug = name[len(_PREFIX):]
|
||||
metadata = read_metadata(slug)
|
||||
out.append(ActiveAgent(
|
||||
backend_name="macos-container",
|
||||
slug=slug,
|
||||
agent_name=metadata.agent_name if metadata else "?",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
services=(),
|
||||
label=metadata.label if metadata else "",
|
||||
color=metadata.color if metadata else "",
|
||||
))
|
||||
return out
|
||||
@@ -1,426 +0,0 @@
|
||||
"""Launch flow for the macOS Apple Container backend.
|
||||
|
||||
This backend keeps the explicit proxy-env enforcement model for v1:
|
||||
the agent container is attached only to a host-only Apple Container
|
||||
network, while the sidecar bundle is attached to a NAT network first
|
||||
and the host-only network second. The sidecar's host-only IP is
|
||||
discovered from `container inspect` and stamped into the agent's
|
||||
HTTP_PROXY / HTTPS_PROXY env vars.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...bottle_state import egress_state_dir, git_gate_state_dir
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import die, info, warn
|
||||
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
|
||||
from ..docker.git_gate import (
|
||||
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from ..docker.sidecar_bundle import (
|
||||
SIDECAR_BUNDLE_DOCKERFILE,
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
)
|
||||
from ..docker.egress import egress_tls_init
|
||||
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
||||
from . import util as container_mod
|
||||
from .bottle import MacosContainerBottle
|
||||
from .bottle_plan import MacosContainerBottlePlan
|
||||
|
||||
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
_AGENT_SLEEP_SECONDS = "2147483647"
|
||||
_GIT_HTTP_PORT = 9420
|
||||
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
|
||||
|
||||
|
||||
def internal_network_name(slug: str) -> str:
|
||||
return f"bot-bottle-net-{slug}"
|
||||
|
||||
|
||||
def egress_network_name(slug: str) -> str:
|
||||
return f"bot-bottle-egress-{slug}"
|
||||
|
||||
|
||||
def sidecar_container_name(slug: str) -> str:
|
||||
return f"bot-bottle-sidecars-{slug}"
|
||||
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
plan: MacosContainerBottlePlan,
|
||||
*,
|
||||
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
|
||||
) -> Generator[MacosContainerBottle, None, None]:
|
||||
"""Build, run, provision, and yield an Apple Container bottle."""
|
||||
stack = ExitStack()
|
||||
bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||
|
||||
def teardown() -> None:
|
||||
teardown_exc: BaseException | None = None
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc: # noqa: W0718 - teardown must continue
|
||||
teardown_exc = exc
|
||||
warn(f"macos-container teardown failed: {exc!r}")
|
||||
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
|
||||
try:
|
||||
plan = _mint_certs(plan)
|
||||
_build_images(plan)
|
||||
|
||||
internal_network = internal_network_name(plan.slug)
|
||||
egress_network = egress_network_name(plan.slug)
|
||||
_create_networks(internal_network, egress_network, stack)
|
||||
|
||||
sidecar_name = sidecar_container_name(plan.slug)
|
||||
container_mod.force_remove_container(sidecar_name)
|
||||
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
|
||||
stack.callback(container_mod.force_remove_container, sidecar_name)
|
||||
_stage_git_gate(plan, sidecar_name)
|
||||
|
||||
sidecar_ip = container_mod.container_ipv4_on_network(
|
||||
sidecar_name, internal_network,
|
||||
)
|
||||
plan = _stamp_agent_urls(plan, sidecar_ip)
|
||||
|
||||
container_mod.force_remove_container(plan.container_name)
|
||||
_start_agent(plan, internal_network, sidecar_ip)
|
||||
stack.callback(container_mod.force_remove_container, plan.container_name)
|
||||
|
||||
bottle = MacosContainerBottle(
|
||||
plan.container_name,
|
||||
teardown,
|
||||
None,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=plan.spec.label or plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
yield bottle
|
||||
finally:
|
||||
teardown()
|
||||
|
||||
|
||||
def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
egress_state_dir(plan.slug),
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
plan.egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
)
|
||||
return dataclasses.replace(plan, egress_plan=egress_plan)
|
||||
|
||||
|
||||
def _build_images(plan: MacosContainerBottlePlan) -> None:
|
||||
container_mod.build_image(
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
_REPO_DIR,
|
||||
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
|
||||
)
|
||||
container_mod.build_image(
|
||||
plan.image,
|
||||
_REPO_DIR,
|
||||
dockerfile=plan.dockerfile_path,
|
||||
)
|
||||
|
||||
|
||||
def _create_networks(
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
stack: ExitStack,
|
||||
) -> None:
|
||||
container_mod.create_network(internal_network, internal=True)
|
||||
stack.callback(container_mod.remove_network, internal_network)
|
||||
container_mod.create_network(egress_network)
|
||||
stack.callback(container_mod.remove_network, egress_network)
|
||||
|
||||
|
||||
def _start_sidecar_bundle(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_name: str,
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
) -> None:
|
||||
argv = _sidecar_run_argv(plan, sidecar_name, internal_network, egress_network)
|
||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, effective_env,
|
||||
)
|
||||
env = {**os.environ, **token_values}
|
||||
info(f"container run sidecar bundle {sidecar_name}")
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True, env=env, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container run for sidecar bundle {sidecar_name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _start_agent(
|
||||
plan: MacosContainerBottlePlan,
|
||||
internal_network: str,
|
||||
sidecar_ip: str,
|
||||
) -> None:
|
||||
argv = _agent_run_argv(plan, internal_network, sidecar_ip)
|
||||
env = {
|
||||
**os.environ,
|
||||
**plan.forwarded_env,
|
||||
}
|
||||
info(f"container run agent {plan.container_name}")
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True, env=env, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container run for agent {plan.container_name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _stamp_agent_urls(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_ip: str,
|
||||
) -> MacosContainerBottlePlan:
|
||||
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
|
||||
supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_url = f"http://{sidecar_ip}:{SUPERVISE_PORT}/"
|
||||
git_gate_url = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_url = f"http://{sidecar_ip}:{_GIT_HTTP_PORT}"
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
agent_proxy_url=proxy_url,
|
||||
agent_git_gate_url=git_gate_url,
|
||||
agent_supervise_url=supervise_url,
|
||||
)
|
||||
|
||||
|
||||
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
|
||||
gp = plan.git_gate_plan
|
||||
if not gp.upstreams:
|
||||
return
|
||||
|
||||
container_mod.exec_container(
|
||||
sidecar_name,
|
||||
[
|
||||
"mkdir",
|
||||
"-p",
|
||||
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
"/git",
|
||||
str(Path(_GIT_GATE_READY_FILE).parent),
|
||||
],
|
||||
)
|
||||
|
||||
for host_path, container_path in _git_gate_files(plan):
|
||||
container_mod.copy_into_container(
|
||||
sidecar_name, host_path, container_path,
|
||||
)
|
||||
|
||||
container_mod.exec_container(
|
||||
sidecar_name,
|
||||
[
|
||||
"sh",
|
||||
"-c",
|
||||
"chmod 755 "
|
||||
f"{GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
|
||||
f"{GIT_GATE_HOOK_IN_CONTAINER} "
|
||||
f"{GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
|
||||
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
|
||||
f"touch {_GIT_GATE_READY_FILE}",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def _git_gate_files(
|
||||
plan: MacosContainerBottlePlan,
|
||||
) -> tuple[tuple[str, str], ...]:
|
||||
gp = plan.git_gate_plan
|
||||
files: list[tuple[str, str]] = [
|
||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
|
||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
|
||||
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
|
||||
]
|
||||
for upstream in gp.upstreams:
|
||||
files.append((
|
||||
expand_tilde(upstream.identity_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key",
|
||||
))
|
||||
if upstream.known_hosts_file:
|
||||
files.append((
|
||||
str(upstream.known_hosts_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts",
|
||||
))
|
||||
return tuple(files)
|
||||
|
||||
|
||||
def _sidecar_run_argv(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_name: str,
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
) -> list[str]:
|
||||
argv = [
|
||||
"container", "run",
|
||||
"--name", sidecar_name,
|
||||
"--detach",
|
||||
"--rm",
|
||||
"--network", egress_network,
|
||||
"--network", internal_network,
|
||||
"--dns", _sidecar_dns(),
|
||||
"--env", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}",
|
||||
]
|
||||
for entry in _sidecar_env_entries(plan):
|
||||
argv += ["--env", entry]
|
||||
for host_path, container_path, read_only in _sidecar_mounts(plan):
|
||||
argv += ["--mount", _mount_spec(host_path, container_path, read_only)]
|
||||
argv.append(SIDECAR_BUNDLE_IMAGE)
|
||||
return argv
|
||||
|
||||
|
||||
def _agent_run_argv(
|
||||
plan: MacosContainerBottlePlan,
|
||||
internal_network: str,
|
||||
sidecar_ip: str,
|
||||
) -> list[str]:
|
||||
argv = [
|
||||
"container", "run",
|
||||
"--name", plan.container_name,
|
||||
"--detach",
|
||||
"--rm",
|
||||
"--network", internal_network,
|
||||
]
|
||||
for entry in _agent_env_entries(plan, sidecar_ip):
|
||||
argv += ["--env", entry]
|
||||
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
|
||||
return argv
|
||||
|
||||
|
||||
def _sidecar_dns() -> str:
|
||||
return container_mod.dns_server()
|
||||
|
||||
|
||||
def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
|
||||
daemons = ["egress"]
|
||||
if plan.git_gate_plan.upstreams:
|
||||
daemons += ["git-gate", "git-http"]
|
||||
if plan.supervise_plan is not None:
|
||||
daemons.append("supervise")
|
||||
return tuple(daemons)
|
||||
|
||||
|
||||
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
|
||||
env: list[str] = []
|
||||
if plan.egress_plan.routes:
|
||||
env.extend(sorted(plan.egress_plan.token_env_map.keys()))
|
||||
if plan.git_gate_plan.upstreams:
|
||||
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
|
||||
if plan.supervise_plan is not None:
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
return tuple(env)
|
||||
|
||||
|
||||
def _sidecar_mounts(
|
||||
plan: MacosContainerBottlePlan,
|
||||
) -> tuple[tuple[str, str, bool], ...]:
|
||||
mounts: list[tuple[str, str, bool]] = []
|
||||
|
||||
ep = plan.egress_plan
|
||||
mounts.append((
|
||||
str(ep.mitmproxy_ca_host_path.parent),
|
||||
str(Path(EGRESS_CA_IN_CONTAINER).parent),
|
||||
False,
|
||||
))
|
||||
if ep.routes:
|
||||
mounts.append((
|
||||
str(_stage_routes_dir(plan)),
|
||||
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent),
|
||||
True,
|
||||
))
|
||||
|
||||
sp = plan.supervise_plan
|
||||
if sp is not None:
|
||||
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
|
||||
|
||||
return tuple(mounts)
|
||||
|
||||
|
||||
def _stage_routes_dir(plan: MacosContainerBottlePlan) -> Path:
|
||||
routes_dir = plan.stage_dir / "macos-container-egress"
|
||||
routes_dir.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(
|
||||
plan.egress_plan.routes_path,
|
||||
routes_dir / Path(EGRESS_ROUTES_IN_CONTAINER).name,
|
||||
)
|
||||
return routes_dir
|
||||
|
||||
|
||||
def _mount_spec(host_path: str, container_path: str, read_only: bool) -> str:
|
||||
spec = f"type=bind,source={host_path},target={container_path}"
|
||||
if read_only:
|
||||
spec += ",readonly"
|
||||
return spec
|
||||
|
||||
|
||||
def _agent_env_entries(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_ip: str,
|
||||
) -> tuple[str, ...]:
|
||||
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
|
||||
no_proxy = _agent_no_proxy(plan, sidecar_ip)
|
||||
env = [
|
||||
f"HTTPS_PROXY={proxy_url}",
|
||||
f"HTTP_PROXY={proxy_url}",
|
||||
f"https_proxy={proxy_url}",
|
||||
f"http_proxy={proxy_url}",
|
||||
f"NO_PROXY={no_proxy}",
|
||||
f"no_proxy={no_proxy}",
|
||||
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
|
||||
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
|
||||
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
|
||||
]
|
||||
if plan.agent_git_gate_url:
|
||||
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
|
||||
if plan.agent_supervise_url:
|
||||
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
|
||||
for name, value in sorted(plan.agent_provision.guest_env.items()):
|
||||
env.append(f"{name}={value}")
|
||||
for name in sorted(plan.forwarded_env.keys()):
|
||||
env.append(name)
|
||||
return tuple(env)
|
||||
|
||||
|
||||
def _agent_no_proxy(plan: MacosContainerBottlePlan, sidecar_ip: str) -> str:
|
||||
hosts = ["localhost", "127.0.0.1", sidecar_ip]
|
||||
return ",".join(hosts)
|
||||
@@ -1,44 +0,0 @@
|
||||
"""Prepare step for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import BottleSpec
|
||||
from . import util as container_mod
|
||||
from .bottle_plan import MacosContainerBottlePlan
|
||||
|
||||
|
||||
def preflight() -> None:
|
||||
container_mod.require_container()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return dict(resolved_env.literals)
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> MacosContainerBottlePlan:
|
||||
return MacosContainerBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
forwarded_env=dict(resolved_env.forwarded),
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -1,388 +0,0 @@
|
||||
"""Host-side primitives for Apple's `container` CLI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import ipaddress
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from typing import Iterable
|
||||
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
_CONTAINER = "container"
|
||||
_DEFAULT_DNS = "1.1.1.1"
|
||||
|
||||
|
||||
def is_macos() -> bool:
|
||||
return platform.system() == "Darwin"
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
return is_macos() and shutil.which(_CONTAINER) is not None
|
||||
|
||||
|
||||
def require_container() -> None:
|
||||
"""Fail with an install pointer if Apple Container is unavailable."""
|
||||
if not is_macos():
|
||||
info("BOT_BOTTLE_BACKEND=macos-container requires macOS.")
|
||||
die("macos-container backend is only supported on macOS")
|
||||
if shutil.which(_CONTAINER) is None:
|
||||
info("Apple Container is required but was not found on PATH.")
|
||||
info("Install: https://github.com/apple/container/releases")
|
||||
die("container not found")
|
||||
_require_container_service()
|
||||
|
||||
|
||||
def _require_container_service() -> None:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "system", "status"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
info("Apple Container system service is not running.")
|
||||
info("Start it with: container system start")
|
||||
die("container system service not running")
|
||||
|
||||
|
||||
def dns_server() -> str:
|
||||
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
|
||||
if override:
|
||||
return override
|
||||
return _host_ipv4_dns() or _DEFAULT_DNS
|
||||
|
||||
|
||||
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
||||
"""Build an OCI image with Apple's BuildKit-backed `container build`."""
|
||||
info(
|
||||
f"building image {ref} from {context} with Apple Container "
|
||||
"(layer cache keeps repeat builds fast)"
|
||||
)
|
||||
_ensure_builder_dns()
|
||||
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
|
||||
if dockerfile:
|
||||
args.extend(["-f", dockerfile])
|
||||
args.append(context)
|
||||
subprocess.run(args, check=True)
|
||||
|
||||
|
||||
def _ensure_builder_dns() -> None:
|
||||
dns = dns_server()
|
||||
status = _builder_status()
|
||||
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
|
||||
if _builder_running(status) and _builder_resolves_build_hosts():
|
||||
if override and not _builder_has_dns(status, dns):
|
||||
_restart_builder_with_dns(dns)
|
||||
return
|
||||
_restart_builder_with_dns(dns)
|
||||
|
||||
|
||||
def _restart_builder_with_dns(dns: str) -> None:
|
||||
subprocess.run(
|
||||
[_CONTAINER, "builder", "stop"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
subprocess.run(
|
||||
[_CONTAINER, "builder", "start", "--dns", dns],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def _host_ipv4_dns() -> str:
|
||||
if not is_macos():
|
||||
return ""
|
||||
result = subprocess.run(
|
||||
["scutil", "--dns"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return ""
|
||||
blocks: list[list[str]] = []
|
||||
current: list[str] = []
|
||||
for line in result.stdout.splitlines():
|
||||
if line.startswith("resolver #") and current:
|
||||
blocks.append(current)
|
||||
current = []
|
||||
current.append(line)
|
||||
if current:
|
||||
blocks.append(current)
|
||||
for direct_only in (True, False):
|
||||
for block in blocks:
|
||||
text = "\n".join(block)
|
||||
if direct_only and "Directly Reachable Address" not in text:
|
||||
continue
|
||||
for line in block:
|
||||
if "nameserver[" not in line or ":" not in line:
|
||||
continue
|
||||
candidate = line.split(":", 1)[1].strip()
|
||||
if _usable_ipv4(candidate):
|
||||
return candidate
|
||||
return ""
|
||||
|
||||
|
||||
def _usable_ipv4(value: str) -> bool:
|
||||
try:
|
||||
address = ipaddress.ip_address(value)
|
||||
except ValueError:
|
||||
return False
|
||||
return (
|
||||
address.version == 4
|
||||
and not address.is_loopback
|
||||
and not address.is_link_local
|
||||
and not address.is_multicast
|
||||
and not address.is_unspecified
|
||||
)
|
||||
|
||||
|
||||
def _builder_status() -> list[dict[str, object]]:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "builder", "status", "--format", "json"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
try:
|
||||
data = json.loads(result.stdout or "[]")
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
if isinstance(data, list):
|
||||
return [entry for entry in data if isinstance(entry, dict)]
|
||||
if isinstance(data, dict):
|
||||
return [data]
|
||||
return []
|
||||
|
||||
|
||||
def _builder_running(status: list[dict[str, object]]) -> bool:
|
||||
for entry in status:
|
||||
entry_status = entry.get("status")
|
||||
if isinstance(entry_status, dict) and entry_status.get("state") == "running":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _builder_dns_nameservers(status: list[dict[str, object]]) -> list[str]:
|
||||
out: list[str] = []
|
||||
for entry in status:
|
||||
config = entry.get("configuration")
|
||||
config_dns = config.get("dns") if isinstance(config, dict) else None
|
||||
nameservers = (
|
||||
config_dns.get("nameservers")
|
||||
if isinstance(config_dns, dict)
|
||||
else None
|
||||
)
|
||||
if not isinstance(nameservers, list):
|
||||
continue
|
||||
out.extend(name for name in nameservers if isinstance(name, str))
|
||||
return out
|
||||
|
||||
|
||||
def _builder_has_dns(status: list[dict[str, object]], dns: str) -> bool:
|
||||
return dns in _builder_dns_nameservers(status)
|
||||
|
||||
|
||||
def _builder_resolves_build_hosts() -> bool:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "exec", "buildkit", "getent", "hosts", "deb.debian.org"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def image_exists(ref: str) -> bool:
|
||||
return _silent_run([_CONTAINER, "image", "inspect", ref]) == 0
|
||||
|
||||
|
||||
def container_exists(name: str) -> bool:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "list", "--all", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
return name in {line.strip() for line in result.stdout.splitlines()}
|
||||
|
||||
|
||||
def force_remove_container(name: str) -> None:
|
||||
if container_exists(name):
|
||||
subprocess.run(
|
||||
[_CONTAINER, "delete", "--force", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def copy_into_container(name: str, host_path: str, container_path: str) -> None:
|
||||
cmd = [_CONTAINER, "cp", host_path, f"{name}:{container_path}"]
|
||||
result = _run_container_op(cmd)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container cp into {name}:{container_path} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def exec_container(name: str, argv: list[str]) -> None:
|
||||
result = _run_container_op([_CONTAINER, "exec", name, *argv])
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container exec in {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]:
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
for _ in range(19):
|
||||
if result.returncode == 0:
|
||||
return result
|
||||
time.sleep(0.1)
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def create_network(name: str, *, internal: bool = False) -> None:
|
||||
args = [
|
||||
_CONTAINER, "network", "create",
|
||||
"--label", "bot-bottle.backend=macos-container",
|
||||
]
|
||||
if internal:
|
||||
args.append("--internal")
|
||||
args.append(name)
|
||||
result = subprocess.run(
|
||||
args, capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return
|
||||
if "already exists" in (result.stderr or "").lower():
|
||||
return
|
||||
die(
|
||||
f"container network create {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def remove_network(name: str) -> None:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "network", "delete", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return
|
||||
|
||||
|
||||
def inspect_container(name: str) -> dict[str, object]:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "inspect", name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container inspect {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
try:
|
||||
data = json.loads(result.stdout or "[]")
|
||||
except json.JSONDecodeError as exc:
|
||||
die(f"container inspect {name} returned malformed JSON: {exc}")
|
||||
if isinstance(data, list) and data and isinstance(data[0], dict):
|
||||
return data[0]
|
||||
if isinstance(data, dict):
|
||||
return data
|
||||
die(f"container inspect {name} returned an unexpected shape")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def container_ipv4_on_network(name: str, network: str) -> str:
|
||||
data = inspect_container(name)
|
||||
status = data.get("status")
|
||||
networks = status.get("networks") if isinstance(status, dict) else None
|
||||
if not isinstance(networks, list):
|
||||
die(f"container inspect {name} did not include status.networks")
|
||||
for entry in networks:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
if entry.get("network") != network:
|
||||
continue
|
||||
raw = entry.get("ipv4Address")
|
||||
if not isinstance(raw, str) or not raw:
|
||||
die(f"container {name} has no IPv4 address on {network}")
|
||||
return raw.split("/", 1)[0]
|
||||
die(f"container {name} is not attached to network {network}")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
"""Return the image digest/ID from `container image inspect`.
|
||||
|
||||
The command returns JSON on current Apple Container releases. Keep
|
||||
parsing narrow and fatal so callers do not cache on an empty key.
|
||||
"""
|
||||
import json
|
||||
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "image", "inspect", ref],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container image inspect for {ref!r} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
try:
|
||||
data = json.loads(result.stdout or "{}")
|
||||
except json.JSONDecodeError as exc:
|
||||
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
|
||||
if isinstance(data, list) and data:
|
||||
data = data[0]
|
||||
if isinstance(data, dict):
|
||||
value = data.get("id") or data.get("digest") or data.get("ID")
|
||||
if value:
|
||||
return str(value)
|
||||
die(f"container image inspect for {ref!r} did not include an image id")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def save(ref: str, output: str) -> None:
|
||||
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
|
||||
|
||||
|
||||
def _silent_run(cmd: Iterable[str]) -> int:
|
||||
return subprocess.run(
|
||||
list(cmd),
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
).returncode
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import Sequence
|
||||
|
||||
from ..agent_provider import runtime_for
|
||||
from ..log import info
|
||||
|
||||
|
||||
@@ -29,13 +30,16 @@ def print_multi(label: str, values: Sequence[str]) -> None:
|
||||
|
||||
|
||||
def visible_agent_env_names(
|
||||
env_names: Sequence[str], *, hidden_env_names: frozenset[str],
|
||||
env_names: Sequence[str], *, agent_provider_template: str,
|
||||
) -> list[str]:
|
||||
"""Env names worth showing in launch summaries.
|
||||
|
||||
Provider-injected placeholder env vars are implementation details:
|
||||
they are non-secret dummy values that satisfy provider CLIs while
|
||||
egress injects the real Authorization header. The plan's
|
||||
`hidden_env_names` carries exactly which names to suppress.
|
||||
Provider auth placeholders (`OPENAI_API_KEY`,
|
||||
`CLAUDE_CODE_OAUTH_TOKEN`) are implementation details: they are
|
||||
non-secret dummy values that satisfy the provider CLI while egress
|
||||
injects the real upstream Authorization header. Showing them in
|
||||
preflight makes the operator think a real key is entering the
|
||||
agent, so hide only that provider-owned placeholder.
|
||||
"""
|
||||
return sorted({name for name in env_names if name and name not in hidden_env_names})
|
||||
hidden = {runtime_for(agent_provider_template).placeholder_env}
|
||||
return sorted({name for name in env_names if name not in hidden})
|
||||
|
||||
@@ -1,122 +0,0 @@
|
||||
"""Shared helpers used by both backends' resolve_plan steps.
|
||||
|
||||
Each helper owns one well-defined step of the per-bottle plan
|
||||
resolution so docker and smolmachines don't repeat the same logic.
|
||||
Backend-specific steps (container names, env-file, per-bottle
|
||||
Dockerfile overrides, subnet allocation) stay in the backend's own
|
||||
resolve_plan.py.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import replace
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from ..agent_provider import AgentProvisionPlan
|
||||
from ..bottle_state import (
|
||||
BottleMetadata,
|
||||
agent_state_dir,
|
||||
bottle_identity,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from ..egress import Egress, EgressPlan
|
||||
from ..git_gate import GitGate, GitGatePlan
|
||||
from ..manifest import ManifestBottle
|
||||
from ..supervise import Supervise, SupervisePlan
|
||||
from . import BottleSpec
|
||||
|
||||
|
||||
def mint_slug(spec: BottleSpec) -> str:
|
||||
"""Return the bottle identity: the recorded identity for a resume,
|
||||
or a freshly minted one for a new start."""
|
||||
return spec.identity or bottle_identity(spec.agent_name)
|
||||
|
||||
|
||||
def write_launch_metadata(
|
||||
slug: str, spec: BottleSpec, *, compose_project: str, backend: str,
|
||||
) -> None:
|
||||
"""Persist launch metadata so `cli.py resume <identity>` can
|
||||
reconstruct the spec. Idempotent — re-writes on resume with a
|
||||
refreshed started_at."""
|
||||
write_metadata(BottleMetadata(
|
||||
identity=slug,
|
||||
agent_name=spec.agent_name,
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
compose_project=compose_project,
|
||||
backend=backend,
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
))
|
||||
|
||||
|
||||
def prepare_agent_state_dir(slug: str, spec: BottleSpec) -> tuple[Path, Path]:
|
||||
"""Create the agent state subdir, write the prompt file.
|
||||
Returns (agent_dir, prompt_file)."""
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
prompt_file.write_text(agent.prompt or "")
|
||||
prompt_file.chmod(0o600)
|
||||
return agent_dir, prompt_file
|
||||
|
||||
|
||||
def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
return GitGate().prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
|
||||
def prepare_egress(
|
||||
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
|
||||
) -> EgressPlan:
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
|
||||
|
||||
|
||||
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
|
||||
"""Prepare the supervise sidecar state dir. Returns None when
|
||||
bottle.supervise is falsy."""
|
||||
if not bottle.supervise:
|
||||
return None
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
return Supervise().prepare(slug, supervise_dir)
|
||||
|
||||
|
||||
def merge_provision_env_vars(provision: AgentProvisionPlan) -> AgentProvisionPlan:
|
||||
"""Fold provision.env_vars into guest_env (setdefault semantics)
|
||||
and return a new plan with the merged guest_env."""
|
||||
merged = dict(provision.guest_env)
|
||||
for key, val in provision.env_vars.items():
|
||||
merged.setdefault(key, val)
|
||||
return replace(provision, guest_env=merged)
|
||||
|
||||
|
||||
def resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
|
||||
"""Resolve a manifest-supplied dockerfile path relative to user_cwd."""
|
||||
path = Path(os.path.expanduser(path_value))
|
||||
if not path.is_absolute():
|
||||
path = Path(spec.user_cwd) / path
|
||||
return str(path)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"merge_provision_env_vars",
|
||||
"mint_slug",
|
||||
"prepare_agent_state_dir",
|
||||
"prepare_egress",
|
||||
"prepare_git_gate",
|
||||
"prepare_supervise",
|
||||
"resolve_manifest_dockerfile",
|
||||
"write_launch_metadata",
|
||||
]
|
||||
@@ -1,11 +1,5 @@
|
||||
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
||||
BottleBackend (PRD 0023).
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
the declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||
smolmachines backend only owns the steps that are about backend
|
||||
infrastructure: CA install (no-op for now), workspace, git copy-in."""
|
||||
BottleBackend (PRD 0023)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -13,20 +7,20 @@ from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from . import prepare as _prepare
|
||||
from . import smolvm as _smolvm
|
||||
from .bottle import SmolmachinesBottle
|
||||
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .provision import ca as _ca
|
||||
from .provision import git as _git
|
||||
from .provision import prompt as _prompt
|
||||
from .provision import skills as _skills
|
||||
from .provision import supervise as _supervise
|
||||
|
||||
|
||||
class SmolmachinesBottleBackend(
|
||||
@@ -45,34 +39,10 @@ class SmolmachinesBottleBackend(
|
||||
runtime check happens at `prepare`."""
|
||||
return _smolvm.is_available()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return _resolve_plan.build_guest_env(resolved_env)
|
||||
|
||||
def _resolve_plan(
|
||||
self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
self, spec: BottleSpec, *, stage_dir: Path
|
||||
) -> SmolmachinesBottlePlan:
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
@@ -81,12 +51,30 @@ class SmolmachinesBottleBackend(
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
|
||||
"""The smolmachines guest reaches the supervise sidecar via a
|
||||
host-published random port the launch step pinned earlier
|
||||
(`http://<loopback_ip>:<random_port>/`). `agent_supervise_url`
|
||||
on the plan is "" when the bottle has no sidecar."""
|
||||
return plan.agent_supervise_url
|
||||
def provision_ca(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_ca.provision_ca(plan, target)
|
||||
|
||||
def provision_prompt(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> str | None:
|
||||
return _prompt.provision_prompt(plan, target)
|
||||
|
||||
def provision_skills(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_skills.provision_skills(plan, target)
|
||||
|
||||
def provision_git(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_git.provision_git(plan, target)
|
||||
|
||||
def provision_supervise(
|
||||
self, plan: SmolmachinesBottlePlan, target: str
|
||||
) -> None:
|
||||
_supervise.provision_supervise(plan, target)
|
||||
|
||||
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
@@ -19,13 +19,10 @@ from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import shlex
|
||||
from typing import Mapping, cast
|
||||
from typing import Mapping
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
from . import pty_resize as _pty_resize
|
||||
from . import smolvm as _smolvm
|
||||
|
||||
@@ -48,11 +45,19 @@ _HOME_FOR = {
|
||||
}
|
||||
|
||||
|
||||
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||
def _env_flags_for(user: str) -> list[str]:
|
||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||
out = [f"HOME={home}", f"USER={user}"]
|
||||
return ["-e", f"HOME={home}", "-e", f"USER={user}"]
|
||||
|
||||
|
||||
def _guest_env_flags(env: Mapping[str, str]) -> list[str]:
|
||||
"""Render `{K: V}` into a flat `-e K=V` argv slice for
|
||||
`smolvm machine exec`. `smolvm machine create -e` set env
|
||||
on PID 1 but it doesn't propagate to fresh exec process
|
||||
trees, so we have to re-pass them every call."""
|
||||
out: list[str] = []
|
||||
for k, v in env.items():
|
||||
out.append(f"{k}={v}")
|
||||
out += ["-e", f"{k}={v}"]
|
||||
return out
|
||||
|
||||
|
||||
@@ -70,16 +75,12 @@ class SmolmachinesBottle(Bottle):
|
||||
guest_env: Mapping[str, str] | None = None,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
) -> None:
|
||||
self.name = machine_name
|
||||
# In-VM path to the agent's prompt file. None when the
|
||||
# agent declared no prompt (file still exists; we just
|
||||
# don't pass --append-system-prompt-file).
|
||||
self.prompt_path = prompt_path
|
||||
self._prompt_path = prompt_path
|
||||
# Env vars the agent process needs (HTTPS_PROXY,
|
||||
# CLAUDE_CODE_OAUTH_TOKEN, manifest-declared bottle env, …).
|
||||
# Forwarded on every `smolvm machine exec` via `-e K=V`
|
||||
@@ -87,10 +88,9 @@ class SmolmachinesBottle(Bottle):
|
||||
self._guest_env = dict(guest_env or {})
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
self.agent_provider_template = (
|
||||
"codex" if agent_command == "codex" else "claude"
|
||||
)
|
||||
|
||||
def agent_argv(
|
||||
self, argv: list[str], *, tty: bool = True,
|
||||
@@ -98,18 +98,13 @@ class SmolmachinesBottle(Bottle):
|
||||
flags = ["smolvm", "machine", "exec", "--name", self.name]
|
||||
if tty:
|
||||
flags += ["-i", "-t"]
|
||||
agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
|
||||
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
|
||||
agent_tail += [
|
||||
"sh", "-lc",
|
||||
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
|
||||
"bot-bottle-agent",
|
||||
]
|
||||
agent_tail.append(self.agent_command)
|
||||
flags += _env_flags_for("node")
|
||||
flags += _guest_env_flags(self._guest_env)
|
||||
agent_tail = [self.agent_command]
|
||||
provider_prompt_args = prompt_args(
|
||||
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
|
||||
self._agent_prompt_mode, self._prompt_path, argv=argv,
|
||||
)
|
||||
if cast(PromptMode, self._agent_prompt_mode) == "read_prompt_file":
|
||||
if self._agent_prompt_mode == "read_prompt_file":
|
||||
agent_tail += argv
|
||||
agent_tail += provider_prompt_args
|
||||
else:
|
||||
@@ -141,16 +136,9 @@ class SmolmachinesBottle(Bottle):
|
||||
UID switches via `runuser -u node --` (not `-l`) so we
|
||||
avoid login-shell wiring. HOME / USER come from `smolvm
|
||||
-e` instead, which sets them on the process env."""
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
return subprocess.run(["sh", "-lc", script], check=False).returncode
|
||||
|
||||
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
|
||||
# early-VM provisioning. Retry once after a short settle so
|
||||
# callers (provision_ca, etc.) don't have to handle it themselves.
|
||||
_SIGKILL_EXIT = 128 + 9
|
||||
return subprocess.run(
|
||||
self.agent_argv(argv, tty=tty), check=False,
|
||||
).returncode
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
"""Run a POSIX shell script as `user` (default `node`) and
|
||||
@@ -160,24 +148,16 @@ class SmolmachinesBottle(Bottle):
|
||||
on both backends. Pass `user="root"` for tests that need
|
||||
root.
|
||||
|
||||
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
|
||||
without invoking a login shell, then sets HOME / USER and the
|
||||
bottle env in the child process.
|
||||
|
||||
Retries once on SIGKILL (exit 137) — libkrun occasionally
|
||||
kills short-lived execs during VM bring-up."""
|
||||
r = self._exec_raw(script, user=user)
|
||||
if r.returncode == self._SIGKILL_EXIT:
|
||||
time.sleep(1.0)
|
||||
r = self._exec_raw(script, user=user)
|
||||
return r
|
||||
|
||||
def _exec_raw(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
argv = [
|
||||
"--", "runuser", "-u", user, "--",
|
||||
"env", *_env_assignments_for(user, self._guest_env),
|
||||
"/bin/sh", "-c", script,
|
||||
]
|
||||
`runuser -u <user> -- /bin/sh -c <script>` switches UID
|
||||
without invoking a login shell; HOME / USER are set via
|
||||
`smolvm -e` (see `_env_flags_for`)."""
|
||||
argv = (
|
||||
_env_flags_for(user)
|
||||
+ _guest_env_flags(self._guest_env)
|
||||
+ ["--", "runuser", "-u", user, "--", "/bin/sh", "-c", script]
|
||||
)
|
||||
# _smolvm.machine_exec expects argv (the bit after `--`);
|
||||
# the -e flags go before, so call smolvm directly.
|
||||
r = subprocess.run(
|
||||
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
||||
capture_output=True, text=True, check=False,
|
||||
|
||||
@@ -8,19 +8,25 @@ in chunk 4."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from ...egress import EgressPlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...log import info
|
||||
from ...pipelock import PipelockProxyPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import BottlePlan
|
||||
from ..print_util import print_multi, visible_agent_env_names
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SmolmachinesBottlePlan(BottlePlan):
|
||||
"""Resolved fields the launch step needs to bring up the bottle.
|
||||
|
||||
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
|
||||
`supervise_plan`, and `agent_provision` from BottlePlan."""
|
||||
Inherits `spec` and `stage_dir` from BottlePlan."""
|
||||
|
||||
slug: str
|
||||
# Per-bottle docker subnet for the sidecar bundle container.
|
||||
@@ -29,6 +35,27 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
bundle_subnet: str
|
||||
bundle_gateway: str
|
||||
bundle_ip: str
|
||||
# smolvm machine name + agent image source. machine_create
|
||||
# boots from a packed `.smolmachine` artifact (pre-baked at
|
||||
# prepare time via `smolvm pack create`); using `--from`
|
||||
# instead of `--image` avoids the registry-pull race we hit
|
||||
# when machine_start tried to fetch on-demand and the libkrun
|
||||
# agent's network attempt got refused by macOS.
|
||||
#
|
||||
# Chunk 2d ships with a public placeholder image (alpine)
|
||||
# since bot-bottle-claude:latest lives in the operator's local
|
||||
# docker daemon and smolvm's crane backend can't read from
|
||||
# there; chunk 4 resolves the agent-image-conversion gap
|
||||
# (push to a registry first, or smolvm grows a docker-daemon
|
||||
# transport).
|
||||
machine_name: str
|
||||
# Agent image ref (docker tag). `launch` runs the
|
||||
# build → save → registry push → smolvm pack pipeline against
|
||||
# this and feeds the resulting `.smolmachine` artifact to
|
||||
# `machine_create --from`. The pipeline runs at launch time
|
||||
# (not prepare time) so the docker build output doesn't garble
|
||||
# the dashboard's preflight modal.
|
||||
agent_image_ref: str
|
||||
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
|
||||
# the guest has no DNS resolver inside the TSI allowlist.
|
||||
# Passed to `smolvm machine create` as `-e K=V` flags.
|
||||
@@ -36,7 +63,12 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
# `--smolfile` is mutually exclusive with `--from`, and
|
||||
# `--from` is the path that avoids the registry-pull race).
|
||||
guest_env: dict[str, str]
|
||||
# Inner Plans for the sidecar bundle daemons. The same shape the
|
||||
# Path to the agent's prompt file on the host. Always written
|
||||
# (mode 0o600) so the in-VM path always exists; the file is
|
||||
# empty when the agent has no prompt — claude-code reads it
|
||||
# via --append-system-prompt-file only when non-empty.
|
||||
prompt_file: Path
|
||||
# Inner Plans for the four bundle daemons. The same shape the
|
||||
# docker backend uses — same `.prepare()` calls produced
|
||||
# them — but our launch step doesn't populate the
|
||||
# docker-specific network fields (internal_network,
|
||||
@@ -44,6 +76,12 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
# docker's `--internal` + egress bridge topology; it's on a
|
||||
# per-bottle bridge with a pinned IP. The unused fields stay
|
||||
# at their dataclass defaults.
|
||||
proxy_plan: PipelockProxyPlan
|
||||
git_gate_plan: GitGatePlan
|
||||
egress_plan: EgressPlan
|
||||
# None when bottle.supervise is False, matching the docker
|
||||
# backend's convention.
|
||||
supervise_plan: SupervisePlan | None
|
||||
# Agent-side endpoints. On Docker Desktop the docker bridge
|
||||
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
||||
# networking; docker container IPs live in the daemon's VM),
|
||||
@@ -55,55 +93,43 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
agent_proxy_url: str = ""
|
||||
agent_git_gate_host: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
agent_command: str = "claude"
|
||||
agent_prompt_mode: PromptMode = "append_file"
|
||||
agent_provider_template: str = "claude"
|
||||
agent_dockerfile_path: str = ""
|
||||
|
||||
@property
|
||||
def machine_name(self) -> str:
|
||||
"""smolvm machine name. `machine_create` boots from a packed
|
||||
`.smolmachine` artifact (pre-baked at prepare time via
|
||||
`smolvm pack create`); using `--from` instead of `--image`
|
||||
avoids the registry-pull race we hit when machine_start tried
|
||||
to fetch on-demand and the libkrun agent's network attempt
|
||||
got refused by macOS."""
|
||||
return self.agent_provision.instance_name
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Compact y/N preflight. Same shape as the Docker
|
||||
backend's so operators see one format across backends."""
|
||||
del remote_control # not surfaced in the compact summary
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
|
||||
@property
|
||||
def agent_image(self) -> str:
|
||||
"""Agent image ref (docker tag). `launch` runs the
|
||||
build → save → registry push → smolvm pack pipeline against
|
||||
this and feeds the resulting `.smolmachine` artifact to
|
||||
`machine_create --from`. The pipeline runs at launch time
|
||||
(not prepare time) so the docker build output doesn't garble
|
||||
the dashboard's preflight modal."""
|
||||
return self.agent_provision.image
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(bottle.env.keys()),
|
||||
agent_provider_template=self.agent_provider_template,
|
||||
)
|
||||
upstreams = [
|
||||
f"{g.Name} → {g.Upstream}" for g in bottle.git
|
||||
]
|
||||
# Use the resolved egress_plan (lowercase `host` on the
|
||||
# plan-level EgressRoute) rather than `bottle.egress.routes`,
|
||||
# which is the manifest's capitalized-attr form.
|
||||
routes = [r.host for r in self.egress_plan.routes]
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
"""Path to the agent's prompt file on the host. Always written
|
||||
(mode 0o600) so the in-VM path always exists; the file is
|
||||
empty when the agent has no prompt — claude-code reads it
|
||||
via --append-system-prompt-file only when non-empty."""
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
return self.agent_git_gate_host
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
return "http"
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
|
||||
@property
|
||||
def agent_dockerfile_path(self) -> str:
|
||||
return self.agent_provision.dockerfile
|
||||
print(file=sys.stderr)
|
||||
info(f"agent : {spec.agent_name}")
|
||||
info(f"provider : {self.agent_provider_template}")
|
||||
print_multi("env ", env_names)
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
identity = manifest.git_identity_summary(spec.agent_name)
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
if upstreams:
|
||||
print_multi(" git gate ", upstreams)
|
||||
if routes:
|
||||
print_multi(" egress ", routes)
|
||||
print(file=sys.stderr)
|
||||
|
||||
@@ -23,7 +23,7 @@ import json
|
||||
import subprocess
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ...bottle_state import read_metadata
|
||||
from ..docker.bottle_state import read_metadata
|
||||
from . import sidecar_bundle as _bundle
|
||||
|
||||
|
||||
@@ -64,15 +64,13 @@ def enumerate_active() -> list[ActiveAgent]:
|
||||
agent_name=metadata.agent_name if metadata else "?",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
services=services_by_slug.get(slug, ()),
|
||||
label=metadata.label if metadata else "",
|
||||
color=metadata.color if metadata else "",
|
||||
))
|
||||
return out
|
||||
|
||||
|
||||
def _query_bundle_services() -> dict[str, tuple[str, ...]]:
|
||||
"""`{slug: ('egress', ...)}` from each running bundle container's
|
||||
`BOT_BOTTLE_SIDECAR_DAEMONS` env var.
|
||||
"""`{slug: ('egress', 'pipelock', ...)}` from each running
|
||||
bundle container's `BOT_BOTTLE_SIDECAR_DAEMONS` env var.
|
||||
Smolmachines bundles all run the PRD-0024 image with the
|
||||
same daemon set declared via env, so one inspect per bundle
|
||||
gets us the picture without exec'ing into the container.
|
||||
|
||||
@@ -9,27 +9,34 @@ guest pointed at the bundle's pinned IP via TSI's
|
||||
exit.
|
||||
|
||||
The bundle's daemons consume the inner Plans the docker backend
|
||||
already produces: egress reads routes + CAs from the EgressPlan.
|
||||
Git-gate + supervise plumb through the same plans the docker
|
||||
backend uses, minus the docker-network fields that don't apply here."""
|
||||
already produces: pipelock reads its yaml + CA from the
|
||||
PipelockProxyPlan; egress reads routes + CAs from the EgressPlan
|
||||
+ EGRESS_UPSTREAM_PROXY pointing at `127.0.0.1:8888` (bundle
|
||||
local), since the agent dials pipelock first (not egress) on the
|
||||
smolmachines path. Git-gate + supervise plumb through the same
|
||||
plans the docker backend uses, minus the docker-network fields
|
||||
that don't apply here."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import time
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...egress import (
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
egress_resolve_token_values,
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
|
||||
from ...pipelock import (
|
||||
PIPELOCK_CA_CERT_IN_CONTAINER,
|
||||
PIPELOCK_CA_KEY_IN_CONTAINER,
|
||||
)
|
||||
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker import util as docker_mod
|
||||
from ..docker.egress import (
|
||||
EGRESS_CA_IN_CONTAINER,
|
||||
EGRESS_PIPELOCK_CA_IN_CONTAINER,
|
||||
EGRESS_PORT as _EGRESS_PORT,
|
||||
egress_tls_init,
|
||||
)
|
||||
@@ -38,10 +45,13 @@ from ..docker.git_gate import (
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
GIT_GATE_PORT as _GIT_GATE_PORT,
|
||||
)
|
||||
from ..docker.pipelock import (
|
||||
BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
PIPELOCK_PORT as _PIPELOCK_PORT_STR,
|
||||
pipelock_tls_init,
|
||||
)
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import warn
|
||||
from ...bottle_state import egress_state_dir, git_gate_state_dir
|
||||
from . import loopback_alias as _loopback
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
@@ -64,8 +74,9 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
|
||||
# Container-internal listening ports for each bundle daemon. The
|
||||
# bundle publishes each one on a random host loopback port (see
|
||||
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks
|
||||
# them up post-start.
|
||||
_GIT_HTTP_PORT = 9420
|
||||
# them up post-start. Pipelock's port is an env-overridable string
|
||||
# in docker.pipelock; coerce to int here.
|
||||
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
|
||||
_SUPERVISE_PORT = SUPERVISE_PORT
|
||||
|
||||
|
||||
@@ -73,217 +84,217 @@ _SUPERVISE_PORT = SUPERVISE_PORT
|
||||
def launch(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
*,
|
||||
provision: Callable[[SmolmachinesBottlePlan, "SmolmachinesBottle"], str | None],
|
||||
provision: Callable[[SmolmachinesBottlePlan, str], str | None],
|
||||
) -> Generator[SmolmachinesBottle, None, None]:
|
||||
"""Build + run the bottle and yield a handle; tear everything
|
||||
down on exit. Errors during bringup unwind any partial state
|
||||
via the ExitStack."""
|
||||
stack = ExitStack()
|
||||
try:
|
||||
loopback_ip, network = _allocate_resources(plan, stack)
|
||||
plan = _mint_certs(plan)
|
||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
||||
plan = _discover_urls(plan, loopback_ip)
|
||||
# 1. Reserve a loopback alias for this bottle. macOS only
|
||||
# routes 127.0.0.1 by default; the per-bottle alias is
|
||||
# what bundles the docker port-publishes and TSI allowlist
|
||||
# against, so this bottle can't reach other bottles' (or
|
||||
# other host services') ports on the loopback. Lazy
|
||||
# sudo-driven on first use per boot. No-op on Linux.
|
||||
_loopback.ensure_pool()
|
||||
loopback_ip = _loopback.allocate(plan.slug)
|
||||
|
||||
# Build the agent image and pack it into a `.smolmachine`
|
||||
# artifact (or hit the per-Dockerfile-digest cache). Runs
|
||||
# here, not in prepare, so the docker-build output doesn't
|
||||
# garble the dashboard's preflight modal.
|
||||
# 2. Per-bottle docker bridge.
|
||||
network = _bundle.bundle_network_name(plan.slug)
|
||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||
stack.callback(_bundle.remove_bundle_network, network)
|
||||
|
||||
# 2. Mint per-bottle CAs and update the inner Plans with
|
||||
# their launch-time paths. pipelock always runs in the
|
||||
# bundle; egress's CA is only minted when the bottle
|
||||
# declares routes (otherwise egress runs idle without
|
||||
# MITM and the CA files would be unused).
|
||||
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
||||
proxy_plan = dataclasses.replace(
|
||||
plan.proxy_plan,
|
||||
ca_cert_host_path=ca_cert_host,
|
||||
ca_key_host_path=ca_key_host,
|
||||
)
|
||||
egress_plan = plan.egress_plan
|
||||
if egress_plan.routes:
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
plan.egress_plan.routes_path.parent,
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
pipelock_ca_host_path=ca_cert_host,
|
||||
# On smolmachines, egress's upstream is pipelock
|
||||
# on the bundle's localhost — they're in the same
|
||||
# container's network namespace.
|
||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
||||
)
|
||||
plan = dataclasses.replace(
|
||||
plan, proxy_plan=proxy_plan, egress_plan=egress_plan,
|
||||
)
|
||||
|
||||
# 3. Build the BundleLaunchSpec from the (now-resolved)
|
||||
# inner Plans: daemon subset, env, bind-mounts, and the
|
||||
# loopback alias to bind published ports against. The
|
||||
# spec's ports_to_publish list expands depending on which
|
||||
# daemons the agent needs to reach from the smolvm guest.
|
||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||
token_env = _resolve_token_env(plan, os.environ)
|
||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||
|
||||
# 4. Discover the host-side ports docker assigned for the
|
||||
# bundle's published container ports, and bind the
|
||||
# agent's URLs to `<loopback_ip>:<host port>`. Docker
|
||||
# container IPs (192.168.x.x in the daemon's bridge)
|
||||
# aren't reachable from the smolvm guest on macOS — TSI
|
||||
# uses macOS networking, and macOS sees the daemon's
|
||||
# bridge via the published-port loopback forward only.
|
||||
#
|
||||
# Proxy hop order matches the docker backend: when the
|
||||
# bottle declares egress routes, the agent's first hop is
|
||||
# egress (for token injection), then pipelock. Without
|
||||
# routes, the agent dials pipelock directly. Whichever
|
||||
# one is "agent-facing" is the daemon whose port we
|
||||
# publish on host loopback; the other stays bundle-
|
||||
# internal as the upstream proxy.
|
||||
if plan.egress_plan.routes:
|
||||
agent_facing_port = _EGRESS_PORT
|
||||
else:
|
||||
agent_facing_port = _PIPELOCK_PORT
|
||||
agent_facing_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
||||
)
|
||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||
agent_git_gate_host = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _GIT_GATE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||
agent_supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||
|
||||
# Stamp the URLs onto the plan + guest_env. provision_git
|
||||
# and provision_supervise read the plan fields; the agent
|
||||
# reads guest_env on every exec_agent.
|
||||
#
|
||||
# NO_PROXY has to include the per-bottle loopback alias —
|
||||
# otherwise claude's HTTPS_PROXY catches direct calls to
|
||||
# the supervise URL (`http://<alias>:<port>/`) and proxies
|
||||
# them through egress, which has no route for the alias
|
||||
# and rejects with "Failed to connect". The git-gate URL
|
||||
# uses git://, not affected by HTTP_PROXY, so the alias
|
||||
# only has to be in NO_PROXY for the MCP / supervise
|
||||
# path. Append rather than overwrite so prepare.py's
|
||||
# `localhost,127.0.0.1` baseline stays in place.
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
"HTTP_PROXY": agent_proxy_url,
|
||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||
}
|
||||
if agent_git_gate_host:
|
||||
guest_env["GIT_GATE_URL"] = f"git://{agent_git_gate_host}"
|
||||
if agent_supervise_url:
|
||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||
plan = dataclasses.replace(
|
||||
plan,
|
||||
guest_env=guest_env,
|
||||
agent_proxy_url=agent_proxy_url,
|
||||
agent_git_gate_host=agent_git_gate_host,
|
||||
agent_supervise_url=agent_supervise_url,
|
||||
)
|
||||
|
||||
# 5. Build the agent image and pack it into a
|
||||
# `.smolmachine` artifact (or hit the per-Dockerfile-digest
|
||||
# cache). Runs here, not in prepare, so the docker-build
|
||||
# output doesn't garble the dashboard's preflight modal:
|
||||
# both the curses-endwin path and the tmux pane-routing
|
||||
# path redirect stderr around `launch` already.
|
||||
agent_from_path = _ensure_smolmachine(
|
||||
plan.agent_image,
|
||||
plan.agent_image_ref,
|
||||
dockerfile=plan.agent_dockerfile_path,
|
||||
)
|
||||
|
||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
||||
_init_vm(plan)
|
||||
|
||||
bottle = SmolmachinesBottle(
|
||||
# smolvm VM. --from carries the pre-packed .smolmachine
|
||||
# artifact; --allow-cidr + -e carry the per-bottle TSI
|
||||
# allowlist + env. The allowlist is the per-bottle
|
||||
# loopback alias — narrowing it to one /32 keeps the
|
||||
# agent from reaching other host loopback services or
|
||||
# other bottles' published ports. Smolfile isn't usable
|
||||
# here — smolvm 0.8.0 makes `--from` and `--smolfile`
|
||||
# mutually exclusive.
|
||||
_smolvm.machine_create(
|
||||
plan.machine_name,
|
||||
prompt_path=None,
|
||||
from_path=agent_from_path,
|
||||
allow_cidrs=[f"{loopback_ip}/32"],
|
||||
env=plan.guest_env,
|
||||
)
|
||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently
|
||||
# dropped when combined with `--from`. Patch the persisted
|
||||
# state DB to set the allowlist before start so the booted
|
||||
# VM's TSI actually enforces. See loopback_alias's module
|
||||
# docstring for the investigation that led here.
|
||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||
_smolvm.machine_start(plan.machine_name)
|
||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||
|
||||
# 6. Repair filesystem ownership + perms that smolvm's
|
||||
# pack process remapped to the host invoker's uid (501
|
||||
# on macOS) rather than preserving the image's expected
|
||||
# ownership.
|
||||
#
|
||||
# - /home/node → node:node so the node user can write
|
||||
# its own dotfiles (claude appendFileSync on
|
||||
# ~/.claude.json otherwise bails with ENOENT/EPERM
|
||||
# and the TUI hangs without surfacing the error).
|
||||
# - /tmp + /var/tmp → root:root mode 1777 so non-root
|
||||
# processes can create their per-uid scratch dirs
|
||||
# (claude-code creates /tmp/claude-<uid>/ as soon as
|
||||
# it spawns a Bash tool call).
|
||||
#
|
||||
# All folded into one sh -c so we only pay one
|
||||
# machine_exec round trip — back-to-back exec calls
|
||||
# right after machine_start hit a SIGKILL race in
|
||||
# libkrun's exec channel (see provision_ca for the
|
||||
# other half of this same workaround).
|
||||
_smolvm.machine_exec(plan.machine_name, [
|
||||
"sh", "-c",
|
||||
"chown -R node:node /home/node && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp",
|
||||
])
|
||||
|
||||
# Wait briefly for the VM to settle. Back-to-back smolvm
|
||||
# machine_exec calls immediately after machine_start
|
||||
# occasionally SIGKILL the in-VM child at ~100ms (looks
|
||||
# like a VM warm-up race in libkrun's exec channel).
|
||||
# 1.5s is empirically enough to dodge it; provisioning
|
||||
# already takes seconds so the wait is amortized.
|
||||
time.sleep(1.5)
|
||||
|
||||
# 7. Provision (CA / prompt / skills / git / supervise).
|
||||
prompt_path = provision(plan, plan.machine_name)
|
||||
|
||||
yield SmolmachinesBottle(
|
||||
plan.machine_name,
|
||||
prompt_path=prompt_path,
|
||||
guest_env=plan.guest_env,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=plan.spec.label or plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
yield bottle
|
||||
finally:
|
||||
_teardown_smolmachines(stack, plan)
|
||||
|
||||
|
||||
def _teardown_smolmachines(
|
||||
stack: ExitStack,
|
||||
plan: SmolmachinesBottlePlan,
|
||||
) -> None:
|
||||
"""Unwind the ExitStack, then revoke any provisioned deploy keys.
|
||||
|
||||
ExitStack errors are caught and logged (non-fatal) so that key
|
||||
revocation always runs. Revocation errors propagate — a stranded
|
||||
deploy key is a security concern the operator must address."""
|
||||
teardown_exc: BaseException | None = None
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc: # noqa: W0718 — teardown must not fail
|
||||
teardown_exc = exc
|
||||
warn(f"smolmachines teardown failed: {exc!r}")
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
|
||||
|
||||
def _allocate_resources(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
stack: ExitStack,
|
||||
) -> tuple[str, str]:
|
||||
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
||||
|
||||
macOS only routes 127.0.0.1 by default; the per-bottle alias
|
||||
scopes TSI's allowlist to this bottle's published ports so the
|
||||
agent can't reach other bottles' or host services' ports on
|
||||
loopback. No-op on Linux."""
|
||||
_loopback.ensure_pool()
|
||||
loopback_ip = _loopback.allocate(plan.slug)
|
||||
network = _bundle.bundle_network_name(plan.slug)
|
||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||
stack.callback(_bundle.remove_bundle_network, network)
|
||||
return loopback_ip, network
|
||||
|
||||
|
||||
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
||||
"""Mint the egress MITM CA and return the plan with CA paths filled."""
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
egress_state_dir(plan.slug),
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
plan.egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
)
|
||||
return dataclasses.replace(plan, egress_plan=egress_plan)
|
||||
|
||||
|
||||
def _start_bundle(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
network: str,
|
||||
loopback_ip: str,
|
||||
stack: ExitStack,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Build the BundleLaunchSpec, resolve token env, start the
|
||||
sidecar bundle container, and register teardown."""
|
||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||
token_env = _resolve_token_env(plan, dict(os.environ))
|
||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||
return plan
|
||||
|
||||
|
||||
def _discover_urls(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
loopback_ip: str,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Discover host-side ports for published container ports and
|
||||
return the plan with URLs + guest_env stamped in.
|
||||
|
||||
Docker container IPs (192.168.x.x in the daemon's bridge)
|
||||
aren't reachable from the smolvm guest on macOS — TSI uses
|
||||
macOS networking, and macOS sees the daemon's bridge via the
|
||||
published-port loopback forward only.
|
||||
|
||||
NO_PROXY includes the per-bottle loopback alias so the
|
||||
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
||||
agent_facing_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _EGRESS_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||
|
||||
agent_git_gate_host = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||
|
||||
agent_supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
"HTTP_PROXY": agent_proxy_url,
|
||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||
}
|
||||
if agent_git_gate_host:
|
||||
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
||||
if agent_supervise_url:
|
||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
guest_env=guest_env,
|
||||
agent_proxy_url=agent_proxy_url,
|
||||
agent_git_gate_host=agent_git_gate_host,
|
||||
agent_supervise_url=agent_supervise_url,
|
||||
)
|
||||
|
||||
|
||||
def _launch_vm(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
agent_from_path: Path,
|
||||
loopback_ip: str,
|
||||
stack: ExitStack,
|
||||
) -> None:
|
||||
"""Create, patch, and start the smolvm VM; register teardown.
|
||||
|
||||
--allow-cidr is the per-bottle loopback alias so the guest can
|
||||
only reach this bottle's bundle ports. force_allowlist patches
|
||||
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
|
||||
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
||||
and --smolfile mutually exclusive."""
|
||||
_smolvm.machine_create(
|
||||
plan.machine_name,
|
||||
from_path=agent_from_path,
|
||||
allow_cidrs=[f"{loopback_ip}/32"],
|
||||
env=plan.guest_env,
|
||||
)
|
||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
|
||||
# when combined with `--from`. Patch the persisted state DB
|
||||
# before start so the booted VM's TSI actually enforces.
|
||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||
_smolvm.machine_start(plan.machine_name)
|
||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||
|
||||
|
||||
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||
"""Repair filesystem ownership and wait for exec channel readiness.
|
||||
|
||||
Ownership repair: smolvm's pack process remaps files to the host
|
||||
invoker's uid (501 on macOS). /home/node must be node:node so
|
||||
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
||||
mode 1777 so non-root processes can create per-uid scratch dirs.
|
||||
All folded into one sh -c to avoid back-to-back exec calls
|
||||
immediately after machine_start (libkrun exec-channel race).
|
||||
|
||||
wait_exec_ready polls until the exec channel is ready for the
|
||||
subsequent provision calls, replacing the empirical sleep."""
|
||||
_smolvm.machine_exec(plan.machine_name, [
|
||||
"sh", "-c",
|
||||
"chown -R node:node /home/node && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp",
|
||||
])
|
||||
_smolvm.wait_exec_ready(plan.machine_name)
|
||||
|
||||
|
||||
def _bundle_launch_spec(
|
||||
@@ -292,23 +303,46 @@ def _bundle_launch_spec(
|
||||
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
||||
|
||||
Daemons in the CSV:
|
||||
- egress is always present.
|
||||
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
|
||||
- egress + pipelock are always present (pipelock is the
|
||||
agent's first hop; egress is its upstream).
|
||||
- git-gate is conditional on plan.git_gate_plan.upstreams.
|
||||
- supervise is conditional on plan.supervise_plan.
|
||||
|
||||
Env + volumes are the union of the sidecar daemons' needs, with
|
||||
Env + volumes are the union of the four daemons' needs, with
|
||||
daemon-private values only (HTTPS_PROXY is scoped to the
|
||||
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
||||
bind-address PR)."""
|
||||
daemons: list[str] = ["egress"]
|
||||
daemons: list[str] = ["egress", "pipelock"]
|
||||
env: list[str] = []
|
||||
volumes: list[tuple[str, str, bool]] = []
|
||||
|
||||
# In this Docker-Desktop-compatible topology, whichever daemon
|
||||
# is "agent-facing" gets its port published on the host
|
||||
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
||||
# other stays bundle-internal. The bundle is NOT reachable by
|
||||
# bridge IP from the smolvm guest, so the
|
||||
# PRD-0023-chunk-3 EGRESS_LISTEN_HOST=127.0.0.1 mitigation
|
||||
# isn't needed: the agent can only dial whatever daemon's
|
||||
# host port we publish, period.
|
||||
|
||||
# --- pipelock ---------------------------------------------
|
||||
pp = plan.proxy_plan
|
||||
volumes += [
|
||||
(str(pp.yaml_path), "/etc/pipelock.yaml", True),
|
||||
(str(pp.ca_cert_host_path), PIPELOCK_CA_CERT_IN_CONTAINER, True),
|
||||
(str(pp.ca_key_host_path), PIPELOCK_CA_KEY_IN_CONTAINER, True),
|
||||
]
|
||||
|
||||
# --- egress -----------------------------------------------
|
||||
ep = plan.egress_plan
|
||||
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
|
||||
if ep.routes:
|
||||
volumes.append((str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True))
|
||||
env.append(f"EGRESS_UPSTREAM_PROXY={ep.pipelock_proxy_url}")
|
||||
env.append(f"EGRESS_UPSTREAM_CA={EGRESS_PIPELOCK_CA_IN_CONTAINER}")
|
||||
volumes += [
|
||||
(str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True),
|
||||
(str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True),
|
||||
(str(ep.pipelock_ca_host_path), EGRESS_PIPELOCK_CA_IN_CONTAINER, True),
|
||||
]
|
||||
# Bare-name entries for upstream-token slots. Their values
|
||||
# come from the docker-run subprocess env (inherited from
|
||||
# the operator's shell), never landing on argv.
|
||||
@@ -316,9 +350,10 @@ def _bundle_launch_spec(
|
||||
env.append(token_env)
|
||||
|
||||
# --- git-gate ---------------------------------------------
|
||||
extra_hosts: list[str] = []
|
||||
gp = plan.git_gate_plan
|
||||
if gp.upstreams:
|
||||
daemons += ["git-gate", "git-http"]
|
||||
daemons.append("git-gate")
|
||||
volumes += [
|
||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
|
||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
|
||||
@@ -351,10 +386,16 @@ def _bundle_launch_spec(
|
||||
|
||||
# Container ports the agent reaches from the smolvm guest —
|
||||
# published on host loopback so the guest can dial via TSI +
|
||||
# macOS networking. Egress is always the agent's HTTP/HTTPS proxy.
|
||||
ports_to_publish: list[int] = [_EGRESS_PORT]
|
||||
# macOS networking. The HTTP/HTTPS chokepoint is whichever
|
||||
# daemon's port we publish: egress when routes are declared
|
||||
# (token injection first, then forwards to bundle-internal
|
||||
# pipelock), pipelock otherwise.
|
||||
if ep.routes:
|
||||
ports_to_publish: list[int] = [_EGRESS_PORT]
|
||||
else:
|
||||
ports_to_publish = [_PIPELOCK_PORT]
|
||||
if gp.upstreams:
|
||||
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||
ports_to_publish.append(_GIT_GATE_PORT)
|
||||
if sp is not None:
|
||||
ports_to_publish.append(_SUPERVISE_PORT)
|
||||
|
||||
@@ -373,13 +414,15 @@ def _bundle_launch_spec(
|
||||
|
||||
|
||||
def _resolve_token_env(
|
||||
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
|
||||
plan: SmolmachinesBottlePlan, host_env: object
|
||||
) -> dict[str, str]:
|
||||
"""Resolve the egress token env-var values from the host's
|
||||
environ so they reach the bundle's process env via docker's
|
||||
`-e NAME` inheritance. Empty when no routes declare auth."""
|
||||
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
|
||||
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||
ep = plan.egress_plan
|
||||
if not ep.routes:
|
||||
return {}
|
||||
return egress_resolve_token_values(ep.token_env_map, dict(host_env))
|
||||
|
||||
|
||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||
|
||||
@@ -42,13 +42,13 @@ import time
|
||||
import uuid
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Generator
|
||||
from typing import Iterator
|
||||
|
||||
from ...log import die
|
||||
|
||||
|
||||
# registry:2.8.3, pinned by digest. Same env-override pattern as the
|
||||
# sidecar image pin in bot_bottle/backend/docker/sidecar_bundle.py.
|
||||
# pipelock image pin in bot_bottle/backend/docker/pipelock.py.
|
||||
REGISTRY_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_REGISTRY_IMAGE",
|
||||
"registry@sha256:a3d8aaa63ed8681a604f1dea0aa03f100d5895b6a58ace528858a7b332415373",
|
||||
@@ -61,10 +61,7 @@ REGISTRY_IMAGE = os.environ.get(
|
||||
# narrow.
|
||||
CRANE_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_CRANE_IMAGE",
|
||||
(
|
||||
"gcr.io/go-containerregistry/crane@sha256:"
|
||||
"0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084"
|
||||
),
|
||||
"gcr.io/go-containerregistry/crane@sha256:0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084",
|
||||
)
|
||||
|
||||
|
||||
@@ -98,7 +95,7 @@ class RegistryHandle:
|
||||
|
||||
|
||||
@contextmanager
|
||||
def ephemeral_registry() -> Generator[RegistryHandle, None, None]:
|
||||
def ephemeral_registry() -> Iterator[RegistryHandle]:
|
||||
"""Bring up a per-session docker network + a `registry:2.8.3`
|
||||
container on it (published on a random host port), yield a
|
||||
`RegistryHandle`, force-remove both on exit.
|
||||
@@ -208,6 +205,7 @@ def _host_port(name: str) -> int:
|
||||
return int(port_str)
|
||||
except ValueError:
|
||||
die(f"unexpected `docker port` output: {line!r}")
|
||||
return -1 # unreachable; die() never returns
|
||||
|
||||
|
||||
def _wait_ready(port: int) -> None:
|
||||
|
||||
@@ -45,8 +45,8 @@ alias gets handed to a new bottle."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import sqlite3
|
||||
@@ -83,14 +83,6 @@ _POOL_START = 16
|
||||
_POOL_END = 31 # inclusive
|
||||
|
||||
|
||||
# File lock that serialises concurrent allocate() calls so two
|
||||
# simultaneous launches can't read the same docker state and claim
|
||||
# the same alias. Narrowed to the allocate() call itself; docker run
|
||||
# runs after the lock is released. Once the container is running it
|
||||
# appears in docker state and future allocate() calls will see it.
|
||||
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
||||
|
||||
|
||||
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
||||
def _pool_addresses() -> list[str]:
|
||||
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
||||
@@ -176,31 +168,20 @@ def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
||||
con.close()
|
||||
|
||||
|
||||
def allocate(_slug: str) -> str:
|
||||
def allocate(slug: str) -> str:
|
||||
"""Pick the lowest-numbered alias from the pool not already
|
||||
in use by a running smolmachines bundle. Bails when the pool
|
||||
is exhausted — the caller should report the limit to the
|
||||
operator. `_slug` is logged for traceability; not otherwise
|
||||
operator. `slug` is logged for traceability; not otherwise
|
||||
used (no on-disk reservation, allocation is purely
|
||||
docker-state-driven).
|
||||
|
||||
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
||||
`127.0.0.1` is fine to share and we skip the alias dance.
|
||||
This still returns a deterministic address so launch.py's
|
||||
callers don't have to branch on platform.
|
||||
|
||||
An exclusive file lock serialises concurrent calls so two
|
||||
simultaneous launches don't read the same docker state and
|
||||
claim the same alias."""
|
||||
callers don't have to branch on platform."""
|
||||
if not _is_macos():
|
||||
return "127.0.0.1"
|
||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
|
||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||
return _allocate_locked()
|
||||
|
||||
|
||||
def _allocate_locked() -> str:
|
||||
in_use = _aliases_in_use()
|
||||
for ip in _pool_addresses():
|
||||
if ip not in in_use:
|
||||
@@ -211,6 +192,7 @@ def _allocate_locked() -> str:
|
||||
f"Stop a running bottle (`smolvm machine ls --json`) or "
|
||||
f"raise _POOL_END in loopback_alias.py."
|
||||
)
|
||||
return "" # unreachable; die() never returns
|
||||
|
||||
|
||||
def _alias_present(ip: str) -> bool:
|
||||
|
||||
@@ -0,0 +1,192 @@
|
||||
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
|
||||
|
||||
Resolves the per-bottle docker subnet + bundle IP and assembles
|
||||
the guest env. The agent's docker image build → smolmachine
|
||||
pack pipeline runs in `launch.launch`, not here, so the
|
||||
dashboard's preflight modal isn't garbled by docker-build output
|
||||
before the operator has confirmed.
|
||||
|
||||
No VM bringup — that's `launch.launch`'s job."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import runtime_for
|
||||
from ...backend import BottleSpec
|
||||
from ...backend.docker.bottle_state import (
|
||||
BottleMetadata,
|
||||
agent_state_dir,
|
||||
bottle_identity,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
pipelock_state_dir,
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from ...egress import Egress
|
||||
from ...git_gate import GitGate
|
||||
from ...pipelock import PipelockProxy
|
||||
from ...supervise import Supervise
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||
|
||||
|
||||
# Gateway ports the bundle exposes inside its container — pipelock
|
||||
# HTTPS proxy, git-gate's git-daemon, supervise's MCP. The agent
|
||||
# inside the smolvm guest dials these on the bundle's pinned IP.
|
||||
_BUNDLE_PIPELOCK_PORT = 8888
|
||||
_BUNDLE_GIT_GATE_PORT = 9418
|
||||
_BUNDLE_SUPERVISE_PORT = 9100
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec, *, stage_dir: Path
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Materialize the smolmachines plan. The bundle's docker
|
||||
subnet + pinned IP are derived from the slug; the agent's
|
||||
`.smolmachine` artifact is built (or cache-hit) here so
|
||||
launch's `machine create --from` boots without a registry
|
||||
pull. Per-bottle guest env + the TSI allow_cidrs land on the
|
||||
plan for launch to pass straight through to
|
||||
`machine create` flags."""
|
||||
smolmachines_preflight()
|
||||
|
||||
manifest = spec.manifest
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
provider = bottle.agent_provider
|
||||
provider_runtime = runtime_for(provider.template)
|
||||
|
||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||
|
||||
# Record minimal metadata so `cli.py resume` can recover the
|
||||
# slug. Same schema as the docker backend.
|
||||
write_metadata(BottleMetadata(
|
||||
identity=slug,
|
||||
agent_name=spec.agent_name,
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
# No compose project for smolmachines bottles; chunk 4
|
||||
# will give dashboard discovery a backend-specific path.
|
||||
compose_project="",
|
||||
))
|
||||
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||
|
||||
# Agent's env: the prepare-time view doesn't yet know the
|
||||
# host loopback ports the bundle's daemons get published on
|
||||
# (those come from docker AFTER `docker run` returns), so
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are
|
||||
# populated in launch.py and stamped onto guest_env there.
|
||||
# What we set here is the part that doesn't depend on
|
||||
# bundle bringup — bottle.env literals, the empty-NO_PROXY
|
||||
# safe default, and the TLS trust env trio
|
||||
# (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / REQUESTS_CA_BUNDLE)
|
||||
# pointing at Debian's update-ca-certificates output bundle.
|
||||
guest_env: dict[str, str] = {
|
||||
**bottle.env,
|
||||
"NO_PROXY": "localhost,127.0.0.1",
|
||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
|
||||
# Inner Plans for the four bundle daemons. The ABCs are
|
||||
# platform-neutral — `.prepare()` writes config files + returns
|
||||
# a Plan dataclass with no backend-specific assumptions. State
|
||||
# dirs are still keyed by slug under the docker backend's
|
||||
# bottle_state layout (shared on-host convention; not a docker
|
||||
# dependency).
|
||||
pipelock_dir = pipelock_state_dir(slug)
|
||||
pipelock_dir.mkdir(parents=True, exist_ok=True)
|
||||
proxy_plan = PipelockProxy().prepare(bottle, slug, pipelock_dir)
|
||||
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = Egress().prepare(bottle, slug, egress_dir)
|
||||
|
||||
# Claude-code refuses to start without *something* it
|
||||
# recognises as a credential. When the bottle has an egress
|
||||
# route carrying the `claude_code_oauth` role marker, egress
|
||||
# strips + re-injects the real Authorization header on the
|
||||
# outbound leg using a token held in egress's own environ — so
|
||||
# the agent gets a non-secret placeholder here (matches the
|
||||
# docker backend's forwarded_env logic in
|
||||
# bot_bottle/backend/docker/prepare.py).
|
||||
has_provider_auth = any(
|
||||
provider_runtime.auth_role in r.roles for r in egress_plan.routes
|
||||
)
|
||||
if has_provider_auth:
|
||||
guest_env[provider_runtime.placeholder_env] = "egress-placeholder"
|
||||
if provider.template == "claude" and has_provider_auth:
|
||||
guest_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
|
||||
guest_env.setdefault("DISABLE_ERROR_REPORTING", "1")
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
supervise_plan = Supervise().prepare(slug, supervise_dir)
|
||||
|
||||
# Prompt file is always written (mode 0o600) so the in-VM
|
||||
# path always exists. Content is the agent's `prompt`
|
||||
# field (markdown body) — empty for agents with no prompt.
|
||||
# claude-code reads it via --append-system-prompt-file only
|
||||
# when non-empty, but the file must exist either way to
|
||||
# match the docker backend's contract.
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
prompt_file.write_text(agent.prompt or "")
|
||||
prompt_file.chmod(0o600)
|
||||
|
||||
machine_name = f"bot-bottle-{slug}"
|
||||
# Stash the agent image ref — `launch.launch` runs the
|
||||
# build → pack pipeline at bringup. Honors BOT_BOTTLE_IMAGE
|
||||
# to match the docker backend's `resolve_plan` default.
|
||||
agent_dockerfile_path = ""
|
||||
if provider.dockerfile:
|
||||
agent_dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
|
||||
image_default = f"bot-bottle-{provider.template}:{slug}"
|
||||
elif provider_runtime.dockerfile:
|
||||
agent_dockerfile_path = provider_runtime.dockerfile
|
||||
image_default = provider_runtime.image
|
||||
else:
|
||||
image_default = provider_runtime.image
|
||||
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
bundle_subnet=subnet,
|
||||
bundle_gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
machine_name=machine_name,
|
||||
agent_image_ref=agent_image_ref,
|
||||
guest_env=guest_env,
|
||||
prompt_file=prompt_file,
|
||||
proxy_plan=proxy_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_command=provider_runtime.command,
|
||||
agent_prompt_mode=provider_runtime.prompt_mode,
|
||||
agent_provider_template=provider.template,
|
||||
agent_dockerfile_path=agent_dockerfile_path,
|
||||
)
|
||||
|
||||
|
||||
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
|
||||
path = Path(os.path.expanduser(path_value))
|
||||
if not path.is_absolute():
|
||||
path = Path(spec.user_cwd) / path
|
||||
return str(path)
|
||||
@@ -1,12 +1,14 @@
|
||||
"""Backend-infrastructure provisioners for the smolmachines backend.
|
||||
"""Provisioning helpers for the smolmachines backend (PRD 0023
|
||||
chunk 4).
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration) live on
|
||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
|
||||
provisioning also moved to the AgentProvider ABC (with Debian/node
|
||||
defaults); user plugins override them for non-standard images.
|
||||
Each method maps onto one of `BottleBackend`'s `provision_*`
|
||||
overrides. They run after the VM is up + the bundle is reachable
|
||||
and copy host-side state (prompt, skills, .git, CA cert,
|
||||
supervise MCP config) into the guest via `smolvm machine cp` /
|
||||
`smolvm machine exec`.
|
||||
|
||||
No modules remain in this subpackage. Workspace copying now runs
|
||||
through `BottleBackend.provision_workspace` against the running
|
||||
bottle for every backend.
|
||||
"""
|
||||
Chunk 4a ships `provision_prompt` and `provision_skills` — the
|
||||
two that don't depend on agent-image tooling (claude-code,
|
||||
update-ca-certificates) beyond `cp` and `mkdir`. provision_ca /
|
||||
provision_git / provision_supervise land once the agent-image
|
||||
gap is solved."""
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Install the per-bottle MITM CA into the smolmachines guest's
|
||||
trust store (PRD 0023 chunk 4d).
|
||||
|
||||
Mirrors `backend.docker.provision.ca`: select the right CA (egress
|
||||
when the bottle has routes, else pipelock), `smolvm machine cp` it
|
||||
to Debian's `/usr/local/share/ca-certificates/` path,
|
||||
`update-ca-certificates` to rebuild the trust bundle, and log the
|
||||
fingerprint once. The selected cert depends on the agent's
|
||||
HTTP_PROXY target — same logic as the docker backend, since the
|
||||
agent dials the same daemons through the same bundle.
|
||||
|
||||
`smolvm machine exec` runs commands as root in the VM (no `-u`
|
||||
flag exists; the VM init is root), so we don't need the explicit
|
||||
`-u 0` the docker backend uses on its `docker exec` calls."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ....log import die
|
||||
from ...util import (
|
||||
AGENT_CA_BUNDLE,
|
||||
AGENT_CA_PATH,
|
||||
log_ca_fingerprint,
|
||||
select_ca_cert,
|
||||
)
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Copy the agent-facing CA cert into the guest, rebuild the
|
||||
trust bundle, emit a one-line fingerprint log. Called from
|
||||
`BottleBackend.provision` after the smolvm guest is up."""
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
||||
|
||||
_smolvm.machine_cp(str(cert_host_path), f"{target}:{AGENT_CA_PATH}")
|
||||
# Mode 0644 — readable to non-root tools in the guest.
|
||||
# update-ca-certificates rebuilds the bundle at AGENT_CA_BUNDLE,
|
||||
# which is what curl / Python ssl / OpenSSL-based tools read by
|
||||
# default. The env trio (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE /
|
||||
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
||||
# `requests` / libraries that don't load the system bundle.
|
||||
#
|
||||
# chown + chmod + update-ca-certificates run in one
|
||||
# `sh -c` so we only pay one machine_exec round trip; the
|
||||
# `&&` chaining surfaces the first failure as the return
|
||||
# code.
|
||||
r = _smolvm.machine_exec(target, [
|
||||
"sh", "-c",
|
||||
f"chown root:root {AGENT_CA_PATH} && "
|
||||
f"chmod 644 {AGENT_CA_PATH} && "
|
||||
f"update-ca-certificates",
|
||||
])
|
||||
if r.returncode != 0 or "1 added" not in (r.stdout or ""):
|
||||
# update-ca-certificates not adding our cert is fatal —
|
||||
# claude-code's TLS handshake against the egress-MITM'd
|
||||
# api.anthropic.com would fail downstream. Bail early
|
||||
# with what we can see (output is captured by smolvm so
|
||||
# we can surface it).
|
||||
die(
|
||||
f"update-ca-certificates didn't add the agent CA "
|
||||
f"(exit {r.returncode}): "
|
||||
f"stdout={(r.stdout or '').strip()!r} "
|
||||
f"stderr={(r.stderr or '').strip()!r}"
|
||||
)
|
||||
|
||||
log_ca_fingerprint(cert_host_path, label)
|
||||
|
||||
|
||||
# Re-exported for the launch/provision_ca caller + tests. The path
|
||||
# constants live in the shared `backend.util` (Debian's
|
||||
# `update-ca-certificates` layout is the same in both backends).
|
||||
__all__ = ["AGENT_CA_BUNDLE", "AGENT_CA_PATH", "provision_ca"]
|
||||
@@ -0,0 +1,141 @@
|
||||
"""Git provisioning inside a running smolmachines bottle
|
||||
(PRD 0023 chunk 4d).
|
||||
|
||||
Three concerns, all about git in the agent:
|
||||
|
||||
1. If --cwd was passed AND the host cwd has a .git, copy that
|
||||
.git into /home/node/workspace/.git so the agent operates on
|
||||
the user's repo.
|
||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||
~/.gitconfig with insteadOf rules so every git operation
|
||||
against a declared upstream transparently hits the per-bottle
|
||||
git-gate. The gate mirrors the upstream in both directions,
|
||||
so URL rewriting is symmetric.
|
||||
3. If the bottle declares `git.user` (issue #86), set
|
||||
`git config --global user.{name,email}` inside the guest so
|
||||
the agent's commits are attributed to that identity.
|
||||
|
||||
Differs from `backend.docker.provision.git` in one address detail:
|
||||
the TSI-allowlisted guest can only reach the bundle's pinned IP
|
||||
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
|
||||
are `git://<bundle_ip>:<port>/<name>.git` rather than the
|
||||
docker backend's `git://git-gate/<name>.git`. The render itself
|
||||
is the shared `git_gate_render_gitconfig` on the platform-neutral
|
||||
git_gate module."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from ....git_gate import git_gate_render_gitconfig
|
||||
from ....log import info
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
# `node` is the agent user from the repo Dockerfile. Override via
|
||||
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
||||
# BOT_BOTTLE_CONTAINER_HOME knob — same purpose, different
|
||||
# transport.
|
||||
_DEFAULT_GUEST_HOME = "/home/node"
|
||||
|
||||
|
||||
def _guest_home() -> str:
|
||||
return os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
||||
|
||||
|
||||
def provision_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Set up git inside the guest. Runs all three subcases; each
|
||||
no-ops when its condition isn't met."""
|
||||
_provision_cwd_git(plan, target)
|
||||
_provision_git_gate_config(plan, target)
|
||||
_provision_git_user(plan, target)
|
||||
|
||||
|
||||
def _provision_cwd_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||
it into <guest_home>/workspace/.git and fix ownership. No-op
|
||||
otherwise."""
|
||||
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
||||
return
|
||||
guest_workspace_git = f"{_guest_home()}/workspace/.git"
|
||||
info(f"copying {plan.spec.user_cwd}/.git -> {target}:{guest_workspace_git}")
|
||||
# mkdir -p the workspace dir so `machine cp` lands the .git
|
||||
# directly there even on first-time bottles.
|
||||
_smolvm.machine_exec(target, ["mkdir", "-p", f"{_guest_home()}/workspace"])
|
||||
_smolvm.machine_cp(
|
||||
f"{plan.spec.user_cwd}/.git", f"{target}:{guest_workspace_git}",
|
||||
)
|
||||
# `machine cp` lands files as root; the agent runs as node so
|
||||
# the workspace tree must be chowned over.
|
||||
_smolvm.machine_exec(
|
||||
target, ["chown", "-R", "node:node", guest_workspace_git],
|
||||
)
|
||||
|
||||
|
||||
def _provision_git_gate_config(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Write ~/.gitconfig in the guest with the git-gate insteadOf
|
||||
rules. No-op when the bottle has no `git` entries."""
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
if not bottle.git:
|
||||
return
|
||||
|
||||
# `127.0.0.1:<host port>` form: the bundle's git-gate port
|
||||
# is published on host loopback at launch time so the
|
||||
# smolvm guest (which can only reach macOS networking via
|
||||
# TSI, not the docker bridge IP) can dial it. launch.py
|
||||
# populates `plan.agent_git_gate_host` after bundle bringup.
|
||||
content = git_gate_render_gitconfig(bottle.git, plan.agent_git_gate_host)
|
||||
|
||||
guest_gitconfig = f"{_guest_home()}/.gitconfig"
|
||||
# Stage the file under the plan's stage_dir so `machine cp`
|
||||
# has a stable host path. The plan's stage_dir is cleaned up
|
||||
# by start.py's session-end teardown.
|
||||
with tempfile.NamedTemporaryFile(
|
||||
"w", dir=str(plan.stage_dir), prefix="gitconfig.",
|
||||
delete=False,
|
||||
) as f:
|
||||
f.write(content)
|
||||
config_file = Path(f.name)
|
||||
os.chmod(config_file, 0o600)
|
||||
|
||||
info(f"writing {guest_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
||||
_smolvm.machine_cp(str(config_file), f"{target}:{guest_gitconfig}")
|
||||
_smolvm.machine_exec(target, ["chown", "node:node", guest_gitconfig])
|
||||
_smolvm.machine_exec(target, ["chmod", "644", guest_gitconfig])
|
||||
|
||||
|
||||
def _provision_git_user(
|
||||
plan: SmolmachinesBottlePlan, target: str,
|
||||
) -> None:
|
||||
"""Apply `git config --global user.{name,email}` inside the
|
||||
guest as the node user so --global lands in the same
|
||||
`/home/node/.gitconfig` that `_provision_git_gate_config`
|
||||
writes to. No-op when the bottle didn't declare `git.user`.
|
||||
|
||||
Runs via `runuser -u node --`; HOME is forced via smolvm's
|
||||
`-e` flag because runuser (without -l) inherits root's
|
||||
HOME=/root, which would put --global in the wrong file."""
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
gu = bottle.git_user
|
||||
if gu.is_empty():
|
||||
return
|
||||
env = {"HOME": _guest_home(), "USER": "node"}
|
||||
if gu.name:
|
||||
info(f"git config --global user.name = {gu.name!r}")
|
||||
_smolvm.machine_exec(
|
||||
target,
|
||||
["runuser", "-u", "node", "--",
|
||||
"git", "config", "--global", "user.name", gu.name],
|
||||
env=env,
|
||||
)
|
||||
if gu.email:
|
||||
info(f"git config --global user.email = {gu.email!r}")
|
||||
_smolvm.machine_exec(
|
||||
target,
|
||||
["runuser", "-u", "node", "--",
|
||||
"git", "config", "--global", "user.email", gu.email],
|
||||
env=env,
|
||||
)
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Copy the agent prompt into a running smolmachines bottle.
|
||||
|
||||
The prompt file is always copied (so the in-guest path always
|
||||
exists) but `--append-system-prompt-file` only fires when the
|
||||
agent actually has a prompt — the return value signals which
|
||||
case, mirroring the docker backend's contract.
|
||||
|
||||
`smolvm machine cp` lands files as root inside the VM; the claude
|
||||
process runs as `node`, so we chown + chmod the prompt after the
|
||||
copy. Same flow as the docker backend's provision_prompt."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
# `node` is the agent user from the repo Dockerfile.
|
||||
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
||||
# BOT_BOTTLE_CONTAINER_HOME knob.
|
||||
_DEFAULT_GUEST_HOME = "/home/node"
|
||||
|
||||
|
||||
def provision_prompt(plan: SmolmachinesBottlePlan, target: str) -> str | None:
|
||||
"""Copy the prompt file into the running smolvm guest, fix
|
||||
ownership/mode. Returns the in-guest path if the agent has a
|
||||
non-empty prompt (drives --append-system-prompt-file), else
|
||||
None. The file is copied either way so the path always
|
||||
exists — mirrors the docker backend's behavior."""
|
||||
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
||||
in_guest_prompt_path = f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
_smolvm.machine_cp(str(plan.prompt_file), f"{target}:{in_guest_prompt_path}")
|
||||
# machine cp lands as root, source's 0o600 mode is preserved —
|
||||
# node can't read its own prompt without these two.
|
||||
_smolvm.machine_exec(target, ["chown", "node:node", in_guest_prompt_path])
|
||||
_smolvm.machine_exec(target, ["chmod", "600", in_guest_prompt_path])
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return in_guest_prompt_path if agent.prompt else None
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Copy host-side skill directories into a running smolmachines
|
||||
bottle.
|
||||
|
||||
Skills are validated on the host before launch by
|
||||
`BottleBackend._validate_skills`; this module assumes that
|
||||
validation has already run. A skill that disappears between
|
||||
validation and copy still dies loudly rather than silently
|
||||
producing a partial guest."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from ....log import die, info
|
||||
from ...util import host_skill_dir
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
# In-guest path mirrors the docker backend's claude-skills
|
||||
# convention (~/.claude/skills/<name>/) under the node user's
|
||||
# home — same path as the real bot-bottle image's
|
||||
# /home/node/.claude/skills (pre-created in the Dockerfile).
|
||||
_DEFAULT_SKILLS_DIR = "/home/node/.claude/skills"
|
||||
|
||||
|
||||
def provision_skills(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Copy each of the agent's named skills from the host's
|
||||
~/.claude/skills/<name>/ into the guest's equivalent path.
|
||||
For each skill: `mkdir -p` the destination, `smolvm machine cp`
|
||||
the host source dir over, then chown the result to node:node so
|
||||
the agent can read it. No-op when the agent has no skills.
|
||||
|
||||
smolvm machine cp on a directory copies recursively (same
|
||||
semantics as `cp -r`); unlike docker cp's trailing-slash
|
||||
convention, smolvm doesn't need the `/.` suffix dance.
|
||||
|
||||
machine cp lands files as root inside the VM, so we chown each
|
||||
skill tree over to node:node after the copy — same pattern as
|
||||
the docker backend's provision_prompt."""
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
|
||||
skills_dir = os.environ.get(
|
||||
"BOT_BOTTLE_GUEST_SKILLS_DIR", _DEFAULT_SKILLS_DIR,
|
||||
)
|
||||
|
||||
_smolvm.machine_exec(target, ["mkdir", "-p", skills_dir])
|
||||
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {target}:{dst}")
|
||||
# Wipe any prior copy so re-runs don't accumulate.
|
||||
_smolvm.machine_exec(target, ["rm", "-rf", dst])
|
||||
_smolvm.machine_cp(src, f"{target}:{dst}")
|
||||
_smolvm.machine_exec(target, ["chown", "-R", "node:node", dst])
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Supervise sidecar provisioning inside a running smolmachines
|
||||
bottle (PRD 0023 chunk 4d; PRD 0013 supervise plane).
|
||||
|
||||
Registers the per-bottle supervise sidecar as an HTTP MCP server
|
||||
in the agent's claude-code config so the agent discovers the
|
||||
stuck-recovery MCP tools (pipelock-block, capability-block) at
|
||||
startup.
|
||||
|
||||
Mirrors `backend.docker.provision.supervise` — same `claude mcp
|
||||
add` call, just dispatched via `smolvm machine exec` instead of
|
||||
`docker exec`, and against `<bundle_ip>:<port>` instead of the
|
||||
short `supervise` alias (no DNS in the TSI-allowlisted guest)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ....log import info, warn
|
||||
from .. import smolvm as _smolvm
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def provision_supervise(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||
"""Run `claude mcp add` inside the guest to register the
|
||||
supervise sidecar in claude-code's user config. No-op when
|
||||
bottle.supervise is False.
|
||||
|
||||
The URL is the agent-side endpoint launch.py populated after
|
||||
bundle bringup — `http://127.0.0.1:<host port>/` rather than
|
||||
the bundle's docker bridge IP, because that bridge isn't
|
||||
reachable from the smolvm guest on macOS.
|
||||
|
||||
Failure is logged but not fatal: the bottle still works (you
|
||||
just can't call supervise tools from the agent until the entry
|
||||
is added manually). The operator sees the warning at launch."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
url = plan.agent_supervise_url
|
||||
info(f"registering supervise MCP server in agent claude config → {url}")
|
||||
# `claude mcp add --scope user` writes to ~/.claude.json. The
|
||||
# agent is the `node` user; smolvm machine_exec runs as root
|
||||
# by default, so we have to switch user explicitly and set
|
||||
# HOME so the config lands in /home/node/.claude.json (where
|
||||
# the agent's claude actually reads it from).
|
||||
r = _smolvm.machine_exec(
|
||||
target,
|
||||
[
|
||||
"runuser", "-u", "node", "--",
|
||||
"env", "HOME=/home/node",
|
||||
"claude", "mcp", "add",
|
||||
"--scope", "user",
|
||||
"--transport", "http",
|
||||
_SUPERVISE_MCP_NAME,
|
||||
url,
|
||||
],
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"claude mcp add --scope user --transport http supervise {url}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["provision_supervise"]
|
||||
@@ -42,7 +42,6 @@ import subprocess
|
||||
import sys
|
||||
import termios
|
||||
import threading
|
||||
from types import FrameType
|
||||
|
||||
|
||||
# How long to wait after the main exec starts before pushing the
|
||||
@@ -68,9 +67,8 @@ def _read_winsize() -> tuple[int, int] | None:
|
||||
- tmux respawn-pane: tmux sets all three to the pane's PTY.
|
||||
- non-TTY (someone piped stdin in tests): none are; the
|
||||
sync just no-ops, which is the right behavior."""
|
||||
for stream in (sys.stdin, sys.stdout, sys.stderr):
|
||||
for fd in (sys.stdin.fileno(), sys.stdout.fileno(), sys.stderr.fileno()):
|
||||
try:
|
||||
fd = stream.fileno()
|
||||
data = fcntl.ioctl(fd, termios.TIOCGWINSZ, b"\x00" * 8)
|
||||
except OSError:
|
||||
continue
|
||||
@@ -125,13 +123,13 @@ def main(argv: list[str]) -> int:
|
||||
machine = argv[0]
|
||||
inner = argv[2:]
|
||||
|
||||
def sync(_signum: int | None = None, _frame: FrameType | None = None) -> None:
|
||||
def sync(*_args) -> None:
|
||||
size = _read_winsize()
|
||||
if size is None:
|
||||
return
|
||||
_push_size(machine, *size)
|
||||
|
||||
signal.signal(signal.SIGWINCH, sync) # type: ignore[arg-type]
|
||||
signal.signal(signal.SIGWINCH, sync)
|
||||
|
||||
proc = subprocess.Popen(inner)
|
||||
# Initial sync is deferred — see _STARTUP_SYNC_DELAY_SEC.
|
||||
|
||||
@@ -1,80 +0,0 @@
|
||||
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
|
||||
|
||||
Resolves the per-bottle docker subnet + bundle IP and assembles
|
||||
the guest env. The agent's docker image build → smolmachine
|
||||
pack pipeline runs in `launch.launch`, not here, so the
|
||||
dashboard's preflight modal isn't garbled by docker-build output
|
||||
before the operator has confirmed.
|
||||
|
||||
No VM bringup — that's `launch.launch`'s job."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .. import BottleSpec
|
||||
from ...env import ResolvedEnv
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||
|
||||
def preflight() -> None:
|
||||
smolmachines_preflight()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
# Agent's env: resolve through resolve_env() so ?prompt entries
|
||||
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
||||
# the Docker backend's contract. Forwarded (secret/interpolated)
|
||||
# values still reach the guest as -e K=V smolvm flags because
|
||||
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
||||
# the known argv-exposure gap documented in PRD 0038.
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
||||
# in launch.py after bundle bringup.
|
||||
return {
|
||||
**resolved_env.literals,
|
||||
**resolved_env.forwarded,
|
||||
"NO_PROXY": "localhost,127.0.0.1",
|
||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Materialize the smolmachines plan. The bundle's docker
|
||||
subnet + pinned IP are derived from the slug; the agent's
|
||||
`.smolmachine` artifact is built (or cache-hit) here so
|
||||
launch's `machine create --from` boots without a registry
|
||||
pull. Per-bottle guest env + the TSI allow_cidrs land on the
|
||||
plan for launch to pass straight through to
|
||||
`machine create` flags."""
|
||||
|
||||
# ==== smolmachines specific setup ====
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
bundle_subnet=subnet,
|
||||
bundle_gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
guest_env=agent_provision_plan.guest_env,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -19,7 +19,7 @@ This module ships the lifecycle primitives only — create
|
||||
network, start bundle, stop bundle, remove network — wrapped
|
||||
around `subprocess.run(["docker", ...])`. Wiring them into the
|
||||
launch flow + populating the `BundleLaunchSpec` from the inner
|
||||
Plans (EgressPlan, …) lands in chunk 2d."""
|
||||
Plans (PipelockProxyPlan, EgressPlan, …) lands in chunk 2d."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -69,7 +69,7 @@ class BundleLaunchSpec:
|
||||
# Daemon subset CSV for BOT_BOTTLE_SIDECAR_DAEMONS. The
|
||||
# supervisor inside the bundle reads it to skip
|
||||
# bottle-irrelevant daemons (e.g. supervise=False bottles).
|
||||
daemons_csv: str = "egress"
|
||||
daemons_csv: str = "egress,pipelock"
|
||||
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name
|
||||
# form inherits the value from the docker-run subprocess env,
|
||||
# matching the docker backend's compose-up secret-forwarding
|
||||
@@ -223,6 +223,7 @@ def bundle_host_port(
|
||||
f"no port mapping on {host_ip} for {container} "
|
||||
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
|
||||
)
|
||||
return -1 # unreachable; die() never returns
|
||||
|
||||
|
||||
def stop_bundle(slug: str) -> None:
|
||||
|
||||
@@ -27,13 +27,11 @@ from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Mapping, Sequence
|
||||
|
||||
|
||||
|
||||
_SMOLVM = "smolvm"
|
||||
|
||||
|
||||
@@ -52,7 +50,7 @@ class SmolvmError(RuntimeError):
|
||||
pack failed, etc.). Carries the captured stderr for the
|
||||
operator-facing log line."""
|
||||
|
||||
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess[str]):
|
||||
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess):
|
||||
self.argv = list(argv)
|
||||
self.returncode = result.returncode
|
||||
self.stdout = result.stdout
|
||||
@@ -65,7 +63,7 @@ class SmolvmError(RuntimeError):
|
||||
|
||||
|
||||
def _smolvm(*args: str, env: Mapping[str, str] | None = None,
|
||||
check: bool = True) -> subprocess.CompletedProcess[str]:
|
||||
check: bool = True) -> subprocess.CompletedProcess:
|
||||
"""One subprocess call into the smolvm CLI. `check=True`
|
||||
raises SmolvmError on non-zero; `check=False` returns the
|
||||
CompletedProcess for the caller to inspect."""
|
||||
@@ -199,34 +197,6 @@ def machine_exec(
|
||||
)
|
||||
|
||||
|
||||
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
||||
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
|
||||
|
||||
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
|
||||
channel needs a brief warm-up before back-to-back exec calls are
|
||||
safe. Polling exits as soon as the channel is ready and fails
|
||||
loudly if the VM never responds."""
|
||||
deadline = time.monotonic() + timeout
|
||||
delay = 0.1
|
||||
while time.monotonic() < deadline:
|
||||
r = machine_exec(name, ["true"])
|
||||
if r.returncode == 0:
|
||||
return
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
break
|
||||
time.sleep(min(delay, remaining))
|
||||
delay = min(delay * 2, 0.5)
|
||||
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
|
||||
raise SmolvmError(
|
||||
argv,
|
||||
subprocess.CompletedProcess(
|
||||
args=argv, returncode=-1, stdout="",
|
||||
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def machine_cp(src: str, dst: str) -> None:
|
||||
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
||||
reference a path inside the VM, bare path for the host. Both
|
||||
|
||||
@@ -21,9 +21,7 @@ def smolmachines_preflight() -> None:
|
||||
die(
|
||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
||||
"PATH. Install with: "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh. "
|
||||
"To use the legacy Docker backend instead, set "
|
||||
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh"
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1,82 +0,0 @@
|
||||
"""Terminal escape-sequence helpers shared across all bottle backends."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
|
||||
|
||||
# color name → (normal_idx, normal_hex, bright_idx, bright_hex, dark_bg_hex)
|
||||
# OSC 4 sets indexed palette entries (affects syntax-highlighted code and any
|
||||
# TUI content that uses indexed colors). dark_bg_hex is used for OSC 11
|
||||
# (default background) — a very dark tint that's visible even when the TUI
|
||||
# uses true/24-bit colors for its own chrome, which would otherwise bypass
|
||||
# the palette entirely.
|
||||
_COLORS: dict[str, tuple[int, str, int, str, str]] = {
|
||||
"black": (0, "#2d2d2d", 8, "#5c5c5c", "#0a0a0a"),
|
||||
"red": (1, "#c0392b", 9, "#e74c3c", "#1a0707"),
|
||||
"green": (2, "#27ae60", 10, "#2ecc71", "#071a09"),
|
||||
"yellow": (3, "#d4ac0d", 11, "#f1c40f", "#1a1507"),
|
||||
"blue": (4, "#2471a3", 12, "#3498db", "#07071a"),
|
||||
"magenta": (5, "#7d3c98", 13, "#9b59b6", "#12071a"),
|
||||
"cyan": (6, "#148f77", 14, "#1abc9c", "#071a1a"),
|
||||
"white": (7, "#bdc3c7", 15, "#ecf0f1", "#111111"),
|
||||
"bright-black": (8, "#5c5c5c", 0, "#2d2d2d", "#111111"),
|
||||
"bright-red": (9, "#e74c3c", 1, "#c0392b", "#200808"),
|
||||
"bright-green": (10, "#2ecc71", 2, "#27ae60", "#082008"),
|
||||
"bright-yellow": (11, "#f1c40f", 3, "#d4ac0d", "#201808"),
|
||||
"bright-blue": (12, "#3498db", 4, "#2471a3", "#080820"),
|
||||
"bright-magenta": (13, "#9b59b6", 5, "#7d3c98", "#160820"),
|
||||
"bright-cyan": (14, "#1abc9c", 6, "#148f77", "#082020"),
|
||||
"bright-white": (15, "#ecf0f1", 7, "#bdc3c7", "#151515"),
|
||||
}
|
||||
|
||||
# OSC 104 resets all indexed palette entries; OSC 111 resets default background.
|
||||
_RESET_PRINTF = "printf '\\033]104\\007\\033]111\\007'"
|
||||
|
||||
|
||||
def palette_printf(color: str) -> str:
|
||||
"""Shell `printf` command that emits OSC 4 + OSC 11 to tint the terminal
|
||||
for *color*: sets the normal/bright palette entries AND the default
|
||||
background to a dark shade of that color. Returns '' if unknown."""
|
||||
entry = _COLORS.get(color)
|
||||
if not entry:
|
||||
return ""
|
||||
n_idx, n_hex, b_idx, b_hex, bg_hex = entry
|
||||
seq = (
|
||||
f"\\033]4;{n_idx};{n_hex}\\007"
|
||||
f"\\033]4;{b_idx};{b_hex}\\007"
|
||||
f"\\033]11;{bg_hex}\\007"
|
||||
)
|
||||
return f"printf '{seq}'"
|
||||
|
||||
|
||||
def exec_shell_script(
|
||||
agent_argv: list[str],
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
) -> str | None:
|
||||
"""Build a shell script string that optionally sets the terminal
|
||||
title and/or palette before running *agent_argv*, and resets the
|
||||
palette + background on exit. Returns None when no decoration is
|
||||
needed — callers should run *agent_argv* directly in that case."""
|
||||
title_cmd = (
|
||||
f"printf '\\033]0;%s\\007' {shlex.quote(terminal_title)}"
|
||||
if terminal_title else ""
|
||||
)
|
||||
pal_cmd = palette_printf(terminal_color)
|
||||
|
||||
if not title_cmd and not pal_cmd:
|
||||
return None
|
||||
|
||||
parts: list[str] = []
|
||||
if title_cmd:
|
||||
parts.append(title_cmd)
|
||||
if pal_cmd:
|
||||
parts.append(pal_cmd)
|
||||
parts.append(shlex.join(agent_argv))
|
||||
parts.append(_RESET_PRINTF)
|
||||
else:
|
||||
# No palette change — exec so the agent replaces the shell.
|
||||
parts.append(f"exec {shlex.join(agent_argv)}")
|
||||
|
||||
return "; ".join(parts)
|
||||
+27
-11
@@ -14,6 +14,7 @@ from ..log import die, info
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..egress import EgressPlan
|
||||
from ..pipelock import PipelockProxyPlan
|
||||
|
||||
|
||||
# Debian-family CA layout, shared by every backend (all guest images
|
||||
@@ -34,20 +35,35 @@ def host_skill_dir(name: str) -> str:
|
||||
return f"{home}/.claude/skills/{name}"
|
||||
|
||||
|
||||
def select_ca_cert(egress_plan: EgressPlan) -> tuple[Path, str]:
|
||||
"""Return the egress MITM CA cert path and label for provision_ca.
|
||||
def select_ca_cert(
|
||||
egress_plan: EgressPlan, proxy_plan: PipelockProxyPlan
|
||||
) -> tuple[Path, str]:
|
||||
"""Pick the agent-facing CA cert (and a short label for the log
|
||||
line) that matches the proxy the agent's HTTP_PROXY points at.
|
||||
Egress wins when the bottle declares any routes (it sits in front
|
||||
of pipelock); else pipelock.
|
||||
|
||||
Launch always mints the CA and re-binds the host path into the
|
||||
egress_plan before provision runs, so an empty/missing path here
|
||||
means launch's bringup is broken — fatal."""
|
||||
cert = egress_plan.mitmproxy_ca_cert_only_host_path
|
||||
if cert == Path() or not cert.is_file():
|
||||
Shared by every backend's `provision_ca`: launch mints the chosen
|
||||
CA(s) and re-binds their host paths into these inner plans before
|
||||
provision runs, so an empty/missing path here means launch's
|
||||
bringup is broken — fatal."""
|
||||
if egress_plan.routes:
|
||||
cert = egress_plan.mitmproxy_ca_cert_only_host_path
|
||||
if cert == Path() or not cert.is_file():
|
||||
die(
|
||||
f"egress CA cert missing at {cert or '(empty)'}; "
|
||||
f"launch must have called egress_tls_init and "
|
||||
f"re-bound the plan before provision"
|
||||
)
|
||||
return cert, "egress"
|
||||
cert = proxy_plan.ca_cert_host_path
|
||||
if not cert or not cert.is_file():
|
||||
die(
|
||||
f"egress CA cert missing at {cert or '(empty)'}; "
|
||||
f"launch must have called egress_tls_init and "
|
||||
f"re-bound the plan before provision"
|
||||
f"pipelock CA cert missing at {cert or '(empty)'}; "
|
||||
f"launch must have called pipelock_tls_init and re-bound "
|
||||
f"the plan before provision"
|
||||
)
|
||||
return cert, "egress"
|
||||
return cert, "pipelock"
|
||||
|
||||
|
||||
def log_ca_fingerprint(cert_host_path: Path, label: str) -> None:
|
||||
|
||||
@@ -1,35 +1,34 @@
|
||||
"""Main CLI dispatcher.
|
||||
|
||||
Commands: cleanup, edit, info, init, list, resume, start, supervise
|
||||
Commands: cleanup, dashboard, edit, info, init, list, resume, start
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
from ..log import Die, die, error
|
||||
from ..manifest import ManifestError
|
||||
from ..log import Die, die
|
||||
from ._common import PROG
|
||||
from . import list as _list_mod
|
||||
from .cleanup import cmd_cleanup
|
||||
from .dashboard import cmd_dashboard
|
||||
from .edit import cmd_edit
|
||||
from .info import cmd_info
|
||||
from .init import cmd_init
|
||||
from .resume import cmd_resume
|
||||
from .start import cmd_start
|
||||
from .supervise import cmd_supervise
|
||||
|
||||
cmd_list = _list_mod.cmd_list
|
||||
|
||||
COMMANDS = {
|
||||
"cleanup": cmd_cleanup,
|
||||
"dashboard": cmd_dashboard,
|
||||
"edit": cmd_edit,
|
||||
"info": cmd_info,
|
||||
"init": cmd_init,
|
||||
"list": cmd_list,
|
||||
"resume": cmd_resume,
|
||||
"start": cmd_start,
|
||||
"supervise": cmd_supervise,
|
||||
}
|
||||
|
||||
|
||||
@@ -37,22 +36,13 @@ def usage() -> None:
|
||||
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
||||
sys.stderr.write("Commands:\n")
|
||||
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
||||
sys.stderr.write(" dashboard view + approve/modify/reject pending supervise proposals (PRD 0013)\n")
|
||||
sys.stderr.write(" edit open an agent in vim for editing\n")
|
||||
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
||||
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
||||
sys.stderr.write(" list list available agents or active containers\n")
|
||||
sys.stderr.write(
|
||||
" resume re-launch a bottle by its identity "
|
||||
"(continues state from PRD 0016)\n"
|
||||
)
|
||||
sys.stderr.write(
|
||||
" start boot a container for a named agent and "
|
||||
"attach an interactive session\n"
|
||||
)
|
||||
sys.stderr.write(
|
||||
" supervise view + approve/modify/reject pending supervise "
|
||||
"proposals (PRD 0013)\n\n"
|
||||
)
|
||||
sys.stderr.write(" resume re-launch a bottle by its identity (continues state from PRD 0016)\n")
|
||||
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n\n")
|
||||
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
|
||||
|
||||
|
||||
@@ -73,11 +63,6 @@ def main(argv: list[str] | None = None) -> int:
|
||||
die(f"unknown command: {command}")
|
||||
try:
|
||||
return handler(rest) or 0
|
||||
except ManifestError as e:
|
||||
# Manifest/config problems surface as a catchable exception;
|
||||
# print the reason and exit non-zero (same UX die() used to give).
|
||||
error(str(e))
|
||||
return 1
|
||||
except Die as e:
|
||||
return e.code if isinstance(e.code, int) else 1
|
||||
except KeyboardInterrupt:
|
||||
|
||||
@@ -14,7 +14,7 @@ REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
|
||||
def read_tty_line() -> str:
|
||||
"""Mirror `IFS= read -r REPLY </dev/tty`. Falls back to stdin."""
|
||||
try:
|
||||
with open("/dev/tty", "r", encoding="utf-8") as tty:
|
||||
with open("/dev/tty", "r") as tty:
|
||||
return tty.readline().rstrip("\n")
|
||||
except OSError:
|
||||
return sys.stdin.readline().rstrip("\n")
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+5
-18
@@ -51,8 +51,7 @@ def cmd_init(argv: list[str]) -> int:
|
||||
die(f"{target_file} exists but is not valid JSON; fix or remove it first")
|
||||
if agent_name in (existing.get("agents") or {}):
|
||||
sys.stderr.write(
|
||||
f'bot-bottle: agent "{agent_name}" already exists in '
|
||||
f'{target_file}. Overwrite? [y/N] '
|
||||
f'bot-bottle: agent "{agent_name}" already exists in {target_file}. Overwrite? [y/N] '
|
||||
)
|
||||
sys.stderr.flush()
|
||||
ow = read_tty_line()
|
||||
@@ -72,10 +71,7 @@ def cmd_init(argv: list[str]) -> int:
|
||||
|
||||
# Prompt
|
||||
print(file=sys.stderr)
|
||||
info(
|
||||
"System prompt — enter text, then a lone '.' on its own line to "
|
||||
"finish (just '.' to leave empty):"
|
||||
)
|
||||
info("System prompt — enter text, then a lone '.' on its own line to finish (just '.' to leave empty):")
|
||||
prompt_lines: list[str] = []
|
||||
while True:
|
||||
line = read_tty_line()
|
||||
@@ -103,10 +99,7 @@ def cmd_init(argv: list[str]) -> int:
|
||||
|
||||
if bottle_name in (existing.get("bottles") or {}):
|
||||
bottle_exists_already = True
|
||||
info(
|
||||
f"Bottle '{bottle_name}' already exists in {target_file}; "
|
||||
f"agent will reference it."
|
||||
)
|
||||
info(f"Bottle '{bottle_name}' already exists in {target_file}; agent will reference it.")
|
||||
else:
|
||||
info(f"Creating new bottle '{bottle_name}'.")
|
||||
bottle_env = _prompt_for_env_vars()
|
||||
@@ -138,14 +131,8 @@ def cmd_init(argv: list[str]) -> int:
|
||||
|
||||
def _prompt_for_env_vars() -> dict[str, str]:
|
||||
print(file=sys.stderr)
|
||||
info(
|
||||
"Env vars — enter each var name then its mode. Press Enter with "
|
||||
"no name to finish."
|
||||
)
|
||||
info(
|
||||
" Modes: secret (prompt at runtime) | interpolated (read from "
|
||||
"host env) | literal (hardcoded value)"
|
||||
)
|
||||
info("Env vars — enter each var name then its mode. Press Enter with no name to finish.")
|
||||
info(" Modes: secret (prompt at runtime) | interpolated (read from host env) | literal (hardcoded value)")
|
||||
out: dict[str, str] = {}
|
||||
while True:
|
||||
print(file=sys.stderr)
|
||||
|
||||
+5
-40
@@ -3,47 +3,12 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
from ..backend import enumerate_active_agents
|
||||
from ..manifest import Manifest
|
||||
from ._common import PROG, USER_CWD
|
||||
|
||||
_ANSI_COLOR_CODES: dict[str, str] = {
|
||||
"black": "\033[30m",
|
||||
"red": "\033[31m",
|
||||
"green": "\033[32m",
|
||||
"yellow": "\033[33m",
|
||||
"blue": "\033[34m",
|
||||
"magenta": "\033[35m",
|
||||
"cyan": "\033[36m",
|
||||
"white": "\033[37m",
|
||||
"bright-black": "\033[90m",
|
||||
"bright-red": "\033[91m",
|
||||
"bright-green": "\033[92m",
|
||||
"bright-yellow": "\033[93m",
|
||||
"bright-blue": "\033[94m",
|
||||
"bright-magenta": "\033[95m",
|
||||
"bright-cyan": "\033[96m",
|
||||
"bright-white": "\033[97m",
|
||||
}
|
||||
_ANSI_RESET = "\033[0m"
|
||||
|
||||
|
||||
def _ansi_label(text: str, color: str) -> str:
|
||||
if not color:
|
||||
return text
|
||||
if not sys.stdout.isatty():
|
||||
return text
|
||||
term = os.environ.get("TERM", "")
|
||||
if term in ("dumb", ""):
|
||||
return text
|
||||
code = _ANSI_COLOR_CODES.get(color)
|
||||
if not code:
|
||||
return text
|
||||
return f"{code}{text}{_ANSI_RESET}"
|
||||
|
||||
|
||||
def cmd_list(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} list", add_help=True)
|
||||
@@ -62,11 +27,11 @@ def cmd_list(argv: list[str]) -> int:
|
||||
if not active:
|
||||
print("no active bot-bottle bottles", file=sys.stderr)
|
||||
return 0
|
||||
# One line per bottle: `<backend>\t<slug>\t<label>\t<services>`.
|
||||
# Tab-separated keeps the format stable for shell pipelines.
|
||||
# One line per bottle: `<backend>\t<slug>\t<agent>\t<status>`.
|
||||
# Tab-separated keeps the format stable for shell pipelines;
|
||||
# the dashboard renders the same data through its own
|
||||
# formatter.
|
||||
for b in active:
|
||||
services = ",".join(b.services) if b.services else "-"
|
||||
display_name = b.label if b.label else b.agent_name
|
||||
colored_name = _ansi_label(display_name, b.color)
|
||||
print(f"{b.backend_name}\t{b.slug}\t{colored_name}\t{services}")
|
||||
print(f"{b.backend_name}\t{b.slug}\t{b.agent_name}\t{services}")
|
||||
return 0
|
||||
|
||||
@@ -18,7 +18,7 @@ from __future__ import annotations
|
||||
import argparse
|
||||
|
||||
from ..backend import BottleSpec
|
||||
from ..bottle_state import read_metadata
|
||||
from ..backend.docker.bottle_state import read_metadata
|
||||
from ..log import die
|
||||
from ..manifest import Manifest
|
||||
from ._common import PROG, USER_CWD
|
||||
@@ -52,10 +52,8 @@ def cmd_resume(argv: list[str]) -> int:
|
||||
user_cwd=metadata.cwd or USER_CWD,
|
||||
identity=metadata.identity,
|
||||
)
|
||||
backend_name = metadata.backend or None
|
||||
return _launch_bottle(
|
||||
spec,
|
||||
dry_run=args.dry_run,
|
||||
remote_control=args.remote_control,
|
||||
backend_name=backend_name,
|
||||
)
|
||||
|
||||
+33
-44
@@ -2,8 +2,10 @@
|
||||
interactive claude-code session. The container is torn down when the
|
||||
session ends.
|
||||
|
||||
The launch core is shared with `cli.py resume <identity>` through
|
||||
the private orchestrator `_launch_bottle`.
|
||||
The launch core is shared with `cli.py resume <identity>` and (PRD
|
||||
0020 chunk 1+) the dashboard's in-process start flow: see the
|
||||
public helpers `prepare_with_preflight`, `attach_agent`, and the
|
||||
private orchestrator `_launch_bottle`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -24,22 +26,21 @@ from ..backend import (
|
||||
known_backend_names,
|
||||
)
|
||||
from ..backend.docker.bottle_plan import DockerBottlePlan
|
||||
from ..bottle_state import (
|
||||
from ..backend.docker.bottle_state import (
|
||||
cleanup_state,
|
||||
is_preserved,
|
||||
mark_preserved,
|
||||
)
|
||||
# from ..backend.docker.capability_apply import snapshot_transcript
|
||||
from ..backend.docker.capability_apply import snapshot_transcript
|
||||
from ..log import info
|
||||
from ..manifest import Manifest
|
||||
from ._common import PROG, USER_CWD, read_tty_line
|
||||
from . import tui
|
||||
|
||||
|
||||
def cmd_start(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
|
||||
parser.add_argument("--cwd", action="store_true", help="copy host cwd into a derived image")
|
||||
parser.add_argument("--remote-control", action="store_true")
|
||||
parser.add_argument(
|
||||
"--backend",
|
||||
@@ -47,51 +48,30 @@ def cmd_start(argv: list[str]) -> int:
|
||||
default=None,
|
||||
help=(
|
||||
"backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND "
|
||||
"or host auto-selection). Overrides the env var when set."
|
||||
"or 'docker'). Overrides the env var when set."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"name",
|
||||
nargs="?",
|
||||
default=None,
|
||||
help="agent name defined in bot-bottle.json (omit to pick interactively)",
|
||||
)
|
||||
parser.add_argument("name", help="agent name defined in bot-bottle.json")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
|
||||
|
||||
manifest = Manifest.resolve(USER_CWD)
|
||||
|
||||
agent_name: str | None = args.name
|
||||
if agent_name is None:
|
||||
agent_name = tui.filter_select(
|
||||
sorted(manifest.agents.keys()),
|
||||
title="Select agent",
|
||||
)
|
||||
if agent_name is None:
|
||||
return 0
|
||||
|
||||
backend_name: str | None = args.backend
|
||||
|
||||
label, color = tui.name_color_modal(default_label=agent_name)
|
||||
|
||||
spec = BottleSpec(
|
||||
manifest=manifest,
|
||||
agent_name=agent_name,
|
||||
agent_name=args.name,
|
||||
copy_cwd=args.cwd,
|
||||
user_cwd=USER_CWD,
|
||||
label=label,
|
||||
color=color,
|
||||
)
|
||||
return _launch_bottle(
|
||||
spec,
|
||||
dry_run=dry_run,
|
||||
remote_control=args.remote_control,
|
||||
backend_name=backend_name,
|
||||
backend_name=args.backend,
|
||||
)
|
||||
|
||||
|
||||
# --- Launch helpers ------------------------------------------------------
|
||||
# --- Public helpers shared with the dashboard (PRD 0020) -----------------
|
||||
|
||||
|
||||
def prepare_with_preflight(
|
||||
@@ -104,11 +84,14 @@ def prepare_with_preflight(
|
||||
backend_name: str | None = None,
|
||||
) -> tuple[DockerBottlePlan | None, str]:
|
||||
"""Run `backend.prepare`, render the preflight summary via the
|
||||
injected callable, prompt y/N via the injected callable.
|
||||
injected callable, prompt y/N via the injected callable. The CLI
|
||||
binds these to stderr/stdin; the dashboard binds them to a
|
||||
curses modal.
|
||||
|
||||
`backend_name` selects which backend prepares the plan
|
||||
(`None` → `$BOT_BOTTLE_BACKEND` → host auto-selection). The CLI
|
||||
passes whatever `--backend` resolved to.
|
||||
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). Dashboard
|
||||
passes the value from its new-agent backend-picker modal; the
|
||||
CLI passes whatever `--backend` resolved to.
|
||||
|
||||
Returns `(plan, identity)`. `plan` is None on dry-run or
|
||||
operator-N, but `identity` is set as soon as `backend.prepare`
|
||||
@@ -133,17 +116,22 @@ def prepare_with_preflight(
|
||||
def attach_agent(
|
||||
bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
|
||||
agent_provider_template: str = "claude",
|
||||
startup_args: tuple[str, ...] = (),
|
||||
) -> int:
|
||||
"""Run the selected provider CLI inside `bottle` as an
|
||||
interactive session. Blocks until the session ends; returns the
|
||||
agent process's exit code.
|
||||
|
||||
`resume=True` adds `--continue` so claude picks up its most
|
||||
recent session non-interactively (no session-picker prompt).
|
||||
First-attach paths (`./cli.py start`) leave it False.
|
||||
recent session non-interactively (no session-picker prompt) —
|
||||
the right shape for the dashboard's Enter re-attach (PRD 0020
|
||||
chunk 3), where a bottle typically has exactly one session.
|
||||
First-attach paths (`./cli.py start`, the dashboard's new-agent
|
||||
flow) leave it False.
|
||||
|
||||
Used as the inner step of `./cli.py start`."""
|
||||
Used as the inner step of `./cli.py start` (one-shot) and by the
|
||||
dashboard, which calls it from inside a `curses.endwin → … →
|
||||
stdscr.refresh()` handoff so the curses surface gets out of the
|
||||
terminal's way while the agent has it."""
|
||||
runtime = runtime_for(agent_provider_template)
|
||||
info(
|
||||
f"attaching interactive {agent_provider_template} session "
|
||||
@@ -152,7 +140,6 @@ def attach_agent(
|
||||
agent_args = list(runtime.bypass_args)
|
||||
if remote_control:
|
||||
agent_args.extend(runtime.remote_control_args)
|
||||
agent_args.extend(startup_args)
|
||||
if resume:
|
||||
agent_args.extend(runtime.resume_args)
|
||||
return bottle.exec_agent(agent_args, tty=True)
|
||||
@@ -161,20 +148,23 @@ def attach_agent(
|
||||
def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
||||
"""Inside the launch context, while the container is still
|
||||
alive: snapshot the transcript and mark for preservation if
|
||||
claude crashed."""
|
||||
claude crashed. Public for the dashboard's death-handling path
|
||||
(PRD 0020 open question 3)."""
|
||||
# FIXME: this captures Claude-specific session state. A follow-up
|
||||
# spike should explore freezing provider-neutral container state
|
||||
# instead of relying on each agent's transcript layout.
|
||||
if not identity:
|
||||
return
|
||||
# snapshot_transcript(identity)
|
||||
snapshot_transcript(identity)
|
||||
if exit_code != 0:
|
||||
mark_preserved(identity)
|
||||
|
||||
|
||||
def settle_state(identity: str) -> None:
|
||||
"""Post-teardown housekeeping: print the resume hint if the
|
||||
state was preserved, otherwise reap the per-bottle state dir."""
|
||||
state was preserved, otherwise reap the per-bottle state dir.
|
||||
Public so the dashboard's explicit-stop path calls the same
|
||||
settlement the CLI uses on context exit."""
|
||||
if not identity:
|
||||
return
|
||||
if is_preserved(identity):
|
||||
@@ -237,7 +227,6 @@ def _launch_bottle(
|
||||
bottle,
|
||||
remote_control=remote_control,
|
||||
agent_provider_template=agent_provider_template,
|
||||
startup_args=plan.agent_provision.startup_args,
|
||||
)
|
||||
info(
|
||||
f"session ended (exit {exit_code}); "
|
||||
|
||||
@@ -1,519 +0,0 @@
|
||||
"""supervise: list pending supervise proposals across all bottles and
|
||||
act on them (approve / modify / reject).
|
||||
|
||||
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
|
||||
approval handler wires to PRD 0016 (capability-block), which rebuilds
|
||||
the bottle Dockerfile. The egress-block tool was removed in issue #198.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import curses
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import traceback
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from .. import supervise as _supervise
|
||||
# from ..bottle_state import read_metadata
|
||||
# from ..backend.docker.capability_apply import (
|
||||
# CapabilityApplyError,
|
||||
# apply_capability_change,
|
||||
# )
|
||||
from ..log import Die, error, info
|
||||
|
||||
|
||||
class CapabilityApplyError(RuntimeError):
|
||||
"""Placeholder while capability_apply is disabled."""
|
||||
|
||||
from ..supervise import (
|
||||
COMPONENT_FOR_TOOL,
|
||||
AuditEntry,
|
||||
Proposal,
|
||||
Response,
|
||||
STATUS_APPROVED,
|
||||
STATUS_MODIFIED,
|
||||
STATUS_REJECTED,
|
||||
TOOL_CAPABILITY_BLOCK,
|
||||
archive_proposal,
|
||||
list_pending_proposals,
|
||||
render_diff,
|
||||
write_audit_entry,
|
||||
write_response,
|
||||
)
|
||||
from ._common import PROG
|
||||
|
||||
|
||||
_REFRESH_INTERVAL_MS = 1000
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QueuedProposal:
|
||||
"""A pending proposal plus the queue dir it was found in."""
|
||||
|
||||
proposal: Proposal
|
||||
queue_dir: Path
|
||||
|
||||
|
||||
# Errors any remediation engine may raise. Caught by the TUI key
|
||||
# handlers and surfaced in the status line so a failed apply keeps
|
||||
# the proposal pending rather than crashing curses.
|
||||
ApplyError = (CapabilityApplyError,)
|
||||
|
||||
|
||||
def discover_pending() -> list[QueuedProposal]:
|
||||
"""Walk ~/.bot-bottle/queue/* and collect pending proposals."""
|
||||
queue_root = _supervise.bot_bottle_root() / "queue"
|
||||
if not queue_root.is_dir():
|
||||
return []
|
||||
out: list[QueuedProposal] = []
|
||||
for slug_dir in sorted(queue_root.iterdir()):
|
||||
if not slug_dir.is_dir():
|
||||
continue
|
||||
for proposal in list_pending_proposals(slug_dir):
|
||||
out.append(QueuedProposal(proposal=proposal, queue_dir=slug_dir))
|
||||
out.sort(key=lambda q: q.proposal.arrival_timestamp)
|
||||
return out
|
||||
|
||||
|
||||
def _approval_status(qp: QueuedProposal, verb: str) -> str:
|
||||
"""Status-line text after a successful approval."""
|
||||
base = f"{verb} {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
||||
return f"{base}; resume: ./cli.py resume {qp.proposal.bottle_slug}"
|
||||
|
||||
|
||||
def _detail_lines(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
green_attr: int = 0,
|
||||
) -> list[tuple[str, int]]:
|
||||
"""Return the detail-view body as (text, curses-attr) tuples."""
|
||||
p = qp.proposal
|
||||
out: list[tuple[str, int]] = [
|
||||
(f"bottle: {p.bottle_slug}", 0),
|
||||
(f"tool: {p.tool}", 0),
|
||||
(f"id: {p.id}", 0),
|
||||
(f"arrived: {p.arrival_timestamp}", 0),
|
||||
(f"queue: {qp.queue_dir}", 0),
|
||||
("", 0),
|
||||
("justification:", 0),
|
||||
]
|
||||
out.extend((" " + line, 0) for line in p.justification.splitlines() or [""])
|
||||
out.extend([
|
||||
("", 0),
|
||||
("proposed file:", 0),
|
||||
])
|
||||
out.extend((line, 0) for line in p.proposed_file.splitlines() or [""])
|
||||
return out
|
||||
|
||||
|
||||
def _suffix_for_tool(tool: str) -> str:
|
||||
if tool == TOOL_CAPABILITY_BLOCK:
|
||||
return ".dockerfile"
|
||||
return ".txt"
|
||||
|
||||
|
||||
# --- Operator actions ------------------------------------------------------
|
||||
|
||||
|
||||
def approve(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
notes: str = "",
|
||||
final_file: str | None = None,
|
||||
) -> None:
|
||||
"""Apply the proposal, write the waiting response, and audit it."""
|
||||
status = STATUS_MODIFIED if final_file is not None else STATUS_APPROVED
|
||||
|
||||
diff_before, diff_after = "", ""
|
||||
# if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
# _meta = read_metadata(qp.proposal.bottle_slug)
|
||||
# if _meta is not None and not _meta.compose_project:
|
||||
# raise CapabilityApplyError(
|
||||
# "capability-block remediation is not supported for smolmachines "
|
||||
# "bottles. Reject this proposal or handle the capability change "
|
||||
# "manually, then restart the bottle."
|
||||
# )
|
||||
# diff_before, diff_after = apply_capability_change(
|
||||
# qp.proposal.bottle_slug, file_to_apply,
|
||||
# )
|
||||
|
||||
response = Response(
|
||||
proposal_id=qp.proposal.id,
|
||||
status=status,
|
||||
notes=notes,
|
||||
final_file=final_file,
|
||||
)
|
||||
write_response(qp.queue_dir, response)
|
||||
_write_audit(
|
||||
qp, action=status, notes=notes,
|
||||
diff_before=diff_before, diff_after=diff_after,
|
||||
)
|
||||
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
archive_proposal(qp.queue_dir, qp.proposal.id)
|
||||
|
||||
|
||||
def reject(qp: QueuedProposal, *, reason: str) -> None:
|
||||
"""Write a rejection response and an audit entry."""
|
||||
response = Response(
|
||||
proposal_id=qp.proposal.id,
|
||||
status=STATUS_REJECTED,
|
||||
notes=reason,
|
||||
final_file=None,
|
||||
)
|
||||
write_response(qp.queue_dir, response)
|
||||
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
|
||||
|
||||
|
||||
def _write_audit(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
action: str,
|
||||
notes: str,
|
||||
diff_before: str,
|
||||
diff_after: str,
|
||||
) -> None:
|
||||
"""Audit log for egress tool."""
|
||||
component = COMPONENT_FOR_TOOL.get(qp.proposal.tool)
|
||||
if component is None:
|
||||
return
|
||||
write_audit_entry(AuditEntry(
|
||||
timestamp=datetime.now(timezone.utc).isoformat(),
|
||||
bottle_slug=qp.proposal.bottle_slug,
|
||||
component=component,
|
||||
operator_action=action,
|
||||
operator_notes=notes,
|
||||
justification=qp.proposal.justification,
|
||||
diff=render_diff(diff_before, diff_after, label=component),
|
||||
))
|
||||
|
||||
|
||||
# --- $EDITOR integration --------------------------------------------------
|
||||
|
||||
|
||||
def edit_in_editor(content: str, *, suffix: str = ".tmp") -> str | None:
|
||||
"""Open `content` in $EDITOR and return edited content, if changed."""
|
||||
editor = os.environ.get("EDITOR", "vim")
|
||||
with tempfile.NamedTemporaryFile(
|
||||
mode="w", suffix=suffix, delete=False, prefix="supervise-modify.",
|
||||
) as f:
|
||||
f.write(content)
|
||||
path = f.name
|
||||
try:
|
||||
subprocess.run([editor, path], check=False)
|
||||
with open(path, encoding="utf-8") as f:
|
||||
edited = f.read()
|
||||
return edited if edited != content else None
|
||||
finally:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# --- TUI -------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_supervise(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} supervise", add_help=True)
|
||||
parser.add_argument(
|
||||
"--once", action="store_true",
|
||||
help="list pending proposals once and exit (no TUI)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.once:
|
||||
return _list_once()
|
||||
try:
|
||||
curses.wrapper(_main_loop)
|
||||
except KeyboardInterrupt:
|
||||
return 130
|
||||
except Die as e:
|
||||
if e.message:
|
||||
error(e.message)
|
||||
else:
|
||||
error("supervise exited on a fatal error (no detail captured).")
|
||||
return e.code if isinstance(e.code, int) else 1
|
||||
except Exception as e: # noqa: W0718 — catch supervise crash for logging
|
||||
log_path = _write_crash_log(e)
|
||||
error(f"supervise crashed: {type(e).__name__}: {e}")
|
||||
error(f"full traceback written to {log_path}")
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
def _write_crash_log(exc: BaseException) -> Path:
|
||||
"""Persist `exc`'s traceback to a stable file under ~/.bot-bottle/."""
|
||||
stamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
body = "".join(
|
||||
traceback.format_exception(type(exc), exc, exc.__traceback__)
|
||||
)
|
||||
entry = f"=== supervise crash {stamp} ===\n{body}\n"
|
||||
try:
|
||||
log_dir = _supervise.bot_bottle_root() / "logs"
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = log_dir / "supervise-crash.log"
|
||||
with path.open("a", encoding="utf-8") as fh:
|
||||
fh.write(entry)
|
||||
return path
|
||||
except OSError:
|
||||
fd, tmp = tempfile.mkstemp(
|
||||
prefix="bot-bottle-supervise-crash-", suffix=".log",
|
||||
)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
fh.write(entry)
|
||||
return Path(tmp)
|
||||
|
||||
|
||||
def _list_once() -> int:
|
||||
pending = discover_pending()
|
||||
if not pending:
|
||||
info("no pending proposals")
|
||||
return 0
|
||||
for qp in pending:
|
||||
sys.stdout.write(
|
||||
f"{qp.proposal.arrival_timestamp} "
|
||||
f"[{qp.proposal.bottle_slug}] "
|
||||
f"{qp.proposal.tool} "
|
||||
f"{qp.proposal.id}\n"
|
||||
)
|
||||
sys.stdout.write(f" {qp.proposal.justification}\n")
|
||||
return 0
|
||||
|
||||
|
||||
def _try_init_green() -> int:
|
||||
"""Initialise a green color pair and return its attr, or 0."""
|
||||
try:
|
||||
curses.start_color()
|
||||
curses.use_default_colors()
|
||||
curses.init_pair(1, curses.COLOR_GREEN, -1)
|
||||
return curses.color_pair(1)
|
||||
except curses.error:
|
||||
return 0
|
||||
|
||||
|
||||
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
|
||||
curses.curs_set(0)
|
||||
stdscr.timeout(_REFRESH_INTERVAL_MS)
|
||||
green_attr = _try_init_green()
|
||||
selected = 0
|
||||
status_line = ""
|
||||
seen_ids: set[str] = set()
|
||||
|
||||
while True:
|
||||
pending = discover_pending()
|
||||
if selected >= len(pending):
|
||||
selected = max(0, len(pending) - 1)
|
||||
|
||||
live_ids = {qp.proposal.id for qp in pending}
|
||||
newly_arrived = live_ids - seen_ids
|
||||
if seen_ids and newly_arrived:
|
||||
try:
|
||||
curses.beep()
|
||||
except curses.error:
|
||||
pass
|
||||
for i, qp in enumerate(pending):
|
||||
if qp.proposal.id in newly_arrived:
|
||||
selected = i
|
||||
break
|
||||
seen_ids = live_ids
|
||||
|
||||
_render(
|
||||
stdscr, pending, selected, status_line,
|
||||
green_attr=green_attr,
|
||||
)
|
||||
|
||||
try:
|
||||
key = stdscr.getch()
|
||||
except KeyboardInterrupt:
|
||||
return
|
||||
|
||||
if key == -1:
|
||||
continue
|
||||
|
||||
status_line = ""
|
||||
|
||||
if key in (ord("q"), 27):
|
||||
return
|
||||
|
||||
if not pending:
|
||||
continue
|
||||
qp = pending[selected]
|
||||
|
||||
if key in (curses.KEY_DOWN, ord("j")):
|
||||
selected = min(selected + 1, len(pending) - 1)
|
||||
elif key in (curses.KEY_UP, ord("k")):
|
||||
selected = max(selected - 1, 0)
|
||||
elif key in (curses.KEY_ENTER, 10, 13):
|
||||
_detail_view(stdscr, qp, green_attr=green_attr)
|
||||
elif key == ord("a"):
|
||||
try:
|
||||
approve(qp)
|
||||
status_line = _approval_status(qp, "approved")
|
||||
except ApplyError as e:
|
||||
status_line = f"apply failed: {e}"
|
||||
elif key == ord("m"):
|
||||
edited = _modify(stdscr, qp)
|
||||
if edited is None:
|
||||
status_line = "modify aborted (no change)"
|
||||
else:
|
||||
try:
|
||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||
status_line = _approval_status(qp, "modified+approved")
|
||||
except ApplyError as e:
|
||||
status_line = f"apply failed: {e}"
|
||||
elif key == ord("r"):
|
||||
reason = _prompt(stdscr, "reject reason: ")
|
||||
if reason:
|
||||
reject(qp, reason=reason)
|
||||
status_line = f"rejected {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
||||
else:
|
||||
status_line = "reject aborted (empty reason)"
|
||||
|
||||
|
||||
def _render(
|
||||
stdscr: "curses._CursesWindow", # type: ignore
|
||||
pending: list[QueuedProposal],
|
||||
selected: int,
|
||||
status_line: str,
|
||||
*,
|
||||
green_attr: int = 0, # noqa: F841 — unused, but required by interface
|
||||
) -> None:
|
||||
stdscr.erase()
|
||||
h, w = stdscr.getmaxyx()
|
||||
header = f"bot-bottle supervise ({len(pending)} pending)"
|
||||
stdscr.addnstr(0, 0, header, w - 1, curses.A_BOLD)
|
||||
stdscr.hline(1, 0, curses.ACS_HLINE, w)
|
||||
|
||||
row = 2
|
||||
if not pending:
|
||||
stdscr.addnstr(
|
||||
row, 2,
|
||||
"no pending proposals; agents will queue here when they call a "
|
||||
"supervise tool",
|
||||
w - 4,
|
||||
)
|
||||
else:
|
||||
for i, qp in enumerate(pending):
|
||||
if row >= h - 3:
|
||||
break
|
||||
p = qp.proposal
|
||||
ts_short = (
|
||||
p.arrival_timestamp.split("T", 1)[1][:8]
|
||||
if "T" in p.arrival_timestamp else p.arrival_timestamp
|
||||
)
|
||||
cursor = "> " if i == selected else " "
|
||||
line = (
|
||||
f"{cursor}{ts_short} "
|
||||
f"[{p.bottle_slug}] {p.tool:<18} {p.id[:8]}"
|
||||
)
|
||||
attr = curses.A_REVERSE if i == selected else curses.A_NORMAL
|
||||
stdscr.addnstr(row, 0, line, w - 1, attr)
|
||||
row += 1
|
||||
if row >= h - 3:
|
||||
break
|
||||
if p.justification:
|
||||
stdscr.addnstr(row, 4, p.justification[: max(0, w - 5)], w - 5)
|
||||
row += 1
|
||||
|
||||
footer = "[j/k] move [Enter] view [a] approve [m] modify [r] reject [q] quit"
|
||||
stdscr.hline(h - 2, 0, curses.ACS_HLINE, w)
|
||||
stdscr.addnstr(h - 1, 0, footer, w - 1, curses.A_DIM)
|
||||
if status_line:
|
||||
stdscr.addnstr(h - 3, 0, status_line, w - 1, curses.A_BOLD)
|
||||
stdscr.refresh()
|
||||
|
||||
|
||||
def _detail_view(
|
||||
stdscr: "curses._CursesWindow", # type: ignore
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
green_attr: int = 0,
|
||||
) -> None:
|
||||
"""Render the full proposal. Scrollable. Press q to return."""
|
||||
lines = _detail_lines(qp, green_attr=green_attr)
|
||||
offset = 0
|
||||
while True:
|
||||
stdscr.erase()
|
||||
h, w = stdscr.getmaxyx()
|
||||
for i, (text, attr) in enumerate(lines[offset:offset + h - 1]):
|
||||
stdscr.addnstr(i, 0, text, w - 1, attr)
|
||||
stdscr.addnstr(
|
||||
h - 1, 0,
|
||||
"[j/k] scroll [g/G] top/bottom [a] approve [m] modify [r] reject [q] back",
|
||||
w - 1, curses.A_DIM,
|
||||
)
|
||||
stdscr.refresh()
|
||||
key = stdscr.getch()
|
||||
if key in (ord("q"), 27):
|
||||
return
|
||||
if key in (curses.KEY_DOWN, ord("j")):
|
||||
offset = min(offset + 1, max(0, len(lines) - 1))
|
||||
elif key in (curses.KEY_UP, ord("k")):
|
||||
offset = max(offset - 1, 0)
|
||||
elif key == ord("g"):
|
||||
offset = 0
|
||||
elif key == ord("G"):
|
||||
offset = max(0, len(lines) - 1)
|
||||
elif key == ord("a"):
|
||||
try:
|
||||
approve(qp)
|
||||
except ApplyError:
|
||||
pass
|
||||
return
|
||||
elif key == ord("m"):
|
||||
edited = _modify(stdscr, qp)
|
||||
if edited is not None:
|
||||
try:
|
||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||
except ApplyError:
|
||||
pass
|
||||
return
|
||||
elif key == ord("r"):
|
||||
reason = _prompt(stdscr, "reject reason: ")
|
||||
if reason:
|
||||
reject(qp, reason=reason)
|
||||
return
|
||||
|
||||
|
||||
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore
|
||||
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
|
||||
suffix = _suffix_for_tool(qp.proposal.tool)
|
||||
curses.endwin()
|
||||
try:
|
||||
edited = edit_in_editor(qp.proposal.proposed_file, suffix=suffix)
|
||||
finally:
|
||||
stdscr.refresh()
|
||||
return edited
|
||||
|
||||
|
||||
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore
|
||||
"""One-line input at the bottom of the screen."""
|
||||
curses.curs_set(1)
|
||||
h, _ = stdscr.getmaxyx()
|
||||
stdscr.move(h - 2, 0)
|
||||
stdscr.clrtoeol()
|
||||
stdscr.addstr(h - 2, 0, label)
|
||||
stdscr.refresh()
|
||||
curses.echo()
|
||||
try:
|
||||
raw = stdscr.getstr(h - 2, len(label), 200)
|
||||
finally:
|
||||
curses.noecho()
|
||||
curses.curs_set(0)
|
||||
return raw.decode("utf-8", errors="replace").strip()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"QueuedProposal",
|
||||
"approve",
|
||||
"cmd_supervise",
|
||||
"discover_pending",
|
||||
"edit_in_editor",
|
||||
"reject",
|
||||
]
|
||||
@@ -1,437 +0,0 @@
|
||||
"""tui.py — minimal curses filter-select picker for CLI prompts.
|
||||
|
||||
Exposed surface:
|
||||
|
||||
filter_select(items, *, title="", tty_path="/dev/tty") -> str | None
|
||||
name_color_modal(default_label, *, tty_path="/dev/tty") -> (str, str)
|
||||
|
||||
Opens /dev/tty directly so the picker works even when stdout/stdin are
|
||||
redirected. Returns the selected item or None on cancel.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import curses
|
||||
import os
|
||||
import sys
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
def filter_select(
|
||||
items: list[str],
|
||||
*,
|
||||
title: str = "",
|
||||
tty_path: str = "/dev/tty",
|
||||
) -> Optional[str]:
|
||||
"""Render a filter-select picker over *items*.
|
||||
|
||||
Returns the selected item string, or ``None`` if the user cancelled
|
||||
(Esc / ``q`` / Ctrl-C / Ctrl-D) or if the terminal is too small.
|
||||
|
||||
The picker opens *tty_path* directly so it works even when
|
||||
stdout/stdin are redirected.
|
||||
"""
|
||||
if not items:
|
||||
return None
|
||||
|
||||
try:
|
||||
tty_fd = open(tty_path, "r+b", buffering=0)
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
try:
|
||||
# Use os.dup() to duplicate the fd so the original file object
|
||||
# and FileIO in _run_picker each manage independent copies,
|
||||
# preventing double-close errors.
|
||||
fd_dup = os.dup(tty_fd.fileno())
|
||||
return _run_picker(items, title=title, tty_fd=fd_dup)
|
||||
finally:
|
||||
tty_fd.close()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal implementation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_KEY_ESC = 27
|
||||
_KEY_CTRL_C = 3
|
||||
_KEY_CTRL_D = 4
|
||||
_KEY_BACKSPACE_WIN = 8
|
||||
_KEY_ENTER_ALT = 10
|
||||
|
||||
_CANCEL_KEYS = frozenset([_KEY_ESC, _KEY_CTRL_C, _KEY_CTRL_D, ord("q")])
|
||||
|
||||
|
||||
def _run_picker(items: list[str], *, title: str, tty_fd: int) -> Optional[str]:
|
||||
"""Drive a curses session on *tty_fd* and return the picked item."""
|
||||
# newterm lets us run curses on an arbitrary fd rather than the
|
||||
# process's controlling tty / stdout — crucial when stdout is piped.
|
||||
os.environ.setdefault("TERM", "xterm-256color")
|
||||
|
||||
# Save / restore the real stdin/stdout so curses newterm can use tty_fd.
|
||||
orig_stdin = sys.__stdin__
|
||||
orig_stdout = sys.__stdout__
|
||||
|
||||
try:
|
||||
import io
|
||||
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode='r+'), write_through=True)
|
||||
sys.__stdin__ = tty_text # type: ignore[assignment]
|
||||
sys.__stdout__ = tty_text # type: ignore[assignment]
|
||||
|
||||
# curses.wrapper calls initscr which honours sys.__stdin__ / __stdout__
|
||||
# on some builds; use newterm where available.
|
||||
screen = curses.initscr()
|
||||
curses.noecho()
|
||||
curses.cbreak()
|
||||
screen.keypad(True)
|
||||
|
||||
try:
|
||||
result = _picker_loop(screen, items, title=title)
|
||||
finally:
|
||||
screen.keypad(False)
|
||||
curses.nocbreak()
|
||||
curses.echo()
|
||||
curses.endwin()
|
||||
except Exception: # noqa: W0718 — curses can raise many error types
|
||||
return None
|
||||
finally:
|
||||
sys.__stdin__ = orig_stdin # type: ignore[assignment]
|
||||
sys.__stdout__ = orig_stdout # type: ignore[assignment]
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _picker_loop(screen: Any, items: list[str], *, title: str) -> Optional[str]:
|
||||
query = ""
|
||||
cursor = 0
|
||||
|
||||
while True:
|
||||
filtered = _filter_items(items, query)
|
||||
|
||||
# Clamp cursor into the visible list.
|
||||
if not filtered:
|
||||
cursor = 0
|
||||
elif cursor >= len(filtered):
|
||||
cursor = len(filtered) - 1
|
||||
|
||||
try:
|
||||
_render(screen, filtered, cursor, query=query, title=title)
|
||||
except curses.error:
|
||||
# Terminal too small or write error — bail out.
|
||||
return None
|
||||
|
||||
try:
|
||||
key = screen.getch()
|
||||
except KeyboardInterrupt:
|
||||
return None
|
||||
|
||||
if key in _CANCEL_KEYS:
|
||||
return None
|
||||
|
||||
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
|
||||
return filtered[cursor] if filtered else None
|
||||
|
||||
if key in (curses.KEY_UP, ord("k")):
|
||||
if cursor > 0:
|
||||
cursor -= 1
|
||||
|
||||
elif key in (curses.KEY_DOWN, ord("j")):
|
||||
if cursor < len(filtered) - 1:
|
||||
cursor += 1
|
||||
|
||||
elif key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
|
||||
query = query[:-1]
|
||||
# After narrowing the filter, keep cursor in range.
|
||||
new_filtered = _filter_items(items, query)
|
||||
if cursor >= len(new_filtered):
|
||||
cursor = max(0, len(new_filtered) - 1)
|
||||
|
||||
elif 32 <= key <= 126:
|
||||
# Printable ASCII — append to query and reset cursor so the
|
||||
# top of the newly-filtered list is selected.
|
||||
query += chr(key)
|
||||
cursor = 0
|
||||
|
||||
|
||||
def _filter_items(items: list[str], query: str) -> list[str]:
|
||||
if not query:
|
||||
return list(items)
|
||||
q = query.lower()
|
||||
return [i for i in items if q in i.lower()]
|
||||
|
||||
|
||||
def _render(screen: Any, filtered: list[str], cursor: int, *, query: str, title: str) -> None:
|
||||
screen.erase()
|
||||
rows, cols = screen.getmaxyx()
|
||||
min_rows = 5
|
||||
|
||||
if rows < min_rows:
|
||||
raise curses.error("terminal too small")
|
||||
|
||||
row = 0
|
||||
|
||||
if title and row < rows - 1:
|
||||
_addstr_safe(screen, row, 0, title[:cols - 1], curses.A_BOLD)
|
||||
row += 1
|
||||
|
||||
filter_label = f"Filter: {query}"
|
||||
if row < rows - 1:
|
||||
_addstr_safe(screen, row, 0, filter_label[:cols - 1])
|
||||
row += 1
|
||||
|
||||
sep = "─" * min(cols - 1, 40)
|
||||
if row < rows - 1:
|
||||
_addstr_safe(screen, row, 0, sep)
|
||||
row += 1
|
||||
|
||||
list_start = row
|
||||
# Reserve two rows for separator + help line at bottom.
|
||||
list_rows = rows - list_start - 2
|
||||
if list_rows < 1:
|
||||
return
|
||||
|
||||
# Scroll window: keep cursor visible.
|
||||
scroll = max(0, cursor - list_rows + 1)
|
||||
visible = filtered[scroll: scroll + list_rows]
|
||||
|
||||
for idx, item in enumerate(visible):
|
||||
abs_idx = scroll + idx
|
||||
attr = curses.A_REVERSE if abs_idx == cursor else curses.A_NORMAL
|
||||
prefix = "> " if abs_idx == cursor else " "
|
||||
line = (prefix + item)[:cols - 1]
|
||||
if row < rows - 1:
|
||||
_addstr_safe(screen, row, 0, line, attr)
|
||||
row += 1
|
||||
|
||||
if row < rows - 1:
|
||||
_addstr_safe(screen, row, 0, sep)
|
||||
row += 1
|
||||
|
||||
help_line = "[↑↓/jk] move [Enter] select [Esc/q] cancel"
|
||||
if row < rows:
|
||||
_addstr_safe(screen, min(rows - 1, row), 0, help_line[:cols - 1])
|
||||
|
||||
screen.refresh()
|
||||
|
||||
|
||||
def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.A_NORMAL) -> None:
|
||||
try:
|
||||
screen.addstr(row, col, text, attr)
|
||||
except curses.error:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# name_color_modal — two-step label + color picker
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_ANSI_COLORS = [
|
||||
"red", "green", "blue", "yellow", "magenta", "cyan", "white", "black",
|
||||
"bright-red", "bright-green", "bright-blue", "bright-yellow",
|
||||
"bright-magenta", "bright-cyan", "bright-white", "bright-black",
|
||||
]
|
||||
|
||||
_CURSES_COLOR_MAP: dict[str, int] = {
|
||||
"black": curses.COLOR_BLACK,
|
||||
"red": curses.COLOR_RED,
|
||||
"green": curses.COLOR_GREEN,
|
||||
"yellow": curses.COLOR_YELLOW,
|
||||
"blue": curses.COLOR_BLUE,
|
||||
"magenta": curses.COLOR_MAGENTA,
|
||||
"cyan": curses.COLOR_CYAN,
|
||||
"white": curses.COLOR_WHITE,
|
||||
}
|
||||
|
||||
_COLOR_NONE = "(none)"
|
||||
|
||||
|
||||
def name_color_modal(
|
||||
default_label: str,
|
||||
*,
|
||||
tty_path: str = "/dev/tty",
|
||||
) -> tuple[str, str]:
|
||||
"""Present a two-step curses modal: first edit the agent label,
|
||||
then optionally pick a color.
|
||||
|
||||
Returns ``(label, color)`` where ``color`` is one of the 16 ANSI
|
||||
color name strings or ``""`` for no color. Falls back to
|
||||
``(default_label, "")`` on any error (terminal too small, not a tty).
|
||||
"""
|
||||
try:
|
||||
tty_fd = open(tty_path, "r+b", buffering=0) # pylint: disable=consider-using-with
|
||||
except OSError:
|
||||
return default_label, ""
|
||||
|
||||
try:
|
||||
fd_dup = os.dup(tty_fd.fileno())
|
||||
return _run_name_color(default_label, tty_fd=fd_dup)
|
||||
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
|
||||
return default_label, ""
|
||||
finally:
|
||||
tty_fd.close()
|
||||
|
||||
|
||||
def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
|
||||
import io
|
||||
orig_stdin = sys.__stdin__
|
||||
orig_stdout = sys.__stdout__
|
||||
try:
|
||||
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode="r+"), write_through=True)
|
||||
sys.__stdin__ = tty_text # type: ignore[assignment]
|
||||
sys.__stdout__ = tty_text # type: ignore[assignment]
|
||||
os.environ.setdefault("TERM", "xterm-256color")
|
||||
|
||||
screen = curses.initscr()
|
||||
curses.noecho()
|
||||
curses.cbreak()
|
||||
screen.keypad(True)
|
||||
try:
|
||||
label = _label_step(screen, default_label)
|
||||
color = _color_step(screen, label)
|
||||
finally:
|
||||
screen.keypad(False)
|
||||
curses.nocbreak()
|
||||
curses.echo()
|
||||
curses.endwin()
|
||||
finally:
|
||||
sys.__stdin__ = orig_stdin # type: ignore[assignment]
|
||||
sys.__stdout__ = orig_stdout # type: ignore[assignment]
|
||||
return label, color
|
||||
|
||||
|
||||
def _label_step(screen: Any, default_label: str) -> str:
|
||||
"""Step 1: edit the label. First printable key replaces the
|
||||
pre-fill; subsequent keys append. Enter confirms."""
|
||||
text = default_label
|
||||
replaced = False # True once the user has typed their first char
|
||||
|
||||
while True:
|
||||
_render_label(screen, text)
|
||||
try:
|
||||
key = screen.getch()
|
||||
except KeyboardInterrupt:
|
||||
return default_label
|
||||
|
||||
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
|
||||
return text.strip() or default_label
|
||||
|
||||
if key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
|
||||
if replaced:
|
||||
text = text[:-1]
|
||||
else:
|
||||
text = ""
|
||||
replaced = True
|
||||
|
||||
elif 32 <= key <= 126:
|
||||
if not replaced:
|
||||
text = chr(key)
|
||||
replaced = True
|
||||
else:
|
||||
text += chr(key)
|
||||
|
||||
|
||||
def _render_label(screen: Any, text: str) -> None:
|
||||
screen.erase()
|
||||
rows, cols = screen.getmaxyx()
|
||||
sep = "─" * min(cols - 1, 40)
|
||||
_addstr_safe(screen, 0, 0, "Name agent", curses.A_BOLD)
|
||||
_addstr_safe(screen, 1, 0, sep)
|
||||
_addstr_safe(screen, 2, 0, text[:cols - 1], curses.A_REVERSE)
|
||||
_addstr_safe(screen, 3, 0, sep)
|
||||
if rows > 5:
|
||||
_addstr_safe(screen, 5, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
|
||||
screen.refresh()
|
||||
|
||||
|
||||
def _color_step(screen: Any, confirmed_label: str) -> str:
|
||||
"""Step 2: pick a color from the list, or skip."""
|
||||
items = [_COLOR_NONE] + _ANSI_COLORS
|
||||
cursor = 0
|
||||
|
||||
# Initialise color pairs once; index 0 = none, 1..16 = palette.
|
||||
color_attrs = _init_color_pairs()
|
||||
|
||||
while True:
|
||||
_render_color(screen, items, cursor, confirmed_label, color_attrs)
|
||||
try:
|
||||
key = screen.getch()
|
||||
except KeyboardInterrupt:
|
||||
return ""
|
||||
|
||||
if key in (ord("q"), _KEY_ESC):
|
||||
return ""
|
||||
|
||||
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
|
||||
chosen = items[cursor]
|
||||
return "" if chosen == _COLOR_NONE else chosen
|
||||
|
||||
if key in (curses.KEY_UP, ord("k")) and cursor > 0:
|
||||
cursor -= 1
|
||||
elif key in (curses.KEY_DOWN, ord("j")) and cursor < len(items) - 1:
|
||||
cursor += 1
|
||||
|
||||
|
||||
def _init_color_pairs() -> dict[str, int]:
|
||||
"""Return {color_name: curses_attr} for the palette items."""
|
||||
attrs: dict[str, int] = {_COLOR_NONE: curses.A_NORMAL}
|
||||
try:
|
||||
curses.start_color()
|
||||
curses.use_default_colors()
|
||||
pair_idx = 2 # pair 1 reserved for other uses
|
||||
for name in _ANSI_COLORS:
|
||||
base = name.replace("bright-", "")
|
||||
fg = _CURSES_COLOR_MAP.get(base, curses.COLOR_WHITE)
|
||||
try:
|
||||
curses.init_pair(pair_idx, fg, -1)
|
||||
attr = curses.color_pair(pair_idx)
|
||||
if name.startswith("bright-"):
|
||||
attr |= curses.A_BOLD
|
||||
attrs[name] = attr
|
||||
pair_idx += 1
|
||||
except curses.error:
|
||||
attrs[name] = curses.A_NORMAL
|
||||
except curses.error:
|
||||
for name in _ANSI_COLORS:
|
||||
attrs[name] = curses.A_NORMAL
|
||||
return attrs
|
||||
|
||||
|
||||
def _render_color(
|
||||
screen: Any,
|
||||
items: list[str],
|
||||
cursor: int,
|
||||
confirmed_label: str,
|
||||
color_attrs: dict[str, int],
|
||||
) -> None:
|
||||
screen.erase()
|
||||
rows, cols = screen.getmaxyx()
|
||||
sep = "─" * min(cols - 1, 40)
|
||||
_addstr_safe(screen, 0, 0, "Name agent", curses.A_BOLD)
|
||||
_addstr_safe(screen, 1, 0, sep)
|
||||
_addstr_safe(screen, 2, 0, confirmed_label[:cols - 1])
|
||||
_addstr_safe(screen, 3, 0, sep)
|
||||
_addstr_safe(screen, 4, 0, "Color (optional)", curses.A_BOLD)
|
||||
|
||||
list_start = 5
|
||||
list_rows = rows - list_start - 2
|
||||
scroll = max(0, cursor - list_rows + 1)
|
||||
visible = items[scroll: scroll + list_rows]
|
||||
|
||||
for idx, name in enumerate(visible):
|
||||
abs_idx = scroll + idx
|
||||
row = list_start + idx
|
||||
if row >= rows - 2:
|
||||
break
|
||||
prefix = "> " if abs_idx == cursor else " "
|
||||
attr = color_attrs.get(name, curses.A_NORMAL)
|
||||
if abs_idx == cursor:
|
||||
attr |= curses.A_REVERSE
|
||||
_addstr_safe(screen, row, 0, (prefix + name)[:cols - 1], attr)
|
||||
|
||||
_addstr_safe(screen, rows - 2, 0, sep)
|
||||
_addstr_safe(
|
||||
screen, rows - 1, 0,
|
||||
"[↑↓/jk] move [Enter] select [Esc/q] skip",
|
||||
curses.A_DIM,
|
||||
)
|
||||
screen.refresh()
|
||||
@@ -1,339 +0,0 @@
|
||||
"""Claude agent provider plugin (PRD 0050, contrib).
|
||||
|
||||
The Claude-specific behavior previously inlined under
|
||||
`agent_provider.agent_provision_plan` (claude.json trust marker,
|
||||
api.anthropic.com egress route, OAuth-token placeholder), plus
|
||||
the `claude mcp add` invocation that registers the supervise
|
||||
sidecar in claude-code's user config (PRD 0013)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ...agent_provider import (
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from ...backend.docker import util as docker_mod
|
||||
from ...egress import EgressRoute
|
||||
from ...log import die, info, warn
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
return f"{guest_home}/.claude/skills"
|
||||
|
||||
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
|
||||
_STATUS_LINE_COLORS = {
|
||||
"black": "\033[30m",
|
||||
"red": "\033[31m",
|
||||
"green": "\033[32m",
|
||||
"yellow": "\033[33m",
|
||||
"blue": "\033[34m",
|
||||
"magenta": "\033[35m",
|
||||
"cyan": "\033[36m",
|
||||
"white": "\033[37m",
|
||||
"bright-black": "\033[90m",
|
||||
"bright-red": "\033[91m",
|
||||
"bright-green": "\033[92m",
|
||||
"bright-yellow": "\033[93m",
|
||||
"bright-blue": "\033[94m",
|
||||
"bright-magenta": "\033[95m",
|
||||
"bright-cyan": "\033[96m",
|
||||
"bright-white": "\033[97m",
|
||||
}
|
||||
|
||||
_CLAUDE_THEME_COLORS = {
|
||||
"black": "black",
|
||||
"red": "red",
|
||||
"green": "green",
|
||||
"yellow": "yellow",
|
||||
"blue": "blue",
|
||||
"magenta": "magenta",
|
||||
"cyan": "cyan",
|
||||
"white": "white",
|
||||
"bright-black": "blackBright",
|
||||
"bright-red": "redBright",
|
||||
"bright-green": "greenBright",
|
||||
"bright-yellow": "yellowBright",
|
||||
"bright-blue": "blueBright",
|
||||
"bright-magenta": "magentaBright",
|
||||
"bright-cyan": "cyanBright",
|
||||
"bright-white": "whiteBright",
|
||||
}
|
||||
|
||||
|
||||
def _status_line_script(label: str, color: str) -> str:
|
||||
if not label:
|
||||
return "#!/bin/sh\nprintf '\\n'\n"
|
||||
label_q = shlex.quote(label)
|
||||
if color and color in _STATUS_LINE_COLORS:
|
||||
return (
|
||||
"#!/bin/sh\n"
|
||||
f"printf '%b%s%b\\n' '{_STATUS_LINE_COLORS[color]}' {label_q} '\\033[0m'\n"
|
||||
)
|
||||
return f"#!/bin/sh\nprintf '%s\\n' {label_q}\n"
|
||||
|
||||
|
||||
def _custom_theme_payload(color: str) -> dict[str, object] | None:
|
||||
theme_color = _CLAUDE_THEME_COLORS.get(color)
|
||||
if not theme_color:
|
||||
return None
|
||||
return {
|
||||
"name": f"Bot-bottle {color}",
|
||||
"base": "dark",
|
||||
"overrides": {
|
||||
"claude": f"ansi:{theme_color}",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="claude",
|
||||
command="claude",
|
||||
image="bot-bottle-claude:latest",
|
||||
prompt_mode="append_file",
|
||||
bypass_args=("--dangerously-skip-permissions",),
|
||||
resume_args=("--continue",),
|
||||
remote_control_args=("--remote-control",),
|
||||
)
|
||||
|
||||
|
||||
class ClaudeAgentProvider(AgentProvider):
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return _RUNTIME
|
||||
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
del forward_host_credentials, host_env, provider_settings
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
guest_home = self.guest_home
|
||||
trusted_path = trusted_project_path or guest_home
|
||||
|
||||
env_vars: dict[str, str] = {
|
||||
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
|
||||
"DISABLE_ERROR_REPORTING": "1",
|
||||
}
|
||||
dirs = (
|
||||
AgentProvisionDir(f"{guest_home}/.claude"),
|
||||
AgentProvisionDir(f"{guest_home}/.claude/themes"),
|
||||
)
|
||||
claude_config = state_dir / "claude.json"
|
||||
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
|
||||
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
||||
payload: dict[str, object] = {
|
||||
"hasCompletedOnboarding": True,
|
||||
"theme": "dark",
|
||||
"bypassPermissionsModeAccepted": True,
|
||||
"projects": claude_projects,
|
||||
}
|
||||
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
|
||||
claude_config.chmod(0o600)
|
||||
files = [
|
||||
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
|
||||
]
|
||||
|
||||
claude_settings = state_dir / "claude-settings.json"
|
||||
claude_settings_payload: dict[str, object] = {}
|
||||
if label or color:
|
||||
statusline_script = state_dir / "claude-statusline.sh"
|
||||
statusline_script.write_text(_status_line_script(label, color))
|
||||
statusline_script.chmod(0o755)
|
||||
files.append(AgentProvisionFile(
|
||||
statusline_script,
|
||||
f"{guest_home}/.claude/statusline.sh",
|
||||
mode="755",
|
||||
))
|
||||
claude_settings_payload["statusLine"] = {
|
||||
"type": "command",
|
||||
"command": "~/.claude/statusline.sh",
|
||||
}
|
||||
theme_payload = _custom_theme_payload(color)
|
||||
if theme_payload is not None:
|
||||
theme_name = f"bot-bottle-{docker_mod.slugify(label or color)}"
|
||||
theme_file = state_dir / f"{theme_name}.json"
|
||||
theme_file.write_text(json.dumps(theme_payload, indent=2) + "\n")
|
||||
theme_file.chmod(0o644)
|
||||
files.append(AgentProvisionFile(
|
||||
theme_file,
|
||||
f"{guest_home}/.claude/themes/{theme_name}.json",
|
||||
))
|
||||
claude_settings_payload["theme"] = f"custom:{theme_name}"
|
||||
if claude_settings_payload:
|
||||
claude_settings.write_text(json.dumps(claude_settings_payload, indent=2) + "\n")
|
||||
claude_settings.chmod(0o600)
|
||||
files.append(AgentProvisionFile(
|
||||
claude_settings,
|
||||
f"{guest_home}/.claude/settings.json",
|
||||
))
|
||||
egress_routes = (EgressRoute(
|
||||
host="api.anthropic.com",
|
||||
auth_scheme="Bearer" if auth_token else "",
|
||||
token_ref=auth_token,
|
||||
),)
|
||||
hidden_env_names: frozenset[str] = frozenset()
|
||||
if auth_token:
|
||||
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
||||
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
||||
|
||||
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
env_vars=env_vars,
|
||||
guest_env=resolved_guest_env,
|
||||
has_prompt=has_prompt,
|
||||
dirs=dirs,
|
||||
files=tuple(files),
|
||||
egress_routes=egress_routes,
|
||||
hidden_env_names=hidden_env_names,
|
||||
)
|
||||
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
||||
on the host into the guest's claude-code skills dir. No-op
|
||||
when the agent has no skills."""
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
"""Copy the prompt file into the guest, fix ownership/mode.
|
||||
Returns the in-guest path iff the agent has a non-empty
|
||||
prompt (drives `--append-system-prompt-file`); the file is
|
||||
copied either way so the path always exists."""
|
||||
prompt_path = _prompt_path(plan.guest_home)
|
||||
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
|
||||
bottle.exec(
|
||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||
user="root",
|
||||
)
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the claude-side declarative provision steps from
|
||||
`plan.agent_provision` — today that's the `claude.json`
|
||||
trust-marker file. Hot-replace this with a richer flow as
|
||||
claude-code's harness shape evolves."""
|
||||
provision = plan.agent_provision
|
||||
for d in provision.dirs:
|
||||
path = shlex.quote(d.guest_path)
|
||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(d.owner)} {path}",
|
||||
f"could not chown {d.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(d.mode)} {path}",
|
||||
f"could not chmod {d.guest_path}",
|
||||
)
|
||||
for command in provision.pre_copy:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
for f in provision.files:
|
||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||
path = shlex.quote(f.guest_path)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(f.owner)} {path}",
|
||||
f"could not chown {f.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(f.mode)} {path}",
|
||||
f"could not chmod {f.guest_path}",
|
||||
)
|
||||
for command in provision.verify:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
"""Run `claude mcp add` inside the agent guest to register the
|
||||
supervise sidecar in claude-code's user config (~/.claude.json).
|
||||
|
||||
Failure is logged but not fatal — the bottle still works without
|
||||
the entry; the operator can register it manually."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
info(f"registering supervise MCP server in agent claude config → {supervise_url}")
|
||||
r = bottle.exec(
|
||||
f"claude mcp add --scope user --transport http "
|
||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||
user="node",
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"claude mcp add --scope user --transport http supervise {supervise_url}"
|
||||
)
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
if detail:
|
||||
detail = f": {detail}"
|
||||
die(f"agent provider provisioning: {error}{detail}")
|
||||
@@ -1,28 +0,0 @@
|
||||
# bot-bottle Codex provider image.
|
||||
#
|
||||
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
|
||||
# non-root node user, and the provider CLI installed globally.
|
||||
|
||||
FROM node:22-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# App-specific deps. Python isn't required by codex itself
|
||||
# (codex is a Node CLI), but is convenient for the agent to shell
|
||||
# out to for ad-hoc scripts. Kept on its own layer so it can be
|
||||
# moved to a downstream image if the base ever needs to shrink.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
|
||||
&& npm cache clean --force
|
||||
|
||||
USER node
|
||||
WORKDIR /home/node
|
||||
|
||||
RUN mkdir -p /home/node/.codex
|
||||
|
||||
CMD ["codex"]
|
||||
@@ -1,283 +0,0 @@
|
||||
"""Codex agent provider plugin (PRD 0050, contrib).
|
||||
|
||||
The Codex-specific behavior previously inlined under
|
||||
`agent_provider.agent_provision_plan` (config.toml trust marker,
|
||||
chatgpt.com / api.openai.com egress routes, optional host-credential
|
||||
forwarding with dummy-auth.json + verify), plus the `codex mcp add`
|
||||
invocation that registers the supervise sidecar in Codex's
|
||||
~/.codex/config.toml (PRD 0050)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from ...agent_provider import (
|
||||
CODEX_HOST_CREDENTIAL_HOSTS,
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionCommand,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
|
||||
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
||||
from ...log import die, info, warn
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
# Codex agents still read skills from the claude-code convention
|
||||
# (~/.claude/skills/) — the bot-bottle-codex image follows the
|
||||
# same layout. If Codex grows native skill discovery later,
|
||||
# change here.
|
||||
return f"{guest_home}/.claude/skills"
|
||||
|
||||
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="codex",
|
||||
command="codex",
|
||||
image="bot-bottle-codex:latest",
|
||||
prompt_mode="read_prompt_file",
|
||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||
resume_args=("resume", "--last"),
|
||||
remote_control_args=(),
|
||||
)
|
||||
|
||||
|
||||
class CodexAgentProvider(AgentProvider):
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return _RUNTIME
|
||||
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
del auth_token, label, color, provider_settings
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
guest_home = self.guest_home
|
||||
trusted_path = trusted_project_path or guest_home
|
||||
|
||||
env_vars: dict[str, str] = {
|
||||
"CODEX_CA_CERTIFICATE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
||||
if forward_host_credentials:
|
||||
env_vars["CODEX_HOME"] = auth_dir
|
||||
|
||||
dirs = [AgentProvisionDir(auth_dir)]
|
||||
files: list[AgentProvisionFile] = []
|
||||
pre_copy: list[AgentProvisionCommand] = []
|
||||
verify: list[AgentProvisionCommand] = []
|
||||
provisioned_env: dict[str, str] = {}
|
||||
|
||||
config_path = f"{auth_dir}/config.toml"
|
||||
config_file = state_dir / "codex-config.toml"
|
||||
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
|
||||
config_file.write_text(
|
||||
f'[projects."{toml_path}"]\n'
|
||||
'trust_level = "trusted"\n'
|
||||
"\n"
|
||||
"[tui]\n"
|
||||
'status_line = ["model-with-reasoning"]\n'
|
||||
'terminal_title = ["spinner", "project"]\n'
|
||||
'theme = "ansi"\n'
|
||||
)
|
||||
config_file.chmod(0o600)
|
||||
files.append(AgentProvisionFile(config_file, config_path))
|
||||
|
||||
egress_routes: list[EgressRoute] = []
|
||||
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
||||
egress_routes.append(EgressRoute(
|
||||
host=host,
|
||||
auth_scheme="Bearer" if forward_host_credentials else "",
|
||||
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
||||
))
|
||||
|
||||
if forward_host_credentials:
|
||||
_host_env = host_env or dict(os.environ)
|
||||
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = (
|
||||
codex_host_access_token(_host_env)
|
||||
)
|
||||
auth_file = state_dir / "codex-auth.json"
|
||||
write_codex_dummy_auth_file(auth_file, _host_env)
|
||||
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
||||
pre_copy.append(AgentProvisionCommand((
|
||||
"find", auth_dir,
|
||||
"-maxdepth", "1",
|
||||
"-type", "f",
|
||||
"(",
|
||||
"-name", "*.sqlite",
|
||||
"-o", "-name", "*.sqlite-*",
|
||||
"-o", "-name", "*.codex-repair-*.bak",
|
||||
")",
|
||||
"-delete",
|
||||
), "codex host credentials: could not reset runtime db files"))
|
||||
verify.append(AgentProvisionCommand((
|
||||
"runuser", "-u", "node", "--",
|
||||
"env",
|
||||
f"HOME={guest_home}",
|
||||
f"CODEX_HOME={auth_dir}",
|
||||
"codex", "login", "status",
|
||||
), (
|
||||
"codex host credentials: dummy auth was copied into the "
|
||||
"guest, but Codex did not accept it"
|
||||
)))
|
||||
|
||||
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
env_vars=env_vars,
|
||||
guest_env=resolved_guest_env,
|
||||
has_prompt=has_prompt,
|
||||
dirs=tuple(dirs),
|
||||
files=tuple(files),
|
||||
pre_copy=tuple(pre_copy),
|
||||
verify=tuple(verify),
|
||||
egress_routes=tuple(egress_routes),
|
||||
provisioned_env=provisioned_env,
|
||||
)
|
||||
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
||||
on the host into the guest. No-op when the agent has no
|
||||
skills."""
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
"""Copy the prompt file into the guest, fix ownership/mode.
|
||||
Codex reads it via the agent's `Read and follow the
|
||||
instructions in <path>.` bootstrap (see `prompt_args`); the
|
||||
file is copied either way so the path always exists."""
|
||||
prompt_path = _prompt_path(plan.guest_home)
|
||||
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
|
||||
bottle.exec(
|
||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||
user="root",
|
||||
)
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the codex-side declarative provision steps from
|
||||
`plan.agent_provision`: the `~/.codex/` dir + config.toml
|
||||
trust marker, plus the dummy-auth.json drop + `codex login
|
||||
status` verify when host-credential forwarding is on."""
|
||||
provision = plan.agent_provision
|
||||
for d in provision.dirs:
|
||||
path = shlex.quote(d.guest_path)
|
||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(d.owner)} {path}",
|
||||
f"could not chown {d.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(d.mode)} {path}",
|
||||
f"could not chmod {d.guest_path}",
|
||||
)
|
||||
for command in provision.pre_copy:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
for f in provision.files:
|
||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||
path = shlex.quote(f.guest_path)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(f.owner)} {path}",
|
||||
f"could not chown {f.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(f.mode)} {path}",
|
||||
f"could not chmod {f.guest_path}",
|
||||
)
|
||||
for command in provision.verify:
|
||||
_exec(bottle, shlex.join(command.argv), command.error)
|
||||
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
"""Run `codex mcp add` inside the agent guest to register the
|
||||
supervise sidecar in Codex's user config (~/.codex/config.toml).
|
||||
|
||||
Mirrors the Claude provider's `claude mcp add` flow — failure
|
||||
is logged but not fatal."""
|
||||
if plan.supervise_plan is None:
|
||||
return
|
||||
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
||||
r = bottle.exec(
|
||||
f"codex mcp add --transport http "
|
||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||
user="node",
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"`codex mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"codex mcp add --transport http supervise {supervise_url}"
|
||||
)
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
if detail:
|
||||
detail = f": {detail}"
|
||||
die(f"agent provider provisioning: {error}{detail}")
|
||||
@@ -1,328 +0,0 @@
|
||||
"""Host Codex auth helpers.
|
||||
|
||||
Reads the host's Codex ChatGPT/device-login auth state and returns only
|
||||
the short-lived access token needed by egress. This module deliberately
|
||||
does not expose refresh tokens or raw auth payloads.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from copy import deepcopy
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from ...log import die
|
||||
from ...util import expand_tilde
|
||||
|
||||
|
||||
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
|
||||
env = os.environ if host_env is None else host_env
|
||||
home = env.get("CODEX_HOME")
|
||||
if home:
|
||||
return Path(expand_tilde(home)) / "auth.json"
|
||||
return Path.home() / ".codex" / "auth.json"
|
||||
|
||||
|
||||
def codex_host_access_token(
|
||||
host_env: dict[str, str] | None = None,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> str:
|
||||
path = codex_auth_path(host_env)
|
||||
if not path.is_file():
|
||||
die(
|
||||
f"codex host credentials: auth file missing at {path}. "
|
||||
"Run `codex login --device-auth` on the host or disable "
|
||||
"agent_provider.forward_host_credentials."
|
||||
)
|
||||
raw = _read_auth_object(path)
|
||||
|
||||
auth_mode = raw.get("auth_mode")
|
||||
if not isinstance(auth_mode, str) or auth_mode == "api_key":
|
||||
die(
|
||||
"codex host credentials: host Codex auth is not user/device "
|
||||
"auth. Run `codex login --device-auth` on the host."
|
||||
)
|
||||
|
||||
tokens = raw.get("tokens")
|
||||
if not isinstance(tokens, dict):
|
||||
die(f"codex host credentials: {path} is missing tokens")
|
||||
tokens_typed = cast(dict[str, object], tokens)
|
||||
access = tokens_typed.get("access_token")
|
||||
if not isinstance(access, str) or not access:
|
||||
die(
|
||||
f"codex host credentials: {path} is missing tokens.access_token. "
|
||||
"Run `codex login --device-auth` on the host."
|
||||
)
|
||||
|
||||
exp = _jwt_exp(access)
|
||||
if exp is None:
|
||||
die("codex host credentials: tokens.access_token is not a JWT with exp")
|
||||
check_now = now or datetime.now(timezone.utc)
|
||||
if exp <= check_now:
|
||||
die(
|
||||
"codex host credentials: host Codex access token is expired. "
|
||||
"Run `codex login --device-auth` on the host and restart the bottle."
|
||||
)
|
||||
return access
|
||||
|
||||
|
||||
def codex_dummy_auth_json(
|
||||
host_env: dict[str, str] | None = None,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> str:
|
||||
"""Return a non-secret `auth.json` that keeps Codex in the host's
|
||||
auth branch while egress owns the real bearer token.
|
||||
|
||||
The dummy access/id tokens carry the *host* token's real `exp` so
|
||||
Codex's proactive refresh lifecycle (it refreshes when its local
|
||||
access token is at/past expiry) tracks the real token instead of
|
||||
firing after an artificial TTL. Codex cannot refresh inside the
|
||||
bottle — the refresh token is a placeholder and the OpenAI token
|
||||
endpoint is off-route — so a shorter dummy exp would drop Codex to
|
||||
the sign-in screen the moment it lapsed, even while egress still
|
||||
holds a valid bearer."""
|
||||
path = codex_auth_path(host_env)
|
||||
access = codex_host_access_token(host_env, now=now)
|
||||
raw = _read_auth_object(path)
|
||||
host_exp = _jwt_exp(access)
|
||||
exp_ts = int(host_exp.timestamp()) if host_exp is not None else None
|
||||
dummy = _redact_codex_auth(deepcopy(raw), now=now, exp_ts=exp_ts)
|
||||
return json.dumps(dummy, indent=2, sort_keys=True) + "\n"
|
||||
|
||||
|
||||
def write_codex_dummy_auth_file(
|
||||
path: Path,
|
||||
host_env: dict[str, str] | None = None,
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(codex_dummy_auth_json(host_env, now=now))
|
||||
path.chmod(0o600)
|
||||
|
||||
|
||||
def _read_auth_object(path: Path) -> dict[str, object]:
|
||||
try:
|
||||
raw = json.loads(path.read_text())
|
||||
except (OSError, json.JSONDecodeError) as e:
|
||||
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
|
||||
if not isinstance(raw, dict):
|
||||
die(f"codex host credentials: {path} must contain a JSON object")
|
||||
return cast(dict[str, object], raw)
|
||||
|
||||
|
||||
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
|
||||
if exp_ts is not None:
|
||||
return exp_ts
|
||||
check_now = now or datetime.now(timezone.utc)
|
||||
return int(check_now.timestamp()) + 3600
|
||||
|
||||
|
||||
def _dummy_timestamp(now: datetime | None = None) -> str:
|
||||
check_now = now or datetime.now(timezone.utc)
|
||||
if check_now.tzinfo is None:
|
||||
check_now = check_now.replace(tzinfo=timezone.utc)
|
||||
check_now = check_now.astimezone(timezone.utc)
|
||||
return check_now.isoformat(timespec="milliseconds").replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _dummy_jwt(now: datetime | None = None, *, exp_ts: int | None = None) -> str:
|
||||
return _encode_dummy_jwt({
|
||||
"exp": _dummy_exp(now, exp_ts),
|
||||
"sub": "bot-bottle-placeholder",
|
||||
})
|
||||
|
||||
|
||||
def _dummy_jwt_from_host(
|
||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||
) -> str:
|
||||
if not isinstance(value, str):
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
parts = value.split(".")
|
||||
if len(parts) < 2:
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
try:
|
||||
payload = json.loads(_b64url_decode(parts[1]))
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
if not isinstance(payload, dict):
|
||||
return _dummy_jwt(now, exp_ts=exp_ts)
|
||||
return _encode_dummy_jwt(_redact_jwt_payload(cast(dict[str, object], payload), now=now, exp_ts=exp_ts))
|
||||
|
||||
|
||||
def _encode_dummy_jwt(payload: dict[str, object]) -> str:
|
||||
def enc(obj: dict[str, object]) -> str:
|
||||
raw = json.dumps(obj, separators=(",", ":")).encode()
|
||||
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
|
||||
|
||||
return f"{enc({'alg': 'none', 'typ': 'JWT'})}.{enc(payload)}.placeholder"
|
||||
|
||||
|
||||
def _redact_jwt_payload(
|
||||
payload: dict[str, object],
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
exp_ts: int | None = None,
|
||||
) -> dict[str, object]:
|
||||
out = _redact_claims(payload)
|
||||
if not isinstance(out, dict):
|
||||
out = {}
|
||||
out_typed: dict[str, object] = cast(dict[str, object], out)
|
||||
out_typed["exp"] = _dummy_exp(now, exp_ts)
|
||||
out_typed.setdefault("sub", "bot-bottle-placeholder")
|
||||
return out_typed
|
||||
|
||||
|
||||
def _redact_claims(value: object) -> object:
|
||||
if isinstance(value, dict):
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in cast(dict[str, object], value).items():
|
||||
lower = key.lower()
|
||||
if key == "https://api.openai.com/profile":
|
||||
out[key] = _redact_profile_claim(inner)
|
||||
elif key == "https://api.openai.com/auth":
|
||||
out[key] = _redact_auth_claim(inner)
|
||||
elif lower == "email":
|
||||
out[key] = "bot-bottle@example.invalid"
|
||||
elif lower == "email_verified":
|
||||
out[key] = True
|
||||
elif lower in {"exp", "iat", "nbf", "auth_time", "pwd_auth_time"}:
|
||||
out[key] = inner if isinstance(inner, (int, float)) else 0
|
||||
elif lower in {"aud", "scp", "amr"}:
|
||||
out[key] = inner if isinstance(inner, list) else []
|
||||
elif isinstance(inner, bool):
|
||||
out[key] = inner
|
||||
elif isinstance(inner, dict):
|
||||
out[key] = {}
|
||||
elif isinstance(inner, list):
|
||||
out[key] = []
|
||||
else:
|
||||
out[key] = "bot-bottle-placeholder"
|
||||
return out
|
||||
if isinstance(value, list):
|
||||
return []
|
||||
return "bot-bottle-placeholder"
|
||||
|
||||
|
||||
def _redact_profile_claim(value: object) -> dict[str, object]:
|
||||
profile = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||
return {
|
||||
"email": "bot-bottle@example.invalid",
|
||||
"email_verified": bool(profile.get("email_verified", True)),
|
||||
}
|
||||
|
||||
|
||||
def _redact_auth_claim(value: object) -> dict[str, object]:
|
||||
auth = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in auth.items():
|
||||
lower = key.lower()
|
||||
if lower == "chatgpt_plan_type" and isinstance(inner, str) and inner:
|
||||
out[key] = inner
|
||||
elif lower == "chatgpt_account_id" and isinstance(inner, str) and inner:
|
||||
# Current Codex uses the selected account id when building
|
||||
# ChatGPT requests. Keep that non-secret identifier aligned
|
||||
# with the host while egress owns the real bearer token.
|
||||
out[key] = inner
|
||||
elif lower == "localhost" and isinstance(inner, bool):
|
||||
out[key] = inner
|
||||
elif isinstance(inner, bool):
|
||||
out[key] = inner
|
||||
elif isinstance(inner, list):
|
||||
out[key] = []
|
||||
elif isinstance(inner, dict):
|
||||
out[key] = {}
|
||||
else:
|
||||
out[key] = "bot-bottle-placeholder"
|
||||
out.setdefault("chatgpt_plan_type", "unknown")
|
||||
out.setdefault("user_id", "bot-bottle-placeholder")
|
||||
out.setdefault("chatgpt_user_id", "bot-bottle-placeholder")
|
||||
out.setdefault("chatgpt_account_id", "bot-bottle-placeholder")
|
||||
return out
|
||||
|
||||
|
||||
def _redact_codex_auth(
|
||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||
) -> object:
|
||||
auth = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in auth.items():
|
||||
lower = key.lower()
|
||||
if lower == "auth_mode" and isinstance(inner, str) and inner:
|
||||
out[key] = inner
|
||||
elif lower == "openai_api_key":
|
||||
out[key] = None
|
||||
elif lower == "last_refresh":
|
||||
# Codex parses this as a timestamp on startup. Keep the
|
||||
# schema valid without copying host-side session metadata.
|
||||
out[key] = _dummy_timestamp(now)
|
||||
elif lower == "tokens":
|
||||
out[key] = _redact_token_block(inner, now=now, exp_ts=exp_ts)
|
||||
else:
|
||||
out[key] = _redact_unknown_auth_value(inner)
|
||||
return out
|
||||
|
||||
|
||||
def _redact_token_block(
|
||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||
) -> dict[str, object]:
|
||||
tokens = cast(dict[str, object], value) if isinstance(value, dict) else {}
|
||||
out: dict[str, object] = {}
|
||||
for key, inner in tokens.items():
|
||||
lower = key.lower()
|
||||
if lower in {"access_token", "id_token"}:
|
||||
out[key] = _dummy_jwt_from_host(inner, now=now, exp_ts=exp_ts)
|
||||
elif lower == "account_id" and isinstance(inner, str) and inner:
|
||||
# Current Codex uses this non-secret selected account id
|
||||
# while egress owns the real bearer token.
|
||||
out[key] = inner
|
||||
else:
|
||||
out[key] = _redact_unknown_auth_value(inner)
|
||||
return out
|
||||
|
||||
|
||||
def _redact_unknown_auth_value(value: object) -> object:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return {}
|
||||
if isinstance(value, list):
|
||||
return []
|
||||
if value is None:
|
||||
return None
|
||||
return "bot-bottle-placeholder"
|
||||
|
||||
|
||||
def _jwt_exp(token: str) -> datetime | None:
|
||||
parts = token.split(".")
|
||||
if len(parts) < 2:
|
||||
return None
|
||||
try:
|
||||
payload = json.loads(_b64url_decode(parts[1]))
|
||||
except (ValueError, json.JSONDecodeError):
|
||||
return None
|
||||
if not isinstance(payload, dict):
|
||||
return None
|
||||
exp = cast(dict[str, object], payload).get("exp")
|
||||
if not isinstance(exp, (int, float)):
|
||||
return None
|
||||
return datetime.fromtimestamp(exp, timezone.utc)
|
||||
|
||||
|
||||
def _b64url_decode(value: str) -> str:
|
||||
padded = value + ("=" * (-len(value) % 4))
|
||||
return base64.urlsafe_b64decode(padded.encode("ascii")).decode("utf-8")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"codex_auth_path",
|
||||
"codex_dummy_auth_json",
|
||||
"codex_host_access_token",
|
||||
"write_codex_dummy_auth_file",
|
||||
]
|
||||
@@ -1,127 +0,0 @@
|
||||
"""Gitea deploy-key provisioner (PRD 0048, contrib).
|
||||
|
||||
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
|
||||
them using the Gitea deploy-key HTTP API. No new Python dependencies —
|
||||
only stdlib `urllib.request` and `subprocess`.
|
||||
|
||||
Required token permissions (Gitea "Applications" → "Generate Token"):
|
||||
- Repository: Read & Write
|
||||
Grants POST /api/v1/repos/{owner}/{repo}/keys (create deploy key)
|
||||
and DELETE /api/v1/repos/{owner}/{repo}/keys/{id} (revoke deploy key).
|
||||
No other scopes are needed."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import tempfile
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from ...deploy_key_provisioner import DeployKeyProvisioner
|
||||
|
||||
|
||||
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
|
||||
"""Manages deploy keys on a Gitea instance."""
|
||||
|
||||
def __init__(self, *, token: str, api_url: str) -> None:
|
||||
self._token = token
|
||||
self._api_url = api_url.rstrip("/")
|
||||
|
||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||
"""Generate an ed25519 keypair, register the public half as a
|
||||
repo deploy key, and return `(key_id, private_key_bytes)`.
|
||||
|
||||
The key is registered with `read_only=False` because git-gate
|
||||
needs push access to forward gitleaks-scanned refs upstream."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
key_path = Path(tmpdir) / "key"
|
||||
subprocess.run(
|
||||
[
|
||||
"ssh-keygen", "-t", "ed25519",
|
||||
"-f", str(key_path),
|
||||
"-N", "",
|
||||
],
|
||||
check=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
private_key = key_path.read_bytes()
|
||||
public_key = key_path.with_suffix(".pub").read_text().strip()
|
||||
|
||||
owner, repo = _split_owner_repo(owner_repo)
|
||||
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys"
|
||||
payload = json.dumps({
|
||||
"key": public_key,
|
||||
"read_only": False,
|
||||
"title": title,
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=payload,
|
||||
headers={
|
||||
"Authorization": f"token {self._token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
body = json.loads(resp.read())
|
||||
except urllib.error.HTTPError as exc:
|
||||
_body = _read_error_body(exc)
|
||||
raise RuntimeError(
|
||||
f"failed to create deploy key for {owner_repo}: "
|
||||
f"HTTP {exc.code} — {_body}"
|
||||
) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise RuntimeError(
|
||||
f"failed to create deploy key for {owner_repo}: {exc.reason}"
|
||||
) from exc
|
||||
|
||||
return str(body["id"]), private_key
|
||||
|
||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||
"""Delete the deploy key. HTTP 404 (already gone) is success.
|
||||
All other errors raise RuntimeError so teardown halts."""
|
||||
owner, repo = _split_owner_repo(owner_repo)
|
||||
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys/{key_id}"
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={"Authorization": f"token {self._token}"},
|
||||
method="DELETE",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req):
|
||||
pass
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
return
|
||||
_body = _read_error_body(exc)
|
||||
raise RuntimeError(
|
||||
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
||||
f"HTTP {exc.code} — {_body}"
|
||||
) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise RuntimeError(
|
||||
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
||||
f"{exc.reason}"
|
||||
) from exc
|
||||
|
||||
|
||||
def _split_owner_repo(owner_repo: str) -> tuple[str, str]:
|
||||
"""Split `'owner/repo'` into `('owner', 'repo')`."""
|
||||
parts = owner_repo.split("/", 1)
|
||||
if len(parts) != 2 or not all(parts):
|
||||
raise ValueError(
|
||||
f"expected 'owner/repo' format, got {owner_repo!r}"
|
||||
)
|
||||
return parts[0], parts[1]
|
||||
|
||||
|
||||
def _read_error_body(exc: urllib.error.HTTPError) -> str:
|
||||
try:
|
||||
return exc.read().decode("utf-8", errors="replace")
|
||||
except Exception: # noqa: broad-exception-caught — safely fallback to empty error message
|
||||
return ""
|
||||
@@ -1,41 +0,0 @@
|
||||
# bot-bottle Pi provider image.
|
||||
#
|
||||
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
|
||||
|
||||
FROM node:22-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
git \
|
||||
ca-certificates \
|
||||
curl \
|
||||
fd-find \
|
||||
ripgrep \
|
||||
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
|
||||
&& npm cache clean --force
|
||||
|
||||
RUN mkdir -p /home/node/.pi/agent \
|
||||
/home/node/.pi/context-mode/sessions \
|
||||
/tmp/pi-subagents-uid-1000 \
|
||||
&& chown -R node:node /home/node/.pi /tmp \
|
||||
&& chmod -R u+rwX /tmp \
|
||||
&& chown root:root /tmp /var/tmp \
|
||||
&& chmod 1777 /tmp /var/tmp
|
||||
|
||||
USER node
|
||||
WORKDIR /home/node
|
||||
|
||||
RUN pi install npm:@harms-haus/pi-cwd \
|
||||
&& pi install npm:pi-web-access \
|
||||
&& pi install npm:context-mode \
|
||||
&& pi install npm:pi-subagents \
|
||||
&& pi install npm:pi-mcp-adapter
|
||||
|
||||
CMD ["pi"]
|
||||
@@ -1 +0,0 @@
|
||||
"""Pi agent provider package."""
|
||||
@@ -1,319 +0,0 @@
|
||||
"""Pi agent provider plugin (PRD 0058, contrib).
|
||||
|
||||
Pi uses ~/.pi/agent/models.json for custom provider/model settings.
|
||||
This provider writes an Ollama-compatible default configuration and
|
||||
lets bottles override the model endpoint and model ids via
|
||||
agent_provider.settings.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from ...agent_provider import (
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from ...egress import EgressRoute
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_DEFAULT_BASE_URL = "http://ollama:11434/v1"
|
||||
_DEFAULT_MODEL = "qwen2.5-coder:7b"
|
||||
_DEFAULT_PROVIDER_NAME = "ollama"
|
||||
_DEFAULT_CONTEXT_WINDOW = 4096
|
||||
_DEFAULT_MAX_TOKENS = 1024
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
return f"{guest_home}/.pi/agent/skills"
|
||||
|
||||
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
|
||||
def _append_system_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.pi/agent/APPEND_SYSTEM.md"
|
||||
|
||||
|
||||
def _models_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.pi/agent/models.json"
|
||||
|
||||
|
||||
def _runtime_state_repair_script(guest_home: str) -> str:
|
||||
home = shlex.quote(guest_home)
|
||||
pi_home = shlex.quote(f"{guest_home}/.pi")
|
||||
context_sessions = shlex.quote(f"{guest_home}/.pi/context-mode/sessions")
|
||||
return (
|
||||
f"mkdir -p {context_sessions} /tmp/pi-subagents-uid-1000 && "
|
||||
f"chown node:node {home} && "
|
||||
f"chown -R node:node {pi_home} /tmp && "
|
||||
"chmod -R u+rwX /tmp && "
|
||||
f"chmod 755 {home} && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp"
|
||||
)
|
||||
|
||||
|
||||
def _settings_value(
|
||||
settings: dict[str, object],
|
||||
key: str,
|
||||
default: object,
|
||||
) -> object:
|
||||
value = settings.get(key)
|
||||
return default if value is None else value
|
||||
|
||||
|
||||
def _settings_int(
|
||||
settings: dict[str, object],
|
||||
key: str,
|
||||
default: int,
|
||||
) -> int:
|
||||
value = _settings_value(settings, key, default)
|
||||
if isinstance(value, bool):
|
||||
return default
|
||||
if isinstance(value, (int, str)):
|
||||
return int(value)
|
||||
return default
|
||||
|
||||
|
||||
def _pi_models_json(
|
||||
settings: dict[str, object],
|
||||
) -> tuple[dict[str, object], str, str, list[str], str]:
|
||||
provider_name = str(
|
||||
_settings_value(settings, "provider", _DEFAULT_PROVIDER_NAME)
|
||||
)
|
||||
base_url = str(_settings_value(settings, "base_url", _DEFAULT_BASE_URL))
|
||||
api = str(_settings_value(settings, "api", "openai-completions"))
|
||||
api_key = settings.get("api_key")
|
||||
api_key_env = str(settings.get("api_key_env", ""))
|
||||
models_raw = _settings_value(settings, "models", [_DEFAULT_MODEL])
|
||||
models = [str(model) for model in models_raw] # type: ignore[union-attr]
|
||||
supports_developer_role = bool(
|
||||
_settings_value(settings, "supports_developer_role", False)
|
||||
)
|
||||
supports_reasoning_effort = bool(
|
||||
_settings_value(settings, "supports_reasoning_effort", False)
|
||||
)
|
||||
max_tokens_field = str(
|
||||
_settings_value(settings, "max_tokens_field", "max_tokens")
|
||||
)
|
||||
context_window = _settings_int(
|
||||
settings, "context_window", _DEFAULT_CONTEXT_WINDOW,
|
||||
)
|
||||
max_tokens = _settings_int(settings, "max_tokens", _DEFAULT_MAX_TOKENS)
|
||||
input_context_window = max(1, context_window - max_tokens)
|
||||
provider: dict[str, object] = {
|
||||
"baseUrl": base_url,
|
||||
"api": api,
|
||||
"compat": {
|
||||
"supportsDeveloperRole": supports_developer_role,
|
||||
"supportsReasoningEffort": supports_reasoning_effort,
|
||||
"maxTokensField": max_tokens_field,
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": model,
|
||||
"name": model,
|
||||
"contextWindow": input_context_window,
|
||||
"maxTokens": max_tokens,
|
||||
}
|
||||
for model in models
|
||||
],
|
||||
}
|
||||
if api_key is not None:
|
||||
provider["apiKey"] = str(api_key)
|
||||
elif api_key_env:
|
||||
provider["apiKey"] = "egress-placeholder"
|
||||
elif provider_name == _DEFAULT_PROVIDER_NAME:
|
||||
provider["apiKey"] = "ollama"
|
||||
payload: dict[str, object] = {
|
||||
"providers": {
|
||||
provider_name: provider,
|
||||
}
|
||||
}
|
||||
return payload, base_url, api_key_env, models, provider_name
|
||||
|
||||
|
||||
def _route_host(base_url: str) -> str:
|
||||
parsed = urlparse(base_url)
|
||||
if not parsed.scheme or not parsed.hostname:
|
||||
die(
|
||||
"agent provider provisioning: pi settings base_url must be an "
|
||||
f"absolute URL (was {base_url!r})"
|
||||
)
|
||||
return parsed.hostname
|
||||
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="pi",
|
||||
command="pi",
|
||||
image="bot-bottle-pi:latest",
|
||||
prompt_mode="append_system_prompt",
|
||||
bypass_args=(),
|
||||
resume_args=(),
|
||||
remote_control_args=(),
|
||||
)
|
||||
|
||||
|
||||
class PiAgentProvider(AgentProvider):
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return _RUNTIME
|
||||
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
del auth_token, forward_host_credentials, host_env, trusted_project_path
|
||||
del label, color
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
guest_home = self.guest_home
|
||||
settings = dict(provider_settings or {})
|
||||
|
||||
models_payload, base_url, api_key_env, models, provider_name = (
|
||||
_pi_models_json(settings)
|
||||
)
|
||||
models_file = state_dir / "pi-models.json"
|
||||
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
|
||||
models_file.chmod(0o600)
|
||||
|
||||
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
|
||||
auth_scheme = "Bearer" if api_key_env else ""
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
guest_env=resolved_guest_env,
|
||||
has_prompt=has_prompt,
|
||||
startup_args=(
|
||||
"--models",
|
||||
",".join(f"{provider_name}/{model}" for model in models),
|
||||
),
|
||||
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
|
||||
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
|
||||
egress_routes=(EgressRoute(
|
||||
host=_route_host(base_url),
|
||||
auth_scheme=auth_scheme,
|
||||
token_ref=api_key_env,
|
||||
),),
|
||||
)
|
||||
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
prompt_path = _prompt_path(plan.guest_home)
|
||||
append_system_path = _append_system_path(plan.guest_home)
|
||||
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
|
||||
bottle.exec(
|
||||
f"mkdir -p {shlex.quote(plan.guest_home)}/.pi/agent && "
|
||||
f"cp {shlex.quote(prompt_path)} {shlex.quote(append_system_path)} && "
|
||||
f"chown node:node {shlex.quote(prompt_path)} "
|
||||
f"{shlex.quote(append_system_path)} && "
|
||||
f"chmod 600 {shlex.quote(prompt_path)} "
|
||||
f"{shlex.quote(append_system_path)}",
|
||||
user="root",
|
||||
)
|
||||
# Pi's `--append-system-prompt` takes literal text, not a file path.
|
||||
# Use its documented APPEND_SYSTEM.md discovery path instead.
|
||||
return None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
provision = plan.agent_provision
|
||||
_exec(
|
||||
bottle,
|
||||
_runtime_state_repair_script(plan.guest_home),
|
||||
"could not prepare pi runtime state",
|
||||
)
|
||||
for d in provision.dirs:
|
||||
path = shlex.quote(d.guest_path)
|
||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(d.owner)} {path}",
|
||||
f"could not chown {d.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(d.mode)} {path}",
|
||||
f"could not chmod {d.guest_path}",
|
||||
)
|
||||
for f in provision.files:
|
||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||
path = shlex.quote(f.guest_path)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(f.owner)} {path}",
|
||||
f"could not chown {f.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(f.mode)} {path}",
|
||||
f"could not chmod {f.guest_path}",
|
||||
)
|
||||
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
del plan, bottle, supervise_url
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
if detail:
|
||||
detail = f": {detail}"
|
||||
die(f"agent provider provisioning: {error}{detail}")
|
||||
@@ -1,52 +0,0 @@
|
||||
"""Deploy-key provisioner interface and factory (PRD 0048).
|
||||
|
||||
The core defines the abstract contract; concrete implementations live
|
||||
in `bot_bottle/contrib/<provider>/deploy_key_provisioner.py`. The
|
||||
factory `get_provisioner` imports contrib modules lazily so that a
|
||||
missing optional dependency in one provider doesn't break unrelated
|
||||
features."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class DeployKeyProvisioner(ABC):
|
||||
"""Manages a single deploy-key lifecycle on a remote forge."""
|
||||
|
||||
@abstractmethod
|
||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
||||
"""Generate a keypair and register the public half as a
|
||||
deploy key on the forge.
|
||||
|
||||
`owner_repo` is the `<owner>/<repo>` path (no `.git` suffix).
|
||||
`title` is the human-readable label shown in the forge UI.
|
||||
|
||||
Returns `(key_id, private_key_bytes)` where `key_id` is opaque
|
||||
to the caller and is only ever passed back to `delete`."""
|
||||
|
||||
@abstractmethod
|
||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
||||
"""Delete the registered deploy key.
|
||||
|
||||
Must not raise if the key is already absent (HTTP 404 is
|
||||
success). Must raise for all other failures so teardown halts."""
|
||||
|
||||
|
||||
def get_provisioner(
|
||||
provider: str, token: str, api_url: str
|
||||
) -> DeployKeyProvisioner:
|
||||
"""Instantiate the contrib provisioner for `provider`.
|
||||
|
||||
Raises `ManifestError` for unknown providers so the error surfaces
|
||||
at parse time rather than at runtime."""
|
||||
if provider == "gitea":
|
||||
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
|
||||
GiteaDeployKeyProvisioner,
|
||||
)
|
||||
return GiteaDeployKeyProvisioner(token=token, api_url=api_url)
|
||||
from .manifest_util import ManifestError
|
||||
raise ManifestError(
|
||||
f"unknown provisioned_key provider: {provider!r}; "
|
||||
f"available: gitea"
|
||||
)
|
||||
@@ -1,291 +0,0 @@
|
||||
"""DLP detectors for the egress proxy (PRD 0053).
|
||||
|
||||
Pure Python, no mitmproxy dependency. Each detector is a module-level
|
||||
function returning `ScanResult | None`.
|
||||
|
||||
Ships flat into the sidecar bundle image alongside
|
||||
`egress_addon_core.py` — both this file and the package source use
|
||||
the same try/except import shim pattern.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import gzip
|
||||
import re
|
||||
import typing
|
||||
import unicodedata
|
||||
from urllib.parse import quote as url_quote
|
||||
|
||||
try:
|
||||
from egress_addon_core import ScanResult # type: ignore[import-not-found]
|
||||
except ImportError: # pragma: no cover - host-side path
|
||||
from .egress_addon_core import ScanResult
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Snippet helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SNIPPET_CONTEXT = 40 # chars of surrounding text to include on each side
|
||||
REDACT = "********" # fixed-width replacement for the matched sensitive value
|
||||
|
||||
|
||||
def _snippet(text: str, start: int, end: int) -> str:
|
||||
"""Return context around a match with the matched span replaced by REDACT."""
|
||||
before = text[max(0, start - SNIPPET_CONTEXT):start].replace("\n", " ").replace("\r", " ")
|
||||
after = text[end:end + SNIPPET_CONTEXT].replace("\n", " ").replace("\r", " ")
|
||||
return f"{before}{REDACT}{after}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Unicode normalization (defeats confusable-char and combining-mark evasion)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _normalize_text(text: str) -> str:
|
||||
# NFKD separates base characters from combining marks and resolves
|
||||
# compatibility equivalents (fullwidth ASCII, ligatures, etc.)
|
||||
decomposed = unicodedata.normalize("NFKD", text)
|
||||
return "".join(
|
||||
ch for ch in decomposed
|
||||
# Strip combining marks inserted between chars to break patterns
|
||||
if unicodedata.category(ch) != "Mn"
|
||||
# Strip control chars; keep common whitespace (\n \r \t)
|
||||
and (unicodedata.category(ch) != "Cc" or ch in "\n\r\t")
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Token patterns detector
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
|
||||
("AWS access key", re.compile(r"AKIA[0-9A-Z]{16}")),
|
||||
("GitHub token (classic)", re.compile(r"ghp_[A-Za-z0-9_]{36}")),
|
||||
("GitHub fine-grained token", re.compile(r"github_pat_[A-Za-z0-9_]{82}")),
|
||||
("Anthropic API key", re.compile(r"sk-ant-[A-Za-z0-9\-_]{93}")),
|
||||
("OpenAI API key", re.compile(r"sk-[A-Za-z0-9]{48}")),
|
||||
("OpenAI project API key", re.compile(r"sk-proj-[A-Za-z0-9_\-]{48,}")),
|
||||
("Stripe live key", re.compile(r"sk_live_[A-Za-z0-9]{24}")),
|
||||
("Generic Bearer JWT", re.compile(r"Bearer\s+[A-Za-z0-9._\-]{50,}")),
|
||||
("HuggingFace token", re.compile(r"hf_[A-Za-z0-9]{34,}")),
|
||||
("Databricks token", re.compile(r"dapi[A-Za-z0-9]{32}")),
|
||||
("Slack token", re.compile(r"xox[baprs]-[A-Za-z0-9]+-[A-Za-z0-9]+-[A-Za-z0-9]{24,}")),
|
||||
("npm token", re.compile(r"npm_[A-Za-z0-9]{36}")),
|
||||
("SendGrid API key", re.compile(r"SG\.[A-Za-z0-9_\-]{22}\.[A-Za-z0-9_\-]{43}")),
|
||||
("PyPI token", re.compile(r"pypi-[A-Za-z0-9_\-]{80,}")),
|
||||
("HashiCorp Vault token", re.compile(r"hvs\.[A-Za-z0-9_\-]{24,}")),
|
||||
)
|
||||
|
||||
|
||||
def scan_token_patterns(text: str, *, location: str = "body") -> ScanResult | None:
|
||||
normalized = _normalize_text(text)
|
||||
for name, pattern in TOKEN_PATTERNS:
|
||||
m = pattern.search(normalized)
|
||||
if m is not None:
|
||||
return ScanResult(
|
||||
severity="block",
|
||||
reason=f"{name} found in {location}",
|
||||
location=location,
|
||||
context=_snippet(text, m.start(), m.end()),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def redact_tokens(
|
||||
text: str,
|
||||
*,
|
||||
env: typing.Mapping[str, str] | None = None,
|
||||
) -> str:
|
||||
"""Replace token pattern matches and (if env given) provisioned secrets with REDACT."""
|
||||
for _, pattern in TOKEN_PATTERNS:
|
||||
text = pattern.sub(REDACT, text)
|
||||
if env is not None:
|
||||
for key, value in env.items():
|
||||
if key.startswith("EGRESS_TOKEN_") and value:
|
||||
for variant in _encoded_variants(value):
|
||||
text = text.replace(variant, REDACT)
|
||||
return text
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Known secrets detector (Phase 1b)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _encoded_variants(secret: str) -> list[str]:
|
||||
"""Return the secret plus common encoded variants for exfil detection."""
|
||||
seen: set[str] = {secret}
|
||||
variants: list[str] = [secret]
|
||||
|
||||
def _add(v: str) -> None:
|
||||
if v not in seen:
|
||||
seen.add(v)
|
||||
variants.append(v)
|
||||
|
||||
secret_bytes = secret.encode("utf-8")
|
||||
|
||||
# Standard base64 — with and without padding
|
||||
b64 = base64.b64encode(secret_bytes).decode("ascii")
|
||||
_add(b64)
|
||||
_add(b64.rstrip("="))
|
||||
|
||||
# URL-safe base64 (JWT/OAuth use -_ alphabet) — with and without padding
|
||||
b64url = base64.urlsafe_b64encode(secret_bytes).decode("ascii")
|
||||
_add(b64url)
|
||||
_add(b64url.rstrip("="))
|
||||
|
||||
# URL percent-encoding
|
||||
_add(url_quote(secret, safe=""))
|
||||
|
||||
# Hex — lowercase and uppercase
|
||||
_add(secret_bytes.hex())
|
||||
_add(secret_bytes.hex().upper())
|
||||
|
||||
# Base32 (TOTP seeds, some DNS-exfil channels)
|
||||
_add(base64.b32encode(secret_bytes).decode("ascii"))
|
||||
|
||||
# gzip + base64 (deterministic: mtime=0); recognisable by H4sI prefix
|
||||
_add(base64.b64encode(gzip.compress(secret_bytes, mtime=0)).decode("ascii"))
|
||||
|
||||
return variants
|
||||
|
||||
|
||||
def scan_known_secrets(
|
||||
text: str,
|
||||
*,
|
||||
location: str = "body",
|
||||
env: typing.Mapping[str, str] | None = None,
|
||||
) -> ScanResult | None:
|
||||
if env is None:
|
||||
return None
|
||||
for key, value in env.items():
|
||||
if not key.startswith("EGRESS_TOKEN_") or not value:
|
||||
continue
|
||||
for variant in _encoded_variants(value):
|
||||
pos = text.find(variant)
|
||||
if pos >= 0:
|
||||
return ScanResult(
|
||||
severity="block",
|
||||
reason=f"provisioned secret from {key} found in {location}",
|
||||
location=location,
|
||||
context=_snippet(text, pos, pos + len(variant)),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Naive prompt injection detector (Phase 2)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DISCLOSURE_PHRASES: tuple[re.Pattern[str], ...] = (
|
||||
re.compile(r"(?i)system\s+prompt"),
|
||||
re.compile(r"(?i)my\s+instructions\s+are"),
|
||||
re.compile(r"(?i)original\s+instructions"),
|
||||
re.compile(r"(?i)secret\s+instructions"),
|
||||
re.compile(r"(?i)hidden\s+rules"),
|
||||
)
|
||||
|
||||
JAILBREAK_PHRASES: tuple[re.Pattern[str], ...] = (
|
||||
re.compile(r"(?i)ignore\s+previous"),
|
||||
re.compile(r"(?i)forget\s+everything"),
|
||||
re.compile(r"(?i)disregard\s+(?:all\s+)?(?:previous|prior)"),
|
||||
re.compile(r"(?i)pretend\s+you\s+are"),
|
||||
re.compile(r"(?i)act\s+as\s+(?:if|though)"),
|
||||
)
|
||||
|
||||
|
||||
PROXIMITY_CHARS = 500
|
||||
|
||||
|
||||
def _closest_pair(
|
||||
a_matches: list[re.Match[str]],
|
||||
b_matches: list[re.Match[str]],
|
||||
) -> tuple[re.Match[str], re.Match[str]] | None:
|
||||
"""Return the pair (a, b) with the smallest character gap, or None."""
|
||||
best: tuple[re.Match[str], re.Match[str]] | None = None
|
||||
best_gap: int | None = None
|
||||
for a in a_matches:
|
||||
for b in b_matches:
|
||||
gap = max(0, max(a.start(), b.start()) - min(a.end(), b.end()))
|
||||
if best_gap is None or gap < best_gap:
|
||||
best_gap = gap
|
||||
best = (a, b)
|
||||
return best
|
||||
|
||||
|
||||
def scan_naive_injection(text: str) -> ScanResult | None:
|
||||
location = "response body"
|
||||
disclosure_hits = [m for p in DISCLOSURE_PHRASES for m in p.finditer(text)]
|
||||
jailbreak_hits = [m for p in JAILBREAK_PHRASES for m in p.finditer(text)]
|
||||
|
||||
if disclosure_hits and jailbreak_hits:
|
||||
pair = _closest_pair(disclosure_hits, jailbreak_hits)
|
||||
if pair is not None:
|
||||
dist = max(0, max(pair[0].start(), pair[1].start()) - min(pair[0].end(), pair[1].end()))
|
||||
if dist <= PROXIMITY_CHARS:
|
||||
first = pair[0] if pair[0].start() <= pair[1].start() else pair[1]
|
||||
return ScanResult(
|
||||
severity="block",
|
||||
reason=(
|
||||
f"disclosure and jailbreak phrases within "
|
||||
f"{dist} chars in {location}"
|
||||
),
|
||||
location=location,
|
||||
context=_snippet(text, first.start(), first.end()),
|
||||
)
|
||||
|
||||
if disclosure_hits:
|
||||
m = disclosure_hits[0]
|
||||
return ScanResult(
|
||||
severity="warn",
|
||||
reason=f"prompt disclosure phrase detected in {location}",
|
||||
location=location,
|
||||
context=_snippet(text, m.start(), m.end()),
|
||||
)
|
||||
|
||||
if jailbreak_hits:
|
||||
m = jailbreak_hits[0]
|
||||
return ScanResult(
|
||||
severity="warn",
|
||||
reason=f"jailbreak phrase detected in {location}",
|
||||
location=location,
|
||||
context=_snippet(text, m.start(), m.end()),
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CRLF injection detector
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# URL-encoded CRLF is never legitimate in a request URL or header value.
|
||||
_CRLF_ENCODED_RE = re.compile(r"%0[dD]%0[aA]", re.ASCII)
|
||||
# Literal CRLF followed by a header-name pattern indicates header injection.
|
||||
_CRLF_HEADER_INJECT_RE = re.compile(r"\r\n[A-Za-z][A-Za-z0-9\-]+\s*:", re.ASCII)
|
||||
|
||||
|
||||
def scan_crlf_injection(text: str) -> ScanResult | None:
|
||||
if _CRLF_ENCODED_RE.search(text):
|
||||
return ScanResult(
|
||||
severity="block",
|
||||
reason="URL-encoded CRLF (%0d%0a) in outbound request",
|
||||
)
|
||||
if _CRLF_HEADER_INJECT_RE.search(text):
|
||||
return ScanResult(
|
||||
severity="block",
|
||||
reason="CRLF header injection pattern in outbound request",
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"REDACT",
|
||||
"SNIPPET_CONTEXT",
|
||||
"TOKEN_PATTERNS",
|
||||
"redact_tokens",
|
||||
"scan_crlf_injection",
|
||||
"scan_known_secrets",
|
||||
"scan_naive_injection",
|
||||
"scan_token_patterns",
|
||||
]
|
||||
+184
-195
@@ -1,59 +1,120 @@
|
||||
"""Per-bottle egress proxy (PRD 0017, PRD 0053).
|
||||
"""Per-bottle egress proxy (PRD 0017).
|
||||
|
||||
Replaces the cred-proxy sidecar (PRD 0010) with a mitmproxy-based
|
||||
sidecar that becomes the agent's `HTTP_PROXY` / `HTTPS_PROXY`. It
|
||||
owns three jobs:
|
||||
|
||||
1. MITM the agent's HTTPS with the per-bottle CA (moved from
|
||||
pipelock).
|
||||
2. Enforce manifest-declared `path_allowlist` per route.
|
||||
3. Inject `Authorization` headers for routes that declare an
|
||||
`auth` block, the same way cred-proxy does today.
|
||||
|
||||
This module defines the abstract proxy (`Egress`), its plan
|
||||
dataclass (`EgressPlan`), and the resolved per-route shape
|
||||
(`EgressRoute`). The sidecar's start/stop lifecycle is backend-
|
||||
specific and lives on concrete subclasses (see
|
||||
`bot_bottle/backend/docker/egress.py`).
|
||||
|
||||
Chunks 1+2 of the PRD: this module + the mitmproxy addon + the Docker
|
||||
lifecycle are wired into the agent's `HTTP_PROXY` path; cred-proxy
|
||||
has been removed. Chunk 3 retargets the cred-proxy-block remediation
|
||||
flow (PRD 0014) at egress and renames the MCP tool.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
from abc import ABC
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from .egress_addon_core import (
|
||||
HeaderMatch as CoreHeaderMatch,
|
||||
MatchEntry as CoreMatchEntry,
|
||||
PathMatch as CorePathMatch,
|
||||
Route,
|
||||
)
|
||||
from .log import die
|
||||
from .manifest import Bottle
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import ManifestBottle
|
||||
|
||||
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||
|
||||
# DNS name agents will dial for the per-bottle egress sidecar.
|
||||
# Backend-agnostic by contract: every concrete backend (Docker today,
|
||||
# others later) attaches this name to its sidecar on the bottle's
|
||||
# internal network. The agent's `HTTP_PROXY` env var resolves to
|
||||
# `http://egress:<port>` once chunk 2 cuts over.
|
||||
EGRESS_HOSTNAME = "egress"
|
||||
|
||||
# In-container path the addon reads. Pre-created in
|
||||
# `Dockerfile.sidecars` so the host bind-mount can drop the file
|
||||
# directly. Content is YAML (hand-rolled by `egress_render_routes`
|
||||
# in the style of `pipelock_render_yaml`, parsed by `yaml_subset`
|
||||
# inside the addon).
|
||||
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressRoute(Route):
|
||||
"""Host-side extension of the addon's `Route`.
|
||||
class EgressRoute:
|
||||
"""One resolved route on the egress sidecar.
|
||||
|
||||
Inherits `host`, `matches`, `auth_scheme`, and `token_env`
|
||||
from `egress_addon_core.Route` — those are the fields that cross the
|
||||
YAML wire into the sidecar. The fields below are host-only and
|
||||
are never serialised to the addon.
|
||||
`host` matches the request's hostname (case-insensitive). The
|
||||
optional `path_allowlist` constrains the URL path; empty tuple
|
||||
means no path-level filtering. The `auth_scheme` / `token_env` /
|
||||
`token_ref` triple is the credential-injection config; empty
|
||||
strings mean "no auth injection" (the manifest's nested `auth`
|
||||
block was omitted).
|
||||
|
||||
`token_ref` is the host env var the CLI reads at launch and forwards
|
||||
into the container's environ under `token_env`.
|
||||
`token_env` is the env-var slot inside the egress container
|
||||
(e.g. `EGRESS_TOKEN_0`); `token_ref` is the host env var
|
||||
the CLI reads at launch and forwards into the container's environ
|
||||
under `token_env`. Routes that share a `token_ref` coalesce to
|
||||
one `token_env` slot.
|
||||
|
||||
`roles` carries the manifest route's role tuple (reserved for
|
||||
future use; always empty today)."""
|
||||
`roles` carries the manifest route's optional role markers (see
|
||||
`manifest.EGRESS_ROLES`). The launch step reads these for
|
||||
side effects like the claude-code OAuth placeholder env."""
|
||||
|
||||
host: str
|
||||
path_allowlist: tuple[str, ...] = ()
|
||||
auth_scheme: str = ""
|
||||
token_env: str = ""
|
||||
token_ref: str = ""
|
||||
roles: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressPlan:
|
||||
"""Output of Egress.prepare; consumed by .start.
|
||||
|
||||
The slug + routes_path + routes + token_env_map fields are
|
||||
filled at prepare time (host-side, side-effect-free on docker).
|
||||
The network + CA + pipelock fields are populated by the backend's
|
||||
launch step via `dataclasses.replace` once those resources
|
||||
exist. Empty defaults are sentinels meaning "not yet set";
|
||||
`.start` validates that they are populated.
|
||||
|
||||
`token_env_map` is `{<token_env in container>: <token_ref on host>}`.
|
||||
The backend's start step reads `os.environ[token_ref]` and
|
||||
forwards the value into the egress container's environ
|
||||
under `token_env`. The plan itself never holds token values —
|
||||
secrets never land in a dataclass that might be logged.
|
||||
|
||||
`mitmproxy_ca_host_path` is the host path of the per-bottle
|
||||
egress CA (single PEM with cert+key concatenated) minted
|
||||
by `egress_tls_init`. `.start` docker-cps it into the
|
||||
sidecar at `~/.mitmproxy/mitmproxy-ca.pem` — mitmproxy reads
|
||||
that file at boot to mint per-host leaf certs.
|
||||
|
||||
`mitmproxy_ca_cert_only_host_path` is the cert-only PEM (no
|
||||
key) for installing into the agent's trust store via
|
||||
`provision_ca`. Separate file rather than re-parsing the
|
||||
concat so secrets and trust artefacts stay on distinct paths.
|
||||
|
||||
`pipelock_ca_host_path` is the host path of the pipelock CA
|
||||
(cert only). `.start` docker-cps it into the sidecar so the
|
||||
proxy's outbound HTTPS client trusts pipelock's MITM on the
|
||||
egress → upstream leg.
|
||||
|
||||
`pipelock_proxy_url` is the URL egress sets as `HTTPS_PROXY`
|
||||
in its environ so outbound HTTPS traverses pipelock — keeping
|
||||
pipelock's hostname allowlist + DLP body scanner on the
|
||||
egress → upstream leg.
|
||||
"""
|
||||
|
||||
slug: str
|
||||
routes_path: Path
|
||||
routes: tuple[EgressRoute, ...]
|
||||
@@ -62,77 +123,77 @@ class EgressPlan:
|
||||
egress_network: str = ""
|
||||
mitmproxy_ca_host_path: Path = Path()
|
||||
mitmproxy_ca_cert_only_host_path: Path = Path()
|
||||
log: int = 0
|
||||
pipelock_ca_host_path: Path = Path()
|
||||
pipelock_proxy_url: str = ""
|
||||
|
||||
|
||||
def egress_manifest_routes(
|
||||
bottle: ManifestBottle,
|
||||
bottle: Bottle,
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
"""Lift each `bottle.egress.routes[]` manifest entry into a
|
||||
resolved EgressRoute. Order is preserved so route lookup at
|
||||
the proxy is stable.
|
||||
|
||||
Token-env slots are assigned per distinct `token_ref`: the first
|
||||
authenticated route with `token_ref` "GH_PAT" gets
|
||||
`EGRESS_TOKEN_0`; a second route with the same `token_ref`
|
||||
shares slot 0. Unauthenticated routes (`auth` omitted) contribute
|
||||
no slot.
|
||||
|
||||
This is the effective set the addon enforces. Provider runtime
|
||||
routes are intentionally not injected implicitly; every allowed
|
||||
host must come from the home-owned bottle manifest."""
|
||||
out: list[EgressRoute] = []
|
||||
slot_for_token: dict[str, str] = {}
|
||||
for r in bottle.egress.routes:
|
||||
core_matches: list[CoreMatchEntry] = []
|
||||
for m in r.Matches:
|
||||
core_paths = tuple(
|
||||
CorePathMatch(type=p.Type, value=p.Value)
|
||||
for p in m.Paths
|
||||
)
|
||||
core_headers = tuple(
|
||||
CoreHeaderMatch(name=h.Name, value=h.Value, type=h.Type)
|
||||
for h in m.Headers
|
||||
)
|
||||
core_matches.append(CoreMatchEntry(
|
||||
paths=core_paths,
|
||||
methods=m.Methods,
|
||||
headers=core_headers,
|
||||
if r.AuthScheme and r.TokenRef:
|
||||
token_env = slot_for_token.get(r.TokenRef)
|
||||
if token_env is None:
|
||||
token_env = f"EGRESS_TOKEN_{len(slot_for_token)}"
|
||||
slot_for_token[r.TokenRef] = token_env
|
||||
out.append(EgressRoute(
|
||||
host=r.Host,
|
||||
path_allowlist=r.PathAllowlist,
|
||||
auth_scheme=r.AuthScheme,
|
||||
token_env=token_env,
|
||||
token_ref=r.TokenRef,
|
||||
roles=r.Role,
|
||||
))
|
||||
else:
|
||||
out.append(EgressRoute(
|
||||
host=r.Host,
|
||||
path_allowlist=r.PathAllowlist,
|
||||
roles=r.Role,
|
||||
))
|
||||
out.append(EgressRoute(
|
||||
host=r.Host,
|
||||
matches=tuple(core_matches),
|
||||
auth_scheme=r.AuthScheme,
|
||||
token_ref=r.TokenRef,
|
||||
roles=r.Role,
|
||||
git_fetch=r.GitFetch,
|
||||
outbound_detectors=r.OutboundDetectors,
|
||||
inbound_detectors=r.InboundDetectors,
|
||||
))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def egress_routes_for_bottle(
|
||||
bottle: ManifestBottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
bottle: Bottle,
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
manifest = egress_manifest_routes(bottle)
|
||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||
merged = list(provider_routes) + [
|
||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||
]
|
||||
return _assign_token_slots(merged)
|
||||
"""Effective egress routes. This is what gets rendered into
|
||||
routes.yaml + what the addon enforces.
|
||||
|
||||
|
||||
def _assign_token_slots(
|
||||
routes: list[EgressRoute],
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
slot_for_ref: dict[str, str] = {}
|
||||
out: list[EgressRoute] = []
|
||||
for r in routes:
|
||||
if r.auth_scheme and r.token_ref:
|
||||
slot = slot_for_ref.get(r.token_ref)
|
||||
if slot is None:
|
||||
slot = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
||||
slot_for_ref[r.token_ref] = slot
|
||||
out.append(dataclasses.replace(r, token_env=slot))
|
||||
else:
|
||||
out.append(r)
|
||||
return tuple(out)
|
||||
Operators that want to allow a host declare it directly in
|
||||
`bottle.egress.routes` as an authenticated route or bare-pass entry
|
||||
(`- host: <name>`). The legacy `bottle.egress.allowlist`
|
||||
folding is gone — egress is the single allowlist surface."""
|
||||
return egress_manifest_routes(bottle)
|
||||
|
||||
|
||||
def egress_token_env_map(
|
||||
routes: tuple[EgressRoute, ...],
|
||||
) -> dict[str, str]:
|
||||
"""Collapse the route list into `{token_env: token_ref}` for the
|
||||
authenticated routes. Routes without `auth` contribute no entry.
|
||||
|
||||
Conflict detection: two routes that share a `token_env` slot but
|
||||
name different `token_ref` host vars is a programming error in
|
||||
`egress_routes_for_bottle`; surface it as a die rather than
|
||||
silently picking one."""
|
||||
out: dict[str, str] = {}
|
||||
for r in routes:
|
||||
if not (r.auth_scheme and r.token_ref and r.token_env):
|
||||
if not r.token_env:
|
||||
continue
|
||||
existing = out.get(r.token_env)
|
||||
if existing is not None and existing != r.token_ref:
|
||||
@@ -145,120 +206,36 @@ def egress_token_env_map(
|
||||
return out
|
||||
|
||||
|
||||
def _route_to_yaml_fields(r: Route) -> dict[str, object]:
|
||||
fields: dict[str, object] = {"host": r.host}
|
||||
if r.auth_scheme and r.token_env:
|
||||
fields["auth_scheme"] = r.auth_scheme
|
||||
fields["token_env"] = r.token_env
|
||||
if r.matches:
|
||||
matches_data: list[dict[str, object]] = []
|
||||
for entry in r.matches:
|
||||
entry_data: dict[str, object] = {}
|
||||
if entry.paths:
|
||||
paths_data: list[dict[str, str]] = []
|
||||
for pm in entry.paths:
|
||||
pd: dict[str, str] = {"value": pm.value}
|
||||
if pm.type != "prefix":
|
||||
pd["type"] = pm.type
|
||||
paths_data.append(pd)
|
||||
entry_data["paths"] = paths_data
|
||||
if entry.methods:
|
||||
entry_data["methods"] = list(entry.methods)
|
||||
if entry.headers:
|
||||
headers_data: list[dict[str, str]] = []
|
||||
for hm in entry.headers:
|
||||
hd: dict[str, str] = {"name": hm.name, "value": hm.value}
|
||||
if hm.type != "exact":
|
||||
hd["type"] = hm.type
|
||||
headers_data.append(hd)
|
||||
entry_data["headers"] = headers_data
|
||||
matches_data.append(entry_data)
|
||||
fields["matches"] = matches_data
|
||||
if r.git_fetch:
|
||||
fields["git"] = {"fetch": True}
|
||||
if r.outbound_detectors is not None or r.inbound_detectors is not None:
|
||||
dlp: dict[str, object] = {}
|
||||
if r.outbound_detectors is not None:
|
||||
dlp["outbound_detectors"] = (
|
||||
False if not r.outbound_detectors
|
||||
else list(r.outbound_detectors)
|
||||
)
|
||||
if r.inbound_detectors is not None:
|
||||
dlp["inbound_detectors"] = (
|
||||
False if not r.inbound_detectors
|
||||
else list(r.inbound_detectors)
|
||||
)
|
||||
fields["dlp"] = dlp
|
||||
return fields
|
||||
|
||||
|
||||
def _render_match_entry(entry: dict[str, object]) -> list[str]:
|
||||
lines: list[str] = []
|
||||
first_key = True
|
||||
if "paths" in entry:
|
||||
lines.append(" - paths:")
|
||||
first_key = False
|
||||
for pd in entry["paths"]: # type: ignore[union-attr]
|
||||
pd_dict: dict[str, str] = pd # type: ignore[assignment]
|
||||
if "type" in pd_dict:
|
||||
lines.append(f' - type: "{pd_dict["type"]}"')
|
||||
lines.append(f' value: "{pd_dict["value"]}"')
|
||||
else:
|
||||
lines.append(f' - value: "{pd_dict["value"]}"')
|
||||
if "methods" in entry:
|
||||
methods_str = ", ".join(f'"{m}"' for m in entry["methods"]) # type: ignore[union-attr]
|
||||
prefix = " - " if first_key else " "
|
||||
lines.append(f'{prefix}methods: [{methods_str}]')
|
||||
first_key = False
|
||||
if "headers" in entry:
|
||||
prefix = " - " if first_key else " "
|
||||
lines.append(f"{prefix}headers:")
|
||||
first_key = False
|
||||
for hd in entry["headers"]: # type: ignore[union-attr]
|
||||
hd_dict: dict[str, str] = hd # type: ignore[assignment]
|
||||
lines.append(f' - name: "{hd_dict["name"]}"')
|
||||
lines.append(f' value: "{hd_dict["value"]}"')
|
||||
if first_key:
|
||||
lines.append(" - {}")
|
||||
return lines
|
||||
|
||||
|
||||
def egress_render_routes(
|
||||
routes: tuple[EgressRoute, ...],
|
||||
*,
|
||||
log: int = 0,
|
||||
) -> str:
|
||||
lines: list[str] = []
|
||||
if log:
|
||||
lines.append(f"log: {log}")
|
||||
lines.append("routes:")
|
||||
"""Serialize the route table for the addon to read.
|
||||
|
||||
YAML content — no token values, no host env-var names. The only
|
||||
thing the addon needs at runtime is the host → path_allowlist
|
||||
+ auth_scheme + in-container env-var mapping. The actual token
|
||||
values arrive via the container's environ.
|
||||
|
||||
Authenticated routes carry `auth_scheme` + `token_env`;
|
||||
unauthenticated routes omit both keys (the addon's parser
|
||||
enforces both-or-neither). Hand-rolled YAML in the style of
|
||||
`pipelock_render_yaml` so the addon's parser
|
||||
(`yaml_subset.parse_yaml_subset`) round-trips it cleanly."""
|
||||
lines: list[str] = ["routes:"]
|
||||
if not routes:
|
||||
lines[-1] = "routes: []"
|
||||
# `routes:` with an empty list on the same line — the parser
|
||||
# needs SOMETHING here. Empty inline list is the cleanest.
|
||||
lines[0] = "routes: []"
|
||||
return "\n".join(lines) + "\n"
|
||||
for r in routes:
|
||||
f = _route_to_yaml_fields(r)
|
||||
lines.append(f' - host: "{f["host"]}"')
|
||||
if "auth_scheme" in f:
|
||||
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
||||
lines.append(f' token_env: "{f["token_env"]}"')
|
||||
if "matches" in f:
|
||||
lines.append(" matches:")
|
||||
for entry in f["matches"]: # type: ignore[union-attr]
|
||||
lines.extend(_render_match_entry(entry)) # type: ignore[arg-type]
|
||||
if "git" in f:
|
||||
git_dict: dict[str, object] = f["git"] # type: ignore
|
||||
lines.append(" git:")
|
||||
if git_dict.get("fetch") is True:
|
||||
lines.append(" fetch: true")
|
||||
if "dlp" in f:
|
||||
dlp_dict: dict[str, object] = f["dlp"] # type: ignore
|
||||
lines.append(" dlp:")
|
||||
for dk, dv in dlp_dict.items():
|
||||
if dv is False:
|
||||
lines.append(f" {dk}: false")
|
||||
elif isinstance(dv, list):
|
||||
items_str = ", ".join(f'"{x}"' for x in dv)
|
||||
lines.append(f" {dk}: [{items_str}]")
|
||||
lines.append(f' - host: "{r.host}"')
|
||||
if r.auth_scheme and r.token_env:
|
||||
lines.append(f' auth_scheme: "{r.auth_scheme}"')
|
||||
lines.append(f' token_env: "{r.token_env}"')
|
||||
if r.path_allowlist:
|
||||
lines.append(" path_allowlist:")
|
||||
for p in r.path_allowlist:
|
||||
lines.append(f' - "{p}"')
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
@@ -266,6 +243,12 @@ def egress_resolve_token_values(
|
||||
token_env_map: dict[str, str],
|
||||
host_env: dict[str, str],
|
||||
) -> dict[str, str]:
|
||||
"""Read `host_env[TokenRef]` for each entry in `token_env_map` and
|
||||
return `{token_env: <value>}`. Dies (with a pointer at the missing
|
||||
var name) if any TokenRef is unset.
|
||||
|
||||
Pure function: takes the host env as an argument so tests can pass
|
||||
a sealed mapping without touching `os.environ`."""
|
||||
out: dict[str, str] = {}
|
||||
for token_env, token_ref in token_env_map.items():
|
||||
value = host_env.get(token_ref)
|
||||
@@ -286,28 +269,34 @@ def egress_resolve_token_values(
|
||||
|
||||
|
||||
class Egress(ABC):
|
||||
def prepare(
|
||||
self,
|
||||
bottle: ManifestBottle,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> EgressPlan:
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||
log = bottle.egress.Log
|
||||
"""The per-bottle egress proxy. Encapsulates the host-side prepare
|
||||
(route lift + routes.yaml render + token-env-map derivation); the
|
||||
sidecar's start/stop lifecycle is backend-specific and lives on
|
||||
concrete subclasses."""
|
||||
|
||||
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> EgressPlan:
|
||||
"""Lift `bottle.egress.routes` into resolved routes,
|
||||
render the routes file (mode 600) under `stage_dir`, and
|
||||
return the plan. Pure host-side, no docker subprocess. The
|
||||
token-env map records the mapping the launch step uses to
|
||||
forward values from the host's environ into the sidecar's
|
||||
environ.
|
||||
|
||||
Returned plan is incomplete: the launch step must fill
|
||||
`internal_network` / `egress_network` / `pipelock_proxy_url`
|
||||
via `dataclasses.replace` before passing it to `.start`."""
|
||||
routes = egress_routes_for_bottle(bottle)
|
||||
routes_path = stage_dir / "egress_routes.yaml"
|
||||
routes_path.write_text(egress_render_routes(routes, log=log))
|
||||
routes_path.write_text(egress_render_routes(routes))
|
||||
routes_path.chmod(0o600)
|
||||
return EgressPlan(
|
||||
slug=slug,
|
||||
routes_path=routes_path,
|
||||
routes=routes,
|
||||
token_env_map=egress_token_env_map(routes),
|
||||
log=log,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
|
||||
"EGRESS_HOSTNAME",
|
||||
"EGRESS_ROUTES_IN_CONTAINER",
|
||||
"Egress",
|
||||
|
||||
+87
-198
@@ -1,7 +1,28 @@
|
||||
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
|
||||
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017).
|
||||
|
||||
Loaded by `mitmdump -s /app/egress_addon.py` inside the
|
||||
egress container."""
|
||||
egress container. Wraps the pure logic from
|
||||
`egress_addon_core` with mitmproxy's HTTPFlow API:
|
||||
|
||||
- At startup, read `EGRESS_ROUTES` (default
|
||||
`/etc/egress/routes.yaml`, JSON content) → routes table.
|
||||
- SIGHUP re-reads the file and atomically swaps the in-memory
|
||||
table. A parse error keeps the old table in place — better to
|
||||
keep serving the old config than to leave the proxy with no
|
||||
routes after a typo.
|
||||
- On each `request`: strip the inbound Authorization header, then
|
||||
consult `decide()` for forward / block / inject-auth and apply
|
||||
the decision to the flow.
|
||||
|
||||
This file imports `mitmproxy` and is never imported on the host —
|
||||
mitmproxy is a container-only dependency. The host's tests target
|
||||
`egress_addon_core`.
|
||||
|
||||
Dockerfile.sidecars copies both this file and
|
||||
`egress_addon_core.py` flat into `/app/`; the absolute import
|
||||
below works because mitmdump runs with `/app` on its sys.path. The
|
||||
parallel file in the package source tree (bot_bottle/) is the
|
||||
build input — not a module the host imports."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -12,61 +33,57 @@ import signal
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
|
||||
from mitmproxy import http # type: ignore[import-not-found]
|
||||
|
||||
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
|
||||
LOG_BLOCKS,
|
||||
LOG_FULL,
|
||||
Config,
|
||||
build_inbound_scan_text,
|
||||
build_outbound_scan_text,
|
||||
decide,
|
||||
decide_git_fetch,
|
||||
is_git_fetch_request,
|
||||
is_git_push_request,
|
||||
load_config,
|
||||
match_route,
|
||||
outbound_scan_headers,
|
||||
scan_inbound,
|
||||
scan_outbound,
|
||||
)
|
||||
|
||||
try:
|
||||
from dlp_detectors import redact_tokens # type: ignore[import-not-found]
|
||||
except ImportError: # pragma: no cover - host-side path
|
||||
from bot_bottle.dlp_detectors import redact_tokens # type: ignore[import-not-found]
|
||||
# Absolute import (NOT `from .egress_addon_core`) — the
|
||||
# container drops both files flat into /app/ so they are sibling
|
||||
# top-level modules to mitmdump's loader, not a package.
|
||||
from egress_addon_core import Route, decide, is_git_push_request, load_routes # type: ignore[import-not-found]
|
||||
|
||||
|
||||
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
|
||||
|
||||
# Magic hostname the addon recognises as an introspection target.
|
||||
# Requests through the proxy for `_egress.local/<path>` are
|
||||
# intercepted and answered with synthetic responses (the addon's
|
||||
# `request` hook sets `flow.response` before any upstream connection).
|
||||
# The hostname is not in DNS — only clients dialing through this
|
||||
# specific egress can reach it, and only via HTTP (no TLS).
|
||||
# Used by the supervise sidecar's `list-egress-routes` MCP
|
||||
# tool to surface the live route table to the agent.
|
||||
INTROSPECT_HOST = "_egress.local"
|
||||
|
||||
|
||||
class EgressAddon:
|
||||
"""The mitmproxy addon. One instance per `mitmdump` process; the
|
||||
request hook is invoked on every CONNECT-decapsulated HTTP/HTTPS
|
||||
request the agent makes."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
|
||||
self.config: Config = Config(routes=())
|
||||
self.routes: tuple[Route, ...] = ()
|
||||
self._reload(initial=True)
|
||||
self._install_sighup()
|
||||
|
||||
def _reload(self, *, initial: bool = False) -> None:
|
||||
try:
|
||||
text = Path(self.routes_path).read_text(encoding="utf-8")
|
||||
new_config = load_config(text)
|
||||
new_routes = load_routes(text)
|
||||
except (OSError, ValueError) as e:
|
||||
tag = "boot" if initial else "SIGHUP"
|
||||
sys.stderr.write(
|
||||
f"egress: {tag} load failed: {e}\n"
|
||||
)
|
||||
if initial:
|
||||
self.config = Config(routes=())
|
||||
# No baseline to fall back on; serve nothing rather
|
||||
# than masquerade as a proxy with a route table the
|
||||
# operator never declared.
|
||||
self.routes = ()
|
||||
return
|
||||
self.config = new_config
|
||||
log_label = ("off", "blocks", "full")[self.config.log]
|
||||
self.routes = new_routes
|
||||
sys.stderr.write(
|
||||
f"egress: loaded {len(self.config.routes)} route(s): "
|
||||
f"{', '.join(r.host for r in self.config.routes)}"
|
||||
f" [log={log_label}]\n"
|
||||
f"egress: loaded {len(self.routes)} route(s): "
|
||||
f"{', '.join(r.host for r in self.routes)}\n"
|
||||
)
|
||||
|
||||
def _install_sighup(self) -> None:
|
||||
@@ -80,9 +97,14 @@ class EgressAddon:
|
||||
signal.signal(signal.SIGHUP, handler)
|
||||
|
||||
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
|
||||
"""Synthesize a response for `_egress.local` requests.
|
||||
Currently supports `/allowlist` which returns the in-memory
|
||||
route table as JSON (host, path_allowlist, auth_scheme,
|
||||
token_env per route — no token VALUES, those live in the
|
||||
container's environ)."""
|
||||
if path == "/allowlist":
|
||||
payload = json.dumps(
|
||||
{"routes": [dataclasses.asdict(r) for r in self.config.routes]},
|
||||
{"routes": [dataclasses.asdict(r) for r in self.routes]},
|
||||
indent=2,
|
||||
).encode("utf-8")
|
||||
flow.response = http.Response.make(
|
||||
@@ -96,194 +118,61 @@ class EgressAddon:
|
||||
{"Content-Type": "text/plain; charset=utf-8"},
|
||||
)
|
||||
|
||||
def _req_ctx(self, flow: http.HTTPFlow) -> dict[str, object]:
|
||||
return {
|
||||
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
|
||||
"method": flow.request.method,
|
||||
"path": redact_tokens(flow.request.path, env=os.environ),
|
||||
}
|
||||
|
||||
def _block(
|
||||
self,
|
||||
flow: http.HTTPFlow,
|
||||
reason: str,
|
||||
ctx: dict[str, object] | None = None,
|
||||
) -> None:
|
||||
if self.config.log >= LOG_BLOCKS:
|
||||
entry: dict[str, object] = {"event": "egress_block", "reason": reason}
|
||||
if ctx:
|
||||
entry.update(ctx)
|
||||
sys.stderr.write(json.dumps(entry) + "\n")
|
||||
flow.response = http.Response.make(
|
||||
403,
|
||||
reason.encode("utf-8"),
|
||||
{"Content-Type": "text/plain; charset=utf-8"},
|
||||
)
|
||||
|
||||
def _log_request(self, flow: http.HTTPFlow) -> None:
|
||||
sys.stderr.write(
|
||||
json.dumps({
|
||||
"event": "egress_request",
|
||||
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
|
||||
"method": flow.request.method,
|
||||
"path": redact_tokens(flow.request.path, env=os.environ),
|
||||
"headers": dict(flow.request.headers),
|
||||
"body": flow.request.get_text(strict=False) or "",
|
||||
})
|
||||
+ "\n"
|
||||
)
|
||||
|
||||
def _log_response(self, flow: http.HTTPFlow) -> None:
|
||||
sys.stderr.write(
|
||||
json.dumps({
|
||||
"event": "egress_response",
|
||||
"host": flow.request.pretty_host,
|
||||
"status": flow.response.status_code,
|
||||
"headers": dict(flow.response.headers),
|
||||
"body": flow.response.get_text(strict=False) or "",
|
||||
})
|
||||
+ "\n"
|
||||
)
|
||||
|
||||
# mitmproxy's addon API: this method name + signature is how
|
||||
# mitmdump discovers the request hook.
|
||||
def request(self, flow: http.HTTPFlow) -> None:
|
||||
request_path, _, query = flow.request.path.partition("?")
|
||||
|
||||
# Introspection: requests to the magic `_egress.local`
|
||||
# host are answered locally with a synthetic response. Check
|
||||
# before the strip-auth + route logic — these requests aren't
|
||||
# real upstream traffic, the agent isn't injecting auth, and
|
||||
# the addon's own decide() would 403 the magic host (it's
|
||||
# never in the routes table).
|
||||
if flow.request.pretty_host == INTROSPECT_HOST:
|
||||
self._serve_introspection(flow, request_path)
|
||||
return
|
||||
|
||||
# DLP outbound scan BEFORE stripping auth — catches tokens the
|
||||
# agent tried to smuggle in any header, path, query param, or body.
|
||||
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
|
||||
route = match_route(self.config.routes, flow.request.pretty_host)
|
||||
if route is not None:
|
||||
body = flow.request.get_text(strict=False) or ""
|
||||
scan_text = build_outbound_scan_text(
|
||||
flow.request.pretty_host,
|
||||
request_path,
|
||||
query,
|
||||
outbound_scan_headers(route, dict(flow.request.headers)),
|
||||
body,
|
||||
)
|
||||
dlp_result = scan_outbound(route, scan_text, os.environ)
|
||||
if dlp_result is not None and dlp_result.severity == "block":
|
||||
ctx = self._req_ctx(flow)
|
||||
if dlp_result.context:
|
||||
ctx = {**ctx, "context": dlp_result.context}
|
||||
self._block(flow, f"egress DLP: {dlp_result.reason}", ctx=ctx)
|
||||
return
|
||||
# Inbound Authorization is always stripped — the agent cannot
|
||||
# smuggle a stolen token through the proxy. If the matched
|
||||
# route declares an auth pair, a fresh header is injected
|
||||
# below.
|
||||
flow.request.headers.pop("authorization", None)
|
||||
|
||||
# Universal HTTPS git-push block. Defense-in-depth: git-gate
|
||||
# (PRD 0008) is the only sanctioned outbound path for git
|
||||
# writes — its pre-receive runs gitleaks. Letting HTTPS push
|
||||
# through egress + auth injection would route around
|
||||
# that scan, so we 403 before any route logic.
|
||||
if is_git_push_request(request_path, query):
|
||||
self._block(
|
||||
flow,
|
||||
"egress: git push over HTTPS is not supported; "
|
||||
"use the bottle.git SSH path (gitleaks-scanned by "
|
||||
"git-gate's pre-receive hook).",
|
||||
ctx=self._req_ctx(flow),
|
||||
flow.response = http.Response.make(
|
||||
403,
|
||||
(
|
||||
b"egress: git push over HTTPS is not supported; "
|
||||
b"use the bottle.git SSH path (gitleaks-scanned by "
|
||||
b"git-gate's pre-receive hook)."
|
||||
),
|
||||
{"Content-Type": "text/plain; charset=utf-8"},
|
||||
)
|
||||
return
|
||||
|
||||
if is_git_fetch_request(request_path, query):
|
||||
git_decision = decide_git_fetch(
|
||||
self.config.routes, flow.request.pretty_host,
|
||||
)
|
||||
if git_decision.action == "block":
|
||||
self._block(
|
||||
flow,
|
||||
git_decision.reason,
|
||||
ctx=self._req_ctx(flow),
|
||||
)
|
||||
return
|
||||
|
||||
# Strip agent-set Authorization after DLP scan so smuggled tokens
|
||||
# are caught above; the route may inject sidecar-owned auth below.
|
||||
flow.request.headers.pop("authorization", None)
|
||||
|
||||
# Build headers mapping for match evaluation
|
||||
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
|
||||
|
||||
decision = decide(
|
||||
self.config.routes,
|
||||
self.routes,
|
||||
flow.request.pretty_host,
|
||||
request_path,
|
||||
os.environ,
|
||||
request_method=flow.request.method,
|
||||
request_headers=req_headers,
|
||||
)
|
||||
|
||||
if decision.action == "block":
|
||||
self._block(flow, decision.reason, ctx=self._req_ctx(flow))
|
||||
flow.response = http.Response.make(
|
||||
403,
|
||||
decision.reason.encode("utf-8"),
|
||||
{"Content-Type": "text/plain; charset=utf-8"},
|
||||
)
|
||||
return
|
||||
|
||||
if decision.inject_authorization is not None:
|
||||
flow.request.headers["authorization"] = decision.inject_authorization
|
||||
|
||||
if self.config.log >= LOG_FULL:
|
||||
self._log_request(flow)
|
||||
|
||||
def response(self, flow: http.HTTPFlow) -> None:
|
||||
"""DLP inbound scan on response headers and body."""
|
||||
route = match_route(self.config.routes, flow.request.pretty_host)
|
||||
if route is None:
|
||||
return
|
||||
if flow.response is None:
|
||||
return
|
||||
if self.config.log >= LOG_FULL:
|
||||
self._log_response(flow)
|
||||
resp_headers = {k.lower(): v for k, v in flow.response.headers.items()}
|
||||
body = flow.response.get_text(strict=False) or ""
|
||||
scan_text = build_inbound_scan_text(resp_headers, body)
|
||||
if not scan_text:
|
||||
return
|
||||
result = scan_inbound(route, scan_text)
|
||||
if result is None:
|
||||
return
|
||||
resp_ctx: dict[str, object] = {
|
||||
**self._req_ctx(flow),
|
||||
"response_status": flow.response.status_code,
|
||||
}
|
||||
if result.context:
|
||||
resp_ctx = {**resp_ctx, "context": result.context}
|
||||
if result.severity == "block":
|
||||
self._block(flow, f"egress DLP: {result.reason}", ctx=resp_ctx)
|
||||
elif result.severity == "warn" and self.config.log >= LOG_BLOCKS:
|
||||
sys.stderr.write(
|
||||
json.dumps({
|
||||
"event": "egress_warn",
|
||||
"reason": f"egress DLP: {result.reason}",
|
||||
**resp_ctx,
|
||||
})
|
||||
+ "\n"
|
||||
)
|
||||
|
||||
def websocket_message(self, flow: http.HTTPFlow) -> None:
|
||||
"""DLP scan on WebSocket frames.
|
||||
|
||||
Outbound frames (from_client) are scanned for credential leakage;
|
||||
inbound frames are scanned for prompt injection. On a block the
|
||||
entire connection is killed — there is no HTTP response surface to
|
||||
write to after the upgrade.
|
||||
"""
|
||||
if flow.websocket is None: # type: ignore[union-attr]
|
||||
return
|
||||
route = match_route(self.config.routes, flow.request.pretty_host)
|
||||
if route is None:
|
||||
return
|
||||
message = flow.websocket.messages[-1] # type: ignore[union-attr]
|
||||
content = message.content.decode("utf-8", errors="replace")
|
||||
if message.from_client:
|
||||
result = scan_outbound(route, content, os.environ)
|
||||
if result is not None and result.severity == "block":
|
||||
sys.stderr.write(f"egress DLP: {result.reason}\n")
|
||||
flow.kill() # type: ignore[union-attr]
|
||||
else:
|
||||
result = scan_inbound(route, content)
|
||||
if result is not None:
|
||||
if result.severity == "block":
|
||||
sys.stderr.write(f"egress DLP: {result.reason}\n")
|
||||
flow.kill() # type: ignore[union-attr]
|
||||
elif result.severity == "warn":
|
||||
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
|
||||
|
||||
|
||||
addons = [EgressAddon()]
|
||||
|
||||
+123
-579
@@ -1,4 +1,4 @@
|
||||
"""Pure logic for the egress mitmproxy addon (PRD 0017, PRD 0053).
|
||||
"""Pure logic for the egress mitmproxy addon (PRD 0017).
|
||||
|
||||
Split out of `egress_addon.py` so the host's unit tests can
|
||||
exercise the parse + decision functions without depending on the
|
||||
@@ -8,277 +8,81 @@ container.
|
||||
|
||||
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
||||
ships flat into the sidecar bundle image alongside this file —
|
||||
see `Dockerfile.sidecars`)."""
|
||||
see `Dockerfile.sidecars`).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import typing
|
||||
from dataclasses import dataclass
|
||||
|
||||
# Absolute import — `yaml_subset.py` is copied flat into the bundle
|
||||
# image's `/app/` next to this file (via `Dockerfile.sidecars`).
|
||||
# The host-side unit tests run with the repo on sys.path, where the
|
||||
# import resolves under the `bot_bottle` package. The try/except
|
||||
# shim picks whichever import works.
|
||||
try:
|
||||
from yaml_subset import YamlSubsetError, parse_yaml_subset # type: ignore[import-not-found]
|
||||
except ImportError: # pragma: no cover - host-side path
|
||||
from .yaml_subset import YamlSubsetError, parse_yaml_subset
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Match types (Gateway API HTTPRoute vocabulary, PRD 0053)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PATH_MATCH_TYPES = ("exact", "prefix", "regex")
|
||||
HEADER_MATCH_TYPES = ("exact", "regex")
|
||||
|
||||
VALID_METHODS = frozenset({
|
||||
"GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "TRACE",
|
||||
"CONNECT",
|
||||
})
|
||||
|
||||
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
|
||||
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PathMatch:
|
||||
type: str # "exact" | "prefix" | "regex"
|
||||
value: str
|
||||
compiled: re.Pattern[str] | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HeaderMatch:
|
||||
name: str
|
||||
value: str
|
||||
type: str = "exact" # "exact" | "regex"
|
||||
compiled: re.Pattern[str] | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MatchEntry:
|
||||
paths: tuple[PathMatch, ...] = ()
|
||||
methods: tuple[str, ...] = ()
|
||||
headers: tuple[HeaderMatch, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Route:
|
||||
"""One row of the egress route table.
|
||||
|
||||
`host` is the request's `Host` header (or SNI hostname) to match
|
||||
against. `path_allowlist` is an optional tuple of absolute path
|
||||
prefixes the request path must start with; empty tuple means no
|
||||
path constraint. `auth_scheme` and `token_env` together form the
|
||||
credential-injection pair (both set or both empty); a non-empty
|
||||
pair tells the addon to overwrite the inbound Authorization with
|
||||
`<auth_scheme> <value-of-environ[token_env]>`.
|
||||
"""
|
||||
|
||||
host: str
|
||||
matches: tuple[MatchEntry, ...] = ()
|
||||
path_allowlist: tuple[str, ...] = ()
|
||||
auth_scheme: str = ""
|
||||
token_env: str = ""
|
||||
git_fetch: bool = False
|
||||
outbound_detectors: tuple[str, ...] | None = None
|
||||
inbound_detectors: tuple[str, ...] | None = None
|
||||
|
||||
|
||||
LOG_OFF = 0 # no logging
|
||||
LOG_BLOCKS = 1 # log block/warn events with request context
|
||||
LOG_FULL = 2 # log block/warn events + full request and response bodies
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Config:
|
||||
routes: tuple[Route, ...]
|
||||
log: int = LOG_OFF
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Decision:
|
||||
"""The result of `decide()`. Either forward (with optional
|
||||
`inject_authorization` header) or block (with a `reason` to surface
|
||||
to the agent)."""
|
||||
|
||||
action: str # "forward" or "block"
|
||||
reason: str = ""
|
||||
inject_authorization: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ScanResult:
|
||||
severity: str # "block" or "warn"
|
||||
reason: str
|
||||
location: str = "" # where the match was found, e.g. "body", "authorization header"
|
||||
context: str = "" # surrounding text with the match replaced by REDACT
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _parse_path_match(idx: int, j: int, raw: object) -> PathMatch:
|
||||
label = f"route[{idx}] matches paths[{j}]"
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{label}: must be an object")
|
||||
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||
ptype = raw_dict.get("type", "prefix")
|
||||
if not isinstance(ptype, str) or ptype not in PATH_MATCH_TYPES:
|
||||
raise ValueError(
|
||||
f"{label}: 'type' must be one of {', '.join(PATH_MATCH_TYPES)} "
|
||||
f"(got {ptype!r})"
|
||||
)
|
||||
value = raw_dict.get("value")
|
||||
if not isinstance(value, str) or not value:
|
||||
raise ValueError(f"{label}: 'value' must be a non-empty string")
|
||||
if ptype in ("exact", "prefix") and not value.startswith("/"):
|
||||
raise ValueError(
|
||||
f"{label}: value {value!r} must start with '/' for "
|
||||
f"type {ptype!r}"
|
||||
)
|
||||
compiled: re.Pattern[str] | None = None
|
||||
if ptype == "regex":
|
||||
try:
|
||||
compiled = re.compile(value)
|
||||
except re.error as e:
|
||||
raise ValueError(
|
||||
f"{label}: regex {value!r} failed to compile: {e}"
|
||||
) from e
|
||||
for k in raw_dict:
|
||||
if k not in ("type", "value"):
|
||||
raise ValueError(f"{label}: unknown key {k!r}")
|
||||
return PathMatch(type=ptype, value=value, compiled=compiled)
|
||||
|
||||
|
||||
def _parse_header_match(idx: int, j: int, raw: object) -> HeaderMatch:
|
||||
label = f"route[{idx}] matches headers[{j}]"
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{label}: must be an object")
|
||||
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||
name = raw_dict.get("name")
|
||||
if not isinstance(name, str) or not name:
|
||||
raise ValueError(f"{label}: 'name' must be a non-empty string")
|
||||
value = raw_dict.get("value")
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"{label}: 'value' must be a string")
|
||||
htype = raw_dict.get("type", "exact")
|
||||
if not isinstance(htype, str) or htype not in HEADER_MATCH_TYPES:
|
||||
raise ValueError(
|
||||
f"{label}: 'type' must be one of {', '.join(HEADER_MATCH_TYPES)} "
|
||||
f"(got {htype!r})"
|
||||
)
|
||||
compiled: re.Pattern[str] | None = None
|
||||
if htype == "regex":
|
||||
try:
|
||||
compiled = re.compile(value)
|
||||
except re.error as e:
|
||||
raise ValueError(
|
||||
f"{label}: regex {value!r} failed to compile: {e}"
|
||||
) from e
|
||||
for k in raw_dict:
|
||||
if k not in ("name", "value", "type"):
|
||||
raise ValueError(f"{label}: unknown key {k!r}")
|
||||
return HeaderMatch(name=name, value=value, type=htype, compiled=compiled)
|
||||
|
||||
|
||||
def _parse_match_entry(idx: int, k: int, raw: object) -> MatchEntry:
|
||||
label = f"route[{idx}] matches[{k}]"
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{label}: must be an object")
|
||||
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||
|
||||
paths: tuple[PathMatch, ...] = ()
|
||||
paths_raw = raw_dict.get("paths")
|
||||
if paths_raw is not None:
|
||||
if not isinstance(paths_raw, list):
|
||||
raise ValueError(f"{label}: 'paths' must be a list")
|
||||
paths_list = typing.cast(list[object], paths_raw)
|
||||
paths = tuple(_parse_path_match(idx, j, p) for j, p in enumerate(paths_list))
|
||||
|
||||
methods: tuple[str, ...] = ()
|
||||
methods_raw = raw_dict.get("methods")
|
||||
if methods_raw is not None:
|
||||
if not isinstance(methods_raw, list):
|
||||
raise ValueError(f"{label}: 'methods' must be a list")
|
||||
methods_list = typing.cast(list[object], methods_raw)
|
||||
normalised: list[str] = []
|
||||
for j, m in enumerate(methods_list):
|
||||
if not isinstance(m, str):
|
||||
raise ValueError(f"{label}: methods[{j}] must be a string")
|
||||
upper = m.upper()
|
||||
if upper not in VALID_METHODS:
|
||||
raise ValueError(
|
||||
f"{label}: methods[{j}] {m!r} is not a valid HTTP method"
|
||||
)
|
||||
normalised.append(upper)
|
||||
methods = tuple(normalised)
|
||||
|
||||
headers: tuple[HeaderMatch, ...] = ()
|
||||
headers_raw = raw_dict.get("headers")
|
||||
if headers_raw is not None:
|
||||
if not isinstance(headers_raw, list):
|
||||
raise ValueError(f"{label}: 'headers' must be a list")
|
||||
headers_list = typing.cast(list[object], headers_raw)
|
||||
headers = tuple(
|
||||
_parse_header_match(idx, j, h) for j, h in enumerate(headers_list)
|
||||
)
|
||||
|
||||
for key in raw_dict:
|
||||
if key not in ("paths", "methods", "headers"):
|
||||
raise ValueError(f"{label}: unknown key {key!r}")
|
||||
|
||||
return MatchEntry(paths=paths, methods=methods, headers=headers)
|
||||
|
||||
|
||||
def _parse_detectors(
|
||||
idx: int,
|
||||
host: str,
|
||||
raw_dict: dict[str, object],
|
||||
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
|
||||
"""Parse the optional `dlp` block on a route, returning
|
||||
(outbound_detectors, inbound_detectors)."""
|
||||
dlp_raw = raw_dict.get("dlp")
|
||||
if dlp_raw is None:
|
||||
return None, None
|
||||
label = f"route[{idx}] ({host})"
|
||||
if not isinstance(dlp_raw, dict):
|
||||
raise ValueError(f"{label}: 'dlp' must be an object")
|
||||
dlp = typing.cast(dict[str, object], dlp_raw)
|
||||
|
||||
def _parse_detector_field(
|
||||
field: str,
|
||||
valid_names: frozenset[str],
|
||||
) -> tuple[str, ...] | None:
|
||||
val = dlp.get(field)
|
||||
if val is None:
|
||||
return None
|
||||
if val is False:
|
||||
return ()
|
||||
if not isinstance(val, list):
|
||||
raise ValueError(
|
||||
f"{label}: dlp.{field} must be false, a list, or omitted"
|
||||
)
|
||||
items = typing.cast(list[object], val)
|
||||
names: list[str] = []
|
||||
for j, item in enumerate(items):
|
||||
if not isinstance(item, str):
|
||||
raise ValueError(
|
||||
f"{label}: dlp.{field}[{j}] must be a string"
|
||||
)
|
||||
if item not in valid_names:
|
||||
raise ValueError(
|
||||
f"{label}: dlp.{field}[{j}] {item!r} is not a valid "
|
||||
f"detector name; valid names: {', '.join(sorted(valid_names))}"
|
||||
)
|
||||
names.append(item)
|
||||
return tuple(names)
|
||||
|
||||
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
|
||||
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
|
||||
|
||||
for k in dlp:
|
||||
if k not in ("outbound_detectors", "inbound_detectors"):
|
||||
raise ValueError(
|
||||
f"{label}: dlp has unknown key {k!r}; accepted keys "
|
||||
f"are 'outbound_detectors', 'inbound_detectors'"
|
||||
)
|
||||
return outbound, inbound
|
||||
|
||||
|
||||
def parse_routes(payload: object) -> tuple[Route, ...]:
|
||||
"""Parse the routes-file payload (already JSON-decoded) into a
|
||||
tuple of `Route`s. Raises `ValueError` on any malformed entry —
|
||||
the caller decides whether to keep the old table or refuse to
|
||||
start.
|
||||
|
||||
Schema:
|
||||
{
|
||||
"routes": [
|
||||
{
|
||||
"host": "api.github.com",
|
||||
"path_allowlist": ["/repos/x/", "/users/x"], # optional
|
||||
"auth_scheme": "Bearer", # optional
|
||||
"token_env": "EGRESS_TOKEN_0" # optional
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
"""
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("routes payload: top-level must be an object")
|
||||
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
|
||||
raw: object = payload_dict.get("routes")
|
||||
raw = payload.get("routes")
|
||||
if not isinstance(raw, list):
|
||||
raise ValueError("routes payload: 'routes' must be a list")
|
||||
raw_list: list[object] = typing.cast(list[object], raw)
|
||||
out: list[Route] = []
|
||||
for i, r in enumerate(raw_list):
|
||||
for i, r in enumerate(raw):
|
||||
out.append(_parse_one(i, r))
|
||||
return tuple(out)
|
||||
|
||||
@@ -287,29 +91,35 @@ def _parse_one(idx: int, raw: object) -> Route:
|
||||
label = f"route[{idx}]"
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{label}: must be an object (got {type(raw).__name__})")
|
||||
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
|
||||
host: object = raw_dict.get("host")
|
||||
host = raw.get("host")
|
||||
if not isinstance(host, str) or not host:
|
||||
raise ValueError(f"{label}: 'host' must be a non-empty string")
|
||||
|
||||
# matches
|
||||
matches: tuple[MatchEntry, ...] = ()
|
||||
matches_raw = raw_dict.get("matches")
|
||||
if matches_raw is not None:
|
||||
if not isinstance(matches_raw, list):
|
||||
raise ValueError(f"{label} ({host}): 'matches' must be a list")
|
||||
matches_list = typing.cast(list[object], matches_raw)
|
||||
matches = tuple(
|
||||
_parse_match_entry(idx, k, m) for k, m in enumerate(matches_list)
|
||||
)
|
||||
path_allow_raw = raw.get("path_allowlist", [])
|
||||
if not isinstance(path_allow_raw, list):
|
||||
raise ValueError(f"{label} ({host}): 'path_allowlist' must be a list")
|
||||
prefixes: list[str] = []
|
||||
for j, p in enumerate(path_allow_raw):
|
||||
if not isinstance(p, str):
|
||||
raise ValueError(
|
||||
f"{label} ({host}): path_allowlist[{j}] must be a string"
|
||||
)
|
||||
if not p.startswith("/"):
|
||||
raise ValueError(
|
||||
f"{label} ({host}): path_allowlist[{j}] {p!r} must be an "
|
||||
f"absolute path prefix starting with '/'"
|
||||
)
|
||||
prefixes.append(p)
|
||||
|
||||
# auth (unchanged wire format)
|
||||
auth_scheme: object = raw_dict.get("auth_scheme", "")
|
||||
token_env: object = raw_dict.get("token_env", "")
|
||||
auth_scheme = raw.get("auth_scheme", "")
|
||||
token_env = raw.get("token_env", "")
|
||||
if not isinstance(auth_scheme, str):
|
||||
raise ValueError(f"{label} ({host}): 'auth_scheme' must be a string")
|
||||
if not isinstance(token_env, str):
|
||||
raise ValueError(f"{label} ({host}): 'token_env' must be a string")
|
||||
# Both-or-neither: 'auth' on the manifest side renders to this
|
||||
# pair atomically. A partial pair here means the renderer or a
|
||||
# hand-edited file is broken.
|
||||
if bool(auth_scheme) != bool(token_env):
|
||||
raise ValueError(
|
||||
f"{label} ({host}): 'auth_scheme' and 'token_env' must be both "
|
||||
@@ -317,50 +127,19 @@ def _parse_one(idx: int, raw: object) -> Route:
|
||||
f"token_env={token_env!r})"
|
||||
)
|
||||
|
||||
# git-over-HTTPS policy
|
||||
git_fetch = False
|
||||
git_raw = raw_dict.get("git")
|
||||
if git_raw is not None:
|
||||
if not isinstance(git_raw, dict):
|
||||
raise ValueError(f"{label} ({host}): 'git' must be an object")
|
||||
git_dict: dict[str, object] = typing.cast(dict[str, object], git_raw)
|
||||
fetch_raw = git_dict.get("fetch", False)
|
||||
if fetch_raw is True or fetch_raw is False:
|
||||
git_fetch = fetch_raw
|
||||
else:
|
||||
raise ValueError(f"{label} ({host}): 'git.fetch' must be a boolean")
|
||||
for k in git_dict:
|
||||
if k != "fetch":
|
||||
raise ValueError(
|
||||
f"{label} ({host}): git has unknown key {k!r}; "
|
||||
"accepted key is 'fetch'"
|
||||
)
|
||||
|
||||
# dlp detectors
|
||||
outbound_detectors, inbound_detectors = _parse_detectors(
|
||||
idx, host, raw_dict,
|
||||
)
|
||||
|
||||
for k in raw_dict:
|
||||
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git"):
|
||||
raise ValueError(
|
||||
f"{label} ({host}): unknown key {k!r}; accepted keys "
|
||||
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git'"
|
||||
)
|
||||
|
||||
return Route(
|
||||
host=host,
|
||||
matches=matches,
|
||||
path_allowlist=tuple(prefixes),
|
||||
auth_scheme=auth_scheme,
|
||||
token_env=token_env,
|
||||
git_fetch=git_fetch,
|
||||
outbound_detectors=outbound_detectors,
|
||||
inbound_detectors=inbound_detectors,
|
||||
)
|
||||
|
||||
|
||||
def load_routes(text: str) -> tuple[Route, ...]:
|
||||
"""Parse YAML text → routes."""
|
||||
"""Parse YAML text → routes. Raises `ValueError` for both
|
||||
decode and shape errors so callers handle them uniformly.
|
||||
`YamlSubsetError` from the parser is a `ValueError` subclass so
|
||||
it already satisfies the same surface; we let it propagate."""
|
||||
try:
|
||||
payload = parse_yaml_subset(text)
|
||||
except YamlSubsetError as e:
|
||||
@@ -368,102 +147,29 @@ def load_routes(text: str) -> tuple[Route, ...]:
|
||||
return parse_routes(payload)
|
||||
|
||||
|
||||
def parse_config(payload: object) -> "Config":
|
||||
"""Parse a full egress config payload (top-level log level + routes)."""
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("routes payload: top-level must be an object")
|
||||
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
|
||||
|
||||
log_raw: object = payload_dict.get("log", LOG_OFF)
|
||||
if log_raw is True or log_raw is False or not isinstance(log_raw, int) \
|
||||
or log_raw not in (LOG_OFF, LOG_BLOCKS, LOG_FULL):
|
||||
raise ValueError(
|
||||
f"routes payload: 'log' must be {LOG_OFF}, {LOG_BLOCKS}, or {LOG_FULL}"
|
||||
)
|
||||
|
||||
routes = parse_routes(payload)
|
||||
return Config(routes=routes, log=log_raw)
|
||||
|
||||
|
||||
def load_config(text: str) -> "Config":
|
||||
"""Parse YAML text → Config (routes + log flag)."""
|
||||
try:
|
||||
payload = parse_yaml_subset(text)
|
||||
except YamlSubsetError as e:
|
||||
raise ValueError(f"routes payload: invalid YAML: {e}") from e
|
||||
return parse_config(payload)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Match evaluation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _path_matches(pm: PathMatch, request_path: str) -> bool:
|
||||
if pm.type == "exact":
|
||||
return request_path == pm.value
|
||||
if pm.type == "prefix":
|
||||
if request_path == pm.value:
|
||||
return True
|
||||
if not pm.value.endswith("/"):
|
||||
return request_path.startswith(pm.value + "/")
|
||||
return request_path.startswith(pm.value)
|
||||
if pm.type == "regex" and pm.compiled is not None:
|
||||
return pm.compiled.search(request_path) is not None
|
||||
return False
|
||||
|
||||
|
||||
def _entry_matches(
|
||||
entry: MatchEntry,
|
||||
request_path: str,
|
||||
request_method: str,
|
||||
request_headers: typing.Mapping[str, str],
|
||||
) -> bool:
|
||||
"""All predicates within a MatchEntry are ANDed."""
|
||||
if entry.paths:
|
||||
if not any(_path_matches(pm, request_path) for pm in entry.paths):
|
||||
return False
|
||||
if entry.methods:
|
||||
if request_method.upper() not in entry.methods:
|
||||
return False
|
||||
if entry.headers:
|
||||
for hm in entry.headers:
|
||||
header_val = request_headers.get(hm.name.lower())
|
||||
if header_val is None:
|
||||
return False
|
||||
if hm.type == "exact":
|
||||
if header_val != hm.value:
|
||||
return False
|
||||
elif hm.type == "regex" and hm.compiled is not None:
|
||||
if not hm.compiled.search(header_val):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def evaluate_matches(
|
||||
route: Route,
|
||||
request_path: str,
|
||||
request_method: str = "GET",
|
||||
request_headers: typing.Mapping[str, str] | None = None,
|
||||
) -> bool:
|
||||
"""Return True if the request matches this route's match entries.
|
||||
Empty matches tuple means all requests match (bare-pass route)."""
|
||||
if not route.matches:
|
||||
return True
|
||||
hdrs: typing.Mapping[str, str] = request_headers or {}
|
||||
return any(
|
||||
_entry_matches(entry, request_path, request_method, hdrs)
|
||||
for entry in route.matches
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Git push detection (unchanged)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def is_git_push_request(path: str, query: str) -> bool:
|
||||
"""Return True if the request is a git smart-HTTP push.
|
||||
|
||||
git push over HTTPS hits two endpoints:
|
||||
GET <repo>/info/refs?service=git-receive-pack (capabilities)
|
||||
POST <repo>/git-receive-pack (the push)
|
||||
|
||||
Fetches use `service=git-upload-pack` / `/git-upload-pack` and
|
||||
are unaffected. Egress-proxy refuses HTTPS push because git-gate's
|
||||
pre-receive gitleaks scan is the gate for outbound git data;
|
||||
routing push through egress would bypass that. Use the
|
||||
bottle.git SSH path if you need to push.
|
||||
|
||||
Universal across routes — the block fires even when no
|
||||
egress route matches the host. A bare-pass route (host with
|
||||
no auth, no path_allowlist) would otherwise let push through to
|
||||
pipelock + upstream untouched.
|
||||
"""
|
||||
if path.endswith("/git-receive-pack"):
|
||||
return True
|
||||
if path.endswith("/info/refs"):
|
||||
# Query string is parsed leniently — `service=git-receive-pack`
|
||||
# may appear with other params in any order.
|
||||
for pair in query.split("&"):
|
||||
k, _, v = pair.partition("=")
|
||||
if k == "service" and v == "git-receive-pack":
|
||||
@@ -471,25 +177,18 @@ def is_git_push_request(path: str, query: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def is_git_fetch_request(path: str, query: str) -> bool:
|
||||
if path.endswith("/git-upload-pack"):
|
||||
return True
|
||||
if path.endswith("/info/refs"):
|
||||
for pair in query.split("&"):
|
||||
k, _, v = pair.partition("=")
|
||||
if k == "service" and v == "git-upload-pack":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Route lookup + decision
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def match_route(
|
||||
routes: typing.Sequence[Route],
|
||||
request_host: str,
|
||||
) -> Route | None:
|
||||
"""Return the first route whose `host` matches `request_host`
|
||||
exactly (case-insensitive). DNS names are case-insensitive.
|
||||
|
||||
Wildcard hosts (`*.foo.com`) are NOT supported — they caused
|
||||
too many edge cases (apex match? cert validation? pipelock
|
||||
mirror mismatch?) for too little payoff. Operators that need
|
||||
multiple subdomains declare them individually (or one common
|
||||
parent host as a bare-pass route)."""
|
||||
target = request_host.lower()
|
||||
for r in routes:
|
||||
if r.host.lower() == target:
|
||||
@@ -502,10 +201,24 @@ def decide(
|
||||
request_host: str,
|
||||
request_path: str,
|
||||
environ: typing.Mapping[str, str],
|
||||
*,
|
||||
request_method: str = "GET",
|
||||
request_headers: typing.Mapping[str, str] | None = None,
|
||||
) -> Decision:
|
||||
"""Pure decision: given a route table + request host + path + env,
|
||||
return what the addon should do with the request.
|
||||
|
||||
- No matching route → BLOCK. The route table is the bottle's
|
||||
egress allowlist; defense-in-depth complements pipelock's
|
||||
hostname gate on the downstream leg. A bottle that wants a
|
||||
host reachable from the agent must declare a route for it
|
||||
(bare-pass route — no `auth`, no `path_allowlist` — is fine
|
||||
for hosts that just need passthrough).
|
||||
- Matching route with `path_allowlist` set, request path doesn't
|
||||
start with any of the allowed prefixes → block with a clear
|
||||
reason.
|
||||
- Matching route with an auth pair → forward + inject
|
||||
Authorization. Token comes from `environ[route.token_env]`;
|
||||
missing/empty values block (route declared auth but the secret
|
||||
isn't here — operator misconfig).
|
||||
"""
|
||||
route = match_route(routes, request_host)
|
||||
if route is None:
|
||||
return Decision(
|
||||
@@ -517,15 +230,15 @@ def decide(
|
||||
),
|
||||
)
|
||||
|
||||
if not evaluate_matches(route, request_path, request_method, request_headers):
|
||||
return Decision(
|
||||
action="block",
|
||||
reason=(
|
||||
f"egress: request {request_method} {request_path!r} "
|
||||
f"does not match any entry in matches for "
|
||||
f"{route.host!r}"
|
||||
),
|
||||
)
|
||||
if route.path_allowlist:
|
||||
if not any(request_path.startswith(p) for p in route.path_allowlist):
|
||||
return Decision(
|
||||
action="block",
|
||||
reason=(
|
||||
f"egress: path {request_path!r} not in "
|
||||
f"path_allowlist for {route.host!r}"
|
||||
),
|
||||
)
|
||||
|
||||
if route.auth_scheme and route.token_env:
|
||||
token = environ.get(route.token_env, "")
|
||||
@@ -545,181 +258,12 @@ def decide(
|
||||
return Decision(action="forward")
|
||||
|
||||
|
||||
def decide_git_fetch(
|
||||
routes: typing.Sequence[Route],
|
||||
request_host: str,
|
||||
) -> Decision:
|
||||
route = match_route(routes, request_host)
|
||||
if route is not None and route.git_fetch:
|
||||
return Decision(action="forward")
|
||||
return Decision(
|
||||
action="block",
|
||||
reason=(
|
||||
"egress: git fetch/clone over HTTPS is not allowed by default; "
|
||||
"use git-gate for declared repos or set "
|
||||
"egress.routes[].git.fetch=true for explicit read-only "
|
||||
"HTTPS Git access."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DLP scan dispatch (PRD 0053)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def build_outbound_scan_text(
|
||||
host: str,
|
||||
path: str,
|
||||
query: str,
|
||||
headers: typing.Mapping[str, str],
|
||||
body: str,
|
||||
) -> str:
|
||||
"""Assemble all outbound request surfaces into one string for DLP scanning.
|
||||
|
||||
Covers hostname (DNS tunnelling), path, query params, all headers, body.
|
||||
"""
|
||||
parts: list[str] = [host, path]
|
||||
if query:
|
||||
parts.append(query)
|
||||
for name, value in headers.items():
|
||||
parts.append(f"{name}: {value}")
|
||||
if body:
|
||||
parts.append(body)
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def outbound_scan_headers(
|
||||
route: Route,
|
||||
headers: typing.Mapping[str, str],
|
||||
) -> dict[str, str]:
|
||||
"""Return request headers that should be included in outbound DLP.
|
||||
|
||||
Routes that inject sidecar-owned auth always strip the agent's
|
||||
Authorization header before forwarding. Scanning that header first
|
||||
creates false positives for provider clients that insist on sending
|
||||
their own bearer-shaped placeholder, while still not changing what
|
||||
reaches the upstream.
|
||||
"""
|
||||
out: dict[str, str] = {}
|
||||
skip_auth = bool(route.auth_scheme and route.token_env)
|
||||
for name, value in headers.items():
|
||||
if skip_auth and name.lower() == "authorization":
|
||||
continue
|
||||
out[name] = value
|
||||
return out
|
||||
|
||||
|
||||
def build_inbound_scan_text(
|
||||
headers: typing.Mapping[str, str],
|
||||
body: str,
|
||||
) -> str:
|
||||
"""Assemble inbound response surfaces into one string for DLP scanning.
|
||||
|
||||
Covers all response headers plus body.
|
||||
"""
|
||||
parts: list[str] = []
|
||||
for name, value in headers.items():
|
||||
parts.append(f"{name}: {value}")
|
||||
if body:
|
||||
parts.append(body)
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def _detector_enabled(
|
||||
configured: tuple[str, ...] | None,
|
||||
name: str,
|
||||
) -> bool:
|
||||
"""Check if a named detector is enabled for a route direction.
|
||||
None means all enabled; empty tuple means all disabled."""
|
||||
if configured is None:
|
||||
return True
|
||||
return name in configured
|
||||
|
||||
|
||||
def scan_outbound(
|
||||
route: Route,
|
||||
body: str | bytes,
|
||||
environ: typing.Mapping[str, str],
|
||||
) -> ScanResult | None:
|
||||
# Lazy import to avoid circular deps and keep dlp_detectors optional
|
||||
# at import time (the sidecar copies it flat alongside this file).
|
||||
try:
|
||||
from dlp_detectors import ( # type: ignore[import-not-found]
|
||||
scan_crlf_injection,
|
||||
scan_known_secrets,
|
||||
scan_token_patterns,
|
||||
)
|
||||
except ImportError: # pragma: no cover - host-side path
|
||||
from .dlp_detectors import ( # type: ignore[import-not-found]
|
||||
scan_crlf_injection,
|
||||
scan_known_secrets,
|
||||
scan_token_patterns,
|
||||
)
|
||||
|
||||
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
|
||||
|
||||
# CRLF injection is never legitimate — runs unconditionally, not gated
|
||||
# by outbound_detectors config.
|
||||
result = scan_crlf_injection(text)
|
||||
if result is not None:
|
||||
return result
|
||||
|
||||
if _detector_enabled(route.outbound_detectors, "token_patterns"):
|
||||
result = scan_token_patterns(text, location="body")
|
||||
if result is not None:
|
||||
return result
|
||||
|
||||
if _detector_enabled(route.outbound_detectors, "known_secrets"):
|
||||
result = scan_known_secrets(text, location="body", env=environ)
|
||||
if result is not None:
|
||||
return result
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def scan_inbound(
|
||||
route: Route,
|
||||
body: str | bytes,
|
||||
) -> ScanResult | None:
|
||||
try:
|
||||
from dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
|
||||
except ImportError: # pragma: no cover - host-side path
|
||||
from .dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
|
||||
|
||||
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
|
||||
|
||||
if _detector_enabled(route.inbound_detectors, "naive_injection_detection"):
|
||||
result = scan_naive_injection(text)
|
||||
if result is not None:
|
||||
return result
|
||||
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"LOG_BLOCKS",
|
||||
"LOG_FULL",
|
||||
"LOG_OFF",
|
||||
"Config",
|
||||
"Decision",
|
||||
"HeaderMatch",
|
||||
"MatchEntry",
|
||||
"PathMatch",
|
||||
"Route",
|
||||
"ScanResult",
|
||||
"build_inbound_scan_text",
|
||||
"build_outbound_scan_text",
|
||||
"decide",
|
||||
"decide_git_fetch",
|
||||
"evaluate_matches",
|
||||
"is_git_push_request",
|
||||
"is_git_fetch_request",
|
||||
"load_config",
|
||||
"load_routes",
|
||||
"match_route",
|
||||
"outbound_scan_headers",
|
||||
"parse_config",
|
||||
"parse_routes",
|
||||
"scan_inbound",
|
||||
"scan_outbound",
|
||||
]
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user