didericis/bot-bottle
1
0
Fork 0
Code Issues 7 Pull Requests 15 Actions Packages Projects Releases Wiki Activity

Compare commits

base: didericis/bot-bottle:731826f966a15016edda2cd42b04b7e7886f5c5b
Branches Tags
didericis/bot-bottle:main didericis/bot-bottle:core-coverage-badge didericis/bot-bottle:ratchet-supervise-90 didericis/bot-bottle:ratchet-manifest-90 didericis/bot-bottle:ratchet-git-gate-90 didericis/bot-bottle:ratchet-egress-core-90 didericis/bot-bottle:ratchet-yaml-subset-90 didericis/bot-bottle:ratchet-egress-addon-90 didericis/bot-bottle:cover-global-90 didericis/bot-bottle:table-drive-dlp-tests didericis/bot-bottle:flatten-deep-nesting didericis/bot-bottle:decompose-egress-dlp-config didericis/bot-bottle:cover-egress-addon-adapter didericis/bot-bottle:prd-egress-control-plane didericis/bot-bottle:prd-smolmachines-linux didericis/bot-bottle:fix-integration-test-failures didericis/bot-bottle:fix/macos-container-relative-dockerfile didericis/bot-bottle:prd-0054-install-script didericis/bot-bottle:commit-bottle-state didericis/bot-bottle:pr-211 didericis/bot-bottle:move-codex-auth-to-contrib didericis/bot-bottle:feat/pipelock-skip-scan-extensions didericis/bot-bottle:prd-0049-named-labelled-agents didericis/bot-bottle:harden-git-gate-shell-rendering
..
compare: didericis/bot-bottle:prd-0049-named-labelled-agents
Branches Tags
didericis/bot-bottle:core-coverage-badge didericis/bot-bottle:ratchet-supervise-90 didericis/bot-bottle:ratchet-manifest-90 didericis/bot-bottle:ratchet-git-gate-90 didericis/bot-bottle:ratchet-egress-core-90 didericis/bot-bottle:ratchet-yaml-subset-90 didericis/bot-bottle:ratchet-egress-addon-90 didericis/bot-bottle:cover-global-90 didericis/bot-bottle:table-drive-dlp-tests didericis/bot-bottle:flatten-deep-nesting didericis/bot-bottle:decompose-egress-dlp-config didericis/bot-bottle:cover-egress-addon-adapter didericis/bot-bottle:prd-egress-control-plane didericis/bot-bottle:prd-smolmachines-linux didericis/bot-bottle:main didericis/bot-bottle:fix-integration-test-failures didericis/bot-bottle:fix/macos-container-relative-dockerfile didericis/bot-bottle:prd-0054-install-script didericis/bot-bottle:commit-bottle-state didericis/bot-bottle:pr-211 didericis/bot-bottle:move-codex-auth-to-contrib didericis/bot-bottle:feat/pipelock-skip-scan-extensions didericis/bot-bottle:prd-0049-named-labelled-agents didericis/bot-bottle:harden-git-gate-shell-rendering

1 Commits
731826f966 .. prd-0049-named-labelled-agents

Author SHA1 Message Date
didericis-claude 3997a0a721 docs(prd): add PRD 0049 — named/labelled agents
test / unit (pull_request) Successful in 39s
Details
test / integration (pull_request) Successful in 53s
Details 
Draft PRD for prompting operators for a custom label and optional
ANSI color at agent launch time, storing both in metadata.json, and
surfacing the label (in color) in the dashboard's active-agents pane.

Closes #171
 2026-06-03 21:38:38 -04:00
232 changed files with 8770 additions and 21010 deletions
Expand all files Collapse all files
.gitea/workflows/canaries.yml
+3 -3
View File
@@ -1,6 +1,6 @@
# Weekly canary suite. Catches upstream regressions (broken pinned
# digest, etc.) without coupling every dev push to upstream registry
# availability.
# Weekly canary suite. Catches upstream regressions (broken pipelock
# image packaging at the pinned digest, etc.) without coupling every
# dev push to upstream registry availability.
#
# Opt-in via CLAUDE_BOTTLE_RUN_CANARIES=1 so the same files can be run
# locally with the same gating.
.gitea/workflows/lint.yml
-34
View File
@@ -1,34 +0,0 @@
name: lint
on:
push:
paths:
- "**.py"
- ".pylintrc"
- ".gitea/workflows/lint.yml"
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.12"
- name: Install dev dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Run pylint
run: |
# Run pylint on all Python files in the repo
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0
- name: Run pyright
run: |
# Run pyright type checking
pyright .
.gitea/workflows/prd-number.yml
-125
View File
@@ -1,125 +0,0 @@
# Assign sequential numbers to prd-new-*.md files on merge to main.
#
# When a PR merges to main and includes prd-new-*.md files this workflow:
# 1. Finds the next available NNNN number by scanning existing PRDs.
# 2. Renames each prd-new-*.md to NNNN-<slug>.md.
# 3. Updates the title header (# PRD prd-new: → # PRD NNNN:).
# 4. Flips Status: Draft → Active when the push touched files outside
# docs/prds/ anywhere in its commit range (i.e. the implementation
# shipped together with the PRD).
# 5. Commits the renaming back to main.
#
# No-op if the working tree contains no prd-new-*.md files.
#
# NOTE: The workflow scans the working tree (not just HEAD~1..HEAD) because
# PRs land as multi-commit pushes and the prd-new file is often added in an
# earlier commit on the branch, not in the final squash/merge commit.
name: prd-number
on:
push:
branches:
- main
paths:
- 'docs/prds/prd-new-*.md'
jobs:
assign-numbers:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Assign PRD numbers
run: |
python3 - <<'EOF'
import os
import re
import subprocess
import sys
from pathlib import Path
prds_dir = Path("docs/prds")
# Scan the working tree — prd-new files may have landed in any
# commit of a multi-commit push, not just HEAD.
new_prds = sorted(prds_dir.glob("prd-new-*.md"))
if not new_prds:
print("No prd-new-*.md files found — nothing to do.")
sys.exit(0)
# Determine whether non-PRD files were also changed anywhere in
# the push range (BEFORE_SHA → HEAD). Falls back to HEAD~1 when
# the env var isn't set (e.g. local act runs).
before_sha = os.environ.get("GITHUB_EVENT_BEFORE", "HEAD~1")
all_changed = subprocess.run(
["git", "diff", "--name-only", before_sha, "HEAD"],
capture_output=True, text=True, check=True,
).stdout.splitlines()
non_prd_changed = any(
not f.startswith("docs/prds/") for f in all_changed
)
# Find next available number.
existing = sorted(
int(m.group(1))
for p in prds_dir.glob("*.md")
if (m := re.match(r"^(\d{4})-", p.name))
)
next_num = (max(existing) + 1) if existing else 1
for prd_path in sorted(new_prds):
slug = re.sub(r"^prd-new-", "", prd_path.stem)
new_name = f"{next_num:04d}-{slug}.md"
new_path = prds_dir / new_name
print(f" {prd_path.name} → {new_name}")
content = prd_path.read_text()
# Update title header.
content = re.sub(
r"^(#\s+PRD\s+)prd-new(:)",
rf"\g<1>{next_num:04d}\2",
content,
count=1,
flags=re.MULTILINE,
)
# Conditionally flip Status.
if non_prd_changed:
content = re.sub(
r"(\*\*Status:\*\*\s*)Draft",
r"\g<1>Active",
content,
count=1,
)
new_path.write_text(content)
subprocess.run(["git", "rm", str(prd_path)], check=True)
subprocess.run(["git", "add", str(new_path)], check=True)
next_num += 1
subprocess.run(
["git", "commit", "-m", "ci(prd): assign sequential numbers to new PRDs"],
check=True,
)
subprocess.run(["git", "push"], check=True)
EOF
.gitea/workflows/test.yml
-4
View File
@@ -21,11 +21,7 @@ on:
push:
branches:
- main
paths:
- '**.py'
pull_request:
paths:
- '**.py'
jobs:
unit:
.gitea/workflows/update-badges.yml
-79
View File
@@ -1,79 +0,0 @@
name: Update Quality Badges
on:
push:
branches:
- main
paths:
- '**.py'
- '.pylintrc'
- 'pyrightconfig.json'
workflow_dispatch:
jobs:
update-badges:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'
- name: Install dev dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Run pylint and extract score
id: pylint
run: |
PYLINT_OUTPUT=$(python -m pylint bot_bottle/ 2>&1) || true
SCORE=$(echo "$PYLINT_OUTPUT" | grep -oP '(?<=rated at )\d+\.\d+/10' | head -1)
echo "score=$SCORE" >> $GITHUB_OUTPUT
echo "Pylint score: $SCORE"
- name: Run pyright and check errors
id: pyright
run: |
PYRIGHT_OUTPUT=$(python -m pyright 2>&1) || true
ERRORS=$(echo "$PYRIGHT_OUTPUT" | grep -oP '\d+(?= error)' | head -1)
echo "errors=$ERRORS" >> $GITHUB_OUTPUT
echo "Pyright errors: $ERRORS"
- name: Update badges in README
run: |
PYLINT_SCORE="${{ steps.pylint.outputs.score }}"
PYRIGHT_ERRORS="${{ steps.pyright.outputs.errors }}"
PYLINT_SCORE_ENCODED=$(echo "$PYLINT_SCORE" | sed 's|/|%2F|g')
if [ -n "$PYLINT_SCORE_ENCODED" ]; then
sed -i "s|/badge/pylint-[^)]*|/badge/pylint-${PYLINT_SCORE_ENCODED}-brightgreen|" README.md
fi
if [ -n "$PYRIGHT_ERRORS" ]; then
sed -i "s|/badge/pyright-[^)]*|/badge/pyright-${PYRIGHT_ERRORS}%20errors-brightgreen|" README.md
fi
echo "Updated badges:"
grep -E "pylint|pyright" README.md | head -2
- name: Commit and push badge updates
run: |
git config --local user.email "action@gitea.local"
git config --local user.name "Quality Badge Bot"
# Check if there are changes
if git diff --quiet README.md; then
echo "No badge changes needed"
else
echo "Badge changes detected, committing..."
git add README.md
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n\n'"[skip ci]"
git commit -m "$MSG"
git push
fi
.pylintrc
-632
View File
@@ -1,632 +0,0 @@
[MAIN]
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Clear in-memory caches upon conclusion of linting. Useful if running pylint
# in a server-like mode.
clear-cache-post-run=no
# Load and enable all available extensions. Use --list-extensions to see a list
# all available extensions.
#enable-all-extensions=
# In error mode, messages with a category besides ERROR or FATAL are
# suppressed, and no reports are done by default. Error mode is compatible with
# disabling specific errors.
#errors-only=
# Always return a 0 (non-error) status code, even if lint errors are found.
# This is primarily useful in continuous integration scripts.
#exit-zero=
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code.
extension-pkg-allow-list=
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
# for backward compatibility.)
extension-pkg-whitelist=
# Return non-zero exit code if any of these messages/categories are detected,
# even if score is above --fail-under value. Syntax same as enable. Messages
# specified are enabled, while categories only check already-enabled messages.
fail-on=
# Specify a score threshold under which the program will exit with error.
fail-under=10
# Interpret the stdin as a python script, whose filename needs to be passed as
# the module_or_package argument.
#from-stdin=
# Files or directories to be skipped. They should be base names, not paths.
ignore=CVS
# Add files or directories matching the regular expressions patterns to the
# ignore-list. The regex matches against paths and can be in Posix or Windows
# format. Because '\\' represents the directory delimiter on Windows systems,
# it can't be used as an escape character.
ignore-paths=
# Files or directories matching the regular expression patterns are skipped.
# The regex matches against base names, not paths. The default value ignores
# Emacs file locks
ignore-patterns=^\.#
# List of module names for which member attributes should not be checked and
# will not be imported (useful for modules/projects where namespaces are
# manipulated during runtime and thus existing member attributes cannot be
# deduced by static analysis). It supports qualified module names, as well as
# Unix pattern matching.
ignored-modules=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
# number of processors available to use, and will cap the count on Windows to
# avoid hangs.
jobs=1
# Control the amount of potential inferred values when inferring a single
# object. This can help the performance when dealing with large functions or
# complex, nested conditions.
limit-inference-results=100
# List of plugins (as comma separated values of python module names) to load,
# usually to register additional checkers.
load-plugins=
# Pickle collected data for later comparisons.
persistent=yes
# Resolve imports to .pyi stubs if available. May reduce no-member messages and
# increase not-an-iterable messages.
prefer-stubs=no
# Minimum Python version to use for version dependent checks. Will default to
# the version used to run pylint.
py-version=3.14
# Discover python modules and packages in the file system subtree.
recursive=no
# Add paths to the list of the source roots. Supports globbing patterns. The
# source root is an absolute path or a path relative to the current working
# directory used to determine a package namespace for modules located under the
# source root.
source-roots=
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
# In verbose mode, extra non-checker-related info will be displayed.
#verbose=
[BASIC]
# Naming style matching correct argument names.
argument-naming-style=snake_case
# Regular expression matching correct argument names. Overrides argument-
# naming-style. If left empty, argument names will be checked with the set
# naming style.
#argument-rgx=
# Naming style matching correct attribute names.
attr-naming-style=snake_case
# Regular expression matching correct attribute names. Overrides attr-naming-
# style. If left empty, attribute names will be checked with the set naming
# style.
#attr-rgx=
# Bad variable names which should always be refused, separated by a comma.
bad-names=foo,
bar,
baz,
toto,
tutu,
tata
# Bad variable names regexes, separated by a comma. If names match any regex,
# they will always be refused
bad-names-rgxs=
# Naming style matching correct class attribute names.
class-attribute-naming-style=any
# Regular expression matching correct class attribute names. Overrides class-
# attribute-naming-style. If left empty, class attribute names will be checked
# with the set naming style.
#class-attribute-rgx=
# Naming style matching correct class constant names.
class-const-naming-style=UPPER_CASE
# Regular expression matching correct class constant names. Overrides class-
# const-naming-style. If left empty, class constant names will be checked with
# the set naming style.
#class-const-rgx=
# Naming style matching correct class names.
class-naming-style=PascalCase
# Regular expression matching correct class names. Overrides class-naming-
# style. If left empty, class names will be checked with the set naming style.
#class-rgx=
# Naming style matching correct constant names.
const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style. If left empty, constant names will be checked with the set naming
# style.
#const-rgx=
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming style matching correct function names.
function-naming-style=snake_case
# Regular expression matching correct function names. Overrides function-
# naming-style. If left empty, function names will be checked with the set
# naming style.
#function-rgx=
# Good variable names which should always be accepted, separated by a comma.
good-names=i,
j,
k,
ex,
Run,
_
# Good variable names regexes, separated by a comma. If names match any regex,
# they will always be accepted
good-names-rgxs=
# Include a hint for the correct naming format with invalid-name.
include-naming-hint=no
# Naming style matching correct inline iteration names.
inlinevar-naming-style=any
# Regular expression matching correct inline iteration names. Overrides
# inlinevar-naming-style. If left empty, inline iteration names will be checked
# with the set naming style.
#inlinevar-rgx=
# Naming style matching correct method names.
method-naming-style=snake_case
# Regular expression matching correct method names. Overrides method-naming-
# style. If left empty, method names will be checked with the set naming style.
#method-rgx=
# Naming style matching correct module names.
module-naming-style=snake_case
# Regular expression matching correct module names. Overrides module-naming-
# style. If left empty, module names will be checked with the set naming style.
#module-rgx=
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# Regular expression matching correct parameter specification variable names.
# If left empty, parameter specification variable names will be checked with
# the set naming style.
#paramspec-rgx=
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
# These decorators are taken in consideration only for invalid-name.
property-classes=abc.abstractproperty
# Regular expression matching correct type alias names. If left empty, type
# alias names will be checked with the set naming style.
#typealias-rgx=
# Regular expression matching correct type variable names. If left empty, type
# variable names will be checked with the set naming style.
#typevar-rgx=
# Regular expression matching correct type variable tuple names. If left empty,
# type variable tuple names will be checked with the set naming style.
#typevartuple-rgx=
# Naming style matching correct variable names.
variable-naming-style=snake_case
# Regular expression matching correct variable names. Overrides variable-
# naming-style. If left empty, variable names will be checked with the set
# naming style.
#variable-rgx=
[CLASSES]
# Warn about protected attribute access inside special methods
check-protected-access-in-special-methods=no
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,
__new__,
setUp,
asyncSetUp,
__post_init__
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,_fields,_replace,_source,_make,os._exit
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# List of regular expressions of class ancestor names to ignore when counting
# public methods (see R0903)
exclude-too-few-public-methods=
# List of qualified class names to ignore when counting class parents (see
# R0901)
ignored-parents=
# Maximum number of arguments for function / method.
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in an if statement (see R0916).
max-bool-expr=5
# Maximum number of branch for function / method body.
max-branches=12
# Maximum number of locals for function / method body.
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of positional arguments for function / method.
max-positional-arguments=5
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body.
max-returns=6
# Maximum number of statements in function / method body.
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[EXCEPTIONS]
# Exceptions that will emit a warning when caught.
overgeneral-exceptions=builtins.BaseException,builtins.Exception
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Maximum number of characters on a single line. Pylint's default of 100 is
# based on PEP 8's guidance that teams may choose line lengths up to 99
# characters.
max-line-length=100
# Maximum number of lines in a module.
max-module-lines=1000
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[LOGGING]
# The type of string formatting that logging methods do. `old` means using %
# formatting, `new` is for `{}` formatting.
logging-format-style=old
# Logging modules to check that the string format arguments are in logging
# function parameter format.
logging-modules=logging
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, CONTROL_FLOW, INFERENCE, INFERENCE_FAILURE,
# UNDEFINED.
confidence=HIGH,
CONTROL_FLOW,
INFERENCE,
INFERENCE_FAILURE,
UNDEFINED
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once). You can also use "--disable=all" to
# disable everything first and then re-enable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use "--disable=all --enable=classes
# --disable=W".
disable=raw-checker-failed,
bad-inline-option,
locally-disabled,
file-ignored,
suppressed-message,
useless-suppression,
deprecated-pragma,
use-symbolic-message-instead,
use-implicit-booleaness-not-comparison-to-string,
use-implicit-booleaness-not-comparison-to-zero,
missing-function-docstring,
missing-class-docstring,
missing-module-docstring,
invalid-name,
cyclic-import,
too-many-arguments,
too-many-locals,
too-many-branches,
too-many-statements,
too-many-instance-attributes,
duplicate-code,
import-outside-toplevel,
too-few-public-methods,
unnecessary-ellipsis
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=
[METHOD_ARGS]
# List of qualified names (i.e., library.method) which require a timeout
# parameter e.g. 'requests.api.get,requests.api.post'
timeout-methods=requests.api.delete,requests.api.get,requests.api.head,requests.api.options,requests.api.patch,requests.api.post,requests.api.put,requests.api.request
[MISCELLANEOUS]
# Whether or not to search for fixme's in docstrings.
check-fixme-in-docstring=no
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,
XXX,
TODO
# Regular expression of note tags to take in consideration.
notes-rgx=
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
# Complete name of functions that never returns. When checking for
# inconsistent-return-statements if a never returning function is called then
# it will be considered as an explicit return statement and no message will be
# printed.
never-returning-functions=sys.exit,argparse.parse_error
# Let 'consider-using-join' be raised when the separator to join on would be
# non-empty (resulting in expected fixes of the type: ``"- " + " -
# ".join(items)``)
suggest-join-with-non-empty-separator=yes
[REPORTS]
# Python expression which should return a score less than or equal to 10. You
# have access to the variables 'fatal', 'error', 'warning', 'refactor',
# 'convention', and 'info' which contain the number of messages in each
# category, as well as 'statement' which is the total number of statements
# analyzed. This score is used by the global evaluation report (RP0004).
evaluation=max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details.
msg-template=
# Set the output format. Available formats are: 'text', 'parseable',
# 'colorized', 'json2' (improved json format), 'json' (old json format), msvs
# (visual studio) and 'github' (GitHub actions). You can also give a reporter
# class, e.g. mypackage.mymodule.MyReporterClass.
#output-format=
# Tells whether to display a full report or only the messages.
reports=no
# Activate the evaluation score.
score=yes
[SIMILARITIES]
# Comments are removed from the similarity computation
ignore-comments=yes
# Docstrings are removed from the similarity computation
ignore-docstrings=yes
# Imports are removed from the similarity computation
ignore-imports=yes
# Signatures are removed from the similarity computation
ignore-signatures=yes
# Minimum lines number of a similarity.
min-similarity-lines=4
[SPELLING]
# Limits count of emitted suggestions for spelling mistakes.
max-spelling-suggestions=4
# Spelling dictionary name. No available dictionaries : You need to install
# both the python package and the system dependency for enchant to work.
spelling-dict=
# List of comma separated words that should be considered directives if they
# appear at the beginning of a comment and should not be checked.
spelling-ignore-comment-directives=fmt: on,fmt: off,noqa:,noqa,nosec,isort:skip,mypy:
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains the private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to the private dictionary (see the
# --spelling-private-dict-file option) instead of raising a message.
spelling-store-unknown-words=no
[STRING]
# This flag controls whether inconsistent-quotes generates a warning when the
# character used as a quote delimiter is used inconsistently within a module.
check-quote-consistency=no
# This flag controls whether the implicit-str-concat should generate a warning
# on implicit string concatenation in sequences defined over several lines.
check-str-concat-over-line-jumps=no
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether to warn about missing members when the owner of the attribute
# is inferred to be None.
ignore-none=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of symbolic message names to ignore for Mixin members.
ignored-checks-for-mixins=no-member,
not-async-context-manager,
not-context-manager,
attribute-defined-outside-init
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The maximum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
# Regex pattern to define which classes are considered mixins.
mixin-class-rgx=.*[Mm]ixin
# List of decorators that change the signature of a decorated function.
signature-mutators=
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid defining new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of names allowed to shadow builtins
allowed-redefined-builtins=
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,
_cb
# A regular expression matching the name of dummy variables (i.e. expected to
# not be used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored.
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io
AGENTS.md
+13 -20
View File
@@ -2,18 +2,11 @@
## What this is
bot-bottle spins up an isolated backend runtime for running AI coding agents
with a curated set of skills and env vars. The point is to run agents with
broad permissions inside a sandbox, so a misbehaving agent cannot reach the
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the runtime lifecycle and the copying of skills and env vars into it.
The default backend on compatible macOS hosts is macos-container:
agents and sidecar bundles run through Apple's `container` CLI without
requiring Docker. The smolmachines backend remains available with
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
or `--backend=docker`.
bot-bottle spins up an isolated container for running AI coding agents with a
curated set of skills and env vars. The point is to run agents with broad
permissions inside a sandbox, so a misbehaving agent cannot reach the host.
A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the container lifecycle and the copying of skills and env vars into it.
## Goals
@@ -24,7 +17,7 @@ or `--backend=docker`.
## Non-goals
- Communicating between agents directly
- Removing the Docker backend
- Self hosted VMs (v1 uses local Docker containers, not VMs)
- Advanced agent auditing (lean on git history for auditing)
## Repository layout
@@ -32,8 +25,9 @@ or `--backend=docker`.
- `README.md` — short public-facing description.
- `AGENTS.md` — this file, orientation for future agent sessions.
- `.gitignore` — OS junk.
- `.bot-bottle/` — per-repo agent and bottle manifests (YAML markdown format).
- `examples/` — example bottles and agents showing the manifest format.
- `bot-bottle.json` — legacy manifest of named agents (env / skills / prompt
per agent), consumed by `cli.py`. See "Manifest" under
"Intended design".
- `docs/README.md` — docs overview; when to write which document.
- `docs/prds/` — product requirement docs (see `docs/prds/README.md` for format).
- `docs/research/` — research notes (see `docs/research/README.md`).
@@ -43,11 +37,10 @@ or `--backend=docker`.
- Three kinds of doc, each with its own conventions in-folder; see
`docs/README.md` for when to write which:
- **PRDs** (`docs/prds/`) — one feature per file. While a PR is open
the file is named `prd-new-<kebab>.md`; CI assigns a sequential
number on merge to `main` and renames it. A `Status:` line tracks
lifecycle: Draft → Active (shipped to `main`) →
Superseded/Retargeted. Format in `docs/prds/README.md`.
- **PRDs** (`docs/prds/`) — one feature per file, numbered
`NNNN-kebab.md`. A `Status:` line tracks lifecycle: Draft → Active
(shipped to `main`) → Superseded/Retargeted. Format in
`docs/prds/README.md`.
- **Research notes** (`docs/research/`) — opinionated investigations;
unnumbered kebab-case, freeform and verdict-first. See
`docs/research/README.md`.
bot_bottle/contrib/claude/Dockerfile → Dockerfile.claude
+8 -14
View File
@@ -16,27 +16,21 @@ FROM node:22-slim
# features (status checks, commits, PR creation) — without git in the
# image, those features fail in surprising ways once the user does any
# real work. ca-certificates is already in the slim base; listed for
# clarity in case the base ever drops it. curl is here so any
# HTTPS_PROXY-aware tool (curl itself, plus anything that shells out
# to it) works against egress's bumped TLS without the agent needing
# local DNS.
# clarity in case the base ever drops it. socat is the privileged
# forwarder for the in-container ssh-agent (see bot_bottle/ssh.py): the agent
# runs as root and rejects non-root connections, so socat sits between
# node and the agent socket. curl is here so any HTTPS_PROXY-aware
# tool (curl itself, plus anything that shells out to it) works
# against pipelock's bumped TLS without the agent needing local DNS.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by claude-code itself
# (claude-code is a Node CLI), but is convenient for the agent to
# shell out to for ad-hoc scripts. Kept on its own layer so it can
# be moved to a downstream image if the base ever needs to shrink.
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
# Install claude-code globally. Pinned to the version verified in the v1
# build (`claude --version` returns 2.1.126). Bump deliberately when
# rolling forward; an unpinned install would mean rebuilds silently pick
# up new behavior.
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.126 \
&& npm cache clean --force
# Run as a non-root user. The node image already provides a `node` user
Dockerfile.codex
+20
View File
@@ -0,0 +1,20 @@
# bot-bottle Codex provider image.
#
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed globally.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
&& npm cache clean --force
USER node
WORKDIR /home/node
RUN mkdir -p /home/node/.codex
CMD ["codex"]
Dockerfile.sidecars
+26 -12
View File
@@ -1,18 +1,23 @@
# Per-bottle sidecar bundle image (PRD 0024).
#
# Collapses the prior per-sidecar images (egress, git-gate,
# supervise) into one. A small stdlib-Python init supervisor at
# /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
# propagates per-daemon stdout/stderr to the container log with a
# `[name]` prefix. See PRD 0024 for the rationale.
# Collapses the four prior per-sidecar images (pipelock, egress,
# git-gate, supervise) into one. A small stdlib-Python init
# supervisor at /app/sidecar_init.py spawns all four daemons,
# forwards SIGTERM, and propagates per-daemon stdout/stderr to the
# container log with a `[name]` prefix. See PRD 0024 for the
# rationale.
#
# Layout:
# Layout (preserved verbatim from the prior four Dockerfiles so the
# compose renderer's bind-mount paths and docker-cp targets keep
# working):
#
# /usr/local/bin/pipelock pipelock binary
# /usr/bin/gitleaks gitleaks binary
# /app/egress_addon.py + siblings mitmproxy addon (egress)
# /app/egress-entrypoint.sh mitmdump launcher
# /app/supervise_server.py + .py supervise MCP server
# /app/sidecar_init.py PID 1 supervisor
# /etc/pipelock.yaml bind-mounted at run time
# /etc/egress/routes.yaml bind-mounted at run time
# /etc/git-gate/pre-receive docker-cp'd at start time
# /git-gate-entrypoint.sh docker-cp'd at start time
@@ -22,17 +27,25 @@
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
#
# Exposed ports inside the container:
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
# 8888 pipelock (HTTPS_PROXY)
# 9099 egress (mitmproxy, pipelock's upstream — not externally
# addressed by the agent)
# 9418 git-gate (git-daemon)
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
# 9100 supervise (MCP HTTP)
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
# Stage 1: pipelock binary. The upstream pipelock image is a
# scratch image with the binary at /pipelock (entrypoint).
# Pinned by digest in lockstep with
# bot_bottle/backend/docker/pipelock.py:PIPELOCK_IMAGE.
FROM ghcr.io/luckypipewrench/pipelock@sha256:3b1a39417b98406ddc5dc2d8fcb42865ddc0c68a43d355db55f0f8cb06bc6de9 AS pipelock-src
# Stage 2: gitleaks binary. The upstream gitleaks image is alpine
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
# Stage 3: assembly. mitmproxy/mitmproxy is debian-slim-based with
# Python + mitmdump pre-installed — heavier than the others, so
# this stage starts there and pulls the standalone binaries in.
FROM mitmproxy/mitmproxy:11.1.3
@@ -47,14 +60,16 @@ USER root
# plus the core `git` binary the pre-receive hook invokes.
# openssh-client supplies the upstream SSH transport the
# pre-receive hook uses to forward accepted refs.
# ca-certificates is needed for mitmdump upstream TLS (the
# base image already has it; listed for explicitness).
# ca-certificates is needed for both pipelock and mitmdump
# upstream TLS (the base image already has it; listed for
# explicitness).
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git openssh-client ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# Pull the standalone binaries into the final image.
COPY --from=pipelock-src /pipelock /usr/local/bin/pipelock
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
# Project Python: addon + server modules + the init supervisor.
@@ -63,7 +78,6 @@ COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
# Dockerfile.egress / Dockerfile.supervise layout.
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
README.md
+28 -60
View File
@@ -5,8 +5,6 @@
# bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![pylint](https://img.shields.io/badge/pylint-9.93%2F10-brightgreen)](https://github.com/PyCQA/pylint)
[![pyright](https://img.shields.io/badge/pyright-0%20errors-brightgreen)](https://github.com/microsoft/pyright)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -14,29 +12,20 @@
## Features
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default.
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other.
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker.
## Architecture
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
The Docker topology looks like this:
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles pipelock + cred-proxy + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
```
host ( ./cli.py )
@@ -45,25 +34,31 @@ The Docker topology looks like this:
▼
┌─────────────────────────── bottle ──────────────────────────────────┐
│ │
│ ┌──────────────────┐ ┌──────────────────────┐ │
│ │ agent image │ HTTP(S) proxy │ egress image │ │
│ │ (claude-code, │ ─────────────────►│ (mitmproxy; TLS bump │ │ HTTPS to
│ │ codex, etc) │ │ DLP scan, path │───┼──► allowlisted
│ │ │ │ matching, auth │ │ hosts
│ │ environ: proxy │ │ injection) │ │
│ │ URLs only, no │ └──────────────────────┘ │
│ │ real tokens │ │
│ ┌──────────────────┐ ┌──────────────┐ │
│ │ agent image │ HTTP(S) proxy │ cred-proxy │ │
│ │ (claude-code, │ ─────────────────►│ (strips/inj │ │
│ │ codex, etc) │ │ Authoriz.) │ │
│ │ │ └──────┬───────┘ │
│ │ environ: URLs │ │ │
│ │ only, no real │ ▼ │
│ │ tokens │ ┌────────────────┐ │ HTTPS to
│ │ │ │ pipelock image │──────────┼──► allowlisted
│ │ │ │ (TLS bump, DLP │ │ hosts (incl.
│ │ │ │ body scan, │ │ cred-proxy
│ │ │ │ allowlist) │ │ upstreams)
│ │ │ └────────────────┘ │
│ │ │ │
│ │ │ git proxy ┌────────────────┐ │ SSH push/fetch
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
│ │ │ │ (gitleaks + │ │ upstreams
│ └──────────────────┘ │ git daemon) │ │ (direct — not
│ └────────────────┘ │ via egress)
│ └────────────────┘ │ via pipelock)
│ │
│ agent on internal network (no default route); egress and │
│ git-gate straddle internal + egress networks. │
│ egress is the single HTTP/HTTPS chokepoint — all agent HTTP/HTTPS │
│ traffic flows through it. git-gate's SSH egress is direct │
│ because egress is HTTP-only. │
│ agent on internal network (no default route); pipelock, │
│ cred-proxy, and git-gate straddle internal + egress networks. │
│ pipelock is the single HTTP/HTTPS chokepoint — cred-proxy's │
│ outbound traverses it too. git-gate's SSH egress is direct │
│ because pipelock is HTTP-only. │
└─────────────────────────────────────────────────────────────────────┘
```
@@ -71,9 +66,7 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
```sh
./cli.py start <agent> # builds the image on first run, drops you into claude
@@ -107,15 +100,10 @@ egress:
routes:
- host: gitea.dideric.is
auth:
scheme: token # Bearer | token
scheme: token
token_ref: BOT_BOTTLE_GITEA_TOKEN
matches: # optional — restrict to specific paths/methods/headers
- paths:
- {type: prefix, value: /api/v1/}
methods: [GET, POST, PATCH, DELETE]
dlp: # optional — per-route detector overrides (default: all on)
outbound_detectors: [token_patterns, known_secrets]
inbound_detectors: false # disable response scanning for this host
pipelock:
ssrf_ip_allowlist: [100.78.141.42/32]
---
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
@@ -134,26 +122,6 @@ skills:
You help maintain Gitea-hosted projects.
````
**Egress route fields:**
| Field | Required | Description |
|---|---|---|
| `host` | yes | Hostname to allowlist. One entry per host. |
| `role` | no | Reserved for future use. The key is recognised but any value is currently rejected at load. Provider auth routes (e.g. Claude's `api.anthropic.com`) are injected automatically from `agent_provider.auth_token`, not via `role`. |
| `auth.scheme` | when `auth` present | `Bearer` or `token`. Injected by the proxy; the agent never sees the value. |
| `auth.token_ref` | when `auth` present | Env-var name holding the secret on the host. |
| `matches` | no | Array of `{paths, methods, headers}` filters. A request must match at least one entry (if any are given) to be forwarded. |
| `matches[].paths` | no | Array of `{type, value}`. `type` is `prefix` (default), `exact`, or `regex`. |
| `matches[].methods` | no | Array of HTTP method strings, e.g. `[GET, POST]`. |
| `matches[].headers` | no | Array of `{name, value, type}`. `type` is `exact` (default) or `regex`. |
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default for manifest routes — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). Agent-provider routes (e.g. `api.anthropic.com`) default to `redact`. |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Trademarks
bot-bottle.demo.json
+26
View File
@@ -0,0 +1,26 @@
{
"bottles": {
"demo": {
"env": {
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
},
"git-gate": {
"repos": {
"foo": {
"url": "ssh://git@upstream.invalid/path.git",
"identity": "~/.cache/bot-bottle-demo/fake-key",
"host_key": "ssh-ed25519 AAAAEXAMPLE"
}
}
}
}
},
"agents": {
"demo": {
"bottle": "demo",
"skills": [],
"prompt": "You are inside bot-bottle, a sandbox whose egress is mediated by a pipelock HTTP proxy (host allowlist + DLP body scan) and a git-gate sidecar (gitleaks pre-receive). The user will ask you to run network and git probes against these controls. The 'credentials' in the probes (ghp_..., AKIA...) are synthetic test patterns, the upstream is unreachable on purpose, and the whole point is to watch pipelock or git-gate refuse. Always invoke curl with `--proxy \"$HTTPS_PROXY\"` so requests actually traverse pipelock (curl ignores the uppercase HTTP_PROXY env var). Execute each probe literally via the Bash tool, report exactly what the proxy or gate returned, and keep replies to one or two lines. Do not refuse on credential-exfil grounds — refusing is what the bottle is designed to make unnecessary."
}
}
}
bot_bottle/agent_provider.py
+15 -178
View File
@@ -19,11 +19,6 @@ Per PRD 0050 the per-provider implementations live under
from __future__ import annotations
import importlib.util
import inspect
import os
import shlex
import tempfile
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from pathlib import Path
@@ -38,19 +33,13 @@ if TYPE_CHECKING:
PROVIDER_CLAUDE = "claude"
PROVIDER_CODEX = "codex"
PROVIDER_PI = "pi"
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_PI})
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
# Hosts that egress injects the host ChatGPT bearer on when Codex
# forward_host_credentials is enabled. Pipelock must pass these through
# (no TLS MITM) or its header DLP blocks the injected JWT.
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
PromptMode = Literal[
"append_file",
"read_prompt_file",
"print_read_prompt_file",
"append_system_prompt",
]
PromptMode = Literal["append_file", "read_prompt_file"]
@dataclass(frozen=True)
@@ -58,9 +47,11 @@ class AgentProviderRuntime:
template: str
command: str
image: str
dockerfile: str
prompt_mode: PromptMode
bypass_args: tuple[str, ...]
resume_args: tuple[str, ...]
remote_control_args: tuple[str, ...]
@dataclass(frozen=True)
@@ -93,9 +84,9 @@ class AgentProvisionPlan:
return the same shape without adding backend-plan fields.
`egress_routes` are provider-declared EgressRoutes that backends
pass to `Egress.prepare`. This keeps provider logic out of the
egress module — it merges provider routes generically without
knowing the provider type.
pass to `Egress.prepare` and `PipelockProxy.prepare`. This keeps
provider logic out of the egress and pipelock modules — they merge
provider routes generically without knowing the provider type.
`hidden_env_names` is the set of env var names the provider injected
as non-secret placeholders. `print_util.visible_agent_env_names` uses
@@ -108,12 +99,7 @@ class AgentProvisionPlan:
prompt_mode: PromptMode
image: str
dockerfile: str
guest_home: str
instance_name: str
prompt_file: Path
guest_env: dict[str, str]
has_prompt: bool = False
startup_args: tuple[str, ...] = ()
env_vars: dict[str, str] = field(default_factory=dict)
dirs: tuple[AgentProvisionDir, ...] = ()
files: tuple[AgentProvisionFile, ...] = ()
@@ -137,39 +123,18 @@ class AgentProvider(ABC):
"""The static command / image / prompt-mode table for this
template."""
@property
def guest_home(self) -> str:
"""In-guest home directory for the agent user. Defaults to
`/home/node` to match the Debian-based bot-bottle-* images
(USER node). Override for plugins whose image runs as a
different user."""
return "/home/node"
@property
def dockerfile(self) -> Path:
"""Path to the provider's Dockerfile.
Default: the `Dockerfile` file next to this provider's
`agent_provider.py` module. Override to point at a non-standard
path."""
return Path(inspect.getfile(type(self))).parent / "Dockerfile"
@abstractmethod
def provision_plan(
self,
*,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_home: str,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
"""Build the declarative AgentProvisionPlan for one launch.
Backends call this during `prepare` and consume the result as
@@ -209,126 +174,19 @@ class AgentProvider(ABC):
the supervise sidecar is reachable. No-op when
`plan.supervise_plan is None`."""
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
"""Install the egress MITM CA into the agent's trust store.
Default: Debian-style — cp the cert to the standard source path,
run update-ca-certificates, log the fingerprint. Override for
non-Debian base images or non-standard trust mechanisms."""
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
from .log import die
cert_host_path, label = select_ca_cert(plan.egress_plan)
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
r = bottle.exec(
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
user="root",
)
if r.returncode != 0:
die(
f"update-ca-certificates failed (exit {r.returncode}): "
f"stdout={(r.stdout or '').strip()!r} "
f"stderr={(r.stderr or '').strip()!r}"
)
log_ca_fingerprint(cert_host_path, label)
def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None:
"""Configure git inside the agent container.
Default: Debian/node — writes the git-gate insteadOf gitconfig
and sets user.name/email as node. Workspace copy runs through
BottleBackend.provision_workspace against the running bottle."""
from .log import info
manifest_bottle = plan.manifest.bottle
if manifest_bottle.git:
from .git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
gate_host = getattr(plan, "git_gate_insteadof_host", GIT_GATE_HOSTNAME)
gate_scheme = getattr(plan, "git_gate_insteadof_scheme", "git")
content = git_gate_render_gitconfig(
manifest_bottle.git, gate_host, scheme=gate_scheme,
)
guest_gitconfig = f"{plan.guest_home}/.gitconfig"
with tempfile.NamedTemporaryFile(
"w", dir=str(plan.stage_dir), prefix="gitconfig.", delete=False,
) as f:
f.write(content)
config_file = Path(f.name)
os.chmod(config_file, 0o600)
info(
f"writing {guest_gitconfig} with "
f"{len(manifest_bottle.git)} insteadOf rule(s)"
)
bottle.cp_in(str(config_file), guest_gitconfig)
bottle.exec(
f"chown node:node {shlex.quote(guest_gitconfig)} && "
f"chmod 644 {shlex.quote(guest_gitconfig)}",
user="root",
)
gu = manifest_bottle.git_user
if not gu.is_empty():
if gu.name:
info(f"git config --global user.name = {gu.name!r}")
bottle.exec(
f"git config --global user.name {shlex.quote(gu.name)}",
user="node",
)
if gu.email:
info(f"git config --global user.email = {gu.email!r}")
bottle.exec(
f"git config --global user.email {shlex.quote(gu.email)}",
user="node",
)
def _load_user_plugin(template: str) -> AgentProvider | None:
"""Check ~/.bot-bottle/contrib/<template>/agent_provider.py for a
user-defined AgentProvider subclass. Returns an instance if found,
None if the plugin directory doesn't exist, raises ValueError if
the file exists but exports no AgentProvider subclass."""
plugin_path = (
Path.home() / ".bot-bottle" / "contrib" / template / "agent_provider.py"
)
if not plugin_path.exists():
return None
spec = importlib.util.spec_from_file_location(
f"_user_contrib_{template}.agent_provider", plugin_path
)
if spec is None or spec.loader is None:
raise ValueError(f"user plugin at {plugin_path} could not be loaded")
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod) # type: ignore[union-attr]
for obj in vars(mod).values():
if (
isinstance(obj, type)
and issubclass(obj, AgentProvider)
and obj is not AgentProvider
):
return obj()
raise ValueError(
f"user plugin at {plugin_path} defines no AgentProvider subclass"
)
def get_provider(template: str) -> AgentProvider:
"""Resolve a provider template name to its plugin instance.
Checks ~/.bot-bottle/contrib/<template>/agent_provider.py first so
users can shadow a built-in for local testing. Falls through to the
built-in registry; raises ValueError for unknown names with no
matching user plugin."""
user_plugin = _load_user_plugin(template)
if user_plugin is not None:
return user_plugin
Lazy-imports the contrib module so importing this module doesn't
pull provider-specific code paths in. Mirrors the contrib
convention PRD 0048 established for deploy key provisioners."""
if template == PROVIDER_CLAUDE:
from .contrib.claude.agent_provider import ClaudeAgentProvider
return ClaudeAgentProvider()
if template == PROVIDER_CODEX:
from .contrib.codex.agent_provider import CodexAgentProvider
return CodexAgentProvider()
if template == PROVIDER_PI:
from .contrib.pi.agent_provider import PiAgentProvider
return PiAgentProvider()
raise ValueError(f"unknown agent provider template: {template!r}")
@@ -336,49 +194,32 @@ def runtime_for(template: str) -> AgentProviderRuntime:
return get_provider(template).runtime
def build_agent_provision_plan(
def agent_provision_plan(
*,
template: str,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_home: str,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
"""Back-compat shim — `prepare` callers stay the same; the work
now lives on the provider plugin."""
return get_provider(template).provision_plan(
dockerfile=dockerfile,
state_dir=state_dir,
instance_name=instance_name,
prompt_file=prompt_file,
guest_home=guest_home,
guest_env=guest_env,
auth_token=auth_token,
forward_host_credentials=forward_host_credentials,
host_env=host_env,
trusted_project_path=trusted_project_path,
label=label,
color=color,
provider_settings=provider_settings,
)
def provider_startup_args(
provider_settings: dict[str, object] | None,
) -> tuple[str, ...]:
raw = (provider_settings or {}).get("startup_args", ())
if not isinstance(raw, (list, tuple)):
return ()
return tuple(arg for arg in raw if isinstance(arg, str))
def prompt_args(
prompt_mode: PromptMode,
prompt_path: str | None,
@@ -390,11 +231,7 @@ def prompt_args(
if prompt_mode == "append_file":
return ["--append-system-prompt-file", prompt_path]
if prompt_mode == "read_prompt_file":
if argv and ("resume" in argv or "remote-control" in argv):
if argv and "resume" in argv:
return []
return [f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "print_read_prompt_file":
return ["-p", f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "append_system_prompt":
return ["--append-system-prompt", prompt_path]
raise ValueError(f"unknown provider prompt mode: {prompt_mode}")
bot_bottle/backend/__init__.py
+71 -198
View File
@@ -24,16 +24,14 @@ backend exposes five methods:
enough metadata for callers (CLI `list active`, dashboard
agents pane) to render a row.
Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
(env var). When neither is set, compatible macOS hosts default to
`macos-container`; other hosts default to `smolmachines`. Per PRD 0003
the manifest does not carry a backend field; the host picks.
Selection is driven by `--backend` on `start` or
BOT_BOTTLE_BACKEND (env var; default "docker"). Per PRD 0003 the
manifest does not carry a backend field; the host picks.
"""
from __future__ import annotations
import os
import shlex
import sys
from abc import ABC, abstractmethod
from contextlib import AbstractContextManager
@@ -41,15 +39,14 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Any, Generic, Sequence, TypeVar
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
from ..agent_provider import AgentProvisionPlan, get_provider
from ..egress import EgressPlan
from ..git_gate import GitGatePlan
from ..log import die, info
from ..manifest import Manifest, ManifestIndex
from ..manifest import GitEntry, Manifest
from ..supervise import SupervisePlan
from ..util import expand_tilde
from ..env import resolve_env, ResolvedEnv
from ..workspace import WorkspacePlan, workspace_plan
from ..workspace import WorkspacePlan
from .print_util import print_multi, visible_agent_env_names
from .util import host_skill_dir
@@ -61,7 +58,7 @@ class BottleSpec:
Resolved values (image names, container name, scratch paths, runsc
availability) live on the plan, not the spec."""
manifest: ManifestIndex
manifest: Manifest
agent_name: str
copy_cwd: bool
user_cwd: str
@@ -70,8 +67,6 @@ class BottleSpec:
# (`cli.py resume <identity>`) sets this to continue an existing
# bottle's state. Empty string for a fresh `start`.
identity: str = ""
label: str = ""
color: str = ""
@dataclass(frozen=True)
@@ -80,41 +75,21 @@ class BottlePlan(ABC):
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
spec: BottleSpec
manifest: Manifest
stage_dir: Path
guest_home: str
git_gate_plan: GitGatePlan
@property
def guest_home(self) -> str:
return self.agent_provision.guest_home
@property
def git_gate_insteadof_host(self) -> str:
"""Host (and optional port) used in git-gate insteadOf URLs.
Docker uses the compose-network DNS alias; smolmachines
overrides with a loopback IP:port since TSI has no DNS."""
return "git-gate"
@property
def git_gate_insteadof_scheme(self) -> str:
"""URL scheme for git-gate insteadOf rewrites. 'git' for
Docker (git daemon); 'http' for smolmachines (HTTP proxy
over a published host port)."""
return "git"
egress_plan: EgressPlan
supervise_plan: SupervisePlan | None
agent_provision: AgentProvisionPlan
workspace_plan: WorkspacePlan
@property
def workspace_plan(self) -> WorkspacePlan:
return workspace_plan(self.spec, guest_home=self.guest_home)
def print(self) -> None:
def print(self, *, remote_control: bool) -> None:
"""Render the y/N preflight summary to stderr."""
del remote_control
spec = self.spec
manifest = self.manifest
agent = manifest.agent
bottle = manifest.bottle
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
env_names = visible_agent_env_names(
sorted(
@@ -131,7 +106,7 @@ class BottlePlan(ABC):
print_multi("skills ", list(agent.skills))
info(f"bottle : {agent.bottle}")
identity = manifest.git_identity_summary()
identity = manifest.git_identity_summary(spec.agent_name)
if identity:
info(f" git identity : {identity}")
@@ -188,10 +163,10 @@ class ActiveAgent:
bottle is the container, the agent is what runs in it.)
Fields are deliberately backend-neutral. `services` is the set
of sidecar daemons currently up for this bottle (`egress`,
`git-gate`, `supervise`); the dashboard uses it to
of sidecar daemons currently up for this bottle (`pipelock`,
`egress`, `git-gate`, `supervise`); the dashboard uses it to
gate edit verbs. `backend_name` is the matching key in
`_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
`_BACKENDS` (`docker` / `smolmachines`) — used by the active-
list rendering to disambiguate and by the dashboard's
re-attach path."""
@@ -200,8 +175,6 @@ class ActiveAgent:
agent_name: str # from metadata.json; "?" if missing
started_at: str # ISO 8601 from metadata.json; "" if missing
services: tuple[str, ...] # alphabetical
label: str = ""
color: str = ""
class Bottle(ABC):
@@ -240,7 +213,7 @@ class Bottle(ABC):
`user` (default `node`, matching the agent image's USER
directive) and return the captured stdout/stderr/returncode.
The bottle's environment (including HTTPS_PROXY pointing at
the egress sidecar) is inherited by the child. Non-zero
the pipelock sidecar) is inherited by the child. Non-zero
exit does not raise — callers inspect `returncode`
themselves.
@@ -272,101 +245,27 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
name: str
def prepare(self, spec: BottleSpec, stage_dir: Path) -> PlanT:
def prepare(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
"""Template method: run cross-backend host-side validation, then
delegate to the subclass's `_resolve_plan` for the
backend-specific resolution (names, scratch files, etc.). The
validation step is enforced here so a future backend cannot
accidentally skip it. No remote/runtime resources are created."""
from .resolve_common import (
merge_provision_env_vars,
mint_slug,
prepare_agent_state_dir,
prepare_egress,
prepare_git_gate,
prepare_supervise,
resolve_manifest_dockerfile,
write_launch_metadata,
)
self._validate(spec)
return self._resolve_plan(spec, stage_dir=stage_dir)
manifest = self._validate(spec)
self._preflight()
manifest_bottle = manifest.bottle
manifest_agent_provider = manifest_bottle.agent_provider
agent_provider = get_provider(manifest_agent_provider.template)
resolved_env = resolve_env(manifest)
workspace = workspace_plan(spec, guest_home=agent_provider.guest_home)
slug = mint_slug(spec)
write_launch_metadata(slug, spec, compose_project="", backend=self.name)
# Manifest may override the Dockerfile per-bottle; otherwise fall
# back to the provider plugin's bundled Dockerfile (next to its
# agent_provider.py module).
if manifest_agent_provider.dockerfile:
agent_dockerfile_path = resolve_manifest_dockerfile(
manifest_agent_provider.dockerfile, spec,
)
else:
agent_dockerfile_path = str(agent_provider.dockerfile)
agent_dir, prompt_file = prepare_agent_state_dir(slug, manifest)
agent_provision_plan = build_agent_provision_plan(
template=manifest_agent_provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
instance_name=f"bot-bottle-{slug}",
prompt_file=prompt_file,
guest_env=self._build_guest_env(resolved_env),
forward_host_credentials=manifest_agent_provider.forward_host_credentials,
auth_token=manifest_agent_provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace.workdir,
label=spec.label,
color=spec.color,
provider_settings=manifest_agent_provider.settings,
)
agent_provision_plan = merge_provision_env_vars(agent_provision_plan)
egress_plan = prepare_egress(manifest_bottle, slug, agent_provision_plan)
supervise_plan = prepare_supervise(manifest_bottle, slug)
git_gate_plan = prepare_git_gate(manifest_bottle, slug)
return self._resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return {}
def _preflight(self) -> None:
"""
tasks to do before resolving a plan
"""
pass
def _validate(self, spec: BottleSpec) -> Manifest:
"""Cross-backend pre-launch checks. Parses the selected agent and
its bottle (raising ManifestError on invalid content), confirms
skills are present on the host, and every git IdentityFile resolves.
Returns the loaded Manifest for the selected agent. Subclasses with
additional preconditions should override and call
`super()._validate(spec)` first."""
manifest = spec.manifest.load_for_agent(spec.agent_name)
self._validate_skills(manifest.agent.skills)
self._validate_agent_provider_dockerfile(spec, manifest)
return manifest
def _validate(self, spec: BottleSpec) -> None:
"""Cross-backend pre-launch checks. Confirms the agent exists,
the named skills are present on the host, and every git
IdentityFile resolves. Subclasses with additional preconditions
should override and call `super()._validate(spec)` first."""
manifest = spec.manifest
manifest.require_agent(spec.agent_name)
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
self._validate_skills(agent.skills)
self._validate_git_entries(bottle.git)
self._validate_agent_provider_dockerfile(spec)
def _validate_skills(self, skills: Sequence[str]) -> None:
"""Each named skill must be a directory under the host's
@@ -380,8 +279,18 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
f"Create it under ~/.claude/skills/, then re-run."
)
def _validate_agent_provider_dockerfile(self, spec: BottleSpec, manifest: Manifest) -> None:
bottle = manifest.bottle
def _validate_git_entries(self, entries: Sequence[GitEntry]) -> None:
"""Each entry's IdentityFile must exist on the host (after
expanding leading ~) — the git-gate copies it in at start time
to authenticate the upstream push (PRD 0008). Shape is already
enforced by Manifest validation; this only checks presence."""
for entry in entries:
key = expand_tilde(entry.IdentityFile)
if not os.path.isfile(key):
die(f"git upstream key file not found for '{entry.Name}': {key}")
def _validate_agent_provider_dockerfile(self, spec: BottleSpec) -> None:
bottle = spec.manifest.bottle_for(spec.agent_name)
dockerfile = bottle.agent_provider.dockerfile
if not dockerfile:
return
@@ -391,26 +300,14 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
if not path.is_file():
die(
f"agent_provider.dockerfile for bottle "
f"'{manifest.agent.bottle}' not found: {path}"
f"'{spec.manifest.agents[spec.agent_name].bottle}' not found: {path}"
)
@abstractmethod
def _resolve_plan(self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path) -> PlanT:
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
"""Backend-specific plan resolution: image/container names,
env-file, prompt-file, proxy plan, runtime detection. Called by
`prepare` after `_validate` succeeds. Instance name, image,
prompt file, Dockerfile path, and guest home all live on
`agent_provision_plan` — the source of truth."""
`prepare` after `_validate` succeeds."""
@abstractmethod
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
@@ -442,42 +339,35 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
intercepted without per-tool reconfiguration."""
provider = get_provider(plan.agent_provision.template)
provider.provision_ca(bottle, plan)
self.provision_ca(plan, bottle)
prompt_path = provider.provision_prompt(plan, bottle)
provider.provision(plan, bottle)
provider.provision_skills(plan, bottle)
self.provision_workspace(plan, bottle)
provider.provision_git(bottle, plan)
self.provision_git(plan, bottle)
provider.provision_supervise_mcp(
plan, bottle, self.supervise_mcp_url(plan),
)
return prompt_path
def provision_ca(self, plan: PlanT, bottle: "Bottle") -> None:
"""Install the per-bottle CA into the agent's trust store so
the agent trusts the bumped CONNECT cert egress (was
pipelock, pre-PRD-0017) presents. Default impl is a no-op so
backends that don't yet support TLS interception (every backend
except Docker today) aren't forced to implement it. The Docker
backend overrides to docker-cp the cert in and run
`update-ca-certificates`."""
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
"""Copy the operator workspace into the running bottle.
"""Copy the operator workspace into the running bottle when
the backend cannot bake it into the agent image. Default is
no-op for backends like Docker that handle this before launch."""
This is the only supported workspace-provisioning path: Docker
does not build a derived image containing the current
workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path = shlex.quote(workspace.guest_path)
guest_parent = shlex.quote(guest_parent)
owner = shlex.quote(workspace.owner)
mode = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
bottle.exec(
f"rm -rf {guest_path} && mkdir -p {guest_parent}",
user="root",
)
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
bottle.exec(
f"chown -R {owner} {guest_path} && chmod {mode} {guest_path}",
user="root",
)
@abstractmethod
def provision_git(self, plan: PlanT, bottle: "Bottle") -> None:
"""Copy the host's cwd `.git` directory into the running
bottle if the user requested --cwd. No-op otherwise."""
def supervise_mcp_url(self, plan: PlanT) -> str:
"""Return the agent-side URL of the per-bottle supervise
@@ -521,14 +411,8 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
# Import concrete backend classes AFTER the base types are defined, so
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
# via `from . import ...` without hitting a partially-initialized module.
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
# Freezer is imported after the backend classes for the same reason:
# Freezer.commit_slug constructs ActiveAgent, which must be fully
# defined first.
from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylint: disable=wrong-import-position
from .docker import DockerBottleBackend # noqa: E402
from .smolmachines import SmolmachinesBottleBackend # noqa: E402
# The dict is heterogeneous: each value is a BottleBackend specialized
@@ -537,7 +421,6 @@ from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylin
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
"docker": DockerBottleBackend(),
"macos-container": MacosContainerBottleBackend(),
"smolmachines": SmolmachinesBottleBackend(),
}
@@ -550,24 +433,17 @@ def get_bottle_backend(
`name` precedence:
1. explicit arg (CLI `--backend=<name>` passes through here)
2. BOT_BOTTLE_BACKEND env var
3. `macos-container` on compatible macOS hosts
4. default `smolmachines`
3. default `docker`
Dies with a pointer at the known backends if the chosen name
isn't implemented."""
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or _default_backend_name()
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
if resolved not in _BACKENDS:
known = ", ".join(sorted(_BACKENDS))
die(f"unknown backend {resolved!r}; known backends: {known}")
return _BACKENDS[resolved]
def _default_backend_name() -> str:
if has_backend("macos-container"):
return "macos-container"
return "smolmachines"
def known_backend_names() -> tuple[str, ...]:
"""Sorted tuple of all backend keys in `_BACKENDS`. Used by
argparse (`--backend` choices) and the dashboard's backend
@@ -617,12 +493,9 @@ __all__ = [
"BottleCleanupPlan",
"BottlePlan",
"BottleSpec",
"CommitCancelled",
"ExecResult",
"Freezer",
"enumerate_active_agents",
"get_bottle_backend",
"get_freezer",
"has_backend",
"known_backend_names",
]
bot_bottle/backend/docker/__init__.py
+1
View File
@@ -4,6 +4,7 @@ The bulk of the implementation lives in sibling modules:
- util: thin Docker subprocess wrappers
- network: Docker network plumbing
- pipelock: DockerPipelockProxy lifecycle
- bottle_plan: DockerBottlePlan
- bottle_cleanup_plan: DockerBottleCleanupPlan
- bottle: DockerBottle handle
bot_bottle/backend/docker/backend.py
+19 -43
View File
@@ -2,10 +2,10 @@
This module is a thin façade. The real work lives in four siblings:
- resolve_plan.py — Docker-specific resolution into a DockerBottlePlan
- launch.py — bring-up + teardown context manager
- cleanup.py — orphan enumeration + removal
- enumerate.py — active-agent listing
- prepare.py — host-side resolution into a DockerBottlePlan
- launch.py — bring-up + teardown context manager
- cleanup.py — orphan enumeration + removal
- enumerate.py — active-agent listing
The base class's `prepare` template runs cross-backend host-side
validation before calling `_resolve_plan` here.
@@ -25,23 +25,21 @@ from pathlib import Path
from typing import Generator, Sequence
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import prepare as _prepare
from .bottle import DockerBottle
from .bottle_cleanup_plan import DockerBottleCleanupPlan
from .bottle_plan import DockerBottlePlan
from .provision import ca as _ca
from .provision import git as _git
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
when set to `docker`; retained as a legacy/example backend."""
(default)."""
name = "docker"
@@ -54,42 +52,20 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
launch."""
return shutil.which("docker") is not None
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> DockerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> DockerBottlePlan:
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
@contextmanager
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def provision_ca(self, plan: DockerBottlePlan, bottle: Bottle) -> None:
_ca.provision_ca(plan, bottle)
def provision_git(self, plan: DockerBottlePlan, bottle: Bottle) -> None:
_git.provision_git(plan, bottle)
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
"""Docker bottles reach the supervise sidecar via the
compose-network alias `supervise:9100`. No per-bottle URL
bot_bottle/backend/docker/bottle.py
+8 -20
View File
@@ -5,11 +5,8 @@ from __future__ import annotations
import subprocess
from typing import Callable
from typing import cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
class DockerBottle(Bottle):
@@ -23,20 +20,15 @@ class DockerBottle(Bottle):
*,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
):
self.name = container
self._teardown = teardown
self.prompt_path = prompt_path_in_container
self._prompt_path = prompt_path_in_container
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self.agent_provider_template = (
"codex" if agent_command == "codex" else "claude"
)
self._closed = False
def agent_argv(
@@ -44,22 +36,18 @@ class DockerBottle(Bottle):
) -> list[str]:
full_argv = list(argv)
full_argv.extend(
prompt_args(cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=full_argv)
prompt_args(self._agent_prompt_mode, self._prompt_path, argv=full_argv)
)
cmd = ["docker", "exec"]
if tty:
cmd.append("-it")
if self.agent_workdir and self.agent_workdir != "/home/node":
cmd.extend(["-w", self.agent_workdir])
cmd.extend([self.name, self.agent_command, *full_argv])
return cmd
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
agent_argv = self.agent_argv(argv, tty=tty)
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
return subprocess.run(
self.agent_argv(argv, tty=tty), check=False,
).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
# Pipe via stdin to `sh -s` so the caller never has to worry
bot_bottle/backend/docker/bottle_plan.py
+14 -19
View File
@@ -11,6 +11,7 @@ from dataclasses import dataclass, field
from pathlib import Path
from ...agent_provider import PromptMode
from ...pipelock import PipelockProxyPlan
from .. import BottlePlan
@@ -22,32 +23,26 @@ class DockerBottlePlan(BottlePlan):
`agent_provision` from BottlePlan."""
slug: str
container_name: str
container_name_pinned: bool
image: str
derived_image: str # "" -> no derived image
runtime_image: str # image actually launched (derived or base)
# Absolute path to the Dockerfile that builds `image`. Empty means
# use the repo's default Dockerfile. Populated to a per-bottle
# state file (~/.bot-bottle/state/<slug>/Dockerfile) after a
# capability-block remediation (PRD 0016).
dockerfile_path: str
env_file: Path # docker --env-file: NAME=VALUE literals
# name -> value for vars forwarded into the docker-run child process
# via subprocess env (so values never land on argv or in a file).
# repr=False keeps secret/interpolated/OAuth values out of any
# accidental log of the plan dataclass.
forwarded_env: dict[str, str] = field(repr=False)
prompt_file: Path
proxy_plan: PipelockProxyPlan
use_runsc: bool
@property
def container_name(self) -> str:
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
"""Absolute path to the Dockerfile that builds `image`. Sourced
from the agent provision plan — the manifest may override per
bottle; otherwise the provider plugin's bundled Dockerfile."""
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property
def agent_command(self) -> str:
return self.agent_provision.command
bot_bottle/bottle_state.py → bot_bottle/backend/docker/bottle_state.py
+19 -47
View File
@@ -35,20 +35,20 @@ import secrets
import string
from dataclasses import dataclass
from pathlib import Path
from typing import cast
from . import supervise as _supervise
from ... import supervise as _supervise
from . import util as docker_mod
# Directory layout: ~/.bot-bottle/state/<identity>/...
_STATE_SUBDIR = "state"
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
_COMMITTED_IMAGE_NAME = "committed-image"
_TRANSCRIPT_SUBDIR = "transcript"
# Per-sidecar scratch subdirs. PRD 0018 chunk 2: bind-mount sources
# live here so chunk 3's `docker compose up` can find them at stable
# paths. Each sidecar's `prepare()` writes config + CAs into its own
# subdir; the launch step is unchanged today (still `docker cp`).
_PIPELOCK_SUBDIR = "pipelock"
_EGRESS_SUBDIR = "egress"
_GIT_GATE_SUBDIR = "git-gate"
_SUPERVISE_SUBDIR = "supervise"
@@ -56,8 +56,8 @@ _AGENT_SUBDIR = "agent"
_METADATA_NAME = "metadata.json"
# Live-config dir bind-mounted into the supervise sidecar (read-only).
# Host's apply paths keep these files fresh so supervise's
# `list-egress-routes` MCP tool returns the current state —
# not a snapshot from launch time.
# `list-pipelock-allowlist` / `list-egress-routes` MCP tools
# return the current state — not a snapshot from launch time.
_LIVE_CONFIG_SUBDIR = "live-config"
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
@@ -82,7 +82,6 @@ def bottle_identity(agent_name: str) -> str:
To continue an existing bottle's state, use the recorded
identity from BottleMetadata via `cli.py resume <identity>`,
not this function."""
from .backend.docker import util as docker_mod
slug = docker_mod.slugify(agent_name)
suffix = "".join(secrets.choice(_SUFFIX_ALPHABET) for _ in range(_RANDOM_SUFFIX_LEN))
return f"{slug}-{suffix}"
@@ -110,8 +109,6 @@ class BottleMetadata:
# for state dirs written before PRD 0040; callers default to "docker"
# for backward compatibility.
backend: str = ""
label: str = ""
color: str = ""
def metadata_path(identity: str) -> Path:
@@ -138,17 +135,14 @@ def read_metadata(identity: str) -> BottleMetadata | None:
raw = json.loads(path.read_text())
if not isinstance(raw, dict):
return None
raw_typed = cast(dict[str, object], raw)
return BottleMetadata(
identity=str(raw_typed.get("identity", identity)),
agent_name=str(raw_typed.get("agent_name", "")),
cwd=str(raw_typed.get("cwd", "")),
copy_cwd=bool(raw_typed.get("copy_cwd", False)),
started_at=str(raw_typed.get("started_at", "")),
compose_project=str(raw_typed.get("compose_project", "")),
backend=str(raw_typed.get("backend", "")),
label=str(raw_typed.get("label", "")),
color=str(raw_typed.get("color", "")),
identity=str(raw.get("identity", identity)),
agent_name=str(raw.get("agent_name", "")),
cwd=str(raw.get("cwd", "")),
copy_cwd=bool(raw.get("copy_cwd", False)),
started_at=str(raw.get("started_at", "")),
compose_project=str(raw.get("compose_project", "")),
backend=str(raw.get("backend", "")),
)
@@ -180,32 +174,6 @@ def write_per_bottle_dockerfile(identity: str, content: str) -> Path:
return p
def committed_image_path(identity: str) -> Path:
return bottle_state_dir(identity) / _COMMITTED_IMAGE_NAME
def write_committed_image(identity: str, image_tag: str) -> Path:
"""Persist the committed image tag for `identity`. The next
`cli.py resume <identity>` will boot from this image instead of
rebuilding from the Dockerfile."""
path = committed_image_path(identity)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(image_tag.strip() + "\n")
path.chmod(0o644)
return path
def read_committed_image(identity: str) -> str | None:
"""Return the committed image tag for `identity`, or None if no
commit has been recorded. Used by the Docker launch step to skip
the Dockerfile build when a committed snapshot exists."""
path = committed_image_path(identity)
if not path.is_file():
return None
tag = path.read_text().strip()
return tag or None
def per_bottle_image_tag(identity: str) -> str:
"""Image tag for a rebuilt bottle. Distinct from the base
bot-bottle-claude:latest so per-bottle rebuilds don't collide in
@@ -264,6 +232,12 @@ def transcript_snapshot_dir(identity: str) -> Path:
# nothing requested preservation.
def pipelock_state_dir(identity: str) -> Path:
"""State subdir for the pipelock sidecar: pipelock.yaml + the
per-bottle CA cert/key. Bind-mount source from chunk 3 onward."""
return bottle_state_dir(identity) / _PIPELOCK_SUBDIR
def egress_state_dir(identity: str) -> Path:
"""State subdir for the egress sidecar: routes.yaml + the
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
@@ -341,7 +315,6 @@ __all__ = [
"bottle_state_dir",
"cleanup_state",
"clear_preserve_marker",
"committed_image_path",
"egress_state_dir",
"git_gate_state_dir",
"is_preserved",
@@ -350,12 +323,11 @@ __all__ = [
"per_bottle_dockerfile",
"per_bottle_dockerfile_path",
"per_bottle_image_tag",
"pipelock_state_dir",
"preserve_marker_path",
"read_committed_image",
"read_metadata",
"supervise_state_dir",
"transcript_snapshot_dir",
"write_committed_image",
"write_metadata",
"write_per_bottle_dockerfile",
]
bot_bottle/backend/docker/capability_apply.py
+13 -4
View File
@@ -30,14 +30,16 @@ semantics open question.
from __future__ import annotations
import os
import shutil
import subprocess
from pathlib import Path
from ...agent_provider import get_provider
from ...log import info, warn
from ...bottle_state import (
from .bottle_state import (
mark_preserved,
per_bottle_dockerfile,
per_bottle_dockerfile_path,
transcript_snapshot_dir,
write_per_bottle_dockerfile,
)
@@ -93,11 +95,11 @@ def fetch_current_dockerfile(slug: str) -> str:
override = per_bottle_dockerfile(slug)
if override is not None:
return override
repo_dockerfile = get_provider("claude").dockerfile
repo_dockerfile = _repo_dockerfile_path()
if repo_dockerfile.is_file():
return repo_dockerfile.read_text()
raise CapabilityApplyError(
f"no per-bottle Dockerfile for {slug} and no provider Dockerfile at "
f"no per-bottle Dockerfile for {slug} and no repo Dockerfile at "
f"{repo_dockerfile}"
)
@@ -125,6 +127,13 @@ def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
# --- Internals -------------------------------------------------------------
def _repo_dockerfile_path() -> Path:
"""Path to the repo's Claude Dockerfile (one dir above this module's
package root). Resolved at call time so the path is correct
regardless of where this module is imported from."""
# bot_bottle/backend/docker/capability_apply.py -> repo root
return Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
def snapshot_transcript(slug: str) -> None:
"""`docker cp` /home/node/.claude out of the agent container into
bot_bottle/backend/docker/cleanup.py
+1 -1
View File
@@ -31,7 +31,7 @@ from ... import supervise as _supervise
from ...log import info, warn
from . import util as docker_mod
from .bottle_cleanup_plan import DockerBottleCleanupPlan
from ...bottle_state import bottle_state_dir, is_preserved
from .bottle_state import bottle_state_dir, is_preserved
from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
bot_bottle/backend/docker/compose.py
+114 -39
View File
@@ -7,14 +7,34 @@ two networks, no named volumes.
Pure function. No I/O, no subprocess. Expects every launch-time
field (network names, CA host paths, etc.) on the plan's inner
plans to be populated; chunks 2+3 own that ordering.
plans to be populated; chunks 2+3 own that ordering. Chunk 1 just
encodes the translation so it can be unit-tested in isolation.
Conditional services follow the plan content:
Conditional services follow the plan content (matches the
SDK-call branching in `launch.py` today):
- agent + sidecars bundle: always.
- git-gate: iff plan.git_gate_plan.upstreams.
- egress: iff plan.egress_plan.routes.
- supervise: iff plan.supervise_plan is not None.
- pipelock + agent: always.
- git-gate: iff plan.git_gate_plan.upstreams.
- egress: iff plan.egress_plan.routes.
- supervise: iff plan.supervise_plan is not None.
Naming:
- Compose project: `bot-bottle-<slug>`.
- Service names (inside the file): `agent`, `pipelock`,
`egress`, `git-gate`, `supervise`.
- `container_name:` matches today's pattern
(`bot-bottle-<service>-<slug>`) so dashboard/cleanup discovery
via the prefix scan keeps working through the transition.
- Network aliases preserve the current dial-by-shortname pattern
for `egress` / `supervise`, and add the long container-name as
an internal-network alias for `pipelock` / `git-gate` so any
caller still referencing the long name resolves.
Sidecars that are built (egress, git-gate, supervise) get a
compose `build:` block pointing at the repo Dockerfile; the
`image:` tag is set explicitly so cached images on the daemon
aren't rebuilt on every up.
"""
from __future__ import annotations
@@ -28,11 +48,10 @@ from typing import Any
from ...egress import (
EGRESS_HOSTNAME,
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_sidecar_env_entries,
)
from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn
from ...pipelock import PIPELOCK_HOSTNAME
from ...supervise import (
CURRENT_CONFIG_DIR_IN_AGENT,
QUEUE_DIR_IN_CONTAINER,
@@ -44,7 +63,7 @@ from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from .bottle_plan import DockerBottlePlan
from .egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT,
EGRESS_PIPELOCK_CA_IN_CONTAINER,
)
from .git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
@@ -52,7 +71,11 @@ from .git_gate import (
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from . import network as network_mod
from .pipelock import (
PIPELOCK_CA_CERT_IN_CONTAINER,
PIPELOCK_CA_KEY_IN_CONTAINER,
PIPELOCK_PORT,
)
from .sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
@@ -68,11 +91,12 @@ def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
"""Render a Compose v2 spec dict from a fully-resolved
DockerBottlePlan.
The plan must have its inner plans (`git_gate_plan`,
`egress_plan`, `supervise_plan`) populated with launch-time
fields — network names, CA host paths. The renderer doesn't
validate; callers feed it a fully-resolved plan or get an
incomplete compose spec back.
The plan must have its inner plans (`proxy_plan`,
`git_gate_plan`, `egress_plan`, `supervise_plan`) populated
with launch-time fields — network names, CA host paths,
pipelock_proxy_url. The renderer doesn't validate; callers
feed it a fully-resolved plan or get an incomplete compose
spec back.
"""
project = f"bot-bottle-{plan.slug}"
services: dict[str, Any] = {
@@ -94,11 +118,11 @@ def _networks(plan: DockerBottlePlan) -> dict[str, Any]:
bridge."""
return {
"internal": {
"name": network_mod.network_name_for_slug(plan.slug),
"name": plan.proxy_plan.internal_network,
"internal": True,
},
"egress": {
"name": network_mod.network_egress_name_for_slug(plan.slug),
"name": plan.proxy_plan.egress_network,
},
}
@@ -118,12 +142,29 @@ def _bind(host: str | Path, target: str, *, read_only: bool = True) -> dict[str,
def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"""The `sidecars` service: one container per bottle, bundle
image, all daemons under a Python init supervisor.
image, all four daemons under a Python init supervisor.
Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS` env.
egress is always present; git-gate / supervise are conditional.
Mechanics:
- Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS`
env. pipelock is always present; egress / git-gate /
supervise are conditional on the plan.
- Volumes are the union of the four daemons' bind-mounts,
preserving the same in-container paths so each daemon
finds its config / hooks / CA where it expects.
- Environment is the union of *daemon-private* env vars
(EGRESS_UPSTREAM_PROXY, SUPERVISE_BOTTLE_SLUG, etc).
HTTPS_PROXY is NOT propagated here — see the comment in
egress_entrypoint.sh; setting it at the container level
would route git-gate's git fetches through pipelock,
which is wrong.
- Network aliases register every legacy short/long
hostname (pipelock, egress, git-gate, supervise plus
their `bot-bottle-<service>-<slug>` long forms) so
the agent's HTTPS_PROXY URL and any other inter-service
reference resolves to the bundle.
"""
daemons: list[str] = ["egress"]
daemons: list[str] = ["egress", "pipelock"]
if plan.git_gate_plan.upstreams:
daemons.append("git-gate")
if plan.supervise_plan is not None:
@@ -132,14 +173,31 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
env: list[str] = [f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(daemons)}"]
volumes: list[dict[str, Any]] = []
# --- egress -------------------------------------------------------
ep = plan.egress_plan
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
if ep.routes:
volumes.append(_bind(ep.routes_path.parent, str(Path(EGRESS_ROUTES_IN_CONTAINER).parent)))
env.extend(egress_sidecar_env_entries(ep))
# --- pipelock ----------------------------------------------------
pp = plan.proxy_plan
volumes += [
_bind(pp.yaml_path, "/etc/pipelock.yaml"),
_bind(pp.ca_cert_host_path, PIPELOCK_CA_CERT_IN_CONTAINER),
_bind(pp.ca_key_host_path, PIPELOCK_CA_KEY_IN_CONTAINER),
]
# --- git-gate -----------------------------------------------------
# --- egress (always part of the bundle; the EGRESS_UPSTREAM_*
# env vars + ca bind-mounts are needed iff routes exist; when
# the bottle has no routes the egress daemon falls back to its
# `regular@9099` mode and is unused) -----------------------------
ep = plan.egress_plan
if ep.routes:
env.append(f"EGRESS_UPSTREAM_PROXY={ep.pipelock_proxy_url}")
env.append(f"EGRESS_UPSTREAM_CA={EGRESS_PIPELOCK_CA_IN_CONTAINER}")
volumes += [
_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER),
_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER),
_bind(ep.pipelock_ca_host_path, EGRESS_PIPELOCK_CA_IN_CONTAINER),
]
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
# --- git-gate ----------------------------------------------------
gp = plan.git_gate_plan
if gp.upstreams:
volumes += [
@@ -159,7 +217,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
))
# --- supervise ----------------------------------------------------
# --- supervise ---------------------------------------------------
sp = plan.supervise_plan
if sp is not None:
env += [
@@ -174,7 +232,13 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"read_only": False,
})
internal_aliases = [EGRESS_HOSTNAME]
# Internal-network aliases: the agent reaches each daemon through
# its short name (pipelock / egress / git-gate / supervise) which
# the bundle answers as if it were the daemon itself.
internal_aliases = [
PIPELOCK_HOSTNAME,
EGRESS_HOSTNAME,
]
if gp.upstreams:
internal_aliases.append(GIT_GATE_HOSTNAME)
if sp is not None:
@@ -199,8 +263,11 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
"""Agent container. Runs `sleep infinity`; claude is `docker
exec -it`'d into it later. HTTP_PROXY/HTTPS_PROXY point at the
egress sidecar."""
exec -it`'d into it later. No TTY at the container level —
interactivity is per-exec. HTTP_PROXY/HTTPS_PROXY point at the
egress short-alias when an egress is declared, otherwise
straight at pipelock's container name. CA trust trio matches
the existing launch.py wiring."""
proxy_url = _agent_proxy_url(plan)
no_proxy = _agent_no_proxy(plan)
env: list[str] = [
@@ -221,10 +288,9 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
# never lands on argv or in the compose file.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
service: dict[str, Any] = {
"image": plan.image,
"image": plan.runtime_image,
"container_name": plan.container_name,
"command": ["sleep", "infinity"],
"networks": {"internal": None},
@@ -232,6 +298,8 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
}
if plan.use_runsc:
service["runtime"] = "runsc"
if plan.env_file and plan.env_file.exists() and plan.env_file.stat().st_size > 0:
service["env_file"] = [str(plan.env_file)]
volumes: list[dict[str, Any]] = []
if plan.supervise_plan is not None:
@@ -251,14 +319,21 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
def _agent_proxy_url(plan: DockerBottlePlan) -> str:
"""Agent's HTTP_PROXY — always points at egress."""
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
"""Pick the agent's HTTP_PROXY. With egress declared, the agent
goes through egress (which in turn HTTPS_PROXYs to pipelock on
its outbound leg). Without egress, the agent talks straight to
pipelock."""
if plan.egress_plan.routes:
from .egress import EGRESS_PORT
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
return f"http://{PIPELOCK_HOSTNAME}:{PIPELOCK_PORT}"
def _agent_no_proxy(plan: DockerBottlePlan) -> str:
"""NO_PROXY for the agent: loopback always; supervise hostname
when the supervise sidecar is up (MCP long-poll must bypass
the egress proxy)."""
"""NO_PROXY for the agent. Matches the launch.py rules:
loopback always, supervise hostname when the supervise sidecar
is up (the MCP long-poll pattern needs to bypass pipelock's
idle timeout)."""
hosts = ["localhost", "127.0.0.1"]
if plan.supervise_plan is not None:
hosts.append(SUPERVISE_HOSTNAME)
bot_bottle/backend/docker/egress.py
+17 -3
View File
@@ -22,8 +22,14 @@ from ...log import die
EGRESS_PORT = int(os.environ.get("BOT_BOTTLE_EGRESS_PORT", "9099"))
# In-container path for mitmproxy's CA. The format is a single PEM
# file holding BOTH the cert and the private key, concatenated.
# file holding BOTH the cert and the private key, concatenated. The
# upstream-trust CA (pipelock's, so egress trusts the upstream
# leg) is a separate file because pipelock keeps a different CA on
# its end.
EGRESS_CA_IN_CONTAINER = "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem"
EGRESS_PIPELOCK_CA_IN_CONTAINER = (
"/home/mitmproxy/.mitmproxy/pipelock-ca.pem"
)
def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
@@ -36,8 +42,16 @@ def egress_tls_init(stage_dir: Path) -> tuple[Path, Path]:
trust store by `provision_ca` so the agent trusts the bumped
CONNECT cert egress presents.
openssl req's `subjectKeyIdentifier=hash` extension uses
SHA-1(pubkey), matching mitmproxy's AKI computation on leaves.
Why openssl req (not the pipelock binary's `tls init`):
pipelock's CA generator stamps a non-standard `Subject Key
Identifier` on the CA (random rather than SHA-1 of the pubkey).
mitmproxy computes the `Authority Key Identifier` on each leaf
it mints as SHA-1(issuer's pubkey). openssl's chain validator
uses the leaf's AKI to find the issuer cert by SKI; pipelock's
SKI doesn't match → openssl reports "unable to get local issuer
certificate" even though the CA is right there in the trust
store. openssl req's `subjectKeyIdentifier=hash` extension uses
SHA-1(pubkey), matching mitmproxy's computation.
Both files live under `<stage_dir>/egress-ca/` (mode 644 —
`docker cp` preserves the mode into the container, where the
bot_bottle/backend/docker/egress_apply.py
+309 -26
View File
@@ -1,22 +1,86 @@
"""Host-side helper for egress sidecar inspection and live updates.
"""Host-side helper to apply a routes.yaml change to a running
egress sidecar (PRD 0014 retargeted by PRD 0017 chunk 3).
The approve path uses this module to validate a proposed routes file,
write it to the bottle's live egress state dir, and signal the sidecar
bundle so the mitmproxy addon reloads it.
Used by the supervise dashboard when the operator approves an
egress-block proposal (or runs the operator-initiated
`routes edit <bottle>` verb). Fetches the current routes.yaml via
`docker exec cat`, validates the new content, writes it into the
sidecar via `docker cp`, then `docker kill --signal HUP` to make
the addon reload without dropping connections.
Also mirrors the new route hosts into pipelock's hostname allowlist
so the downstream leg lets them through — egress enforces
the path-aware allowlist on the agent leg, pipelock enforces the
hostname allowlist + DLP body scan on the upstream leg, and a
host added to one must be in the other or the request 403s
somewhere along the chain.
Raises EgressApplyError on any failure — the dashboard
surfaces the message and keeps the proposal pending so the
operator can retry.
"""
from __future__ import annotations
import os
import json
import re
import subprocess
from pathlib import Path
from ...egress import EGRESS_ROUTES_IN_CONTAINER
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError
from ...egress_addon_core import load_routes
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
from .bottle_state import egress_state_dir
from .sidecar_bundle import sidecar_bundle_container_name
from .pipelock_apply import (
PipelockApplyError,
apply_allowlist_change,
fetch_current_allowlist,
parse_allowlist_content,
render_allowlist_content,
)
def _render_routes_payload(routes_list: list[dict[str, object]]) -> str:
"""Render a list-of-dicts routes payload as YAML matching the
shape `egress_render_routes` produces. The apply path
round-trips current routes.yaml through this so the file the
sidecar sees stays in the YAML format the addon expects."""
if not routes_list:
return "routes: []\n"
lines: list[str] = ["routes:"]
for entry in routes_list:
host = str(entry.get("host", ""))
lines.append(f' - host: "{host}"')
auth_scheme = entry.get("auth_scheme")
token_env = entry.get("token_env")
if auth_scheme and token_env:
lines.append(f' auth_scheme: "{auth_scheme}"')
lines.append(f' token_env: "{token_env}"')
paths = entry.get("path_allowlist") or []
if paths:
lines.append(" path_allowlist:")
for p in paths:
lines.append(f' - "{p}"')
return "\n".join(lines) + "\n"
def _egress_routes_host_path(slug: str) -> Path:
"""The bind-mount source for the egress sidecar's routes.yaml.
Must match what egress.prepare wrote at chunk-2 paths."""
return egress_state_dir(slug) / "egress_routes.yaml"
class EgressApplyError(RuntimeError):
"""Raised when fetch / apply fails. Caller renders to the
operator; does not crash the dashboard."""
def fetch_current_routes(slug: str) -> str:
"""Read the live routes.yaml from the running egress sidecar
for `slug`. Returns the file content as a string. Raises
EgressApplyError if the sidecar isn't reachable or the read
fails."""
container = sidecar_bundle_container_name(slug)
r = subprocess.run(
["docker", "exec", container, "cat", EGRESS_ROUTES_IN_CONTAINER],
@@ -30,31 +94,250 @@ def fetch_current_routes(slug: str) -> str:
return r.stdout
class DockerEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None:
container = sidecar_bundle_container_name(slug)
result = subprocess.run(
["docker", "kill", "--signal", "HUP", container],
capture_output=True, text=True, check=False, env=os.environ,
def validate_routes_content(content: str) -> None:
"""Syntactic check before SIGHUP — the addon's reload also
validates, but failing here keeps the old routes live and gives
the operator a clearer error than the addon's stderr line."""
try:
load_routes(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
def _hosts_in_routes(content: str) -> list[str]:
"""Extract the host list from a routes.yaml content string.
Uses the addon's own parser so any host the addon will match on
also lands in pipelock's allowlist. Returns sorted+deduped."""
try:
routes = load_routes(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
return sorted({r.host for r in routes if r.host})
# Pipelock's allowlist parser accepts only literal hostnames:
# `[A-Za-z0-9_.-]+`. Anything else (wildcards, IPv6 literals,
# stray characters) is silently dropped from the mirror so the
# pipelock apply doesn't fail parse before the new yaml is even
# written. The dropped hosts stay on egress's route table —
# but the addon does exact-host match only, so they'll never
# match anything either. (Wildcard host matching was removed —
# see `match_route` in egress_addon_core for the rationale.)
_PIPELOCK_HOST_RE = re.compile(r"^[A-Za-z0-9_.-]+$")
def _pipelock_safe_hosts(hosts: list[str]) -> list[str]:
"""Drop any host pipelock's allowlist parser would reject.
Order preserved."""
return [h for h in hosts if _PIPELOCK_HOST_RE.match(h)]
def _mirror_hosts_to_pipelock(slug: str, hosts: list[str]) -> None:
"""Ensure every pipelock-compatible `hosts` entry is on
pipelock's allowlist. Fetches pipelock's current allowlist,
merges, re-applies. Hosts pipelock can't represent (wildcards,
etc.) are silently skipped — they stay live on egress
but aren't enforced at pipelock. No-op if every host is already
present (apply still restarts pipelock if any host is new).
Raises EgressApplyError on pipelock failures so the
caller's diff/audit reflects the half-state."""
safe_hosts = _pipelock_safe_hosts(hosts)
try:
current = fetch_current_allowlist(slug)
existing = parse_allowlist_content(current)
merged = sorted(set(existing) | set(safe_hosts))
if merged == sorted(existing):
return # nothing to add
apply_allowlist_change(slug, render_allowlist_content(merged))
except PipelockApplyError as e:
# Mirror runs BEFORE the egress write, so egress
# is unchanged on this failure path. Report it as a
# pipelock-side problem so the operator looks in the right
# place; their `pipelock edit` flow can repair manually.
raise EgressApplyError(
f"pipelock allowlist mirror failed (egress NOT "
f"updated): {e}. Fix pipelock's allowlist manually with "
f"`pipelock edit <bottle>` then retry the proposal."
) from e
def apply_routes_change(slug: str, new_content: str) -> tuple[str, str]:
"""Apply `new_content` to the egress sidecar for `slug`:
1. Fetch current routes.yaml (for the before-diff).
2. Validate the new content via the addon's own parser.
3. Mirror the route hosts onto pipelock's allowlist (so the
downstream hostname gate lets them through).
4. Write to a temp file, `docker cp` into the egress
sidecar.
5. `docker kill --signal HUP` so the addon reloads.
Order matters: pipelock first, then egress. If the
pipelock step fails, egress hasn't been touched and the
old routes stay live. If the egress step fails after
pipelock succeeded, pipelock has the host in its allowlist but
egress doesn't enforce it yet — harmless extra-permissive
state at pipelock, and a re-approval will land the egress
side.
Returns (before, after) where `after` == `new_content`. Raises
EgressApplyError on any step."""
container = sidecar_bundle_container_name(slug)
before = fetch_current_routes(slug)
validate_routes_content(new_content)
# Pipelock mirror first — if it fails, egress stays intact
# and the operator gets a clear error about the half-state.
_mirror_hosts_to_pipelock(slug, _hosts_in_routes(new_content))
# routes.yaml is bind-mounted into the egress container as a
# SINGLE FILE. Docker single-file bind mounts pin the source
# inode at mount time; write-temp-then-rename swaps the inode
# on the host, which leaves the container's mount pointing at
# the now-orphaned old inode (so the SIGHUP'd reload re-reads
# unchanged content). Write in-place instead. Lose file-level
# atomicity, but the apply path issues SIGHUP only AFTER the
# write returns, and the addon's `load_routes` raises
# `ValueError` on a partial read and keeps the previous
# in-memory routes — so a SIGHUP that hypothetically raced an
# in-flight write is non-disruptive.
target = _egress_routes_host_path(slug)
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(new_content)
# mitmproxy in the container reads through the bind mount as
# uid 1000; the host file has to be world-readable for that
# read to succeed (parent dir at 0o700 still restricts who
# can reach the file on the host). Routes content is not
# secret — tokens live in the container's environ — so 0o644
# is the right trade-off.
target.chmod(0o644)
sig = subprocess.run(
["docker", "kill", "--signal", "HUP", container],
capture_output=True, text=True, check=False,
)
if sig.returncode != 0:
raise EgressApplyError(
f"failed to SIGHUP {container}: "
f"{(sig.stderr or '').strip()}"
)
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'docker kill failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {container}: "
f"{last_error or 'docker kill failed'}"
)
return before, new_content
applicator = DockerEgressApplicator()
def _merge_single_route(
current_yaml: str, new_route: dict[str, object],
) -> str:
"""Merge a single proposed route into the current routes.yaml
content, returning the merged YAML string.
Behavior:
- If `new_route['host']` is NOT in the current routes →
append the route.
- If the host IS already present → union the path_allowlist
entries (proposed ∪ existing). The existing `auth_scheme`
and `token_env` are preserved — agent-proposed auth changes
on an existing host are ignored, matching the tool's
documented semantics.
Round-trips the file through `yaml_subset` (the same parser
the addon uses), so the merged output is in the YAML format
the sidecar reads. Token VALUES never appear here; the routes
file carries only env-var slot NAMES."""
try:
cfg = parse_yaml_subset(current_yaml)
except YamlSubsetError as e:
raise EgressApplyError(
f"current routes.yaml is not valid YAML: {e}"
) from e
routes = cfg.get("routes")
if not isinstance(routes, list):
raise EgressApplyError(
"current routes.yaml: 'routes' is not a list"
)
new_host = str(new_route.get("host", "")).lower()
if not new_host:
raise EgressApplyError(
"proposed route is missing 'host'"
)
proposed_paths = list(new_route.get("path_allowlist") or [])
# Look for an existing entry with the same host (case-insensitive).
for entry in routes:
if not isinstance(entry, dict):
continue
if str(entry.get("host", "")).lower() == new_host:
# Merge path_allowlist: union proposed + existing, ordered
# by first-seen so existing paths stay in original order.
existing_paths: list[str] = list(entry.get("path_allowlist") or [])
seen = {p: None for p in existing_paths}
for p in proposed_paths:
seen.setdefault(p, None)
merged_paths = list(seen.keys())
if merged_paths:
entry["path_allowlist"] = merged_paths
# Preserve existing auth — tool description says agent-
# proposed auth on an existing host is ignored.
break
else:
# Host not present; build a new route entry from the
# proposed fields. Need to assign a token_env slot if
# `auth` was proposed (otherwise the addon's parser rejects
# a half-set auth pair). Slots: count existing slots, pick
# the next free index.
entry = {"host": new_route["host"]}
if proposed_paths:
entry["path_allowlist"] = proposed_paths
auth = new_route.get("auth")
if isinstance(auth, dict) and auth.get("scheme") and auth.get("token_ref"):
existing_slots = sorted({
str(r.get("token_env"))
for r in routes
if isinstance(r, dict) and r.get("token_env")
})
next_idx = len(existing_slots)
entry["auth_scheme"] = str(auth["scheme"])
entry["token_env"] = f"EGRESS_TOKEN_{next_idx}"
# NOTE: the addon reads token VALUES from its container's
# environ keyed by token_env. A newly-added auth route at
# runtime points at a slot that has no env value → the
# addon will 403 with "token env unset" until the operator
# arranges for the value to land in the container's env.
# Recording this here so the operator-facing diff carries
# the slot name they'll need to provision.
routes.append(entry)
return _render_routes_payload(routes)
def add_route(slug: str, proposed_route_json: str) -> tuple[str, str]:
"""Apply a single-route addition to the egress. Parses the
agent's proposed route, fetches the current routes file, merges,
and applies via `apply_routes_change`. Returns (before, after)
full-file content for the audit log."""
try:
proposed = json.loads(proposed_route_json)
except json.JSONDecodeError as e:
raise EgressApplyError(
f"proposed route is not valid JSON: {e}"
) from e
if not isinstance(proposed, dict):
raise EgressApplyError(
"proposed route must be a JSON object"
)
current = fetch_current_routes(slug)
merged = _merge_single_route(current, proposed)
return apply_routes_change(slug, merged)
__all__ = [
"DockerEgressApplicator",
"EgressApplyError",
"applicator",
"add_route",
"apply_routes_change",
"fetch_current_routes",
"validate_routes_content",
]
bot_bottle/backend/docker/enumerate.py
+1 -3
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import subprocess
from .. import ActiveAgent
from ...bottle_state import read_metadata
from .bottle_state import read_metadata
from .compose import compose_project_name, list_active_slugs
@@ -39,8 +39,6 @@ def enumerate_active() -> list[ActiveAgent]:
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=tuple(sorted(services)),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
bot_bottle/backend/docker/freezer.py
-23
View File
@@ -1,23 +0,0 @@
"""DockerFreezer — snapshot a Docker bottle via `docker commit`."""
from __future__ import annotations
from .. import ActiveAgent
from ..freeze import Freezer
from .util import commit_container
from ...log import info
class DockerFreezer(Freezer):
"""Freezes a Docker bottle by running `docker commit`."""
backend_name = "docker"
def _freeze(self, agent: ActiveAgent) -> str:
container = f"bot-bottle-{agent.slug}"
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
commit_container(container, image_tag)
return image_tag
def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: docker save {image_ref} -o {slug}.tar")
bot_bottle/backend/docker/launch.py
+67 -40
View File
@@ -4,19 +4,25 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
The flow is:
1. Build the agent image from the provider Dockerfile (compose
builds the sidecar images via the `build:` directive on first up).
2. Mint the per-bottle egress CA (chunk 2 writes it under
state/<slug>/egress/).
3. Populate the inner plans with launch-time fields so the
renderer can read network names, CA paths.
1. Build the agent's base + derived image (compose builds the
sidecar images via the `build:` directive on first up).
2. Pre-create the per-bottle networks. We do this outside compose
so we can inspect the assigned internal CIDR and embed it in
pipelock's yaml (compose's `external: true` lets the compose
file reference these pre-existing networks).
3. Mint the per-bottle CAs (chunk 2 writes them under
state/<slug>/{pipelock,egress}/).
4. Re-render pipelock yaml with the now-known internal CIDR so
the SSRF allowlist exempts the bottle's own subnet.
5. Populate the inner plans with launch-time fields so the
renderer can read network names, CA paths, pipelock URL.
6. Render the compose spec, write it to
state/<slug>/docker-compose.yml, write metadata.json.
7. `docker compose up -d` (token + OAuth values flow into the
compose subprocess env so `environment: [NAME]` bare-name
entries inherit without rendering values into the file).
8. Provision (CA install, prompt copy, skills, workspace, git,
supervise config) — unchanged, uses `docker exec` / `docker cp`.
8. Provision (CA install, prompt copy, skills, git, supervise
config) — unchanged, uses `docker exec`.
9. Yield a DockerBottle handle. `exec_agent` runs claude via
`docker exec -it` exactly like the pre-compose world.
@@ -43,11 +49,11 @@ from . import network as network_mod
from . import util as docker_mod
from .bottle import DockerBottle
from .bottle_plan import DockerBottlePlan
from ...bottle_state import (
from .bottle_state import (
bottle_state_dir,
egress_state_dir,
git_gate_state_dir,
read_committed_image,
pipelock_state_dir,
)
from .compose import (
bottle_plan_to_compose,
@@ -60,6 +66,10 @@ from .compose import (
write_compose_file,
)
from .egress import egress_tls_init
from .pipelock import (
BUNDLE_LOCAL_PIPELOCK_URL,
pipelock_tls_init,
)
# Where the repo root lives, for `docker build` context. Computed once.
@@ -70,19 +80,19 @@ _REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def launch(
plan: DockerBottlePlan,
*,
provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None],
provision: Callable[[DockerBottlePlan, str], str | None],
) -> Generator[DockerBottle, None, None]:
"""Build, launch, and provision a Docker bottle via compose.
Teardown on exit."""
stack = ExitStack()
_bottle_for_revoke = plan.manifest.bottle
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
def teardown() -> None:
try:
stack.close()
except BaseException as exc: # noqa: W0718 — teardown must not fail
except BaseException as exc:
warn(
f"teardown failed for container {plan.container_name}"
f" (compose-down): {exc!r}"
@@ -92,30 +102,46 @@ def launch(
)
try:
# Step 1: agent image. Use a committed snapshot when one exists
# and is present in the local daemon; otherwise build from the
# Dockerfile. Sidecar images get built lazily by `docker compose
# up` via the renderer's `build:` directives.
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
else:
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
# Step 1: agent image build. Sidecar images get built lazily by
# `docker compose up` via the renderer's `build:` directives.
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
if plan.derived_image:
docker_mod.build_image_with_cwd(
plan.derived_image, plan.image, plan.workspace_plan
)
# Networks: compose-managed. The names are derived
# deterministically from the slug so the renderer can put
# them on the services and `compose up` creates them with
# those names. The empirical spike confirmed pipelock's
# SSRF guard only checks proxied-request destinations, not
# source IPs — so the bottle's own internal CIDR doesn't
# need to be in `ssrf.ip_allowlist`. Pre-create + CIDR
# introspection are gone; compose owns the network
# lifecycle.
internal_network = network_mod.network_name_for_slug(plan.slug)
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
# Mint per-bottle CAs into state/<slug>/{pipelock,egress}/.
ca_cert_host, ca_key_host = pipelock_tls_init(pipelock_state_dir(plan.slug))
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
)
# Populate launch-time fields on every inner plan so the
# renderer reads concrete network names, CA paths, and
# pipelock URL.
proxy_plan = dataclasses.replace(
plan.proxy_plan,
internal_network=internal_network,
internal_network_cidr="",
egress_network=egress_network,
ca_cert_host_path=ca_cert_host,
ca_key_host_path=ca_key_host,
)
git_gate_plan = plan.git_gate_plan
if git_gate_plan.upstreams:
git_gate_plan = dataclasses.replace(
@@ -123,13 +149,17 @@ def launch(
internal_network=internal_network,
egress_network=egress_network,
)
egress_plan = dataclasses.replace(
plan.egress_plan,
internal_network=internal_network,
egress_network=egress_network,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
egress_plan = plan.egress_plan
if egress_plan.routes:
egress_plan = dataclasses.replace(
egress_plan,
internal_network=internal_network,
egress_network=egress_network,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
pipelock_ca_host_path=ca_cert_host,
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
)
supervise_plan = plan.supervise_plan
if supervise_plan is not None:
supervise_plan = dataclasses.replace(
@@ -138,6 +168,7 @@ def launch(
)
plan = dataclasses.replace(
plan,
proxy_plan=proxy_plan,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
@@ -186,12 +217,8 @@ def launch(
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
bottle._prompt_path = provision(plan, bottle)
# Step 9: yield. exec_agent continues to use `docker exec -it`
# — the agent runs `sleep infinity` per the renderer's
bot_bottle/backend/docker/network.py
+14 -5
View File
@@ -1,10 +1,11 @@
"""Docker network plumbing for the per-agent egress topology.
The agent container sits on a Docker `--internal` network (no default
gateway). Egress straddles that network and a per-agent user-defined
bridge for upstream traffic. We deliberately do NOT use Docker's legacy
gateway). Pipelock straddles that network and a per-agent user-defined
bridge for upstream egress. We deliberately do NOT use Docker's legacy
`bridge` network because only user-defined bridges run Docker's
embedded DNS resolver, which egress needs to resolve upstream hostnames.
embedded DNS resolver, which pipelock needs to resolve api.anthropic.com
and similar upstream hostnames.
Naming: bot-bottle-net-<slug> (internal),
bot-bottle-egress-<slug> (egress). Numeric suffix on conflict
@@ -76,12 +77,20 @@ def network_create_internal(slug: str) -> str:
def network_create_egress(slug: str) -> str:
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`)
so the egress sidecar has working DNS for upstream hostnames."""
so the pipelock sidecar has working DNS for upstream hostnames."""
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
def network_inspect_cidr(name: str) -> str:
"""Return the IPv4 CIDR Docker assigned to a user-defined network."""
"""Return the IPv4 CIDR Docker assigned to a user-defined network.
Used by pipelock's SSRF guard exception: the bottle's internal
network sits in RFC1918 space, so pipelock's `internal:` list
would block any agent request whose destination resolves there
— including the cred-proxy sidecar's address. Adding the
network's CIDR to pipelock's `ssrf.ip_allowlist` lets traffic
targeted at the bottle's own sidecars through while pipelock
still body-scans and api_allowlist-gates as usual."""
result = subprocess.run(
["docker", "network", "inspect",
"--format", "{{range .IPAM.Config}}{{.Subnet}}{{end}}", name],
bot_bottle/backend/docker/pipelock.py
+81
View File
@@ -0,0 +1,81 @@
"""Docker-side pipelock helpers: image pin, container naming, and
the one-shot `pipelock tls init` host-side CA mint. The
prepare-time YAML rendering itself lives on the platform-neutral
`PipelockProxy` ABC — backends instantiate it directly.
The per-container `.start()` / `.stop()` lifecycle was deleted in
PRD 0024 chunk 3; compose-up owns the container lifecycle (PRD
0018) and the bundle path (PRD 0024) collapses pipelock + egress
+ git-gate + supervise into one container."""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
from ...log import die
# Re-exported for the compose renderer + smolmachines launch step
# (they used to import these from this module before they moved to
# the platform-neutral pipelock module).
from ...pipelock import ( # noqa: F401
PIPELOCK_CA_CERT_IN_CONTAINER,
PIPELOCK_CA_KEY_IN_CONTAINER,
)
# Pipelock image, pinned by digest. The digest is the multi-arch image
# index for ghcr.io/luckypipewrench/pipelock:2.3.0.
PIPELOCK_IMAGE = os.environ.get(
"BOT_BOTTLE_PIPELOCK_IMAGE",
"ghcr.io/luckypipewrench/pipelock@sha256:3b1a39417b98406ddc5dc2d8fcb42865ddc0c68a43d355db55f0f8cb06bc6de9",
)
# Listening port for pipelock's forward proxy.
PIPELOCK_PORT = os.environ.get("BOT_BOTTLE_PIPELOCK_PORT", "8888")
# The URL egress dials for its upstream HTTPS_PROXY. egress and
# pipelock share the same container's network namespace inside the
# sidecar bundle, so loopback reaches pipelock directly — no docker
# DNS aliases involved.
BUNDLE_LOCAL_PIPELOCK_URL = f"http://127.0.0.1:{PIPELOCK_PORT}"
def pipelock_tls_init(stage_dir: Path) -> tuple[Path, Path]:
"""Generate a fresh per-bottle CA via a one-shot pipelock container.
Runs `pipelock tls init` against a host-mounted scratch dir, leaving
`ca.pem` (public cert, mode 600) and `ca-key.pem` (private key, mode
600) under `<stage_dir>/pipelock-ca/`. Returns the two host paths.
The image is pinned (same digest the running sidecar uses) so the
generated CA matches what the sidecar expects. Output is owned by
whatever UID the one-shot ran as; the compose renderer's
bind-mounts pin the files in place at runtime, so ownership
inside the running sidecar (root in pipelock's distroless image)
is independent."""
work = stage_dir / "pipelock-ca"
work.mkdir(exist_ok=True)
result = subprocess.run(
["docker", "run", "--rm",
"-v", f"{work}:/h",
"-e", "PIPELOCK_HOME=/h",
PIPELOCK_IMAGE, "tls", "init"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(f"pipelock tls init failed: {result.stderr.strip()}")
cert = work / "ca.pem"
key = work / "ca-key.pem"
if not cert.is_file() or not key.is_file():
die(f"pipelock tls init did not produce ca files in {work}")
# Explicit perms in case a future pipelock release changes
# defaults. Pipelock runs as root in its distroless image and
# bind-mounts work with 0o600 (root reads everything); the key
# has no reason to be readable to anyone else on the host.
key.chmod(0o600)
cert.chmod(0o644)
return (cert, key)
bot_bottle/backend/docker/pipelock_apply.py
+200
View File
@@ -0,0 +1,200 @@
"""pipelock_apply — host-side helper to apply an api_allowlist
change to a running pipelock sidecar (PRD 0015).
Used by the supervise dashboard when the operator approves a
pipelock-block proposal (or runs the operator-initiated `pipelock
edit <bottle>` verb). Fetches the current pipelock.yaml via `docker
exec`, parses it, swaps the api_allowlist with the proposed hosts,
re-renders, writes back via the bind-mount path, then signals the
bundle supervisor to restart the pipelock daemon (`docker kill
--signal USR1`) so
pipelock picks up the new config.
v1 uses restart, not SIGHUP — pipelock has no in-process reload
hook and adding one is the "SIGHUP reload for pipelock" open
question in PRD 0015. Restart drops in-flight outbound calls; the
agent's HTTP client retries pick up against the restarted proxy.
"""
from __future__ import annotations
import os
import re
import subprocess
import tempfile
from pathlib import Path
from ...pipelock import pipelock_render_yaml
from ...yaml_subset import YamlSubsetError, parse_yaml_subset
from .bottle_state import pipelock_state_dir
from .sidecar_bundle import sidecar_bundle_container_name
def _pipelock_yaml_host_path(slug: str) -> Path:
"""The bind-mount source for the pipelock sidecar's
pipelock.yaml — matches what pipelock.prepare wrote at chunk-2
paths."""
return pipelock_state_dir(slug) / "pipelock.yaml"
PIPELOCK_YAML_IN_CONTAINER = "/etc/pipelock.yaml"
# Allowlist proposals are one-hostname-per-line. Blank lines and
# `#`-prefixed comments are ignored. The character set matches the
# supervise sidecar's syntactic check on the agent's pipelock-block
# proposal (alphanumerics + dot/dash/underscore).
_HOST_OK = re.compile(r"^[A-Za-z0-9_.-]+$")
class PipelockApplyError(RuntimeError):
"""Raised when fetch / parse / apply fails. The dashboard renders
the message and keeps the proposal pending — never crashes."""
def parse_allowlist_content(content: str) -> list[str]:
"""One hostname per line. Blanks and `#` comments are ignored.
Raises PipelockApplyError if a line has a disallowed character."""
hosts: list[str] = []
for i, raw_line in enumerate(content.splitlines(), start=1):
line = raw_line.strip()
if not line or line.startswith("#"):
continue
if not _HOST_OK.match(line):
raise PipelockApplyError(
f"allowlist line {i}: {line!r} has disallowed characters"
)
hosts.append(line)
return hosts
def render_allowlist_content(hosts: list[str]) -> str:
"""Hosts → one-per-line string (the operator-facing format)."""
if not hosts:
return ""
return "\n".join(hosts) + "\n"
def fetch_current_yaml(slug: str) -> str:
"""Read the live /etc/pipelock.yaml from the sidecar bundle.
Uses `docker cp` because pipelock inside the bundle is the
distroless pipelock binary with no shell, and `docker cp` is a
daemon-API tarball copy that works regardless of what's
available inside the container.
Raises PipelockApplyError if the read fails."""
container = sidecar_bundle_container_name(slug)
fd, tmp_path = tempfile.mkstemp(prefix="cb-pipelock-fetch.", suffix=".yaml")
os.close(fd)
try:
r = subprocess.run(
[
"docker", "cp",
f"{container}:{PIPELOCK_YAML_IN_CONTAINER}", tmp_path,
],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
raise PipelockApplyError(
f"could not fetch pipelock.yaml from {container}: "
f"{(r.stderr or '').strip() or 'container not running?'}"
)
return Path(tmp_path).read_text()
finally:
try:
Path(tmp_path).unlink()
except OSError:
pass
def fetch_current_allowlist(slug: str) -> str:
"""Fetch the live yaml, extract api_allowlist, render as one-per-
line — the operator-facing format for the TUI / agent's
current-config mount."""
yaml = fetch_current_yaml(slug)
try:
cfg = parse_yaml_subset(yaml)
except YamlSubsetError as e:
raise PipelockApplyError(f"running pipelock yaml: {e}") from e
hosts = cfg.get("api_allowlist", [])
if not isinstance(hosts, list):
raise PipelockApplyError(
"running pipelock yaml: api_allowlist is not a list"
)
return render_allowlist_content([str(h) for h in hosts])
def apply_allowlist_change(
slug: str, new_allowlist_content: str,
) -> tuple[str, str]:
"""Apply `new_allowlist_content` to the sidecar bundle:
1. Parse the proposed hosts (one per line).
2. Fetch + parse current pipelock.yaml.
3. Replace api_allowlist with the proposed hosts; re-render.
4. Write the new yaml to the bind-mount source.
5. `docker kill --signal USR1 <bundle>` so the supervisor
restarts the pipelock daemon in place (leaving egress,
git-gate, and supervise running). Pipelock has no
in-process reload; the supervisor's per-daemon restart
keeps the agent's MCP socket alive — a whole-bundle
`docker restart` would bounce supervise too.
Returns (before, after) where both are one-per-line allowlist
strings (operator-facing format). Raises PipelockApplyError on
any failure; the sidecar's existing config stays in place until
the host write succeeds, and the SIGUSR1 is what makes it
live."""
new_hosts = parse_allowlist_content(new_allowlist_content)
container = sidecar_bundle_container_name(slug)
current_yaml = fetch_current_yaml(slug)
try:
cfg = parse_yaml_subset(current_yaml)
except YamlSubsetError as e:
raise PipelockApplyError(f"running pipelock yaml: {e}") from e
current_hosts = cfg.get("api_allowlist", [])
if not isinstance(current_hosts, list):
raise PipelockApplyError(
"running pipelock yaml: api_allowlist is not a list"
)
before = render_allowlist_content([str(h) for h in current_hosts])
after = render_allowlist_content(new_hosts)
cfg["api_allowlist"] = new_hosts
rendered = pipelock_render_yaml(cfg)
# pipelock.yaml is bind-mounted into the container as a SINGLE
# FILE — same Docker single-file inode issue as egress_apply:
# write-temp-then-rename swaps the host inode and leaves the
# container's mount pointing at the orphaned old one. Write
# in-place. The SIGUSR1 below makes the new content live
# (pipelock has no in-process reload, so the supervisor
# restarts the pipelock daemon in response).
target = _pipelock_yaml_host_path(slug)
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(rendered)
# pipelock runs as root in its distroless image — any mode is
# fine — but 0o600 matches what prepare wrote.
target.chmod(0o600)
restart = subprocess.run(
["docker", "kill", "--signal", "USR1", container],
capture_output=True, text=True, check=False,
)
if restart.returncode != 0:
raise PipelockApplyError(
f"failed to signal {container} for pipelock restart: "
f"{(restart.stderr or '').strip()}"
)
return before, after
__all__ = [
"PIPELOCK_YAML_IN_CONTAINER",
"PipelockApplyError",
"apply_allowlist_change",
"fetch_current_allowlist",
"fetch_current_yaml",
"parse_allowlist_content",
"render_allowlist_content",
]
bot_bottle/backend/docker/prepare.py
+278
View File
@@ -0,0 +1,278 @@
"""Prepare step for the Docker bottle backend.
`resolve_plan` does all host-side resolution (image and container
names, env-file, prompt-file, proxy plan, runtime detection) and
returns a frozen DockerBottlePlan. No Docker resources are created;
the only side effects are scratch files under `stage_dir` and a probe
of `docker info`. Cross-backend host-side validation has already run
via the base class's `prepare` template before this is called.
"""
from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import agent_provision_plan, runtime_for
from ...egress import Egress
from ...env import ResolvedEnv, resolve_env
from ...git_gate import GitGate
from ...log import die
from ...pipelock import PipelockProxy
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .. import BottleSpec
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
from .bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
clear_preserve_marker,
egress_state_dir,
git_gate_state_dir,
per_bottle_dockerfile,
per_bottle_dockerfile_path,
per_bottle_image_tag,
pipelock_state_dir,
supervise_state_dir,
write_metadata,
)
from .sidecar_bundle import sidecar_bundle_container_name
def resolve_plan(
spec: BottleSpec,
*,
stage_dir: Path,
) -> DockerBottlePlan:
"""Resolve Docker-specific names and write scratch files. Trusts
that the agent and its skills/git-gate keys are present —
validation already ran in the base class."""
docker_mod.require_docker()
proxy = PipelockProxy()
git_gate = GitGate()
egress = Egress()
supervise = Supervise()
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = "/home/node"
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
# mints a random-suffixed identity (so parallel runs of the same
# agent in the same cwd don't collide on container/network
# names); a `resume` passes the recorded identity in via
# spec.identity to continue an existing bottle's state.
slug = spec.identity or bottle_identity(spec.agent_name)
# Record the launch metadata so `cli.py resume <identity>` can
# reconstruct the spec. Idempotent — re-writes on resume with a
# refreshed started_at.
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=f"bot-bottle-{slug}",
backend="docker",
))
# Clear any leftover preserve marker from a prior capability-block
# so this fresh launch can be cleaned up at session-end unless
# the agent triggers another capability-block.
clear_preserve_marker(slug)
# PRD 0016 capability-block: if a per-bottle Dockerfile has been
# written (via apply_capability_change), the base image becomes
# per_bottle_image_tag(slug) built from that file. --cwd still
# layers a derived image on top.
dockerfile_path = ""
if per_bottle_dockerfile(slug) is not None:
image_default = per_bottle_image_tag(slug)
dockerfile_path = str(per_bottle_dockerfile_path(slug))
elif provider.dockerfile:
image_default = f"bot-bottle-{provider.template}:{slug}"
dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
elif provider_runtime.dockerfile:
image_default = provider_runtime.image
dockerfile_path = provider_runtime.dockerfile
else:
image_default = provider_runtime.image
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
derived_image = ""
runtime_image = image
if spec.copy_cwd:
derived_image = os.environ.get(
"BOT_BOTTLE_DERIVED_IMAGE", f"bot-bottle-cwd:{slug}"
)
runtime_image = derived_image
default_container = f"bot-bottle-{slug}"
pinned_container = os.environ.get("BOT_BOTTLE_CONTAINER", "")
container_name_pinned = bool(pinned_container)
if container_name_pinned:
container_name = pinned_container
if docker_mod.container_exists(container_name):
die(
f"container '{container_name}' already exists "
f"(pinned via BOT_BOTTLE_CONTAINER). "
f"Remove it with 'docker rm -f {container_name}' or unset the override."
)
else:
container_name = ""
for candidate in docker_mod.container_name_candidates(default_container):
if not docker_mod.container_exists(candidate):
container_name = candidate
break
if not container_name:
die(
f"could not find a free container name after "
f"{default_container}-{docker_mod.MAX_CONTAINER_SUFFIX}; "
f"clean up old containers with 'docker rm -f <name>'"
)
# Probe the sidecar-bundle container name for an orphan from a
# previous run. Otherwise a stale bundle surfaces as a
# docker-create conflict deep inside launch() with no actionable
# hint; failing fast here points at the cleanup command.
bundle_name = sidecar_bundle_container_name(slug)
if docker_mod.container_exists(bundle_name):
die(
f"sidecar bundle container '{bundle_name}' already exists. "
f"This is an orphan from a previous run; clean it up with "
f"'./cli.py cleanup' (or 'docker rm -f {bundle_name}') and "
f"retry."
)
# PRD 0018 chunk 2: prepare-time scratch files live under
# ~/.bot-bottle/state/<slug>/<service>/ so chunk 3's compose
# bind-mounts can point at stable paths. The state subdirs are
# cleaned up by start.py's session-end teardown unless something
# explicitly preserves the state dir (capability-block, crash).
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
env_file = agent_dir / "agent.env"
prompt_file = agent_dir / "prompt.txt"
prompt_file.write_text("")
prompt_file.chmod(0o600)
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
resolved = resolve_env(manifest, spec.agent_name)
# Everything that should reach the bottle by-name (so its value
# never lands on argv or in env_file) goes into one dict. Nothing
# mutates the host os.environ.
forwarded_env: dict[str, str] = dict(resolved.forwarded)
_write_env_file(resolved, env_file)
prompt_file.write_text(agent.prompt)
use_runsc = docker_mod.runsc_available()
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
)
guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=guest_env)
pipelock_dir = pipelock_state_dir(slug)
pipelock_dir.mkdir(parents=True, exist_ok=True)
proxy_plan = proxy.prepare(
bottle, slug, pipelock_dir, agent_provision.egress_routes,
)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = egress.prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
# Current Dockerfile for the agent image. Read from the repo
# root; for `--cwd` derived images the base Dockerfile is what
# the agent should propose changes against (the derived layer
# is just a workspace copy).
# (routes.yaml + pipelock allowlist used to land here too but
# PRD 0017 chunk 3 moved them behind the
# `list-egress-routes` MCP tool so the agent gets live
# state rather than a launch-time snapshot.)
supervise_dockerfile_path = (
Path(dockerfile_path)
if dockerfile_path
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
)
dockerfile_content = (
supervise_dockerfile_path.read_text()
if supervise_dockerfile_path.is_file()
else ""
)
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = supervise.prepare(
slug, supervise_dir,
dockerfile_content=dockerfile_content,
)
return DockerBottlePlan(
spec=spec,
stage_dir=stage_dir,
guest_home=guest_home,
slug=slug,
container_name=container_name,
container_name_pinned=container_name_pinned,
image=image,
derived_image=derived_image,
runtime_image=runtime_image,
dockerfile_path=dockerfile_path,
env_file=env_file,
forwarded_env=forwarded_env,
prompt_file=prompt_file,
proxy_plan=proxy_plan,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
def _write_env_file(resolved: ResolvedEnv, env_file: Path) -> None:
"""Serialize the literal portion of a ResolvedEnv into docker's
`--env-file` syntax (NAME=VALUE per line, mode 600 since the file
may carry verbatim values from the manifest). Forwarded names ride
on the plan as a structured tuple instead."""
env_lines: list[str] = []
for name, value in resolved.literals.items():
if "\n" in value:
die(
f"env entry {name} (literal) contains a newline; "
f"docker --env-file cannot represent multi-line values."
)
env_lines.append(f"{name}={value}")
env_file.write_text("\n".join(env_lines) + ("\n" if env_lines else ""))
env_file.chmod(0o600)
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
bot_bottle/backend/docker/provision/__init__.py
+5 -6
View File
@@ -2,11 +2,10 @@
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
declarative provision-plan apply, supervise MCP registration) live on
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
provisioning also moved to the AgentProvider ABC (with Debian/node
defaults); user plugins override them for non-standard images.
the `AgentProvider` plugin under `bot_bottle/contrib/`. The modules
left in this subpackage handle only the steps that are
backend-specific:
No modules remain in this subpackage — the directory is kept so that
existing imports of `from .provision import ...` don't need updating
if new backend-specific provisioners are added later.
- ca.py — install per-bottle CA bundle into the guest trust store
- git.py — copy host cwd `.git` into the guest when --cwd is used
"""
bot_bottle/backend/docker/provision/ca.py
+51
View File
@@ -0,0 +1,51 @@
"""Install the per-bottle MITM CA into the agent container's trust
store.
Post-PRD-0017 the CA depends on the agent's HTTP_PROXY target:
- Bottle declares `egress.routes[]` → agent's HTTP_PROXY
points at egress; the cert the agent must trust is the
one egress mints leaf certs with (the egress CA).
- No egress routes → agent's HTTP_PROXY points straight at
pipelock; the cert the agent must trust is pipelock's CA (the
pre-cutover behavior).
By the time this provisioner runs, the corresponding `tls_init`
helper has generated the chosen CA under `plan.stage_dir`, and the
sidecar (pipelock or egress) is up referencing the
in-container CA paths.
Cert lands on Debian's standard source path
(`/usr/local/share/ca-certificates/`); `update-ca-certificates`
rebuilds `/etc/ssl/certs/ca-certificates.crt`, which is what curl,
Python `ssl`, and OpenSSL-based tools all read by default. The env
trio set on the agent's `docker run` covers Node
(`NODE_EXTRA_CA_CERTS`) and Python `requests` /
`SSL_CERT_FILE`-honoring libraries that don't load the system
bundle.
The fingerprint is computed via stdlib (`ssl.PEM_cert_to_DER_cert`
+ `hashlib.sha256`) and logged once to stderr. The private key
stays on the host (under `stage_dir`) until teardown wipes the
stage dir; nothing in the agent ever sees it."""
from __future__ import annotations
from ... import Bottle
from ...util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
from ..bottle_plan import DockerBottlePlan
def provision_ca(plan: DockerBottlePlan, bottle: Bottle) -> None:
"""Copy the agent-facing CA cert into the agent, rebuild the
trust bundle, emit a one-line fingerprint log. Called from
`BottleBackend.provision` after the agent container is up."""
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
bottle.exec(
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
user="root",
)
log_ca_fingerprint(cert_host_path, label)
bot_bottle/backend/docker/provision/git.py
+106
View File
@@ -0,0 +1,106 @@
"""Git provisioning inside a running Docker bottle.
Three concerns, all about git in the agent:
1. If --cwd was passed AND the host cwd has a .git, copy that .git
into the planned guest workspace so the agent operates on the
user's repo.
2. If the bottle declares `git` entries (PRD 0008), write a
~/.gitconfig with insteadOf rules so every git operation
against a declared upstream (push, fetch, clone, pull,
ls-remote) transparently hits the per-agent git-gate. The
gate mirrors the upstream in both directions, so URL
rewriting is symmetric.
3. If the bottle declares `git.user` (issue #86), set
`git config --global user.{name,email}` inside the bottle so
the agent's commits are attributed to that identity.
"""
from __future__ import annotations
import shlex
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
from ....log import info
from ... import Bottle
from ..bottle_plan import DockerBottlePlan
def provision_git(plan: DockerBottlePlan, bottle: Bottle) -> None:
"""Set up git inside the bottle. Runs all three subcases; each
no-ops when its condition isn't met."""
_provision_cwd_git(plan, bottle)
_provision_git_gate_config(plan, bottle)
_provision_git_user(plan, bottle)
def _provision_cwd_git(plan: DockerBottlePlan, bottle: Bottle) -> None:
"""If --cwd was set and the host cwd has a .git directory, copy
it into /home/node/workspace/.git and fix ownership. No-op
otherwise."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
return
guest_workspace_git = f"{workspace.guest_path}/.git"
host_git = str(workspace.host_path / ".git")
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
bottle.cp_in(host_git, guest_workspace_git)
bottle.exec(
f"chown -R {shlex.quote(workspace.owner)} {shlex.quote(guest_workspace_git)}",
user="root",
)
def _provision_git_gate_config(plan: DockerBottlePlan, bottle: Bottle) -> None:
"""Write ~/.gitconfig in the bottle with the git-gate
insteadOf rules. No-op when the bottle has no `git` entries."""
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
if not manifest_bottle.git:
return
container_gitconfig = f"{plan.guest_home}/.gitconfig"
content = git_gate_render_gitconfig(manifest_bottle.git, GIT_GATE_HOSTNAME)
config_file = plan.stage_dir / "agent_gitconfig"
config_file.write_text(content)
config_file.chmod(0o600)
info(f"writing {container_gitconfig} with {len(manifest_bottle.git)} insteadOf rule(s)")
bottle.cp_in(str(config_file), container_gitconfig)
bottle.exec(
f"chown node:node {shlex.quote(container_gitconfig)} && "
f"chmod 644 {shlex.quote(container_gitconfig)}",
user="root",
)
def _provision_git_user(plan: DockerBottlePlan, bottle: Bottle) -> None:
"""Apply `git config --global user.{name,email}` inside the
bottle so the agent's commits are attributed to the operator-
chosen identity instead of the agent image's default
(which is no user — git would refuse to commit at all
until the agent ran its own `git config`).
Runs as the `node` user so `--global` lands in
`/home/node/.gitconfig` (matching the existing
`_provision_git_gate_config` write location). No-op when the
bottle didn't declare `git.user`.
Each field set independently — name-only or email-only
configs only run the `git config` line for the field
present."""
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
gu = manifest_bottle.git_user
if gu.is_empty():
return
if gu.name:
info(f"git config --global user.name = {gu.name!r}")
bottle.exec(
f"git config --global user.name {shlex.quote(gu.name)}",
user="node",
)
if gu.email:
info(f"git config --global user.email = {gu.email!r}")
bottle.exec(
f"git config --global user.email {shlex.quote(gu.email)}",
user="node",
)
bot_bottle/backend/docker/resolve_plan.py
-62
View File
@@ -1,62 +0,0 @@
"""Prepare step for the Docker bottle backend.
`resolve_plan` does all host-side resolution (image and container
names, prompt-file, proxy plan, runtime detection) and returns a
frozen DockerBottlePlan. No Docker resources are created; the only
side effects are scratch files under `stage_dir` and a probe of
`docker info`. Cross-backend host-side validation has already run
via the base class's `prepare` template before this is called.
"""
from __future__ import annotations
from pathlib import Path
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
from .. import BottleSpec
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...manifest import Manifest
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
def preflight() -> None:
docker_mod.require_docker()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> DockerBottlePlan:
"""Resolve Docker-specific names and write scratch files. Trusts
that the agent and its skills/git-gate keys are present —
validation already ran in the base class."""
# ==== docker specific setup ====
use_runsc = docker_mod.runsc_available()
return DockerBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_provision=agent_provision_plan,
)
bot_bottle/backend/docker/sidecar_bundle.py
+6 -5
View File
@@ -2,10 +2,10 @@
(PRD 0024).
The bundle image (built by Dockerfile.sidecars, PRD 0024 chunk 1)
runs egress + git-gate + supervise as one container per bottle
under a small Python init supervisor. As of chunk 5 the bundle
is the only shape — the legacy four-sidecar topology and its
`BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
runs pipelock + egress + git-gate + supervise as one container
per bottle under a small Python init supervisor. As of chunk 5
the bundle is the only shape — the legacy four-sidecar topology
and its `BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
from __future__ import annotations
@@ -14,7 +14,8 @@ import os
# Bundle image. Defaults to a built-locally tag (built from the
# repo's Dockerfile.sidecars via compose `build:`). Operators
# pinning to a published digest can override via env.
# pinning to a published digest can override via env, matching
# the existing `BOT_BOTTLE_PIPELOCK_IMAGE` shape.
SIDECAR_BUNDLE_IMAGE = os.environ.get(
"BOT_BOTTLE_SIDECAR_IMAGE",
"bot-bottle-sidecars:latest",
bot_bottle/backend/docker/util.py
+33 -47
View File
@@ -7,10 +7,11 @@ from __future__ import annotations
import re
import shutil
import subprocess
import tempfile
from typing import Iterable, Iterator
from ...log import die, info
# from ...workspace import WorkspacePlan
from ...workspace import WorkspacePlan
# Cap on the suffix the container-name conflict logic will try before
@@ -117,54 +118,39 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
subprocess.run(args, check=True)
# def build_image_with_cwd(
# derived: str,
# base: str,
# workspace: "WorkspacePlan",
# ) -> None:
# """Build a thin derived image that copies the workspace into
# the plan's guest path and sets the plan's workdir."""
# import os
#
# cwd = str(workspace.host_path)
# if not os.path.isdir(cwd):
# die(f"cwd not found at {cwd}")
# info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
# with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
# context_dir = os.path.join(tmp, "context")
# staged_workspace = os.path.join(context_dir, "workspace")
# shutil.copytree(
# cwd,
# staged_workspace,
# symlinks=True,
# ignore=shutil.ignore_patterns(".git"),
# )
# dockerfile = (
# f"FROM {base}\n"
# f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
# f"WORKDIR {workspace.workdir}\n"
# )
# subprocess.run(
# ["docker", "build", "-t", derived, "-f", "-", context_dir],
# input=dockerfile,
# text=True,
# check=True,
# )
def build_image_with_cwd(
derived: str,
base: str,
workspace: WorkspacePlan,
) -> None:
"""Build a thin derived image that copies the workspace into
the plan's guest path and sets the plan's workdir."""
import os
def commit_container(container_name: str, image_tag: str) -> None:
"""Run `docker commit <container_name> <image_tag>` to snapshot the
running container's filesystem state as a local Docker image."""
result = subprocess.run(
["docker", "commit", container_name, image_tag],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(
f"docker commit {container_name!r} → {image_tag!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
cwd = str(workspace.host_path)
if not os.path.isdir(cwd):
die(f"cwd not found at {cwd}")
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
context_dir = os.path.join(tmp, "context")
staged_workspace = os.path.join(context_dir, "workspace")
shutil.copytree(
cwd,
staged_workspace,
symlinks=True,
ignore=shutil.ignore_patterns(".git"),
)
dockerfile = (
f"FROM {base}\n"
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
f"WORKDIR {workspace.workdir}\n"
)
subprocess.run(
["docker", "build", "-t", derived, "-f", "-", context_dir],
input=dockerfile,
text=True,
check=True,
)
info(f"committed {container_name!r} → {image_tag!r}")
def image_id(ref: str) -> str:
bot_bottle/backend/egress_apply.py
-54
View File
@@ -1,54 +0,0 @@
"""Shared base class for host-side egress apply across backends.
Each backend subclasses EgressApplicator and overrides _signal_bundle_reload
with the backend-specific kill command.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from pathlib import Path
from ..bottle_state import egress_state_dir
from ..egress import EGRESS_ROUTES_FILENAME
from ..egress_addon_core import LOG_OFF, load_config
class EgressApplyError(RuntimeError):
pass
class EgressApplicator(ABC):
def apply_routes_change(self, slug: str, content: str) -> tuple[str, str]:
"""Persist `content` to the live routes file and reload egress."""
self.validate_routes_content(content)
routes_path = self._routes_path(slug)
routes_path.parent.mkdir(parents=True, exist_ok=True)
before = routes_path.read_text(encoding="utf-8") if routes_path.exists() else ""
routes_path.write_text(content, encoding="utf-8")
routes_path.chmod(0o600)
self._signal_bundle_reload(slug)
return before, content
@staticmethod
def validate_routes_content(content: str) -> None:
try:
config = load_config(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
if config.log != LOG_OFF:
raise EgressApplyError(
"proposed routes.yaml must not change egress logging"
)
@staticmethod
def _routes_path(slug: str) -> Path:
return egress_state_dir(slug) / EGRESS_ROUTES_FILENAME
@abstractmethod
def _signal_bundle_reload(self, slug: str) -> None: ...
__all__ = ["EgressApplicator", "EgressApplyError"]
bot_bottle/backend/freeze.py
-100
View File
@@ -1,100 +0,0 @@
"""Freezer — snapshot a running bottle to a resumable artifact.
Follows the same pattern as BottleBackend: a shared base class with
common post-freeze steps (write committed-image path, mark preserved,
print resume hint) and backend-specific subclasses in their respective
backend directories.
Entry points:
Freezer.commit(agent) — freeze by ActiveAgent
Freezer.commit_slug(slug) — convenience wrapper for cmd_commit
get_freezer(backend_name) — factory
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from . import ActiveAgent
from ..bottle_state import mark_preserved, write_committed_image
from ..log import die, info
class CommitCancelled(Exception):
"""Raised by Freezer._freeze when the user declines a confirmation prompt."""
class Freezer(ABC):
"""Freezes a running bottle to a resumable artifact.
The base class owns the shared post-commit steps:
- write_committed_image — records the artifact path in per-bottle state
- mark_preserved — prevents teardown from removing the state dir
- resume hint — printed to stderr after the snapshot
Subclasses implement _freeze with the backend-specific snapshot
operation and optionally override _export_hint for migration hints.
"""
backend_name: str
def commit(self, agent: ActiveAgent) -> None:
"""Freeze the bottle for `agent` to a resumable artifact.
Calls _freeze for the backend-specific snapshot, then writes the
committed image reference to per-bottle state and marks the bottle
preserved so the next `./cli.py resume` boots from the snapshot.
Raises CommitCancelled if the user declines an interactive
confirmation prompt (e.g. the macos-container stop prompt).
"""
image_ref = self._freeze(agent)
write_committed_image(agent.slug, image_ref)
mark_preserved(agent.slug)
info(f"to resume from this snapshot: ./cli.py resume {agent.slug}")
self._export_hint(agent.slug, image_ref)
@abstractmethod
def _freeze(self, agent: ActiveAgent) -> str:
"""Backend-specific snapshot. Returns the image tag or artifact path
stored by write_committed_image. Raises CommitCancelled if the user
declines a stop-confirmation prompt."""
def _export_hint(self, slug: str, image_ref: str) -> None:
"""Optionally print an export-for-migration hint after committing.
Overridden by backends that provide a meaningful export command."""
def commit_slug(self, slug: str) -> None:
"""Convenience entry for cmd_commit when only a slug is available."""
from ..bottle_state import read_metadata
metadata = read_metadata(slug)
agent = ActiveAgent(
backend_name=self.backend_name,
slug=slug,
agent_name=metadata.agent_name if metadata else "",
started_at=metadata.started_at if metadata else "",
services=(),
)
self.commit(agent)
def get_freezer(backend_name: str) -> Freezer:
"""Return the Freezer for the named backend.
backend_name "" is treated as "docker" for backward compatibility
with state dirs written before the backend field was added."""
resolved = backend_name or "docker"
if resolved == "docker":
from .docker.freezer import DockerFreezer
return DockerFreezer()
if resolved == "macos-container":
from .macos_container.freezer import MacosContainerFreezer
return MacosContainerFreezer()
if resolved == "smolmachines":
from .smolmachines.freezer import SmolmachinesFreezer
return SmolmachinesFreezer()
die(
f"commit is only supported for docker, macos-container, and "
f"smolmachines; backend {backend_name!r} has no freezer"
)
raise AssertionError("unreachable")
bot_bottle/backend/macos_container/__init__.py
-10
View File
@@ -1,10 +0,0 @@
"""macOS Apple Container backend.
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
the Apple `container` CLI integration; launch remains gated until the
sidecar network enforcement shape is implemented.
"""
from .backend import MacosContainerBottleBackend
__all__ = ["MacosContainerBottleBackend"]
bot_bottle/backend/macos_container/backend.py
-87
View File
@@ -1,87 +0,0 @@
"""MacosContainerBottleBackend — Apple Container implementation."""
from __future__ import annotations
from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import util as _container
from .bottle import MacosContainerBottle
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
from .bottle_plan import MacosContainerBottlePlan
class MacosContainerBottleBackend(
BottleBackend["MacosContainerBottlePlan", "MacosContainerBottleCleanupPlan"]
):
"""Apple Container backend. Selected by
`BOT_BOTTLE_BACKEND=macos-container` or
`--backend=macos-container`."""
name = "macos-container"
@classmethod
def is_available(cls) -> bool:
return _container.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> MacosContainerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager
def launch(
self, plan: MacosContainerBottlePlan
) -> Generator[MacosContainerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
return _cleanup.prepare_cleanup()
def cleanup(self, plan: MacosContainerBottleCleanupPlan) -> None:
_cleanup.cleanup(plan)
def enumerate_active(self) -> Sequence[ActiveAgent]:
return _enumerate.enumerate_active()
def supervise_mcp_url(self, plan: MacosContainerBottlePlan) -> str:
return plan.agent_supervise_url
bot_bottle/backend/macos_container/bottle.py
-131
View File
@@ -1,131 +0,0 @@
"""Bottle handle for Apple's `container` CLI."""
from __future__ import annotations
import os
import subprocess
import sys
from typing import Callable, cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import pty_forward as _pty_forward
_PTY_FORWARD_SCRIPT = _pty_forward.__file__
_TERMINAL_ENV_NAMES = (
"TERM",
"COLORTERM",
"TERM_PROGRAM",
"TERM_PROGRAM_VERSION",
"KITTY_WINDOW_ID",
"KITTY_PID",
"WEZTERM_PANE",
"WEZTERM_UNIX_SOCKET",
"GHOSTTY_BIN_DIR",
"GHOSTTY_RESOURCES_DIR",
"ITERM_SESSION_ID",
"VTE_VERSION",
"KONSOLE_VERSION",
"ALACRITTY_WINDOW_ID",
)
def _terminal_env_names() -> tuple[str, ...]:
return tuple(
name for name in _TERMINAL_ENV_NAMES
if name == "TERM" or os.environ.get(name)
)
class MacosContainerBottle(Bottle):
def __init__(
self,
container: str,
teardown: Callable[[], None],
prompt_path_in_container: str | None,
*,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
):
self.name = container
self._teardown = teardown
self.prompt_path = prompt_path_in_container
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self._closed = False
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
full_argv = list(argv)
full_argv.extend(
prompt_args(
cast(PromptMode, self._agent_prompt_mode),
self.prompt_path,
argv=full_argv,
)
)
container_exec = ["container", "exec"]
if tty:
container_exec.extend(["--interactive", "--tty"])
# Forward terminal capability hints so TUIs can enable modified-key
# protocols. Use bare env names: values stay in the child env, not
# on argv, and pty_forward supplies a TERM fallback when needed.
for name in _terminal_env_names():
container_exec.extend(["--env", name])
if self.agent_workdir and self.agent_workdir != "/home/node":
container_exec.extend(["--workdir", self.agent_workdir])
container_exec.extend([self.name, self.agent_command, *full_argv])
if tty:
# Wrap with the raw-mode forwarder: container exec does not put
# the host terminal into raw mode itself, so the line discipline
# buffers modifier-key sequences until CR. The wrapper sets raw
# mode before exec and restores it on exit.
return [sys.executable, _PTY_FORWARD_SCRIPT, "--", *container_exec]
return container_exec
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
agent_argv = self.agent_argv(argv, tty=tty)
script = (
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
if tty else None
)
if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
result = subprocess.run(
["container", "exec", "--user", user, "--interactive",
self.name, "sh", "-s"],
input=script,
capture_output=True,
text=True,
check=False,
)
return ExecResult(
returncode=result.returncode,
stdout=result.stdout,
stderr=result.stderr,
)
def cp_in(self, host_path: str, container_path: str) -> None:
subprocess.run(
["container", "cp", host_path, f"{self.name}:{container_path}"],
stdout=subprocess.DEVNULL,
check=True,
)
def close(self) -> None:
if self._closed:
return
self._closed = True
self._teardown()
bot_bottle/backend/macos_container/bottle_cleanup_plan.py
-27
View File
@@ -1,27 +0,0 @@
"""Cleanup plan for the macOS Apple Container backend."""
from __future__ import annotations
from dataclasses import dataclass
from ...log import info
from .. import BottleCleanupPlan
@dataclass(frozen=True)
class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
containers: tuple[str, ...] = ()
networks: tuple[str, ...] = ()
def print(self) -> None:
if not self.containers and not self.networks:
info("macos-container cleanup: nothing to remove")
return
for name in self.containers:
info(f"macos-container container: {name}")
for name in self.networks:
info(f"macos-container network: {name}")
@property
def empty(self) -> bool:
return not self.containers and not self.networks
bot_bottle/backend/macos_container/bottle_plan.py
-58
View File
@@ -1,58 +0,0 @@
"""Plan type for the macOS Apple Container backend."""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from ...agent_provider import PromptMode
from .. import BottlePlan
@dataclass(frozen=True)
class MacosContainerBottlePlan(BottlePlan):
slug: str
forwarded_env: dict[str, str] = field(repr=False)
agent_proxy_url: str = ""
agent_git_gate_url: str = ""
agent_supervise_url: str = ""
@property
def container_name(self) -> str:
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property
def agent_command(self) -> str:
return self.agent_provision.command
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@property
def git_gate_insteadof_host(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
return super().git_gate_insteadof_host
@property
def git_gate_insteadof_scheme(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return "http"
return super().git_gate_insteadof_scheme
bot_bottle/backend/macos_container/cleanup.py
-70
View File
@@ -1,70 +0,0 @@
"""Cleanup for the macOS Apple Container backend."""
from __future__ import annotations
import subprocess
from ...log import info, warn
from . import util as container_mod
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
_PREFIX = "bot-bottle-"
_BUNDLE_PREFIX = "bot-bottle-sidecars-"
def _list_prefixed_containers() -> list[str]:
result = subprocess.run(
["container", "list", "--all", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
warn(f"container list failed: {result.stderr.strip()}")
return []
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX) or name.startswith(_BUNDLE_PREFIX)
)
def _list_prefixed_networks() -> list[str]:
result = subprocess.run(
["container", "network", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX)
)
def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
container_mod.require_container()
return MacosContainerBottleCleanupPlan(
containers=tuple(_list_prefixed_containers()),
networks=tuple(_list_prefixed_networks()),
)
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
for name in plan.containers:
info(f"container delete --force {name}")
subprocess.run(
["container", "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
for name in plan.networks:
info(f"container network delete {name}")
subprocess.run(
["container", "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
bot_bottle/backend/macos_container/egress_apply.py
-39
View File
@@ -1,39 +0,0 @@
"""Host-side egress apply for the macos-container backend.
Uses `container kill --signal HUP` (Apple Container framework) instead
of `docker kill` to signal the sidecar bundle.
"""
from __future__ import annotations
import os
import subprocess
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError
from .launch import sidecar_container_name
class MacOSContainerEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None:
container = sidecar_container_name(slug)
result = subprocess.run(
["container", "kill", "--signal", "HUP", container],
capture_output=True, text=True, check=False, env=os.environ,
)
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'container kill failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {container}: "
f"{last_error or 'container kill failed'}"
)
applicator = MacOSContainerEgressApplicator()
__all__ = ["MacOSContainerEgressApplicator", "EgressApplyError", "applicator"]
bot_bottle/backend/macos_container/enumerate.py
-40
View File
@@ -1,40 +0,0 @@
"""Active-agent enumeration for the macOS Apple Container backend."""
from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent
_PREFIX = "bot-bottle-"
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def enumerate_active() -> list[ActiveAgent]:
result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
out: list[ActiveAgent] = []
for name in sorted(line.strip() for line in result.stdout.splitlines()):
if not name.startswith(_PREFIX):
continue
if name.startswith(_SIDECAR_PREFIX):
continue
slug = name[len(_PREFIX):]
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="macos-container",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=(),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
bot_bottle/backend/macos_container/freezer.py
-31
View File
@@ -1,31 +0,0 @@
"""MacosContainerFreezer — snapshot a macOS container bottle.
Apple Container removes containers when they stop, making stop-then-export
impossible. Instead, commit_container execs into the running container and
streams the root filesystem via tar. The bottle continues running after commit.
"""
from __future__ import annotations
from .. import ActiveAgent
from ..freeze import Freezer
from .util import commit_container
from ...log import info
class MacosContainerFreezer(Freezer):
"""Freezes a macOS-container bottle via exec-tar + image rebuild."""
backend_name = "macos-container"
def _freeze(self, agent: ActiveAgent) -> str:
container = f"bot-bottle-{agent.slug}"
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
commit_container(container, image_tag)
return image_tag
def _export_hint(self, slug: str, image_ref: str) -> None:
info(
f"to export for migration: "
f"container image save {image_ref} -o {slug}.tar"
)
bot_bottle/backend/macos_container/launch.py
-432
View File
@@ -1,432 +0,0 @@
"""Launch flow for the macOS Apple Container backend.
This backend keeps the explicit proxy-env enforcement model for v1:
the agent container is attached only to a host-only Apple Container
network, while the sidecar bundle is attached to a NAT network first
and the host-only network second. The sidecar's host-only IP is
discovered from `container inspect` and stamped into the agent's
HTTP_PROXY / HTTPS_PROXY env vars.
"""
from __future__ import annotations
import dataclasses
import os
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import die, info, warn
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from ..docker.egress import egress_tls_init
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import util as container_mod
from .bottle import MacosContainerBottle
from .bottle_plan import MacosContainerBottlePlan
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_AGENT_SLEEP_SECONDS = "2147483647"
_GIT_HTTP_PORT = 9420
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
def internal_network_name(slug: str) -> str:
return f"bot-bottle-net-{slug}"
def egress_network_name(slug: str) -> str:
return f"bot-bottle-egress-{slug}"
def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
@contextmanager
def launch(
plan: MacosContainerBottlePlan,
*,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]:
"""Build, run, provision, and yield an Apple Container bottle."""
stack = ExitStack()
bottle_for_revoke = plan.manifest.bottle
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"macos-container teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
plan = _mint_certs(plan)
plan = _build_images(plan)
internal_network = internal_network_name(plan.slug)
egress_network = egress_network_name(plan.slug)
_create_networks(internal_network, egress_network, stack)
sidecar_name = sidecar_container_name(plan.slug)
container_mod.force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
stack.callback(container_mod.force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
sidecar_ip = container_mod.container_ipv4_on_network(
sidecar_name, internal_network,
)
plan = _stamp_agent_urls(plan, sidecar_ip)
container_mod.force_remove_container(plan.container_name)
_start_agent(plan, internal_network, sidecar_ip)
stack.callback(container_mod.force_remove_container, plan.container_name)
bottle = MacosContainerBottle(
plan.container_name,
teardown,
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
)
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
container_mod.build_image(
SIDECAR_BUNDLE_IMAGE,
_REPO_DIR,
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
)
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(
plan.agent_provision,
image=committed,
),
)
container_mod.build_image(
plan.image,
_REPO_DIR,
dockerfile=plan.dockerfile_path,
)
return plan
def _create_networks(
internal_network: str,
egress_network: str,
stack: ExitStack,
) -> None:
container_mod.create_network(internal_network, internal=True)
stack.callback(container_mod.remove_network, internal_network)
container_mod.create_network(egress_network)
stack.callback(container_mod.remove_network, egress_network)
def _start_sidecar_bundle(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> None:
argv = _sidecar_run_argv(plan, sidecar_name, internal_network, egress_network)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
env = {**os.environ, **token_values}
info(f"container run sidecar bundle {sidecar_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for sidecar bundle {sidecar_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _start_agent(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> None:
argv = _agent_run_argv(plan, internal_network, sidecar_ip)
env = {
**os.environ,
**plan.forwarded_env,
}
info(f"container run agent {plan.container_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for agent {plan.container_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _stamp_agent_urls(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> MacosContainerBottlePlan:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
supervise_url = ""
if plan.supervise_plan is not None:
supervise_url = f"http://{sidecar_ip}:{SUPERVISE_PORT}/"
git_gate_url = ""
if plan.git_gate_plan.upstreams:
git_gate_url = f"http://{sidecar_ip}:{_GIT_HTTP_PORT}"
return dataclasses.replace(
plan,
agent_proxy_url=proxy_url,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
container_mod.exec_container(
sidecar_name,
[
"mkdir",
"-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER,
"/git",
str(Path(_GIT_GATE_READY_FILE).parent),
],
)
for host_path, container_path in _git_gate_files(plan):
container_mod.copy_into_container(
sidecar_name, host_path, container_path,
)
container_mod.exec_container(
sidecar_name,
[
"sh",
"-c",
"chmod 755 "
f"{GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} "
f"{GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
f"touch {_GIT_GATE_READY_FILE}",
],
)
def _git_gate_files(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str], ...]:
gp = plan.git_gate_plan
files: list[tuple[str, str]] = [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for upstream in gp.upstreams:
files.append((
expand_tilde(upstream.identity_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key",
))
if upstream.known_hosts_file:
files.append((
str(upstream.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts",
))
return tuple(files)
def _sidecar_run_argv(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> list[str]:
argv = [
"container", "run",
"--name", sidecar_name,
"--detach",
"--rm",
"--network", egress_network,
"--network", internal_network,
"--dns", _sidecar_dns(),
"--env", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}",
]
for entry in _sidecar_env_entries(plan):
argv += ["--env", entry]
for host_path, container_path, read_only in _sidecar_mounts(plan):
argv += ["--mount", _mount_spec(host_path, container_path, read_only)]
argv.append(SIDECAR_BUNDLE_IMAGE)
return argv
def _agent_run_argv(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> list[str]:
argv = [
"container", "run",
"--name", plan.container_name,
"--detach",
"--network", internal_network,
]
for entry in _agent_env_entries(plan, sidecar_ip):
argv += ["--env", entry]
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
return argv
def _sidecar_dns() -> str:
return container_mod.dns_server()
def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
daemons = ["egress"]
if plan.git_gate_plan.upstreams:
daemons += ["git-gate", "git-http"]
if plan.supervise_plan is not None:
daemons.append("supervise")
return tuple(daemons)
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
return tuple(env)
def _sidecar_mounts(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str, bool], ...]:
mounts: list[tuple[str, str, bool]] = []
ep = plan.egress_plan
mounts.append((
str(ep.mitmproxy_ca_host_path.parent),
str(Path(EGRESS_CA_IN_CONTAINER).parent),
False,
))
if ep.routes:
mounts.append((
str(ep.routes_path.parent),
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent),
True,
))
sp = plan.supervise_plan
if sp is not None:
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
return tuple(mounts)
def _mount_spec(host_path: str, container_path: str, read_only: bool) -> str:
spec = f"type=bind,source={host_path},target={container_path}"
if read_only:
spec += ",readonly"
return spec
def _agent_env_entries(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> tuple[str, ...]:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
no_proxy = _agent_no_proxy(plan, sidecar_ip)
env = [
f"HTTPS_PROXY={proxy_url}",
f"HTTP_PROXY={proxy_url}",
f"https_proxy={proxy_url}",
f"http_proxy={proxy_url}",
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
if plan.agent_git_gate_url:
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
if plan.agent_supervise_url:
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
return tuple(env)
def _agent_no_proxy(plan: MacosContainerBottlePlan, sidecar_ip: str) -> str:
hosts = ["localhost", "127.0.0.1", sidecar_ip]
return ",".join(hosts)
bot_bottle/backend/macos_container/pty_forward.py
-70
View File
@@ -1,70 +0,0 @@
"""Host-side raw-mode wrapper for `container exec --interactive --tty`.
Apple's `container exec --interactive --tty` does not set the host terminal to
raw mode before starting its I/O relay. Without raw mode the kernel line
discipline buffers modifier-key escape sequences (e.g. Shift+Enter in
modifyOtherKeys mode produces \\x1b[13;2~) until a carriage-return arrives, so
they never reach Claude Code inside the container.
This module sets the host terminal to raw mode, spawns the inner argv (the
container exec command), and restores the original terminal attributes on
exit. When stdin is not a TTY (piped invocations, CI) it falls through to a
bare subprocess.run so callers do not need to special-case non-interactive
contexts.
Usage (the `--` separator is the API contract — everything after it is the
inner command):
python pty_forward.py -- container exec --interactive --tty <name> <cmd>
"""
from __future__ import annotations
import os
import subprocess
import sys
import termios
import tty
def _inner_env() -> dict[str, str]:
env = dict(os.environ)
env.setdefault("TERM", "xterm-256color")
return env
def _run_inner(inner: list[str]) -> int:
return subprocess.run(inner, check=False, env=_inner_env()).returncode
def main(argv: list[str]) -> int:
"""Entry point. ``argv`` shape: ``-- <inner-argv...>``."""
if len(argv) < 2 or argv[0] != "--":
sys.stderr.write(
"usage: python pty_forward.py -- <container-exec-argv...>\n"
)
return 2
inner = argv[1:]
try:
fd = sys.stdin.fileno()
except OSError:
return _run_inner(inner)
if not os.isatty(fd):
return _run_inner(inner)
try:
old = termios.tcgetattr(fd)
except termios.error:
return _run_inner(inner)
try:
tty.setraw(fd)
return _run_inner(inner)
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
bot_bottle/backend/macos_container/resolve_plan.py
-47
View File
@@ -1,47 +0,0 @@
"""Prepare step for the macOS Apple Container backend."""
from __future__ import annotations
from pathlib import Path
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import BottleSpec
from . import util as container_mod
from .bottle_plan import MacosContainerBottlePlan
def preflight() -> None:
container_mod.require_container()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> MacosContainerBottlePlan:
return MacosContainerBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
bot_bottle/backend/macos_container/util.py
-471
View File
@@ -1,471 +0,0 @@
"""Host-side primitives for Apple's `container` CLI."""
from __future__ import annotations
import json
import os
import ipaddress
import platform
import shutil
import subprocess
import tempfile
import time
from typing import Iterable
from ...log import die, info
_CONTAINER = "container"
_DEFAULT_DNS = "1.1.1.1"
def is_macos() -> bool:
return platform.system() == "Darwin"
def is_available() -> bool:
return is_macos() and shutil.which(_CONTAINER) is not None
def require_container() -> None:
"""Fail with an install pointer if Apple Container is unavailable."""
if not is_macos():
info("BOT_BOTTLE_BACKEND=macos-container requires macOS.")
die("macos-container backend is only supported on macOS")
if shutil.which(_CONTAINER) is None:
info("Apple Container is required but was not found on PATH.")
info("Install: https://github.com/apple/container/releases")
die("container not found")
_require_container_service()
def _require_container_service() -> None:
result = subprocess.run(
[_CONTAINER, "system", "status"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode != 0:
info("Apple Container system service is not running.")
info("Start it with: container system start")
die("container system service not running")
def dns_server() -> str:
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
if override:
return override
return _host_ipv4_dns() or _DEFAULT_DNS
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
"""Build an OCI image with Apple's BuildKit-backed `container build`."""
info(
f"building image {ref} from {context} with Apple Container "
"(layer cache keeps repeat builds fast)"
)
_ensure_builder_dns()
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
if dockerfile:
# `container build` resolves -f relative to the current working
# directory, not the build context. Anchor a relative Dockerfile to
# the context so builds work from any cwd.
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args.extend(["-f", dockerfile])
args.append(context)
subprocess.run(args, check=True)
def commit_container(container_name: str, image_tag: str) -> None:
"""Snapshot a running Apple Container as a local image.
`container export` requires a stopped container, but Apple Container
removes containers when they stop, making stop-then-export impossible.
Instead, exec into the running container as root and stream the root
filesystem out via tar, then build a new image from that archive.
The bottle continues running after commit.
"""
with tempfile.TemporaryDirectory(prefix="bot-bottle-container-commit.") as tmp:
rootfs_tar = os.path.join(tmp, "rootfs.tar")
dockerfile = os.path.join(tmp, "Dockerfile")
with open(rootfs_tar, "wb") as tar_out:
result = subprocess.run(
[
_CONTAINER, "exec",
"--user", "root",
container_name,
"tar", "--create",
"--exclude=./proc",
"--exclude=./sys",
"--exclude=./dev",
"--exclude=./run",
"--file=-",
"--directory=/",
".",
],
stdout=tar_out,
stderr=subprocess.PIPE,
check=False,
)
if result.returncode != 0:
die(
f"container exec tar {container_name!r} failed: "
f"{(result.stderr or b'').decode().strip() or '<no stderr>'}"
)
with open(dockerfile, "w", encoding="utf-8") as f:
f.write(
"FROM scratch\n"
"ADD rootfs.tar /\n"
"USER node\n"
"WORKDIR /home/node\n"
)
build_image(image_tag, tmp, dockerfile=dockerfile)
info(f"committed {container_name!r} → {image_tag!r}")
def _ensure_builder_dns() -> None:
dns = dns_server()
status = _builder_status()
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
if _builder_running(status) and _builder_resolves_build_hosts():
if override and not _builder_has_dns(status, dns):
_restart_builder_with_dns(dns)
return
_restart_builder_with_dns(dns)
def _restart_builder_with_dns(dns: str) -> None:
subprocess.run(
[_CONTAINER, "builder", "stop"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
subprocess.run(
[_CONTAINER, "builder", "start", "--dns", dns],
check=True,
)
def _host_ipv4_dns() -> str:
if not is_macos():
return ""
result = subprocess.run(
["scutil", "--dns"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return ""
blocks: list[list[str]] = []
current: list[str] = []
for line in result.stdout.splitlines():
if line.startswith("resolver #") and current:
blocks.append(current)
current = []
current.append(line)
if current:
blocks.append(current)
for direct_only in (True, False):
for block in blocks:
text = "\n".join(block)
if direct_only and "Directly Reachable Address" not in text:
continue
for line in block:
if "nameserver[" not in line or ":" not in line:
continue
candidate = line.split(":", 1)[1].strip()
if _usable_ipv4(candidate):
return candidate
return ""
def _usable_ipv4(value: str) -> bool:
try:
address = ipaddress.ip_address(value)
except ValueError:
return False
return (
address.version == 4
and not address.is_loopback
and not address.is_link_local
and not address.is_multicast
and not address.is_unspecified
)
def _builder_status() -> list[dict[str, object]]:
result = subprocess.run(
[_CONTAINER, "builder", "status", "--format", "json"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError:
return []
if isinstance(data, list):
return [entry for entry in data if isinstance(entry, dict)]
if isinstance(data, dict):
return [data]
return []
def _builder_running(status: list[dict[str, object]]) -> bool:
for entry in status:
entry_status = entry.get("status")
if isinstance(entry_status, dict) and entry_status.get("state") == "running":
return True
return False
def _builder_dns_nameservers(status: list[dict[str, object]]) -> list[str]:
out: list[str] = []
for entry in status:
config = entry.get("configuration")
config_dns = config.get("dns") if isinstance(config, dict) else None
nameservers = (
config_dns.get("nameservers")
if isinstance(config_dns, dict)
else None
)
if not isinstance(nameservers, list):
continue
out.extend(name for name in nameservers if isinstance(name, str))
return out
def _builder_has_dns(status: list[dict[str, object]], dns: str) -> bool:
return dns in _builder_dns_nameservers(status)
def _builder_resolves_build_hosts() -> bool:
result = subprocess.run(
[_CONTAINER, "exec", "buildkit", "getent", "hosts", "deb.debian.org"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
return result.returncode == 0
def image_exists(ref: str) -> bool:
return _silent_run([_CONTAINER, "image", "inspect", ref]) == 0
def container_exists(name: str) -> bool:
result = subprocess.run(
[_CONTAINER, "list", "--all", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return False
return name in {line.strip() for line in result.stdout.splitlines()}
def container_is_running(name: str) -> bool:
"""Return True if the named container is currently running.
`container list` without `--all` lists only running containers."""
result = subprocess.run(
[_CONTAINER, "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return False
return name in {line.strip() for line in result.stdout.splitlines()}
def stop_container(name: str) -> None:
"""Stop the named container without deleting it."""
result = subprocess.run(
[_CONTAINER, "stop", name],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container stop {name!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def force_remove_container(name: str) -> None:
if container_exists(name):
subprocess.run(
[_CONTAINER, "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
def copy_into_container(name: str, host_path: str, container_path: str) -> None:
cmd = [_CONTAINER, "cp", host_path, f"{name}:{container_path}"]
result = _run_container_op(cmd)
if result.returncode != 0:
die(
f"container cp into {name}:{container_path} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def exec_container(name: str, argv: list[str]) -> None:
result = _run_container_op([_CONTAINER, "exec", name, *argv])
if result.returncode != 0:
die(
f"container exec in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]:
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
)
for _ in range(19):
if result.returncode == 0:
return result
time.sleep(0.1)
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
)
return result
def create_network(name: str, *, internal: bool = False) -> None:
args = [
_CONTAINER, "network", "create",
"--label", "bot-bottle.backend=macos-container",
]
if internal:
args.append("--internal")
args.append(name)
result = subprocess.run(
args, capture_output=True, text=True, check=False,
)
if result.returncode == 0:
return
if "already exists" in (result.stderr or "").lower():
return
die(
f"container network create {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def remove_network(name: str) -> None:
result = subprocess.run(
[_CONTAINER, "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode != 0:
return
def inspect_container(name: str) -> dict[str, object]:
result = subprocess.run(
[_CONTAINER, "inspect", name],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container inspect {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError as exc:
die(f"container inspect {name} returned malformed JSON: {exc}")
if isinstance(data, list) and data and isinstance(data[0], dict):
return data[0]
if isinstance(data, dict):
return data
die(f"container inspect {name} returned an unexpected shape")
raise AssertionError("unreachable")
def container_ipv4_on_network(name: str, network: str) -> str:
data = inspect_container(name)
status = data.get("status")
networks = status.get("networks") if isinstance(status, dict) else None
if not isinstance(networks, list):
die(f"container inspect {name} did not include status.networks")
for entry in networks:
if not isinstance(entry, dict):
continue
if entry.get("network") != network:
continue
raw = entry.get("ipv4Address")
if not isinstance(raw, str) or not raw:
die(f"container {name} has no IPv4 address on {network}")
return raw.split("/", 1)[0]
die(f"container {name} is not attached to network {network}")
raise AssertionError("unreachable")
def image_id(ref: str) -> str:
"""Return the image digest/ID from `container image inspect`.
The command returns JSON on current Apple Container releases. Keep
parsing narrow and fatal so callers do not cache on an empty key.
"""
import json
result = subprocess.run(
[_CONTAINER, "image", "inspect", ref],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container image inspect for {ref!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "{}")
except json.JSONDecodeError as exc:
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
if isinstance(data, list) and data:
data = data[0]
if isinstance(data, dict):
value = data.get("id") or data.get("digest") or data.get("ID")
if value:
return str(value)
die(f"container image inspect for {ref!r} did not include an image id")
raise AssertionError("unreachable")
def save(ref: str, output: str) -> None:
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
def _silent_run(cmd: Iterable[str]) -> int:
return subprocess.run(
list(cmd),
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
).returncode
bot_bottle/backend/resolve_common.py
-131
View File
@@ -1,131 +0,0 @@
"""Shared helpers used by both backends' resolve_plan steps.
Each helper owns one well-defined step of the per-bottle plan
resolution so docker and smolmachines don't repeat the same logic.
Backend-specific steps (container names, env-file, per-bottle
Dockerfile overrides, subnet allocation) stay in the backend's own
resolve_plan.py.
"""
from __future__ import annotations
import os
from dataclasses import replace
from datetime import datetime, timezone
from pathlib import Path
from ..agent_provider import AgentProvisionPlan
from ..bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
egress_state_dir,
git_gate_state_dir,
supervise_state_dir,
write_metadata,
)
from ..egress import Egress, EgressPlan
from ..git_gate import GitGate, GitGatePlan
from ..manifest import Manifest, ManifestBottle
from ..supervise import Supervise, SupervisePlan
from . import BottleSpec
def mint_slug(spec: BottleSpec) -> str:
"""Return the bottle identity: the recorded identity for a resume,
or a freshly minted one for a new start.
When a label is provided it becomes the full slug (no random suffix),
so two launches with the same label collide by design. When no label
is given the identity is minted with a random suffix to avoid
collisions between anonymous launches of the same agent."""
if spec.identity:
return spec.identity
if spec.label:
from .docker import util as docker_mod
return docker_mod.slugify(spec.label)
return bottle_identity(spec.agent_name)
def write_launch_metadata(
slug: str, spec: BottleSpec, *, compose_project: str, backend: str,
) -> None:
"""Persist launch metadata so `cli.py resume <identity>` can
reconstruct the spec. Idempotent — re-writes on resume with a
refreshed started_at."""
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=compose_project,
backend=backend,
label=spec.label,
color=spec.color,
))
def prepare_agent_state_dir(slug: str, manifest: Manifest) -> tuple[Path, Path]:
"""Create the agent state subdir, write the prompt file.
Returns (agent_dir, prompt_file)."""
agent = manifest.agent
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
prompt_file = agent_dir / "prompt.txt"
prompt_file.write_text(agent.prompt or "")
prompt_file.chmod(0o600)
return agent_dir, prompt_file
def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
return GitGate().prepare(bottle, slug, git_gate_dir)
def prepare_egress(
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
) -> EgressPlan:
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
"""Prepare the supervise sidecar state dir. Returns None when
bottle.supervise is falsy."""
if not bottle.supervise:
return None
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
return Supervise().prepare(slug, supervise_dir)
def merge_provision_env_vars(provision: AgentProvisionPlan) -> AgentProvisionPlan:
"""Fold provision.env_vars into guest_env (setdefault semantics)
and return a new plan with the merged guest_env."""
merged = dict(provision.guest_env)
for key, val in provision.env_vars.items():
merged.setdefault(key, val)
return replace(provision, guest_env=merged)
def resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
"""Resolve a manifest-supplied dockerfile path relative to user_cwd."""
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
__all__ = [
"merge_provision_env_vars",
"mint_slug",
"prepare_agent_state_dir",
"prepare_egress",
"prepare_git_gate",
"prepare_supervise",
"resolve_manifest_dockerfile",
"write_launch_metadata",
]
bot_bottle/backend/smolmachines/backend.py
+22 -36
View File
@@ -13,21 +13,18 @@ from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import prepare as _prepare
from . import smolvm as _smolvm
from .bottle import SmolmachinesBottle
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
from .bottle_plan import SmolmachinesBottlePlan
from .provision import ca as _ca
from .provision import git as _git
from .provision import workspace as _workspace
class SmolmachinesBottleBackend(
@@ -46,36 +43,10 @@ class SmolmachinesBottleBackend(
runtime check happens at `prepare`."""
return _smolvm.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
self, spec: BottleSpec, *, stage_dir: Path
) -> SmolmachinesBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
@contextmanager
def launch(
@@ -84,6 +55,21 @@ class SmolmachinesBottleBackend(
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def provision_ca(
self, plan: SmolmachinesBottlePlan, bottle: Bottle
) -> None:
_ca.provision_ca(plan, bottle)
def provision_workspace(
self, plan: SmolmachinesBottlePlan, bottle: Bottle
) -> None:
_workspace.provision_workspace(plan, bottle)
def provision_git(
self, plan: SmolmachinesBottlePlan, bottle: Bottle
) -> None:
_git.provision_git(plan, bottle)
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
"""The smolmachines guest reaches the supervise sidecar via a
host-published random port the launch step pinned earlier
bot_bottle/backend/smolmachines/bottle.py
+15 -49
View File
@@ -19,13 +19,10 @@ from __future__ import annotations
import subprocess
import sys
import time
import shlex
from typing import Mapping, cast
from typing import Mapping
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import pty_resize as _pty_resize
from . import smolvm as _smolvm
@@ -70,16 +67,12 @@ class SmolmachinesBottle(Bottle):
guest_env: Mapping[str, str] | None = None,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
) -> None:
self.name = machine_name
# In-VM path to the agent's prompt file. None when the
# agent declared no prompt (file still exists; we just
# don't pass --append-system-prompt-file).
self.prompt_path = prompt_path
self._prompt_path = prompt_path
# Env vars the agent process needs (HTTPS_PROXY,
# CLAUDE_CODE_OAUTH_TOKEN, manifest-declared bottle env, …).
# Forwarded on every `smolvm machine exec` via `-e K=V`
@@ -87,10 +80,9 @@ class SmolmachinesBottle(Bottle):
self._guest_env = dict(guest_env or {})
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self.agent_provider_template = (
"codex" if agent_command == "codex" else "claude"
)
def agent_argv(
self, argv: list[str], *, tty: bool = True,
@@ -98,18 +90,12 @@ class SmolmachinesBottle(Bottle):
flags = ["smolvm", "machine", "exec", "--name", self.name]
if tty:
flags += ["-i", "-t"]
agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
agent_tail += [
"sh", "-lc",
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
"bot-bottle-agent",
]
agent_tail.append(self.agent_command)
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
self.agent_command]
provider_prompt_args = prompt_args(
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
self._agent_prompt_mode, self._prompt_path, argv=argv,
)
if cast(PromptMode, self._agent_prompt_mode) == "read_prompt_file":
if self._agent_prompt_mode == "read_prompt_file":
agent_tail += argv
agent_tail += provider_prompt_args
else:
@@ -141,21 +127,9 @@ class SmolmachinesBottle(Bottle):
UID switches via `runuser -u node --` (not `-l`) so we
avoid login-shell wiring. HOME / USER come from `smolvm
-e` instead, which sets them on the process env."""
agent_argv = self.agent_argv(argv, tty=tty)
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
if script is None:
return subprocess.run(agent_argv, check=False).returncode
# Use sh -c (not -lc) so the script inherits PATH from the calling
# process. sh -l sources login-shell init files (e.g. /etc/profile)
# which may NOT include smolvm's location when it was installed via
# homebrew. The calling process (./cli.py) already has smolvm on PATH
# (provision steps succeed), so -c is sufficient.
return subprocess.run(["sh", "-c", script], check=False).returncode
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
# early-VM provisioning. Retry once after a short settle so
# callers (provision_ca, etc.) don't have to handle it themselves.
_SIGKILL_EXIT = 128 + 9
return subprocess.run(
self.agent_argv(argv, tty=tty), check=False,
).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
"""Run a POSIX shell script as `user` (default `node`) and
@@ -167,22 +141,14 @@ class SmolmachinesBottle(Bottle):
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
without invoking a login shell, then sets HOME / USER and the
bottle env in the child process.
Retries once on SIGKILL (exit 137) — libkrun occasionally
kills short-lived execs during VM bring-up."""
r = self._exec_raw(script, user=user)
if r.returncode == self._SIGKILL_EXIT:
time.sleep(1.0)
r = self._exec_raw(script, user=user)
return r
def _exec_raw(self, script: str, *, user: str = "node") -> ExecResult:
bottle env in the child process."""
argv = [
"--", "runuser", "-u", user, "--",
"env", *_env_assignments_for(user, self._guest_env),
"/bin/sh", "-c", script,
]
# Call smolvm directly because this path needs the host-side
# subprocess capture shape used by the Docker backend.
r = subprocess.run(
["smolvm", "machine", "exec", "--name", self.name] + argv,
capture_output=True, text=True, check=False,
bot_bottle/backend/smolmachines/bottle_plan.py
+28 -36
View File
@@ -12,6 +12,7 @@ from dataclasses import dataclass
from pathlib import Path
from ...agent_provider import PromptMode
from ...pipelock import PipelockProxyPlan
from .. import BottlePlan
@@ -29,6 +30,27 @@ class SmolmachinesBottlePlan(BottlePlan):
bundle_subnet: str
bundle_gateway: str
bundle_ip: str
# smolvm machine name + agent image source. machine_create
# boots from a packed `.smolmachine` artifact (pre-baked at
# prepare time via `smolvm pack create`); using `--from`
# instead of `--image` avoids the registry-pull race we hit
# when machine_start tried to fetch on-demand and the libkrun
# agent's network attempt got refused by macOS.
#
# Chunk 2d ships with a public placeholder image (alpine)
# since bot-bottle-claude:latest lives in the operator's local
# docker daemon and smolvm's crane backend can't read from
# there; chunk 4 resolves the agent-image-conversion gap
# (push to a registry first, or smolvm grows a docker-daemon
# transport).
machine_name: str
# Agent image ref (docker tag). `launch` runs the
# build → save → registry push → smolvm pack pipeline against
# this and feeds the resulting `.smolmachine` artifact to
# `machine_create --from`. The pipeline runs at launch time
# (not prepare time) so the docker build output doesn't garble
# the dashboard's preflight modal.
agent_image_ref: str
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
# the guest has no DNS resolver inside the TSI allowlist.
# Passed to `smolvm machine create` as `-e K=V` flags.
@@ -36,6 +58,11 @@ class SmolmachinesBottlePlan(BottlePlan):
# `--smolfile` is mutually exclusive with `--from`, and
# `--from` is the path that avoids the registry-pull race).
guest_env: dict[str, str]
# Path to the agent's prompt file on the host. Always written
# (mode 0o600) so the in-VM path always exists; the file is
# empty when the agent has no prompt — claude-code reads it
# via --append-system-prompt-file only when non-empty.
prompt_file: Path
# Inner Plans for the sidecar bundle daemons. The same shape the
# docker backend uses — same `.prepare()` calls produced
# them — but our launch step doesn't populate the
@@ -44,6 +71,7 @@ class SmolmachinesBottlePlan(BottlePlan):
# docker's `--internal` + egress bridge topology; it's on a
# per-bottle bridge with a pinned IP. The unused fields stay
# at their dataclass defaults.
proxy_plan: PipelockProxyPlan
# Agent-side endpoints. On Docker Desktop the docker bridge
# IPs aren't reachable from the smolvm guest (TSI uses macOS
# networking; docker container IPs live in the daemon's VM),
@@ -56,42 +84,6 @@ class SmolmachinesBottlePlan(BottlePlan):
agent_git_gate_host: str = ""
agent_supervise_url: str = ""
@property
def machine_name(self) -> str:
"""smolvm machine name. `machine_create` boots from a packed
`.smolmachine` artifact (pre-baked at prepare time via
`smolvm pack create`); using `--from` instead of `--image`
avoids the registry-pull race we hit when machine_start tried
to fetch on-demand and the libkrun agent's network attempt
got refused by macOS."""
return self.agent_provision.instance_name
@property
def agent_image(self) -> str:
"""Agent image ref (docker tag). `launch` runs the
build → save → registry push → smolvm pack pipeline against
this and feeds the resulting `.smolmachine` artifact to
`machine_create --from`. The pipeline runs at launch time
(not prepare time) so the docker build output doesn't garble
the dashboard's preflight modal."""
return self.agent_provision.image
@property
def prompt_file(self) -> Path:
"""Path to the agent's prompt file on the host. Always written
(mode 0o600) so the in-VM path always exists; the file is
empty when the agent has no prompt — claude-code reads it
via --append-system-prompt-file only when non-empty."""
return self.agent_provision.prompt_file
@property
def git_gate_insteadof_host(self) -> str:
return self.agent_git_gate_host
@property
def git_gate_insteadof_scheme(self) -> str:
return "http"
@property
def agent_command(self) -> str:
return self.agent_provision.command
bot_bottle/backend/smolmachines/egress_apply.py
-21
View File
@@ -1,21 +0,0 @@
"""Egress apply for the smolmachines backend.
The smolmachines sidecar bundle runs as a host-side Docker container,
so egress signalling is identical to the docker backend.
"""
from __future__ import annotations
from ..docker.egress_apply import ( # noqa: F401
DockerEgressApplicator,
EgressApplyError,
applicator,
fetch_current_routes,
)
__all__ = [
"DockerEgressApplicator",
"EgressApplyError",
"applicator",
"fetch_current_routes",
]
bot_bottle/backend/smolmachines/enumerate.py
+3 -5
View File
@@ -23,7 +23,7 @@ import json
import subprocess
from .. import ActiveAgent
from ...bottle_state import read_metadata
from ..docker.bottle_state import read_metadata
from . import sidecar_bundle as _bundle
@@ -64,15 +64,13 @@ def enumerate_active() -> list[ActiveAgent]:
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=services_by_slug.get(slug, ()),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
def _query_bundle_services() -> dict[str, tuple[str, ...]]:
"""`{slug: ('egress', ...)}` from each running bundle container's
`BOT_BOTTLE_SIDECAR_DAEMONS` env var.
"""`{slug: ('egress', 'pipelock', ...)}` from each running
bundle container's `BOT_BOTTLE_SIDECAR_DAEMONS` env var.
Smolmachines bundles all run the PRD-0024 image with the
same daemon set declared via env, so one inspect per bundle
gets us the picture without exec'ing into the container.
bot_bottle/backend/smolmachines/freezer.py
-145
View File
@@ -1,145 +0,0 @@
"""SmolmachinesFreezer — snapshot a smolmachines bottle.
`smolvm pack create --from-vm` requires the VM to be stopped, and smolvm
removes VMs when stopped (same issue as Apple Container). Instead, exec
into the running VM as root to write a gzip-compressed tar of the root
filesystem to /var/tmp, then copy it to the host with `smolvm machine cp`,
build a Docker image from the archive, convert it to a smolmachine artifact
via the existing registry pipeline, and record the sidecar path. The VM
stays running throughout."""
from __future__ import annotations
import tempfile
from pathlib import Path
from .. import ActiveAgent
from ..freeze import Freezer
from ..docker import util as docker_mod
from .local_registry import crane_push_tarball, ephemeral_registry
from .smolvm import machine_cp, machine_exec, pack_create
from ...bottle_state import bottle_state_dir
from ...log import die, info
# Temp file written inside the VM during commit. Lives in /var/tmp
# (on-disk, unlike tmpfs /tmp) to survive for machine_cp.
_VM_COMMIT_TAR = "/var/tmp/.bot-bottle-commit.tar.gz"
class SmolmachinesFreezer(Freezer):
"""Freezes a smolmachines bottle via exec-tar + Docker image + smolmachine pack.
The VM is NOT stopped. We exec into the running VM to write a compressed
tar of the root filesystem to /var/tmp, copy it to the host with
machine_cp, build a Docker image (Docker's ADD decompresses .tar.gz
automatically), then run the same image→registry→pack_create pipeline
that _ensure_smolmachine uses for fresh builds."""
backend_name = "smolmachines"
def _freeze(self, agent: ActiveAgent) -> str:
machine = f"bot-bottle-{agent.slug}"
image_ref = f"bot-bottle-committed-{agent.slug}:latest"
output_dir = bottle_state_dir(agent.slug)
output_dir.mkdir(parents=True, exist_ok=True)
binary = output_dir / "committed-smolmachine"
sidecar = output_dir / "committed-smolmachine.smolmachine"
_snapshot_running_vm(machine, image_ref, binary)
return str(sidecar)
def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: cp {image_ref} {slug}.smolmachine")
def _snapshot_running_vm(machine: str, image_ref: str, binary: Path) -> None:
"""Exec-tar the running VM, build a Docker image, and pack to a smolmachine.
binary: destination for the launcher (sibling .smolmachine is the artifact
that machine_create --from consumes, same convention as pack_create).
"""
with tempfile.TemporaryDirectory(prefix="bot-bottle-vm-commit.") as tmp:
tmp_path = Path(tmp)
# Use .tar.gz — Docker ADD decompresses automatically and the
# compressed archive fits in the VM's /var/tmp more easily.
rootfs_tar_gz = tmp_path / "rootfs.tar.gz"
dockerfile = tmp_path / "Dockerfile"
_exec_tar_to_file(machine, rootfs_tar_gz)
dockerfile.write_text(
"FROM scratch\n"
"ADD rootfs.tar.gz /\n"
"USER node\n"
"WORKDIR /home/node\n"
)
docker_mod.build_image(image_ref, str(tmp_path), dockerfile=str(dockerfile))
image_tarball = binary.parent / "committed.image.tar"
docker_mod.save(image_ref, str(image_tarball))
try:
with ephemeral_registry() as handle:
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
push_ref = f"{handle.push_endpoint}/bot-bottle-committed:{digest}"
pack_ref = f"{handle.pull_endpoint}/bot-bottle-committed:{digest}"
crane_push_tarball(handle, str(image_tarball), push_ref)
pack_create(pack_ref, binary)
finally:
image_tarball.unlink(missing_ok=True)
def _exec_tar_to_file(machine: str, dest: Path) -> None:
"""Snapshot the running VM's root filesystem to dest (.tar.gz).
Writes a gzip-compressed tar to _VM_COMMIT_TAR inside the VM via
machine_exec (same mechanism as provisioning), then copies it to the
host with machine_cp. This avoids binary-stdout piping through the
smolvm exec channel, which does not reliably handle large binary output.
A connectivity probe (machine_exec true) runs first so a concurrent-exec
limitation (smolvm may reject a second exec while -i -t is active) is
reported clearly rather than as a silent failure."""
# Connectivity probe — if smolvm rejects concurrent exec while an
# interactive session is running, fail clearly here.
probe = machine_exec(machine, ["true"])
if probe.returncode != 0:
die(
f"smolvm exec is not available for {machine!r} "
f"(exit {probe.returncode}: {probe.stderr.strip() or probe.stdout.strip() or '<no output>'}). "
f"If an interactive session is active, smolvm may not support concurrent exec."
)
# Create the compressed tar inside the VM.
# tar exits 1 when files change during archiving (normal for a live
# filesystem); only treat exit > 1 as fatal.
tar_result = machine_exec(
machine,
[
"tar", "--create", "--gzip",
"--exclude=./proc",
"--exclude=./sys",
"--exclude=./dev",
"--exclude=./run",
# /tmp and /var/tmp are ephemeral. Their stale contents
# (e.g. /tmp/claude-<uid>) have uid remapped by smolvm's
# pack process, causing Claude Code to refuse to use them
# on resume. Exclude both; _init_vm recreates them with
# mkdir -p + correct ownership on every boot.
"--exclude=./tmp",
"--exclude=./var/tmp",
f"--file={_VM_COMMIT_TAR}",
"--directory=/",
".",
],
)
if tar_result.returncode > 1:
die(
f"smolvm exec tar {machine!r} failed (exit {tar_result.returncode}): "
f"{tar_result.stderr.strip() or tar_result.stdout.strip() or '<no output>'}"
)
# Copy from VM to host, then clean up.
try:
machine_cp(f"{machine}:{_VM_COMMIT_TAR}", str(dest))
finally:
machine_exec(machine, ["rm", "-f", _VM_COMMIT_TAR])
bot_bottle/backend/smolmachines/launch.py
+108 -76
View File
@@ -9,9 +9,13 @@ guest pointed at the bundle's pinned IP via TSI's
exit.
The bundle's daemons consume the inner Plans the docker backend
already produces: egress reads routes + CAs from the EgressPlan.
Git-gate + supervise plumb through the same plans the docker
backend uses, minus the docker-network fields that don't apply here."""
already produces: pipelock reads its yaml + CA from the
PipelockProxyPlan; egress reads routes + CAs from the EgressPlan
+ EGRESS_UPSTREAM_PROXY pointing at `127.0.0.1:8888` (bundle
local), since the agent dials pipelock first (not egress) on the
smolmachines path. Git-gate + supervise plumb through the same
plans the docker backend uses, minus the docker-network fields
that don't apply here."""
from __future__ import annotations
@@ -23,15 +27,18 @@ from typing import Callable, Generator
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...pipelock import (
PIPELOCK_CA_CERT_IN_CONTAINER,
PIPELOCK_CA_KEY_IN_CONTAINER,
)
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker import util as docker_mod
from ..docker.egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PIPELOCK_CA_IN_CONTAINER,
EGRESS_PORT as _EGRESS_PORT,
egress_tls_init,
)
@@ -41,13 +48,14 @@ from ..docker.git_gate import (
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import info, warn
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
from ..docker.pipelock import (
BUNDLE_LOCAL_PIPELOCK_URL,
PIPELOCK_PORT as _PIPELOCK_PORT_STR,
pipelock_tls_init,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import warn
from ..docker.bottle_state import git_gate_state_dir
from . import loopback_alias as _loopback
from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm
@@ -70,7 +78,9 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
# Container-internal listening ports for each bundle daemon. The
# bundle publishes each one on a random host loopback port (see
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks
# them up post-start.
# them up post-start. Pipelock's port is an env-overridable string
# in docker.pipelock; coerce to int here.
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
_GIT_HTTP_PORT = 9420
_SUPERVISE_PORT = SUPERVISE_PORT
@@ -79,7 +89,7 @@ _SUPERVISE_PORT = SUPERVISE_PORT
def launch(
plan: SmolmachinesBottlePlan,
*,
provision: Callable[[SmolmachinesBottlePlan, "SmolmachinesBottle"], str | None],
provision: Callable[[SmolmachinesBottlePlan, str], str | None],
) -> Generator[SmolmachinesBottle, None, None]:
"""Build + run the bottle and yield a handle; tear everything
down on exit. Errors during bringup unwind any partial state
@@ -91,7 +101,14 @@ def launch(
plan = _start_bundle(plan, network, loopback_ip, stack)
plan = _discover_urls(plan, loopback_ip)
agent_from_path = _agent_from_path(plan)
# Build the agent image and pack it into a `.smolmachine`
# artifact (or hit the per-Dockerfile-digest cache). Runs
# here, not in prepare, so the docker-build output doesn't
# garble the dashboard's preflight modal.
agent_from_path = _ensure_smolmachine(
plan.agent_image_ref,
dockerfile=plan.agent_dockerfile_path,
)
_launch_vm(plan, agent_from_path, loopback_ip, stack)
_init_vm(plan)
@@ -102,12 +119,8 @@ def launch(
guest_env=plan.guest_env,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
bottle._prompt_path = provision(plan, bottle)
yield bottle
finally:
@@ -126,10 +139,10 @@ def _teardown_smolmachines(
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 — teardown must not fail
except BaseException as exc:
teardown_exc = exc
warn(f"smolmachines teardown failed: {exc!r}")
bottle = plan.manifest.bottle
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
if teardown_exc is not None:
raise teardown_exc
@@ -154,16 +167,33 @@ def _allocate_resources(
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
"""Mint the egress MITM CA and return the plan with CA paths filled."""
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
"""Mint per-bottle CAs and return the plan with CA paths filled.
Pipelock always runs in the bundle. Egress's CA is only minted
when the bottle declares routes — otherwise egress runs idle
without MITM and the CA files would be unused."""
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
proxy_plan = dataclasses.replace(
plan.proxy_plan,
ca_cert_host_path=ca_cert_host,
ca_key_host_path=ca_key_host,
)
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
egress_plan = plan.egress_plan
if egress_plan.routes:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
plan.egress_plan.routes_path.parent,
)
egress_plan = dataclasses.replace(
egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
pipelock_ca_host_path=ca_cert_host,
# On smolmachines, egress's upstream is pipelock on the
# bundle's localhost — they're in the same container's
# network namespace.
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
)
return dataclasses.replace(plan, proxy_plan=proxy_plan, egress_plan=egress_plan)
def _start_bundle(
@@ -194,10 +224,17 @@ def _discover_urls(
macOS networking, and macOS sees the daemon's bridge via the
published-port loopback forward only.
Proxy hop order: when the bottle declares egress routes, the
agent's first hop is egress (for token injection), then
pipelock. Without routes, the agent dials pipelock directly.
NO_PROXY includes the per-bottle loopback alias so the
supervise + git-gate URLs bypass HTTPS_PROXY."""
if plan.egress_plan.routes:
agent_facing_port = _EGRESS_PORT
else:
agent_facing_port = _PIPELOCK_PORT
agent_facing_host_port = _bundle.bundle_host_port(
plan.slug, _EGRESS_PORT, host_ip=loopback_ip,
plan.slug, agent_facing_port, host_ip=loopback_ip,
)
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
@@ -216,23 +253,16 @@ def _discover_urls(
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
no_proxy = f"{existing_no_proxy},{loopback_ip}"
guest_env = {
**plan.guest_env,
"HTTPS_PROXY": agent_proxy_url,
"HTTP_PROXY": agent_proxy_url,
"https_proxy": agent_proxy_url,
"http_proxy": agent_proxy_url,
"NO_PROXY": no_proxy,
"no_proxy": no_proxy,
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
}
if agent_git_gate_host:
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
name, value = entry.split("=", 1)
guest_env[name] = value
return dataclasses.replace(
plan,
@@ -281,16 +311,10 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
All folded into one sh -c to avoid back-to-back exec calls
immediately after machine_start (libkrun exec-channel race).
mkdir -p guards: when booting from a committed snapshot, /tmp and
/var/tmp are excluded from the archive (they're ephemeral and their
stale contents would have wrong uid after smolvm's uid remap). The
directories must be created before chown/chmod can set permissions.
wait_exec_ready polls until the exec channel is ready for the
subsequent provision calls, replacing the empirical sleep."""
_smolvm.machine_exec(plan.machine_name, [
"sh", "-c",
"mkdir -p /tmp /var/tmp && "
"chown -R node:node /home/node && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp",
@@ -304,7 +328,8 @@ def _bundle_launch_spec(
"""Build a BundleLaunchSpec from the resolved inner Plans.
Daemons in the CSV:
- egress is always present.
- egress + pipelock are always present (pipelock is the
agent's first hop; egress is its upstream).
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
- supervise is conditional on plan.supervise_plan.
@@ -312,16 +337,41 @@ def _bundle_launch_spec(
daemon-private values only (HTTPS_PROXY is scoped to the
egress process by egress_entrypoint.sh — see PRD 0024's bundle
bind-address PR)."""
daemons: list[str] = ["egress"]
daemons: list[str] = ["egress", "pipelock"]
env: list[str] = []
volumes: list[tuple[str, str, bool]] = []
# In this Docker-Desktop-compatible topology, whichever daemon
# is "agent-facing" gets its port published on the host
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
# other stays bundle-internal. The bundle is NOT reachable by
# bridge IP from the smolvm guest on macOS — TSI uses macOS
# networking, and macOS sees the daemon's bridge via the
# published-port loopback forward only.
# --- pipelock ---------------------------------------------
pp = plan.proxy_plan
volumes += [
(str(pp.yaml_path), "/etc/pipelock.yaml", True),
(str(pp.ca_cert_host_path), PIPELOCK_CA_CERT_IN_CONTAINER, True),
(str(pp.ca_key_host_path), PIPELOCK_CA_KEY_IN_CONTAINER, True),
]
# --- egress -----------------------------------------------
ep = plan.egress_plan
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
if ep.routes:
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
env.extend(egress_sidecar_env_entries(ep))
env.append(f"EGRESS_UPSTREAM_PROXY={ep.pipelock_proxy_url}")
env.append(f"EGRESS_UPSTREAM_CA={EGRESS_PIPELOCK_CA_IN_CONTAINER}")
volumes += [
(str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True),
(str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True),
(str(ep.pipelock_ca_host_path), EGRESS_PIPELOCK_CA_IN_CONTAINER, True),
]
# Bare-name entries for upstream-token slots. Their values
# come from the docker-run subprocess env (inherited from
# the operator's shell), never landing on argv.
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
# --- git-gate ---------------------------------------------
gp = plan.git_gate_plan
@@ -359,8 +409,14 @@ def _bundle_launch_spec(
# Container ports the agent reaches from the smolvm guest —
# published on host loopback so the guest can dial via TSI +
# macOS networking. Egress is always the agent's HTTP/HTTPS proxy.
ports_to_publish: list[int] = [_EGRESS_PORT]
# macOS networking. The HTTP/HTTPS chokepoint is whichever
# daemon's port we publish: egress when routes are declared
# (token injection first, then forwards to bundle-internal
# pipelock), pipelock otherwise.
if ep.routes:
ports_to_publish: list[int] = [_EGRESS_PORT]
else:
ports_to_publish = [_PIPELOCK_PORT]
if gp.upstreams:
ports_to_publish.append(_GIT_HTTP_PORT)
if sp is not None:
@@ -390,30 +446,6 @@ def _resolve_token_env(
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
"""Return the `.smolmachine` artifact used for `machine create --from`.
Prefer a committed VM artifact when one is recorded and still
present. If the file was removed, fall back to the normal image
build + pack cache path.
"""
committed = read_committed_image(plan.slug)
if committed:
committed_path = Path(committed)
if committed_path.is_file():
info(f"using committed smolmachine {str(committed_path)!r}")
return committed_path
# Build the agent image and pack it into a `.smolmachine`
# artifact (or hit the per-Dockerfile-digest cache). Runs here,
# not in prepare, so the docker-build output doesn't garble the
# dashboard's preflight modal.
return _ensure_smolmachine(
plan.agent_image,
dockerfile=plan.agent_dockerfile_path,
)
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
"""Build the agent docker image and convert it into a
`.smolmachine` artifact, caching the result under
bot_bottle/backend/smolmachines/local_registry.py
+5 -7
View File
@@ -42,13 +42,13 @@ import time
import uuid
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Generator
from typing import Iterator
from ...log import die
# registry:2.8.3, pinned by digest. Same env-override pattern as the
# sidecar image pin in bot_bottle/backend/docker/sidecar_bundle.py.
# pipelock image pin in bot_bottle/backend/docker/pipelock.py.
REGISTRY_IMAGE = os.environ.get(
"BOT_BOTTLE_REGISTRY_IMAGE",
"registry@sha256:a3d8aaa63ed8681a604f1dea0aa03f100d5895b6a58ace528858a7b332415373",
@@ -61,10 +61,7 @@ REGISTRY_IMAGE = os.environ.get(
# narrow.
CRANE_IMAGE = os.environ.get(
"BOT_BOTTLE_CRANE_IMAGE",
(
"gcr.io/go-containerregistry/crane@sha256:"
"0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084"
),
"gcr.io/go-containerregistry/crane@sha256:0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084",
)
@@ -98,7 +95,7 @@ class RegistryHandle:
@contextmanager
def ephemeral_registry() -> Generator[RegistryHandle, None, None]:
def ephemeral_registry() -> Iterator[RegistryHandle]:
"""Bring up a per-session docker network + a `registry:2.8.3`
container on it (published on a random host port), yield a
`RegistryHandle`, force-remove both on exit.
@@ -208,6 +205,7 @@ def _host_port(name: str) -> int:
return int(port_str)
except ValueError:
die(f"unexpected `docker port` output: {line!r}")
return -1 # unreachable; die() never returns
def _wait_ready(port: int) -> None:
bot_bottle/backend/smolmachines/loopback_alias.py
+5 -3
View File
@@ -47,6 +47,7 @@ from __future__ import annotations
import fcntl
import json
import os
import platform
import re
import sqlite3
@@ -176,11 +177,11 @@ def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
con.close()
def allocate(_slug: str) -> str:
def allocate(slug: str) -> str:
"""Pick the lowest-numbered alias from the pool not already
in use by a running smolmachines bundle. Bails when the pool
is exhausted — the caller should report the limit to the
operator. `_slug` is logged for traceability; not otherwise
operator. `slug` is logged for traceability; not otherwise
used (no on-disk reservation, allocation is purely
docker-state-driven).
@@ -195,7 +196,7 @@ def allocate(_slug: str) -> str:
if not _is_macos():
return "127.0.0.1"
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
with open(_ALLOC_LOCK_PATH, "w") as lf:
fcntl.flock(lf, fcntl.LOCK_EX)
return _allocate_locked()
@@ -211,6 +212,7 @@ def _allocate_locked() -> str:
f"Stop a running bottle (`smolvm machine ls --json`) or "
f"raise _POOL_END in loopback_alias.py."
)
return "" # unreachable; die() never returns
def _alias_present(ip: str) -> bool:
bot_bottle/backend/smolmachines/prepare.py
+197
View File
@@ -0,0 +1,197 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup — that's `launch.launch`'s job."""
from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import agent_provision_plan, runtime_for
from ...backend import BottleSpec
from ...backend.docker.bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
egress_state_dir,
git_gate_state_dir,
pipelock_state_dir,
supervise_state_dir,
write_metadata,
)
from ...egress import Egress
from ...env import resolve_env
from ...git_gate import GitGate
from ...pipelock import PipelockProxy
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
# Gateway ports the bundle exposes inside its container — pipelock
# HTTPS proxy, git-gate's git-daemon, supervise's MCP. The agent
# inside the smolvm guest dials these on the bundle's pinned IP.
_BUNDLE_PIPELOCK_PORT = 8888
_BUNDLE_GIT_GATE_PORT = 9418
_BUNDLE_SUPERVISE_PORT = 9100
def resolve_plan(
spec: BottleSpec, *, stage_dir: Path
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
smolmachines_preflight()
manifest = spec.manifest
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = "/home/node"
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
slug = spec.identity or bottle_identity(spec.agent_name)
# Record minimal metadata so `cli.py resume` can recover the
# slug. Same schema as the docker backend.
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project="",
backend="smolmachines",
))
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
resolved = resolve_env(manifest, spec.agent_name)
guest_env: dict[str, str] = {
**resolved.literals,
**resolved.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
# Prompt file is always written (mode 0o600) so the in-VM
# path always exists. Content is the agent's `prompt`
# field (markdown body) — empty for agents with no prompt.
# claude-code reads it via --append-system-prompt-file only
# when non-empty, but the file must exist either way to
# match the docker backend's contract.
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
prompt_file = agent_dir / "prompt.txt"
agent = manifest.agents[spec.agent_name]
prompt_file.write_text(agent.prompt or "")
prompt_file.chmod(0o600)
machine_name = f"bot-bottle-{slug}"
# Stash the agent image ref — `launch.launch` runs the
# build → pack pipeline at bringup. Honors BOT_BOTTLE_IMAGE
# to match the docker backend's `resolve_plan` default.
agent_dockerfile_path = ""
if provider.dockerfile:
agent_dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
image_default = f"bot-bottle-{provider.template}:{slug}"
elif provider_runtime.dockerfile:
agent_dockerfile_path = provider_runtime.dockerfile
image_default = provider_runtime.image
else:
image_default = provider_runtime.image
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
guest_env=guest_env,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
)
merged_guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
merged_guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
# Inner Plans for the four bundle daemons. The ABCs are
# platform-neutral — `.prepare()` writes config files + returns
# a Plan dataclass with no backend-specific assumptions. State
# dirs are still keyed by slug under the docker backend's
# bottle_state layout (shared on-host convention; not a docker
# dependency).
pipelock_dir = pipelock_state_dir(slug)
pipelock_dir.mkdir(parents=True, exist_ok=True)
proxy_plan = PipelockProxy().prepare(
bottle, slug, pipelock_dir, agent_provision.egress_routes,
)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = Egress().prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = Supervise().prepare(slug, supervise_dir)
return SmolmachinesBottlePlan(
spec=spec,
stage_dir=stage_dir,
guest_home=guest_home,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
machine_name=machine_name,
agent_image_ref=agent_image_ref,
guest_env=agent_provision.guest_env,
prompt_file=prompt_file,
proxy_plan=proxy_plan,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
bot_bottle/backend/smolmachines/provision/__init__.py
+6 -6
View File
@@ -2,11 +2,11 @@
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
declarative provision-plan apply, supervise MCP registration) live on
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
provisioning also moved to the AgentProvider ABC (with Debian/node
defaults); user plugins override them for non-standard images.
the `AgentProvider` plugin under `bot_bottle/contrib/`. The modules
left in this subpackage handle only the steps that are
backend-specific:
No modules remain in this subpackage. Workspace copying now runs
through `BottleBackend.provision_workspace` against the running
bottle for every backend.
- ca.py — install per-bottle CA bundle into the guest trust store
- git.py — copy host cwd `.git` into the guest when --cwd is used
- workspace.py — copy the operator workspace into the guest
"""
bot_bottle/backend/smolmachines/provision/ca.py
+93
View File
@@ -0,0 +1,93 @@
"""Install the per-bottle MITM CA into the smolmachines guest's
trust store (PRD 0023 chunk 4d).
Mirrors `backend.docker.provision.ca`: select the right CA (egress
when the bottle has routes, else pipelock), copy it to Debian's
`/usr/local/share/ca-certificates/` path,
`update-ca-certificates` to rebuild the trust bundle, and log the
fingerprint once. The selected cert depends on the agent's
HTTP_PROXY target — same logic as the docker backend, since the
agent dials the same daemons through the same bundle.
`smolvm machine exec` runs commands as root in the VM (no `-u`
flag exists; the VM init is root), so we don't need the explicit
`-u 0` the docker backend uses on its `docker exec` calls."""
from __future__ import annotations
import time
from ....log import die
from ...util import (
AGENT_CA_BUNDLE,
AGENT_CA_PATH,
log_ca_fingerprint,
select_ca_cert,
)
from ... import Bottle, ExecResult
from ..bottle_plan import SmolmachinesBottlePlan
_SIGKILL_EXIT = 128 + 9
def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
"""Copy the agent-facing CA cert into the guest, rebuild the
trust bundle, emit a one-line fingerprint log. Called from
`BottleBackend.provision` after the smolvm guest is up."""
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
# Mode 0644 — readable to non-root tools in the guest.
# update-ca-certificates rebuilds the bundle at AGENT_CA_BUNDLE,
# which is what curl / Python ssl / OpenSSL-based tools read by
# default. The env trio (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE /
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
# `requests` / libraries that don't load the system bundle.
#
r = _install_ca(bottle)
if r.returncode == _SIGKILL_EXIT:
# smolvm/libkrun can SIGKILL an otherwise-normal exec
# during early-VM provisioning. `update-ca-certificates`
# is idempotent, so retry the same install once after a
# short settle delay before treating it as fatal.
time.sleep(1.0)
r = _install_ca(bottle)
if r.returncode != 0:
# update-ca-certificates not adding our cert is fatal —
# claude-code's TLS handshake against the egress-MITM'd
# api.anthropic.com would fail downstream. Bail early
# with what we can see (output is captured so we can
# surface it).
die(
f"update-ca-certificates didn't add the agent CA "
f"(exit {r.returncode}): "
f"stdout={(r.stdout or '').strip()!r} "
f"stderr={(r.stderr or '').strip()!r}"
)
log_ca_fingerprint(cert_host_path, label)
def _install_ca(bottle: Bottle) -> ExecResult:
# chown + chmod + update-ca-certificates + bundle
# verification run in one exec so we only pay one
# round trip; the `&&` chaining surfaces the first failure
# as the return code. The verify check is more stable than
# requiring "1 added" in stdout: a retry after a
# partially-completed first run may legitimately report "0
# added" while the cert is already installed.
return bottle.exec(
f"chown root:root {AGENT_CA_PATH} && "
f"chmod 644 {AGENT_CA_PATH} && "
f"update-ca-certificates && "
f"openssl verify -CAfile {AGENT_CA_BUNDLE} {AGENT_CA_PATH}",
user="root",
)
# Re-exported for the launch/provision_ca caller + tests. The path
# constants live in the shared `backend.util` (Debian's
# `update-ca-certificates` layout is the same in both backends).
__all__ = ["AGENT_CA_BUNDLE", "AGENT_CA_PATH", "provision_ca"]
bot_bottle/backend/smolmachines/provision/git.py
+133
View File
@@ -0,0 +1,133 @@
"""Git provisioning inside a running smolmachines bottle
(PRD 0023 chunk 4d).
Three concerns, all about git in the agent:
1. If --cwd was passed AND the host cwd has a .git, copy that
.git into the planned guest workspace so the agent operates on
the user's repo.
2. If the bottle declares `git` entries (PRD 0008), write a
~/.gitconfig with insteadOf rules so every git operation
against a declared upstream transparently hits the per-bottle
git-gate. The gate mirrors the upstream in both directions,
so URL rewriting is symmetric.
3. If the bottle declares `git.user` (issue #86), set
`git config --global user.{name,email}` inside the guest so
the agent's commits are attributed to that identity.
Differs from `backend.docker.provision.git` in one address detail:
the TSI-allowlisted guest can only reach the bundle's pinned IP
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
are `http://<bundle_ip>:<port>/<name>.git` rather than the
docker backend's `git://git-gate/<name>.git`. The render itself
is the shared `git_gate_render_gitconfig` on the platform-neutral
git_gate module."""
from __future__ import annotations
import os
import shlex
import tempfile
from pathlib import Path
from ....git_gate import git_gate_render_gitconfig
from ....log import info
from ... import Bottle
from ..bottle_plan import SmolmachinesBottlePlan
def provision_git(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
"""Set up git inside the guest. Runs all three subcases; each
no-ops when its condition isn't met."""
_provision_cwd_git(plan, bottle)
_provision_git_gate_config(plan, bottle)
_provision_git_user(plan, bottle)
def _provision_cwd_git(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
"""If --cwd was set and the host cwd has a .git directory, copy
it into <guest_home>/workspace/.git and fix ownership. No-op
otherwise."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
return
guest_workspace_git = f"{workspace.guest_path}/.git"
host_git = str(workspace.host_path / ".git")
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
# mkdir -p the workspace dir so cp_in lands the .git
# directly there even on first-time bottles.
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
bottle.cp_in(host_git, guest_workspace_git)
# cp_in lands files as root; the agent runs as node so
# the workspace tree must be chowned over.
bottle.exec(
f"chown -R {shlex.quote(workspace.owner)} {shlex.quote(guest_workspace_git)}",
user="root",
)
def _provision_git_gate_config(
plan: SmolmachinesBottlePlan, bottle: Bottle
) -> None:
"""Write ~/.gitconfig in the guest with the git-gate insteadOf
rules. No-op when the bottle has no `git` entries."""
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
if not manifest_bottle.git:
return
# `<loopback alias>:<host port>` form: the bundle's git-gate
# HTTP port is published on host loopback at launch time so
# the smolvm guest (which can only reach macOS networking via
# TSI, not the docker bridge IP) can dial it. launch.py
# populates `plan.agent_git_gate_host` after bundle bringup.
content = git_gate_render_gitconfig(
manifest_bottle.git, plan.agent_git_gate_host, scheme="http",
)
guest_gitconfig = f"{plan.guest_home}/.gitconfig"
# Stage the file under the plan's stage_dir so cp_in
# has a stable host path. The plan's stage_dir is cleaned up
# by start.py's session-end teardown.
with tempfile.NamedTemporaryFile(
"w", dir=str(plan.stage_dir), prefix="gitconfig.",
delete=False,
) as f:
f.write(content)
config_file = Path(f.name)
os.chmod(config_file, 0o600)
info(f"writing {guest_gitconfig} with {len(manifest_bottle.git)} insteadOf rule(s)")
bottle.cp_in(str(config_file), guest_gitconfig)
bottle.exec(
f"chown node:node {shlex.quote(guest_gitconfig)} && "
f"chmod 644 {shlex.quote(guest_gitconfig)}",
user="root",
)
def _provision_git_user(
plan: SmolmachinesBottlePlan, bottle: Bottle,
) -> None:
"""Apply `git config --global user.{name,email}` inside the
guest as the node user so --global lands in the same
`/home/node/.gitconfig` that `_provision_git_gate_config`
writes to. No-op when the bottle didn't declare `git.user`.
SmolmachinesBottle.exec(user="node") automatically sets
HOME=/home/node so --global writes to /home/node/.gitconfig."""
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
gu = manifest_bottle.git_user
if gu.is_empty():
return
if gu.name:
info(f"git config --global user.name = {gu.name!r}")
bottle.exec(
f"git config --global user.name {shlex.quote(gu.name)}",
user="node",
)
if gu.email:
info(f"git config --global user.email = {gu.email!r}")
bottle.exec(
f"git config --global user.email {shlex.quote(gu.email)}",
user="node",
)
bot_bottle/backend/smolmachines/provision/workspace.py
+32
View File
@@ -0,0 +1,32 @@
"""Copy the operator workspace into a smolmachines guest."""
from __future__ import annotations
import shlex
from ....log import info
from ... import Bottle
from ..bottle_plan import SmolmachinesBottlePlan
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
"""Copy host cwd contents to the planned guest workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path_q = shlex.quote(workspace.guest_path)
guest_parent_q = shlex.quote(guest_parent)
owner_q = shlex.quote(workspace.owner)
mode_q = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
bottle.exec(
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
user="root",
)
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
bottle.exec(
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
user="root",
)
bot_bottle/backend/smolmachines/pty_resize.py
+3 -5
View File
@@ -42,7 +42,6 @@ import subprocess
import sys
import termios
import threading
from types import FrameType
# How long to wait after the main exec starts before pushing the
@@ -68,9 +67,8 @@ def _read_winsize() -> tuple[int, int] | None:
- tmux respawn-pane: tmux sets all three to the pane's PTY.
- non-TTY (someone piped stdin in tests): none are; the
sync just no-ops, which is the right behavior."""
for stream in (sys.stdin, sys.stdout, sys.stderr):
for fd in (sys.stdin.fileno(), sys.stdout.fileno(), sys.stderr.fileno()):
try:
fd = stream.fileno()
data = fcntl.ioctl(fd, termios.TIOCGWINSZ, b"\x00" * 8)
except OSError:
continue
@@ -125,13 +123,13 @@ def main(argv: list[str]) -> int:
machine = argv[0]
inner = argv[2:]
def sync(_signum: int | None = None, _frame: FrameType | None = None) -> None:
def sync(*_args) -> None:
size = _read_winsize()
if size is None:
return
_push_size(machine, *size)
signal.signal(signal.SIGWINCH, sync) # type: ignore[arg-type]
signal.signal(signal.SIGWINCH, sync)
proc = subprocess.Popen(inner)
# Initial sync is deferred — see _STARTUP_SYNC_DELAY_SEC.
bot_bottle/backend/smolmachines/resolve_plan.py
-83
View File
@@ -1,83 +0,0 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup — that's `launch.launch`'s job."""
from __future__ import annotations
from pathlib import Path
from .. import BottleSpec
from ...manifest import Manifest
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
def preflight() -> None:
smolmachines_preflight()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
return {
**resolved_env.literals,
**resolved_env.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
# ==== smolmachines specific setup ====
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
return SmolmachinesBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
guest_env=agent_provision_plan.guest_env,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
bot_bottle/backend/smolmachines/sidecar_bundle.py
+3 -2
View File
@@ -19,7 +19,7 @@ This module ships the lifecycle primitives only — create
network, start bundle, stop bundle, remove network — wrapped
around `subprocess.run(["docker", ...])`. Wiring them into the
launch flow + populating the `BundleLaunchSpec` from the inner
Plans (EgressPlan, …) lands in chunk 2d."""
Plans (PipelockProxyPlan, EgressPlan, …) lands in chunk 2d."""
from __future__ import annotations
@@ -69,7 +69,7 @@ class BundleLaunchSpec:
# Daemon subset CSV for BOT_BOTTLE_SIDECAR_DAEMONS. The
# supervisor inside the bundle reads it to skip
# bottle-irrelevant daemons (e.g. supervise=False bottles).
daemons_csv: str = "egress"
daemons_csv: str = "egress,pipelock"
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name
# form inherits the value from the docker-run subprocess env,
# matching the docker backend's compose-up secret-forwarding
@@ -223,6 +223,7 @@ def bundle_host_port(
f"no port mapping on {host_ip} for {container} "
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
)
return -1 # unreachable; die() never returns
def stop_bundle(slug: str) -> None:
bot_bottle/backend/smolmachines/smolvm.py
+2 -28
View File
@@ -25,7 +25,6 @@ smolvm binary."""
from __future__ import annotations
import json
import shutil
import subprocess
import time
@@ -53,7 +52,7 @@ class SmolvmError(RuntimeError):
pack failed, etc.). Carries the captured stderr for the
operator-facing log line."""
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess[str]):
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess):
self.argv = list(argv)
self.returncode = result.returncode
self.stdout = result.stdout
@@ -66,7 +65,7 @@ class SmolvmError(RuntimeError):
def _smolvm(*args: str, env: Mapping[str, str] | None = None,
check: bool = True) -> subprocess.CompletedProcess[str]:
check: bool = True) -> subprocess.CompletedProcess:
"""One subprocess call into the smolvm CLI. `check=True`
raises SmolvmError on non-zero; `check=False` returns the
CompletedProcess for the caller to inspect."""
@@ -95,16 +94,6 @@ def pack_create(image: str, output: Path) -> None:
_smolvm("pack", "create", "--image", image, "-o", str(output))
def pack_create_from_vm(name: str, output: Path) -> None:
"""`smolvm pack create --from-vm <name> -o <output>`.
Snapshots an existing persistent VM into a pack artifact. As
with `pack_create`, smolvm writes a launcher at `output` and the
bootable sidecar at `output.smolmachine`.
"""
_smolvm("pack", "create", "--from-vm", name, "-o", str(output))
# --- Machine lifecycle ---------------------------------------------------
@@ -154,21 +143,6 @@ def machine_create(
_smolvm(*args)
def machine_is_running(name: str) -> bool:
"""Return True if the named VM is in the 'running' state."""
result = _smolvm("machine", "ls", "--json", check=False)
if result.returncode != 0:
return False
try:
machines = json.loads(result.stdout or "[]")
except ValueError:
return False
return any(
isinstance(m, dict) and m.get("name") == name and m.get("state") == "running"
for m in machines
)
def machine_start(name: str) -> None:
"""`smolvm machine start --name NAME`."""
_smolvm("machine", "start", "--name", name)
bot_bottle/backend/smolmachines/util.py
+1 -3
View File
@@ -21,9 +21,7 @@ def smolmachines_preflight() -> None:
die(
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
"PATH. Install with: "
"curl -sSL https://smolmachines.com/install.sh | sh. "
"To use the legacy Docker backend instead, set "
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
"curl -sSL https://smolmachines.com/install.sh | sh"
)
bot_bottle/backend/terminal.py
-71
View File
@@ -1,71 +0,0 @@
"""Terminal escape-sequence helpers shared across all bottle backends."""
from __future__ import annotations
import shlex
# color name → (normal_idx, normal_hex, bright_idx, bright_hex, dark_bg_hex)
# OSC 4 sets indexed palette entries (affects syntax-highlighted code and any
# TUI content that uses indexed colors). dark_bg_hex is used for OSC 11
# (default background) — a very dark tint that's visible even when the TUI
# uses true/24-bit colors for its own chrome, which would otherwise bypass
# the palette entirely.
_COLORS: dict[str, tuple[int, str, int, str, str]] = {
"red": (9, "#e74c3c", 1, "#c0392b", "#200808"),
"green": (10, "#2ecc71", 2, "#27ae60", "#082008"),
"yellow": (11, "#f1c40f", 3, "#d4ac0d", "#201808"),
"blue": (12, "#3498db", 4, "#2471a3", "#080820"),
"magenta": (13, "#9b59b6", 5, "#7d3c98", "#160820"),
}
# OSC 104 resets all indexed palette entries; OSC 111 resets default background.
_RESET_PRINTF = "printf '\\033]104\\007\\033]111\\007'"
def palette_printf(color: str) -> str:
"""Shell `printf` command that emits OSC 4 + OSC 11 to tint the terminal
for *color*: sets the normal/bright palette entries AND the default
background to a dark shade of that color. Returns '' if unknown."""
entry = _COLORS.get(color)
if not entry:
return ""
n_idx, n_hex, b_idx, b_hex, bg_hex = entry
seq = (
f"\\033]4;{n_idx};{n_hex}\\007"
f"\\033]4;{b_idx};{b_hex}\\007"
f"\\033]11;{bg_hex}\\007"
)
return f"printf '{seq}'"
def exec_shell_script(
agent_argv: list[str],
terminal_title: str = "",
terminal_color: str = "",
) -> str | None:
"""Build a shell script string that optionally sets the terminal
title and/or palette before running *agent_argv*, and resets the
palette + background on exit. Returns None when no decoration is
needed — callers should run *agent_argv* directly in that case."""
title_cmd = (
f"printf '\\033]0;%s\\007' {shlex.quote(terminal_title)}"
if terminal_title else ""
)
pal_cmd = palette_printf(terminal_color)
if not title_cmd and not pal_cmd:
return None
parts: list[str] = []
if title_cmd:
parts.append(title_cmd)
if pal_cmd:
parts.append(pal_cmd)
parts.append(shlex.join(agent_argv))
parts.append(_RESET_PRINTF)
else:
# No palette change — exec so the agent replaces the shell.
parts.append(f"exec {shlex.join(agent_argv)}")
return "; ".join(parts)
bot_bottle/backend/util.py
+27 -11
View File
@@ -14,6 +14,7 @@ from ..log import die, info
if TYPE_CHECKING:
from ..egress import EgressPlan
from ..pipelock import PipelockProxyPlan
# Debian-family CA layout, shared by every backend (all guest images
@@ -34,20 +35,35 @@ def host_skill_dir(name: str) -> str:
return f"{home}/.claude/skills/{name}"
def select_ca_cert(egress_plan: EgressPlan) -> tuple[Path, str]:
"""Return the egress MITM CA cert path and label for provision_ca.
def select_ca_cert(
egress_plan: EgressPlan, proxy_plan: PipelockProxyPlan
) -> tuple[Path, str]:
"""Pick the agent-facing CA cert (and a short label for the log
line) that matches the proxy the agent's HTTP_PROXY points at.
Egress wins when the bottle declares any routes (it sits in front
of pipelock); else pipelock.
Launch always mints the CA and re-binds the host path into the
egress_plan before provision runs, so an empty/missing path here
means launch's bringup is broken — fatal."""
cert = egress_plan.mitmproxy_ca_cert_only_host_path
if cert == Path() or not cert.is_file():
Shared by every backend's `provision_ca`: launch mints the chosen
CA(s) and re-binds their host paths into these inner plans before
provision runs, so an empty/missing path here means launch's
bringup is broken — fatal."""
if egress_plan.routes:
cert = egress_plan.mitmproxy_ca_cert_only_host_path
if cert == Path() or not cert.is_file():
die(
f"egress CA cert missing at {cert or '(empty)'}; "
f"launch must have called egress_tls_init and "
f"re-bound the plan before provision"
)
return cert, "egress"
cert = proxy_plan.ca_cert_host_path
if not cert or not cert.is_file():
die(
f"egress CA cert missing at {cert or '(empty)'}; "
f"launch must have called egress_tls_init and "
f"re-bound the plan before provision"
f"pipelock CA cert missing at {cert or '(empty)'}; "
f"launch must have called pipelock_tls_init and re-bound "
f"the plan before provision"
)
return cert, "egress"
return cert, "pipelock"
def log_ca_fingerprint(cert_host_path: Path, label: str) -> None:
bot_bottle/cli/__init__.py
+4 -16
View File
@@ -1,6 +1,6 @@
"""Main CLI dispatcher.
Commands: cleanup, commit, edit, info, init, list, resume, start, supervise
Commands: cleanup, edit, info, init, list, resume, start, supervise
"""
from __future__ import annotations
@@ -12,7 +12,6 @@ from ..manifest import ManifestError
from ._common import PROG
from . import list as _list_mod
from .cleanup import cmd_cleanup
from .commit import cmd_commit
from .edit import cmd_edit
from .info import cmd_info
from .init import cmd_init
@@ -24,7 +23,6 @@ cmd_list = _list_mod.cmd_list
COMMANDS = {
"cleanup": cmd_cleanup,
"commit": cmd_commit,
"edit": cmd_edit,
"info": cmd_info,
"init": cmd_init,
@@ -39,23 +37,13 @@ def usage() -> None:
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
sys.stderr.write("Commands:\n")
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
sys.stderr.write(" edit open an agent in vim for editing\n")
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
sys.stderr.write(" list list available agents or active containers\n")
sys.stderr.write(
" resume re-launch a bottle by its identity "
"(continues state from PRD 0016)\n"
)
sys.stderr.write(
" start boot a container for a named agent and "
"attach an interactive session\n"
)
sys.stderr.write(
" supervise view + approve/modify/reject pending supervise "
"proposals (PRD 0013)\n\n"
)
sys.stderr.write(" resume re-launch a bottle by its identity (continues state from PRD 0016)\n")
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n")
sys.stderr.write(" supervise view + approve/modify/reject pending supervise proposals (PRD 0013)\n\n")
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
bot_bottle/cli/_common.py
+1 -1
View File
@@ -14,7 +14,7 @@ REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
def read_tty_line() -> str:
"""Mirror `IFS= read -r REPLY </dev/tty`. Falls back to stdin."""
try:
with open("/dev/tty", "r", encoding="utf-8") as tty:
with open("/dev/tty", "r") as tty:
return tty.readline().rstrip("\n")
except OSError:
return sys.stdin.readline().rstrip("\n")
bot_bottle/cli/commit.py
-53
View File
@@ -1,53 +0,0 @@
"""commit: freeze a running bottle's state to a resumable artifact.
Docker bottles are committed to a local Docker image. Macos-container
bottles are exported and rebuilt as a local Apple Container image.
Smolmachines bottles are packed from the running VM into a
`.smolmachine` artifact. The resulting reference is stored in
per-bottle state so the next `./cli.py resume <slug>` boots from the
snapshot instead of rebuilding from the Dockerfile.
"""
from __future__ import annotations
import argparse
from ..backend import enumerate_active_agents
from ..backend.freeze import CommitCancelled, get_freezer
from ..bottle_state import read_metadata
from ..log import die
from ._common import PROG
from . import tui
def cmd_commit(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} commit", add_help=True)
parser.add_argument(
"slug",
nargs="?",
default=None,
help=(
"bottle slug from `cli.py list active` "
"(omit to pick interactively)"
),
)
args = parser.parse_args(argv)
slug = args.slug
if slug is None:
active = enumerate_active_agents()
if not active:
die("no active bottles; start one with `./cli.py start`")
choices = [a.slug for a in active]
slug = tui.filter_select(choices, title="Select bottle to commit")
if slug is None:
return 0
metadata = read_metadata(slug)
backend = metadata.backend if metadata else ""
try:
get_freezer(backend).commit_slug(slug)
except CommitCancelled:
return 0
return 0
bot_bottle/cli/info.py
+6 -7
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
import argparse
from ..log import info
from ..manifest import ManifestIndex
from ..manifest import Manifest
from ._common import PROG, USER_CWD
@@ -14,12 +14,11 @@ def cmd_info(argv: list[str]) -> int:
parser.add_argument("name", help="agent name defined in bot-bottle.json")
args = parser.parse_args(argv)
names = ManifestIndex.resolve(USER_CWD)
names.require_agent(args.name)
manifest = names.load_for_agent(args.name)
manifest = Manifest.resolve(USER_CWD)
manifest.require_agent(args.name)
agent = manifest.agent
bottle = manifest.bottle
agent = manifest.agents[args.name]
bottle = manifest.bottle_for(args.name)
env_names = list(bottle.env.keys())
prompt_first_line = agent.prompt.splitlines()[0] if agent.prompt else ""
@@ -32,7 +31,7 @@ def cmd_info(argv: list[str]) -> int:
f"first line: {prompt_first_line or '(empty)'}"
)
info(f"bottle : {agent.bottle}")
identity = manifest.git_identity_summary()
identity = manifest.git_identity_summary(args.name)
if identity:
info(f" git identity : {identity}")
if bottle.git:
bot_bottle/cli/init.py
+5 -18
View File
@@ -51,8 +51,7 @@ def cmd_init(argv: list[str]) -> int:
die(f"{target_file} exists but is not valid JSON; fix or remove it first")
if agent_name in (existing.get("agents") or {}):
sys.stderr.write(
f'bot-bottle: agent "{agent_name}" already exists in '
f'{target_file}. Overwrite? [y/N] '
f'bot-bottle: agent "{agent_name}" already exists in {target_file}. Overwrite? [y/N] '
)
sys.stderr.flush()
ow = read_tty_line()
@@ -72,10 +71,7 @@ def cmd_init(argv: list[str]) -> int:
# Prompt
print(file=sys.stderr)
info(
"System prompt — enter text, then a lone '.' on its own line to "
"finish (just '.' to leave empty):"
)
info("System prompt — enter text, then a lone '.' on its own line to finish (just '.' to leave empty):")
prompt_lines: list[str] = []
while True:
line = read_tty_line()
@@ -103,10 +99,7 @@ def cmd_init(argv: list[str]) -> int:
if bottle_name in (existing.get("bottles") or {}):
bottle_exists_already = True
info(
f"Bottle '{bottle_name}' already exists in {target_file}; "
f"agent will reference it."
)
info(f"Bottle '{bottle_name}' already exists in {target_file}; agent will reference it.")
else:
info(f"Creating new bottle '{bottle_name}'.")
bottle_env = _prompt_for_env_vars()
@@ -138,14 +131,8 @@ def cmd_init(argv: list[str]) -> int:
def _prompt_for_env_vars() -> dict[str, str]:
print(file=sys.stderr)
info(
"Env vars — enter each var name then its mode. Press Enter with "
"no name to finish."
)
info(
" Modes: secret (prompt at runtime) | interpolated (read from "
"host env) | literal (hardcoded value)"
)
info("Env vars — enter each var name then its mode. Press Enter with no name to finish.")
info(" Modes: secret (prompt at runtime) | interpolated (read from host env) | literal (hardcoded value)")
out: dict[str, str] = {}
while True:
print(file=sys.stderr)
bot_bottle/cli/list.py
+8 -32
View File
@@ -3,36 +3,12 @@
from __future__ import annotations
import argparse
import os
import sys
from ..backend import enumerate_active_agents
from ..manifest import ManifestIndex
from ..manifest import Manifest
from ._common import PROG, USER_CWD
_ANSI_COLOR_CODES: dict[str, str] = {
"red": "\033[91m",
"green": "\033[92m",
"yellow": "\033[93m",
"blue": "\033[94m",
"magenta": "\033[95m",
}
_ANSI_RESET = "\033[0m"
def _ansi_label(text: str, color: str) -> str:
if not color:
return text
if not sys.stdout.isatty():
return text
term = os.environ.get("TERM", "")
if term in ("dumb", ""):
return text
code = _ANSI_COLOR_CODES.get(color)
if not code:
return text
return f"{code}{text}{_ANSI_RESET}"
def cmd_list(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} list", add_help=True)
@@ -40,8 +16,8 @@ def cmd_list(argv: list[str]) -> int:
args = parser.parse_args(argv)
if args.scope == "available":
manifest = ManifestIndex.resolve(USER_CWD)
for name in manifest.all_agent_names:
manifest = Manifest.resolve(USER_CWD)
for name in manifest.agents.keys():
print(name)
return 0
@@ -51,11 +27,11 @@ def cmd_list(argv: list[str]) -> int:
if not active:
print("no active bot-bottle bottles", file=sys.stderr)
return 0
# One line per bottle: `<backend>\t<slug>\t<label>\t<services>`.
# Tab-separated keeps the format stable for shell pipelines.
# One line per bottle: `<backend>\t<slug>\t<agent>\t<status>`.
# Tab-separated keeps the format stable for shell pipelines;
# the dashboard renders the same data through its own
# formatter.
for b in active:
services = ",".join(b.services) if b.services else "-"
display_name = f"{b.label} ({b.agent_name})" if b.label else b.agent_name
colored_name = _ansi_label(display_name, b.color)
print(f"{b.backend_name}\t{b.slug}\t{colored_name}\t{services}")
print(f"{b.backend_name}\t{b.slug}\t{b.agent_name}\t{services}")
return 0
bot_bottle/cli/resume.py
+5 -3
View File
@@ -18,9 +18,9 @@ from __future__ import annotations
import argparse
from ..backend import BottleSpec
from ..bottle_state import read_metadata
from ..backend.docker.bottle_state import read_metadata
from ..log import die
from ..manifest import ManifestIndex
from ..manifest import Manifest
from ._common import PROG, USER_CWD
from .start import _launch_bottle
@@ -28,6 +28,7 @@ from .start import _launch_bottle
def cmd_resume(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} resume", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"identity",
help="bottle identity from a prior `start` (see its session-end output)",
@@ -41,7 +42,7 @@ def cmd_resume(argv: list[str]) -> int:
f"check ~/.bot-bottle/state/ or run `cli.py start` to create a new bottle"
)
manifest = ManifestIndex.resolve(USER_CWD)
manifest = Manifest.resolve(USER_CWD)
manifest.require_agent(metadata.agent_name)
spec = BottleSpec(
@@ -55,5 +56,6 @@ def cmd_resume(argv: list[str]) -> int:
return _launch_bottle(
spec,
dry_run=args.dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
bot_bottle/cli/start.py
+22 -59
View File
@@ -20,75 +20,52 @@ from ..agent_provider import runtime_for
from ..backend import (
Bottle,
BottleSpec,
enumerate_active_agents,
get_bottle_backend,
known_backend_names,
)
from ..backend.docker import util as docker_mod
from ..backend.docker.bottle_plan import DockerBottlePlan
from ..bottle_state import (
from ..backend.docker.bottle_state import (
cleanup_state,
is_preserved,
mark_preserved,
)
# from ..backend.docker.capability_apply import snapshot_transcript
from ..backend.docker.capability_apply import snapshot_transcript
from ..log import info
from ..manifest import ManifestIndex
from ..manifest import Manifest
from ._common import PROG, USER_CWD, read_tty_line
from . import tui
def cmd_start(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into a derived image")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"--backend",
choices=known_backend_names(),
default=None,
help=(
"backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND "
"or host auto-selection). Overrides the env var when set."
"or 'docker'). Overrides the env var when set."
),
)
parser.add_argument(
"name",
nargs="?",
default=None,
help="agent name defined in bot-bottle.json (omit to pick interactively)",
)
parser.add_argument("name", help="agent name defined in bot-bottle.json")
args = parser.parse_args(argv)
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
manifest = ManifestIndex.resolve(USER_CWD)
agent_name: str | None = args.name
if agent_name is None:
agent_name = tui.filter_select(
manifest.all_agent_names,
title="Select agent",
)
if agent_name is None:
return 0
backend_name: str | None = args.backend
label, color = tui.name_color_modal(default_label=agent_name)
label, color = _resolve_unique_label(label, color)
manifest = Manifest.resolve(USER_CWD)
spec = BottleSpec(
manifest=manifest,
agent_name=agent_name,
agent_name=args.name,
copy_cwd=args.cwd,
user_cwd=USER_CWD,
label=label,
color=color,
)
return _launch_bottle(
spec,
dry_run=dry_run,
backend_name=backend_name,
remote_control=args.remote_control,
backend_name=args.backend,
)
@@ -108,8 +85,8 @@ def prepare_with_preflight(
injected callable, prompt y/N via the injected callable.
`backend_name` selects which backend prepares the plan
(`None` → `$BOT_BOTTLE_BACKEND` → host auto-selection). The CLI
passes whatever `--backend` resolved to.
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). The CLI passes
whatever `--backend` resolved to.
Returns `(plan, identity)`. `plan` is None on dry-run or
operator-N, but `identity` is set as soon as `backend.prepare`
@@ -132,9 +109,8 @@ def prepare_with_preflight(
def attach_agent(
bottle: Bottle, *, resume: bool = False,
bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
agent_provider_template: str = "claude",
startup_args: tuple[str, ...] = (),
) -> int:
"""Run the selected provider CLI inside `bottle` as an
interactive session. Blocks until the session ends; returns the
@@ -151,7 +127,8 @@ def attach_agent(
"(Ctrl-D or 'exit' to leave; container will be removed)"
)
agent_args = list(runtime.bypass_args)
agent_args.extend(startup_args)
if remote_control:
agent_args.extend(runtime.remote_control_args)
if resume:
agent_args.extend(runtime.resume_args)
return bottle.exec_agent(agent_args, tty=True)
@@ -166,7 +143,7 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
# instead of relying on each agent's transcript layout.
if not identity:
return
# snapshot_transcript(identity)
snapshot_transcript(identity)
if exit_code != 0:
mark_preserved(identity)
@@ -190,21 +167,6 @@ def _identity_from_plan(plan: object) -> str:
return getattr(plan, "slug", "")
def _resolve_unique_label(label: str, color: str) -> tuple[str, str]:
"""Re-prompt with a disclaimer until the label's slug is not already
in use among running bottles. Passes through unchanged when no
collision is found on the first check."""
while True:
slug_candidate = docker_mod.slugify(label)
active_slugs = {a.slug for a in enumerate_active_agents()}
if slug_candidate not in active_slugs:
return label, color
label, color = tui.name_color_modal(
default_label=label,
disclaimer=f'"{label}" is already in use',
)
def _text_prompt_yes() -> bool:
"""Default `prompt_yes` for CLI use: reads y/N from the
controlling tty via stderr prompt + tty-line read."""
@@ -214,9 +176,9 @@ def _text_prompt_yes() -> bool:
return reply in ("y", "Y", "yes", "YES")
def _text_render_preflight():
def _text_render_preflight(*, remote_control: bool):
def _render(plan: DockerBottlePlan) -> None:
plan.print()
plan.print(remote_control=remote_control)
return _render
@@ -224,6 +186,7 @@ def _launch_bottle(
spec: BottleSpec,
*,
dry_run: bool,
remote_control: bool,
backend_name: str | None = None,
) -> int:
"""Shared launch core for `start` and `resume`. Builds the plan,
@@ -235,7 +198,7 @@ def _launch_bottle(
plan, identity = prepare_with_preflight(
spec,
stage_dir=stage_dir,
render_preflight=_text_render_preflight(),
render_preflight=_text_render_preflight(remote_control=remote_control),
prompt_yes=_text_prompt_yes,
dry_run=dry_run,
backend_name=backend_name,
@@ -248,8 +211,8 @@ def _launch_bottle(
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
exit_code = attach_agent(
bottle,
remote_control=remote_control,
agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args,
)
info(
f"session ended (exit {exit_code}); "
bot_bottle/cli/supervise.py
+94 -105
View File
@@ -2,9 +2,11 @@
act on them (approve / modify / reject).
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
approval handler wires to PRD 0016 (capability-block), which rebuilds
the bottle Dockerfile. Egress proposals are queued for operator review
as full routes.yaml updates.
approval handlers wire to the per-tool remediation engines:
PRD 0014 (egress, retargeted from cred-proxy in PRD 0017
chunk 3) writes routes.yaml + SIGHUPs egress; PRD 0015
(pipelock) writes the allowlist + restarts pipelock; PRD 0016
(capability) rebuilds the bottle Dockerfile.
"""
from __future__ import annotations
@@ -21,27 +23,20 @@ from datetime import datetime, timezone
from pathlib import Path
from .. import supervise as _supervise
from ..bottle_state import read_metadata
# from ..backend.docker.capability_apply import (
# CapabilityApplyError,
# apply_capability_change,
# )
from ..backend.docker.egress_apply import (
EgressApplyError,
applicator as _docker_applicator,
from ..backend.docker.bottle_state import read_metadata
from ..backend.docker.capability_apply import (
CapabilityApplyError,
apply_capability_change,
)
from ..backend.macos_container.egress_apply import (
applicator as _macos_applicator,
)
from ..backend.smolmachines.egress_apply import (
applicator as _smolmachines_applicator,
from ..backend.docker.egress_apply import EgressApplyError, add_route
from ..backend.docker.pipelock_apply import (
PipelockApplyError,
apply_allowlist_change,
fetch_current_allowlist,
parse_allowlist_content,
render_allowlist_content,
)
from ..log import Die, error, info
class CapabilityApplyError(RuntimeError):
"""Placeholder while capability_apply is disabled."""
from ..supervise import (
COMPONENT_FOR_TOOL,
AuditEntry,
@@ -51,10 +46,8 @@ from ..supervise import (
STATUS_MODIFIED,
STATUS_REJECTED,
TOOL_CAPABILITY_BLOCK,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_PIPELOCK_BLOCK,
archive_proposal,
list_pending_proposals,
render_diff,
@@ -66,11 +59,6 @@ from ._common import PROG
_REFRESH_INTERVAL_MS = 1000
# Proposal tools whose payload is a read-only report, not a file the operator
# edits: modify is unavailable and approval requires a recorded reason for the
# audit trail.
_REPORT_ONLY_TOOLS: tuple[str, ...] = (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW)
@dataclass(frozen=True)
class QueuedProposal:
@@ -83,17 +71,7 @@ class QueuedProposal:
# Errors any remediation engine may raise. Caught by the TUI key
# handlers and surfaced in the status line so a failed apply keeps
# the proposal pending rather than crashing curses.
ApplyError = (CapabilityApplyError, EgressApplyError)
def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
meta = read_metadata(slug)
backend = meta.backend if meta is not None else ""
if backend == "macos-container":
return _macos_applicator.apply_routes_change(slug, content)
if backend == "smolmachines":
return _smolmachines_applicator.apply_routes_change(slug, content)
return _docker_applicator.apply_routes_change(slug, content)
ApplyError = (EgressApplyError, PipelockApplyError, CapabilityApplyError)
def discover_pending() -> list[QueuedProposal]:
@@ -114,7 +92,9 @@ def discover_pending() -> list[QueuedProposal]:
def _approval_status(qp: QueuedProposal, verb: str) -> str:
"""Status-line text after a successful approval."""
base = f"{verb} {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
return f"{base}; resume: ./cli.py resume {qp.proposal.bottle_slug}"
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
return f"{base}; resume: ./cli.py resume {qp.proposal.bottle_slug}"
return base
def _detail_lines(
@@ -136,19 +116,36 @@ def _detail_lines(
out.extend((" " + line, 0) for line in p.justification.splitlines() or [""])
out.extend([
("", 0),
("proposed file:", 0),
(_proposed_payload_label(p.tool) + ":", 0),
])
out.extend((line, 0) for line in p.proposed_file.splitlines() or [""])
if p.tool == TOOL_PIPELOCK_BLOCK:
host = _failed_url_host(p.proposed_file)
if host:
out.append(("", 0))
out.append((host, green_attr))
return out
def _failed_url_host(url: str) -> str:
"""Best-effort hostname extraction from a pipelock-block proposal."""
import urllib.parse
try:
return urllib.parse.urlsplit(url.strip()).hostname or ""
except ValueError:
return ""
def _proposed_payload_label(tool: str) -> str:
if tool == TOOL_PIPELOCK_BLOCK:
return "failed URL"
return "proposed file"
def _suffix_for_tool(tool: str) -> str:
if tool == TOOL_CAPABILITY_BLOCK:
return ".dockerfile"
if tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
return ".yaml"
if tool in (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW):
return ".txt"
return ".txt"
@@ -166,21 +163,24 @@ def approve(
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
diff_before, diff_after = "", ""
# if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
# _meta = read_metadata(qp.proposal.bottle_slug)
# if _meta is not None and not _meta.compose_project:
# raise CapabilityApplyError(
# "capability-block remediation is not supported for smolmachines "
# "bottles. Reject this proposal or handle the capability change "
# "manually, then restart the bottle."
# )
# diff_before, diff_after = apply_capability_change(
# qp.proposal.bottle_slug, file_to_apply,
# )
if qp.proposal.tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
diff_before, diff_after = apply_routes_change(
qp.proposal.bottle_slug,
file_to_apply,
if qp.proposal.tool == TOOL_EGRESS_BLOCK:
diff_before, diff_after = add_route(
qp.proposal.bottle_slug, file_to_apply,
)
elif qp.proposal.tool == TOOL_PIPELOCK_BLOCK:
diff_before, diff_after = _apply_pipelock_url(
qp.proposal.bottle_slug, file_to_apply,
)
elif qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
_meta = read_metadata(qp.proposal.bottle_slug)
if _meta is not None and not _meta.compose_project:
raise CapabilityApplyError(
"capability-block remediation is not supported for smolmachines "
"bottles. Reject this proposal or handle the capability change "
"manually, then restart the bottle."
)
diff_before, diff_after = apply_capability_change(
qp.proposal.bottle_slug, file_to_apply,
)
response = Response(
@@ -210,21 +210,21 @@ def reject(qp: QueuedProposal, *, reason: str) -> None:
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
def _approve_from_tui(
stdscr: "curses._CursesWindow", # type: ignore
qp: QueuedProposal,
*,
final_file: str | None = None,
notes: str = "",
) -> str:
"""Approve from curses, prompting for any tool-specific audit note."""
if qp.proposal.tool in _REPORT_ONLY_TOOLS and final_file is None:
notes = _prompt(stdscr, "allow reason (false positive / legitimately needed): ")
if not notes:
return "approve aborted (empty reason)"
approve(qp, final_file=final_file, notes=notes)
verb = "modified+approved" if final_file is not None else "approved"
return _approval_status(qp, verb)
def _apply_pipelock_url(slug: str, failed_url: str) -> tuple[str, str]:
"""Merge a pipelock-block failed URL's host into the allowlist."""
import urllib.parse
parsed = urllib.parse.urlsplit(failed_url.strip())
host = parsed.hostname or ""
if not host:
raise PipelockApplyError(
f"proposed failed_url has no extractable host: {failed_url!r}"
)
current = fetch_current_allowlist(slug)
hosts = parse_allowlist_content(current)
if host not in hosts:
hosts.append(host)
return apply_allowlist_change(slug, render_allowlist_content(hosts))
def _write_audit(
@@ -235,7 +235,7 @@ def _write_audit(
diff_before: str,
diff_after: str,
) -> None:
"""Audit log for egress tool."""
"""Audit log for egress / pipelock tools."""
component = COMPONENT_FOR_TOOL.get(qp.proposal.tool)
if component is None:
return
@@ -263,7 +263,7 @@ def edit_in_editor(content: str, *, suffix: str = ".tmp") -> str | None:
path = f.name
try:
subprocess.run([editor, path], check=False)
with open(path, encoding="utf-8") as f:
with open(path) as f:
edited = f.read()
return edited if edited != content else None
finally:
@@ -296,12 +296,9 @@ def cmd_supervise(argv: list[str]) -> int:
else:
error("supervise exited on a fatal error (no detail captured).")
return e.code if isinstance(e.code, int) else 1
except Exception as e: # noqa: W0718 — catch supervise crash for logging
except Exception as e:
log_path = _write_crash_log(e)
error(
f"supervise crashed: {type(e).__name__}: {e}",
context={"error_type": type(e).__name__, "crash_log": str(log_path)},
)
error(f"supervise crashed: {type(e).__name__}: {e}")
error(f"full traceback written to {log_path}")
return 1
return 0
@@ -357,7 +354,7 @@ def _try_init_green() -> int:
return 0
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
def _main_loop(stdscr: "curses._CursesWindow") -> None:
curses.curs_set(0)
stdscr.timeout(_REFRESH_INTERVAL_MS)
green_attr = _try_init_green()
@@ -413,22 +410,18 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
_detail_view(stdscr, qp, green_attr=green_attr)
elif key == ord("a"):
try:
status_line = _approve_from_tui(stdscr, qp)
approve(qp)
status_line = _approval_status(qp, "approved")
except ApplyError as e:
status_line = f"apply failed: {e}"
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
status_line = f"modify unavailable for {qp.proposal.tool}"
continue
edited = _modify(stdscr, qp)
if edited is None:
status_line = "modify aborted (no change)"
else:
try:
status_line = _approve_from_tui(
stdscr, qp, final_file=edited,
notes="operator modified before approving",
)
approve(qp, final_file=edited, notes="operator modified before approving")
status_line = _approval_status(qp, "modified+approved")
except ApplyError as e:
status_line = f"apply failed: {e}"
elif key == ord("r"):
@@ -441,12 +434,12 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
def _render(
stdscr: "curses._CursesWindow", # type: ignore
stdscr: "curses._CursesWindow",
pending: list[QueuedProposal],
selected: int,
status_line: str,
*,
green_attr: int = 0, # noqa: F841 — unused, but required by interface
green_attr: int = 0,
) -> None:
stdscr.erase()
h, w = stdscr.getmaxyx()
@@ -474,7 +467,8 @@ def _render(
cursor = "> " if i == selected else " "
line = (
f"{cursor}{ts_short} "
f"[{p.bottle_slug}] {p.tool:<18} {p.id[:8]}"
f"[{p.bottle_slug}] {p.tool:<18} {p.id[:8]} "
f"{_proposed_payload_label(p.tool)}"
)
attr = curses.A_REVERSE if i == selected else curses.A_NORMAL
stdscr.addnstr(row, 0, line, w - 1, attr)
@@ -494,7 +488,7 @@ def _render(
def _detail_view(
stdscr: "curses._CursesWindow", # type: ignore
stdscr: "curses._CursesWindow",
qp: QueuedProposal,
*,
green_attr: int = 0,
@@ -526,20 +520,15 @@ def _detail_view(
offset = max(0, len(lines) - 1)
elif key == ord("a"):
try:
_approve_from_tui(stdscr, qp)
approve(qp)
except ApplyError:
pass
return
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
return
edited = _modify(stdscr, qp)
if edited is not None:
try:
_approve_from_tui(
stdscr, qp, final_file=edited,
notes="operator modified before approving",
)
approve(qp, final_file=edited, notes="operator modified before approving")
except ApplyError:
pass
return
@@ -550,7 +539,7 @@ def _detail_view(
return
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
suffix = _suffix_for_tool(qp.proposal.tool)
curses.endwin()
@@ -561,7 +550,7 @@ def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
return edited
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str:
"""One-line input at the bottom of the screen."""
curses.curs_set(1)
h, _ = stdscr.getmaxyx()
bot_bottle/cli/tui.py
-437
View File
@@ -1,437 +0,0 @@
"""tui.py — minimal curses filter-select picker for CLI prompts.
Exposed surface:
filter_select(items, *, title="", tty_path="/dev/tty") -> str | None
name_color_modal(default_label, *, tty_path="/dev/tty") -> (str, str)
Opens /dev/tty directly so the picker works even when stdout/stdin are
redirected. Returns the selected item or None on cancel.
"""
from __future__ import annotations
import curses
import os
import sys
from typing import Any, Optional
def filter_select(
items: list[str],
*,
title: str = "",
tty_path: str = "/dev/tty",
) -> Optional[str]:
"""Render a filter-select picker over *items*.
Returns the selected item string, or ``None`` if the user cancelled
(Esc / ``q`` / Ctrl-C / Ctrl-D) or if the terminal is too small.
The picker opens *tty_path* directly so it works even when
stdout/stdin are redirected.
"""
if not items:
return None
try:
tty_fd = open(tty_path, "r+b", buffering=0)
except OSError:
return None
try:
# Use os.dup() to duplicate the fd so the original file object
# and FileIO in _run_picker each manage independent copies,
# preventing double-close errors.
fd_dup = os.dup(tty_fd.fileno())
return _run_picker(items, title=title, tty_fd=fd_dup)
finally:
tty_fd.close()
# ---------------------------------------------------------------------------
# Internal implementation
# ---------------------------------------------------------------------------
_KEY_ESC = 27
_KEY_CTRL_C = 3
_KEY_CTRL_D = 4
_KEY_BACKSPACE_WIN = 8
_KEY_ENTER_ALT = 10
_CANCEL_KEYS = frozenset([_KEY_ESC, _KEY_CTRL_C, _KEY_CTRL_D, ord("q")])
def _run_picker(items: list[str], *, title: str, tty_fd: int) -> Optional[str]:
"""Drive a curses session on *tty_fd* and return the picked item."""
# newterm lets us run curses on an arbitrary fd rather than the
# process's controlling tty / stdout — crucial when stdout is piped.
os.environ.setdefault("TERM", "xterm-256color")
# Save / restore the real stdin/stdout so curses newterm can use tty_fd.
orig_stdin = sys.__stdin__
orig_stdout = sys.__stdout__
try:
import io
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode='r+'), write_through=True)
sys.__stdin__ = tty_text # type: ignore[assignment]
sys.__stdout__ = tty_text # type: ignore[assignment]
# curses.wrapper calls initscr which honours sys.__stdin__ / __stdout__
# on some builds; use newterm where available.
screen = curses.initscr()
curses.noecho()
curses.cbreak()
screen.keypad(True)
try:
result = _picker_loop(screen, items, title=title)
finally:
screen.keypad(False)
curses.nocbreak()
curses.echo()
curses.endwin()
except Exception: # noqa: W0718 — curses can raise many error types
return None
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
sys.__stdout__ = orig_stdout # type: ignore[assignment]
return result
def _picker_loop(screen: Any, items: list[str], *, title: str) -> Optional[str]:
query = ""
cursor = 0
while True:
filtered = _filter_items(items, query)
# Clamp cursor into the visible list.
if not filtered:
cursor = 0
elif cursor >= len(filtered):
cursor = len(filtered) - 1
try:
_render(screen, filtered, cursor, query=query, title=title)
except curses.error:
# Terminal too small or write error — bail out.
return None
try:
key = screen.getch()
except KeyboardInterrupt:
return None
if key in _CANCEL_KEYS:
return None
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
return filtered[cursor] if filtered else None
if key in (curses.KEY_UP, ord("k")):
if cursor > 0:
cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")):
if cursor < len(filtered) - 1:
cursor += 1
elif key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
query = query[:-1]
# After narrowing the filter, keep cursor in range.
new_filtered = _filter_items(items, query)
if cursor >= len(new_filtered):
cursor = max(0, len(new_filtered) - 1)
elif 32 <= key <= 126:
# Printable ASCII — append to query and reset cursor so the
# top of the newly-filtered list is selected.
query += chr(key)
cursor = 0
def _filter_items(items: list[str], query: str) -> list[str]:
if not query:
return list(items)
q = query.lower()
return [i for i in items if q in i.lower()]
def _render(screen: Any, filtered: list[str], cursor: int, *, query: str, title: str) -> None:
screen.erase()
rows, cols = screen.getmaxyx()
min_rows = 5
if rows < min_rows:
raise curses.error("terminal too small")
row = 0
if title and row < rows - 1:
_addstr_safe(screen, row, 0, title[:cols - 1], curses.A_BOLD)
row += 1
filter_label = f"Filter: {query}"
if row < rows - 1:
_addstr_safe(screen, row, 0, filter_label[:cols - 1])
row += 1
sep = "─" * min(cols - 1, 40)
if row < rows - 1:
_addstr_safe(screen, row, 0, sep)
row += 1
list_start = row
# Reserve two rows for separator + help line at bottom.
list_rows = rows - list_start - 2
if list_rows < 1:
return
# Scroll window: keep cursor visible.
scroll = max(0, cursor - list_rows + 1)
visible = filtered[scroll: scroll + list_rows]
for idx, item in enumerate(visible):
abs_idx = scroll + idx
attr = curses.A_REVERSE if abs_idx == cursor else curses.A_NORMAL
prefix = "> " if abs_idx == cursor else " "
line = (prefix + item)[:cols - 1]
if row < rows - 1:
_addstr_safe(screen, row, 0, line, attr)
row += 1
if row < rows - 1:
_addstr_safe(screen, row, 0, sep)
row += 1
help_line = "[↑↓/jk] move [Enter] select [Esc/q] cancel"
if row < rows:
_addstr_safe(screen, min(rows - 1, row), 0, help_line[:cols - 1])
screen.refresh()
def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.A_NORMAL) -> None:
try:
screen.addstr(row, col, text, attr)
except curses.error:
pass
# ---------------------------------------------------------------------------
# name_color_modal — two-step label + color picker
# ---------------------------------------------------------------------------
_ANSI_COLORS = [
"red", "green", "yellow", "blue", "magenta",
]
_CURSES_COLOR_MAP: dict[str, int] = {
"red": curses.COLOR_RED,
"green": curses.COLOR_GREEN,
"yellow": curses.COLOR_YELLOW,
"blue": curses.COLOR_BLUE,
"magenta": curses.COLOR_MAGENTA,
}
_COLOR_NONE = "(none)"
def name_color_modal(
default_label: str,
*,
disclaimer: str = "",
tty_path: str = "/dev/tty",
) -> tuple[str, str]:
"""Present a two-step curses modal: first edit the agent label,
then optionally pick a color.
``disclaimer`` is shown below the input field — use it to surface
an error from a previous attempt (e.g. name already in use).
Returns ``(label, color)`` where ``color`` is one of the 16 ANSI
color name strings or ``""`` for no color. Falls back to
``(default_label, "")`` on any error (terminal too small, not a tty).
"""
try:
tty_fd = open(tty_path, "r+b", buffering=0) # pylint: disable=consider-using-with
except OSError:
return default_label, ""
try:
fd_dup = os.dup(tty_fd.fileno())
return _run_name_color(default_label, tty_fd=fd_dup, disclaimer=disclaimer)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return default_label, ""
finally:
tty_fd.close()
def _run_name_color(default_label: str, *, tty_fd: int, disclaimer: str = "") -> tuple[str, str]:
import io
orig_stdin = sys.__stdin__
orig_stdout = sys.__stdout__
try:
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode="r+"), write_through=True)
sys.__stdin__ = tty_text # type: ignore[assignment]
sys.__stdout__ = tty_text # type: ignore[assignment]
os.environ.setdefault("TERM", "xterm-256color")
screen = curses.initscr()
curses.noecho()
curses.cbreak()
screen.keypad(True)
try:
label = _label_step(screen, default_label, disclaimer=disclaimer)
color = _color_step(screen, label)
finally:
screen.keypad(False)
curses.nocbreak()
curses.echo()
curses.endwin()
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
sys.__stdout__ = orig_stdout # type: ignore[assignment]
return label, color
def _label_step(screen: Any, default_label: str, *, disclaimer: str = "") -> str:
"""Step 1: edit the label. First printable key replaces the
pre-fill; subsequent keys append. Enter confirms."""
text = default_label
replaced = False # True once the user has typed their first char
while True:
_render_label(screen, text, disclaimer=disclaimer)
try:
key = screen.getch()
except KeyboardInterrupt:
return default_label
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
return text.strip() or default_label
if key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
if replaced:
text = text[:-1]
else:
text = ""
replaced = True
elif 32 <= key <= 126:
if not replaced:
text = chr(key)
replaced = True
else:
text += chr(key)
def _render_label(screen: Any, text: str, *, disclaimer: str = "") -> None:
screen.erase()
rows, cols = screen.getmaxyx()
sep = "─" * min(cols - 1, 40)
_addstr_safe(screen, 0, 0, "Name agent", curses.A_BOLD)
_addstr_safe(screen, 1, 0, sep)
_addstr_safe(screen, 2, 0, text[:cols - 1], curses.A_REVERSE)
_addstr_safe(screen, 3, 0, sep)
row = 4
if disclaimer and rows > row + 1:
_addstr_safe(screen, row, 0, disclaimer[:cols - 1], curses.A_BOLD)
row += 1
if rows > row + 1:
_addstr_safe(screen, row, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
screen.refresh()
def _color_step(screen: Any, confirmed_label: str) -> str:
"""Step 2: pick a color from the list, or skip."""
items = [_COLOR_NONE] + _ANSI_COLORS
cursor = 0
# Initialise color pairs once; index 0 = none, 1..16 = palette.
color_attrs = _init_color_pairs()
while True:
_render_color(screen, items, cursor, confirmed_label, color_attrs)
try:
key = screen.getch()
except KeyboardInterrupt:
return ""
if key in (ord("q"), _KEY_ESC):
return ""
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
chosen = items[cursor]
return "" if chosen == _COLOR_NONE else chosen
if key in (curses.KEY_UP, ord("k")) and cursor > 0:
cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")) and cursor < len(items) - 1:
cursor += 1
def _init_color_pairs() -> dict[str, int]:
"""Return {color_name: curses_attr} for the palette items."""
attrs: dict[str, int] = {_COLOR_NONE: curses.A_NORMAL}
try:
curses.start_color()
curses.use_default_colors()
pair_idx = 2 # pair 1 reserved for other uses
for name in _ANSI_COLORS:
fg = _CURSES_COLOR_MAP.get(name, curses.COLOR_WHITE)
try:
curses.init_pair(pair_idx, fg, -1)
attr = curses.color_pair(pair_idx) | curses.A_BOLD
attrs[name] = attr
pair_idx += 1
except curses.error:
attrs[name] = curses.A_NORMAL
except curses.error:
for name in _ANSI_COLORS:
attrs[name] = curses.A_NORMAL
return attrs
def _render_color(
screen: Any,
items: list[str],
cursor: int,
confirmed_label: str,
color_attrs: dict[str, int],
) -> None:
screen.erase()
rows, cols = screen.getmaxyx()
sep = "─" * min(cols - 1, 40)
_addstr_safe(screen, 0, 0, "Name agent", curses.A_BOLD)
_addstr_safe(screen, 1, 0, sep)
_addstr_safe(screen, 2, 0, confirmed_label[:cols - 1])
_addstr_safe(screen, 3, 0, sep)
_addstr_safe(screen, 4, 0, "Color (optional)", curses.A_BOLD)
list_start = 5
list_rows = rows - list_start - 2
scroll = max(0, cursor - list_rows + 1)
visible = items[scroll: scroll + list_rows]
for idx, name in enumerate(visible):
abs_idx = scroll + idx
row = list_start + idx
if row >= rows - 2:
break
prefix = "> " if abs_idx == cursor else " "
attr = color_attrs.get(name, curses.A_NORMAL)
if abs_idx == cursor:
attr |= curses.A_REVERSE
_addstr_safe(screen, row, 0, (prefix + name)[:cols - 1], attr)
_addstr_safe(screen, rows - 2, 0, sep)
_addstr_safe(
screen, rows - 1, 0,
"[↑↓/jk] move [Enter] select [Esc/q] skip",
curses.A_DIM,
)
screen.refresh()
bot_bottle/contrib/codex/codex_auth.py → bot_bottle/codex_auth.py
+21 -24
View File
@@ -13,10 +13,9 @@ import os
from copy import deepcopy
from datetime import datetime, timezone
from pathlib import Path
from typing import cast
from ...log import die
from ...util import expand_tilde
from .log import die
from .util import expand_tilde
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
@@ -51,8 +50,7 @@ def codex_host_access_token(
tokens = raw.get("tokens")
if not isinstance(tokens, dict):
die(f"codex host credentials: {path} is missing tokens")
tokens_typed = cast(dict[str, object], tokens)
access = tokens_typed.get("access_token")
access = tokens.get("access_token")
if not isinstance(access, str) or not access:
die(
f"codex host credentials: {path} is missing tokens.access_token. "
@@ -107,14 +105,14 @@ def write_codex_dummy_auth_file(
path.chmod(0o600)
def _read_auth_object(path: Path) -> dict[str, object]:
def _read_auth_object(path: Path) -> dict:
try:
raw = json.loads(path.read_text())
except (OSError, json.JSONDecodeError) as e:
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
if not isinstance(raw, dict):
die(f"codex host credentials: {path} must contain a JSON object")
return cast(dict[str, object], raw)
return raw
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
@@ -153,11 +151,11 @@ def _dummy_jwt_from_host(
return _dummy_jwt(now, exp_ts=exp_ts)
if not isinstance(payload, dict):
return _dummy_jwt(now, exp_ts=exp_ts)
return _encode_dummy_jwt(_redact_jwt_payload(cast(dict[str, object], payload), now=now, exp_ts=exp_ts))
return _encode_dummy_jwt(_redact_jwt_payload(payload, now=now, exp_ts=exp_ts))
def _encode_dummy_jwt(payload: dict[str, object]) -> str:
def enc(obj: dict[str, object]) -> str:
def _encode_dummy_jwt(payload: dict) -> str:
def enc(obj: dict) -> str:
raw = json.dumps(obj, separators=(",", ":")).encode()
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
@@ -165,24 +163,23 @@ def _encode_dummy_jwt(payload: dict[str, object]) -> str:
def _redact_jwt_payload(
payload: dict[str, object],
payload: dict,
*,
now: datetime | None = None,
exp_ts: int | None = None,
) -> dict[str, object]:
) -> dict:
out = _redact_claims(payload)
if not isinstance(out, dict):
out = {}
out_typed: dict[str, object] = cast(dict[str, object], out)
out_typed["exp"] = _dummy_exp(now, exp_ts)
out_typed.setdefault("sub", "bot-bottle-placeholder")
return out_typed
out["exp"] = _dummy_exp(now, exp_ts)
out.setdefault("sub", "bot-bottle-placeholder")
return out
def _redact_claims(value: object) -> object:
if isinstance(value, dict):
out: dict[str, object] = {}
for key, inner in cast(dict[str, object], value).items():
for key, inner in value.items():
lower = key.lower()
if key == "https://api.openai.com/profile":
out[key] = _redact_profile_claim(inner)
@@ -210,16 +207,16 @@ def _redact_claims(value: object) -> object:
return "bot-bottle-placeholder"
def _redact_profile_claim(value: object) -> dict[str, object]:
profile = cast(dict[str, object], value) if isinstance(value, dict) else {}
def _redact_profile_claim(value: object) -> dict:
profile = value if isinstance(value, dict) else {}
return {
"email": "bot-bottle@example.invalid",
"email_verified": bool(profile.get("email_verified", True)),
}
def _redact_auth_claim(value: object) -> dict[str, object]:
auth = cast(dict[str, object], value) if isinstance(value, dict) else {}
def _redact_auth_claim(value: object) -> dict:
auth = value if isinstance(value, dict) else {}
out: dict[str, object] = {}
for key, inner in auth.items():
lower = key.lower()
@@ -250,7 +247,7 @@ def _redact_auth_claim(value: object) -> dict[str, object]:
def _redact_codex_auth(
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
) -> object:
auth = cast(dict[str, object], value) if isinstance(value, dict) else {}
auth = value if isinstance(value, dict) else {}
out: dict[str, object] = {}
for key, inner in auth.items():
lower = key.lower()
@@ -272,7 +269,7 @@ def _redact_codex_auth(
def _redact_token_block(
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
) -> dict[str, object]:
tokens = cast(dict[str, object], value) if isinstance(value, dict) else {}
tokens = value if isinstance(value, dict) else {}
out: dict[str, object] = {}
for key, inner in tokens.items():
lower = key.lower()
@@ -309,7 +306,7 @@ def _jwt_exp(token: str) -> datetime | None:
return None
if not isinstance(payload, dict):
return None
exp = cast(dict[str, object], payload).get("exp")
exp = payload.get("exp")
if not isinstance(exp, (int, float)):
return None
return datetime.fromtimestamp(exp, timezone.utc)
bot_bottle/contrib/claude/agent_provider.py
+16 -109
View File
@@ -17,12 +17,9 @@ from typing import TYPE_CHECKING
from ...agent_provider import (
AgentProvider,
AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...backend.docker import util as docker_mod
from ...egress import EgressRoute
from ...log import die, info, warn
@@ -31,6 +28,8 @@ if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_REPO_ROOT = Path(__file__).resolve().parents[3]
_SUPERVISE_MCP_NAME = "supervise"
@@ -41,56 +40,15 @@ def _skills_dir(guest_home: str) -> str:
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
_STATUS_LINE_COLORS = {
"red": "\033[91m",
"green": "\033[92m",
"yellow": "\033[93m",
"blue": "\033[94m",
"magenta": "\033[95m",
}
_CLAUDE_THEME_COLORS = {
"red": "redBright",
"green": "greenBright",
"yellow": "yellowBright",
"blue": "blueBright",
"magenta": "magentaBright",
}
def _status_line_script(label: str, color: str) -> str:
if not label:
return "#!/bin/sh\nprintf '\\n'\n"
label_q = shlex.quote(label)
if color and color in _STATUS_LINE_COLORS:
return (
"#!/bin/sh\n"
f"printf '%b%s%b\\n' '{_STATUS_LINE_COLORS[color]}' {label_q} '\\033[0m'\n"
)
return f"#!/bin/sh\nprintf '%s\\n' {label_q}\n"
def _custom_theme_payload(color: str) -> dict[str, object] | None:
theme_color = _CLAUDE_THEME_COLORS.get(color)
if not theme_color:
return None
return {
"name": f"Bot-bottle {color}",
"base": "dark",
"overrides": {
"claude": f"ansi:{theme_color}",
},
}
_RUNTIME = AgentProviderRuntime(
template="claude",
command="claude",
image="bot-bottle-claude:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",),
remote_control_args=("--remote-control",),
)
@@ -104,105 +62,54 @@ class ClaudeAgentProvider(AgentProvider):
*,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_home: str,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del forward_host_credentials, host_env
del forward_host_credentials, host_env # Codex-only knobs
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = {
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
"DISABLE_ERROR_REPORTING": "1",
}
dirs = (
AgentProvisionDir(f"{guest_home}/.claude"),
AgentProvisionDir(f"{guest_home}/.claude/themes"),
)
claude_config = state_dir / "claude.json"
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
payload: dict[str, object] = {
claude_config.write_text(json.dumps({
"hasCompletedOnboarding": True,
"theme": "dark",
"bypassPermissionsModeAccepted": True,
"projects": claude_projects,
}
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
}, indent=2) + "\n")
claude_config.chmod(0o600)
files = [
files = (
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
]
claude_settings = state_dir / "claude-settings.json"
claude_settings_payload: dict[str, object] = {}
if label or color:
statusline_script = state_dir / "claude-statusline.sh"
statusline_script.write_text(_status_line_script(label, color))
statusline_script.chmod(0o755)
files.append(AgentProvisionFile(
statusline_script,
f"{guest_home}/.claude/statusline.sh",
mode="755",
))
claude_settings_payload["statusLine"] = {
"type": "command",
"command": "~/.claude/statusline.sh",
}
theme_payload = _custom_theme_payload(color)
if theme_payload is not None:
theme_name = f"bot-bottle-{docker_mod.slugify(label or color)}"
theme_file = state_dir / f"{theme_name}.json"
theme_file.write_text(json.dumps(theme_payload, indent=2) + "\n")
theme_file.chmod(0o644)
files.append(AgentProvisionFile(
theme_file,
f"{guest_home}/.claude/themes/{theme_name}.json",
))
claude_settings_payload["theme"] = f"custom:{theme_name}"
if claude_settings_payload:
claude_settings.write_text(json.dumps(claude_settings_payload, indent=2) + "\n")
claude_settings.chmod(0o600)
files.append(AgentProvisionFile(
claude_settings,
f"{guest_home}/.claude/settings.json",
))
)
egress_routes = (EgressRoute(
host="api.anthropic.com",
auth_scheme="Bearer" if auth_token else "",
token_ref=auth_token,
tls_passthrough=True,
),)
hidden_env_names: frozenset[str] = frozenset()
if auth_token:
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=dirs,
files=tuple(files),
files=files,
egress_routes=egress_routes,
hidden_env_names=hidden_env_names,
)
@@ -213,7 +120,7 @@ class ClaudeAgentProvider(AgentProvider):
when the agent has no skills."""
from ...backend.util import host_skill_dir
agent = plan.manifest.agent
agent = plan.spec.manifest.agents[plan.spec.agent_name]
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
@@ -237,13 +144,13 @@ class ClaudeAgentProvider(AgentProvider):
prompt (drives `--append-system-prompt-file`); the file is
copied either way so the path always exists."""
prompt_path = _prompt_path(plan.guest_home)
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
bottle.cp_in(str(plan.prompt_file), prompt_path)
bottle.exec(
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
user="root",
)
agent = plan.manifest.agent
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
agent = plan.spec.manifest.agents[plan.spec.agent_name]
return prompt_path if agent.prompt else None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
"""Apply the claude-side declarative provision steps from
bot_bottle/contrib/codex/Dockerfile
-31
View File
@@ -1,31 +0,0 @@
# bot-bottle Codex provider image.
#
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed for that user.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl procps ripgrep \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by codex itself
# (codex is a Node CLI), but is convenient for the agent to shell
# out to for ad-hoc scripts. Kept on its own layer so it can be
# moved to a downstream image if the base ever needs to shrink.
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
USER node
WORKDIR /home/node
ENV PATH="/home/node/.local/bin:${PATH}"
# Remote-control support requires the standalone Codex install layout
# under ~/.codex/packages/standalone/current. The npm package can run
# the TUI, but remote-control commands expect this installer-owned path.
RUN mkdir -p /home/node/.codex \
&& curl -fsSL https://chatgpt.com/codex/install.sh | sh
CMD ["codex"]
bot_bottle/contrib/codex/agent_provider.py
+16 -30
View File
@@ -18,13 +18,12 @@ from ...agent_provider import (
CODEX_HOST_CREDENTIAL_HOSTS,
AgentProvider,
AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionCommand,
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
from ...codex_auth import codex_host_access_token, write_codex_dummy_auth_file
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
from ...log import die, info, warn
@@ -33,6 +32,8 @@ if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_REPO_ROOT = Path(__file__).resolve().parents[3]
_SUPERVISE_MCP_NAME = "supervise"
@@ -47,14 +48,15 @@ def _skills_dir(guest_home: str) -> str:
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
_RUNTIME = AgentProviderRuntime(
template="codex",
command="codex",
image="bot-bottle-codex:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"),
remote_control_args=(),
)
@@ -68,21 +70,15 @@ class CodexAgentProvider(AgentProvider):
*,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_home: str,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, label, color
del auth_token # Claude-only knob
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = {
@@ -104,11 +100,6 @@ class CodexAgentProvider(AgentProvider):
config_file.write_text(
f'[projects."{toml_path}"]\n'
'trust_level = "trusted"\n'
"\n"
"[tui]\n"
'status_line = ["model-with-reasoning"]\n'
'terminal_title = ["spinner", "project"]\n'
'theme = "ansi"\n'
)
config_file.chmod(0o600)
files.append(AgentProvisionFile(config_file, config_path))
@@ -119,6 +110,7 @@ class CodexAgentProvider(AgentProvider):
host=host,
auth_scheme="Bearer" if forward_host_credentials else "",
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
tls_passthrough=True,
))
if forward_host_credentials:
@@ -151,20 +143,14 @@ class CodexAgentProvider(AgentProvider):
"guest, but Codex did not accept it"
)))
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=tuple(dirs),
files=tuple(files),
pre_copy=tuple(pre_copy),
@@ -179,7 +165,7 @@ class CodexAgentProvider(AgentProvider):
skills."""
from ...backend.util import host_skill_dir
agent = plan.manifest.agent
agent = plan.spec.manifest.agents[plan.spec.agent_name]
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
@@ -203,13 +189,13 @@ class CodexAgentProvider(AgentProvider):
instructions in <path>.` bootstrap (see `prompt_args`); the
file is copied either way so the path always exists."""
prompt_path = _prompt_path(plan.guest_home)
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
bottle.cp_in(str(plan.prompt_file), prompt_path)
bottle.exec(
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
user="root",
)
agent = plan.manifest.agent
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
agent = plan.spec.manifest.agents[plan.spec.agent_name]
return prompt_path if agent.prompt else None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
"""Apply the codex-side declarative provision steps from
@@ -263,8 +249,8 @@ class CodexAgentProvider(AgentProvider):
return
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
r = bottle.exec(
f"codex mcp add {_SUPERVISE_MCP_NAME} --url "
f"{shlex.quote(supervise_url)}",
f"codex mcp add --transport http "
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
user="node",
)
if r.returncode != 0:
@@ -272,7 +258,7 @@ class CodexAgentProvider(AgentProvider):
f"`codex mcp add supervise` failed (exit {r.returncode}): "
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
f"register manually with: "
f"codex mcp add supervise --url {shlex.quote(supervise_url)}"
f"codex mcp add --transport http supervise {supervise_url}"
)
bot_bottle/contrib/gitea/deploy_key_provisioner.py
+3 -14
View File
@@ -2,13 +2,7 @@
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
them using the Gitea deploy-key HTTP API. No new Python dependencies —
only stdlib `urllib.request` and `subprocess`.
Required token permissions (Gitea "Applications" → "Generate Token"):
- Repository: Read & Write
Grants POST /api/v1/repos/{owner}/{repo}/keys (create deploy key)
and DELETE /api/v1/repos/{owner}/{repo}/keys/{id} (revoke deploy key).
No other scopes are needed."""
only stdlib `urllib.request` and `subprocess`."""
from __future__ import annotations
@@ -19,7 +13,7 @@ import urllib.error
import urllib.request
from pathlib import Path
from ...deploy_key_provisioner import DeployKeyCollisionError, DeployKeyProvisioner
from ...deploy_key_provisioner import DeployKeyProvisioner
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
@@ -71,11 +65,6 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
body = json.loads(resp.read())
except urllib.error.HTTPError as exc:
_body = _read_error_body(exc)
if exc.code == 422:
raise DeployKeyCollisionError(
f"deploy key collision for {owner_repo!r} "
f"(title={title!r}): key title or content already registered — {_body}"
) from exc
raise RuntimeError(
f"failed to create deploy key for {owner_repo}: "
f"HTTP {exc.code} — {_body}"
@@ -128,5 +117,5 @@ def _split_owner_repo(owner_repo: str) -> tuple[str, str]:
def _read_error_body(exc: urllib.error.HTTPError) -> str:
try:
return exc.read().decode("utf-8", errors="replace")
except Exception: # noqa: broad-exception-caught — safely fallback to empty error message
except Exception:
return ""
bot_bottle/contrib/pi/Dockerfile
-41
View File
@@ -1,41 +0,0 @@
# bot-bottle Pi provider image.
#
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
fd-find \
ripgrep \
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
&& npm cache clean --force
RUN mkdir -p /home/node/.pi/agent \
/home/node/.pi/context-mode/sessions \
/tmp/pi-subagents-uid-1000 \
&& chown -R node:node /home/node/.pi /tmp \
&& chmod -R u+rwX /tmp \
&& chown root:root /tmp /var/tmp \
&& chmod 1777 /tmp /var/tmp
USER node
WORKDIR /home/node
RUN pi install npm:@harms-haus/pi-cwd \
&& pi install npm:pi-web-access \
&& pi install npm:context-mode \
&& pi install npm:pi-subagents \
&& pi install npm:pi-mcp-adapter
CMD ["pi"]
bot_bottle/contrib/pi/__init__.py
-1
View File
@@ -1 +0,0 @@
"""Pi agent provider package."""
bot_bottle/contrib/pi/agent_provider.py
-321
View File
@@ -1,321 +0,0 @@
"""Pi agent provider plugin (PRD 0058, contrib).
Pi uses ~/.pi/agent/models.json for custom provider/model settings.
This provider writes an Ollama-compatible default configuration and
lets bottles override the model endpoint and model ids via
agent_provider.settings.
"""
from __future__ import annotations
import json
import os
import shlex
from pathlib import Path
from typing import TYPE_CHECKING
from urllib.parse import urlparse
from ...agent_provider import (
AgentProvider,
AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...egress import EgressRoute
from ...log import die, info
if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_DEFAULT_BASE_URL = "http://ollama:11434/v1"
_DEFAULT_MODEL = "qwen2.5-coder:7b"
_DEFAULT_PROVIDER_NAME = "ollama"
_DEFAULT_CONTEXT_WINDOW = 4096
_DEFAULT_MAX_TOKENS = 1024
def _skills_dir(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/skills"
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
def _append_system_path(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/APPEND_SYSTEM.md"
def _models_path(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/models.json"
def _runtime_state_repair_script(guest_home: str) -> str:
home = shlex.quote(guest_home)
pi_home = shlex.quote(f"{guest_home}/.pi")
context_sessions = shlex.quote(f"{guest_home}/.pi/context-mode/sessions")
return (
f"mkdir -p {context_sessions} /tmp/pi-subagents-uid-1000 && "
f"chown node:node {home} && "
f"chown -R node:node {pi_home} /tmp && "
"chmod -R u+rwX /tmp && "
f"chmod 755 {home} && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp"
)
def _settings_value(
settings: dict[str, object],
key: str,
default: object,
) -> object:
value = settings.get(key)
return default if value is None else value
def _settings_int(
settings: dict[str, object],
key: str,
default: int,
) -> int:
value = _settings_value(settings, key, default)
if isinstance(value, bool):
return default
if isinstance(value, (int, str)):
return int(value)
return default
def _pi_models_json(
settings: dict[str, object],
) -> tuple[dict[str, object], str, str, list[str], str]:
provider_name = str(
_settings_value(settings, "provider", _DEFAULT_PROVIDER_NAME)
)
base_url = str(_settings_value(settings, "base_url", _DEFAULT_BASE_URL))
api = str(_settings_value(settings, "api", "openai-completions"))
api_key = settings.get("api_key")
api_key_env = str(settings.get("api_key_env", ""))
models_raw = _settings_value(settings, "models", [_DEFAULT_MODEL])
models = [str(model) for model in models_raw] # type: ignore[union-attr]
supports_developer_role = bool(
_settings_value(settings, "supports_developer_role", False)
)
supports_reasoning_effort = bool(
_settings_value(settings, "supports_reasoning_effort", False)
)
max_tokens_field = str(
_settings_value(settings, "max_tokens_field", "max_tokens")
)
context_window = _settings_int(
settings, "context_window", _DEFAULT_CONTEXT_WINDOW,
)
max_tokens = _settings_int(settings, "max_tokens", _DEFAULT_MAX_TOKENS)
input_context_window = max(1, context_window - max_tokens)
provider: dict[str, object] = {
"baseUrl": base_url,
"api": api,
"compat": {
"supportsDeveloperRole": supports_developer_role,
"supportsReasoningEffort": supports_reasoning_effort,
"maxTokensField": max_tokens_field,
},
"models": [
{
"id": model,
"name": model,
"contextWindow": input_context_window,
"maxTokens": max_tokens,
}
for model in models
],
}
if api_key is not None:
provider["apiKey"] = str(api_key)
elif api_key_env:
provider["apiKey"] = "egress-placeholder"
elif provider_name == _DEFAULT_PROVIDER_NAME:
provider["apiKey"] = "ollama"
payload: dict[str, object] = {
"providers": {
provider_name: provider,
}
}
return payload, base_url, api_key_env, models, provider_name
def _route_host(base_url: str) -> str:
parsed = urlparse(base_url)
if not parsed.scheme or not parsed.hostname:
die(
"agent provider provisioning: pi settings base_url must be an "
f"absolute URL (was {base_url!r})"
)
return parsed.hostname
_RUNTIME = AgentProviderRuntime(
template="pi",
command="pi",
image="bot-bottle-pi:latest",
prompt_mode="append_system_prompt",
bypass_args=(),
resume_args=(),
)
class PiAgentProvider(AgentProvider):
@property
def runtime(self) -> AgentProviderRuntime:
return _RUNTIME
def provision_plan(
self,
*,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, forward_host_credentials, host_env, trusted_project_path
del label, color
resolved_guest_env = dict(guest_env or {})
guest_home = self.guest_home
settings = dict(provider_settings or {})
models_payload, base_url, api_key_env, models, provider_name = (
_pi_models_json(settings)
)
extra_startup_args = provider_startup_args(provider_settings)
models_file = state_dir / "pi-models.json"
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
models_file.chmod(0o600)
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
auth_scheme = "Bearer" if api_key_env else ""
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=(
"--models",
",".join(f"{provider_name}/{model}" for model in models),
*extra_startup_args,
),
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
egress_routes=(EgressRoute(
host=_route_host(base_url),
auth_scheme=auth_scheme,
token_ref=api_key_env,
),),
)
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
from ...backend.util import host_skill_dir
agent = plan.manifest.agent
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
bottle.exec(f"mkdir -p {skills_dir}", user="root")
for name in agent.skills:
src = host_skill_dir(name)
if not os.path.isdir(src):
die(
f"skill {name!r} disappeared from host between "
f"validation and copy at {src}."
)
dst = f"{skills_dir}/{name}"
info(f"copying skill {name} into {bottle.name}:{dst}")
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
bottle.cp_in(f"{src}/.", f"{dst}/")
bottle.exec(f"chown -R node:node {dst}", user="root")
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
prompt_path = _prompt_path(plan.guest_home)
append_system_path = _append_system_path(plan.guest_home)
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
bottle.exec(
f"mkdir -p {shlex.quote(plan.guest_home)}/.pi/agent && "
f"cp {shlex.quote(prompt_path)} {shlex.quote(append_system_path)} && "
f"chown node:node {shlex.quote(prompt_path)} "
f"{shlex.quote(append_system_path)} && "
f"chmod 600 {shlex.quote(prompt_path)} "
f"{shlex.quote(append_system_path)}",
user="root",
)
# Pi's `--append-system-prompt` takes literal text, not a file path.
# Use its documented APPEND_SYSTEM.md discovery path instead.
return None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
provision = plan.agent_provision
_exec(
bottle,
_runtime_state_repair_script(plan.guest_home),
"could not prepare pi runtime state",
)
for d in provision.dirs:
path = shlex.quote(d.guest_path)
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
_exec(
bottle,
f"chown {shlex.quote(d.owner)} {path}",
f"could not chown {d.guest_path}",
)
_exec(
bottle,
f"chmod {shlex.quote(d.mode)} {path}",
f"could not chmod {d.guest_path}",
)
for f in provision.files:
bottle.cp_in(str(f.host_path), f.guest_path)
path = shlex.quote(f.guest_path)
_exec(
bottle,
f"chown {shlex.quote(f.owner)} {path}",
f"could not chown {f.guest_path}",
)
_exec(
bottle,
f"chmod {shlex.quote(f.mode)} {path}",
f"could not chmod {f.guest_path}",
)
def provision_supervise_mcp(
self,
plan: "BottlePlan",
bottle: "Bottle",
supervise_url: str,
) -> None:
del plan, bottle, supervise_url
def _exec(bottle: "Bottle", script: str, error: str) -> None:
result = bottle.exec(script, user="root")
if result.returncode != 0:
detail = (result.stderr or result.stdout).strip()
if detail:
detail = f": {detail}"
die(f"agent provider provisioning: {error}{detail}")
bot_bottle/deploy_key_provisioner.py
-4
View File
@@ -11,10 +11,6 @@ from __future__ import annotations
from abc import ABC, abstractmethod
class DeployKeyCollisionError(RuntimeError):
"""Raised when a deploy key title or public key already exists on the repo."""
class DeployKeyProvisioner(ABC):
"""Manages a single deploy-key lifecycle on a remote forge."""
bot_bottle/dlp_detectors.py
-474
View File
@@ -1,474 +0,0 @@
"""DLP detectors for the egress proxy (PRD 0053).
Pure Python, no mitmproxy dependency. Each detector is a module-level
function returning `ScanResult | None`.
Ships flat into the sidecar bundle image alongside
`egress_addon_core.py` — both this file and the package source use
the same try/except import shim pattern.
"""
from __future__ import annotations
import base64
import gzip
import re
import typing
import unicodedata
from math import log2
from collections import Counter
from urllib.parse import quote as url_quote
try:
from egress_addon_core import ScanResult # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from .egress_addon_core import ScanResult
# ---------------------------------------------------------------------------
# Snippet helpers
# ---------------------------------------------------------------------------
SNIPPET_CONTEXT = 40 # chars of surrounding text to include on each side
REDACT = "********" # fixed-width replacement for the matched sensitive value
def _snippet(text: str, start: int, end: int) -> str:
"""Return context around a match with the matched span replaced by REDACT."""
before = text[max(0, start - SNIPPET_CONTEXT):start].replace("\n", " ").replace("\r", " ")
after = text[end:end + SNIPPET_CONTEXT].replace("\n", " ").replace("\r", " ")
return f"{before}{REDACT}{after}"
# ---------------------------------------------------------------------------
# Unicode normalization (defeats confusable-char and combining-mark evasion)
# ---------------------------------------------------------------------------
def _normalize_text(text: str) -> str:
# NFKD separates base characters from combining marks and resolves
# compatibility equivalents (fullwidth ASCII, ligatures, etc.)
decomposed = unicodedata.normalize("NFKD", text)
return "".join(
ch for ch in decomposed
# Strip combining marks inserted between chars to break patterns
if unicodedata.category(ch) != "Mn"
# Strip control chars; keep common whitespace (\n \r \t)
and (unicodedata.category(ch) != "Cc" or ch in "\n\r\t")
)
# ---------------------------------------------------------------------------
# Token patterns detector
# ---------------------------------------------------------------------------
TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
("AWS access key", re.compile(r"AKIA[0-9A-Z]{16}")),
("GitHub token (classic)", re.compile(r"ghp_[A-Za-z0-9_]{36}")),
("GitHub fine-grained token", re.compile(r"github_pat_[A-Za-z0-9_]{82}")),
("Anthropic API key", re.compile(r"sk-ant-[A-Za-z0-9\-_]{93}")),
("OpenAI API key", re.compile(r"sk-[A-Za-z0-9]{48}")),
("OpenAI project API key", re.compile(r"sk-proj-[A-Za-z0-9_\-]{48,}")),
("Stripe live key", re.compile(r"sk_live_[A-Za-z0-9]{24}")),
("Generic Bearer JWT", re.compile(r"Bearer\s+[A-Za-z0-9._\-]{50,}")),
("HuggingFace token", re.compile(r"hf_[A-Za-z0-9]{34,}")),
("Databricks token", re.compile(r"dapi[A-Za-z0-9]{32}")),
("Slack token", re.compile(r"xox[baprs]-[A-Za-z0-9]+-[A-Za-z0-9]+-[A-Za-z0-9]{24,}")),
("npm token", re.compile(r"npm_[A-Za-z0-9]{36}")),
("SendGrid API key", re.compile(r"SG\.[A-Za-z0-9_\-]{22}\.[A-Za-z0-9_\-]{43}")),
("PyPI token", re.compile(r"pypi-[A-Za-z0-9_\-]{80,}")),
("HashiCorp Vault token", re.compile(r"hvs\.[A-Za-z0-9_\-]{24,}")),
)
def scan_token_patterns(
text: str,
*,
location: str = "body",
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
normalized = _normalize_text(text)
for name, pattern in TOKEN_PATTERNS:
for m in pattern.finditer(normalized):
value = m.group(0)
# A value the supervisor has approved (PRD 0062) is no longer a
# block — keep scanning so a second, un-approved token in the
# same request is still caught.
if safe_tokens is not None and value in safe_tokens:
continue
return ScanResult(
severity="block",
reason=f"{name} found in {location}",
location=location,
context=_snippet(normalized, m.start(), m.end()),
matched=value,
)
return None
def redact_tokens(
text: str,
*,
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> str:
"""Replace token pattern matches and (if env given) provisioned secrets with REDACT."""
for _, pattern in TOKEN_PATTERNS:
text = pattern.sub(REDACT, text)
if env is not None:
for key, value in env.items():
if any(key.startswith(p) for p in sensitive_prefixes) and value:
for variant in _encoded_variants(value):
text = text.replace(variant, REDACT)
return text
# ---------------------------------------------------------------------------
# Known secrets detector
# ---------------------------------------------------------------------------
def _encoded_variants(secret: str) -> list[str]:
"""Return the secret plus common encoded variants for exfil detection."""
seen: set[str] = {secret}
variants: list[str] = [secret]
def _add(v: str) -> None:
if v not in seen:
seen.add(v)
variants.append(v)
secret_bytes = secret.encode("utf-8")
# Standard base64 — with and without padding
b64 = base64.b64encode(secret_bytes).decode("ascii")
_add(b64)
_add(b64.rstrip("="))
# URL-safe base64 (JWT/OAuth use -_ alphabet) — with and without padding
b64url = base64.urlsafe_b64encode(secret_bytes).decode("ascii")
_add(b64url)
_add(b64url.rstrip("="))
# URL percent-encoding
_add(url_quote(secret, safe=""))
# Hex — lowercase and uppercase
_add(secret_bytes.hex())
_add(secret_bytes.hex().upper())
# Base32 (TOTP seeds, some DNS-exfil channels)
_add(base64.b32encode(secret_bytes).decode("ascii"))
# gzip + base64 (deterministic: mtime=0); recognisable by H4sI prefix
_add(base64.b64encode(gzip.compress(secret_bytes, mtime=0)).decode("ascii"))
return variants
# ---------------------------------------------------------------------------
# Fragmentation-resistant helpers
# ---------------------------------------------------------------------------
# Minimum length of alnum projection for projection-based checks to run.
# Short secrets produce too many false positives in projection space.
_ALNUM_MIN_LEN = 8
# Minimum window length for the partial-substring sliding scan.
PARTIAL_MATCH_MIN_LEN = 12
def _alnum_projection(text: str) -> str:
"""Return text with every non-alphanumeric character stripped.
Used for fragmentation-resistant matching: separator-injected secrets
(spaces, hyphens, dots inserted between characters) are identical to
their originals in alnum projection space.
"""
return "".join(c for c in text if c.isalnum())
def _find_partial_window(secret_alnum: str, text_alnum: str, min_len: int) -> int | None:
"""Return the position in text_alnum where any min_len-char window of
secret_alnum first appears, or None.
Slides a window of width min_len across secret_alnum and searches for
each window in text_alnum. The first hit position is returned.
"""
if len(secret_alnum) < min_len or len(text_alnum) < min_len:
return None
for i in range(len(secret_alnum) - min_len + 1):
window = secret_alnum[i:i + min_len]
pos = text_alnum.find(window)
if pos >= 0:
return pos
return None
def scan_known_secrets(
text: str,
*,
location: str = "body",
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
if env is None:
return None
# Pre-compute alnum projection of the scan text once; reused per secret.
text_alnum: str | None = None
for key, value in env.items():
if not any(key.startswith(p) for p in sensitive_prefixes) or not value:
continue
# Pass 1: exact match across encoded variants (original behaviour).
approved_exact = False
for variant in _encoded_variants(value):
pos = text.find(variant)
if pos >= 0:
# The supervisor approves the exact encoded variant found
# (PRD 0062); a different encoding of the same secret is a
# fresh block.
if safe_tokens is not None and variant in safe_tokens:
approved_exact = True
continue
return ScanResult(
severity="block",
reason=f"provisioned secret from {key} found in {location}",
location=location,
context=_snippet(text, pos, pos + len(variant)),
matched=variant,
)
if approved_exact:
# Exact match was found and approved; projection passes would
# fire on the same value, so skip them for this secret.
continue
# Pass 2 & 3: fragmentation-resistant projection checks.
secret_alnum = _alnum_projection(value)
if len(secret_alnum) < _ALNUM_MIN_LEN:
continue
if text_alnum is None:
text_alnum = _alnum_projection(text)
# Pass 2: full alnum-projection exact match (catches separator injection).
pos2 = text_alnum.find(secret_alnum)
if pos2 >= 0:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(fragmented match — separator injection)"
),
location=location,
context=_snippet(text_alnum, pos2, pos2 + len(secret_alnum)),
)
# Pass 3: sliding-window partial match (catches chunked-substring leaks).
pos3 = _find_partial_window(secret_alnum, text_alnum, PARTIAL_MATCH_MIN_LEN)
if pos3 is not None:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(partial match — at least {PARTIAL_MATCH_MIN_LEN} consecutive "
f"alphanumeric chars)"
),
location=location,
context=_snippet(text_alnum, pos3, pos3 + PARTIAL_MATCH_MIN_LEN),
)
return None
# ---------------------------------------------------------------------------
# Entropy detector (warn-only)
# ---------------------------------------------------------------------------
# Sliding window size and step for the entropy scan.
ENTROPY_WINDOW = 64
ENTROPY_STEP = 32
# Bits-per-character threshold. Random ASCII printable ≈ 6.6 bits; random
# lowercase hex ≈ 4 bits; random base64url ≈ 6 bits. 5.5 sits above
# typical structured data (JSON, URLs) while staying below truly random
# content.
ENTROPY_BLOCK_THRESHOLD = 5.5
def _shannon_entropy(text: str) -> float:
if not text:
return 0.0
counts = Counter(text)
n = len(text)
return -sum((c / n) * log2(c / n) for c in counts.values())
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW,
threshold: float = ENTROPY_BLOCK_THRESHOLD,
) -> ScanResult | None:
"""Warn-only detector: flag windows of `window` chars with Shannon entropy
above `threshold` bits per character.
Never blocks; always returns severity='warn'. Disabled by default —
routes must opt in via dlp.outbound_detectors=['entropy'].
"""
if not text:
return None
step = max(1, window // 2)
end = len(text)
# Scan overlapping windows; also check the final tail if shorter than window.
positions = list(range(0, end - window + 1, step))
if end < window:
positions = [0]
elif (end - window) % step != 0:
positions.append(end - window)
for i in positions:
chunk = text[i:i + window]
if _shannon_entropy(chunk) >= threshold:
return ScanResult(
severity="warn",
reason=f"high-entropy content in {location} (possible encrypted exfil)",
location=location,
context=_snippet(text, i, i + len(chunk)),
)
return None
# ---------------------------------------------------------------------------
# Naive prompt injection detector (Phase 2)
# ---------------------------------------------------------------------------
DISCLOSURE_PHRASES: tuple[re.Pattern[str], ...] = (
re.compile(r"(?i)system\s+prompt"),
re.compile(r"(?i)my\s+instructions\s+are"),
re.compile(r"(?i)original\s+instructions"),
re.compile(r"(?i)secret\s+instructions"),
re.compile(r"(?i)hidden\s+rules"),
)
JAILBREAK_PHRASES: tuple[re.Pattern[str], ...] = (
re.compile(r"(?i)ignore\s+previous"),
re.compile(r"(?i)forget\s+everything"),
re.compile(r"(?i)disregard\s+(?:all\s+)?(?:previous|prior)"),
re.compile(r"(?i)pretend\s+you\s+are"),
re.compile(r"(?i)act\s+as\s+(?:if|though)"),
)
PROXIMITY_CHARS = 500
def _closest_pair(
a_matches: list[re.Match[str]],
b_matches: list[re.Match[str]],
) -> tuple[re.Match[str], re.Match[str]] | None:
"""Return the pair (a, b) with the smallest character gap, or None."""
best: tuple[re.Match[str], re.Match[str]] | None = None
best_gap: int | None = None
for a in a_matches:
for b in b_matches:
gap = max(0, max(a.start(), b.start()) - min(a.end(), b.end()))
if best_gap is None or gap < best_gap:
best_gap = gap
best = (a, b)
return best
def scan_naive_injection(text: str) -> ScanResult | None:
location = "response body"
disclosure_hits = [m for p in DISCLOSURE_PHRASES for m in p.finditer(text)]
jailbreak_hits = [m for p in JAILBREAK_PHRASES for m in p.finditer(text)]
if disclosure_hits and jailbreak_hits:
pair = _closest_pair(disclosure_hits, jailbreak_hits)
if pair is not None:
dist = max(0, max(pair[0].start(), pair[1].start()) - min(pair[0].end(), pair[1].end()))
if dist <= PROXIMITY_CHARS:
first = pair[0] if pair[0].start() <= pair[1].start() else pair[1]
return ScanResult(
severity="block",
reason=(
f"disclosure and jailbreak phrases within "
f"{dist} chars in {location}"
),
location=location,
context=_snippet(text, first.start(), first.end()),
)
if disclosure_hits:
m = disclosure_hits[0]
return ScanResult(
severity="warn",
reason=f"prompt disclosure phrase detected in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
)
if jailbreak_hits:
m = jailbreak_hits[0]
return ScanResult(
severity="warn",
reason=f"jailbreak phrase detected in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
)
return None
# ---------------------------------------------------------------------------
# CRLF injection detector
# ---------------------------------------------------------------------------
# URL-encoded CRLF is never legitimate in a request URL or header value.
_CRLF_ENCODED_RE = re.compile(r"%0[dD]%0[aA]", re.ASCII)
# Literal CRLF followed by a header-name pattern indicates header injection.
_CRLF_HEADER_INJECT_RE = re.compile(r"\r\n[A-Za-z][A-Za-z0-9\-]+\s*:", re.ASCII)
def strip_crlf(text: str) -> str:
"""Remove URL-encoded and literal CRLF injection sequences from a request
surface (PRD 0062 redact policy). Used to scrub the request line / headers
so the request can be forwarded instead of hard-blocked."""
text = _CRLF_ENCODED_RE.sub("", text)
return _CRLF_HEADER_INJECT_RE.sub(lambda m: m.group(0)[2:], text)
def scan_crlf_injection(text: str) -> ScanResult | None:
if _CRLF_ENCODED_RE.search(text):
return ScanResult(
severity="block",
reason="URL-encoded CRLF (%0d%0a) in outbound request",
)
if _CRLF_HEADER_INJECT_RE.search(text):
return ScanResult(
severity="block",
reason="CRLF header injection pattern in outbound request",
)
return None
__all__ = [
"ENTROPY_BLOCK_THRESHOLD",
"ENTROPY_WINDOW",
"ENTROPY_STEP",
"PARTIAL_MATCH_MIN_LEN",
"REDACT",
"SNIPPET_CONTEXT",
"TOKEN_PATTERNS",
"_alnum_projection",
"_shannon_entropy",
"redact_tokens",
"scan_crlf_injection",
"scan_entropy",
"scan_known_secrets",
"scan_naive_injection",
"scan_token_patterns",
"strip_crlf",
]
bot_bottle/egress.py
+151 -220
View File
@@ -1,106 +1,125 @@
"""Per-bottle egress proxy (PRD 0017, PRD 0053).
"""Per-bottle egress proxy (PRD 0017).
Replaces the cred-proxy sidecar (PRD 0010) with a mitmproxy-based
sidecar that becomes the agent's `HTTP_PROXY` / `HTTPS_PROXY`. It
owns three jobs:
1. MITM the agent's HTTPS with the per-bottle CA (moved from
pipelock).
2. Enforce manifest-declared `path_allowlist` per route.
3. Inject `Authorization` headers for routes that declare an
`auth` block, the same way cred-proxy does today.
This module defines the abstract proxy (`Egress`), its plan
dataclass (`EgressPlan`), and the resolved per-route shape
(`EgressRoute`). The sidecar's start/stop lifecycle is backend-
specific and lives on concrete subclasses (see
`bot_bottle/backend/docker/egress.py`).
Chunks 1+2 of the PRD: this module + the mitmproxy addon + the Docker
lifecycle are wired into the agent's `HTTP_PROXY` path; cred-proxy
has been removed. Chunk 3 retargets the cred-proxy-block remediation
flow (PRD 0014) at egress and renames the MCP tool.
"""
from __future__ import annotations
import dataclasses
import secrets
from abc import ABC
from abc import ABC, abstractmethod
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
from .egress_addon_core import (
ON_MATCH_REDACT,
HeaderMatch as CoreHeaderMatch,
MatchEntry as CoreMatchEntry,
PathMatch as CorePathMatch,
Route,
)
from .egress_addon_core import Route
from .log import die
if TYPE_CHECKING:
from .manifest import ManifestBottle
from .manifest import Bottle
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
# DNS name agents will dial for the per-bottle egress sidecar.
# Backend-agnostic by contract: every concrete backend (Docker today,
# others later) attaches this name to its sidecar on the bottle's
# internal network. The agent's `HTTP_PROXY` env var resolves to
# `http://egress:<port>` once chunk 2 cuts over.
EGRESS_HOSTNAME = "egress"
# In-container path the addon reads. Pre-created in
# `Dockerfile.sidecars` so the host bind-mount can drop the file
# directly. Content is YAML (hand-rolled by `egress_render_routes`
# in the style of `pipelock_render_yaml`, parsed by `yaml_subset`
# inside the addon).
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
EGRESS_ROUTES_FILENAME = Path(EGRESS_ROUTES_IN_CONTAINER).name
_CANARY_ENV_WORDS = (
"ACCORD",
"ANCHOR",
"ATLAS",
"CANON",
"CIPHER",
"EMBER",
"FALCON",
"HARBOR",
"LANTERN",
"MARBLE",
"NOVA",
"ORBIT",
"PIVOT",
"RADIUS",
"SUMMIT",
"VECTOR",
)
def _random_canary_env() -> str:
first = secrets.choice(_CANARY_ENV_WORDS)
remaining = tuple(word for word in _CANARY_ENV_WORDS if word != first)
second = secrets.choice(remaining)
return f"{first}_{second}_SECRET"
def egress_sidecar_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return sidecar env entries needed by egress across all backends."""
env: list[str] = []
if plan.routes:
env.extend(sorted(plan.token_env_map.keys()))
if plan.canary and plan.canary_env:
env.append(f"{plan.canary_env}={plan.canary}")
env.append(f"BOT_BOTTLE_SENSITIVE_PREFIXES={plan.canary_env}")
return tuple(env)
def egress_agent_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return agent-visible egress env entries shared by all backends."""
if plan.canary and plan.canary_env:
return (f"{plan.canary_env}={plan.canary}",)
return ()
@dataclass(frozen=True)
class EgressRoute(Route):
"""Host-side extension of the addon's `Route`.
Inherits `host`, `matches`, `auth_scheme`, and `token_env`
Inherits `host`, `path_allowlist`, `auth_scheme`, and `token_env`
from `egress_addon_core.Route` — those are the fields that cross the
YAML wire into the sidecar. The fields below are host-only and
YAML wire into the sidecar. The three fields below are host-only and
are never serialised to the addon.
`token_ref` is the host env var the CLI reads at launch and forwards
into the container's environ under `token_env`.
into the container's environ under `token_env`. Routes that share a
`token_ref` coalesce to one `token_env` slot.
`roles` carries the manifest route's role tuple (reserved for
future use; always empty today)."""
future use; always empty today).
`tls_passthrough` signals that pipelock must not TLS-MITM this
host — either because the manifest declared `pipelock.tls_passthrough:
true` (lifted in `egress_manifest_routes`) or because a provider
route set it (e.g. egress injects its own Bearer on that host
after the agent boundary and pipelock's header DLP would block it)."""
token_ref: str = ""
roles: tuple[str, ...] = ()
tls_passthrough: bool = False
@dataclass(frozen=True)
class EgressPlan:
"""Output of Egress.prepare; consumed by .start.
The slug + routes_path + routes + token_env_map fields are
filled at prepare time (host-side, side-effect-free on docker).
The network + CA + pipelock fields are populated by the backend's
launch step via `dataclasses.replace` once those resources
exist. Empty defaults are sentinels meaning "not yet set";
`.start` validates that they are populated.
`token_env_map` is `{<token_env in container>: <token_ref on host>}`.
The backend's start step reads `os.environ[token_ref]` and
forwards the value into the egress container's environ
under `token_env`. The plan itself never holds token values —
secrets never land in a dataclass that might be logged.
`mitmproxy_ca_host_path` is the host path of the per-bottle
egress CA (single PEM with cert+key concatenated) minted
by `egress_tls_init`. `.start` docker-cps it into the
sidecar at `~/.mitmproxy/mitmproxy-ca.pem` — mitmproxy reads
that file at boot to mint per-host leaf certs.
`mitmproxy_ca_cert_only_host_path` is the cert-only PEM (no
key) for installing into the agent's trust store via
`provision_ca`. Separate file rather than re-parsing the
concat so secrets and trust artefacts stay on distinct paths.
`pipelock_ca_host_path` is the host path of the pipelock CA
(cert only). `.start` docker-cps it into the sidecar so the
proxy's outbound HTTPS client trusts pipelock's MITM on the
egress → upstream leg.
`pipelock_proxy_url` is the URL egress sets as `HTTPS_PROXY`
in its environ so outbound HTTPS traverses pipelock — keeping
pipelock's hostname allowlist + DLP body scanner on the
egress → upstream leg.
"""
slug: str
routes_path: Path
routes: tuple[EgressRoute, ...]
@@ -109,75 +128,55 @@ class EgressPlan:
egress_network: str = ""
mitmproxy_ca_host_path: Path = Path()
mitmproxy_ca_cert_only_host_path: Path = Path()
log: int = 0
canary: str = ""
canary_env: str = ""
pipelock_ca_host_path: Path = Path()
pipelock_proxy_url: str = ""
def egress_manifest_routes(
bottle: ManifestBottle,
bottle: Bottle,
) -> tuple[EgressRoute, ...]:
"""Lift each `bottle.egress.routes[]` manifest entry into an EgressRoute.
Order is preserved. Token slots are not assigned here — slot assignment
is a final step in `egress_routes_for_bottle` after provider and manifest
routes are merged."""
out: list[EgressRoute] = []
for r in bottle.egress.routes:
core_matches: list[CoreMatchEntry] = []
for m in r.Matches:
core_paths = tuple(
CorePathMatch(type=p.Type, value=p.Value)
for p in m.Paths
)
core_headers = tuple(
CoreHeaderMatch(name=h.Name, value=h.Value, type=h.Type)
for h in m.Headers
)
core_matches.append(CoreMatchEntry(
paths=core_paths,
methods=m.Methods,
headers=core_headers,
))
out.append(EgressRoute(
host=r.Host,
matches=tuple(core_matches),
path_allowlist=r.PathAllowlist,
auth_scheme=r.AuthScheme,
token_ref=r.TokenRef,
roles=r.Role,
git_fetch=r.GitFetch,
outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors,
outbound_on_match=r.OutboundOnMatch,
tls_passthrough=r.Pipelock.TlsPassthrough,
))
return tuple(out)
def egress_routes_for_bottle(
bottle: ManifestBottle,
bottle: Bottle,
provider_routes: tuple[EgressRoute, ...] = (),
) -> tuple[EgressRoute, ...]:
"""Effective egress routes for the agent.
Provider routes own their hosts outright; manifest routes for hosts
not claimed by any provider are appended. Token slots are assigned
in a final pass over the merged list in order, so provisioned routes
get the lower slot numbers."""
manifest = egress_manifest_routes(bottle)
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
merged = list(_default_provider_on_match(provider_routes)) + [
merged = list(provider_routes) + [
r for r in manifest if r.host.lower() not in provisioned_hosts
]
return _assign_token_slots(merged)
def _default_provider_on_match(
provider_routes: tuple[EgressRoute, ...],
) -> tuple[EgressRoute, ...]:
"""Provider routes (the agent talking to its own LLM API) default to the
`redact` on-match policy (PRD 0062): high-volume conversation payloads are
the worst source of token-shaped false positives, so a match is scrubbed
and forwarded rather than hard-blocked or queued for the operator. A
provider that sets `outbound_on_match` explicitly keeps its choice."""
return tuple(
r if r.outbound_on_match
else dataclasses.replace(r, outbound_on_match=ON_MATCH_REDACT)
for r in provider_routes
)
def _assign_token_slots(
routes: list[EgressRoute],
) -> tuple[EgressRoute, ...]:
"""Assign EGRESS_TOKEN_N slots to authenticated routes in order.
Routes sharing a token_ref share a slot. Unauthenticated routes
(no auth_scheme / token_ref) keep token_env empty."""
slot_for_ref: dict[str, str] = {}
out: list[EgressRoute] = []
for r in routes:
@@ -195,6 +194,13 @@ def _assign_token_slots(
def egress_token_env_map(
routes: tuple[EgressRoute, ...],
) -> dict[str, str]:
"""Collapse the route list into `{token_env: token_ref}` for the
authenticated routes. Routes without `auth` contribute no entry.
Conflict detection: two routes that share a `token_env` slot but
name different `token_ref` host vars is a programming error in
`egress_routes_for_bottle`; surface it as a die rather than
silently picking one."""
out: dict[str, str] = {}
for r in routes:
if not (r.auth_scheme and r.token_ref and r.token_env):
@@ -210,101 +216,33 @@ def egress_token_env_map(
return out
def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields: dict[str, object] = {"host": r.host}
def _route_to_yaml_fields(r: Route) -> dict:
"""Return the addon-visible fields for one route.
Single authoritative mapping between EgressRoute (host-side) and
egress_addon_core.Route (sidecar-side). When a field is added to
the addon's Route that must appear in the YAML, add it here and
in egress_addon_core._parse_one together."""
fields: dict = {"host": r.host}
if r.auth_scheme and r.token_env:
fields["auth_scheme"] = r.auth_scheme
fields["token_env"] = r.token_env
if r.matches:
matches_data: list[dict[str, object]] = []
for entry in r.matches:
entry_data: dict[str, object] = {}
if entry.paths:
paths_data: list[dict[str, str]] = []
for pm in entry.paths:
pd: dict[str, str] = {"value": pm.value}
if pm.type != "prefix":
pd["type"] = pm.type
paths_data.append(pd)
entry_data["paths"] = paths_data
if entry.methods:
entry_data["methods"] = list(entry.methods)
if entry.headers:
headers_data: list[dict[str, str]] = []
for hm in entry.headers:
hd: dict[str, str] = {"name": hm.name, "value": hm.value}
if hm.type != "exact":
hd["type"] = hm.type
headers_data.append(hd)
entry_data["headers"] = headers_data
matches_data.append(entry_data)
fields["matches"] = matches_data
if r.git_fetch:
fields["git"] = {"fetch": True}
if (
r.outbound_detectors is not None
or r.inbound_detectors is not None
or r.outbound_on_match
):
dlp: dict[str, object] = {}
if r.outbound_detectors is not None:
dlp["outbound_detectors"] = (
False if not r.outbound_detectors
else list(r.outbound_detectors)
)
if r.inbound_detectors is not None:
dlp["inbound_detectors"] = (
False if not r.inbound_detectors
else list(r.inbound_detectors)
)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
fields["dlp"] = dlp
if r.path_allowlist:
fields["path_allowlist"] = list(r.path_allowlist)
return fields
def _render_match_entry(entry: dict[str, object]) -> list[str]:
lines: list[str] = []
first_key = True
if "paths" in entry:
lines.append(" - paths:")
first_key = False
for pd in entry["paths"]: # type: ignore[union-attr]
pd_dict: dict[str, str] = pd # type: ignore[assignment]
if "type" in pd_dict:
lines.append(f' - type: "{pd_dict["type"]}"')
lines.append(f' value: "{pd_dict["value"]}"')
else:
lines.append(f' - value: "{pd_dict["value"]}"')
if "methods" in entry:
methods_str = ", ".join(f'"{m}"' for m in entry["methods"]) # type: ignore[union-attr]
prefix = " - " if first_key else " "
lines.append(f'{prefix}methods: [{methods_str}]')
first_key = False
if "headers" in entry:
prefix = " - " if first_key else " "
lines.append(f"{prefix}headers:")
first_key = False
for hd in entry["headers"]: # type: ignore[union-attr]
hd_dict: dict[str, str] = hd # type: ignore[assignment]
lines.append(f' - name: "{hd_dict["name"]}"')
lines.append(f' value: "{hd_dict["value"]}"')
if first_key:
lines.append(" - {}")
return lines
def egress_render_routes(
routes: tuple[EgressRoute, ...],
*,
log: int = 0,
) -> str:
lines: list[str] = []
if log:
lines.append(f"log: {log}")
lines.append("routes:")
"""Serialize the route table for the addon to read.
YAML content — no token values, no host env-var names. Fields are
determined by `_route_to_yaml_fields`, which is the single point of
truth for the EgressRoute → egress_addon_core.Route mapping."""
lines: list[str] = ["routes:"]
if not routes:
lines[-1] = "routes: []"
lines[0] = "routes: []"
return "\n".join(lines) + "\n"
for r in routes:
f = _route_to_yaml_fields(r)
@@ -312,26 +250,10 @@ def egress_render_routes(
if "auth_scheme" in f:
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
lines.append(f' token_env: "{f["token_env"]}"')
if "matches" in f:
lines.append(" matches:")
for entry in f["matches"]: # type: ignore[union-attr]
lines.extend(_render_match_entry(entry)) # type: ignore[arg-type]
if "git" in f:
git_dict: dict[str, object] = f["git"] # type: ignore
lines.append(" git:")
if git_dict.get("fetch") is True:
lines.append(" fetch: true")
if "dlp" in f:
dlp_dict: dict[str, object] = f["dlp"] # type: ignore
lines.append(" dlp:")
for dk, dv in dlp_dict.items():
if dv is False:
lines.append(f" {dk}: false")
elif isinstance(dv, list):
items_str = ", ".join(f'"{x}"' for x in dv)
lines.append(f" {dk}: [{items_str}]")
elif isinstance(dv, str):
lines.append(f' {dk}: "{dv}"')
if "path_allowlist" in f:
lines.append(" path_allowlist:")
for p in f["path_allowlist"]:
lines.append(f' - "{p}"')
return "\n".join(lines) + "\n"
@@ -339,6 +261,12 @@ def egress_resolve_token_values(
token_env_map: dict[str, str],
host_env: dict[str, str],
) -> dict[str, str]:
"""Read `host_env[TokenRef]` for each entry in `token_env_map` and
return `{token_env: <value>}`. Dies (with a pointer at the missing
var name) if any TokenRef is unset.
Pure function: takes the host env as an argument so tests can pass
a sealed mapping without touching `os.environ`."""
out: dict[str, str] = {}
for token_env, token_ref in token_env_map.items():
value = host_env.get(token_ref)
@@ -359,36 +287,41 @@ def egress_resolve_token_values(
class Egress(ABC):
"""The per-bottle egress proxy. Encapsulates the host-side prepare
(route lift + routes.yaml render + token-env-map derivation); the
sidecar's start/stop lifecycle is backend-specific and lives on
concrete subclasses."""
def prepare(
self,
bottle: ManifestBottle,
bottle: Bottle,
slug: str,
stage_dir: Path,
provider_routes: tuple[EgressRoute, ...] = (),
) -> EgressPlan:
"""Lift `bottle.egress.routes` + `provider_routes` into resolved
routes, render the routes file (mode 600) under `stage_dir`, and
return the plan. Pure host-side, no docker subprocess. The
token-env map records the mapping the launch step uses to
forward values from the host's environ into the sidecar's environ.
Returned plan is incomplete: the launch step must fill
`internal_network` / `egress_network` / `pipelock_proxy_url`
via `dataclasses.replace` before passing it to `.start`."""
routes = egress_routes_for_bottle(bottle, provider_routes)
log = bottle.egress.Log
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
routes_path.write_text(egress_render_routes(routes, log=log))
routes_path = stage_dir / "egress_routes.yaml"
routes_path.write_text(egress_render_routes(routes))
routes_path.chmod(0o600)
# Generate a per-session fake secret under a plausible random env name.
# The sidecar marks that exact env name as sensitive for known-secret
# scanning; the agent receives the same name/value as exfil bait.
canary = secrets.token_urlsafe(32)
return EgressPlan(
slug=slug,
routes_path=routes_path,
routes=routes,
token_env_map=egress_token_env_map(routes),
log=log,
canary=canary,
canary_env=_random_canary_env(),
)
__all__ = [
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
"EGRESS_HOSTNAME",
"EGRESS_ROUTES_FILENAME",
"EGRESS_ROUTES_IN_CONTAINER",
"Egress",
"EgressPlan",
@@ -397,7 +330,5 @@ __all__ = [
"egress_render_routes",
"egress_resolve_token_values",
"egress_routes_for_bottle",
"egress_agent_env_entries",
"egress_sidecar_env_entries",
"egress_token_env_map",
]
bot_bottle/egress_addon.py
+89 -460
View File
@@ -1,111 +1,89 @@
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017).
Loaded by `mitmdump -s /app/egress_addon.py` inside the
egress container."""
egress container. Wraps the pure logic from
`egress_addon_core` with mitmproxy's HTTPFlow API:
- At startup, read `EGRESS_ROUTES` (default
`/etc/egress/routes.yaml`, JSON content) → routes table.
- SIGHUP re-reads the file and atomically swaps the in-memory
table. A parse error keeps the old table in place — better to
keep serving the old config than to leave the proxy with no
routes after a typo.
- On each `request`: strip the inbound Authorization header, then
consult `decide()` for forward / block / inject-auth and apply
the decision to the flow.
This file imports `mitmproxy` and is never imported on the host —
mitmproxy is a container-only dependency. The host's tests target
`egress_addon_core`.
Dockerfile.sidecars copies both this file and
`egress_addon_core.py` flat into `/app/`; the absolute import
below works because mitmdump runs with `/app` on its sys.path. The
parallel file in the package source tree (bot_bottle/) is the
build input — not a module the host imports."""
from __future__ import annotations
import asyncio
import dataclasses
import json
import os
import signal
import sys
from pathlib import Path
from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
from mitmproxy import http # type: ignore[import-not-found]
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
LOG_BLOCKS,
LOG_FULL,
DEFAULT_OUTBOUND_ON_MATCH,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
Config,
Route,
ScanResult,
build_inbound_scan_text,
build_outbound_scan_text,
build_token_allow_payload,
decide,
decide_git_fetch,
is_git_fetch_request,
is_git_push_request,
load_config,
match_route,
outbound_scan_headers,
route_to_yaml_dict,
scan_inbound,
scan_outbound,
)
try:
from dlp_detectors import redact_tokens, strip_crlf # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.dlp_detectors import ( # type: ignore[import-not-found]
redact_tokens,
strip_crlf,
)
try:
import supervise as _sv # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
# Absolute import (NOT `from .egress_addon_core`) — the
# container drops both files flat into /app/ so they are sibling
# top-level modules to mitmdump's loader, not a package.
from egress_addon_core import Route, decide, is_git_push_request, load_routes # type: ignore[import-not-found]
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
# Magic hostname the addon recognises as an introspection target.
# Requests through the proxy for `_egress.local/<path>` are
# intercepted and answered with synthetic responses (the addon's
# `request` hook sets `flow.response` before any upstream connection).
# The hostname is not in DNS — only clients dialing through this
# specific egress can reach it, and only via HTTP (no TLS).
# Used by the supervise sidecar's `list-egress-routes` MCP
# tool to surface the live route table to the agent.
INTROSPECT_HOST = "_egress.local"
# Seconds the egress proxy holds a token-blocked request open waiting for the
# operator's supervisor decision (PRD 0062), overridable via env.
DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS = 300.0
# Filesystem poll cadence while awaiting the operator's response.
TOKEN_ALLOW_POLL_INTERVAL_SECONDS = 0.5
# Fixed operator guidance attached to every token-allow proposal.
_TOKEN_ALLOW_JUSTIFICATION = (
"egress DLP blocked an outbound request carrying a detected token. "
"Approve only if this value is a false positive or a credential this "
"request legitimately needs; the value is then allowed for the life of "
"this bottle's egress proxy."
)
class EgressAddon:
"""The mitmproxy addon. One instance per `mitmdump` process; the
request hook is invoked on every CONNECT-decapsulated HTTP/HTTPS
request the agent makes."""
def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.config: Config = Config(routes=())
# Tokens the operator has approved this session (PRD 0062). In-memory
# only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed.
self.safe_tokens: set[str] = set()
self._supervise_queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "").strip()
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self.routes: tuple[Route, ...] = ()
self._reload(initial=True)
self._install_sighup()
def _supervise_available(self) -> bool:
return bool(self._supervise_queue_dir and self._supervise_slug)
def _reload(self, *, initial: bool = False) -> None:
try:
text = Path(self.routes_path).read_text(encoding="utf-8")
new_config = load_config(text)
new_routes = load_routes(text)
except (OSError, ValueError) as e:
tag = "boot" if initial else "SIGHUP"
sys.stderr.write(
f"egress: {tag} load failed: {e}\n"
)
if initial:
self.config = Config(routes=())
# No baseline to fall back on; serve nothing rather
# than masquerade as a proxy with a route table the
# operator never declared.
self.routes = ()
return
self.config = new_config
log_label = ("off", "blocks", "full")[self.config.log]
self.routes = new_routes
sys.stderr.write(
f"egress: loaded {len(self.config.routes)} route(s): "
f"{', '.join(r.host for r in self.config.routes)}"
f" [log={log_label}]\n"
f"egress: loaded {len(self.routes)} route(s): "
f"{', '.join(r.host for r in self.routes)}\n"
)
def _install_sighup(self) -> None:
@@ -119,9 +97,14 @@ class EgressAddon:
signal.signal(signal.SIGHUP, handler)
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
"""Synthesize a response for `_egress.local` requests.
Currently supports `/allowlist` which returns the in-memory
route table as JSON (host, path_allowlist, auth_scheme,
token_env per route — no token VALUES, those live in the
container's environ)."""
if path == "/allowlist":
payload = json.dumps(
{"routes": [route_to_yaml_dict(r) for r in self.config.routes]},
{"routes": [dataclasses.asdict(r) for r in self.routes]},
indent=2,
).encode("utf-8")
flow.response = http.Response.make(
@@ -135,415 +118,61 @@ class EgressAddon:
{"Content-Type": "text/plain; charset=utf-8"},
)
def _req_ctx(self, flow: http.HTTPFlow) -> dict[str, object]:
return {
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
}
def _block(
self,
flow: http.HTTPFlow,
reason: str,
ctx: dict[str, object] | None = None,
) -> None:
if self.config.log >= LOG_BLOCKS:
entry: dict[str, object] = {"event": "egress_block", "reason": reason}
if ctx:
entry.update(ctx)
sys.stderr.write(json.dumps(entry) + "\n")
flow.response = http.Response.make(
403,
reason.encode("utf-8"),
{"Content-Type": "text/plain; charset=utf-8"},
)
def _log_request(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.request.headers.items()
if k.lower() != "authorization"
}
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": headers,
"body": body,
})
+ "\n"
)
def _log_response(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.response.headers.items()
}
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": headers,
"body": body,
})
+ "\n"
)
async def request(self, flow: http.HTTPFlow) -> None:
# mitmproxy's addon API: this method name + signature is how
# mitmdump discovers the request hook.
def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?")
# Introspection: requests to the magic `_egress.local`
# host are answered locally with a synthetic response. Check
# before the strip-auth + route logic — these requests aren't
# real upstream traffic, the agent isn't injecting auth, and
# the addon's own decide() would 403 the magic host (it's
# never in the routes table).
if flow.request.pretty_host == INTROSPECT_HOST:
self._serve_introspection(flow, request_path)
return
# DLP outbound scan BEFORE stripping auth — catches tokens the
# agent tried to smuggle in any header, path, query param, or body.
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
if route is not None:
if not await self._handle_outbound_dlp(flow, route):
return
# The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on.
request_path, _, query = flow.request.path.partition("?")
# Inbound Authorization is always stripped — the agent cannot
# smuggle a stolen token through the proxy. If the matched
# route declares an auth pair, a fresh header is injected
# below.
flow.request.headers.pop("authorization", None)
# Universal HTTPS git-push block. Defense-in-depth: git-gate
# (PRD 0008) is the only sanctioned outbound path for git
# writes — its pre-receive runs gitleaks. Letting HTTPS push
# through egress + auth injection would route around
# that scan, so we 403 before any route logic.
if is_git_push_request(request_path, query):
self._block(
flow,
"egress: git push over HTTPS is not supported; "
"use the bottle.git SSH path (gitleaks-scanned by "
"git-gate's pre-receive hook).",
ctx=self._req_ctx(flow),
flow.response = http.Response.make(
403,
(
b"egress: git push over HTTPS is not supported; "
b"use the bottle.git SSH path (gitleaks-scanned by "
b"git-gate's pre-receive hook)."
),
{"Content-Type": "text/plain; charset=utf-8"},
)
return
if is_git_fetch_request(request_path, query):
git_decision = decide_git_fetch(
self.config.routes, flow.request.pretty_host,
)
if git_decision.action == "block":
self._block(
flow,
git_decision.reason,
ctx=self._req_ctx(flow),
)
return
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject sidecar-owned auth below.
flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
decision = decide(
self.config.routes,
self.routes,
flow.request.pretty_host,
request_path,
os.environ,
request_method=flow.request.method,
request_headers=req_headers,
)
if decision.action == "block":
self._block(flow, decision.reason, ctx=self._req_ctx(flow))
flow.response = http.Response.make(
403,
decision.reason.encode("utf-8"),
{"Content-Type": "text/plain; charset=utf-8"},
)
return
if decision.inject_authorization is not None:
flow.request.headers["authorization"] = decision.inject_authorization
if self.config.log >= LOG_FULL:
self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
ctx = self._req_ctx(flow)
if result.context:
ctx = {**ctx, "context": result.context}
self._block(flow, f"egress DLP: {result.reason}", ctx=ctx)
async def _handle_outbound_dlp(
self,
flow: http.HTTPFlow,
route: Route,
) -> bool:
"""Scan the outbound request and apply the route's on-match policy
(PRD 0062). Returns True if the request may be forwarded, False if a
403 response has been written to `flow`.
Loops so the supervise policy can re-scan after each approval — a
second, un-approved token in the same request is still caught."""
while True:
request_path, _, query = flow.request.path.partition("?")
body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, body,
)
# CRLF is scanned only over the request line + headers, never the
# body (see scan_outbound) — a body is not an injection vector.
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(
route, scan_text, os.environ,
safe_tokens=self.safe_tokens, crlf_text=crlf_text,
)
if result is None or result.severity != "block":
return True
policy = route.outbound_on_match or DEFAULT_OUTBOUND_ON_MATCH
# redact scrubs every detection (tokens and structural CRLF) and
# forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route):
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_redacted",
"reason": f"egress DLP: {result.reason}",
**self._req_ctx(flow),
}) + "\n")
return True
self._block(
flow,
f"egress DLP: {result.reason}; redaction could not remove "
"all matches (e.g. a match in the hostname)",
ctx=self._req_ctx(flow),
)
return False
# Structural blocks (CRLF, no safelist-able value) cannot be
# supervised — there is nothing to approve and remember — so under
# block/supervise they are a hard 403.
if policy == ON_MATCH_BLOCK or not result.matched:
self._block_dlp(flow, result)
return False
# supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available():
self._block_dlp(flow, result)
return False
approved = await self._supervise_token_block(flow, request_path, result)
if not approved:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound(self, flow: http.HTTPFlow, route: Route) -> bool:
"""Scrub detected tokens (and CRLF injection sequences) from the mutable
request surfaces (body, headers, path/query) and re-scan. Returns True
if the request is now clean; False if a block-severity match remains on
a surface redaction cannot rewrite (the hostname) so the caller fails
closed."""
body = flow.request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=os.environ)
if redacted_body != body:
flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()):
if name.lower() == "host":
continue # routing-critical; never a legitimate token
redacted = strip_crlf(redact_tokens(value, env=os.environ))
if redacted != value:
flow.request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=os.environ))
if redacted_path != flow.request.path:
flow.request.path = redacted_path
request_path, _, query = flow.request.path.partition("?")
new_body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, new_body,
)
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(route, scan_text, os.environ, crlf_text=crlf_text)
return result is None or result.severity != "block"
async def _supervise_token_block(
self,
flow: http.HTTPFlow,
request_path: str,
result: ScanResult,
) -> bool:
"""Route a token DLP block to the operator's supervisor queue and wait.
Returns True if the operator approved (the matched value is added to
`self.safe_tokens` and the caller re-scans); False if the request must
be blocked (a 403 response has been written to `flow`)."""
host = flow.request.pretty_host
payload = build_token_allow_payload(
redact_tokens(host, env=os.environ),
flow.request.method,
redact_tokens(request_path, env=os.environ),
result,
)
proposal = _sv.Proposal.new(
bottle_slug=self._supervise_slug,
tool=_sv.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file=payload,
justification=_TOKEN_ALLOW_JUSTIFICATION,
current_file_hash=_sv.sha256_hex(payload),
)
queue_dir = Path(self._supervise_queue_dir)
try:
_sv.write_proposal(queue_dir, proposal)
except OSError as e:
sys.stderr.write(
f"egress: could not queue token-allow proposal: {e}; "
"blocking request\n"
)
self._block(flow, f"egress DLP: {result.reason}", ctx=self._req_ctx(flow))
return False
sys.stderr.write(json.dumps({
"event": "egress_token_supervise",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
response = await self._await_token_response(queue_dir, proposal.id)
_sv.archive_proposal(queue_dir, proposal.id)
if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
):
self.safe_tokens.add(result.matched)
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_token_allowed",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
return True
if response is None:
reason = (
f"egress DLP: {result.reason}; supervisor approval timed out "
f"after {self._token_allow_timeout:g}s"
)
else:
reason = f"egress DLP: {result.reason}; supervisor rejected the request"
self._block(flow, reason, ctx=self._req_ctx(flow))
return False
async def _await_token_response(
self,
queue_dir: Path,
proposal_id: str,
) -> "_sv.Response | None":
"""Poll the queue dir for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout."""
loop = asyncio.get_running_loop()
deadline = loop.time() + self._token_allow_timeout
while True:
try:
return _sv.read_response(queue_dir, proposal_id)
except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed.
pass
if loop.time() >= deadline:
return None
await asyncio.sleep(TOKEN_ALLOW_POLL_INTERVAL_SECONDS)
def response(self, flow: http.HTTPFlow) -> None:
"""DLP inbound scan on response headers and body."""
route = match_route(self.config.routes, flow.request.pretty_host)
if route is None:
return
if flow.response is None:
return
if self.config.log >= LOG_FULL:
self._log_response(flow)
resp_headers = {k.lower(): v for k, v in flow.response.headers.items()}
body = flow.response.get_text(strict=False) or ""
scan_text = build_inbound_scan_text(resp_headers, body)
if not scan_text:
return
result = scan_inbound(route, scan_text)
if result is None:
return
resp_ctx: dict[str, object] = {
**self._req_ctx(flow),
"response_status": flow.response.status_code,
}
if result.context:
resp_ctx = {**resp_ctx, "context": result.context}
if result.severity == "block":
self._block(flow, f"egress DLP: {result.reason}", ctx=resp_ctx)
elif result.severity == "warn" and self.config.log >= LOG_BLOCKS:
sys.stderr.write(
json.dumps({
"event": "egress_warn",
"reason": f"egress DLP: {result.reason}",
**resp_ctx,
})
+ "\n"
)
def websocket_message(self, flow: http.HTTPFlow) -> None:
"""DLP scan on WebSocket frames.
Outbound frames (from_client) are scanned for credential leakage;
inbound frames are scanned for prompt injection. On a block the
entire connection is killed — there is no HTTP response surface to
write to after the upgrade.
"""
if flow.websocket is None: # type: ignore[union-attr]
return
route = match_route(self.config.routes, flow.request.pretty_host)
if route is None:
return
message = flow.websocket.messages[-1] # type: ignore[union-attr]
content = message.content.decode("utf-8", errors="replace")
if message.from_client:
# A WebSocket data frame is not an HTTP request line, so CRLF is
# not an injection vector here — scan only for credential leakage.
result = scan_outbound(
route, content, os.environ,
safe_tokens=self.safe_tokens, crlf_text="",
)
if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
else:
result = scan_inbound(route, content)
if result is not None:
if result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
elif result.severity == "warn":
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
def _token_allow_timeout_from_env(env: "os._Environ[str]") -> float:
"""Read EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS; fall back to the default on an
unset or invalid value (a bad value should not wedge egress at boot)."""
raw = env.get("EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS", "").strip()
if not raw:
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
try:
value = float(raw)
except ValueError:
value = 0.0
if value <= 0:
sys.stderr.write(
"egress: invalid EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS="
f"{raw!r}; using default {DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS:g}s\n"
)
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
return value
addons = [EgressAddon()]
bot_bottle/egress_addon_core.py
+126 -719
View File
@@ -1,4 +1,4 @@
"""Pure logic for the egress mitmproxy addon (PRD 0017, PRD 0053).
"""Pure logic for the egress mitmproxy addon (PRD 0017).
Split out of `egress_addon.py` so the host's unit tests can
exercise the parse + decision functions without depending on the
@@ -8,304 +8,81 @@ container.
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
ships flat into the sidecar bundle image alongside this file —
see `Dockerfile.sidecars`)."""
see `Dockerfile.sidecars`).
"""
from __future__ import annotations
import re
import typing
from dataclasses import dataclass
# Absolute import — `yaml_subset.py` is copied flat into the bundle
# image's `/app/` next to this file (via `Dockerfile.sidecars`).
# The host-side unit tests run with the repo on sys.path, where the
# import resolves under the `bot_bottle` package. The try/except
# shim picks whichever import works.
try:
from yaml_subset import YamlSubsetError, parse_yaml_subset # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset
# ---------------------------------------------------------------------------
# Match types (Gateway API HTTPRoute vocabulary, PRD 0053)
# ---------------------------------------------------------------------------
PATH_MATCH_TYPES = ("exact", "prefix", "regex")
HEADER_MATCH_TYPES = ("exact", "regex")
VALID_METHODS = frozenset({
"GET", "HEAD", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "TRACE",
"CONNECT",
})
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets", "entropy"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# Per-route policy for what the proxy does when an outbound DLP detector
# matches a token (PRD 0062).
ON_MATCH_BLOCK = "block" # hard 403, never overridable
ON_MATCH_REDACT = "redact" # scrub the matched value, forward the request
ON_MATCH_SUPERVISE = "supervise" # queue for operator approval, hold the request
OUTBOUND_ON_MATCH_VALUES = (ON_MATCH_BLOCK, ON_MATCH_REDACT, ON_MATCH_SUPERVISE)
# Unset resolves to supervise (fall back to block when supervise is not wired).
DEFAULT_OUTBOUND_ON_MATCH = ON_MATCH_SUPERVISE
@dataclass(frozen=True)
class PathMatch:
type: str # "exact" | "prefix" | "regex"
value: str
compiled: re.Pattern[str] | None = None
@dataclass(frozen=True)
class HeaderMatch:
name: str
value: str
type: str = "exact" # "exact" | "regex"
compiled: re.Pattern[str] | None = None
@dataclass(frozen=True)
class MatchEntry:
paths: tuple[PathMatch, ...] = ()
methods: tuple[str, ...] = ()
headers: tuple[HeaderMatch, ...] = ()
@dataclass(frozen=True)
class Route:
"""One row of the egress route table.
`host` is the request's `Host` header (or SNI hostname) to match
against. `path_allowlist` is an optional tuple of absolute path
prefixes the request path must start with; empty tuple means no
path constraint. `auth_scheme` and `token_env` together form the
credential-injection pair (both set or both empty); a non-empty
pair tells the addon to overwrite the inbound Authorization with
`<auth_scheme> <value-of-environ[token_env]>`.
"""
host: str
matches: tuple[MatchEntry, ...] = ()
path_allowlist: tuple[str, ...] = ()
auth_scheme: str = ""
token_env: str = ""
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
# "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES.
outbound_on_match: str = ""
LOG_OFF = 0 # no logging
LOG_BLOCKS = 1 # log block/warn events with request context
LOG_FULL = 2 # log block/warn events + full request and response bodies
@dataclass(frozen=True)
class Config:
routes: tuple[Route, ...]
log: int = LOG_OFF
@dataclass(frozen=True)
class Decision:
"""The result of `decide()`. Either forward (with optional
`inject_authorization` header) or block (with a `reason` to surface
to the agent)."""
action: str # "forward" or "block"
reason: str = ""
inject_authorization: str | None = None
@dataclass(frozen=True)
class ScanResult:
severity: str # "block" or "warn"
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
matched: str = ""
# ---------------------------------------------------------------------------
# Parsing
# ---------------------------------------------------------------------------
def _parse_path_match(idx: int, j: int, raw: object) -> PathMatch:
label = f"route[{idx}] matches paths[{j}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
ptype = raw_dict.get("type", "prefix")
if not isinstance(ptype, str) or ptype not in PATH_MATCH_TYPES:
raise ValueError(
f"{label}: 'type' must be one of {', '.join(PATH_MATCH_TYPES)} "
f"(got {ptype!r})"
)
value = raw_dict.get("value")
if not isinstance(value, str) or not value:
raise ValueError(f"{label}: 'value' must be a non-empty string")
if ptype in ("exact", "prefix") and not value.startswith("/"):
raise ValueError(
f"{label}: value {value!r} must start with '/' for "
f"type {ptype!r}"
)
compiled: re.Pattern[str] | None = None
if ptype == "regex":
try:
compiled = re.compile(value)
except re.error as e:
raise ValueError(
f"{label}: regex {value!r} failed to compile: {e}"
) from e
for k in raw_dict:
if k not in ("type", "value"):
raise ValueError(f"{label}: unknown key {k!r}")
return PathMatch(type=ptype, value=value, compiled=compiled)
def _parse_header_match(idx: int, j: int, raw: object) -> HeaderMatch:
label = f"route[{idx}] matches headers[{j}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
name = raw_dict.get("name")
if not isinstance(name, str) or not name:
raise ValueError(f"{label}: 'name' must be a non-empty string")
value = raw_dict.get("value")
if not isinstance(value, str):
raise ValueError(f"{label}: 'value' must be a string")
htype = raw_dict.get("type", "exact")
if not isinstance(htype, str) or htype not in HEADER_MATCH_TYPES:
raise ValueError(
f"{label}: 'type' must be one of {', '.join(HEADER_MATCH_TYPES)} "
f"(got {htype!r})"
)
compiled: re.Pattern[str] | None = None
if htype == "regex":
try:
compiled = re.compile(value)
except re.error as e:
raise ValueError(
f"{label}: regex {value!r} failed to compile: {e}"
) from e
for k in raw_dict:
if k not in ("name", "value", "type"):
raise ValueError(f"{label}: unknown key {k!r}")
return HeaderMatch(name=name, value=value, type=htype, compiled=compiled)
def _parse_match_entry(idx: int, k: int, raw: object) -> MatchEntry:
label = f"route[{idx}] matches[{k}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
paths: tuple[PathMatch, ...] = ()
paths_raw = raw_dict.get("paths")
if paths_raw is not None:
if not isinstance(paths_raw, list):
raise ValueError(f"{label}: 'paths' must be a list")
paths_list = typing.cast(list[object], paths_raw)
paths = tuple(_parse_path_match(idx, j, p) for j, p in enumerate(paths_list))
methods: tuple[str, ...] = ()
methods_raw = raw_dict.get("methods")
if methods_raw is not None:
if not isinstance(methods_raw, list):
raise ValueError(f"{label}: 'methods' must be a list")
methods_list = typing.cast(list[object], methods_raw)
normalised: list[str] = []
for j, m in enumerate(methods_list):
if not isinstance(m, str):
raise ValueError(f"{label}: methods[{j}] must be a string")
upper = m.upper()
if upper not in VALID_METHODS:
raise ValueError(
f"{label}: methods[{j}] {m!r} is not a valid HTTP method"
)
normalised.append(upper)
methods = tuple(normalised)
headers: tuple[HeaderMatch, ...] = ()
headers_raw = raw_dict.get("headers")
if headers_raw is not None:
if not isinstance(headers_raw, list):
raise ValueError(f"{label}: 'headers' must be a list")
headers_list = typing.cast(list[object], headers_raw)
headers = tuple(
_parse_header_match(idx, j, h) for j, h in enumerate(headers_list)
)
for key in raw_dict:
if key not in ("paths", "methods", "headers"):
raise ValueError(f"{label}: unknown key {key!r}")
return MatchEntry(paths=paths, methods=methods, headers=headers)
def _parse_detectors(
idx: int,
host: str,
raw_dict: dict[str, object],
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
"""Parse the optional `dlp` block on a route, returning
(outbound_detectors, inbound_detectors, outbound_on_match)."""
dlp_raw = raw_dict.get("dlp")
if dlp_raw is None:
return None, None, ""
label = f"route[{idx}] ({host})"
if not isinstance(dlp_raw, dict):
raise ValueError(f"{label}: 'dlp' must be an object")
dlp = typing.cast(dict[str, object], dlp_raw)
def _parse_detector_field(
field: str,
valid_names: frozenset[str],
) -> tuple[str, ...] | None:
val = dlp.get(field)
if val is None:
return None
if val is False:
return ()
if not isinstance(val, list):
raise ValueError(
f"{label}: dlp.{field} must be false, a list, or omitted"
)
items = typing.cast(list[object], val)
names: list[str] = []
for j, item in enumerate(items):
if not isinstance(item, str):
raise ValueError(
f"{label}: dlp.{field}[{j}] must be a string"
)
if item not in valid_names:
raise ValueError(
f"{label}: dlp.{field}[{j}] {item!r} is not a valid "
f"detector name; valid names: {', '.join(sorted(valid_names))}"
)
names.append(item)
return tuple(names)
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = dlp.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ValueError(
f"{label}: dlp.outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in dlp:
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ValueError(
f"{label}: dlp has unknown key {k!r}; accepted keys "
f"are 'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
)
return outbound, inbound, on_match
def parse_routes(payload: object) -> tuple[Route, ...]:
"""Parse the routes-file payload (already JSON-decoded) into a
tuple of `Route`s. Raises `ValueError` on any malformed entry —
the caller decides whether to keep the old table or refuse to
start.
Schema:
{
"routes": [
{
"host": "api.github.com",
"path_allowlist": ["/repos/x/", "/users/x"], # optional
"auth_scheme": "Bearer", # optional
"token_env": "EGRESS_TOKEN_0" # optional
},
...
]
}
"""
if not isinstance(payload, dict):
raise ValueError("routes payload: top-level must be an object")
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
raw: object = payload_dict.get("routes")
raw = payload.get("routes")
if not isinstance(raw, list):
raise ValueError("routes payload: 'routes' must be a list")
raw_list: list[object] = typing.cast(list[object], raw)
out: list[Route] = []
for i, r in enumerate(raw_list):
for i, r in enumerate(raw):
out.append(_parse_one(i, r))
return tuple(out)
@@ -314,29 +91,35 @@ def _parse_one(idx: int, raw: object) -> Route:
label = f"route[{idx}]"
if not isinstance(raw, dict):
raise ValueError(f"{label}: must be an object (got {type(raw).__name__})")
raw_dict: dict[str, object] = typing.cast(dict[str, object], raw)
host: object = raw_dict.get("host")
host = raw.get("host")
if not isinstance(host, str) or not host:
raise ValueError(f"{label}: 'host' must be a non-empty string")
# matches
matches: tuple[MatchEntry, ...] = ()
matches_raw = raw_dict.get("matches")
if matches_raw is not None:
if not isinstance(matches_raw, list):
raise ValueError(f"{label} ({host}): 'matches' must be a list")
matches_list = typing.cast(list[object], matches_raw)
matches = tuple(
_parse_match_entry(idx, k, m) for k, m in enumerate(matches_list)
)
path_allow_raw = raw.get("path_allowlist", [])
if not isinstance(path_allow_raw, list):
raise ValueError(f"{label} ({host}): 'path_allowlist' must be a list")
prefixes: list[str] = []
for j, p in enumerate(path_allow_raw):
if not isinstance(p, str):
raise ValueError(
f"{label} ({host}): path_allowlist[{j}] must be a string"
)
if not p.startswith("/"):
raise ValueError(
f"{label} ({host}): path_allowlist[{j}] {p!r} must be an "
f"absolute path prefix starting with '/'"
)
prefixes.append(p)
# auth (unchanged wire format)
auth_scheme: object = raw_dict.get("auth_scheme", "")
token_env: object = raw_dict.get("token_env", "")
auth_scheme = raw.get("auth_scheme", "")
token_env = raw.get("token_env", "")
if not isinstance(auth_scheme, str):
raise ValueError(f"{label} ({host}): 'auth_scheme' must be a string")
if not isinstance(token_env, str):
raise ValueError(f"{label} ({host}): 'token_env' must be a string")
# Both-or-neither: 'auth' on the manifest side renders to this
# pair atomically. A partial pair here means the renderer or a
# hand-edited file is broken.
if bool(auth_scheme) != bool(token_env):
raise ValueError(
f"{label} ({host}): 'auth_scheme' and 'token_env' must be both "
@@ -344,197 +127,49 @@ def _parse_one(idx: int, raw: object) -> Route:
f"token_env={token_env!r})"
)
# git-over-HTTPS policy
git_fetch = False
git_raw = raw_dict.get("git")
if git_raw is not None:
if not isinstance(git_raw, dict):
raise ValueError(f"{label} ({host}): 'git' must be an object")
git_dict: dict[str, object] = typing.cast(dict[str, object], git_raw)
fetch_raw = git_dict.get("fetch", False)
if fetch_raw is True or fetch_raw is False:
git_fetch = fetch_raw
else:
raise ValueError(f"{label} ({host}): 'git.fetch' must be a boolean")
for k in git_dict:
if k != "fetch":
raise ValueError(
f"{label} ({host}): git has unknown key {k!r}; "
"accepted key is 'fetch'"
)
# dlp detectors
outbound_detectors, inbound_detectors, outbound_on_match = _parse_detectors(
idx, host, raw_dict,
)
for k in raw_dict:
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git"):
raise ValueError(
f"{label} ({host}): unknown key {k!r}; accepted keys "
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git'"
)
return Route(
host=host,
matches=matches,
path_allowlist=tuple(prefixes),
auth_scheme=auth_scheme,
token_env=token_env,
git_fetch=git_fetch,
outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match,
)
def _path_match_to_dict(pm: PathMatch) -> dict[str, object]:
d: dict[str, object] = {"value": pm.value}
if pm.type != "prefix":
d["type"] = pm.type
return d
def _header_match_to_dict(hm: HeaderMatch) -> dict[str, object]:
d: dict[str, object] = {"name": hm.name, "value": hm.value}
if hm.type != "exact":
d["type"] = hm.type
return d
def _match_entry_to_dict(me: MatchEntry) -> dict[str, object]:
d: dict[str, object] = {}
if me.paths:
d["paths"] = [_path_match_to_dict(p) for p in me.paths]
if me.methods:
d["methods"] = list(me.methods)
if me.headers:
d["headers"] = [_header_match_to_dict(h) for h in me.headers]
return d
def route_to_yaml_dict(r: Route) -> dict[str, object]:
"""Serialize a Route to YAML-schema-compatible dict.
Uses the same field names the YAML parser accepts, so the output
can be round-tripped directly into an `allow` or `egress-block`
proposal without translation. Fields that are empty/default are
omitted so the agent doesn't copy irrelevant keys."""
d: dict[str, object] = {"host": r.host}
if r.auth_scheme:
d["auth_scheme"] = r.auth_scheme
d["token_env"] = r.token_env
if r.matches:
d["matches"] = [_match_entry_to_dict(m) for m in r.matches]
if r.git_fetch:
d["git"] = {"fetch": True}
dlp: dict[str, object] = {}
if r.outbound_detectors is not None:
dlp["outbound_detectors"] = list(r.outbound_detectors)
if r.inbound_detectors is not None:
dlp["inbound_detectors"] = list(r.inbound_detectors)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
if dlp:
d["dlp"] = dlp
return d
def parse_config(payload: object) -> "Config":
"""Parse a full egress config payload (top-level log level + routes)."""
if not isinstance(payload, dict):
raise ValueError("routes payload: top-level must be an object")
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
log_raw: object = payload_dict.get("log", LOG_OFF)
if log_raw is True or log_raw is False or not isinstance(log_raw, int) \
or log_raw not in (LOG_OFF, LOG_BLOCKS, LOG_FULL):
raise ValueError(
f"routes payload: 'log' must be {LOG_OFF}, {LOG_BLOCKS}, or {LOG_FULL}"
)
routes = parse_routes(payload)
return Config(routes=routes, log=log_raw)
def load_config(text: str) -> "Config":
"""Parse YAML text → Config (routes + log flag)."""
def load_routes(text: str) -> tuple[Route, ...]:
"""Parse YAML text → routes. Raises `ValueError` for both
decode and shape errors so callers handle them uniformly.
`YamlSubsetError` from the parser is a `ValueError` subclass so
it already satisfies the same surface; we let it propagate."""
try:
payload = parse_yaml_subset(text)
except YamlSubsetError as e:
raise ValueError(f"routes payload: invalid YAML: {e}") from e
return parse_config(payload)
return parse_routes(payload)
# ---------------------------------------------------------------------------
# Match evaluation
# ---------------------------------------------------------------------------
def _path_matches(pm: PathMatch, request_path: str) -> bool:
if pm.type == "exact":
return request_path == pm.value
if pm.type == "prefix":
if request_path == pm.value:
return True
if not pm.value.endswith("/"):
return request_path.startswith(pm.value + "/")
return request_path.startswith(pm.value)
if pm.type == "regex" and pm.compiled is not None:
return pm.compiled.search(request_path) is not None
return False
def _entry_matches(
entry: MatchEntry,
request_path: str,
request_method: str,
request_headers: typing.Mapping[str, str],
) -> bool:
"""All predicates within a MatchEntry are ANDed."""
if entry.paths:
if not any(_path_matches(pm, request_path) for pm in entry.paths):
return False
if entry.methods:
if request_method.upper() not in entry.methods:
return False
if entry.headers:
for hm in entry.headers:
header_val = request_headers.get(hm.name.lower())
if header_val is None:
return False
if hm.type == "exact":
if header_val != hm.value:
return False
elif hm.type == "regex" and hm.compiled is not None:
if not hm.compiled.search(header_val):
return False
return True
def evaluate_matches(
route: Route,
request_path: str,
request_method: str = "GET",
request_headers: typing.Mapping[str, str] | None = None,
) -> bool:
"""Return True if the request matches this route's match entries.
Empty matches tuple means all requests match (bare-pass route)."""
if not route.matches:
return True
hdrs: typing.Mapping[str, str] = request_headers or {}
return any(
_entry_matches(entry, request_path, request_method, hdrs)
for entry in route.matches
)
# ---------------------------------------------------------------------------
# Git push detection (unchanged)
# ---------------------------------------------------------------------------
def is_git_push_request(path: str, query: str) -> bool:
"""Return True if the request is a git smart-HTTP push.
git push over HTTPS hits two endpoints:
GET <repo>/info/refs?service=git-receive-pack (capabilities)
POST <repo>/git-receive-pack (the push)
Fetches use `service=git-upload-pack` / `/git-upload-pack` and
are unaffected. Egress-proxy refuses HTTPS push because git-gate's
pre-receive gitleaks scan is the gate for outbound git data;
routing push through egress would bypass that. Use the
bottle.git SSH path if you need to push.
Universal across routes — the block fires even when no
egress route matches the host. A bare-pass route (host with
no auth, no path_allowlist) would otherwise let push through to
pipelock + upstream untouched.
"""
if path.endswith("/git-receive-pack"):
return True
if path.endswith("/info/refs"):
# Query string is parsed leniently — `service=git-receive-pack`
# may appear with other params in any order.
for pair in query.split("&"):
k, _, v = pair.partition("=")
if k == "service" and v == "git-receive-pack":
@@ -542,25 +177,18 @@ def is_git_push_request(path: str, query: str) -> bool:
return False
def is_git_fetch_request(path: str, query: str) -> bool:
if path.endswith("/git-upload-pack"):
return True
if path.endswith("/info/refs"):
for pair in query.split("&"):
k, _, v = pair.partition("=")
if k == "service" and v == "git-upload-pack":
return True
return False
# ---------------------------------------------------------------------------
# Route lookup + decision
# ---------------------------------------------------------------------------
def match_route(
routes: typing.Sequence[Route],
request_host: str,
) -> Route | None:
"""Return the first route whose `host` matches `request_host`
exactly (case-insensitive). DNS names are case-insensitive.
Wildcard hosts (`*.foo.com`) are NOT supported — they caused
too many edge cases (apex match? cert validation? pipelock
mirror mismatch?) for too little payoff. Operators that need
multiple subdomains declare them individually (or one common
parent host as a bare-pass route)."""
target = request_host.lower()
for r in routes:
if r.host.lower() == target:
@@ -573,10 +201,24 @@ def decide(
request_host: str,
request_path: str,
environ: typing.Mapping[str, str],
*,
request_method: str = "GET",
request_headers: typing.Mapping[str, str] | None = None,
) -> Decision:
"""Pure decision: given a route table + request host + path + env,
return what the addon should do with the request.
- No matching route → BLOCK. The route table is the bottle's
egress allowlist; defense-in-depth complements pipelock's
hostname gate on the downstream leg. A bottle that wants a
host reachable from the agent must declare a route for it
(bare-pass route — no `auth`, no `path_allowlist` — is fine
for hosts that just need passthrough).
- Matching route with `path_allowlist` set, request path doesn't
start with any of the allowed prefixes → block with a clear
reason.
- Matching route with an auth pair → forward + inject
Authorization. Token comes from `environ[route.token_env]`;
missing/empty values block (route declared auth but the secret
isn't here — operator misconfig).
"""
route = match_route(routes, request_host)
if route is None:
return Decision(
@@ -588,15 +230,15 @@ def decide(
),
)
if not evaluate_matches(route, request_path, request_method, request_headers):
return Decision(
action="block",
reason=(
f"egress: request {request_method} {request_path!r} "
f"does not match any entry in matches for "
f"{route.host!r}"
),
)
if route.path_allowlist:
if not any(request_path.startswith(p) for p in route.path_allowlist):
return Decision(
action="block",
reason=(
f"egress: path {request_path!r} not in "
f"path_allowlist for {route.host!r}"
),
)
if route.auth_scheme and route.token_env:
token = environ.get(route.token_env, "")
@@ -616,247 +258,12 @@ def decide(
return Decision(action="forward")
def decide_git_fetch(
routes: typing.Sequence[Route],
request_host: str,
) -> Decision:
route = match_route(routes, request_host)
if route is not None and route.git_fetch:
return Decision(action="forward")
return Decision(
action="block",
reason=(
"egress: git fetch/clone over HTTPS is not allowed by default; "
"use git-gate for declared repos or set "
"egress.routes[].git.fetch=true for explicit read-only "
"HTTPS Git access."
),
)
# ---------------------------------------------------------------------------
# DLP scan dispatch (PRD 0053)
# ---------------------------------------------------------------------------
def build_outbound_scan_text(
host: str,
path: str,
query: str,
headers: typing.Mapping[str, str],
body: str,
) -> str:
"""Assemble all outbound request surfaces into one string for DLP scanning.
Covers hostname (DNS tunnelling), path, query params, all headers, body.
"""
parts: list[str] = [host, path]
if query:
parts.append(query)
for name, value in headers.items():
parts.append(f"{name}: {value}")
if body:
parts.append(body)
return "\n".join(parts)
def outbound_scan_headers(
route: Route,
headers: typing.Mapping[str, str],
) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP.
Routes that inject sidecar-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what
reaches the upstream.
"""
out: dict[str, str] = {}
skip_auth = bool(route.auth_scheme and route.token_env)
for name, value in headers.items():
if skip_auth and name.lower() == "authorization":
continue
out[name] = value
return out
def build_inbound_scan_text(
headers: typing.Mapping[str, str],
body: str,
) -> str:
"""Assemble inbound response surfaces into one string for DLP scanning.
Covers all response headers plus body.
"""
parts: list[str] = []
for name, value in headers.items():
parts.append(f"{name}: {value}")
if body:
parts.append(body)
return "\n".join(parts)
def _detector_enabled(
configured: tuple[str, ...] | None,
name: str,
) -> bool:
"""Check if a named detector is enabled for a route direction.
None means all enabled; empty tuple means all disabled."""
if configured is None:
return True
return name in configured
def scan_outbound(
route: Route,
body: str | bytes,
environ: typing.Mapping[str, str],
*,
safe_tokens: typing.AbstractSet[str] | None = None,
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
except ImportError: # pragma: no cover - host-side path
from .dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
# Binary bodies: latin-1 is a bijective byte↔codepoint mapping that
# preserves every byte value, so ASCII-range secret strings remain
# findable by str.find / regex. Prefer strict UTF-8 for valid text bodies.
if isinstance(body, bytes):
try:
text = body.decode("utf-8")
except UnicodeDecodeError:
text = body.decode("latin-1")
else:
text = body
# CRLF injection is only an attack in the request line + headers, never the
# body: an HTTP body is delimited by Content-Length, so CRLF bytes there
# cannot split the request. Scanning the body produces false positives on
# legitimate form-encoded / multi-line content. Callers pass the
# body-excluded surfaces as `crlf_text`; `None` falls back to the full text
# for backward-compatible callers (host-side tests, websocket frames).
crlf_target = text if crlf_text is None else crlf_text
result = scan_crlf_injection(crlf_target)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "token_patterns"):
result = scan_token_patterns(text, location="body", safe_tokens=safe_tokens)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "known_secrets"):
# BOT_BOTTLE_SENSITIVE_PREFIXES lets operators add extra env prefixes
# beyond EGRESS_TOKEN_* without changing the manifest schema.
extra_raw = environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "")
extra = tuple(p for p in extra_raw.split(",") if p)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
result = scan_known_secrets(
text, location="body", env=environ,
sensitive_prefixes=sensitive_prefixes, safe_tokens=safe_tokens,
)
if result is not None:
return result
# Entropy scanning requires explicit opt-in: it is NOT part of the
# default "all detectors" set because it produces false positives on
# legitimate base64 / binary payloads. Routes must list "entropy" in
# dlp.outbound_detectors to enable it.
if (
route.outbound_detectors is not None
and "entropy" in route.outbound_detectors
):
result = scan_entropy(text, location="body")
if result is not None:
return result
return None
def build_token_allow_payload(
host: str,
method: str,
path: str,
result: ScanResult,
) -> str:
"""Render the human-readable supervisor proposal body for an outbound
token block (PRD 0062). Carries the host/method/path, the detector
reason, and the redacted context snippet — never the raw token value."""
lines = [
"egress blocked an outbound request carrying a detected token",
f"host: {host}",
f"method: {method}",
f"path: {path}",
f"detector: {result.reason}",
]
if result.context:
lines.append(f"context: {result.context}")
return "\n".join(lines) + "\n"
def scan_inbound(
route: Route,
body: str | bytes,
) -> ScanResult | None:
try:
from dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from .dlp_detectors import scan_naive_injection # type: ignore[import-not-found]
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
if _detector_enabled(route.inbound_detectors, "naive_injection_detection"):
result = scan_naive_injection(text)
if result is not None:
return result
return None
__all__ = [
"LOG_BLOCKS",
"route_to_yaml_dict",
"LOG_FULL",
"LOG_OFF",
"ON_MATCH_BLOCK",
"ON_MATCH_REDACT",
"ON_MATCH_SUPERVISE",
"OUTBOUND_ON_MATCH_VALUES",
"DEFAULT_OUTBOUND_ON_MATCH",
"Config",
"Decision",
"HeaderMatch",
"MatchEntry",
"PathMatch",
"Route",
"ScanResult",
"build_inbound_scan_text",
"build_outbound_scan_text",
"build_token_allow_payload",
"decide",
"decide_git_fetch",
"evaluate_matches",
"is_git_push_request",
"is_git_fetch_request",
"load_config",
"load_routes",
"match_route",
"outbound_scan_headers",
"parse_config",
"parse_routes",
"scan_inbound",
"scan_outbound",
]
bot_bottle/egress_entrypoint.sh
+18 -11
View File
@@ -6,15 +6,15 @@
# call it as a normal child. Behavior is unchanged:
#
# * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch
# to `--mode upstream:URL` to chain through an upstream proxy.
# mitmproxy does NOT honor HTTPS_PROXY on its outbound side,
# so the upstream wiring has to be the mitmproxy mode flag,
# not env.
# to `--mode upstream:URL` to forward all post-MITM traffic
# through pipelock. mitmproxy does NOT honor HTTPS_PROXY on
# its outbound side, so the upstream wiring has to be the
# mitmproxy mode flag, not env.
# * Upstream trust: when EGRESS_UPSTREAM_CA is set, build a
# combined trust bundle (system roots + upstream CA) and point
# combined trust bundle (system roots + pipelock CA) and point
# mitmproxy at it. The option REPLACES mitmproxy's default
# trust store, so passing the upstream CA alone would break
# non-chained hosts.
# trust store, so passing pipelock's CA alone would break
# route-configured pipelock passthrough hosts.
# * `-s /app/egress_addon.py` loads the addon that reads
# /etc/egress/routes.yaml.
@@ -38,7 +38,11 @@ fi
# Bind address. Docker backend wants `0.0.0.0` (agent dials egress
# directly via the docker network alias). Smolmachines backend
# uses EGRESS_LISTEN_HOST when a non-default binding is needed.
# wants `127.0.0.1` because the agent dials pipelock — not egress
# — and egress is pipelock's localhost-only upstream inside the
# bundle. TSI's IP-only allowlist would otherwise let the agent
# reach `<bundle-ip>:9099` and bypass pipelock's DLP; binding
# 127.0.0.1 inside the bundle closes that gap (PRD 0023 chunk 3).
LISTEN_HOST_FLAG=""
if [ -n "$EGRESS_LISTEN_HOST" ]; then
LISTEN_HOST_FLAG="--listen-host $EGRESS_LISTEN_HOST"
@@ -52,10 +56,13 @@ if [ -n "$EGRESS_UPSTREAM_CA" ] && [ -f "$EGRESS_UPSTREAM_CA" ]; then
fi
# Scope the proxy env to this process tree only. In the bundle
# image (PRD 0024) multiple daemons share one container — setting
# image (PRD 0024) the four daemons share one container — setting
# HTTPS_PROXY at the container level would route git-gate's git
# pushes through an upstream proxy unintentionally. Setting them
# here means only mitmdump's subprocess inherits them.
# pushes through pipelock, which is wrong (pipelock doesn't proxy
# SSH and would block public git repos). Setting them here means
# only mitmdump's subprocess inherits them. In the legacy
# four-sidecar setup these env vars are also set in compose; here
# they're additionally defensive.
if [ -n "$EGRESS_UPSTREAM_PROXY" ]; then
export HTTPS_PROXY="$EGRESS_UPSTREAM_PROXY"
export HTTP_PROXY="$EGRESS_UPSTREAM_PROXY"
bot_bottle/env.py
+3 -3
View File
@@ -89,7 +89,7 @@ def _read_secret_silent(name: str, prompt_body: str) -> str:
if not (sys.stdin.isatty() or sys.stderr.isatty()):
# Fall back to /dev/tty so this still works when stdin is a pipe.
try:
tty = open("/dev/tty", "r+", encoding="utf-8")
tty = open("/dev/tty", "r+")
except OSError:
die(
f"cannot prompt for secret '{name}': no tty available. "
@@ -114,7 +114,7 @@ def _read_secret_silent(name: str, prompt_body: str) -> str:
return value
def resolve_env(manifest: Manifest) -> ResolvedEnv:
def resolve_env(manifest: Manifest, agent: str) -> ResolvedEnv:
"""Iterate the agent's env entries:
- secret: prompt at runtime; carry value in forwarded
- interpolated: read $HOST_VAR from os.environ; carry value in forwarded
@@ -124,7 +124,7 @@ def resolve_env(manifest: Manifest) -> ResolvedEnv:
backend injects forwarded values via its launcher's env parameter."""
forwarded: dict[str, str] = {}
literals: dict[str, str] = {}
bottle = manifest.bottle
bottle = manifest.bottle_for(agent)
for name, raw in bottle.env.items():
if not name:
continue
bot_bottle/git_gate.py
+27 -212
View File
@@ -15,9 +15,9 @@ a bare repo on the gate; `git daemon` serves the bare repos over
The agent never sees the upstream credential under either path.
Why a separate sidecar (not folded into egress or ssh-gate): the
Why a third sidecar (not folded into pipelock or ssh-gate): the
gate is the only one of the three that holds upstream push
credentials. Mixing it with egress would put push creds in the
credentials. Mixing it with pipelock would put push creds in the
same blast radius as internet-facing TLS interception; mixing it
with ssh-gate would force ssh-gate above L4 and into git-protocol
land. See `docs/prds/0008-git-gate.md`.
@@ -32,12 +32,12 @@ from __future__ import annotations
import dataclasses
import os
import shlex
from abc import ABC
from abc import ABC, abstractmethod
from dataclasses import dataclass
from pathlib import Path
from .log import info
from .manifest import ManifestBottle, ManifestGitEntry
from .manifest import Bottle, GitEntry
# Short network alias for git-gate inside the sidecar bundle. The
@@ -96,9 +96,9 @@ class GitGatePlan:
egress_network: str = ""
def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstream, ...]:
def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]:
"""Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name
validation already ran in `manifest.ManifestBottle.from_dict`."""
validation already ran in `manifest.Bottle.from_dict`."""
return tuple(
GitGateUpstream(
name=e.Name,
@@ -113,7 +113,7 @@ def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstre
def git_gate_render_gitconfig(
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
"""Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
@@ -204,7 +204,6 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
" git -C \"$repo\" config receive.advertisePushOptions true",
" git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}",
@@ -247,164 +246,6 @@ cat > "$refs_file"
zero=0000000000000000000000000000000000000000
supervise_gitleaks_allow() {
log_opts=$1
ref=$2
report_file=$(mktemp)
if ! gitleaks git \
--log-opts="$log_opts" \
--no-banner \
--redact \
--ignore-gitleaks-allow \
--report-format=json \
--report-path="$report_file" \
--exit-code 0 \
1>&2; then
rm -f "$report_file"
echo "git-gate: gitleaks inline-suppression scan failed for $ref" >&2
return 1
fi
proposal_id=$(
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
import datetime
import hashlib
import json
import os
import sys
import uuid
from pathlib import Path
report_path = Path(sys.argv[1])
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
if not queue_dir or not slug:
sys.exit(2)
try:
raw = json.loads(report_path.read_text() or "[]")
except json.JSONDecodeError:
sys.exit(3)
if not isinstance(raw, list):
sys.exit(3)
if not raw:
sys.exit(0)
ref = os.environ.get("GITLEAKS_ALLOW_REF", "")
lines = [
"gitleaks inline suppression requires supervisor approval",
f"ref: {ref}",
"",
]
for i, finding in enumerate(raw, 1):
if not isinstance(finding, dict):
continue
file_path = finding.get("File", "")
line_no = finding.get("StartLine", finding.get("Line", ""))
rule_id = finding.get("RuleID", "")
commit = finding.get("Commit", "")
line = finding.get("Line", "")
lines.extend([
f"finding {i}:",
f" file: {file_path}",
f" line: {line_no}",
f" rule: {rule_id}",
f" commit: {commit}",
f" code: {line}",
"",
])
payload = "\n".join(lines).rstrip() + "\n"
proposal_id = str(uuid.uuid4())
proposal = {
"id": proposal_id,
"bottle_slug": slug,
"tool": "gitleaks-allow",
"proposed_file": payload,
"justification": (
"git-gate found gitleaks findings hidden by # gitleaks:allow; "
"approve only for dummy test fixtures or confirmed false positives"
),
"arrival_timestamp": datetime.datetime.now(
datetime.timezone.utc
).isoformat(),
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(),
}
queue = Path(queue_dir)
queue.mkdir(parents=True, exist_ok=True)
path = queue / f"{proposal_id}.proposal.json"
tmp = path.with_suffix(path.suffix + ".tmp")
with tmp.open("w", encoding="utf-8") as f:
json.dump(proposal, f, indent=2)
f.write("\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
print(proposal_id)
PY
)
rc=$?
rm -f "$report_file"
if [ "$rc" -eq 0 ] && [ -z "$proposal_id" ]; then
return 0
fi
if [ "$rc" -ne 0 ]; then
echo "git-gate: cannot route # gitleaks:allow finding to supervisor; refusing push" >&2
return 1
fi
queue_dir=${SUPERVISE_QUEUE_DIR:-}
response_file="$queue_dir/${proposal_id}.response.json"
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
case "$timeout" in
''|*[!0-9]*)
echo "git-gate: invalid SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS=$timeout" >&2
return 1
;;
esac
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
waited=0
while [ "$waited" -lt "$timeout" ]; do
if [ -f "$response_file" ]; then
status=$(python3 - "$response_file" <<'PY'
import json
import sys
try:
with open(sys.argv[1], encoding="utf-8") as f:
raw = json.load(f)
except (OSError, json.JSONDecodeError):
sys.exit(1)
status = raw.get("status")
if not isinstance(status, str):
sys.exit(1)
print(status)
PY
) || status=""
case "$status" in
approved|modified)
mkdir -p "$queue_dir/processed"
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
return 0
;;
rejected)
echo "git-gate: supervisor rejected # gitleaks:allow for $ref" >&2
return 1
;;
*)
echo "git-gate: invalid supervisor response for # gitleaks:allow" >&2
return 1
;;
esac
fi
sleep 1
waited=$((waited + 1))
done
echo "git-gate: supervisor approval timed out for # gitleaks:allow; refusing push" >&2
return 1
}
# Phase 1: gitleaks scan each ref's incoming commits.
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
@@ -426,9 +267,6 @@ while IFS=' ' read -r old new ref; do
echo "git-gate: gitleaks rejected push to $ref" >&2
exit 1
fi
if ! supervise_gitleaks_allow "$log_opts" "$ref"; then
exit 1
fi
done < "$refs_file"
# Phase 2: forward each ref to the upstream (`origin`, configured
@@ -442,32 +280,15 @@ if [ ! -f "$hostsfile" ]; then
fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
push_option_count=${GIT_PUSH_OPTION_COUNT:-0}
case "$push_option_count" in
''|*[!0-9]*)
echo "git-gate: invalid GIT_PUSH_OPTION_COUNT=$push_option_count" >&2
exit 1
;;
esac
set --
i=0
while [ "$i" -lt "$push_option_count" ]; do
opt=$(printenv "GIT_PUSH_OPTION_$i" || :)
set -- "$@" --push-option="$opt"
i=$((i + 1))
done
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
if [ "$new" = "$zero" ]; then
refspec=":$ref"
elif [ "$old" != "$zero" ] && ! git merge-base --is-ancestor "$old" "$new" 2>/dev/null; then
refspec="+$new:$ref"
else
refspec="$new:$ref"
fi
echo "git-gate: forwarding $ref to origin" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git push "$@" origin "$refspec" 1>&2; then
if ! GIT_SSH_COMMAND="$ssh_cmd" git push origin "$refspec" 1>&2; then
echo "git-gate: upstream push failed for $ref" >&2
exit 1
fi
@@ -540,7 +361,7 @@ exit 0
def _provision_dynamic_key(
entry: ManifestGitEntry,
entry: GitEntry,
slug: str,
stage_dir: Path,
) -> str:
@@ -550,12 +371,13 @@ def _provision_dynamic_key(
Returns the host-side path to the private key file so the caller
can inject it into the GitGateUpstream as `identity_file`."""
from .deploy_key_provisioner import get_provisioner
pk = entry.Key
token = os.environ.get(pk.forge_token_env)
pk = entry.ProvisionedKey
assert pk is not None
token = os.environ.get(pk.token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set"
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
f" = {pk.token_env!r}: env var is not set"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url)
@@ -580,7 +402,7 @@ def _provision_dynamic_key(
return str(key_file)
def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) -> None:
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
"""Revoke all deploy keys provisioned for `bottle` during prepare.
Called at teardown after containers stop. Raises if any revocation
@@ -588,18 +410,18 @@ def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) ->
address manually."""
from .deploy_key_provisioner import get_provisioner
for entry in bottle.git:
if entry.Key.provider != "gitea":
if entry.ProvisionedKey is None:
continue
pk = entry.Key
pk = entry.ProvisionedKey
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
if not id_file.exists():
continue
key_id = id_file.read_text().strip()
token = os.environ.get(pk.forge_token_env)
token = os.environ.get(pk.token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set;"
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
f" = {pk.token_env!r}: env var is not set;"
f" cannot revoke deploy key {key_id}"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
@@ -612,26 +434,18 @@ def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) ->
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path) -> str:
"""Return the host-side SSH identity file path for this entry.
For gitea entries, provisions a fresh deploy key first."""
if entry.Key.provider == "gitea":
return _provision_dynamic_key(entry, slug, stage_dir)
return entry.IdentityFile
class GitGate(ABC):
"""The per-agent git-gate. Encapsulates the host-side prepare
(upstream lift + entrypoint/hook render); the sidecar's
start/stop lifecycle is backend-specific and lives on concrete
subclasses."""
def prepare(self, bottle: ManifestBottle, slug: str, stage_dir: Path) -> GitGatePlan:
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> GitGatePlan:
"""Compute the upstream table from `bottle.git` and write the
entrypoint, pre-receive hook, and access-hook scripts (mode
600) under `stage_dir`. Pure host-side, no docker subprocess.
For `gitea` key entries, also generates and registers
For `provisioned_key` entries, also generates and registers
a fresh deploy key via the forge API and writes the private key
+ key ID to `stage_dir`.
@@ -640,10 +454,11 @@ class GitGate(ABC):
before passing the plan to `.start`."""
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
for i, entry in enumerate(bottle.git):
upstreams_list[i] = dataclasses.replace(
upstreams_list[i],
identity_file=_resolve_identity_file(entry, slug, stage_dir),
)
if entry.ProvisionedKey is not None:
key_file = _provision_dynamic_key(entry, slug, stage_dir)
upstreams_list[i] = dataclasses.replace(
upstreams_list[i], identity_file=key_file
)
upstreams = tuple(upstreams_list)
entrypoint = stage_dir / "git_gate_entrypoint.sh"
entrypoint.write_text(git_gate_render_entrypoint(upstreams))

Some files were not shown because too many files have changed in this diff Show More