Compare commits

..

50 Commits

Author SHA1 Message Date
Quality Badge Bot d7fbe8e8a9 chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors
- Coverage: 79%

[skip ci]
2026-06-25 20:11:29 +00:00
didericis 50f5b3aa7f ci(badges): add coverage percentage to quality badges
test / integration (pull_request) Successful in 16s
test / unit (pull_request) Successful in 43s
test / unit (push) Successful in 44s
lint / lint (push) Successful in 1m49s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m53s
The update-badges workflow only refreshed pylint and pyright. Add a
coverage step that runs the unit suite under coverage.py, extracts the
TOTAL percentage, and updates a new coverage badge in the README.
Also trigger the workflow on .coveragerc changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
2026-06-25 15:13:44 -04:00
didericis-claude 45a096413f fix: add type annotations to __exit__ context manager (pyright)
lint / lint (push) Successful in 1m47s
test / unit (pull_request) Successful in 43s
test / integration (pull_request) Successful in 16s
2026-06-25 15:03:06 -04:00
didericis c6479d62e4 test: add coverage for git gate and supervise server 2026-06-25 15:03:06 -04:00
didericis d0cad3a559 chore: ignore coverage data 2026-06-25 15:03:06 -04:00
didericis c2ddac1be5 test: fix integration coverage failures 2026-06-25 15:03:06 -04:00
didericis 446414144e test: tune coverage exclusions 2026-06-25 15:03:06 -04:00
didericis 8188d6304e ci: add coverage.py reporting 2026-06-25 15:03:06 -04:00
github-actions[bot] 9f7c067e85 ci(prd): assign sequential numbers to new PRDs 2026-06-25 11:42:07 +00:00
didericis-codex 90e84a52e6 fix: remove unused supervise import for pyright
test / unit (pull_request) Successful in 37s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
Update Quality Badges / update-badges (push) Failing after 1m18s
prd-number / assign-numbers (push) Successful in 23s
test / unit (push) Successful in 33s
test / integration (push) Successful in 17s
2026-06-25 05:45:55 -04:00
didericis-claude 75755a472f refactor: drop redundant single-parent fast path in _resolve_one_bottle
lint / lint (push) Failing after 1m50s
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 18s
_fold_parents with one name returns after the first resolve; the
single-element branch was a verbatim copy of the general path.
2026-06-25 05:10:03 -04:00
didericis-claude 2f3dc57fa9 fix: resolve pyright reportUnnecessaryIsInstance in _resolve_one_bottle
Validate list entries against object-typed raw_list before narrowing to
list[str], so the isinstance(pname, str) check is not redundant.
2026-06-25 05:10:03 -04:00
didericis-claude 302920e290 feat: support multiple parents in bottle extends:
Allow extends: to accept a list of bottle names in addition to a plain
string. Parents are resolved independently and folded left-to-right
into a single combined parent before the child is merged on top, so
orthogonal concerns (base env, networking, agent provider) can live in
separate bottles without forcing a linear chain.

Merge rules for the parent fold: env dict-merge with later winning on
collision; git-gate.user per-field overlay; git-gate.repos union by
name with later winning per-field on same name; egress.routes
concatenated; all scalar fields (supervise, agent_provider, egress.log)
use last-wins. The existing child-wins-over-all-parents rule is
unchanged. Cycle detection, diamond deduplication, and missing/invalid
parent errors all work across multi-parent graphs.

Closes #268
2026-06-25 05:10:03 -04:00
Quality Badge Bot ca1b4afaea chore: update quality badges
- Pylint: 9.93/10
- Pyright: 1 errors

[skip ci]
2026-06-25 09:06:44 +00:00
didericis-codex d2072b13be feat!: remove capability apply
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 18s
lint / lint (push) Failing after 1m53s
test / unit (push) Successful in 40s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m37s
2026-06-25 08:58:28 +00:00
didericis-codex 36c5b7025b feat: add ripgrep to agent images
lint / lint (push) Successful in 1m48s
2026-06-25 04:32:53 -04:00
didericis-claude 515a95a79d fix: escape quotes/newlines in YAML and gitconfig emitters
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
test / unit (push) Successful in 32s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m18s
Closes #258.

`egress_render_routes` and `_render_match_entry` now pass all manifest
strings (host, auth_scheme, token_env, path/header values) through
`_yaml_str_escape` before interpolating into double-quoted YAML scalars,
preventing stray `"` or newlines from corrupting routes.yaml.

`git_gate_render_gitconfig` now calls `_gitconfig_validate_value` on
each Upstream value (and the derived alias) before writing the
`insteadOf` line, rejecting any value containing a newline that would
inject arbitrary gitconfig keys.
2026-06-25 04:23:13 -04:00
didericis-claude 0bace7615a refactor: rename GIT_GATE_DAEMON_TIMEOUT_SECS to GIT_GATE_TIMEOUT_SECS
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
test / unit (push) Successful in 35s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m20s
The constant now covers the daemon path, the HTTP backend access-hook,
and the git http-backend CGI subprocess, so 'daemon' in the name was
too narrow. Updated the comment to list all three current uses.
2026-06-25 04:12:43 -04:00
didericis-claude c0d3f16519 refactor: import GIT_GATE_DAEMON_TIMEOUT_SECS instead of duplicating the value 2026-06-25 04:12:43 -04:00
didericis-claude 508c537deb fix: add explicit timeouts to subprocess and HTTP calls in git-gate paths
Closes #255. Without timeouts, a hung upstream during the access-hook
or git http-backend CGI call (git_http_backend.py) and a stalled Gitea
API during deploy-key provisioning (contrib/gitea/deploy_key_provisioner.py)
could wedge a sidecar indefinitely. Adds GIT_HTTP_BACKEND_TIMEOUT_SECS
(30s) to both subprocess.run calls in the HTTP backend, mirroring the
existing GIT_GATE_DAEMON_TIMEOUT_SECS on the daemon path. Adds
_API_TIMEOUT_SECS (30s) and _KEYGEN_TIMEOUT_SECS (10s) to the Gitea
provisioner's urlopen and ssh-keygen calls. Tests verify the timeout
values are forwarded in all four call sites.
2026-06-25 04:12:43 -04:00
didericis-claude d99dba037c feat(supervise): typed RPC error taxonomy for dispatch
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
test / unit (push) Successful in 32s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m20s
Introduce _RpcClientError and _RpcInternalError as distinct subclasses
of _RpcError so the dispatcher can handle bad requests and server-side
faults differently — returning client errors verbatim and logging
internal faults with their cause before replying ERR_INTERNAL.

Wrap write_proposal and archive_proposal IO with _RpcInternalError
so OS failures surface through the typed path instead of the bare
Exception fallback. All existing raise _RpcError(...) call sites
converted to _RpcClientError.

Closes #253
2026-06-25 04:02:39 -04:00
didericis-claude 9a878bd885 fix: guard CGI Status-line parse in _write_cgi_response
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 16s
lint / lint (push) Successful in 1m47s
test / unit (push) Successful in 32s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m19s
An empty or non-numeric Status: header from git http-backend raised
ValueError/IndexError that escaped the handler thread. Wrap the parse
in a try/except and fall back to HTTP 500 instead.

Closes #254
2026-06-25 03:47:05 -04:00
didericis 0f72843150 fix(macos-container): anchor relative Dockerfile path to build context
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m49s
test / unit (push) Successful in 33s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m19s
`container build` resolves -f relative to the current working directory,
not the build context, so builds failed from any cwd other than the repo
root. Anchor a relative Dockerfile to the context before passing it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 03:27:46 -04:00
didericis fd6b14fb32 fix: route remote control through provider startup args
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 18s
lint / lint (push) Successful in 1m46s
test / unit (push) Successful in 30s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m23s
2026-06-25 03:08:47 -04:00
didericis-claude 9f9aa2e762 refactor: remove load_routes, use load_config(...).routes in tests
test / unit (pull_request) Successful in 48s
test / integration (pull_request) Successful in 26s
lint / lint (push) Successful in 1m45s
test / unit (push) Successful in 32s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m21s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 06:07:47 +00:00
didericis-codex 454baaf3a1 fix(egress): validate proposed full config
lint / lint (push) Successful in 2m23s
test / unit (pull_request) Successful in 47s
test / integration (pull_request) Successful in 28s
2026-06-25 05:25:42 +00:00
github-actions[bot] 8a092504b8 ci(prd): assign sequential numbers to new PRDs 2026-06-25 04:50:36 +00:00
didericis-codex e7dacf7d86 fix: satisfy pyright for log redaction tests
test / integration (pull_request) Successful in 29s
prd-number / assign-numbers (push) Successful in 1m6s
Update Quality Badges / update-badges (push) Successful in 1m40s
test / unit (pull_request) Successful in 52s
lint / lint (push) Successful in 2m20s
test / unit (push) Successful in 50s
test / integration (push) Successful in 27s
2026-06-25 00:32:42 -04:00
didericis-claude 9b929d0684 fix(egress): strip injected Authorization and redact bodies in LOG_FULL path
_log_request and _log_response wrote headers and bodies to stderr verbatim.
_log_request also included the sidecar-injected upstream Authorization value,
exposing live bearer tokens on every allowed request under LOG_FULL.

Apply redact_tokens to all header values and bodies in both log functions;
exclude the authorization header from _log_request entirely since its value
is always a live sidecar-injected credential by the time _log_request runs.

Closes #257
2026-06-25 00:32:42 -04:00
github-actions[bot] ec41f629a4 ci(prd): assign sequential numbers to new PRDs 2026-06-25 04:19:30 +00:00
didericis-codex d9a9eef276 docs: remove prd-new code citations
test / integration (pull_request) Successful in 46s
test / unit (pull_request) Successful in 1m4s
lint / lint (push) Successful in 2m36s
prd-number / assign-numbers (push) Successful in 1m24s
test / integration (push) Successful in 34s
test / unit (push) Successful in 52s
Update Quality Badges / update-badges (push) Successful in 2m11s
2026-06-25 03:57:41 +00:00
didericis-codex 5204b98777 refactor(egress): centralize launch env entries
lint / lint (push) Successful in 2m12s
test / unit (pull_request) Successful in 43s
test / integration (pull_request) Successful in 25s
2026-06-25 03:35:24 +00:00
didericis-codex 14ae89580a fix(egress): wire canary env for smolmachines
lint / lint (push) Successful in 2m16s
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 23s
2026-06-25 03:31:51 +00:00
didericis-codex 4808ef557a fix(egress): randomize canary secret env name
lint / lint (push) Successful in 2m15s
test / unit (pull_request) Successful in 45s
test / integration (pull_request) Successful in 26s
2026-06-25 03:25:37 +00:00
didericis-codex 0a7e166b35 fix(tests): remove unused dlp entropy import
lint / lint (push) Successful in 2m8s
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 23s
2026-06-24 23:09:11 -04:00
didericis-claude a920203730 fix(dlp): skip projection passes when exact variant is safe-listed
When a supervisor-approved safe-token exactly matched an env secret
(Pass 1), Passes 2 & 3 (alnum projection) still ran and re-blocked on
the same value.  Track whether any variant was found-and-approved and
skip the projection passes for that secret in that case.
2026-06-24 23:09:11 -04:00
didericis-claude e02fab15d0 docs(prd): flip prd-new-strengthen-outbound-exfil-detection Draft → Active
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis-claude 11cf12188d feat(egress): inject per-session canary token into sidecar and agent environments
EgressPlan gains a `canary: str` field (default "") populated in Egress.prepare()
using secrets.token_urlsafe(32).  Each launched bottle:

  - sidecar receives EGRESS_TOKEN_CANARY=<value> (literal env entry, scanned by
    existing known-secrets detector without any detector code changes)
  - agent receives BOT_BOTTLE_CANARY=<value> (visible fake secret that signals
    exfiltration with zero false positives if it appears in outbound traffic)

Docker compose and macos-container backends updated; smolmachines shares docker
compose and so picks this up automatically.  Unit tests cover canary uniqueness,
detection via scan_known_secrets, and EgressPlan backward-compat default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis-claude 701df6cb2f feat(dlp): fragmentation resistance, entropy detector, broadened known-value scan
- _alnum_projection(): strip non-alphanumeric chars for separator-injection detection
- scan_known_secrets() gains two extra passes per secret after exact-variant matching:
  alnum-projection exact match (catches hyphens/spaces between secret chars) and a
  sliding-window partial-match scan (catches chunked substrings ≥ PARTIAL_MATCH_MIN_LEN)
- scan_known_secrets() accepts sensitive_prefixes param (default ("EGRESS_TOKEN_",))
  so redact_tokens and call-sites can extend the scanned env-var prefix set
- scan_entropy() warn-only detector flagging windows with Shannon entropy ≥ 5.5 bits/char
- "entropy" added to OUTBOUND_DETECTOR_NAMES; scan_outbound opts it in only when
  explicitly listed in dlp.outbound_detectors (never part of the default "all" set)
- scan_outbound reads BOT_BOTTLE_SENSITIVE_PREFIXES from environ to extend
  scan_known_secrets beyond EGRESS_TOKEN_* without schema changes
- Binary bodies decoded via latin-1 fallback (bijective byte↔codepoint) instead
  of utf-8 errors=replace, preserving ASCII secret strings in binary payloads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis-claude ea6bc5a170 docs: draft PRD prd-new for strengthen-outbound-exfil-detection
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis ecaae708f7 feat(provider): support startup args settings
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 26s
lint / lint (push) Successful in 2m12s
test / unit (push) Successful in 41s
test / integration (push) Successful in 26s
Update Quality Badges / update-badges (push) Successful in 2m9s
2026-06-24 22:51:27 -04:00
didericis-claude 2e790268b0 fix(deploy-key): raise DeployKeyCollisionError on 422 key conflicts
lint / lint (push) Successful in 2m7s
test / unit (push) Successful in 46s
test / integration (push) Successful in 25s
Update Quality Badges / update-badges (push) Successful in 2m7s
Gitea returns HTTP 422 when a deploy key title or public key content
already exists on the repo. The provisioner previously surfaced this
as a generic RuntimeError with the raw status code. Introduce
DeployKeyCollisionError (a RuntimeError subclass) in the base module
and detect 422 in GiteaDeployKeyProvisioner.create so callers can
catch collisions explicitly and the error message names the repo and
title involved.
2026-06-25 02:23:12 +00:00
didericis-claude a421d1d688 Rename TOOL_ALLOW to TOOL_EGRESS_ALLOW
lint / lint (push) Successful in 3m50s
test / unit (push) Successful in 38s
test / integration (push) Successful in 23s
Update Quality Badges / update-badges (push) Successful in 1m38s
The constant and its MCP tool name ("allow" → "egress-allow") were the
only supervise tools without an egress-scoped identifier, despite the
tool being egress-only (routes.yaml payload, COMPONENT_FOR_TOOL maps
it to "egress", always grouped with TOOL_EGRESS_BLOCK). The rename
brings it in line with TOOL_EGRESS_BLOCK and TOOL_EGRESS_TOKEN_ALLOW,
and adds TOOL_EGRESS_ALLOW and TOOL_EGRESS_BLOCK to __all__ (both were
previously absent).
2026-06-25 01:23:10 +00:00
didericis d2d50be65a Restructure PRD 0062 to the init-prd template
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 25s
lint / lint (push) Successful in 2m8s
test / unit (push) Successful in 41s
test / integration (push) Successful in 25s
Update Quality Badges / update-badges (push) Successful in 1m32s
Conform the PRD to the standard PRD-new skeleton: add a Scope section
(In scope / Out of scope), rename Design -> Proposed Design and split
its prose into New services / Existing code touched / Data model
changes / External dependencies, fold the old Implementation chunks
into In scope, and add a References section. No change in substance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 21:10:31 -04:00
didericis 1ad710a041 Default agent-provider routes to the redact on-match policy
lint / lint (push) Successful in 1m42s
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 16s
Provider routes (the agent talking to its own LLM API — api.anthropic.com,
the Codex backend, etc.) carry the whole conversation payload, which is the
worst source of token-shaped false positives. egress_routes_for_bottle now
fills outbound_on_match=redact on any provider route that doesn't set it
explicitly, so a match there is scrubbed and forwarded rather than blocked
or queued for the operator. A provider that sets the policy keeps its
choice; manifest routes still default to supervise.

Tests: provider route gets redact default, explicit provider policy
preserved, manifest route unaffected. README + PRD 0062 updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 20:40:36 -04:00
didericis b411577e76 Stop scanning the request body for CRLF injection
lint / lint (push) Successful in 1m41s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 18s
A 403 "egress DLP: URL-encoded CRLF (%0d%0a)" was firing on legitimate
requests (e.g. the Claude Code login flow) and bypassing the on-match
policy entirely, because CRLF blocks carry no matched value and were
routed straight to a hard 403.

Root cause: CRLF injection is only an attack in the request line and
headers. An HTTP body is delimited by Content-Length, so CRLF bytes in
the body cannot split the request — but the scan flattened the body into
the same blob it checked, so form-encoded / multi-line body content
(which legitimately contains %0d%0a) tripped it.

Fix:
- scan_outbound takes a crlf_text param; the addon scans CRLF only over
  the body-excluded request line + headers. crlf_text=None keeps the
  old full-blob behavior for host-side callers/tests; the websocket path
  passes "" since a data frame is not a request line.
- The redact policy now also scrubs CRLF (new strip_crlf helper) from the
  path and headers, so redact is a complete escape hatch and structural
  CRLF in the URL/headers can be forwarded when a route opts into it.

Tests: strip_crlf unit tests; scan_outbound crlf_text body-exclusion and
backward-compat tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 20:37:26 -04:00
didericis cdfaaa3de8 Add dlp.outbound_on_match policy (block | redact | supervise)
lint / lint (push) Successful in 1m41s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 18s
Give each egress route a policy for what the proxy does when an outbound
DLP detector matches a token, defaulting to the supervise flow added in
the previous commit. The goal is cutting false-positive friction without
weakening default-deny.

- redact: scrub the matched value(s) from the body, non-host headers, and
  path/query via redact_tokens, then re-scan. Forward if clean; fail
  closed with a 403 if a match remains on a surface redaction can't
  rewrite (the hostname, or a unicode-evasion token). For routes where a
  token-shaped value is noise the upstream doesn't need.
- block: the original hard 403, never overridable.
- supervise (default, unset): hold the request for operator approval.

Structural blocks (CRLF, no safelist-able value) stay hard 403s under
every policy.

Threads outbound_on_match from the bottle manifest (manifest_egress)
through the resolved EgressRoute and rendered routes.yaml (egress.py) to
the addon's Route (egress_addon_core), and round-trips it via the
list-egress-routes introspection endpoint. The allow/egress-block tool
descriptions document the new key.

Tests: manifest parse/validation, core parse/validation, full
manifest->render->addon round-trip for redact. README + PRD 0062 updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 16:50:13 -04:00
didericis 7f2352287e PRD 0062: supervisor override for egress token blocks
lint / lint (push) Successful in 1m42s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
When the outbound DLP catches a token, route the block through the
existing supervisor approval queue instead of returning 403 outright.
The egress proxy holds the request open until the operator answers, then
remembers an approved value for the life of the proxy so the request --
and later ones carrying it -- flow through. Fails closed on rejection,
timeout, malformed response, or when supervise is disabled.

- ScanResult.matched carries the raw matched substring (sidecar-only;
  never logged or written to the proposal). scan_outbound and the token
  detectors take a safe_tokens set and skip approved values, continuing
  past a safelisted match so a second secret in the same request is
  still caught.
- New egress-token-allow proposal tool, written directly to the queue by
  the addon (the gitleaks-allow pattern from PRD 0061). build_token_allow
  _payload renders host/method/path/detector reason + redacted context.
- Async request hook polls the queue without stalling the proxy event
  loop; EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS (default 300) bounds the wait.
- Supervisor TUI renders egress-token-allow like gitleaks-allow: report
  only, modify unavailable, approval requires a recorded reason.
- Unit tests for the matched/safe-tokens plumbing, payload builder, tool
  constant round-trip, and TUI paths; README + PRD 0062.

Closes #261.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 16:12:50 -04:00
didericis 7cb967770e feat(log): add leveled severity and structured context to log wrappers
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
lint / lint (push) Successful in 1m39s
test / unit (push) Successful in 31s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m32s
log.py was bare print-to-stderr wrappers with no levels or attributable
context (issue #252). Add:

- Ordered severities (debug/info/warn/error) gated by
  BOT_BOTTLE_LOG_LEVEL (default info). debug is silent by default;
  error always surfaces (nothing sits above it), so the fatal die path
  stays visible regardless of configured level.
- An optional `context` mapping on every wrapper, rendered as a
  parseable ` [k=v ...]` suffix (keys sorted; whitespace/quoted values
  quoted) so failures can be filtered and correlated.

Default output with no context is byte-identical to the original lines,
so the 100+ existing single-string call sites are unaffected. Wires the
supervise crash path (the example the issue names) to attach error_type
and crash_log context. Adds test_log.py (backward-compat, context
rendering, level gating, die surfacing).

Closes #252.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-24 15:37:57 -04:00
didericis 80eca740d6 docs(research): replace unsourced "20% malicious skills" with cited empirical figures
The "~20% of ClawHub skills malicious" claim had no traceable source and
is contradicted by the empirical literature. Replace with the Jan 2026
large-scale study (98,380-skill snapshot: 157 confirmed malicious, ~71%
credential harvesters, exfiltration overwhelmingly naive) and add the
arXiv citation. The corrected figures still support the supply-chain
threat point and are defensible under scrutiny.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-24 09:32:19 -04:00
86 changed files with 3970 additions and 1234 deletions
+9
View File
@@ -0,0 +1,9 @@
[run]
branch = True
source = .
[report]
omit =
bot_bottle/egress_addon.py
bot_bottle/cli/tui.py
tests/*
+7 -1
View File
@@ -39,8 +39,14 @@ jobs:
with:
python-version: "3.12"
- name: Install dev requirements
run: python3 -m pip install -r requirements-dev.txt
- name: Run unit tests
run: python3 -m unittest discover -t . -s tests/unit -v
run: python3 -m coverage run -m unittest discover -t . -s tests/unit -v
- name: Report unit coverage
run: python3 -m coverage report -m
integration:
runs-on: ubuntu-latest
+15 -2
View File
@@ -8,6 +8,7 @@ on:
- '**.py'
- '.pylintrc'
- 'pyrightconfig.json'
- '.coveragerc'
workflow_dispatch:
jobs:
@@ -45,10 +46,19 @@ jobs:
echo "errors=$ERRORS" >> $GITHUB_OUTPUT
echo "Pyright errors: $ERRORS"
- name: Run coverage and extract percentage
id: coverage
run: |
python -m coverage run -m unittest discover -t . -s tests/unit > /dev/null 2>&1 || true
PERCENT=$(python -m coverage report 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Coverage: $PERCENT%"
- name: Update badges in README
run: |
PYLINT_SCORE="${{ steps.pylint.outputs.score }}"
PYRIGHT_ERRORS="${{ steps.pyright.outputs.errors }}"
COVERAGE_PERCENT="${{ steps.coverage.outputs.percent }}"
PYLINT_SCORE_ENCODED=$(echo "$PYLINT_SCORE" | sed 's|/|%2F|g')
@@ -58,9 +68,12 @@ jobs:
if [ -n "$PYRIGHT_ERRORS" ]; then
sed -i "s|/badge/pyright-[^)]*|/badge/pyright-${PYRIGHT_ERRORS}%20errors-brightgreen|" README.md
fi
if [ -n "$COVERAGE_PERCENT" ]; then
sed -i "s|/badge/coverage-[^)]*|/badge/coverage-${COVERAGE_PERCENT}%25-brightgreen|" README.md
fi
echo "Updated badges:"
grep -E "pylint|pyright" README.md | head -2
grep -E "pylint|pyright|coverage" README.md | head -3
- name: Commit and push badge updates
run: |
@@ -73,7 +86,7 @@ jobs:
else
echo "Badge changes detected, committing..."
git add README.md
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n\n'"[skip ci]"
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n'"- Coverage: ${{ steps.coverage.outputs.percent }}%"$'\n\n'"[skip ci]"
git commit -m "$MSG"
git push
fi
+1
View File
@@ -22,3 +22,4 @@ venv/
.pytest_cache/
.mypy_cache/
.ruff_cache/
.coverage
+10 -41
View File
@@ -7,41 +7,28 @@
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![pylint](https://img.shields.io/badge/pylint-9.93%2F10-brightgreen)](https://github.com/PyCQA/pylint)
[![pyright](https://img.shields.io/badge/pyright-0%20errors-brightgreen)](https://github.com/microsoft/pyright)
[![coverage](https://img.shields.io/badge/coverage-79%25-brightgreen)](https://coverage.readthedocs.io/)
**Run any coding agent like it might be compromised — and lose nothing when it is.**
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
bot-bottle is a provider-neutral, security-first substrate for autonomous agents. Bring Claude Code, Codex, or your own harness; each one runs in an ephemeral, per-agent "bottle" it cannot modify, where every byte of egress is scanned for exfiltration and capabilities are narrowed to exactly what the task declares.
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
**Problem:** You want to let a coding agent run unsupervised, but a prompt-injected or misbehaving agent — or a poisoned repo, MCP server, or skill — can wreck your environment or exfiltrate your secrets. Locking yourself to one vendor's cloud doesn't fix that; it just moves the blast radius.
**Solution:** A neutral control plane that runs *whatever agent you choose* inside an isolation boundary the agent can't touch: TLS-bumped egress allowlisting, outbound/inbound DLP, gitleaks-gated pushes, and host secrets the agent never sees. Swap the agent; keep the guarantees.
## Why bot-bottle
### A neutral substrate — bring your own agent
- **Provider-agnostic by design** — Claude and Codex ship built in; any other agent (Gemini, Aider, a local-model wrapper) is a drop-in plugin at `~/.bot-bottle/contrib/<name>/` — no fork, no PR against this repo. The manifest accepts any provider template, and the isolation, egress, and git guarantees are identical across all of them.
- **One control plane, every harness** — the same bottle, egress policy, and supervise flow wrap whichever agent you run, so switching or mixing providers doesn't change your security posture.
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
### An isolation boundary the agent can't touch
## Features
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
### Isolation that matches your host
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
Per-provider auth (Claude long-lived OAuth token; Codex opt-in host device-auth forwarding) and per-provider images (`Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile) are configured on the bottle — see [Manifest](#manifest).
## Architecture
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
@@ -83,27 +70,6 @@ The Docker topology looks like this:
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
## Install
Install the CLI with the bootstrap script:
```sh
curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
```
The script checks Python 3.11+, checks Docker daemon reachability, creates the `~/.bot-bottle/` config directories, installs the Python package with `pipx` when available or `pip --user` otherwise, then runs:
```sh
bot-bottle doctor
```
Python-native installers can use the package metadata directly:
```sh
pipx install git+https://gitea.dideric.is/didericis/bot-bottle.git
uv tool install git+https://gitea.dideric.is/didericis/bot-bottle.git
```
## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
@@ -184,8 +150,11 @@ You help maintain Gitea-hosted projects.
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default for manifest routes — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). Agent-provider routes (e.g. `api.anthropic.com`) default to `redact`. |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Trademarks
-96
View File
@@ -1,96 +0,0 @@
# Per-bottle sidecar bundle image (PRD 0024).
#
# Collapses the prior per-sidecar images (egress, git-gate,
# supervise) into one. A small stdlib-Python init supervisor at
# /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
# propagates per-daemon stdout/stderr to the container log with a
# `[name]` prefix. See PRD 0024 for the rationale.
#
# Layout:
#
# /usr/bin/gitleaks gitleaks binary
# /app/egress_addon.py + siblings mitmproxy addon (egress)
# /app/egress-entrypoint.sh mitmdump launcher
# /app/supervise_server.py + .py supervise MCP server
# /app/sidecar_init.py PID 1 supervisor
# /etc/egress/routes.yaml bind-mounted at run time
# /etc/git-gate/pre-receive docker-cp'd at start time
# /git-gate-entrypoint.sh docker-cp'd at start time
# /git-gate/creds/* docker-cp'd at start time
# /git/* bare repos, populated at runtime
# /run/supervise/queue/ bind-mounted at run time
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
#
# Exposed ports inside the container:
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
# 9418 git-gate (git-daemon)
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
# 9100 supervise (MCP HTTP)
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
# Python + mitmdump pre-installed — heavier than the others, so
# this stage starts there and pulls the standalone binaries in.
FROM mitmproxy/mitmproxy:11.1.3
# Run as root inside the bundle. The bundle is the isolation
# boundary; per-daemon user separation inside it is not load-bearing
# and complicates the supervisor's spawn path.
USER root
# Runtime system deps:
# git supplies the `git daemon` subcommand (no separate package)
# plus the core `git` binary the pre-receive hook invokes.
# openssh-client supplies the upstream SSH transport the
# pre-receive hook uses to forward accepted refs.
# ca-certificates is needed for mitmdump upstream TLS (the
# base image already has it; listed for explicitness).
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git openssh-client ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# Pull the standalone binaries into the final image.
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
# Project Python: addon + server modules + the init supervisor.
# Kept flat under /app/ so mitmdump's loader resolves them as
# top-level siblings (absolute imports), matching the prior
# Dockerfile.egress / Dockerfile.supervise layout.
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
RUN chmod +x /app/egress-entrypoint.sh
# Pre-create runtime directories the compose renderer + start
# step expect to exist. `docker cp` does not create intermediate
# dirs, and bind mounts won't either if the parent is missing.
RUN mkdir -p \
/etc/egress \
/etc/git-gate \
/git-gate/creds \
/git \
/run/supervise/queue \
/home/mitmproxy/.mitmproxy
# Documentation only — the compose renderer publishes whichever
# subset the bottle uses.
EXPOSE 8888 9099 9418 9420 9100
# WORKDIR matches Dockerfile.supervise's prior layout so the
# in-app same-dir import in supervise_server.py stays deterministic.
WORKDIR /app
# PID 1 is the supervisor. It owns signal handling and exit-code
# propagation; no `exec` chain in the entrypoint itself.
ENTRYPOINT ["python3", "/app/sidecar_init.py"]
+10 -2
View File
@@ -61,7 +61,6 @@ class AgentProviderRuntime:
prompt_mode: PromptMode
bypass_args: tuple[str, ...]
resume_args: tuple[str, ...]
remote_control_args: tuple[str, ...]
@dataclass(frozen=True)
@@ -371,6 +370,15 @@ def build_agent_provision_plan(
)
def provider_startup_args(
provider_settings: dict[str, object] | None,
) -> tuple[str, ...]:
raw = (provider_settings or {}).get("startup_args", ())
if not isinstance(raw, (list, tuple)):
return ()
return tuple(arg for arg in raw if isinstance(arg, str))
def prompt_args(
prompt_mode: PromptMode,
prompt_path: str | None,
@@ -382,7 +390,7 @@ def prompt_args(
if prompt_mode == "append_file":
return ["--append-system-prompt-file", prompt_path]
if prompt_mode == "read_prompt_file":
if argv and "resume" in argv:
if argv and ("resume" in argv or "remote-control" in argv):
return []
return [f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "print_read_prompt_file":
+1 -2
View File
@@ -109,9 +109,8 @@ class BottlePlan(ABC):
def workspace_plan(self) -> WorkspacePlan:
return workspace_plan(self.spec, guest_home=self.guest_home)
def print(self, *, remote_control: bool) -> None:
def print(self) -> None:
"""Render the y/N preflight summary to stderr."""
del remote_control
spec = self.spec
manifest = self.manifest
agent = manifest.agent
@@ -1,211 +0,0 @@
"""capability_apply — host-side orchestrator for capability-block
remediation (PRD 0016).
On approval of a capability-block proposal, the dashboard calls
apply_capability_change(slug, new_dockerfile) which:
1. Snapshots the agent's transcript dir to
~/.bot-bottle/state/<slug>/transcript/ (best-effort).
2. Pushes the agent's working tree via `git push` (best-effort —
no upstream / no commits / no git repo all skip with a log).
3. Writes the new Dockerfile to
~/.bot-bottle/state/<slug>/Dockerfile (PRD 0016 Phase 1
state). The next `cli.py start <agent>` picks it up.
4. Force-removes the agent container + all sidecars + the
per-bottle networks. Idempotent — missing resources are not
errors.
Returns (before, after) Dockerfile contents so the dashboard can
record / render the diff. (capability-block has no audit log per
PRD 0013 — the per-bottle Dockerfile state is its own record.)
This is "fire-and-forget" from the agent's perspective: by the time
the dashboard writes the response file the supervise sidecar is
gone, so the agent's tool call connection drops without ever
receiving the response. The replacement agent (next manual
`cli.py start`) sees the new Dockerfile and starts from there.
v1 does not auto-relaunch — see PRD 0016's capability-block return
semantics open question.
"""
from __future__ import annotations
import shutil
import subprocess
from ...agent_provider import get_provider
from ...log import info, warn
from ...bottle_state import (
mark_preserved,
per_bottle_dockerfile,
transcript_snapshot_dir,
write_per_bottle_dockerfile,
)
from .sidecar_bundle import sidecar_bundle_container_name
# Agent home inside the container (per the repo Dockerfile's
# `USER node` + `WORKDIR /home/node`). Used to locate the transcript
# dir + the workspace dir for git push.
_AGENT_HOME_IN_CONTAINER = "/home/node"
_AGENT_TRANSCRIPT_IN_CONTAINER = f"{_AGENT_HOME_IN_CONTAINER}/.claude"
_AGENT_WORKSPACE_IN_CONTAINER = f"{_AGENT_HOME_IN_CONTAINER}/workspace"
# Per-bottle resource name patterns (mirroring prepare.py).
def _agent_container_name(slug: str) -> str:
return f"bot-bottle-{slug}"
def _per_bottle_container_names(slug: str) -> list[str]:
"""All container names that belong to this bottle. Missing
containers are silently skipped by the teardown helper, so it's
fine to include names that don't exist for a given bottle."""
return [
_agent_container_name(slug),
sidecar_bundle_container_name(slug),
]
def _per_bottle_network_names(slug: str) -> list[str]:
return [
f"bot-bottle-net-{slug}",
f"bot-bottle-egress-{slug}",
]
class CapabilityApplyError(RuntimeError):
"""Raised when the apply fails in a way that should keep the
proposal pending (so the operator can retry). Best-effort
failures (transcript snapshot, git push) do not raise — they
just log and proceed."""
# --- Public helpers --------------------------------------------------------
def fetch_current_dockerfile(slug: str) -> str:
"""Return the Dockerfile content the next `cli.py start <agent>`
would use for this bottle. If a per-bottle override exists, that
one; otherwise the repo's Dockerfile.
Used by the operator-edit verb to show the current source of
truth, and by apply_capability_change for the before-diff."""
override = per_bottle_dockerfile(slug)
if override is not None:
return override
repo_dockerfile = get_provider("claude").dockerfile
if repo_dockerfile.is_file():
return repo_dockerfile.read_text()
raise CapabilityApplyError(
f"no per-bottle Dockerfile for {slug} and no provider Dockerfile at "
f"{repo_dockerfile}"
)
def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
"""End-to-end capability-block remediation. See module docstring
for the sequence. Returns (before, after) Dockerfile content."""
if not new_dockerfile.strip():
raise CapabilityApplyError("proposed Dockerfile is empty")
before = fetch_current_dockerfile(slug)
snapshot_transcript(slug)
_push_working_tree(slug)
write_per_bottle_dockerfile(slug, new_dockerfile)
# Set the preserve marker BEFORE teardown so cli.py's session-end
# cleanup sees it and keeps the state dir intact for the
# operator's `cli.py resume <identity>`. Without the marker the
# state dir would be deleted as part of normal session end.
mark_preserved(slug)
_teardown_bottle(slug)
return before, new_dockerfile
# --- Internals -------------------------------------------------------------
def snapshot_transcript(slug: str) -> None:
"""`docker cp` /home/node/.claude out of the agent container into
~/.bot-bottle/state/<slug>/transcript/. Best-effort: missing
container, missing dir, or cp error all log a warning and return.
The transcript is what `claude --resume` reads to pick up where
the agent left off.
Called from two places:
- capability-apply, before tearing the bottle down.
- cli.py's session-end path, before the launch context closes,
so a crash or normal exit also leaves a transcript on disk
(deleted along with the state dir on clean exit, kept on
crash or capability-block per the preserve marker)."""
container = _agent_container_name(slug)
dest = transcript_snapshot_dir(slug)
if dest.exists():
# Remove any prior snapshot so the new one is a clean copy.
shutil.rmtree(dest, ignore_errors=True)
dest.parent.mkdir(parents=True, exist_ok=True)
r = subprocess.run(
["docker", "cp", f"{container}:{_AGENT_TRANSCRIPT_IN_CONTAINER}", str(dest)],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"transcript snapshot skipped "
f"({(r.stderr or '').strip() or 'no transcript dir in container?'})"
)
return
info(f"transcript snapshotted to {dest}")
def _push_working_tree(slug: str) -> None:
"""`docker exec <agent> git push` from /home/node/workspace.
Best-effort: not-a-git-repo, no upstream, nothing-to-push, no
network all log a warning and return. The replacement bottle
will pick up whatever's actually upstream."""
container = _agent_container_name(slug)
r = subprocess.run(
[
"docker", "exec", container, "sh", "-c",
f"cd {_AGENT_WORKSPACE_IN_CONTAINER} && "
f"git rev-parse --is-inside-work-tree >/dev/null 2>&1 && "
f"git push origin HEAD 2>&1 || true",
],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"capability-apply: git push skipped "
f"({(r.stderr or '').strip() or 'docker exec failed'})"
)
return
output = (r.stdout or "").strip()
if output:
info(f"capability-apply: git push: {output}")
else:
info("capability-apply: git push ran (no output — likely not a git workspace)")
def _teardown_bottle(slug: str) -> None:
"""Force-remove all per-bottle docker resources. Idempotent —
`docker rm -f` / `docker network rm` silently ignore missing
names, so this can be called even mid-rebuild."""
info(f"capability-apply: tearing down bottle {slug}")
for name in _per_bottle_container_names(slug):
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
for net in _per_bottle_network_names(slug):
subprocess.run(
["docker", "network", "rm", net],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
__all__ = [
"CapabilityApplyError",
"apply_capability_change",
"fetch_current_dockerfile",
"snapshot_transcript",
]
+6 -21
View File
@@ -28,11 +28,12 @@ from typing import Any
from ...egress import (
EGRESS_HOSTNAME,
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_sidecar_env_entries,
)
from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn
from ...supervise import (
CURRENT_CONFIG_DIR_IN_AGENT,
QUEUE_DIR_IN_CONTAINER,
SUPERVISE_HOSTNAME,
SUPERVISE_PORT,
@@ -58,17 +59,10 @@ from .sidecar_bundle import (
)
# Repo root or installed site-packages root, used as the build context for
# Dockerfiles that COPY bot_bottle source files.
# Repo root, used as the build context for the bundle Dockerfile.
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def _sidecar_bundle_dockerfile() -> str:
if (Path(_REPO_DIR) / SIDECAR_BUNDLE_DOCKERFILE).is_file():
return SIDECAR_BUNDLE_DOCKERFILE
return f"bot_bottle/{SIDECAR_BUNDLE_DOCKERFILE}"
def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
"""Render a Compose v2 spec dict from a fully-resolved
DockerBottlePlan.
@@ -142,8 +136,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
if ep.routes:
volumes.append(_bind(ep.routes_path.parent, str(Path(EGRESS_ROUTES_IN_CONTAINER).parent)))
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate -----------------------------------------------------
gp = plan.git_gate_plan
@@ -190,7 +183,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"image": SIDECAR_BUNDLE_IMAGE,
"build": {
"context": _REPO_DIR,
"dockerfile": _sidecar_bundle_dockerfile(),
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
},
"container_name": sidecar_bundle_container_name(plan.slug),
"networks": {
@@ -227,6 +220,7 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
# never lands on argv or in the compose file.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
service: dict[str, Any] = {
"image": plan.image,
@@ -238,15 +232,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
if plan.use_runsc:
service["runtime"] = "runsc"
volumes: list[dict[str, Any]] = []
if plan.supervise_plan is not None:
volumes.append(_bind(
plan.supervise_plan.current_config_dir,
CURRENT_CONFIG_DIR_IN_AGENT,
))
if volumes:
service["volumes"] = volumes
# The init supervisor inside the bundle owns intra-bundle
# daemon ordering, so the agent only waits for the bundle
# container itself.
+3 -4
View File
@@ -12,10 +12,9 @@ from __future__ import annotations
import os
# Bundle image. Defaults to a built-locally tag. Source checkouts
# build from the repo-root Dockerfile.sidecars; installed packages
# build from the packaged copy under bot_bottle/.
# Operators pinning to a published digest can override via env.
# Bundle image. Defaults to a built-locally tag (built from the
# repo's Dockerfile.sidecars via compose `build:`). Operators
# pinning to a published digest can override via env.
SIDECAR_BUNDLE_IMAGE = os.environ.get(
"BOT_BOTTLE_SIDECAR_IMAGE",
"bot-bottle-sidecars:latest",
+6 -2
View File
@@ -11,7 +11,7 @@ from pathlib import Path
from ..bottle_state import egress_state_dir
from ..egress import EGRESS_ROUTES_FILENAME
from ..egress_addon_core import load_routes
from ..egress_addon_core import LOG_OFF, load_config
class EgressApplyError(RuntimeError):
@@ -33,11 +33,15 @@ class EgressApplicator(ABC):
@staticmethod
def validate_routes_content(content: str) -> None:
try:
load_routes(content)
config = load_config(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
if config.log != LOG_OFF:
raise EgressApplyError(
"proposed routes.yaml must not change egress logging"
)
@staticmethod
def _routes_path(slug: str) -> Path:
+8 -4
View File
@@ -22,7 +22,12 @@ from ...bottle_state import (
git_gate_state_dir,
read_committed_image,
)
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import die, info, warn
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
@@ -350,9 +355,7 @@ def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
env: list[str] = []
if plan.egress_plan.routes:
env.extend(sorted(plan.egress_plan.token_env_map.keys()))
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
@@ -420,6 +423,7 @@ def _agent_env_entries(
env.append(f"{name}={value}")
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
return tuple(env)
@@ -68,6 +68,11 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
_ensure_builder_dns()
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
if dockerfile:
# `container build` resolves -f relative to the current working
# directory, not the build context. Anchor a relative Dockerfile to
# the context so builds work from any cwd.
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args.extend(["-f", dockerfile])
args.append(context)
subprocess.run(args, check=True)
+6 -5
View File
@@ -23,7 +23,9 @@ from typing import Callable, Generator
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
@@ -228,6 +230,9 @@ def _discover_urls(
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
name, value = entry.split("=", 1)
guest_env[name] = value
return dataclasses.replace(
plan,
@@ -316,11 +321,7 @@ def _bundle_launch_spec(
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
if ep.routes:
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
# Bare-name entries for upstream-token slots. Their values
# come from the docker-run subprocess env (inherited from
# the operator's shell), never landing on argv.
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate ---------------------------------------------
gp = plan.git_gate_plan
+10 -16
View File
@@ -1,8 +1,7 @@
"""Per-bottle persistent state (PRD 0016).
"""Per-bottle persistent state.
Holds the per-bottle Dockerfile override that capability-block
remediation writes, the transcript snapshot the state-preservation
helper saves before teardown, and the launch metadata that lets
Holds optional per-bottle Dockerfile overrides, the transcript snapshot
the state-preservation helper saves before teardown, and the launch metadata that lets
`cli.py resume <identity>` reconstruct a bottle's spec. State
lives at:
@@ -61,7 +60,7 @@ _METADATA_NAME = "metadata.json"
_LIVE_CONFIG_SUBDIR = "live-config"
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
# Empty marker file. capability_apply writes it before teardown so
# Empty marker file. Session preservation writes it before teardown so
# cli.py's session-end cleanup knows to preserve the state dir for
# `cli.py resume <identity>`. Absent = clean up.
_PRESERVE_MARKER = ".preserve"
@@ -164,8 +163,7 @@ def per_bottle_dockerfile_path(identity: str) -> Path:
def per_bottle_dockerfile(identity: str) -> str | None:
"""Return the per-bottle Dockerfile content if present, else
None. None means: use the repo's Dockerfile (the original
pre-capability-block behavior)."""
None. None means: use the provider or manifest Dockerfile."""
p = per_bottle_dockerfile_path(identity)
if p.is_file():
return p.read_text()
@@ -249,9 +247,7 @@ def write_live_config(
def transcript_snapshot_dir(identity: str) -> Path:
"""Where capability_apply stashes the agent's transcript before
teardown, so the next `cli.py start <agent>` can offer to
resume from it."""
"""Where agent session snapshots are kept for resume flows."""
return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR
@@ -278,8 +274,7 @@ def git_gate_state_dir(identity: str) -> Path:
def supervise_state_dir(identity: str) -> Path:
"""State subdir for the supervise sidecar's current-config dir
(bind-mounted into the agent at /etc/bot-bottle/current-config).
"""State subdir reserved for supervise sidecar bind-mount sources.
The queue dir is intentionally NOT under here it lives at
~/.bot-bottle/queue/<slug>/ alongside the audit logs, so it
survives state-dir cleanup."""
@@ -301,9 +296,8 @@ def preserve_marker_path(identity: str) -> Path:
def mark_preserved(identity: str) -> Path:
"""Mark this bottle's state for preservation across session
teardown. Written by capability_apply.apply_capability_change so
cli.py's session-end cleanup leaves the state dir intact for a
subsequent `cli.py resume`."""
teardown so cli.py's session-end cleanup leaves the state dir
intact for a subsequent `cli.py resume`."""
path = preserve_marker_path(identity)
path.parent.mkdir(parents=True, exist_ok=True)
path.touch()
@@ -316,7 +310,7 @@ def is_preserved(identity: str) -> bool:
def clear_preserve_marker(identity: str) -> None:
"""Idempotent removal. Called at fresh launch (start or resume)
so a marker left from a prior capability-block doesn't keep
so a marker left from a prior preserved session doesn't keep
state alive past the next normal session-end."""
try:
preserve_marker_path(identity).unlink()
+1 -4
View File
@@ -1,6 +1,6 @@
"""Main CLI dispatcher.
Commands: cleanup, commit, doctor, edit, info, init, list, resume, start, supervise
Commands: cleanup, commit, edit, info, init, list, resume, start, supervise
"""
from __future__ import annotations
@@ -13,7 +13,6 @@ from ._common import PROG
from . import list as _list_mod
from .cleanup import cmd_cleanup
from .commit import cmd_commit
from .doctor import cmd_doctor
from .edit import cmd_edit
from .info import cmd_info
from .init import cmd_init
@@ -26,7 +25,6 @@ cmd_list = _list_mod.cmd_list
COMMANDS = {
"cleanup": cmd_cleanup,
"commit": cmd_commit,
"doctor": cmd_doctor,
"edit": cmd_edit,
"info": cmd_info,
"init": cmd_init,
@@ -42,7 +40,6 @@ def usage() -> None:
sys.stderr.write("Commands:\n")
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
sys.stderr.write(" doctor check Python, Docker, and bot-bottle config prerequisites\n")
sys.stderr.write(" edit open an agent in vim for editing\n")
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
+1 -1
View File
@@ -6,7 +6,7 @@ import os
import sys
from pathlib import Path
PROG = Path(sys.argv[0]).name or "bot-bottle"
PROG = "cli.py"
USER_CWD = os.getcwd()
REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
+2 -3
View File
@@ -13,9 +13,8 @@ dirs are shared layout, so docker is the single owner of that
bucket.
State dirs with `.preserve` are intentionally never touched they
hold capability-block rebuilds or crash snapshots the operator may
want to `resume`. Manual `rm -rf ~/.bot-bottle/state/<identity>`
is the path for those.
hold preserved sessions the operator may want to `resume`. Manual
`rm -rf ~/.bot-bottle/state/<identity>` is the path for those.
"""
from __future__ import annotations
-73
View File
@@ -1,73 +0,0 @@
"""doctor: validate host prerequisites for running bot-bottle."""
from __future__ import annotations
import argparse
import shutil
import subprocess
import sys
from pathlib import Path
from ._common import PROG
def _ok(label: str, detail: str) -> None:
print(f"ok: {label}: {detail}")
def _fail(label: str, detail: str) -> None:
print(f"fail: {label}: {detail}")
def _check_python() -> bool:
version = sys.version_info
detail = f"{version.major}.{version.minor}.{version.micro}"
if version >= (3, 11):
_ok("python", detail)
return True
_fail("python", f"{detail}; need 3.11 or newer")
return False
def _check_docker() -> bool:
docker = shutil.which("docker")
if not docker:
_fail("docker", "docker command not found")
return False
try:
result = subprocess.run(
[docker, "info"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
timeout=10,
)
except (OSError, subprocess.TimeoutExpired) as exc:
_fail("docker", f"daemon check failed: {exc}")
return False
if result.returncode == 0:
_ok("docker", "daemon reachable")
return True
_fail("docker", "daemon not reachable")
return False
def _check_config_dir() -> bool:
config = Path.home() / ".bot-bottle"
if config.is_dir():
_ok("config", str(config))
return True
_fail("config", f"{config} does not exist")
return False
def cmd_doctor(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} doctor", add_help=True)
parser.parse_args(argv)
checks = (
_check_python(),
_check_docker(),
_check_config_dir(),
)
return 0 if all(checks) else 1
+4 -7
View File
@@ -4,13 +4,12 @@ Reads ~/.bot-bottle/state/<identity>/metadata.json to recover the
(agent_name, cwd, copy_cwd) the bottle was originally started with,
then runs the same launch core as `start` but pinned to the
recorded identity so the new bottle picks up any per-bottle Dockerfile
(from capability-block apply) and transcript snapshot under the same
state dir.
override and transcript snapshot under the same state dir.
Use case: an agent calls capability-block, the dashboard approves
and tears down the bottle, the operator runs
Use case: an interrupted or preserved bottle needs to be relaunched;
the operator runs
./cli.py resume <identity>
to bring up the replacement with the new capabilities baked in.
to bring up the replacement from the recorded state.
"""
from __future__ import annotations
@@ -28,7 +27,6 @@ from .start import _launch_bottle
def cmd_resume(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} resume", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"identity",
help="bottle identity from a prior `start` (see its session-end output)",
@@ -56,6 +54,5 @@ def cmd_resume(argv: list[str]) -> int:
return _launch_bottle(
spec,
dry_run=args.dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
+6 -17
View File
@@ -31,7 +31,6 @@ from ..bottle_state import (
is_preserved,
mark_preserved,
)
# from ..backend.docker.capability_apply import snapshot_transcript
from ..log import info
from ..manifest import ManifestIndex
from ._common import PROG, USER_CWD, read_tty_line
@@ -42,7 +41,6 @@ def cmd_start(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"--backend",
choices=known_backend_names(),
@@ -89,7 +87,6 @@ def cmd_start(argv: list[str]) -> int:
return _launch_bottle(
spec,
dry_run=dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
@@ -134,7 +131,7 @@ def prepare_with_preflight(
def attach_agent(
bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
bottle: Bottle, *, resume: bool = False,
agent_provider_template: str = "claude",
startup_args: tuple[str, ...] = (),
) -> int:
@@ -153,8 +150,6 @@ def attach_agent(
"(Ctrl-D or 'exit' to leave; container will be removed)"
)
agent_args = list(runtime.bypass_args)
if remote_control:
agent_args.extend(runtime.remote_control_args)
agent_args.extend(startup_args)
if resume:
agent_args.extend(runtime.resume_args)
@@ -218,9 +213,9 @@ def _text_prompt_yes() -> bool:
return reply in ("y", "Y", "yes", "YES")
def _text_render_preflight(*, remote_control: bool):
def _text_render_preflight():
def _render(plan: DockerBottlePlan) -> None:
plan.print(remote_control=remote_control)
plan.print()
return _render
@@ -228,7 +223,6 @@ def _launch_bottle(
spec: BottleSpec,
*,
dry_run: bool,
remote_control: bool,
backend_name: str | None = None,
) -> int:
"""Shared launch core for `start` and `resume`. Builds the plan,
@@ -240,7 +234,7 @@ def _launch_bottle(
plan, identity = prepare_with_preflight(
spec,
stage_dir=stage_dir,
render_preflight=_text_render_preflight(remote_control=remote_control),
render_preflight=_text_render_preflight(),
prompt_yes=_text_prompt_yes,
dry_run=dry_run,
backend_name=backend_name,
@@ -253,7 +247,6 @@ def _launch_bottle(
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
exit_code = attach_agent(
bottle,
remote_control=remote_control,
agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args,
)
@@ -263,12 +256,8 @@ def _launch_bottle(
)
# While the container is still alive: always snapshot the
# transcript and — if the agent exited non-zero — mark
# the state for preservation. Capability-block already
# did both before triggering teardown from the dashboard;
# this picks up crashes / Ctrl-Cs / OOM kills the same
# way. snapshot_transcript is best-effort so the
# capability-block path's prior snapshot isn't clobbered
# when the container is already gone.
# the state for preservation. This picks up crashes /
# Ctrl-Cs / OOM kills before cleanup removes the state dir.
if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code)
return 0
+28 -46
View File
@@ -2,9 +2,8 @@
act on them (approve / modify / reject).
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
approval handler wires to PRD 0016 (capability-block), which rebuilds
the bottle Dockerfile. Egress proposals are queued for operator review
as full routes.yaml updates.
Egress proposals are queued for operator review as full routes.yaml
updates.
"""
from __future__ import annotations
@@ -22,10 +21,6 @@ from pathlib import Path
from .. import supervise as _supervise
from ..bottle_state import read_metadata
# from ..backend.docker.capability_apply import (
# CapabilityApplyError,
# apply_capability_change,
# )
from ..backend.docker.egress_apply import (
EgressApplyError,
applicator as _docker_applicator,
@@ -38,10 +33,6 @@ from ..backend.smolmachines.egress_apply import (
)
from ..log import Die, error, info
class CapabilityApplyError(RuntimeError):
"""Placeholder while capability_apply is disabled."""
from ..supervise import (
COMPONENT_FOR_TOOL,
AuditEntry,
@@ -50,11 +41,10 @@ from ..supervise import (
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOL_CAPABILITY_BLOCK,
TOOL_ALLOW,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
archive_proposal,
TOOL_EGRESS_TOKEN_ALLOW,
list_pending_proposals,
render_diff,
write_audit_entry,
@@ -65,6 +55,11 @@ from ._common import PROG
_REFRESH_INTERVAL_MS = 1000
# Proposal tools whose payload is a read-only report, not a file the operator
# edits: modify is unavailable and approval requires a recorded reason for the
# audit trail.
_REPORT_ONLY_TOOLS: tuple[str, ...] = (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW)
@dataclass(frozen=True)
class QueuedProposal:
@@ -77,7 +72,7 @@ class QueuedProposal:
# Errors any remediation engine may raise. Caught by the TUI key
# handlers and surfaced in the status line so a failed apply keeps
# the proposal pending rather than crashing curses.
ApplyError = (CapabilityApplyError, EgressApplyError)
ApplyError = (EgressApplyError,)
def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
@@ -137,11 +132,9 @@ def _detail_lines(
def _suffix_for_tool(tool: str) -> str:
if tool == TOOL_CAPABILITY_BLOCK:
return ".dockerfile"
if tool in (TOOL_ALLOW, TOOL_EGRESS_BLOCK):
if tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
return ".yaml"
if tool == TOOL_GITLEAKS_ALLOW:
if tool in (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW):
return ".txt"
return ".txt"
@@ -160,18 +153,7 @@ def approve(
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
diff_before, diff_after = "", ""
# if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
# _meta = read_metadata(qp.proposal.bottle_slug)
# if _meta is not None and not _meta.compose_project:
# raise CapabilityApplyError(
# "capability-block remediation is not supported for smolmachines "
# "bottles. Reject this proposal or handle the capability change "
# "manually, then restart the bottle."
# )
# diff_before, diff_after = apply_capability_change(
# qp.proposal.bottle_slug, file_to_apply,
# )
if qp.proposal.tool in (TOOL_ALLOW, TOOL_EGRESS_BLOCK):
if qp.proposal.tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
diff_before, diff_after = apply_routes_change(
qp.proposal.bottle_slug,
file_to_apply,
@@ -188,9 +170,6 @@ def approve(
qp, action=status, notes=notes,
diff_before=diff_before, diff_after=diff_after,
)
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
archive_proposal(qp.queue_dir, qp.proposal.id)
def reject(qp: QueuedProposal, *, reason: str) -> None:
"""Write a rejection response and an audit entry."""
@@ -212,8 +191,8 @@ def _approve_from_tui(
notes: str = "",
) -> str:
"""Approve from curses, prompting for any tool-specific audit note."""
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW and final_file is None:
notes = _prompt(stdscr, "allow reason (test fixture/false positive): ")
if qp.proposal.tool in _REPORT_ONLY_TOOLS and final_file is None:
notes = _prompt(stdscr, "allow reason (false positive / legitimately needed): ")
if not notes:
return "approve aborted (empty reason)"
approve(qp, final_file=final_file, notes=notes)
@@ -292,7 +271,10 @@ def cmd_supervise(argv: list[str]) -> int:
return e.code if isinstance(e.code, int) else 1
except Exception as e: # noqa: W0718 — catch supervise crash for logging
log_path = _write_crash_log(e)
error(f"supervise crashed: {type(e).__name__}: {e}")
error(
f"supervise crashed: {type(e).__name__}: {e}",
context={"error_type": type(e).__name__, "crash_log": str(log_path)},
)
error(f"full traceback written to {log_path}")
return 1
return 0
@@ -337,7 +319,7 @@ def _list_once() -> int:
return 0
def _try_init_green() -> int:
def _try_init_green() -> int: # pragma: no cover
"""Initialise a green color pair and return its attr, or 0."""
try:
curses.start_color()
@@ -348,7 +330,7 @@ def _try_init_green() -> int:
return 0
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore # pragma: no cover
curses.curs_set(0)
stdscr.timeout(_REFRESH_INTERVAL_MS)
green_attr = _try_init_green()
@@ -408,8 +390,8 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
except ApplyError as e:
status_line = f"apply failed: {e}"
elif key == ord("m"):
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW:
status_line = "modify unavailable for gitleaks-allow"
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
status_line = f"modify unavailable for {qp.proposal.tool}"
continue
edited = _modify(stdscr, qp)
if edited is None:
@@ -438,7 +420,7 @@ def _render(
status_line: str,
*,
green_attr: int = 0, # noqa: F841 — unused, but required by interface
) -> None:
) -> None: # pragma: no cover
stdscr.erase()
h, w = stdscr.getmaxyx()
header = f"bot-bottle supervise ({len(pending)} pending)"
@@ -489,7 +471,7 @@ def _detail_view(
qp: QueuedProposal,
*,
green_attr: int = 0,
) -> None:
) -> None: # pragma: no cover
"""Render the full proposal. Scrollable. Press q to return."""
lines = _detail_lines(qp, green_attr=green_attr)
offset = 0
@@ -522,7 +504,7 @@ def _detail_view(
pass
return
elif key == ord("m"):
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW:
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
return
edited = _modify(stdscr, qp)
if edited is not None:
@@ -541,7 +523,7 @@ def _detail_view(
return
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore # pragma: no cover
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
suffix = _suffix_for_tool(qp.proposal.tool)
curses.endwin()
@@ -552,7 +534,7 @@ def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
return edited
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore # pragma: no cover
"""One-line input at the bottom of the screen."""
curses.curs_set(1)
h, _ = stdscr.getmaxyx()
+1 -1
View File
@@ -21,7 +21,7 @@ FROM node:22-slim
# to it) works against egress's bumped TLS without the agent needing
# local DNS.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl \
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by claude-code itself
+4 -2
View File
@@ -20,6 +20,7 @@ from ...agent_provider import (
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...backend.docker import util as docker_mod
from ...egress import EgressRoute
@@ -90,7 +91,6 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",),
remote_control_args=("--remote-control",),
)
@@ -115,8 +115,9 @@ class ClaudeAgentProvider(AgentProvider):
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del forward_host_credentials, host_env, provider_settings
del forward_host_credentials, host_env
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
@@ -199,6 +200,7 @@ class ClaudeAgentProvider(AgentProvider):
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=dirs,
files=tuple(files),
egress_routes=egress_routes,
+9 -6
View File
@@ -1,12 +1,12 @@
# bot-bottle Codex provider image.
#
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed globally.
# non-root node user, and the provider CLI installed for that user.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl \
&& apt-get install -y --no-install-recommends git ca-certificates curl procps ripgrep \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by codex itself
@@ -17,12 +17,15 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
&& npm cache clean --force
USER node
WORKDIR /home/node
RUN mkdir -p /home/node/.codex
ENV PATH="/home/node/.local/bin:${PATH}"
# Remote-control support requires the standalone Codex install layout
# under ~/.codex/packages/standalone/current. The npm package can run
# the TUI, but remote-control commands expect this installer-owned path.
RUN mkdir -p /home/node/.codex \
&& curl -fsSL https://chatgpt.com/codex/install.sh | sh
CMD ["codex"]
+4 -2
View File
@@ -22,6 +22,7 @@ from ...agent_provider import (
AgentProvisionCommand,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
@@ -54,7 +55,6 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"),
remote_control_args=(),
)
@@ -79,8 +79,9 @@ class CodexAgentProvider(AgentProvider):
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, label, color, provider_settings
del auth_token, label, color
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
@@ -163,6 +164,7 @@ class CodexAgentProvider(AgentProvider):
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=tuple(dirs),
files=tuple(files),
pre_copy=tuple(pre_copy),
@@ -19,7 +19,12 @@ import urllib.error
import urllib.request
from pathlib import Path
from ...deploy_key_provisioner import DeployKeyProvisioner
from ...deploy_key_provisioner import DeployKeyCollisionError, DeployKeyProvisioner
# Timeout for ssh-keygen and Gitea API HTTP calls. A hung Gitea instance at
# prepare time would stall bottle launch indefinitely without this bound.
_API_TIMEOUT_SECS = 30
_KEYGEN_TIMEOUT_SECS = 10
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
@@ -46,6 +51,7 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=_KEYGEN_TIMEOUT_SECS,
)
private_key = key_path.read_bytes()
public_key = key_path.with_suffix(".pub").read_text().strip()
@@ -67,10 +73,15 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
method="POST",
)
try:
with urllib.request.urlopen(req) as resp:
with urllib.request.urlopen(req, timeout=_API_TIMEOUT_SECS) as resp:
body = json.loads(resp.read())
except urllib.error.HTTPError as exc:
_body = _read_error_body(exc)
if exc.code == 422:
raise DeployKeyCollisionError(
f"deploy key collision for {owner_repo!r} "
f"(title={title!r}): key title or content already registered — {_body}"
) from exc
raise RuntimeError(
f"failed to create deploy key for {owner_repo}: "
f"HTTP {exc.code}{_body}"
@@ -93,7 +104,7 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
method="DELETE",
)
try:
with urllib.request.urlopen(req):
with urllib.request.urlopen(req, timeout=_API_TIMEOUT_SECS):
pass
except urllib.error.HTTPError as exc:
if exc.code == 404:
+3 -1
View File
@@ -21,6 +21,7 @@ from ...agent_provider import (
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...egress import EgressRoute
from ...log import die, info
@@ -165,7 +166,6 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="append_system_prompt",
bypass_args=(),
resume_args=(),
remote_control_args=(),
)
@@ -199,6 +199,7 @@ class PiAgentProvider(AgentProvider):
models_payload, base_url, api_key_env, models, provider_name = (
_pi_models_json(settings)
)
extra_startup_args = provider_startup_args(provider_settings)
models_file = state_dir / "pi-models.json"
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
models_file.chmod(0o600)
@@ -219,6 +220,7 @@ class PiAgentProvider(AgentProvider):
startup_args=(
"--models",
",".join(f"{provider_name}/{model}" for model in models),
*extra_startup_args,
),
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
+4
View File
@@ -11,6 +11,10 @@ from __future__ import annotations
from abc import ABC, abstractmethod
class DeployKeyCollisionError(RuntimeError):
"""Raised when a deploy key title or public key already exists on the repo."""
class DeployKeyProvisioner(ABC):
"""Manages a single deploy-key lifecycle on a remote forge."""
+190 -7
View File
@@ -15,6 +15,8 @@ import gzip
import re
import typing
import unicodedata
from math import log2
from collections import Counter
from urllib.parse import quote as url_quote
try:
@@ -78,16 +80,27 @@ TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
)
def scan_token_patterns(text: str, *, location: str = "body") -> ScanResult | None:
def scan_token_patterns(
text: str,
*,
location: str = "body",
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
normalized = _normalize_text(text)
for name, pattern in TOKEN_PATTERNS:
m = pattern.search(normalized)
if m is not None:
for m in pattern.finditer(normalized):
value = m.group(0)
# A value the supervisor has approved (PRD 0062) is no longer a
# block — keep scanning so a second, un-approved token in the
# same request is still caught.
if safe_tokens is not None and value in safe_tokens:
continue
return ScanResult(
severity="block",
reason=f"{name} found in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
context=_snippet(normalized, m.start(), m.end()),
matched=value,
)
return None
@@ -96,20 +109,21 @@ def redact_tokens(
text: str,
*,
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> str:
"""Replace token pattern matches and (if env given) provisioned secrets with REDACT."""
for _, pattern in TOKEN_PATTERNS:
text = pattern.sub(REDACT, text)
if env is not None:
for key, value in env.items():
if key.startswith("EGRESS_TOKEN_") and value:
if any(key.startswith(p) for p in sensitive_prefixes) and value:
for variant in _encoded_variants(value):
text = text.replace(variant, REDACT)
return text
# ---------------------------------------------------------------------------
# Known secrets detector (Phase 1b)
# Known secrets detector
# ---------------------------------------------------------------------------
def _encoded_variants(secret: str) -> list[str]:
@@ -150,26 +164,179 @@ def _encoded_variants(secret: str) -> list[str]:
return variants
# ---------------------------------------------------------------------------
# Fragmentation-resistant helpers
# ---------------------------------------------------------------------------
# Minimum length of alnum projection for projection-based checks to run.
# Short secrets produce too many false positives in projection space.
_ALNUM_MIN_LEN = 8
# Minimum window length for the partial-substring sliding scan.
PARTIAL_MATCH_MIN_LEN = 12
def _alnum_projection(text: str) -> str:
"""Return text with every non-alphanumeric character stripped.
Used for fragmentation-resistant matching: separator-injected secrets
(spaces, hyphens, dots inserted between characters) are identical to
their originals in alnum projection space.
"""
return "".join(c for c in text if c.isalnum())
def _find_partial_window(secret_alnum: str, text_alnum: str, min_len: int) -> int | None:
"""Return the position in text_alnum where any min_len-char window of
secret_alnum first appears, or None.
Slides a window of width min_len across secret_alnum and searches for
each window in text_alnum. The first hit position is returned.
"""
if len(secret_alnum) < min_len or len(text_alnum) < min_len:
return None
for i in range(len(secret_alnum) - min_len + 1):
window = secret_alnum[i:i + min_len]
pos = text_alnum.find(window)
if pos >= 0:
return pos
return None
def scan_known_secrets(
text: str,
*,
location: str = "body",
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
if env is None:
return None
# Pre-compute alnum projection of the scan text once; reused per secret.
text_alnum: str | None = None
for key, value in env.items():
if not key.startswith("EGRESS_TOKEN_") or not value:
if not any(key.startswith(p) for p in sensitive_prefixes) or not value:
continue
# Pass 1: exact match across encoded variants (original behaviour).
approved_exact = False
for variant in _encoded_variants(value):
pos = text.find(variant)
if pos >= 0:
# The supervisor approves the exact encoded variant found
# (PRD 0062); a different encoding of the same secret is a
# fresh block.
if safe_tokens is not None and variant in safe_tokens:
approved_exact = True
continue
return ScanResult(
severity="block",
reason=f"provisioned secret from {key} found in {location}",
location=location,
context=_snippet(text, pos, pos + len(variant)),
matched=variant,
)
if approved_exact:
# Exact match was found and approved; projection passes would
# fire on the same value, so skip them for this secret.
continue
# Pass 2 & 3: fragmentation-resistant projection checks.
secret_alnum = _alnum_projection(value)
if len(secret_alnum) < _ALNUM_MIN_LEN:
continue
if text_alnum is None:
text_alnum = _alnum_projection(text)
# Pass 2: full alnum-projection exact match (catches separator injection).
pos2 = text_alnum.find(secret_alnum)
if pos2 >= 0:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(fragmented match — separator injection)"
),
location=location,
context=_snippet(text_alnum, pos2, pos2 + len(secret_alnum)),
)
# Pass 3: sliding-window partial match (catches chunked-substring leaks).
pos3 = _find_partial_window(secret_alnum, text_alnum, PARTIAL_MATCH_MIN_LEN)
if pos3 is not None:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(partial match — at least {PARTIAL_MATCH_MIN_LEN} consecutive "
f"alphanumeric chars)"
),
location=location,
context=_snippet(text_alnum, pos3, pos3 + PARTIAL_MATCH_MIN_LEN),
)
return None
# ---------------------------------------------------------------------------
# Entropy detector (warn-only)
# ---------------------------------------------------------------------------
# Sliding window size and step for the entropy scan.
ENTROPY_WINDOW = 64
ENTROPY_STEP = 32
# Bits-per-character threshold. Random ASCII printable ≈ 6.6 bits; random
# lowercase hex ≈ 4 bits; random base64url ≈ 6 bits. 5.5 sits above
# typical structured data (JSON, URLs) while staying below truly random
# content.
ENTROPY_BLOCK_THRESHOLD = 5.5
def _shannon_entropy(text: str) -> float:
if not text:
return 0.0
counts = Counter(text)
n = len(text)
return -sum((c / n) * log2(c / n) for c in counts.values())
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW,
threshold: float = ENTROPY_BLOCK_THRESHOLD,
) -> ScanResult | None:
"""Warn-only detector: flag windows of `window` chars with Shannon entropy
above `threshold` bits per character.
Never blocks; always returns severity='warn'. Disabled by default
routes must opt in via dlp.outbound_detectors=['entropy'].
"""
if not text:
return None
step = max(1, window // 2)
end = len(text)
# Scan overlapping windows; also check the final tail if shorter than window.
positions = list(range(0, end - window + 1, step))
if end < window:
positions = [0]
elif (end - window) % step != 0:
positions.append(end - window)
for i in positions:
chunk = text[i:i + window]
if _shannon_entropy(chunk) >= threshold:
return ScanResult(
severity="warn",
reason=f"high-entropy content in {location} (possible encrypted exfil)",
location=location,
context=_snippet(text, i, i + len(chunk)),
)
return None
@@ -265,6 +432,14 @@ _CRLF_ENCODED_RE = re.compile(r"%0[dD]%0[aA]", re.ASCII)
_CRLF_HEADER_INJECT_RE = re.compile(r"\r\n[A-Za-z][A-Za-z0-9\-]+\s*:", re.ASCII)
def strip_crlf(text: str) -> str:
"""Remove URL-encoded and literal CRLF injection sequences from a request
surface (PRD 0062 redact policy). Used to scrub the request line / headers
so the request can be forwarded instead of hard-blocked."""
text = _CRLF_ENCODED_RE.sub("", text)
return _CRLF_HEADER_INJECT_RE.sub(lambda m: m.group(0)[2:], text)
def scan_crlf_injection(text: str) -> ScanResult | None:
if _CRLF_ENCODED_RE.search(text):
return ScanResult(
@@ -280,12 +455,20 @@ def scan_crlf_injection(text: str) -> ScanResult | None:
__all__ = [
"ENTROPY_BLOCK_THRESHOLD",
"ENTROPY_WINDOW",
"ENTROPY_STEP",
"PARTIAL_MATCH_MIN_LEN",
"REDACT",
"SNIPPET_CONTEXT",
"TOKEN_PATTERNS",
"_alnum_projection",
"_shannon_entropy",
"redact_tokens",
"scan_crlf_injection",
"scan_entropy",
"scan_known_secrets",
"scan_naive_injection",
"scan_token_patterns",
"strip_crlf",
]
+102 -11
View File
@@ -10,12 +10,14 @@ specific and lives on concrete subclasses (see
from __future__ import annotations
import dataclasses
import secrets
from abc import ABC
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
from .egress_addon_core import (
ON_MATCH_REDACT,
HeaderMatch as CoreHeaderMatch,
MatchEntry as CoreMatchEntry,
PathMatch as CorePathMatch,
@@ -33,6 +35,50 @@ EGRESS_HOSTNAME = "egress"
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
EGRESS_ROUTES_FILENAME = Path(EGRESS_ROUTES_IN_CONTAINER).name
_CANARY_ENV_WORDS = (
"ACCORD",
"ANCHOR",
"ATLAS",
"CANON",
"CIPHER",
"EMBER",
"FALCON",
"HARBOR",
"LANTERN",
"MARBLE",
"NOVA",
"ORBIT",
"PIVOT",
"RADIUS",
"SUMMIT",
"VECTOR",
)
def _random_canary_env() -> str:
first = secrets.choice(_CANARY_ENV_WORDS)
remaining = tuple(word for word in _CANARY_ENV_WORDS if word != first)
second = secrets.choice(remaining)
return f"{first}_{second}_SECRET"
def egress_sidecar_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return sidecar env entries needed by egress across all backends."""
env: list[str] = []
if plan.routes:
env.extend(sorted(plan.token_env_map.keys()))
if plan.canary and plan.canary_env:
env.append(f"{plan.canary_env}={plan.canary}")
env.append(f"BOT_BOTTLE_SENSITIVE_PREFIXES={plan.canary_env}")
return tuple(env)
def egress_agent_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return agent-visible egress env entries shared by all backends."""
if plan.canary and plan.canary_env:
return (f"{plan.canary_env}={plan.canary}",)
return ()
@dataclass(frozen=True)
class EgressRoute(Route):
@@ -64,6 +110,8 @@ class EgressPlan:
mitmproxy_ca_host_path: Path = Path()
mitmproxy_ca_cert_only_host_path: Path = Path()
log: int = 0
canary: str = ""
canary_env: str = ""
def egress_manifest_routes(
@@ -95,6 +143,7 @@ def egress_manifest_routes(
git_fetch=r.GitFetch,
outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors,
outbound_on_match=r.OutboundOnMatch,
))
return tuple(out)
@@ -105,12 +154,27 @@ def egress_routes_for_bottle(
) -> tuple[EgressRoute, ...]:
manifest = egress_manifest_routes(bottle)
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
merged = list(provider_routes) + [
merged = list(_default_provider_on_match(provider_routes)) + [
r for r in manifest if r.host.lower() not in provisioned_hosts
]
return _assign_token_slots(merged)
def _default_provider_on_match(
provider_routes: tuple[EgressRoute, ...],
) -> tuple[EgressRoute, ...]:
"""Provider routes (the agent talking to its own LLM API) default to the
`redact` on-match policy (PRD 0062): high-volume conversation payloads are
the worst source of token-shaped false positives, so a match is scrubbed
and forwarded rather than hard-blocked or queued for the operator. A
provider that sets `outbound_on_match` explicitly keeps its choice."""
return tuple(
r if r.outbound_on_match
else dataclasses.replace(r, outbound_on_match=ON_MATCH_REDACT)
for r in provider_routes
)
def _assign_token_slots(
routes: list[EgressRoute],
) -> tuple[EgressRoute, ...]:
@@ -146,6 +210,17 @@ def egress_token_env_map(
return out
def _yaml_str_escape(s: str) -> str:
"""Escape a string for use inside a YAML double-quoted scalar."""
return (
s.replace("\\", "\\\\")
.replace('"', '\\"')
.replace("\n", "\\n")
.replace("\r", "\\r")
.replace("\t", "\\t")
)
def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields: dict[str, object] = {"host": r.host}
if r.auth_scheme and r.token_env:
@@ -177,7 +252,11 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields["matches"] = matches_data
if r.git_fetch:
fields["git"] = {"fetch": True}
if r.outbound_detectors is not None or r.inbound_detectors is not None:
if (
r.outbound_detectors is not None
or r.inbound_detectors is not None
or r.outbound_on_match
):
dlp: dict[str, object] = {}
if r.outbound_detectors is not None:
dlp["outbound_detectors"] = (
@@ -189,6 +268,8 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
False if not r.inbound_detectors
else list(r.inbound_detectors)
)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
fields["dlp"] = dlp
return fields
@@ -202,12 +283,12 @@ def _render_match_entry(entry: dict[str, object]) -> list[str]:
for pd in entry["paths"]: # type: ignore[union-attr]
pd_dict: dict[str, str] = pd # type: ignore[assignment]
if "type" in pd_dict:
lines.append(f' - type: "{pd_dict["type"]}"')
lines.append(f' value: "{pd_dict["value"]}"')
lines.append(f' - type: "{_yaml_str_escape(pd_dict["type"])}"')
lines.append(f' value: "{_yaml_str_escape(pd_dict["value"])}"')
else:
lines.append(f' - value: "{pd_dict["value"]}"')
lines.append(f' - value: "{_yaml_str_escape(pd_dict["value"])}"')
if "methods" in entry:
methods_str = ", ".join(f'"{m}"' for m in entry["methods"]) # type: ignore[union-attr]
methods_str = ", ".join(f'"{_yaml_str_escape(m)}"' for m in entry["methods"]) # type: ignore[union-attr]
prefix = " - " if first_key else " "
lines.append(f'{prefix}methods: [{methods_str}]')
first_key = False
@@ -217,8 +298,8 @@ def _render_match_entry(entry: dict[str, object]) -> list[str]:
first_key = False
for hd in entry["headers"]: # type: ignore[union-attr]
hd_dict: dict[str, str] = hd # type: ignore[assignment]
lines.append(f' - name: "{hd_dict["name"]}"')
lines.append(f' value: "{hd_dict["value"]}"')
lines.append(f' - name: "{_yaml_str_escape(hd_dict["name"])}"')
lines.append(f' value: "{_yaml_str_escape(hd_dict["value"])}"')
if first_key:
lines.append(" - {}")
return lines
@@ -238,10 +319,10 @@ def egress_render_routes(
return "\n".join(lines) + "\n"
for r in routes:
f = _route_to_yaml_fields(r)
lines.append(f' - host: "{f["host"]}"')
lines.append(f' - host: "{_yaml_str_escape(str(f["host"]))}"')
if "auth_scheme" in f:
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
lines.append(f' token_env: "{f["token_env"]}"')
lines.append(f' auth_scheme: "{_yaml_str_escape(str(f["auth_scheme"]))}"')
lines.append(f' token_env: "{_yaml_str_escape(str(f["token_env"]))}"')
if "matches" in f:
lines.append(" matches:")
for entry in f["matches"]: # type: ignore[union-attr]
@@ -260,6 +341,8 @@ def egress_render_routes(
elif isinstance(dv, list):
items_str = ", ".join(f'"{x}"' for x in dv)
lines.append(f" {dk}: [{items_str}]")
elif isinstance(dv, str):
lines.append(f' {dk}: "{_yaml_str_escape(dv)}"')
return "\n".join(lines) + "\n"
@@ -299,12 +382,18 @@ class Egress(ABC):
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
routes_path.write_text(egress_render_routes(routes, log=log))
routes_path.chmod(0o600)
# Generate a per-session fake secret under a plausible random env name.
# The sidecar marks that exact env name as sensitive for known-secret
# scanning; the agent receives the same name/value as exfil bait.
canary = secrets.token_urlsafe(32)
return EgressPlan(
slug=slug,
routes_path=routes_path,
routes=routes,
token_env_map=egress_token_env_map(routes),
log=log,
canary=canary,
canary_env=_random_canary_env(),
)
__all__ = [
@@ -319,5 +408,7 @@ __all__ = [
"egress_render_routes",
"egress_resolve_token_values",
"egress_routes_for_bottle",
"egress_agent_env_entries",
"egress_sidecar_env_entries",
"egress_token_env_map",
]
+282 -22
View File
@@ -5,6 +5,7 @@ egress container."""
from __future__ import annotations
import asyncio
import json
import os
import signal
@@ -16,9 +17,15 @@ from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
LOG_BLOCKS,
LOG_FULL,
DEFAULT_OUTBOUND_ON_MATCH,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
Config,
Route,
ScanResult,
build_inbound_scan_text,
build_outbound_scan_text,
build_token_allow_payload,
decide,
decide_git_fetch,
is_git_fetch_request,
@@ -32,23 +39,55 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
)
try:
from dlp_detectors import redact_tokens # type: ignore[import-not-found]
from dlp_detectors import redact_tokens, strip_crlf # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.dlp_detectors import redact_tokens # type: ignore[import-not-found]
from bot_bottle.dlp_detectors import ( # type: ignore[import-not-found]
redact_tokens,
strip_crlf,
)
try:
import supervise as _sv # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
INTROSPECT_HOST = "_egress.local"
# Seconds the egress proxy holds a token-blocked request open waiting for the
# operator's supervisor decision (PRD 0062), overridable via env.
DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS = 300.0
# Filesystem poll cadence while awaiting the operator's response.
TOKEN_ALLOW_POLL_INTERVAL_SECONDS = 0.5
# Fixed operator guidance attached to every token-allow proposal.
_TOKEN_ALLOW_JUSTIFICATION = (
"egress DLP blocked an outbound request carrying a detected token. "
"Approve only if this value is a false positive or a credential this "
"request legitimately needs; the value is then allowed for the life of "
"this bottle's egress proxy."
)
class EgressAddon:
def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.config: Config = Config(routes=())
# Tokens the operator has approved this session (PRD 0062). In-memory
# only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed.
self.safe_tokens: set[str] = set()
self._supervise_queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "").strip()
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True)
self._install_sighup()
def _supervise_available(self) -> bool:
return bool(self._supervise_queue_dir and self._supervise_slug)
def _reload(self, *, initial: bool = False) -> None:
try:
text = Path(self.routes_path).read_text(encoding="utf-8")
@@ -121,31 +160,42 @@ class EgressAddon:
)
def _log_request(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.request.headers.items()
if k.lower() != "authorization"
}
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": dict(flow.request.headers),
"body": flow.request.get_text(strict=False) or "",
"headers": headers,
"body": body,
})
+ "\n"
)
def _log_response(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.response.headers.items()
}
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": dict(flow.response.headers),
"body": flow.response.get_text(strict=False) or "",
"headers": headers,
"body": body,
})
+ "\n"
)
def request(self, flow: http.HTTPFlow) -> None:
async def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?")
if flow.request.pretty_host == INTROSPECT_HOST:
@@ -157,21 +207,11 @@ class EgressAddon:
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
if route is not None:
body = flow.request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
outbound_scan_headers(route, dict(flow.request.headers)),
body,
)
dlp_result = scan_outbound(route, scan_text, os.environ)
if dlp_result is not None and dlp_result.severity == "block":
ctx = self._req_ctx(flow)
if dlp_result.context:
ctx = {**ctx, "context": dlp_result.context}
self._block(flow, f"egress DLP: {dlp_result.reason}", ctx=ctx)
if not await self._handle_outbound_dlp(flow, route):
return
# The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on.
request_path, _, query = flow.request.path.partition("?")
if is_git_push_request(request_path, query):
self._block(
@@ -221,6 +261,202 @@ class EgressAddon:
if self.config.log >= LOG_FULL:
self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
ctx = self._req_ctx(flow)
if result.context:
ctx = {**ctx, "context": result.context}
self._block(flow, f"egress DLP: {result.reason}", ctx=ctx)
async def _handle_outbound_dlp(
self,
flow: http.HTTPFlow,
route: Route,
) -> bool:
"""Scan the outbound request and apply the route's on-match policy
(PRD 0062). Returns True if the request may be forwarded, False if a
403 response has been written to `flow`.
Loops so the supervise policy can re-scan after each approval a
second, un-approved token in the same request is still caught."""
while True:
request_path, _, query = flow.request.path.partition("?")
body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, body,
)
# CRLF is scanned only over the request line + headers, never the
# body (see scan_outbound) — a body is not an injection vector.
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(
route, scan_text, os.environ,
safe_tokens=self.safe_tokens, crlf_text=crlf_text,
)
if result is None or result.severity != "block":
return True
policy = route.outbound_on_match or DEFAULT_OUTBOUND_ON_MATCH
# redact scrubs every detection (tokens and structural CRLF) and
# forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route):
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_redacted",
"reason": f"egress DLP: {result.reason}",
**self._req_ctx(flow),
}) + "\n")
return True
self._block(
flow,
f"egress DLP: {result.reason}; redaction could not remove "
"all matches (e.g. a match in the hostname)",
ctx=self._req_ctx(flow),
)
return False
# Structural blocks (CRLF, no safelist-able value) cannot be
# supervised — there is nothing to approve and remember — so under
# block/supervise they are a hard 403.
if policy == ON_MATCH_BLOCK or not result.matched:
self._block_dlp(flow, result)
return False
# supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available():
self._block_dlp(flow, result)
return False
approved = await self._supervise_token_block(flow, request_path, result)
if not approved:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound(self, flow: http.HTTPFlow, route: Route) -> bool:
"""Scrub detected tokens (and CRLF injection sequences) from the mutable
request surfaces (body, headers, path/query) and re-scan. Returns True
if the request is now clean; False if a block-severity match remains on
a surface redaction cannot rewrite (the hostname) so the caller fails
closed."""
body = flow.request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=os.environ)
if redacted_body != body:
flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()):
if name.lower() == "host":
continue # routing-critical; never a legitimate token
redacted = strip_crlf(redact_tokens(value, env=os.environ))
if redacted != value:
flow.request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=os.environ))
if redacted_path != flow.request.path:
flow.request.path = redacted_path
request_path, _, query = flow.request.path.partition("?")
new_body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, new_body,
)
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(route, scan_text, os.environ, crlf_text=crlf_text)
return result is None or result.severity != "block"
async def _supervise_token_block(
self,
flow: http.HTTPFlow,
request_path: str,
result: ScanResult,
) -> bool:
"""Route a token DLP block to the operator's supervisor queue and wait.
Returns True if the operator approved (the matched value is added to
`self.safe_tokens` and the caller re-scans); False if the request must
be blocked (a 403 response has been written to `flow`)."""
host = flow.request.pretty_host
payload = build_token_allow_payload(
redact_tokens(host, env=os.environ),
flow.request.method,
redact_tokens(request_path, env=os.environ),
result,
)
proposal = _sv.Proposal.new(
bottle_slug=self._supervise_slug,
tool=_sv.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file=payload,
justification=_TOKEN_ALLOW_JUSTIFICATION,
current_file_hash=_sv.sha256_hex(payload),
)
queue_dir = Path(self._supervise_queue_dir)
try:
_sv.write_proposal(queue_dir, proposal)
except OSError as e:
sys.stderr.write(
f"egress: could not queue token-allow proposal: {e}; "
"blocking request\n"
)
self._block(flow, f"egress DLP: {result.reason}", ctx=self._req_ctx(flow))
return False
sys.stderr.write(json.dumps({
"event": "egress_token_supervise",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
response = await self._await_token_response(queue_dir, proposal.id)
_sv.archive_proposal(queue_dir, proposal.id)
if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
):
self.safe_tokens.add(result.matched)
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_token_allowed",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
return True
if response is None:
reason = (
f"egress DLP: {result.reason}; supervisor approval timed out "
f"after {self._token_allow_timeout:g}s"
)
else:
reason = f"egress DLP: {result.reason}; supervisor rejected the request"
self._block(flow, reason, ctx=self._req_ctx(flow))
return False
async def _await_token_response(
self,
queue_dir: Path,
proposal_id: str,
) -> "_sv.Response | None":
"""Poll the queue dir for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout."""
loop = asyncio.get_running_loop()
deadline = loop.time() + self._token_allow_timeout
while True:
try:
return _sv.read_response(queue_dir, proposal_id)
except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed.
pass
if loop.time() >= deadline:
return None
await asyncio.sleep(TOKEN_ALLOW_POLL_INTERVAL_SECONDS)
def response(self, flow: http.HTTPFlow) -> None:
"""DLP inbound scan on response headers and body."""
route = match_route(self.config.routes, flow.request.pretty_host)
@@ -272,7 +508,12 @@ class EgressAddon:
message = flow.websocket.messages[-1] # type: ignore[union-attr]
content = message.content.decode("utf-8", errors="replace")
if message.from_client:
result = scan_outbound(route, content, os.environ)
# A WebSocket data frame is not an HTTP request line, so CRLF is
# not an injection vector here — scan only for credential leakage.
result = scan_outbound(
route, content, os.environ,
safe_tokens=self.safe_tokens, crlf_text="",
)
if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
@@ -286,4 +527,23 @@ class EgressAddon:
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
def _token_allow_timeout_from_env(env: "os._Environ[str]") -> float:
"""Read EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS; fall back to the default on an
unset or invalid value (a bad value should not wedge egress at boot)."""
raw = env.get("EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS", "").strip()
if not raw:
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
try:
value = float(raw)
except ValueError:
value = 0.0
if value <= 0:
sys.stderr.write(
"egress: invalid EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS="
f"{raw!r}; using default {DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS:g}s\n"
)
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
return value
addons = [EgressAddon()]
+110 -24
View File
@@ -34,9 +34,18 @@ VALID_METHODS = frozenset({
"CONNECT",
})
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets", "entropy"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# Per-route policy for what the proxy does when an outbound DLP detector
# matches a token (PRD 0062).
ON_MATCH_BLOCK = "block" # hard 403, never overridable
ON_MATCH_REDACT = "redact" # scrub the matched value, forward the request
ON_MATCH_SUPERVISE = "supervise" # queue for operator approval, hold the request
OUTBOUND_ON_MATCH_VALUES = (ON_MATCH_BLOCK, ON_MATCH_REDACT, ON_MATCH_SUPERVISE)
# Unset resolves to supervise (fall back to block when supervise is not wired).
DEFAULT_OUTBOUND_ON_MATCH = ON_MATCH_SUPERVISE
@dataclass(frozen=True)
class PathMatch:
@@ -69,6 +78,8 @@ class Route:
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
# "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES.
outbound_on_match: str = ""
LOG_OFF = 0 # no logging
@@ -95,6 +106,11 @@ class ScanResult:
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
matched: str = ""
# ---------------------------------------------------------------------------
@@ -218,12 +234,12 @@ def _parse_detectors(
idx: int,
host: str,
raw_dict: dict[str, object],
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
"""Parse the optional `dlp` block on a route, returning
(outbound_detectors, inbound_detectors)."""
(outbound_detectors, inbound_detectors, outbound_on_match)."""
dlp_raw = raw_dict.get("dlp")
if dlp_raw is None:
return None, None
return None, None, ""
label = f"route[{idx}] ({host})"
if not isinstance(dlp_raw, dict):
raise ValueError(f"{label}: 'dlp' must be an object")
@@ -260,13 +276,24 @@ def _parse_detectors(
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = dlp.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ValueError(
f"{label}: dlp.outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in dlp:
if k not in ("outbound_detectors", "inbound_detectors"):
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ValueError(
f"{label}: dlp has unknown key {k!r}; accepted keys "
f"are 'outbound_detectors', 'inbound_detectors'"
f"are 'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
)
return outbound, inbound
return outbound, inbound, on_match
def parse_routes(payload: object) -> tuple[Route, ...]:
@@ -337,7 +364,7 @@ def _parse_one(idx: int, raw: object) -> Route:
)
# dlp detectors
outbound_detectors, inbound_detectors = _parse_detectors(
outbound_detectors, inbound_detectors, outbound_on_match = _parse_detectors(
idx, host, raw_dict,
)
@@ -356,6 +383,7 @@ def _parse_one(idx: int, raw: object) -> Route:
git_fetch=git_fetch,
outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match,
)
@@ -404,20 +432,13 @@ def route_to_yaml_dict(r: Route) -> dict[str, object]:
dlp["outbound_detectors"] = list(r.outbound_detectors)
if r.inbound_detectors is not None:
dlp["inbound_detectors"] = list(r.inbound_detectors)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
if dlp:
d["dlp"] = dlp
return d
def load_routes(text: str) -> tuple[Route, ...]:
"""Parse YAML text → routes."""
try:
payload = parse_yaml_subset(text)
except YamlSubsetError as e:
raise ValueError(f"routes payload: invalid YAML: {e}") from e
return parse_routes(payload)
def parse_config(payload: object) -> "Config":
"""Parse a full egress config payload (top-level log level + routes)."""
if not isinstance(payload, dict):
@@ -690,43 +711,103 @@ def scan_outbound(
route: Route,
body: str | bytes,
environ: typing.Mapping[str, str],
*,
safe_tokens: typing.AbstractSet[str] | None = None,
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
except ImportError: # pragma: no cover - host-side path
from .dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
# Binary bodies: latin-1 is a bijective byte↔codepoint mapping that
# preserves every byte value, so ASCII-range secret strings remain
# findable by str.find / regex. Prefer strict UTF-8 for valid text bodies.
if isinstance(body, bytes):
try:
text = body.decode("utf-8")
except UnicodeDecodeError:
text = body.decode("latin-1")
else:
text = body
# CRLF injection is never legitimate — runs unconditionally, not gated
# by outbound_detectors config.
result = scan_crlf_injection(text)
# CRLF injection is only an attack in the request line + headers, never the
# body: an HTTP body is delimited by Content-Length, so CRLF bytes there
# cannot split the request. Scanning the body produces false positives on
# legitimate form-encoded / multi-line content. Callers pass the
# body-excluded surfaces as `crlf_text`; `None` falls back to the full text
# for backward-compatible callers (host-side tests, websocket frames).
crlf_target = text if crlf_text is None else crlf_text
result = scan_crlf_injection(crlf_target)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "token_patterns"):
result = scan_token_patterns(text, location="body")
result = scan_token_patterns(text, location="body", safe_tokens=safe_tokens)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "known_secrets"):
result = scan_known_secrets(text, location="body", env=environ)
# BOT_BOTTLE_SENSITIVE_PREFIXES lets operators add extra env prefixes
# beyond EGRESS_TOKEN_* without changing the manifest schema.
extra_raw = environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "")
extra = tuple(p for p in extra_raw.split(",") if p)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
result = scan_known_secrets(
text, location="body", env=environ,
sensitive_prefixes=sensitive_prefixes, safe_tokens=safe_tokens,
)
if result is not None:
return result
# Entropy scanning requires explicit opt-in: it is NOT part of the
# default "all detectors" set because it produces false positives on
# legitimate base64 / binary payloads. Routes must list "entropy" in
# dlp.outbound_detectors to enable it.
if (
route.outbound_detectors is not None
and "entropy" in route.outbound_detectors
):
result = scan_entropy(text, location="body")
if result is not None:
return result
return None
def build_token_allow_payload(
host: str,
method: str,
path: str,
result: ScanResult,
) -> str:
"""Render the human-readable supervisor proposal body for an outbound
token block (PRD 0062). Carries the host/method/path, the detector
reason, and the redacted context snippet never the raw token value."""
lines = [
"egress blocked an outbound request carrying a detected token",
f"host: {host}",
f"method: {method}",
f"path: {path}",
f"detector: {result.reason}",
]
if result.context:
lines.append(f"context: {result.context}")
return "\n".join(lines) + "\n"
def scan_inbound(
route: Route,
body: str | bytes,
@@ -751,6 +832,11 @@ __all__ = [
"route_to_yaml_dict",
"LOG_FULL",
"LOG_OFF",
"ON_MATCH_BLOCK",
"ON_MATCH_REDACT",
"ON_MATCH_SUPERVISE",
"OUTBOUND_ON_MATCH_VALUES",
"DEFAULT_OUTBOUND_ON_MATCH",
"Config",
"Decision",
"HeaderMatch",
@@ -760,13 +846,13 @@ __all__ = [
"ScanResult",
"build_inbound_scan_text",
"build_outbound_scan_text",
"build_token_allow_payload",
"decide",
"decide_git_fetch",
"evaluate_matches",
"is_git_push_request",
"is_git_fetch_request",
"load_config",
"load_routes",
"match_route",
"outbound_scan_headers",
"parse_config",
+17 -6
View File
@@ -43,10 +43,10 @@ from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the sidecar bundle. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate"
# Bound half-open git client sessions. If an agent/tool runner is
# interrupted during push, git daemon should reap the receive-pack
# child instead of keeping the gate wedged indefinitely.
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
# git daemon (--timeout/--init-timeout), the access-hook subprocess in
# git_http_backend, and the git http-backend CGI subprocess.
GIT_GATE_TIMEOUT_SECS = 15
@dataclass(frozen=True)
@@ -112,6 +112,15 @@ def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstre
)
def _gitconfig_validate_value(field: str, value: str) -> None:
"""Raise ValueError if value contains characters that break gitconfig line syntax."""
if "\n" in value or "\r" in value:
raise ValueError(
f"git-gate: {field} contains a newline, which would inject "
f"arbitrary gitconfig keys; rejecting manifest entry"
)
def git_gate_render_gitconfig(
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
@@ -136,6 +145,7 @@ def git_gate_render_gitconfig(
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
]
for entry in entries:
_gitconfig_validate_value(f"repos[{entry.Name!r}].url", entry.Upstream)
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
out.append(f"\tinsteadOf = {entry.Upstream}\n")
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
@@ -148,6 +158,7 @@ def git_gate_render_gitconfig(
f"ssh://{entry.UpstreamUser}@{entry.RemoteKey}{port}/"
f"{entry.UpstreamPath}"
)
_gitconfig_validate_value(f"repos[{entry.Name!r}].url (resolved alias)", alias)
out.append(f"\tinsteadOf = {alias}\n")
return "".join(out)
@@ -217,8 +228,8 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
"",
"exec git daemon \\",
" --reuseaddr \\",
f" --timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
f" --timeout={GIT_GATE_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_TIMEOUT_SECS} \\",
" --base-path=/git \\",
" --export-all \\",
" --enable=receive-pack \\",
+11 -1
View File
@@ -16,6 +16,8 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlsplit
from .git_gate import GIT_GATE_TIMEOUT_SECS
DEFAULT_PORT = 9420
@@ -47,6 +49,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
[hook_path, "upload-pack", str(repo_dir), peer, peer],
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
if hook.returncode != 0:
detail = (hook.stderr or hook.stdout).decode(
@@ -110,6 +113,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
env=env,
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
self._write_cgi_response(proc.stdout)
@@ -148,7 +152,13 @@ class GitHttpHandler(BaseHTTPRequestHandler):
key, _, value = line.decode("latin1").partition(":")
value = value.strip()
if key.lower() == "status":
status = int(value.split()[0])
try:
status = int(value.split()[0])
except (ValueError, IndexError):
self.log_message(
"malformed CGI Status header %r; using 500", value,
)
status = 500
else:
headers.append((key, value))
self.send_response(status)
+96 -10
View File
@@ -1,21 +1,107 @@
"""Tiny logging wrappers. All output goes to stderr."""
"""Tiny logging wrappers. All output goes to stderr.
Two capabilities layer onto the bare wrappers (issue #252):
- **Levels.** `debug` / `info` / `warn` / `error` carry an ordered
severity. Output is gated by `BOT_BOTTLE_LOG_LEVEL` (debug | info |
warn | error; default `info`). A message emits when its severity is
at or above the threshold, so `debug` is silent by default and
`error` always surfaces (nothing sits above it) which keeps the
fatal `die` path visible regardless of the configured level.
- **Context.** Every wrapper takes an optional `context` mapping that
renders as a parseable ` [k=v ...]` suffix (keys sorted; values with
whitespace/quotes are quoted), so failures can be filtered and
correlated instead of being flat strings.
With no `context` and the default level, output is byte-identical to the
original `bot-bottle: <msg>` / `bot-bottle: warning: <msg>` /
`bot-bottle: error: <msg>` lines the 100+ existing call sites are
unaffected.
"""
from __future__ import annotations
import os
import sys
from typing import NoReturn
from typing import Mapping, NoReturn
# Ordered severities. Gaps left between values so intermediate levels
# can be added later without renumbering.
DEBUG = 10
INFO = 20
WARN = 30
ERROR = 40
_LEVEL_NAMES: dict[str, int] = {
"debug": DEBUG,
"info": INFO,
"warn": WARN,
"warning": WARN,
"error": ERROR,
}
# Default threshold when BOT_BOTTLE_LOG_LEVEL is unset or unrecognised.
_DEFAULT_THRESHOLD = INFO
_LOG_LEVEL_ENV = "BOT_BOTTLE_LOG_LEVEL"
def info(msg: str) -> None:
print(f"bot-bottle: {msg}", file=sys.stderr)
def _threshold() -> int:
"""Resolve the active level threshold from the environment.
Read per-call (not cached) so the level can be changed at runtime
and so tests can patch `os.environ` without a reload. Unknown values
fall back to the default rather than raising logging must never be
the thing that crashes the process."""
raw = os.environ.get(_LOG_LEVEL_ENV, "")
return _LEVEL_NAMES.get(raw.strip().lower(), _DEFAULT_THRESHOLD)
def warn(msg: str) -> None:
print(f"bot-bottle: warning: {msg}", file=sys.stderr)
def _format_context(context: Mapping[str, object] | None) -> str:
"""Render a context mapping as a ` [k=v k2=v2]` suffix.
Keys are sorted for stable, diffable output. Values that are empty or
contain whitespace or a quote are wrapped in double quotes (with inner
quotes escaped) so each `k=v` pair stays parseable. Empty/None context
renders as the empty string."""
if not context:
return ""
parts: list[str] = []
for key in sorted(context):
value = str(context[key])
if value == "" or any(ch.isspace() for ch in value) or '"' in value:
value = '"' + value.replace('"', '\\"') + '"'
parts.append(f"{key}={value}")
return " [" + " ".join(parts) + "]"
def error(msg: str) -> None:
print(f"bot-bottle: error: {msg}", file=sys.stderr)
def _emit(
level: int,
label: str,
msg: str,
context: Mapping[str, object] | None,
) -> None:
if level < _threshold():
return
prefix = f"{label}: " if label else ""
sys.stderr.write(f"bot-bottle: {prefix}{msg}{_format_context(context)}\n")
def debug(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(DEBUG, "debug", msg, context)
def info(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(INFO, "", msg, context)
def warn(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(WARN, "warning", msg, context)
def error(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(ERROR, "error", msg, context)
class Die(SystemExit):
@@ -31,6 +117,6 @@ class Die(SystemExit):
self.message = message
def die(msg: str) -> NoReturn:
error(msg)
def die(msg: str, *, context: Mapping[str, object] | None = None) -> NoReturn:
error(msg, context=context)
raise Die(1, msg)
+2 -4
View File
@@ -113,10 +113,8 @@ class ManifestBottle:
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
# Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
# default, issue #249), the launch step brings up a supervise
# sidecar that exposes MCP tools to the agent (egress-block,
# capability-block) plus mounts the current-config dir read-only
# into the agent at /etc/bot-bottle/current-config. Set
# `supervise: false` to skip the sidecar and mount.
# sidecar that exposes egress MCP tools to the agent. Set
# `supervise: false` to skip the sidecar.
supervise: bool = True
@classmethod
+28 -6
View File
@@ -199,13 +199,10 @@ def _parse_provider_settings(
) -> dict[str, object]:
if raw is None:
return {}
if template != "pi":
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings is only "
"supported for template 'pi'"
)
settings = as_json_object(raw, f"bottle '{bottle_name}' agent_provider.settings")
allowed = {
common_allowed = {"startup_args"}
pi_allowed = {
"provider",
"base_url",
"api",
@@ -218,12 +215,37 @@ def _parse_provider_settings(
"supports_developer_role",
"supports_reasoning_effort",
}
if template == "pi":
allowed = common_allowed | pi_allowed
elif template in ("claude", "codex"):
allowed = common_allowed
elif template not in PROVIDER_TEMPLATES:
return dict(settings)
else:
allowed = common_allowed
for key in settings:
if key not in allowed:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings has unknown "
f"key {key!r}; allowed: {', '.join(sorted(allowed))}"
)
startup_args = settings.get("startup_args")
if startup_args is not None:
if not isinstance(startup_args, list):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.startup_args "
f"must be an array of strings"
)
for i, arg in enumerate(startup_args):
if not isinstance(arg, str) or not arg:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings."
f"startup_args[{i}] must be a non-empty string"
)
if template != "pi":
return dict(settings)
for key in ("provider", "base_url", "api", "api_key", "api_key_env"):
value = settings.get(key)
if value is not None and (not isinstance(value, str) or not value):
+22 -5
View File
@@ -21,6 +21,9 @@ VALID_METHODS = frozenset({
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# What the proxy does on an outbound token match (PRD 0062).
OUTBOUND_ON_MATCH_VALUES = ("block", "redact", "supervise")
def validate_egress_routes(
bottle_name: str,
@@ -67,6 +70,7 @@ class ManifestEgressRoute:
GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None
InboundDetectors: tuple[str, ...] | None = None
OutboundOnMatch: str = ""
@classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
@@ -161,8 +165,9 @@ class ManifestEgressRoute:
# --- dlp ---
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
outbound_on_match = ""
if "dlp" in d:
outbound_detectors, inbound_detectors = _parse_dlp_block(
outbound_detectors, inbound_detectors, outbound_on_match = _parse_dlp_block(
label, d.get("dlp"),
)
@@ -201,6 +206,7 @@ class ManifestEgressRoute:
GitFetch=git_fetch,
OutboundDetectors=outbound_detectors,
InboundDetectors=inbound_detectors,
OutboundOnMatch=outbound_on_match,
)
@@ -323,7 +329,7 @@ def _parse_header_match(
def _parse_dlp_block(
route_label: str,
raw: object,
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
label = f"{route_label} dlp"
d = as_json_object(raw, label)
@@ -358,13 +364,24 @@ def _parse_dlp_block(
outbound = _parse_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = d.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ManifestError(
f"{label} outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in d:
if k not in ("outbound_detectors", "inbound_detectors"):
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are "
f"'outbound_detectors', 'inbound_detectors'"
f"'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
)
return outbound, inbound
return outbound, inbound, on_match
LOG_LEVELS = frozenset({0, 1, 2})
+110 -18
View File
@@ -49,33 +49,125 @@ def _resolve_one_bottle(
repos_cache[name] = _resolve_repos_raw({}, child_raw)
return bottle
if not isinstance(parent_name_raw, str):
# Normalize to list, accepting both str and list[str].
raw_list: list[object]
if isinstance(parent_name_raw, str):
raw_list = [parent_name_raw]
elif isinstance(parent_name_raw, list):
raw_list = parent_name_raw
else:
raise ManifestError(
f"bottle '{name}' extends must be a string "
f"bottle '{name}' extends must be a string or list of strings "
f"(was {type(parent_name_raw).__name__})"
)
parent_name: str = parent_name_raw
if parent_name == name:
raise ManifestError(
f"bottle '{name}' extends itself; remove the "
f"self-reference"
)
if parent_name not in raws:
avail = ", ".join(sorted(raws.keys())) or "(none)"
raise ManifestError(
f"bottle '{name}' extends '{parent_name}' which is not "
f"defined. Available bottles: {avail}"
)
parent = _resolve_one_bottle(
parent_name, raws, cache, repos_cache, seen + (name,)
# Validate each entry before resolving any of them.
parent_names: list[str] = []
for i, pname in enumerate(raw_list):
if not isinstance(pname, str):
raise ManifestError(
f"bottle '{name}' extends[{i}] must be a string "
f"(was {type(pname).__name__})"
)
parent_names.append(pname)
if pname == name:
raise ManifestError(
f"bottle '{name}' extends itself; remove the self-reference"
)
if pname not in raws:
avail = ", ".join(sorted(raws.keys())) or "(none)"
raise ManifestError(
f"bottle '{name}' extends '{pname}' which is not "
f"defined. Available bottles: {avail}"
)
combined_parent, combined_repos_raw = _fold_parents(
parent_names, raws, cache, repos_cache, seen + (name,)
)
merged_repos_raw = _resolve_repos_raw(repos_cache[parent_name], child_raw)
bottle = _merge_bottles(parent, child_raw, merged_repos_raw, name)
merged_repos_raw = _resolve_repos_raw(combined_repos_raw, child_raw)
bottle = _merge_bottles(combined_parent, child_raw, merged_repos_raw, name)
cache[name] = bottle
repos_cache[name] = merged_repos_raw
return bottle
def _fold_parents(
parent_names: list[str],
raws: dict[str, dict[str, object]],
cache: dict[str, ManifestBottle],
repos_cache: dict[str, dict[str, object]],
seen: tuple[str, ...],
) -> tuple[ManifestBottle, dict[str, object]]:
"""Resolve each parent and fold them left-to-right.
Later parents win over earlier ones on conflict. The `seen` tuple
carries the current bottle's name so cycle detection works across
every parent edge in the multi-parent graph."""
first = parent_names[0]
effective = _resolve_one_bottle(first, raws, cache, repos_cache, seen)
effective_repos_raw = repos_cache[first]
for pname in parent_names[1:]:
later = _resolve_one_bottle(pname, raws, cache, repos_cache, seen)
later_repos_raw = repos_cache[pname]
effective, effective_repos_raw = _fold_two_bottles(
effective, effective_repos_raw, later, later_repos_raw
)
return effective, effective_repos_raw
def _fold_two_bottles(
earlier: ManifestBottle,
earlier_repos_raw: dict[str, object],
later: ManifestBottle,
later_repos_raw: dict[str, object],
) -> tuple[ManifestBottle, dict[str, object]]:
"""Combine two resolved parent bottles; later wins over earlier."""
from .manifest import ManifestBottle, ManifestGitUser
from .manifest_egress import ManifestEgressConfig
from .manifest_git import parse_git_gate_config
from .manifest_util import as_json_object
merged_env = {**earlier.env, **later.env}
merged_git_user = ManifestGitUser(
name=later.git_user.name or earlier.git_user.name,
email=later.git_user.email or earlier.git_user.email,
)
# Repos: union by name; for same-name entries, later wins per-field.
# Unlike _resolve_repos_raw, an empty later_repos_raw means "no repos
# declared" — it does NOT clear the earlier parent's repos.
names = list(earlier_repos_raw) + [
n for n in later_repos_raw if n not in earlier_repos_raw
]
merged_repos_raw: dict[str, object] = {
n: {
**as_json_object(earlier_repos_raw.get(n, {}), "earlier parent repo"),
**as_json_object(later_repos_raw.get(n, {}), "later parent repo"),
}
for n in names
}
if merged_repos_raw:
merged_git, _ = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
else:
merged_git = ()
# Egress: routes concatenate; scalar fields use last-wins.
merged_egress = ManifestEgressConfig(
routes=earlier.egress.routes + later.egress.routes,
Log=later.egress.Log,
)
return ManifestBottle(
env=merged_env,
agent_provider=later.agent_provider,
git=merged_git,
git_user=merged_git_user,
egress=merged_egress,
supervise=later.supervise,
), merged_repos_raw
def _merge_bottles(
parent: ManifestBottle,
child_raw: dict[str, object],
+2
View File
@@ -87,5 +87,7 @@ def load_bottle_chain_from_dir(
parent = fm.get("extends")
if isinstance(parent, str):
to_load.append(parent)
elif isinstance(parent, list):
to_load.extend(p for p in parent if isinstance(p, str))
return resolve_bottles(raws)[bottle_name]
+20 -45
View File
@@ -2,11 +2,10 @@
The supervise plane is the per-bottle MCP sidecar plus its host-side
queue/audit support. The sidecar (bot_bottle.supervise_server)
sits on the bottle's internal network and exposes three MCP tools the
agent calls when it hits a stuck-recovery category:
sits on the bottle's internal network and exposes MCP tools the agent
calls when it needs an operator-reviewed egress change:
* egress-block / allow agent proposes a new routes.yaml
* capability-block agent proposes a new agent Dockerfile
Each tool call: the agent passes the full proposed file plus a
justification text. The sidecar validates the proposal syntactically,
@@ -48,16 +47,18 @@ from pathlib import Path
SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100
TOOL_CAPABILITY_BLOCK = "capability-block"
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_ALLOW = "allow"
TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
# Written directly by the egress addon (not an agent-facing MCP tool) when an
# outbound DLP token block is routed to the operator for override (PRD 0062).
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
TOOLS: tuple[str, ...] = (
TOOL_ALLOW,
TOOL_CAPABILITY_BLOCK,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
@@ -71,12 +72,8 @@ TOOLS: tuple[str, ...] = (
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
# capability-block has no on-disk config the operator edits in place
# (the Dockerfile is rebuilt, not patched), so it has no audit log
# here — those changes are captured by git history + the rebuild record
# laid down in PRD 0016.
COMPONENT_FOR_TOOL: dict[str, str] = {
TOOL_ALLOW: "egress",
TOOL_EGRESS_ALLOW: "egress",
TOOL_EGRESS_BLOCK: "egress",
}
@@ -90,8 +87,6 @@ STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
ACTION_OPERATOR_EDIT = "operator-edit"
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
CURRENT_CONFIG_DIR_IN_AGENT = "/etc/bot-bottle/current-config"
DEFAULT_POLL_INTERVAL_SEC = 0.5
@@ -434,59 +429,39 @@ def sha256_hex(content: str) -> str:
# --- Sidecar plan + abstract lifecycle -------------------------------------
# Filename of the staged Dockerfile inside the agent's read-only
# current-config mount. The capability-block tool's description
# points the agent at this exact path so it can read the current
# Dockerfile and propose modifications.
#
# routes.yaml + allowlist used to live here too; PRD 0017 chunk 3
# moved them behind the `list-egress-routes` MCP tool (live state
# from egress's introspection endpoint) so the agent always sees
# current data rather than a launch-time snapshot.
CURRENT_CONFIG_DOCKERFILE = "Dockerfile"
@dataclass(frozen=True)
class SupervisePlan:
"""Output of Supervise.prepare; consumed by .start.
`queue_dir` is the host directory bind-mounted into the sidecar
at /run/supervise/queue. `current_config_dir` is the host
directory bind-mounted (read-only) into the *agent* container
at /etc/bot-bottle/current-config currently holds only the
Dockerfile snapshot (routes.yaml + allowlist moved to the
`list-egress-routes` MCP tool). `internal_network` is
empty at prepare time; the backend's launch step fills it via
dataclasses.replace before calling .start."""
at /run/supervise/queue. `internal_network` is empty at prepare
time; the backend's launch step fills it via dataclasses.replace
before calling .start."""
slug: str
queue_dir: Path
current_config_dir: Path
internal_network: str = ""
class Supervise(ABC):
"""Per-bottle supervise sidecar. Encapsulates the host-side
prepare (queue dir + current-config staging); the sidecar's
start/stop lifecycle is backend-specific."""
prepare (queue dir staging); the sidecar's start/stop lifecycle
is backend-specific."""
def prepare(
self,
slug: str,
stage_dir: Path,
) -> SupervisePlan:
"""Stage the per-bottle queue dir on the host and the
current-config dir under `stage_dir`. Returns the plan;
`internal_network` must be set by the launch step before
"""Stage the per-bottle queue dir on the host. Returns the
plan; `internal_network` must be set by the launch step before
.start runs."""
del stage_dir
queue_dir = queue_dir_for_slug(slug)
queue_dir.mkdir(parents=True, exist_ok=True)
current_config_dir = stage_dir / "current-config"
current_config_dir.mkdir(parents=True, exist_ok=True)
return SupervisePlan(
slug=slug,
queue_dir=queue_dir,
current_config_dir=current_config_dir,
)
# --- Helpers ---------------------------------------------------------------
@@ -537,8 +512,6 @@ __all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"COMPONENT_FOR_TOOL",
"CURRENT_CONFIG_DIR_IN_AGENT",
"CURRENT_CONFIG_DOCKERFILE",
"DEFAULT_POLL_INTERVAL_SEC",
"Proposal",
"QUEUE_DIR_IN_CONTAINER",
@@ -554,8 +527,10 @@ __all__ = [
"TOOLS",
"EGRESS_FORWARD_PROXY",
"EGRESS_INTROSPECT_URL",
"TOOL_CAPABILITY_BLOCK",
"TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK",
"TOOL_GITLEAKS_ALLOW",
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"archive_proposal",
"audit_dir",
+64 -66
View File
@@ -1,8 +1,8 @@
"""Supervise sidecar HTTP server (PRD 0013).
Per-bottle MCP server exposing tools the agent calls to propose config
changes when stuck. The tools are `allow`, `egress-block`,
`capability-block`, and `list-egress-routes`.
Per-bottle MCP server exposing tools the agent calls to propose egress
config changes when stuck. The tools are `egress-allow`,
`egress-block`, and `list-egress-routes`.
Each queued tool call:
@@ -47,11 +47,11 @@ from pathlib import Path
try:
# Same-directory imports inside the bundle container; these files are
# COPYed flat under /app by Dockerfile.sidecars.
from egress_addon_core import load_routes
from egress_addon_core import LOG_OFF, load_config
import supervise as _sv
except ModuleNotFoundError:
# Package imports for host-side tests and tooling.
from .egress_addon_core import load_routes
from .egress_addon_core import LOG_OFF, load_config
from . import supervise as _sv
@@ -90,19 +90,19 @@ def parse_jsonrpc(body: bytes) -> JsonRpcRequest:
try:
raw = json.loads(body)
except json.JSONDecodeError as e:
raise _RpcError(ERR_PARSE, f"parse error: {e}") from e
raise _RpcClientError(ERR_PARSE, f"parse error: {e}") from e
if not isinstance(raw, dict):
raise _RpcError(ERR_INVALID_REQUEST, "request must be a JSON object")
raise _RpcClientError(ERR_INVALID_REQUEST, "request must be a JSON object")
if raw.get("jsonrpc") != JSONRPC_VERSION:
raise _RpcError(ERR_INVALID_REQUEST, "jsonrpc field must be '2.0'")
raise _RpcClientError(ERR_INVALID_REQUEST, "jsonrpc field must be '2.0'")
method = raw.get("method")
if not isinstance(method, str):
raise _RpcError(ERR_INVALID_REQUEST, "method must be a string")
raise _RpcClientError(ERR_INVALID_REQUEST, "method must be a string")
params = raw.get("params", {})
if params is None:
params = {}
if not isinstance(params, dict):
raise _RpcError(ERR_INVALID_PARAMS, "params must be an object")
raise _RpcClientError(ERR_INVALID_PARAMS, "params must be an object")
rpc_id = raw.get("id", _NO_ID)
is_notification = rpc_id is _NO_ID
return JsonRpcRequest(
@@ -117,12 +117,23 @@ _NO_ID = object()
class _RpcError(Exception):
"""Base class for all typed RPC errors that surface as JSON-RPC error responses."""
def __init__(self, code: int, message: str):
super().__init__(message)
self.code = code
self.message = message
class _RpcClientError(_RpcError):
"""Caller sent a bad request; returned verbatim, no server-side logging."""
class _RpcInternalError(_RpcError):
"""Server-side fault; logged at ERROR with cause, always returns ERR_INTERNAL."""
def __init__(self, message: str) -> None:
super().__init__(ERR_INTERNAL, message)
def jsonrpc_result(request_id: object, result: object) -> bytes:
payload = {"jsonrpc": JSONRPC_VERSION, "id": request_id, "result": result}
return (json.dumps(payload) + "\n").encode("utf-8")
@@ -148,7 +159,7 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
"allowlist. Returns JSON with one entry per allowed host, "
"each carrying its matches rules (if any) and whether "
"the proxy injects Authorization for the route. Use this "
"before composing an `allow` or `egress-block` proposal so "
"before composing an `egress-allow` or `egress-block` proposal so "
"the new routes file extends the live one rather than "
"replacing it."
),
@@ -159,7 +170,7 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
},
},
{
"name": _sv.TOOL_ALLOW,
"name": _sv.TOOL_EGRESS_ALLOW,
"description": (
"Request operator approval to change the bottle's egress "
"allowlist. Pass the full proposed routes.yaml content, not "
@@ -187,6 +198,7 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
" dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format."
),
@@ -228,6 +240,7 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
" dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format."
),
@@ -240,42 +253,13 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
"required": ["routes_yaml", "justification"],
},
},
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"description": (
"Call when the bottle is missing a tool, skill, permission, "
"or env var you need — something that lives in the agent "
"Dockerfile rather than in the egress routes. "
"Read the current Dockerfile from "
"/etc/bot-bottle/current-config/Dockerfile, compose a "
"modified version, and pass the full new file plus a "
"justification. On approval the supervisor rebuilds the "
"bottle from the new Dockerfile and starts a replacement on "
"the same branch (wired in PRD 0016; v1 acknowledges only)."
),
"inputSchema": {
"type": "object",
"properties": {
"dockerfile": {
"type": "string",
"description": "Full proposed Dockerfile content.",
},
"justification": {
"type": "string",
"description": "Why this capability is needed.",
},
},
"required": ["dockerfile", "justification"],
},
},
]
# Map each proposal tool to the input field that carries the agent's
# payload (stored in Proposal.proposed_file).
PROPOSED_FILE_FIELD: dict[str, str] = {
_sv.TOOL_ALLOW: "routes_yaml",
_sv.TOOL_CAPABILITY_BLOCK: "dockerfile",
_sv.TOOL_EGRESS_ALLOW: "routes_yaml",
_sv.TOOL_EGRESS_BLOCK: "routes_yaml",
}
@@ -288,21 +272,22 @@ def validate_proposed_file(tool: str, content: str) -> None:
catches obvious paste-errors / wrong-tool selections before they
enter the queue."""
if not content.strip():
raise _RpcError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
if tool == _sv.TOOL_CAPABILITY_BLOCK:
# Dockerfiles are too varied to validate syntactically beyond
# non-empty. The operator reads the diff in the TUI.
pass
elif tool in (_sv.TOOL_ALLOW, _sv.TOOL_EGRESS_BLOCK):
raise _RpcClientError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
if tool in (_sv.TOOL_EGRESS_ALLOW, _sv.TOOL_EGRESS_BLOCK):
try:
load_routes(content)
config = load_config(content)
except ValueError as e:
raise _RpcError(
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{tool}: proposed routes.yaml is not valid: {e}",
) from e
if config.log != LOG_OFF:
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{tool}: proposed routes.yaml must not change egress logging",
)
else:
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
raise _RpcClientError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
# --- MCP handlers ----------------------------------------------------------
@@ -375,17 +360,17 @@ def handle_tools_call(
doesn't need operator approval."""
name = params.get("name")
if not isinstance(name, str):
raise _RpcError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
if name == _sv.TOOL_LIST_EGRESS_ROUTES:
return handle_list_egress_routes(typing.cast(dict[str, object], params.get("arguments", {})), config)
args_raw = params.get("arguments", {})
if not isinstance(args_raw, dict):
raise _RpcError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
justification = args_raw.get("justification")
if not isinstance(justification, str) or not justification.strip():
raise _RpcError(
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{name}: 'justification' is required and must be a non-empty string",
)
@@ -394,13 +379,13 @@ def handle_tools_call(
file_field = PROPOSED_FILE_FIELD[name]
proposed_file = args_raw.get(file_field)
if not isinstance(proposed_file, str):
raise _RpcError(
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{name}: '{file_field}' is required and must be a string",
)
validate_proposed_file(name, proposed_file)
else:
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {name!r}")
raise _RpcClientError(ERR_INVALID_PARAMS, f"unknown tool {name!r}")
proposal = _sv.Proposal.new(
bottle_slug=config.bottle_slug,
@@ -409,7 +394,10 @@ def handle_tools_call(
justification=justification,
current_file_hash=_sv.sha256_hex(proposed_file),
)
_sv.write_proposal(config.queue_dir, proposal)
try:
_sv.write_proposal(config.queue_dir, proposal)
except OSError as e:
raise _RpcInternalError(f"failed to write proposal to queue: {e}") from e
sys.stderr.write(
f"supervise: queued proposal {proposal.id} ({name}) "
f"for bottle {config.bottle_slug}; waiting for operator...\n"
@@ -429,7 +417,10 @@ def handle_tools_call(
"content": [{"type": "text", "text": text}],
"isError": False,
}
_sv.archive_proposal(config.queue_dir, proposal.id)
try:
_sv.archive_proposal(config.queue_dir, proposal.id)
except OSError as e:
raise _RpcInternalError(f"failed to archive proposal: {e}") from e
text = format_response_text(response)
return {
@@ -463,9 +454,8 @@ def format_pending_response_text(timeout_seconds: float) -> str:
# --- HTTP transport --------------------------------------------------------
# Max request body the server accepts. Generous because Dockerfile
# proposals can be a few KB; routes.json is small. 1 MB is well above
# any realistic config file.
# Max request body the server accepts. 1 MB is well above any realistic
# routes.yaml proposal.
MAX_BODY_BYTES = 1 * 1024 * 1024
@@ -505,7 +495,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
req = parse_jsonrpc(body)
except _RpcError as e:
except _RpcClientError as e:
self._write_jsonrpc(jsonrpc_error(None, e.code, e.message))
return
@@ -513,11 +503,19 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
result = self._dispatch(req, config)
except _RpcError as e:
except _RpcClientError as e:
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
return
except Exception as e: # noqa: W0718 — catch-all for RPC dispatch errors
sys.stderr.write(f"supervise: internal error: {e}\n")
except _RpcInternalError as e:
cause = e.__cause__
detail = f": {cause}" if cause else ""
sys.stderr.write(f"supervise: internal error: {e.message}{detail}\n")
sys.stderr.flush()
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
return
except Exception as e: # noqa: W0718 — unexpected errors
sys.stderr.write(f"supervise: unexpected error: {type(e).__name__}: {e}\n")
sys.stderr.flush()
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
return
@@ -536,7 +534,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
return handle_tools_list(req.params)
if method == "tools/call":
return handle_tools_call(req.params, config)
raise _RpcError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def _write_jsonrpc(self, body: bytes) -> None:
self.send_response(200)
@@ -0,0 +1,210 @@
# PRD 0062: Supervisor override for egress token blocks
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-24
- **Issue:** #261
## Summary
Give each egress route a policy for what happens when an outbound DLP detector
matches a token, via `dlp.outbound_on_match: block | redact | supervise`
(default `supervise`):
- **`supervise`** (default) — route the block through the existing supervisor
approval queue instead of returning `403` immediately. The proxy holds the
request open until the operator approves or rejects it. On approval the
matched token is added to an in-memory "safe tokens" set so the request — and
any later request carrying the same token — flows through without
re-prompting.
- **`redact`** — scrub the matched value(s) from the request and forward it,
no operator in the loop. For routes where a token-shaped value is noise the
upstream doesn't need (telemetry/log sinks). Fails closed if a match lands on
a surface redaction can't rewrite (the hostname).
- **`block`** — the original hard `403`; never overridable. For routes where a
detected token must always stop.
The motivating goal is reducing friction from false positives without weakening
the default-deny posture: supervise keeps a human in the loop, redact is an
explicit per-route opt-in, and block stays available for sensitive routes.
## Problem
The outbound DLP detectors (`token_patterns`, `known_secrets`) are
deliberately aggressive: any string that looks like a credential is blocked
before it leaves the bottle. That is the right default, but it produces false
positives — a token-shaped value that is not actually a secret, or a credential
the agent legitimately needs to send to a declared host. Today the only
recovery is for the operator to notice the `egress DLP` 403 in the logs and
hand-edit the route's `dlp.outbound_detectors`, which disables the detector for
the whole route rather than allowing the one value.
The operator has no in-the-loop signal that a token block happened and no
fine-grained way to say "this specific value is fine."
## Goals / Success Criteria
1. An outbound DLP **token** block (a `ScanResult` carrying a matched secret
value) creates a supervisor proposal instead of an immediate `403`.
2. The egress proxy holds the blocked request open, polling for the operator's
response up to a bounded timeout.
3. The proposal shows the operator the host, method, path, the detector reason,
and a **redacted** context snippet — never the raw token value.
4. On `approved`/`modified`, the matched token value is added to an in-memory
safe-tokens set and the request proceeds normally; later requests carrying
the same value skip the block.
5. On `rejected`, timeout, malformed response, or missing supervisor wiring,
the request fails closed with the same `403` as today.
6. Structural blocks that carry no token value (CRLF injection) and the
route-not-allowlisted / git blocks are unchanged — they stay hard `403`s and
keep their existing agent-driven `allow` / `egress-block` MCP path.
7. The proxy event loop is not stalled while waiting: the wait is asynchronous,
so other flows keep being served.
## Non-goals
- Persisting the safe-tokens set across egress restarts. It lives in process
memory only; a restart re-prompts. (The issue explicitly defers persistence.)
- Supervising inbound (prompt-injection) blocks or WebSocket frame blocks.
WebSocket frames still honour the safe-tokens set for already-approved values
but cannot wait for approval (there is no response surface after upgrade).
- Generalising an approved secret across encodings. The safe-tokens set matches
the exact value the detector found.
- Replacing the per-route `dlp.outbound_detectors` override. That remains the
way to turn a detector off wholesale.
- Making `redact` the default. Silent redaction of a true false positive
corrupts legitimate data, so it is opt-in per route; `supervise` (human in
the loop) stays the default.
## Scope
### In scope
The minimum cut that ships, in build order:
1. **Core**`ScanResult.matched`; thread `safe_tokens` through
`scan_outbound` / the token detectors; `build_token_allow_payload`.
2. **Supervise + TUI**`TOOL_EGRESS_TOKEN_ALLOW`; TUI suffix, modify guard,
required approval reason.
3. **Addon glue** — async `request`, safe-tokens set, proposal write + async
poll, allow/block decision; pass `safe_tokens` into the WebSocket path.
4. **On-match policy**`dlp.outbound_on_match` through manifest → render →
addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in
the addon's outbound handler.
5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README
egress + supervisor notes.
### Out of scope
The deferrals enumerated under **Non-goals** — restart persistence, inbound /
WebSocket-frame supervision, cross-encoding generalisation, replacing
`dlp.outbound_detectors`, and making `redact` the default.
## Proposed Design
### New services / components
A new proposal tool constant `egress-token-allow` (`TOOL_EGRESS_TOKEN_ALLOW`)
is added to `supervise.TOOLS`, and the egress addon gains an in-memory
safe-tokens set plus the policy-dispatch path that drives it.
On an outbound block the addon dispatches on the resolved policy:
- **Structural blocks always 403.** A `ScanResult` with no `matched` value
(CRLF injection) is a hard `403` regardless of policy — there is nothing to
redact or safelist.
- **`redact`** runs `redact_tokens` over the body, non-`host` header values,
and path/query, then re-scans. If the re-scan is clean the (rewritten)
request is forwarded; if a block-severity match remains (e.g. in the
hostname, or a unicode-evasion token redaction can't reach) it fails closed
with a `403`.
- **`block`** writes the `403` immediately.
- **`supervise`** runs the queue-and-wait loop, falling back to `block` when
supervise isn't wired for the bottle.
For `supervise`, the addon writes the proposal directly to
`SUPERVISE_QUEUE_DIR` (the queue is bind-mounted into the sidecar bundle and
shared by every daemon, exactly as git-gate's `gitleaks-allow` proposal in PRD
0061 does). The proposal's `proposed_file` is a human-readable text payload
built by `build_token_allow_payload`:
```
egress blocked an outbound request carrying a detected token
host: api.example.com
method: POST
path: /v1/ingest
detector: OpenAI API key found in body
context: ...before ******** after...
```
The justification tells the operator to approve only if the value is a false
positive or a credential the request legitimately needs. The addon then polls
`<proposal-id>.response.json` for `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default
300). `approved`/`modified` allow the request and add the value to the
safe-tokens set; `rejected`, malformed responses, and timeout fail the request
closed. The proposal + response are archived to `processed/` after a decision.
Because the wait happens inside mitmproxy's asyncio loop, the addon's `request`
hook is async and polls with `asyncio.sleep`, so concurrent flows are
unaffected.
### Existing code touched
- **Policy threading.** `dlp.outbound_on_match` is a per-route enum threaded
from the bottle manifest (`manifest_egress`) through the resolved route
(`egress.EgressRoute`), the rendered `routes.yaml` (`egress_render_routes`),
and the addon's `Route` (`egress_addon_core`). Unset renders nothing and
resolves to `supervise` at request time. The `list-egress-routes`
introspection endpoint round-trips it so the agent's proposals preserve it.
- **Provider-route default.** Agent-provider routes (the agent talking to its
own LLM API — `api.anthropic.com`, the Codex backend, etc.) are the worst
source of token-shaped false positives because the whole conversation payload
flows through them. `egress_routes_for_bottle` fills `outbound_on_match=redact`
on any provider route that doesn't set it explicitly; a provider that sets the
policy keeps its choice, and manifest routes are unaffected (they default to
`supervise`).
- **Scanners.** `scan_outbound` (and the token detectors `scan_token_patterns`
/ `scan_known_secrets` it calls) accept a `safe_tokens` set. A match whose
value is in `safe_tokens` is skipped, so an approved token no longer blocks;
the scanners keep searching past a safelisted match so a second, un-approved
secret in the same request is still caught. The WebSocket path is passed the
same `safe_tokens` set.
- **Supervisor UI.** `cli/supervise.py` renders `egress-token-allow` like
`gitleaks-allow`: the text payload is shown, modify is unavailable (there is
no file patch to edit), and approval prompts for a non-empty reason recorded
in the response notes. There is no on-disk config diff, so — like
`gitleaks-allow` and `capability-block` — it writes no egress audit-log entry.
- **Failure handling.** If `SUPERVISE_QUEUE_DIR` / `SUPERVISE_BOTTLE_SLUG` are
unset (supervise disabled for the bottle), the addon skips the queue and
returns the existing `403`. Any error writing the proposal or reading the
response also fails closed.
### Data model changes
- New per-route manifest field `dlp.outbound_on_match: block | redact |
supervise`, rendered into `routes.yaml` (omitted when unset).
- `ScanResult` gains a `matched: str = ""` field carrying the raw substring the
detector matched. The token detectors populate it; the structural CRLF
detector leaves it empty. The value stays inside the egress sidecar process —
never written to a log line (logs use the redacted `context`) nor to the
proposal file.
- Proposal text payload (above) plus `<proposal-id>.response.json` in
`SUPERVISE_QUEUE_DIR`, archived to `processed/` after a decision.
- New env var `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default 300).
### External dependencies
None. Reuses the existing supervisor queue (`SUPERVISE_QUEUE_DIR`) and the
mitmproxy addon framework already in the egress sidecar.
## Open questions
- Should `known_secrets` (provisioned `EGRESS_TOKEN_*` exfiltration) be
override-able at all, or only `token_patterns`? This PRD allows both —
approval is an explicit operator decision and the safe-tokens set matches the
exact found value — but a future revision could restrict `known_secrets` to
reject-only.
## References
- Issue #261
- PRD 0061 — `gitleaks-allow` supervisor proposal pattern this reuses.
@@ -0,0 +1,189 @@
# PRD 0063: Strengthen outbound exfiltration detection
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-25
- **Issue:** #259
## Summary
The current DLP system has two detection tiers: weak (vendor token regexes)
and strong (exact matching of provisioned `EGRESS_TOKEN_*` values in multiple
encodings). This PRD adds four complementary hardening layers in ranked order
of impact:
1. **Canary tokens** — inject a per-session fake secret into the agent's
environment; any outbound appearance is a zero-false-positive exfil signal.
2. **Broadened known-value scanning** — extend the strong tier beyond
`EGRESS_TOKEN_*` via a `BOT_BOTTLE_SENSITIVE_PREFIXES` sidecar env var.
3. **Fragmentation-resistant matching** — catch separator injection and
long-substring leaks via alnum projection and sliding-window substring
search.
4. **Entropy scoring** — warn-only flag on high-entropy blobs as a
complementary signal for encrypted exfil.
## Problem
After PRD 0056 the outbound scan covers all HTTP request surfaces (host, path,
query, headers, body). But the content-based strong tier only matches
`EGRESS_TOKEN_*` values and their encodings. Three gaps remain:
| Gap | How an agent exploits it |
|-----|--------------------------|
| No canary | There is no "can't be false-positive" signal |
| Narrow prefix filter | Sensitive values provisioned under other env prefixes (MCP keys, API keys injected via the agent provider) are invisible to `scan_known_secrets` |
| Fragmentation blindness | Inserting `-`, space, or `\n` between secret characters turns an exact-match into a miss |
## Goals / Success Criteria
1. Each launched bottle has a unique canary token in the agent's environment
under a randomized `WORD_WORD_SECRET` env var name. The egress sidecar gets
the same env var and registers that exact name through
`BOT_BOTTLE_SENSITIVE_PREFIXES`. Any outbound appearance of the canary
blocks the request as a known-secret match.
2. `scan_known_secrets` accepts a `sensitive_prefixes` parameter (default:
`("EGRESS_TOKEN_",)`). `scan_outbound` reads
`BOT_BOTTLE_SENSITIVE_PREFIXES` from `environ` and merges those prefixes
in, so operators can mark additional env vars as scanned values without
changing the manifest schema.
3. For every secret that passes exact-match, a secondary alnum-projection pass
checks for the secret with all non-alphanumeric characters stripped. This
catches separator-injection evasion (`MY-SECRET` → body contains
`MY SECRET`).
4. A sliding-window partial-match pass checks for long-enough contiguous
substrings of the secret's alnum projection in the text's alnum projection.
Any match ≥ `PARTIAL_MATCH_MIN_LEN` (12 chars) blocks with reason
`"partial match"`.
5. A new `scan_entropy` detector flags outbound text windows with Shannon
entropy ≥ `ENTROPY_BLOCK_THRESHOLD` (5.5 bits/char) at **warn** severity
only. It is registered under the new detector name `"entropy"` in
`OUTBOUND_DETECTOR_NAMES` and disabled by default (routes must opt in).
6. Binary request bodies are decoded via `latin-1` instead of
`utf-8 errors="replace"`, preserving every byte value and allowing
ASCII-range secrets to be found within binary payloads.
7. All new behaviour is unit-tested; existing tests pass unchanged.
## Non-goals
- Rolling per-host buffer for split-across-requests detection (state in the
stateless addon is complex; deferred).
- Additional vendor regexes.
- ML / embedding-based detection.
- Entropy-based hard blocks (warn only per the issue).
## Design
### Canary token flow
```
Egress.prepare()
canary = secrets.token_urlsafe(32)
canary_env = <random WORD_WORD_SECRET>
EgressPlan(canary=canary, canary_env=canary_env, ...)
Docker compose render:
sidecar env: <canary_env>=<canary>
sidecar env: BOT_BOTTLE_SENSITIVE_PREFIXES=<canary_env>
agent env: <canary_env>=<canary> ← visible to agent as a "secret"
macos-container launch: same literals added to sidecar + agent env entries
```
The sidecar uses `BOT_BOTTLE_SENSITIVE_PREFIXES` to make the random canary env
name part of the existing `scan_known_secrets` detector without adding a
manifest schema field.
### Broadened known-value scanning
`scan_known_secrets` gains a `sensitive_prefixes` parameter:
```python
def scan_known_secrets(
text: str,
*,
location: str = "body",
env: Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> ScanResult | None:
```
`scan_outbound` reads `BOT_BOTTLE_SENSITIVE_PREFIXES` (comma-separated list
of additional prefixes) from `environ` and appends them:
```python
extra = tuple(
p for p in environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "").split(",") if p
)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
```
`redact_tokens` receives the same treatment for consistent redaction.
### Fragmentation-resistant matching
A new helper `_alnum_projection(text)` strips all non-alphanumeric characters.
`scan_known_secrets` runs two passes per secret:
1. **Exact pass** — existing encoded-variant loop (unchanged).
2. **Alnum-projection pass** — if the secret's alnum projection has ≥ 8 chars,
check if it appears in the text's alnum projection. Match → block with
`"fragmented match (separator injection)"` reason.
3. **Partial-substring pass** — if the secret's alnum projection has ≥
`PARTIAL_MATCH_MIN_LEN` chars (12), slide a window of that length across the
secret's projection and look for each window in the text's alnum projection.
First match → block with `"partial match"` reason.
All three passes run only for the `"known_secrets"` detector; the token-pattern
and entropy detectors are unchanged.
### Entropy scoring
New public function:
```python
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW, # 64
threshold: float = ENTROPY_BLOCK_THRESHOLD, # 5.5
) -> ScanResult | None:
```
Slides a window of `window` characters across `text` in steps of `window // 2`.
If any window's Shannon entropy exceeds `threshold`, returns a **warn**-severity
`ScanResult`. Never blocks.
`OUTBOUND_DETECTOR_NAMES` gains `"entropy"`. Routes opt in via their `dlp`
block; entropy scanning is **off by default** to avoid false-positive noise on
legitimate binary payloads.
### Binary body handling
In `scan_outbound`, the bytes → str decoding changes from:
```python
body.decode("utf-8", errors="replace")
```
to:
```python
body.decode("utf-8") if body is str else body.decode("latin-1")
```
`latin-1` is a bijective byte↔codepoint mapping; every byte value is preserved
as its corresponding Latin-1 code point, so ASCII-range secret strings remain
intact and `str.find` / regex still locate them correctly. The fallback from
strict UTF-8 is tried first so valid UTF-8 bodies are decoded faithfully.
## Implementation
Delivered in three commits on the same branch:
1. **DLP detector changes**`_alnum_projection`, fragmentation passes,
`scan_entropy`, broadened `scan_known_secrets`, updated `scan_outbound` and
`redact_tokens`; all accompanying unit tests.
2. **Canary injection**`EgressPlan.canary`, `Egress.prepare()`,
Docker compose + macos-container backend injection.
3. **PRD flip**`Status: Draft → Active`.
@@ -0,0 +1,85 @@
# PRD 0064: LOG_FULL egress logging credential redaction
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-25
- **Issue:** #257
## Summary
The `LOG_FULL` egress logging path (`_log_request` and `_log_response` in `egress_addon.py`) writes request/response headers and bodies to stderr without redaction and includes the sidecar-injected upstream `Authorization` header verbatim. This PR applies `redact_tokens` to header values and bodies in both log functions and strips the injected `Authorization` header from request logs entirely.
## Problem
`LOG_FULL` (log level 2) is intended for debugging egress traffic. When active it calls `_log_request` and `_log_response`. Both functions have two related bugs:
1. **Injected `Authorization` header exposure.** `_log_request` is called *after* the sidecar injects upstream credentials (`flow.request.headers["authorization"] = decision.inject_authorization`). The full header dict — including the live credential — is serialized to stderr. Any log collector that ingests the egress container's stderr will receive the upstream bearer token in plaintext.
2. **Unredacted bodies and header values.** Neither `_log_request` nor `_log_response` passes body or header values through `redact_tokens`. By contrast, `_req_ctx` (used for block/warn events) already calls `redact_tokens` on path and host. Any provisioned secret or recognized token pattern that appears in a request body, response body, or non-Authorization header value will be logged verbatim under `LOG_FULL`.
These two bugs compose: an agent that enables `LOG_FULL` and simultaneously triggers a request that carries a known token gains a write path from credentials → egress logs.
## Goals / Success Criteria
- `_log_request` never logs the `authorization` header in any form.
- `_log_request` applies `redact_tokens(value, env=os.environ)` to every other header value before serializing.
- `_log_request` applies `redact_tokens(body, env=os.environ)` to the request body before logging.
- `_log_response` applies `redact_tokens(value, env=os.environ)` to every response header value before logging.
- `_log_response` applies `redact_tokens(body, env=os.environ)` to the response body before logging.
- Unit tests cover each of the five cases above.
## Non-goals
- Redacting host or path in the full-log path (already covered by `_req_ctx` for block/warn events; `_log_request` already calls `redact_tokens` on host and path).
- Suppressing `LOG_FULL` or adding a new log level.
- Changing the outbound DLP scan logic.
## Design
### `_log_request`
```python
def _log_request(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.request.headers.items()
if k.lower() != "authorization"
}
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": headers,
"body": body,
})
+ "\n"
)
```
The `authorization` key is excluded because by the time `_log_request` is called the sidecar has already injected the upstream credential (`decision.inject_authorization`). Logging it would write a live bearer token to stderr on every allowed request. There is no safe subset to log — the value is always a live credential or empty.
### `_log_response`
```python
def _log_response(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.response.headers.items()
}
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": headers,
"body": body,
})
+ "\n"
)
```
Response headers don't carry injected credentials, so no header name is suppressed — only the values are scrubbed by `redact_tokens`.
+166
View File
@@ -0,0 +1,166 @@
# PRD 0065: Multi-parent `extends:` for bottles
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-25
- **Issue:** #268
- **Extends:** PRD 0025 (`0025-bottle-extends.md`)
## Summary
Allow a bottle's `extends:` field to accept either a single bottle name (existing
behavior) or a list of bottle names (new). Multiple parents are resolved
independently and folded left-to-right into a single effective parent before the
child is merged on top. This lets orthogonal concerns (base env, networking/egress,
agent provider) live in separate bottles and be composed without forcing them into a
linear chain.
## Problem
PRD 0025 shipped single-parent `extends:` and listed "No multi-parent inheritance"
as a non-goal. In practice, users want to compose multiple orthogonal bottles — a
base environment, a networking profile, and an agent-provider override — without
creating a three-level linear chain that couples unrelated parents to each other.
The linear chain workaround has two problems:
1. **Ordering constraint.** `networking extends base` works, but then
`agent extends networking` can't also pick up `base` without going through
`networking`, coupling two unrelated concerns.
2. **Quadratic duplication.** N orthogonal bottles require O(N²) chain variants
(one chain per permutation of applied concerns).
Multi-parent `extends:` removes both constraints: each orthogonal concern stays in
its own bottle, and the child bottle is the only place that names the combination.
## Goals / Success Criteria
- `extends:` accepts a list of strings in addition to a plain string.
- Backward compat: existing single-string `extends:` is unchanged.
- Parents are resolved left-to-right; later entries win on conflict.
- Child wins over all parents (unchanged from PRD 0025).
- Cycle detection covers multi-parent graphs, not just linear chains.
- Diamond inheritance: a shared ancestor is resolved once (via the existing cache).
- Invalid list entries (non-string, undefined bottle, self-reference) die at parse
with clear messages.
- `manifest_loader.py`'s `load_bottle_chain_from_dir` enqueues all parents from a
list `extends:` so the resolver sees every bottle in the graph.
## Non-goals
- No change to the agent-vs-bottle trust boundary (PRD 0025 "Alternatives
considered" option 2 stays rejected).
- No MRO / C3 linearization. Left-to-right fold is sufficient for the expected use
cases.
- No preflight display of per-field provenance across multiple parents (same open
question as PRD 0025; remains a follow-up).
## Design
### Schema
`extends:` now accepts either form:
```yaml
# single parent (unchanged)
extends: base
# multiple parents (new)
extends: [base, networking]
```
Both forms are normalized to a list internally. A list with one element behaves
identically to the string form.
### Merge rules for multi-parent fold
Parents are folded pairwise left-to-right before the child merge. For each step in
the fold, the "earlier" bottle is the running accumulator and the "later" bottle is
the next parent. Rules per field:
| Field | Fold rule |
|--------------------|--------------------------------------------------------------|
| `env` | dict merge; later wins on key collision |
| `git-gate.user` | per-field overlay; later's non-empty fields win |
| `git-gate.repos` | union by name; for same-name entries, later wins per-field |
| `egress.routes` | concatenate (earlier first, later appended) |
| `egress.log` | later wins (last-wins) |
| `agent_provider` | later wins (last-wins) |
| `supervise` | later wins (last-wins) |
After the fold, the combined parent is merged against the child using the existing
PRD 0025 rules (child always wins). The child's `egress.routes` appends to the
combined parent's concatenated routes; `validate_egress_routes` runs once on the
final merged set and catches duplicate hosts.
### Algorithm
```
extends: [p1, p2, p3]
fold:
combined = resolve(p1)
combined = fold_two(combined, resolve(p2))
combined = fold_two(combined, resolve(p3))
merge:
result = _merge_bottles(combined, child_raw, name)
```
`fold_two(earlier, later)` applies the rules in the table above. Cycle detection
(the `seen` tuple) is passed to each parent resolution call unchanged — if any
parent's chain circles back to the current bottle, it is caught. The `cache` dict
ensures a shared ancestor is only resolved once across all parents.
### Error cases
| Condition | Error message shape |
|----------------------------------------|------------------------------------------------------------------|
| `extends` is not a string or list | `extends must be a string or list of strings (was <type>)` |
| A list entry is not a string | `extends[<i>] must be a string (was <type>)` |
| A list entry names an undefined bottle | `extends '<name>' which is not defined. Available bottles: ...` |
| A list entry is the bottle itself | `extends itself; remove the self-reference` |
| Cycle through any parent edge | `is in an extends cycle: <chain>` |
## Implementation
### `bot_bottle/manifest_extends.py`
- `_resolve_one_bottle`: accept `str | list[str]` for `extends`; normalize to list;
validate each entry; for a single-entry list fall through to the existing
single-parent path; for multiple entries call `_fold_parents` then
`_merge_bottles`.
- `_fold_parents(parent_names, raws, cache, repos_cache, seen)`: resolve each
parent and fold pairwise left-to-right; return `(effective_bottle,
effective_repos_raw)`.
- `_fold_two_bottles(earlier, earlier_repos_raw, later, later_repos_raw)`: apply
the fold rules above; return `(folded_bottle, folded_repos_raw)`.
### `bot_bottle/manifest_loader.py`
- `load_bottle_chain_from_dir`: when `extends` is a list, enqueue all parent names
for loading (previously only `isinstance(parent, str)` was handled).
### `tests/unit/test_manifest_extends.py`
- `TestExtendsErrors.test_non_string_extends_dies`: update to use an integer
`extends` value (a list is now valid).
- New class `TestExtendsMultiParent` covering all cases listed in the issue.
## Testing strategy
Unit tests via `ManifestIndex.from_json_obj` (same resolver surface used by all
paths). No integration test changes needed — downstream code consumes the already-
merged bottle and is unchanged.
Test cases:
- Two-parent list: env union, egress routes concat, git repos union
- Last-parent-wins on scalar (supervise, agent_provider)
- Child wins over all parents on conflict
- Diamond: two parents share an ancestor; ancestor resolved once
- Single-element list: identical to string form
- Non-string extends value → ManifestError
- Non-string list entry → ManifestError
- Undefined bottle in list → ManifestError
- Self-reference in list → ManifestError
- Cycle through multi-parent edge → ManifestError
-75
View File
@@ -1,75 +0,0 @@
# PRD prd-new: Install script
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-06
- **Issue:** #197
## Summary
Add a proper Python package distribution and a thin `install.sh` bootstrapper so users can install bot-bottle with a single command without cloning the repo.
## Problem
There is currently no install path for new users. The only way to run bot-bottle is to clone the repo and invoke `cli.py` directly. This blocks any HN-style public demo: readers want `curl | sh` or `pipx install`, not a manual clone-and-configure flow.
## Goals / Success Criteria
- `curl -fsSL <url>/install.sh | sh` (or equivalent) leaves a working `bot-bottle` command on PATH.
- Python-native users can install with `pipx install bot-bottle` or `uv tool install bot-bottle`.
- `install.sh` validates prerequisites (Python ≥ 3.11, Docker) and exits with a clear message if they are missing. It does not silently install Docker.
- `install.sh` runs `bot-bottle doctor` (or equivalent diagnostic) after install to confirm the environment is ready.
- The package has no runtime pip dependencies (stdlib-only, matching the existing constraint).
## Non-goals
- Bundling a Python runtime or producing a standalone binary.
- Automatic Docker installation.
- Plugin architecture changes (out of scope; see issue #197 for future direction).
- Publishing to PyPI in this PR — the package structure is the deliverable; publishing is a separate step.
## Design
### Package structure
Add a minimal `pyproject.toml` at the repo root:
```toml
[project]
name = "bot-bottle"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = []
[project.scripts]
bot-bottle = "bot_bottle.cli:main"
```
The existing `bot_bottle/` package and `cli.py` entry point already contain the logic; this just wires up the standard entry point. `cli.py` may need a small refactor to expose a `main()` callable if it uses `if __name__ == "__main__"` only.
### `install.sh`
A thin bootstrapper that:
1. Checks `python3 --version` ≥ 3.11; exits with instructions if not met.
2. Checks `docker info` exits 0; exits with instructions if Docker is not running.
3. Installs via `pipx` if available, otherwise falls back to `pip install --user`.
4. Runs `bot-bottle doctor` to verify the install.
The script must be idempotent (safe to re-run) and must not require `sudo`.
### `bot-bottle doctor`
A new subcommand that checks and reports:
- Python version.
- Docker daemon reachability.
- Whether `~/.bot-bottle/` config directory exists.
Exits 0 if all checks pass, non-zero otherwise.
## Decisions
- `install.sh` is hosted from the repo's raw Gitea URL for now:
`https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh`.
- Should `version` in `pyproject.toml` be driven by a git tag at build time (e.g. via `hatch-vcs`) or kept as a static string? Static is simpler for now.
@@ -22,7 +22,7 @@ escapes**, and **whether credentials are short-lived and scoped**.
- Outbound: Docker containers have full internet access by default; no egress monitoring on most home networks
- Lateral movement: compromised container can reach the LAN — NAS, other machines, internal services
- Notable: CVE-2025-59536 (CVSS 8.7, Feb 2026) — a poisoned `.claude/settings.json` in a repo gives RCE when Claude Code opens it. `--dangerously-skip-permissions` removes the last gate.
- Supply chain: MCP servers, skills, and npm packages pulled during agent execution. ~20% of ClawHub skills were found malicious in early 2026.
- Supply chain: MCP servers, skills, and npm packages pulled during agent execution. A Jan 2026 large-scale empirical study of a 98,380-skill snapshot confirmed 157 malicious skills, ~71% of them credential harvesters. Exfiltration was overwhelmingly naive — plaintext HTTP to hardcoded endpoints; under 10% used any code obfuscation, and concealment was mostly at the documentation level, not the code level. ([Malicious Agent Skills in the Wild](https://arxiv.org/html/2602.06547v1), arXiv:2602.06547)
**What local topology protects:**
- No inbound attack surface — nothing listening on a public port
-50
View File
@@ -1,50 +0,0 @@
#!/bin/sh
set -eu
PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}"
MIN_PYTHON="3.11"
say() {
printf 'bot-bottle install: %s\n' "$*" >&2
}
die() {
say "error: $*"
exit 1
}
command -v python3 >/dev/null 2>&1 || die "python3 is required (version ${MIN_PYTHON} or newer)"
python3 - <<'PY' || die "python3 3.11 or newer is required"
import sys
raise SystemExit(0 if sys.version_info >= (3, 11) else 1)
PY
command -v docker >/dev/null 2>&1 || die "Docker is required; install Docker and start the daemon, then re-run this script"
docker info >/dev/null 2>&1 || die "Docker is installed but the daemon is not reachable; start Docker and re-run this script"
mkdir -p \
"${HOME}/.bot-bottle/agents" \
"${HOME}/.bot-bottle/bottles" \
"${HOME}/.bot-bottle/contrib"
if command -v pipx >/dev/null 2>&1; then
say "installing with pipx"
pipx install --force "${PACKAGE_SPEC}"
else
say "pipx not found; installing with python3 -m pip --user"
python3 -m pip install --user --upgrade "${PACKAGE_SPEC}"
fi
if command -v bot-bottle >/dev/null 2>&1; then
BOT_BOTTLE_BIN="bot-bottle"
elif [ -x "${HOME}/.local/bin/bot-bottle" ]; then
BOT_BOTTLE_BIN="${HOME}/.local/bin/bot-bottle"
say "using ${BOT_BOTTLE_BIN}; add ${HOME}/.local/bin to PATH for future shells"
else
die "bot-bottle was installed but is not on PATH"
fi
say "running bot-bottle doctor"
"${BOT_BOTTLE_BIN}" doctor
-27
View File
@@ -1,27 +0,0 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "bot-bottle"
version = "0.1.0"
description = "Self-hosted sandbox for AI coding agents with egress controls"
readme = "README.md"
requires-python = ">=3.11"
license = { text = "Apache-2.0" }
dependencies = []
[project.scripts]
bot-bottle = "bot_bottle.cli:main"
[tool.setuptools.packages.find]
include = ["bot_bottle*"]
[tool.setuptools.package-data]
bot_bottle = [
"Dockerfile.sidecars",
"egress_entrypoint.sh",
"contrib/claude/Dockerfile",
"contrib/codex/Dockerfile",
"contrib/pi/Dockerfile",
]
+1
View File
@@ -4,3 +4,4 @@
pylint>=3.0.0
pyright>=1.1.300
coverage>=7.0.0
+7 -4
View File
@@ -92,9 +92,9 @@ class TestSandboxEscape(unittest.TestCase):
"on PATH: curl -sSL https://smolmachines.com/install.sh | sh"
)
# Throwaway "identity file" for the git-gate's `identity` field.
# It need not be a real SSH key: test 5 reaches gitleaks before
# any SSH attempt anyway.
# Throwaway static key for the git-gate fixture. It need not
# be a real SSH key: test 5 reaches gitleaks before any SSH
# attempt anyway.
fd, kp = tempfile.mkstemp(prefix="sandbox-test-key.")
os.close(fd)
cls._key_path = Path(kp)
@@ -123,7 +123,10 @@ class TestSandboxEscape(unittest.TestCase):
"git-gate": {"repos": {
"throwaway": {
"url": "ssh://git@unreachable.invalid:22/throwaway.git",
"identity": str(cls._key_path),
"key": {
"provider": "static",
"path": str(cls._key_path),
},
},
}},
},
@@ -198,6 +198,7 @@ class TestSmolmachinesLaunch(unittest.TestCase):
# connect fails, which is the property chunk 3 will
# preserve once egress is actually running.
r = self.bottle.exec(
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
f"curl -s --show-error --max-time 3 http://{self.plan.bundle_ip}:9099 "
"2>&1 || true"
)
+46
View File
@@ -168,6 +168,34 @@ class TestAgentProviderRuntime(unittest.TestCase):
self.assertEqual("~/.claude/statusline.sh", settings["statusLine"]["command"])
self.assertEqual("custom:bot-bottle-research-ui", settings["theme"])
def test_claude_plan_uses_startup_args_from_provider_settings(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = build_agent_provision_plan(
template="claude",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
provider_settings={
"startup_args": ["--model", "opus"],
},
)
self.assertEqual(("--model", "opus"), plan.startup_args)
def test_codex_plan_uses_startup_args_from_provider_settings(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = build_agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
provider_settings={
"startup_args": ["--model", "gpt-5-codex"],
},
)
self.assertEqual(("--model", "gpt-5-codex"), plan.startup_args)
def test_codex_forward_host_credentials_populates_egress_routes(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
home = Path(tmp) / "host-codex"
@@ -394,6 +422,24 @@ class TestAgentProviderRuntime(unittest.TestCase):
self.assertNotIn("OPENROUTER_API_KEY", plan.guest_env)
self.assertTrue(provider["compat"]["supportsReasoningEffort"])
def test_pi_plan_appends_startup_args_from_provider_settings(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = build_agent_provision_plan(
template="pi",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
provider_settings={
"models": ["qwen3:14b"],
"startup_args": ["--no-stream"],
},
)
self.assertEqual(
("--models", "ollama/qwen3:14b", "--no-stream"),
plan.startup_args,
)
def test_pi_prompt_mode_appends_system_prompt_interactively(self):
self.assertEqual(
["--append-system-prompt", "/home/node/.bot-bottle-prompt.txt"],
+2 -2
View File
@@ -115,8 +115,8 @@ class TestBottleIdentity(unittest.TestCase):
class TestPreserveMarker(_FakeHomeMixin, unittest.TestCase):
"""The .preserve marker is how capability_apply tells cli.py's
session-end cleanup to keep the state dir instead of removing it."""
"""The .preserve marker tells cli.py's session-end cleanup to keep
the state dir instead of removing it."""
def setUp(self):
self._setup_fake_home()
-51
View File
@@ -1,51 +0,0 @@
"""Unit: `bot-bottle doctor` host prerequisite checks."""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest.mock import MagicMock, patch
from bot_bottle.cli import doctor
class TestDoctor(unittest.TestCase):
def test_success_when_prerequisites_present(self):
with tempfile.TemporaryDirectory() as tmp, patch.object(
doctor.Path, "home", return_value=Path(tmp),
), patch.object(
doctor.shutil, "which", return_value="/usr/bin/docker",
), patch.object(
doctor.subprocess, "run",
return_value=MagicMock(returncode=0),
):
Path(tmp, ".bot-bottle").mkdir()
self.assertEqual(0, doctor.cmd_doctor([]))
def test_missing_config_fails(self):
with tempfile.TemporaryDirectory() as tmp, patch.object(
doctor.Path, "home", return_value=Path(tmp),
), patch.object(
doctor.shutil, "which", return_value="/usr/bin/docker",
), patch.object(
doctor.subprocess, "run",
return_value=MagicMock(returncode=0),
):
self.assertEqual(1, doctor.cmd_doctor([]))
def test_missing_docker_fails_before_daemon_check(self):
with tempfile.TemporaryDirectory() as tmp, patch.object(
doctor.Path, "home", return_value=Path(tmp),
), patch.object(
doctor.shutil, "which", return_value=None,
), patch.object(
doctor.subprocess, "run",
) as run:
Path(tmp, ".bot-bottle").mkdir()
self.assertEqual(1, doctor.cmd_doctor([]))
run.assert_not_called()
if __name__ == "__main__":
unittest.main()
+23 -2
View File
@@ -29,8 +29,8 @@ class _FakeHomeMixin:
class TestCaptureSessionState(_FakeHomeMixin, unittest.TestCase):
# snapshot_transcript is commented out (capability_apply is disabled);
# capture_claude_session_state now only handles the preserve marker.
# capture_claude_session_state handles the preserve marker for
# non-zero agent exits.
def setUp(self):
self._setup_fake_home()
@@ -102,6 +102,27 @@ class TestAttachAgent(unittest.TestCase):
bottle.argv,
)
def test_remote_control_is_provider_startup_arg(self):
class Bottle:
argv: list[str] = []
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
self.argv = list(argv)
return 0
bottle = Bottle()
exit_code = start_mod.attach_agent(
bottle, # type: ignore[arg-type]
agent_provider_template="codex",
startup_args=("remote-control",),
)
self.assertEqual(0, exit_code)
self.assertEqual(
["--dangerously-bypass-approvals-and-sandbox", "remote-control"],
bottle.argv,
)
if __name__ == "__main__":
unittest.main()
+26 -26
View File
@@ -80,7 +80,11 @@ def _git_gate_plan(upstreams: tuple[GitGateUpstream, ...] = ()) -> GitGatePlan:
)
def _egress_plan(routes: tuple[EgressRoute, ...] = ()) -> EgressPlan:
def _egress_plan(
routes: tuple[EgressRoute, ...] = (),
*,
canary: bool = False,
) -> EgressPlan:
token_env_map = {
r.token_env: r.token_ref
for r in routes
@@ -95,6 +99,8 @@ def _egress_plan(routes: tuple[EgressRoute, ...] = ()) -> EgressPlan:
egress_network=f"bot-bottle-egress-{SLUG}",
mitmproxy_ca_host_path=STATE / "egress-ca" / "mitmproxy-ca.pem",
mitmproxy_ca_cert_only_host_path=STATE / "egress-ca" / "ca.pem",
canary="fake-canary-value" if canary else "",
canary_env="CANON_ALPHA_SECRET" if canary else "",
)
@@ -102,7 +108,6 @@ def _supervise_plan() -> SupervisePlan:
return SupervisePlan(
slug=SLUG,
queue_dir=STATE / "supervise" / "queue",
current_config_dir=STATE / "supervise" / "current-config",
internal_network=f"bot-bottle-net-{SLUG}",
)
@@ -112,6 +117,7 @@ def _plan(
with_git: bool = False,
with_egress: bool = False,
supervise: bool = False,
canary: bool = False,
) -> DockerBottlePlan:
"""Build a fully-resolved DockerBottlePlan. Toggles cover the
matrix the renderer's conditional-service logic branches on."""
@@ -150,7 +156,7 @@ def _plan(
slug=SLUG,
forwarded_env={"CLAUDE_CODE_OAUTH_TOKEN": "x"},
git_gate_plan=_git_gate_plan(upstreams),
egress_plan=_egress_plan(routes),
egress_plan=_egress_plan(routes, canary=canary),
supervise_plan=_supervise_plan() if supervise else None,
use_runsc=False,
agent_provision=AgentProvisionPlan(
@@ -264,18 +270,11 @@ class TestAgentAlwaysPresent(unittest.TestCase):
s = bottle_plan_to_compose(_plan(**kwargs))["services"]["agent"]
self.assertEqual(["sidecars"], s["depends_on"])
def test_agent_current_config_mount_only_with_supervise(self):
def test_agent_has_no_current_config_mount_with_supervise(self):
with_sv = bottle_plan_to_compose(_plan(supervise=True))["services"]["agent"]
self.assertTrue(any(
v["target"] == "/etc/bot-bottle/current-config"
for v in with_sv.get("volumes", [])
))
self.assertNotIn("volumes", with_sv)
without_sv = bottle_plan_to_compose(_plan(supervise=False))["services"]["agent"]
# Either no volumes key at all, or no current-config target.
self.assertFalse(any(
v["target"] == "/etc/bot-bottle/current-config"
for v in without_sv.get("volumes", [])
))
self.assertNotIn("volumes", without_sv)
class TestSidecarBundleShape(unittest.TestCase):
@@ -301,19 +300,6 @@ class TestSidecarBundleShape(unittest.TestCase):
self.assertEqual("bot-bottle-sidecars:latest", sc["image"])
self.assertEqual("Dockerfile.sidecars", sc["build"]["dockerfile"])
def test_bundle_uses_packaged_dockerfile_when_root_missing(self):
from bot_bottle.backend.docker import compose as compose_mod
original = compose_mod._REPO_DIR
try:
compose_mod._REPO_DIR = "/tmp/does-not-exist"
self.assertEqual(
"bot_bottle/Dockerfile.sidecars",
compose_mod._sidecar_bundle_dockerfile(),
)
finally:
compose_mod._REPO_DIR = original
def test_bundle_container_name_uses_sidecars_prefix(self):
sc = self._render()["services"]["sidecars"]
self.assertEqual(f"bot-bottle-sidecars-{SLUG}", sc["container_name"])
@@ -388,6 +374,20 @@ class TestSidecarBundleShape(unittest.TestCase):
env_strings = sc["environment"]
self.assertNotIn("EGRESS_TOKEN_0", env_strings)
def test_canary_env_registered_as_sensitive_in_sidecar(self):
sc = self._render(canary=True)["services"]["sidecars"]
env_strings = sc["environment"]
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", env_strings)
self.assertIn(
"BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET",
env_strings,
)
def test_canary_env_visible_to_agent(self):
agent = self._render(canary=True)["services"]["agent"]
env_strings = agent["environment"]
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", env_strings)
def test_supervise_env_present_when_active(self):
sc = self._render(supervise=True)["services"]["sidecars"]
env_strings = sc["environment"]
@@ -75,7 +75,6 @@ def _plan(
supervise_plan = SupervisePlan(
slug="demo-abc12",
queue_dir=Path("/tmp/queue"),
current_config_dir=Path("/tmp/current-config"),
)
return DockerBottlePlan(
spec=spec,
+9 -1
View File
@@ -29,6 +29,9 @@ from bot_bottle.supervise import SupervisePlan
_URL = "http://supervise:9100/"
_CODEX_DOCKERFILE = (
Path(__file__).resolve().parents[2] / "bot_bottle/contrib/codex/Dockerfile"
)
def _make_bottle(exec_result: ExecResult | None = None) -> MagicMock:
@@ -75,7 +78,6 @@ def _plan(
supervise_plan = SupervisePlan(
slug="demo-abc12",
queue_dir=Path("/tmp/queue"),
current_config_dir=Path("/tmp/current-config"),
)
return DockerBottlePlan(
spec=spec,
@@ -276,6 +278,12 @@ class TestCodexProvision(unittest.TestCase):
)
class TestCodexDockerfile(unittest.TestCase):
def test_installs_procps_for_remote_control_pid_management(self):
dockerfile = _CODEX_DOCKERFILE.read_text()
self.assertIn("procps", dockerfile)
class TestCodexSuperviseMcp(unittest.TestCase):
def test_noop_when_supervise_disabled(self):
bottle = _make_bottle()
@@ -10,8 +10,11 @@ from unittest.mock import MagicMock, patch
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
GiteaDeployKeyProvisioner,
_API_TIMEOUT_SECS,
_KEYGEN_TIMEOUT_SECS,
_split_owner_repo,
)
from bot_bottle.deploy_key_provisioner import DeployKeyCollisionError
def _provisioner() -> GiteaDeployKeyProvisioner:
@@ -82,6 +85,25 @@ class TestCreate(unittest.TestCase):
self.assertEqual(str(fake_key_id), key_id)
self.assertEqual(fake_private, private_bytes)
def test_create_passes_timeout_to_ssh_keygen_and_urlopen(self):
provisioner = _provisioner()
with patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.subprocess.run"
) as mock_run, patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.urllib.request.urlopen"
) as mock_urlopen, patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.Path.read_bytes",
return_value=b"PRIVATE",
), patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.Path.read_text",
return_value="ssh-ed25519 AAAA\n",
):
mock_urlopen.return_value = _urlopen_response({"id": 1})
provisioner.create("owner/repo", "title")
self.assertEqual(_KEYGEN_TIMEOUT_SECS, mock_run.call_args.kwargs.get("timeout"))
self.assertEqual(_API_TIMEOUT_SECS, mock_urlopen.call_args.kwargs.get("timeout"))
def test_create_raises_on_http_error(self):
provisioner = _provisioner()
with patch(
@@ -100,6 +122,30 @@ class TestCreate(unittest.TestCase):
provisioner.create("owner/repo", "title")
self.assertIn("403", str(ctx.exception))
def test_create_raises_collision_error_on_422(self):
provisioner = _provisioner()
collision_body = json.dumps({
"errors": ["Key content already exists on this repository"],
"message": "422 Unprocessable Entity",
})
with patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.subprocess.run"
), patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.urllib.request.urlopen",
side_effect=_http_error(422, collision_body),
), patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.Path.read_bytes",
return_value=b"pk",
), patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.Path.read_text",
return_value="ssh-ed25519 AAAA\n",
):
with self.assertRaises(DeployKeyCollisionError) as ctx:
provisioner.create("owner/repo", "my-title")
msg = str(ctx.exception)
self.assertIn("owner/repo", msg)
self.assertIn("my-title", msg)
class TestDelete(unittest.TestCase):
def test_delete_calls_correct_endpoint(self):
@@ -114,6 +160,16 @@ class TestDelete(unittest.TestCase):
self.assertIn("/api/v1/repos/didericis/bot-bottle/keys/99", req.full_url)
self.assertEqual("DELETE", req.get_method())
def test_delete_passes_timeout_to_urlopen(self):
provisioner = _provisioner()
with patch(
"bot_bottle.contrib.gitea.deploy_key_provisioner.urllib.request.urlopen"
) as mock_urlopen:
mock_urlopen.return_value = _urlopen_response({})
provisioner.delete("owner/repo", "7")
self.assertEqual(_API_TIMEOUT_SECS, mock_urlopen.call_args.kwargs.get("timeout"))
def test_delete_tolerates_404(self):
provisioner = _provisioner()
with patch(
+250 -2
View File
@@ -1,18 +1,23 @@
"""Unit: DLP detectors (PRD 0053).
Tests for token pattern scanning, known secret detection, and
naive prompt injection detection."""
Tests for token pattern scanning, known secret detection, fragmentation-
resistant matching, entropy scoring, and naive prompt injection detection."""
import base64
import gzip
import unittest
from bot_bottle.dlp_detectors import (
ENTROPY_BLOCK_THRESHOLD,
PARTIAL_MATCH_MIN_LEN,
REDACT,
_alnum_projection,
_encoded_variants,
_normalize_text,
_shannon_entropy,
redact_tokens,
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_naive_injection,
scan_token_patterns,
@@ -445,5 +450,248 @@ class TestKnownSecretsNewVariants(unittest.TestCase):
self.assertIsNotNone(result)
class TestMatchedAndSafeTokens(unittest.TestCase):
"""PRD 0062: detectors carry the raw matched value, and a safelisted
value is skipped so the supervisor can approve a specific token."""
def test_token_pattern_sets_matched(self):
token = "ghp_" + "A" * 36
result = scan_token_patterns(f"token: {token}")
assert result is not None
self.assertEqual(token, result.matched)
def test_safe_token_is_skipped(self):
token = "ghp_" + "A" * 36
self.assertIsNone(
scan_token_patterns(f"token: {token}", safe_tokens={token})
)
def test_safe_token_does_not_mask_other_token(self):
safe = "ghp_" + "A" * 36
other = "AKIAIOSFODNN7EXAMPLE"
result = scan_token_patterns(
f"a={safe} b={other}", safe_tokens={safe},
)
assert result is not None
self.assertEqual(other, result.matched)
self.assertIn("AWS", result.reason)
def test_known_secret_sets_matched_and_safelist_skips(self):
secret = "supersecretvalue123"
env = {"EGRESS_TOKEN_FOO": secret}
result = scan_known_secrets(f"x={secret}", env=env)
assert result is not None
self.assertEqual(secret, result.matched)
self.assertIsNone(
scan_known_secrets(f"x={secret}", env=env, safe_tokens={secret})
)
def test_crlf_block_has_no_matched_value(self):
result = scan_crlf_injection("path%0d%0aHost: evil")
assert result is not None
self.assertEqual("", result.matched)
class TestStripCrlf(unittest.TestCase):
def test_removes_url_encoded_crlf(self):
from bot_bottle.dlp_detectors import strip_crlf
out = strip_crlf("next=%0d%0aX-Injected: evil")
self.assertNotRegex(out, r"%0[dD]%0[aA]")
def test_removes_literal_header_injection(self):
from bot_bottle.dlp_detectors import strip_crlf
out = strip_crlf("value\r\nX-Injected: evil")
self.assertIsNone(scan_crlf_injection(out))
def test_leaves_clean_text_unchanged(self):
from bot_bottle.dlp_detectors import strip_crlf
self.assertEqual("/api/v1/data?q=hello", strip_crlf("/api/v1/data?q=hello"))
class TestAlnumProjection(unittest.TestCase):
def test_alphanumeric_unchanged(self):
self.assertEqual("abc123XYZ", _alnum_projection("abc123XYZ"))
def test_strips_hyphens(self):
self.assertEqual("mysecretvalue", _alnum_projection("my-secret-value"))
def test_strips_spaces(self):
self.assertEqual("mysecretvalue", _alnum_projection("my secret value"))
def test_strips_dots_and_underscores(self):
self.assertEqual("mysecretvalue", _alnum_projection("my.secret_value"))
def test_empty_string(self):
self.assertEqual("", _alnum_projection(""))
def test_all_special_chars(self):
self.assertEqual("", _alnum_projection("!@#$%^&*()"))
class TestFragmentationResistantMatching(unittest.TestCase):
"""scan_known_secrets catches separator-injection and partial-substring evasion."""
# Secrets long enough that their alnum projections are ≥ 8 chars.
SECRET = "supersecrettoken99"
ENV = {"EGRESS_TOKEN_0": SECRET}
def test_exact_match_still_works(self):
result = scan_known_secrets(f"key={self.SECRET}", env=self.ENV)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
def test_separator_injection_blocked(self):
# Hyphens inserted between chars of the secret.
fragmented = "-".join(self.SECRET)
result = scan_known_secrets(f"data={fragmented}", env=self.ENV)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
self.assertIn("separator injection", result.reason)
def test_space_separator_blocked(self):
fragmented = " ".join(self.SECRET)
result = scan_known_secrets(f"body: {fragmented}", env=self.ENV)
self.assertIsNotNone(result)
assert result is not None
self.assertIn("separator injection", result.reason)
def test_partial_substring_blocked(self):
# First PARTIAL_MATCH_MIN_LEN alnum chars of the secret, no separators.
partial = _alnum_projection(self.SECRET)[:PARTIAL_MATCH_MIN_LEN]
result = scan_known_secrets(f"x={partial}&y=other", env=self.ENV)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
self.assertIn("partial match", result.reason)
def test_short_secret_skips_projection(self):
# Secrets shorter than _ALNUM_MIN_LEN in alnum projection are not
# fragmentation-checked (too many false positives).
short_env = {"EGRESS_TOKEN_0": "abc"}
# "a b c" has alnum projection "abc" (3 chars, < 8); should not block.
self.assertIsNone(scan_known_secrets("a b c", env=short_env))
def test_clean_text_not_blocked(self):
self.assertIsNone(scan_known_secrets("nothing to see here", env=self.ENV))
def test_sensitive_prefixes_param_extra_prefix(self):
env = {"MY_CRED_0": self.SECRET, "IGNORED": "other"}
result = scan_known_secrets(
f"key={self.SECRET}",
env=env,
sensitive_prefixes=("MY_CRED_",),
)
self.assertIsNotNone(result)
assert result is not None
self.assertIn("MY_CRED_0", result.reason)
def test_sensitive_prefixes_default_only_egress_token(self):
# A value under a non-EGRESS_TOKEN_ key is ignored with default prefixes.
env = {"MY_CRED_0": self.SECRET}
self.assertIsNone(scan_known_secrets(f"key={self.SECRET}", env=env))
def test_canary_prefix_detected(self):
canary_value = "canary-fake-secret-value-xyz"
env = {"CANON_ALPHA_SECRET": canary_value}
result = scan_known_secrets(
f"x={canary_value}",
env=env,
sensitive_prefixes=("CANON_ALPHA_SECRET",),
)
self.assertIsNotNone(result)
assert result is not None
self.assertIn("CANON_ALPHA_SECRET", result.reason)
class TestRedactTokensBroadenedPrefixes(unittest.TestCase):
SECRET = "my-provisioned-secret"
def test_default_redacts_egress_token(self):
env = {"EGRESS_TOKEN_0": self.SECRET}
out = redact_tokens(f"val={self.SECRET}", env=env)
self.assertNotIn(self.SECRET, out)
self.assertIn(REDACT, out)
def test_extra_prefix_redacted(self):
env = {"MY_SECRET_KEY": self.SECRET}
out = redact_tokens(
f"val={self.SECRET}",
env=env,
sensitive_prefixes=("MY_SECRET_",),
)
self.assertNotIn(self.SECRET, out)
self.assertIn(REDACT, out)
def test_non_matching_prefix_not_redacted(self):
env = {"MY_SECRET_KEY": self.SECRET}
out = redact_tokens(f"val={self.SECRET}", env=env)
# Default prefixes only include EGRESS_TOKEN_ → secret not redacted
self.assertIn(self.SECRET, out)
class TestShannonEntropy(unittest.TestCase):
def test_empty_string_zero(self):
self.assertEqual(0.0, _shannon_entropy(""))
def test_single_char_zero(self):
self.assertEqual(0.0, _shannon_entropy("aaaaaa"))
def test_two_equal_chars_one_bit(self):
self.assertAlmostEqual(1.0, _shannon_entropy("abababab"), places=10)
def test_high_entropy_random_like(self):
# Uniform 64-char string over 64 distinct symbols has entropy 6 bits.
import string
alphabet = (string.ascii_letters + string.digits + "+/")[:64]
text = alphabet # each char appears exactly once
self.assertAlmostEqual(6.0, _shannon_entropy(text), places=10)
class TestScanEntropy(unittest.TestCase):
def test_empty_returns_none(self):
self.assertIsNone(scan_entropy(""))
def test_low_entropy_returns_none(self):
# Highly repetitive text has low entropy.
self.assertIsNone(scan_entropy("a" * 200))
def test_high_entropy_warns(self):
# Build a 64-char string with entropy > ENTROPY_BLOCK_THRESHOLD.
# Use all 64 distinct printable chars to maximise entropy (~6 bits).
import string
alphabet = (string.ascii_letters + string.digits + "+/")[:64]
result = scan_entropy(alphabet, threshold=ENTROPY_BLOCK_THRESHOLD)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("warn", result.severity)
self.assertIn("high-entropy", result.reason)
def test_never_blocks(self):
import string
alphabet = (string.ascii_letters + string.digits + "+/")[:64]
result = scan_entropy(alphabet)
# scan_entropy is warn-only; it must never return severity="block".
if result is not None:
self.assertNotEqual("block", result.severity)
def test_location_in_result(self):
import string
alphabet = (string.ascii_letters + string.digits + "+/")[:64]
result = scan_entropy(alphabet, location="authorization header")
if result is not None:
self.assertIn("authorization header", result.location)
def test_structured_json_no_warn(self):
# Typical JSON has low entropy and should not be flagged.
json_body = '{"status": "ok", "message": "hello world", "count": 42}'
self.assertIsNone(scan_entropy(json_body))
def test_short_text_below_window(self):
# Text shorter than the window: checked as one chunk.
# Use a uniform string to ensure it won't be flagged.
self.assertIsNone(scan_entropy("abcde", threshold=ENTROPY_BLOCK_THRESHOLD))
if __name__ == "__main__":
unittest.main()
+10
View File
@@ -136,6 +136,16 @@ class TestClaudeArgv(unittest.TestCase):
argv,
)
def test_codex_remote_control_startup_arg_does_not_receive_initial_prompt(self):
argv = _codex_bottle("/home/node/.bot-bottle-prompt.txt").agent_argv(
["--dangerously-bypass-approvals-and-sandbox", "remote-control"],
)
self.assertEqual(
["docker", "exec", "-it", "bot-bottle-dev-abc", "codex",
"--dangerously-bypass-approvals-and-sandbox", "remote-control"],
argv,
)
def test_codex_resume_does_not_append_initial_prompt(self):
argv = _codex_bottle("/home/node/.bot-bottle-prompt.txt").agent_argv(
["--dangerously-bypass-approvals-and-sandbox", "resume", "--last"],
+2 -2
View File
@@ -65,8 +65,8 @@ class TestOrphanStateDirs(_FakeHomeMixin, unittest.TestCase):
)
def test_preserve_marker_skips_dir(self):
# Preserve marker = capability-block or crash auto-preserve;
# the user explicitly wanted this dir kept for `resume`.
# Preserve marker means the user explicitly wanted this dir
# kept for `resume`.
bottle_state.write_per_bottle_dockerfile("kept-ccc", "FROM x\n")
bottle_state.mark_preserved("kept-ccc")
self.assertEqual(
@@ -31,7 +31,6 @@ class _Provider(AgentProvider):
return AgentProviderRuntime(
template="test", command="test", image="",
prompt_mode="append_file", bypass_args=(), resume_args=(),
remote_control_args=(),
)
def provision_plan(self, **kwargs): # type: ignore[override]
raise NotImplementedError
+231 -6
View File
@@ -1,15 +1,22 @@
"""Unit: Egress route lift + routes.yaml render + token
resolution (PRD 0017, PRD 0053)."""
import tempfile
import unittest
from pathlib import Path
from bot_bottle.egress import (
CODEX_HOST_CREDENTIAL_TOKEN_REF,
Egress,
EgressPlan,
EgressRoute,
_yaml_str_escape,
egress_agent_env_entries,
egress_manifest_routes,
egress_render_routes,
egress_resolve_token_values,
egress_routes_for_bottle,
egress_sidecar_env_entries,
egress_token_env_map,
)
from bot_bottle.log import Die
@@ -202,6 +209,23 @@ class TestProviderRouteMerge(unittest.TestCase):
self.assertEqual((), routes[0].matches)
self.assertEqual({}, egress_token_env_map(routes))
def test_provider_route_defaults_to_redact_on_match(self):
b = _bottle([])
pr = EgressRoute(host="api.anthropic.com")
routes = egress_routes_for_bottle(b, (pr,))
self.assertEqual("redact", routes[0].outbound_on_match)
def test_provider_route_explicit_on_match_preserved(self):
b = _bottle([])
pr = EgressRoute(host="api.anthropic.com", outbound_on_match="supervise")
routes = egress_routes_for_bottle(b, (pr,))
self.assertEqual("supervise", routes[0].outbound_on_match)
def test_manifest_route_does_not_get_redact_default(self):
b = _bottle([{"host": "api.example.com"}])
routes = egress_routes_for_bottle(b)
self.assertEqual("", routes[0].outbound_on_match)
def test_two_provider_routes_with_same_token_ref_share_slot(self):
b = _bottle([])
routes = egress_routes_for_bottle(b, (
@@ -299,7 +323,7 @@ class TestRenderRoutes(unittest.TestCase):
self.assertEqual([], parse_yaml_subset(rendered)["routes"])
def test_round_trip_through_addon_core(self):
from bot_bottle.egress_addon_core import load_routes
from bot_bottle.egress_addon_core import load_config
b = _bottle([
{"host": "api.github.com",
"auth": {"scheme": "Bearer", "token_ref": "GH_PAT"},
@@ -310,7 +334,7 @@ class TestRenderRoutes(unittest.TestCase):
{"host": "api.anthropic.com"},
])
routes = egress_routes_for_bottle(b)
addon_routes = load_routes(egress_render_routes(routes))
addon_routes = load_config(egress_render_routes(routes)).routes
self.assertEqual(3, len(addon_routes))
self.assertEqual("Bearer", addon_routes[0].auth_scheme)
self.assertEqual("EGRESS_TOKEN_0", addon_routes[0].token_env)
@@ -318,24 +342,41 @@ class TestRenderRoutes(unittest.TestCase):
self.assertEqual("", addon_routes[2].auth_scheme)
def test_dlp_round_trips(self):
from bot_bottle.egress_addon_core import load_routes
from bot_bottle.egress_addon_core import load_config
b = _bottle([{"host": "x.example", "dlp": {
"outbound_detectors": ["token_patterns"],
"inbound_detectors": False,
}}])
routes = egress_routes_for_bottle(b)
rendered = egress_render_routes(routes)
addon_routes = load_routes(rendered)
addon_routes = load_config(rendered).routes
self.assertEqual(("token_patterns",), addon_routes[0].outbound_detectors)
self.assertEqual((), addon_routes[0].inbound_detectors)
def test_outbound_on_match_round_trips(self):
from bot_bottle.egress_addon_core import load_config
b = _bottle([{"host": "logs.example", "dlp": {
"outbound_on_match": "redact",
}}])
routes = egress_routes_for_bottle(b)
rendered = egress_render_routes(routes)
self.assertIn('outbound_on_match: "redact"', rendered)
addon_routes = load_config(rendered).routes
self.assertEqual("redact", addon_routes[0].outbound_on_match)
def test_outbound_on_match_default_omitted_from_render(self):
b = _bottle([{"host": "x.example"}])
routes = egress_routes_for_bottle(b)
rendered = egress_render_routes(routes)
self.assertNotIn("outbound_on_match", rendered)
def test_git_fetch_policy_round_trips(self):
from bot_bottle.egress_addon_core import load_routes
from bot_bottle.egress_addon_core import load_config
b = _bottle([{"host": "github.com", "git": {"fetch": True}}])
routes = egress_routes_for_bottle(b)
rendered = egress_render_routes(routes)
self.assertEqual({"fetch": True}, self._parsed(routes)[0]["git"])
addon_routes = load_routes(rendered)
addon_routes = load_config(rendered).routes
self.assertTrue(addon_routes[0].git_fetch)
def test_log_zero_omitted_from_render(self):
@@ -379,6 +420,76 @@ class TestRenderRoutes(unittest.TestCase):
self.assertEqual(LOG_BLOCKS, cfg.log)
class TestYamlStrEscape(unittest.TestCase):
"""_yaml_str_escape produces safe YAML double-quoted scalar content."""
def test_plain_string_unchanged(self):
self.assertEqual("api.example.com", _yaml_str_escape("api.example.com"))
def test_double_quote_escaped(self):
self.assertEqual('\\"', _yaml_str_escape('"'))
def test_backslash_escaped(self):
self.assertEqual("\\\\", _yaml_str_escape("\\"))
def test_newline_escaped(self):
self.assertEqual("\\n", _yaml_str_escape("\n"))
def test_carriage_return_escaped(self):
self.assertEqual("\\r", _yaml_str_escape("\r"))
def test_tab_escaped(self):
self.assertEqual("\\t", _yaml_str_escape("\t"))
def test_combined(self):
self.assertEqual('\\"\\n\\\\', _yaml_str_escape('"\n\\'))
class TestRenderRoutesEscaping(unittest.TestCase):
"""Stray quotes/newlines in manifest strings do not corrupt routes.yaml."""
@staticmethod
def _parsed(routes) -> list[dict]: # type: ignore
return parse_yaml_subset(egress_render_routes(routes))["routes"] # type: ignore
def test_host_with_double_quote_round_trips(self):
routes = (EgressRoute(host='bad"host.example'),)
parsed = self._parsed(routes)
self.assertEqual('bad"host.example', parsed[0]["host"])
def test_host_with_newline_round_trips(self):
routes = (EgressRoute(host="host\nextra.example"),)
parsed = self._parsed(routes)
self.assertEqual("host\nextra.example", parsed[0]["host"])
def test_auth_scheme_with_double_quote_round_trips(self):
routes = (EgressRoute(
host="api.example",
auth_scheme='Bear"er',
token_env="EGRESS_TOKEN_0",
),)
parsed = self._parsed(routes)
self.assertEqual('Bear"er', parsed[0]["auth_scheme"])
def test_path_value_with_double_quote_round_trips(self):
from bot_bottle.egress_addon_core import PathMatch, MatchEntry
routes = (EgressRoute(
host="api.example",
matches=(MatchEntry(paths=(PathMatch(type="prefix", value='/v1/"quoted"/'),)),),
),)
parsed = self._parsed(routes)
self.assertEqual('/v1/"quoted"/', parsed[0]["matches"][0]["paths"][0]["value"])
def test_header_value_with_double_quote_round_trips(self):
from bot_bottle.egress_addon_core import HeaderMatch, MatchEntry
routes = (EgressRoute(
host="api.example",
matches=(MatchEntry(headers=(HeaderMatch(name="x-h", value='val"ue'),)),),
),)
parsed = self._parsed(routes)
self.assertEqual('val"ue', parsed[0]["matches"][0]["headers"][0]["value"])
class TestResolveTokenValues(unittest.TestCase):
def test_reads_host_env(self):
out = egress_resolve_token_values(
@@ -409,5 +520,119 @@ class TestResolveTokenValues(unittest.TestCase):
self.assertEqual({"EGRESS_TOKEN_0": "codex-access-token"}, out)
class TestCanaryGeneration(unittest.TestCase):
"""Egress.prepare() generates a unique canary token per session."""
def _bottle_obj(self):
return ManifestIndex.from_json_obj({
"bottles": {"dev": {"egress": {"routes": []}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}).bottles["dev"]
def _make_plan(self) -> EgressPlan:
# Use a concrete no-op subclass so we can call prepare() without
# a real backend.
class _TestEgress(Egress):
pass
e = _TestEgress()
with tempfile.TemporaryDirectory() as td:
return e.prepare(self._bottle_obj(), "test-slug", Path(td))
def test_canary_is_non_empty(self):
plan = self._make_plan()
self.assertIsInstance(plan.canary, str)
self.assertGreater(len(plan.canary), 0)
self.assertRegex(plan.canary_env, r"^[A-Z]+_[A-Z]+_SECRET$")
def test_canary_is_unique_per_session(self):
with tempfile.TemporaryDirectory() as td:
bottle = self._bottle_obj()
class _TestEgress(Egress):
pass
e = _TestEgress()
plan_a = e.prepare(bottle, "slug-a", Path(td))
plan_b = e.prepare(bottle, "slug-b", Path(td))
self.assertNotEqual(plan_a.canary, plan_b.canary)
def test_canary_detected_by_scan_known_secrets(self):
from bot_bottle.dlp_detectors import scan_known_secrets
plan = self._make_plan()
env = {plan.canary_env: plan.canary}
result = scan_known_secrets(
f"exfil={plan.canary}",
env=env,
sensitive_prefixes=(plan.canary_env,),
)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
self.assertIn(plan.canary_env, result.reason)
def test_egress_plan_canary_field_default_empty(self):
# Verify EgressPlan can be constructed with an empty canary (backward compat).
from pathlib import Path
plan = EgressPlan(
slug="s",
routes_path=Path("/tmp/r.yaml"),
routes=(),
token_env_map={},
)
self.assertEqual("", plan.canary)
self.assertEqual("", plan.canary_env)
class TestEgressEnvEntries(unittest.TestCase):
def test_sidecar_entries_include_route_tokens_and_canary_scan_prefix(self):
plan = EgressPlan(
slug="s",
routes_path=Path("/tmp/r.yaml"),
routes=(EgressRoute(host="api.example"),),
token_env_map={"EGRESS_TOKEN_1": "T1", "EGRESS_TOKEN_0": "T0"},
canary="fake-canary-value",
canary_env="CANON_ALPHA_SECRET",
)
self.assertEqual(
(
"EGRESS_TOKEN_0",
"EGRESS_TOKEN_1",
"CANON_ALPHA_SECRET=fake-canary-value",
"BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET",
),
egress_sidecar_env_entries(plan),
)
def test_agent_entries_include_only_canary_bait(self):
plan = EgressPlan(
slug="s",
routes_path=Path("/tmp/r.yaml"),
routes=(),
token_env_map={},
canary="fake-canary-value",
canary_env="CANON_ALPHA_SECRET",
)
self.assertEqual(
("CANON_ALPHA_SECRET=fake-canary-value",),
egress_agent_env_entries(plan),
)
def test_canary_entries_omitted_when_name_missing(self):
plan = EgressPlan(
slug="s",
routes_path=Path("/tmp/r.yaml"),
routes=(),
token_env_map={},
canary="fake-canary-value",
)
self.assertEqual((), egress_sidecar_env_entries(plan))
self.assertEqual((), egress_agent_env_entries(plan))
if __name__ == "__main__":
unittest.main()
+234 -38
View File
@@ -22,15 +22,16 @@ from bot_bottle.egress_addon_core import (
MatchEntry,
PathMatch,
Route,
ScanResult,
build_inbound_scan_text,
build_outbound_scan_text,
build_token_allow_payload,
decide,
decide_git_fetch,
evaluate_matches,
is_git_fetch_request,
is_git_push_request,
load_config,
load_routes,
match_route,
outbound_scan_headers,
parse_config,
@@ -267,46 +268,24 @@ class TestParseDlp(unittest.TestCase):
"dlp": {"wat": True},
}]})
def test_outbound_on_match_default_empty(self):
routes = parse_routes({"routes": [{"host": "x.example"}]})
self.assertEqual("", routes[0].outbound_on_match)
# --- load_routes ---------------------------------------------------------
def test_outbound_on_match_parsed(self):
for policy in ("block", "redact", "supervise"):
routes = parse_routes({"routes": [{
"host": "x.example",
"dlp": {"outbound_on_match": policy},
}]})
self.assertEqual(policy, routes[0].outbound_on_match)
class TestLoadRoutes(unittest.TestCase):
def test_yaml_text_round_trip(self):
routes = load_routes(
'routes:\n'
' - host: "api.example"\n'
)
self.assertEqual(1, len(routes))
self.assertEqual("api.example", routes[0].host)
def test_full_route_shape_parses(self):
routes = load_routes(
'routes:\n'
' - host: "api.example"\n'
' auth_scheme: "Bearer"\n'
' token_env: "EGRESS_TOKEN_0"\n'
' matches:\n'
' - paths:\n'
' - value: "/v1/"\n'
' - type: "exact"\n'
' value: "/messages"\n'
)
self.assertEqual(1, len(routes))
r = routes[0]
self.assertEqual("api.example", r.host)
self.assertEqual("Bearer", r.auth_scheme)
self.assertEqual("EGRESS_TOKEN_0", r.token_env)
self.assertEqual(1, len(r.matches))
self.assertEqual(2, len(r.matches[0].paths))
def test_empty_routes_list(self):
routes = load_routes("routes: []\n")
self.assertEqual((), routes)
def test_invalid_yaml_raises_value_error(self):
def test_outbound_on_match_invalid_rejected(self):
with self.assertRaises(ValueError):
load_routes("routes:\n\t- host: x\n")
parse_routes({"routes": [{
"host": "x.example",
"dlp": {"outbound_on_match": "nope"},
}]})
# --- load_config / parse_config ------------------------------------------
@@ -357,6 +336,33 @@ class TestLoadConfig(unittest.TestCase):
with self.assertRaises(ValueError):
parse_config("not a dict")
def test_empty_routes_list(self):
cfg = load_config("routes: []\n")
self.assertEqual((), cfg.routes)
def test_full_route_shape_parses(self):
cfg = load_config(
'routes:\n'
' - host: "api.example"\n'
' auth_scheme: "Bearer"\n'
' token_env: "EGRESS_TOKEN_0"\n'
' matches:\n'
' - paths:\n'
' - value: "/v1/"\n'
' - type: "exact"\n'
' value: "/messages"\n'
)
r = cfg.routes[0]
self.assertEqual("api.example", r.host)
self.assertEqual("Bearer", r.auth_scheme)
self.assertEqual("EGRESS_TOKEN_0", r.token_env)
self.assertEqual(1, len(r.matches))
self.assertEqual(2, len(r.matches[0].paths))
def test_invalid_yaml_raises_value_error(self):
with self.assertRaises(ValueError):
load_config("routes:\n\t- host: x\n")
# --- evaluate_matches ---------------------------------------------------
@@ -1167,5 +1173,195 @@ class TestScanInbound(unittest.TestCase):
self.assertEqual("block", result.severity)
class TestScanOutboundSafeTokens(unittest.TestCase):
"""PRD 0062: scan_outbound threads the supervisor-approved safe-tokens
set into the token detectors."""
def test_safe_token_allows_request(self):
text = build_outbound_scan_text(
host="api.example.com", path="/v1/data", query="",
headers={}, body=f"key={_AWS_KEY}",
)
self.assertIsNone(
scan_outbound(_ROUTE, text, {}, safe_tokens={_AWS_KEY})
)
def test_unrelated_safe_token_still_blocks(self):
text = build_outbound_scan_text(
host="api.example.com", path="/v1/data", query="",
headers={}, body=f"key={_AWS_KEY}",
)
result = scan_outbound(_ROUTE, text, {}, safe_tokens={"ghp_" + "A" * 36})
self.assertIsNotNone(result)
assert result is not None
self.assertEqual(_AWS_KEY, result.matched)
class TestScanOutboundCrlfText(unittest.TestCase):
"""PRD 0062: CRLF is scanned only over the request line + headers
(crlf_text), never the body a body is not an injection vector."""
def test_body_crlf_not_flagged_when_crlf_text_excludes_body(self):
# A form-encoded multi-line body legitimately contains %0d%0a.
body = "comment=line1%0d%0aline2"
full = build_outbound_scan_text(
host="api.example.com", path="/submit", query="",
headers={}, body=body,
)
crlf_text = build_outbound_scan_text(
host="api.example.com", path="/submit", query="",
headers={}, body="",
)
self.assertIsNone(scan_outbound(_ROUTE, full, {}, crlf_text=crlf_text))
def test_request_line_crlf_still_flagged(self):
full = build_outbound_scan_text(
host="api.example.com", path="/p", query="next=%0d%0aX:evil",
headers={}, body="",
)
crlf_text = full
result = scan_outbound(_ROUTE, full, {}, crlf_text=crlf_text)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
def test_default_crlf_text_scans_full_blob(self):
# Backward compatibility: crlf_text=None scans everything (body too).
full = build_outbound_scan_text(
host="api.example.com", path="/submit", query="",
headers={}, body="x=%0d%0aX:evil",
)
self.assertIsNotNone(scan_outbound(_ROUTE, full, {}))
class TestBuildTokenAllowPayload(unittest.TestCase):
def test_payload_includes_context_and_no_raw_token(self):
result = ScanResult(
severity="block",
reason="AWS access key found in body",
location="body",
context="key=******** tail",
matched=_AWS_KEY,
)
payload = build_token_allow_payload(
"api.example.com", "POST", "/v1/ingest", result,
)
self.assertIn("host: api.example.com", payload)
self.assertIn("method: POST", payload)
self.assertIn("path: /v1/ingest", payload)
self.assertIn("AWS access key found in body", payload)
self.assertIn("key=******** tail", payload)
# The raw matched value must never appear in the proposal file.
self.assertNotIn(_AWS_KEY, payload)
def test_payload_omits_context_line_when_empty(self):
result = ScanResult(severity="block", reason="r", matched="x")
payload = build_token_allow_payload("h", "GET", "/", result)
self.assertNotIn("context:", payload)
class TestScanOutboundEnhanced(unittest.TestCase):
"""scan_outbound changes: binary decode, entropy detector,
broadened known-value prefixes, fragmentation resistance."""
_ROUTE = Route(host="api.example.com")
_ROUTE_ENTROPY = Route(
host="api.example.com",
outbound_detectors=("entropy",),
)
def test_binary_body_latin1_decode_finds_ascii_secret(self):
# Body contains valid ASCII secret surrounded by non-UTF-8 bytes.
secret = "supersecrettoken99"
env = {"EGRESS_TOKEN_0": secret}
# Wrap the secret in bytes that are invalid UTF-8.
body = b"\x80\x81" + secret.encode("ascii") + b"\xff"
result = scan_outbound(self._ROUTE, body, env)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
def test_binary_body_valid_utf8_decoded_correctly(self):
env = {"EGRESS_TOKEN_0": "mysecret"}
# Valid UTF-8 body — should be decoded as UTF-8, not latin-1.
body = "clean body with mysecret".encode("utf-8")
result = scan_outbound(self._ROUTE, body, env)
self.assertIsNotNone(result)
def test_entropy_detector_off_by_default(self):
import string
# High-entropy content should NOT warn if the route has no entropy detector.
alphabet = (string.ascii_letters + string.digits + "+/")[:64]
result = scan_outbound(self._ROUTE, alphabet, {})
self.assertIsNone(result)
def test_entropy_detector_warns_when_enabled(self):
import string
alphabet = (string.ascii_letters + string.digits + "+/")[:64]
result = scan_outbound(self._ROUTE_ENTROPY, alphabet, {})
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("warn", result.severity)
def test_bot_bottle_sensitive_prefixes_env_var(self):
# When the sidecar env contains BOT_BOTTLE_SENSITIVE_PREFIXES,
# scan_outbound should scan those additional prefixes.
secret = "extra-sensitive-value-abc"
env = {
"MY_CRED_KEY": secret,
"BOT_BOTTLE_SENSITIVE_PREFIXES": "MY_CRED_",
}
result = scan_outbound(self._ROUTE, f"x={secret}", env)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
def test_bot_bottle_sensitive_prefixes_multiple(self):
secret = "my-api-key-value-xyz"
env = {
"ANTHROPIC_API_0": secret,
"BOT_BOTTLE_SENSITIVE_PREFIXES": "ANTHROPIC_API_,OTHER_",
}
result = scan_outbound(self._ROUTE, f"auth={secret}", env)
self.assertIsNotNone(result)
def test_canary_detected_via_random_secret_env_name(self):
# The fake secret uses a randomized env name that the sidecar marks
# as sensitive through BOT_BOTTLE_SENSITIVE_PREFIXES.
canary = "canaryvalue12345abcdef"
env = {
"CANON_ALPHA_SECRET": canary,
"BOT_BOTTLE_SENSITIVE_PREFIXES": "CANON_ALPHA_SECRET",
}
result = scan_outbound(self._ROUTE, f"data={canary}", env)
self.assertIsNotNone(result)
assert result is not None
self.assertEqual("block", result.severity)
self.assertIn("CANON_ALPHA_SECRET", result.reason)
def test_fragmented_canary_blocked(self):
# Canary with separators injected is still caught.
canary = "supersecretcanary99"
env = {
"CANON_ALPHA_SECRET": canary,
"BOT_BOTTLE_SENSITIVE_PREFIXES": "CANON_ALPHA_SECRET",
}
fragmented = "-".join(canary)
result = scan_outbound(self._ROUTE, f"x={fragmented}", env)
self.assertIsNotNone(result)
class TestOutboundDetectorNames(unittest.TestCase):
def test_entropy_in_outbound_detector_names(self):
from bot_bottle.egress_addon_core import OUTBOUND_DETECTOR_NAMES
self.assertIn("entropy", OUTBOUND_DETECTOR_NAMES)
def test_known_secrets_in_outbound_detector_names(self):
from bot_bottle.egress_addon_core import OUTBOUND_DETECTOR_NAMES
self.assertIn("known_secrets", OUTBOUND_DETECTOR_NAMES)
def test_token_patterns_in_outbound_detector_names(self):
from bot_bottle.egress_addon_core import OUTBOUND_DETECTOR_NAMES
self.assertIn("token_patterns", OUTBOUND_DETECTOR_NAMES)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,274 @@
"""Unit: LOG_FULL credential redaction in _log_request / _log_response (issue #257).
egress_addon.py is sidecar-only code that depends on mitmproxy, which is
not installed on the host. This file pre-populates sys.modules with the
minimum mocks needed so EgressAddon can be imported and tested without the
real mitmproxy package."""
from __future__ import annotations
import json
import sys
import types
import unittest
from io import StringIO
from typing import Any
from unittest.mock import patch
# ---------------------------------------------------------------------------
# Sidecar-import shims — must run before importing egress_addon
# ---------------------------------------------------------------------------
def _ensure_shims() -> None:
if "mitmproxy" not in sys.modules:
_mm = types.ModuleType("mitmproxy")
_mh = types.ModuleType("mitmproxy.http")
setattr(_mm, "http", _mh)
sys.modules["mitmproxy"] = _mm
sys.modules["mitmproxy.http"] = _mh
if "egress_addon_core" not in sys.modules:
import bot_bottle.egress_addon_core as _core
sys.modules["egress_addon_core"] = _core
_ensure_shims()
from bot_bottle.egress_addon import EgressAddon # noqa: E402 (import after shims)
from bot_bottle.egress_addon_core import Config, LOG_FULL # noqa: E402
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _addon() -> EgressAddon:
"""Return a bare EgressAddon with LOG_FULL config and no routes file."""
a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = Config(routes=(), log=LOG_FULL)
a.safe_tokens = set()
a._supervise_queue_dir = ""
a._supervise_slug = ""
a._token_allow_timeout = 300.0
return a
class _Headers:
def __init__(self, d: dict[str, str]) -> None:
self._d = d
def items(self) -> list[tuple[str, str]]:
return list(self._d.items())
class _Request:
def __init__(
self,
host: str = "api.example.com",
method: str = "POST",
path: str = "/v1/messages",
headers: dict[str, str] | None = None,
body: str = "",
) -> None:
self.pretty_host = host
self.method = method
self.path = path
self.headers = _Headers(headers or {})
self._body = body
def get_text(self, *, strict: bool = True) -> str:
return self._body
class _Response:
def __init__(
self,
status_code: int = 200,
headers: dict[str, str] | None = None,
body: str = "",
) -> None:
self.status_code = status_code
self.headers = _Headers(headers or {})
self._body = body
def get_text(self, *, strict: bool = True) -> str:
return self._body
class _Flow:
def __init__(
self,
request: _Request | None = None,
response: _Response | None = None,
) -> None:
self.request = request or _Request()
self.response = response or _Response()
def _log_request(addon: EgressAddon, flow: _Flow) -> dict[str, Any]:
buf = StringIO()
with patch("sys.stderr", buf):
addon._log_request(flow) # type: ignore[arg-type]
return json.loads(buf.getvalue())
def _log_response(addon: EgressAddon, flow: _Flow) -> dict[str, Any]:
buf = StringIO()
with patch("sys.stderr", buf):
addon._log_response(flow) # type: ignore[arg-type]
return json.loads(buf.getvalue())
# ---------------------------------------------------------------------------
# _log_request — authorization header stripped
# ---------------------------------------------------------------------------
class TestLogRequestAuthorizationStripped(unittest.TestCase):
def test_lowercase_authorization_excluded(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(headers={"authorization": "Bearer sk-real-secret"}))
entry = _log_request(addon, flow)
self.assertNotIn("authorization", entry["headers"])
def test_titlecase_authorization_excluded(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(headers={"Authorization": "Bearer sk-real-secret"}))
entry = _log_request(addon, flow)
self.assertNotIn("Authorization", entry["headers"])
self.assertNotIn("authorization", entry["headers"])
def test_non_auth_headers_retained(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(headers={
"authorization": "Bearer sk-real-secret",
"content-type": "application/json",
}))
entry = _log_request(addon, flow)
self.assertIn("content-type", entry["headers"])
self.assertEqual("application/json", entry["headers"]["content-type"])
def test_no_authorization_header_logs_all_others(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(headers={"x-request-id": "abc"}))
entry = _log_request(addon, flow)
self.assertEqual({"x-request-id": "abc"}, entry["headers"])
# ---------------------------------------------------------------------------
# _log_request — body redaction
# ---------------------------------------------------------------------------
_OPENAI_KEY = "sk-" + "A" * 48
class TestLogRequestBodyRedacted(unittest.TestCase):
def test_token_pattern_in_body_scrubbed(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(body=f"key={_OPENAI_KEY}"))
entry = _log_request(addon, flow)
self.assertNotIn(_OPENAI_KEY, entry["body"])
self.assertIn("********", entry["body"])
def test_provisioned_secret_in_body_scrubbed(self) -> None:
addon = _addon()
secret = "provisioned-egress-secret-xyz"
flow = _Flow(request=_Request(body=f"token={secret}"))
with patch.dict("os.environ", {"EGRESS_TOKEN_0": secret}):
entry = _log_request(addon, flow)
self.assertNotIn(secret, entry["body"])
self.assertIn("********", entry["body"])
def test_clean_body_preserved(self) -> None:
addon = _addon()
payload = '{"model": "claude-3", "max_tokens": 1024}'
flow = _Flow(request=_Request(body=payload))
entry = _log_request(addon, flow)
self.assertEqual(payload, entry["body"])
# ---------------------------------------------------------------------------
# _log_request — non-authorization header value redaction
# ---------------------------------------------------------------------------
class TestLogRequestHeaderValuesRedacted(unittest.TestCase):
def test_token_in_custom_header_scrubbed(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(headers={"x-api-key": _OPENAI_KEY}))
entry = _log_request(addon, flow)
self.assertNotIn(_OPENAI_KEY, entry["headers"].get("x-api-key", ""))
self.assertIn("********", entry["headers"].get("x-api-key", ""))
def test_clean_header_value_preserved(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(headers={"accept": "application/json"}))
entry = _log_request(addon, flow)
self.assertEqual("application/json", entry["headers"]["accept"])
# ---------------------------------------------------------------------------
# _log_response — body redaction
# ---------------------------------------------------------------------------
class TestLogResponseBodyRedacted(unittest.TestCase):
def test_token_pattern_in_response_body_scrubbed(self) -> None:
addon = _addon()
flow = _Flow(
request=_Request(),
response=_Response(body=f'{{"key": "{_OPENAI_KEY}"}}'),
)
entry = _log_response(addon, flow)
self.assertNotIn(_OPENAI_KEY, entry["body"])
self.assertIn("********", entry["body"])
def test_provisioned_secret_in_response_body_scrubbed(self) -> None:
addon = _addon()
secret = "provisioned-egress-secret-xyz"
flow = _Flow(
request=_Request(),
response=_Response(body=f'{{"token": "{secret}"}}'),
)
with patch.dict("os.environ", {"EGRESS_TOKEN_0": secret}):
entry = _log_response(addon, flow)
self.assertNotIn(secret, entry["body"])
self.assertIn("********", entry["body"])
def test_clean_response_body_preserved(self) -> None:
addon = _addon()
flow = _Flow(request=_Request(), response=_Response(body='{"result": "ok"}'))
entry = _log_response(addon, flow)
self.assertEqual('{"result": "ok"}', entry["body"])
# ---------------------------------------------------------------------------
# _log_response — response header value redaction
# ---------------------------------------------------------------------------
class TestLogResponseHeaderValuesRedacted(unittest.TestCase):
def test_token_in_response_header_scrubbed(self) -> None:
addon = _addon()
flow = _Flow(
request=_Request(),
response=_Response(headers={"set-cookie": f"token={_OPENAI_KEY}"}),
)
entry = _log_response(addon, flow)
cookie_val = entry["headers"].get("set-cookie", "")
self.assertNotIn(_OPENAI_KEY, cookie_val)
self.assertIn("********", cookie_val)
def test_clean_response_header_preserved(self) -> None:
addon = _addon()
flow = _Flow(
request=_Request(),
response=_Response(headers={"content-type": "application/json"}),
)
entry = _log_response(addon, flow)
self.assertEqual("application/json", entry["headers"]["content-type"])
if __name__ == "__main__":
unittest.main()
+9
View File
@@ -54,6 +54,15 @@ class TestValidateRoutesContent(unittest.TestCase):
' auth_scheme: "Bearer"\n'
)
def test_rejects_log_full(self):
with self.assertRaises(EgressApplyError) as cm:
applicator.validate_routes_content(
'log: 2\n'
'routes:\n'
' - host: "x.example"\n'
)
self.assertIn("must not change egress logging", str(cm.exception))
class TestApplyRoutesChange(unittest.TestCase):
def setUp(self):
+65
View File
@@ -4,6 +4,7 @@ import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle.git_gate import (
GitGate,
@@ -13,6 +14,8 @@ from bot_bottle.git_gate import (
git_gate_render_access_hook,
git_gate_render_entrypoint,
git_gate_render_hook,
revoke_git_gate_provisioned_keys,
_resolve_identity_file,
git_gate_upstreams_for_bottle,
)
from bot_bottle.manifest import ManifestIndex
@@ -328,6 +331,68 @@ class TestPrepare(unittest.TestCase):
self.assertIn("exec git daemon", content)
class TestDynamicKeyProvisioning(unittest.TestCase):
def setUp(self):
self.stage = Path(tempfile.mkdtemp())
def tearDown(self):
import shutil
shutil.rmtree(self.stage, ignore_errors=True)
def _gitea_manifest(self):
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"git-gate": {
"repos": {
"repo": {
"url": "ssh://git@gitea.example.com/org/repo.git",
"key": {
"provider": "gitea",
"forge_token_env": "GITEA_TOKEN",
},
"host_key": "ssh-ed25519 AAAA...",
},
},
}
}
},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def test_resolve_identity_file_static_uses_entry_path(self):
entry = fixture_with_git().bottles["dev"].git[0]
self.assertEqual(entry.IdentityFile, _resolve_identity_file(entry, "demo", self.stage))
def test_resolve_identity_file_gitea_provisions_key(self):
entry = self._gitea_manifest().bottles["dev"].git[0]
with patch("bot_bottle.git_gate._provision_dynamic_key", return_value="/tmp/provisioned-key") as mock_provision:
self.assertEqual("/tmp/provisioned-key", _resolve_identity_file(entry, "demo", self.stage))
mock_provision.assert_called_once()
def test_revoke_skips_non_gitea_and_missing_id_file(self):
revoke_git_gate_provisioned_keys(fixture_with_git().bottles["dev"], self.stage)
def test_revoke_calls_delete_for_gitea_entry(self):
bottle = self._gitea_manifest().bottles["dev"]
(self.stage / "repo-deploy-key-id").write_text("123\n")
with patch.dict("os.environ", {"GITEA_TOKEN": "token"}), patch(
"bot_bottle.deploy_key_provisioner.get_provisioner"
) as mock_get_provisioner:
provisioner = mock_get_provisioner.return_value
revoke_git_gate_provisioned_keys(bottle, self.stage)
mock_get_provisioner.assert_called_once()
provisioner.delete.assert_called_once_with("org/repo", "123")
def test_revoke_missing_token_raises(self):
bottle = self._gitea_manifest().bottles["dev"]
(self.stage / "repo-deploy-key-id").write_text("123\n")
with patch.dict("os.environ", {}, clear=True), self.assertRaises(RuntimeError) as cm:
revoke_git_gate_provisioned_keys(bottle, self.stage)
self.assertIn("env var is not set", str(cm.exception))
class TestShellEscaping(unittest.TestCase):
"""Regression tests: all three render functions must produce syntactically
valid sh code even when names and upstream URLs contain shell-special
+107
View File
@@ -9,6 +9,7 @@ import urllib.request
from pathlib import Path
from unittest import mock
from bot_bottle.git_gate import GIT_GATE_TIMEOUT_SECS
from bot_bottle.git_http_backend import GitHttpHandler, MAX_BODY_BYTES
@@ -150,6 +151,61 @@ class TestGitHttpBackend(unittest.TestCase):
)
self.assertEqual("git/test", env["HTTP_USER_AGENT"])
def test_subprocess_calls_include_timeout(self):
"""Both subprocess.run calls (access-hook and git http-backend) must
pass timeout= so a hung upstream cannot wedge the sidecar."""
from http.server import ThreadingHTTPServer
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
(root / "repo.git").mkdir()
old_root = os.environ.get("GIT_PROJECT_ROOT")
os.environ["GIT_PROJECT_ROOT"] = str(root)
self.addCleanup(self._restore_env, old_root)
old_hook = os.environ.get("GIT_GATE_ACCESS_HOOK")
hook = root / "access-hook"
hook.write_text("#!/bin/sh\nexit 0\n")
hook.chmod(0o700)
os.environ["GIT_GATE_ACCESS_HOOK"] = str(hook)
self.addCleanup(self._restore_hook, old_hook)
server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
backend_response = (
b"Status: 200 OK\r\n"
b"Content-Type: application/x-git-upload-pack-result\r\n"
b"\r\n"
b"0000"
)
calls = [
subprocess.CompletedProcess(["hook"], 0, b"", b""),
subprocess.CompletedProcess(["git"], 0, backend_response, b""),
]
with mock.patch(
"bot_bottle.git_http_backend.subprocess.run",
side_effect=calls,
) as run:
req = urllib.request.Request(
f"http://127.0.0.1:{server.server_port}"
"/repo.git/git-upload-pack",
data=b"",
method="POST",
)
with urllib.request.urlopen(req, timeout=5):
pass
for call in run.call_args_list:
self.assertEqual(
GIT_GATE_TIMEOUT_SECS,
call.kwargs.get("timeout"),
f"subprocess.run call missing timeout: {call}",
)
def test_access_hook_denial_is_logged_to_stdout(self):
"""When the access-hook exits non-zero we still return 403 to the
client, but the hook's stderr must also appear on the handler's
@@ -256,6 +312,57 @@ class TestGitHttpBackend(unittest.TestCase):
os.environ["GIT_GATE_ACCESS_HOOK"] = value
class TestMalformedStatusHeader(unittest.TestCase):
"""Malformed CGI Status: headers must not propagate as unhandled exceptions;
the handler should fall back to HTTP 500."""
def setUp(self):
from http.server import ThreadingHTTPServer
import tempfile
self._tmp = tempfile.mkdtemp()
os.environ["GIT_PROJECT_ROOT"] = self._tmp
self._server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
self._thread = threading.Thread(
target=self._server.serve_forever, daemon=True,
)
self._thread.start()
self._port = self._server.server_port
def tearDown(self):
self._server.shutdown()
self._server.server_close()
os.environ.pop("GIT_PROJECT_ROOT", None)
import shutil
shutil.rmtree(self._tmp, ignore_errors=True)
def _get_with_backend_response(self, cgi_response: bytes) -> int:
with mock.patch(
"bot_bottle.git_http_backend.subprocess.run",
return_value=mock.Mock(returncode=0, stdout=cgi_response),
):
req = urllib.request.Request(
f"http://127.0.0.1:{self._port}/repo.git/info/refs",
method="GET",
)
try:
with urllib.request.urlopen(req, timeout=3) as resp:
return resp.status
except urllib.error.HTTPError as e: # type: ignore
return e.code
def test_empty_status_value_returns_500(self):
status = self._get_with_backend_response(
b"Status: \r\nContent-Type: text/plain\r\n\r\n"
)
self.assertEqual(500, status)
def test_non_numeric_status_returns_500(self):
status = self._get_with_backend_response(
b"Status: bad\r\nContent-Type: text/plain\r\n\r\n"
)
self.assertEqual(500, status)
class TestContentLengthBounds(unittest.TestCase):
"""PRD 0041: malformed or oversized Content-Length is rejected before
git http-backend is invoked."""
-34
View File
@@ -1,34 +0,0 @@
"""Unit: install.sh static contract checks."""
from __future__ import annotations
import subprocess
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
class TestInstallScript(unittest.TestCase):
def test_shell_syntax(self):
result = subprocess.run(
["sh", "-n", str(ROOT / "install.sh")],
check=False,
capture_output=True,
text=True,
)
self.assertEqual("", result.stderr)
self.assertEqual(0, result.returncode)
def test_contract_phrases(self):
script = (ROOT / "install.sh").read_text(encoding="utf-8")
self.assertIn("python3", script)
self.assertIn("docker info", script)
self.assertIn("pipx install --force", script)
self.assertIn("pip install --user --upgrade", script)
self.assertIn('"${BOT_BOTTLE_BIN}" doctor', script)
if __name__ == "__main__":
unittest.main()
+127
View File
@@ -0,0 +1,127 @@
"""Unit: leveled + structured logging wrappers (issue #252).
Locks three properties of bot_bottle.log:
- backward compatibility default output is byte-identical to the
original bare wrappers, so the 100+ existing single-string call
sites are unaffected;
- context rendering an optional mapping becomes a parseable
` [k=v ...]` suffix;
- level gating BOT_BOTTLE_LOG_LEVEL filters by severity, debug is
silent by default, and error always surfaces.
"""
from __future__ import annotations
import contextlib
import io
import unittest
from typing import Callable
from unittest import mock
from bot_bottle import log
def _capture(
fn: Callable[..., None],
*args: object,
env: dict[str, str] | None = None,
**kwargs: object,
) -> str:
buf = io.StringIO()
patched = mock.patch.dict("os.environ", env or {}, clear=False)
with patched, contextlib.redirect_stderr(buf):
fn(*args, **kwargs)
return buf.getvalue()
class TestBackwardCompat(unittest.TestCase):
"""No context + default level → exactly the legacy lines."""
def test_info(self):
self.assertEqual("bot-bottle: hello\n", _capture(log.info, "hello"))
def test_warn(self):
self.assertEqual(
"bot-bottle: warning: careful\n", _capture(log.warn, "careful")
)
def test_error(self):
self.assertEqual(
"bot-bottle: error: boom\n", _capture(log.error, "boom")
)
class TestContext(unittest.TestCase):
def test_appends_sorted_parseable_suffix(self):
out = _capture(
log.error, "rpc failed", context={"slug": "abc123", "code": "-32603"}
)
# keys sorted: code before slug
self.assertEqual(
"bot-bottle: error: rpc failed [code=-32603 slug=abc123]\n", out
)
def test_quotes_values_with_whitespace(self):
out = _capture(
log.info, "did thing", context={"path": "/a b/c", "ok": "yes"}
)
self.assertEqual(
'bot-bottle: did thing [ok=yes path="/a b/c"]\n', out
)
def test_empty_context_is_noop_suffix(self):
self.assertEqual(
"bot-bottle: x\n", _capture(log.info, "x", context={})
)
class TestLevels(unittest.TestCase):
def test_debug_silent_by_default(self):
self.assertEqual("", _capture(log.debug, "trace"))
def test_debug_emits_when_level_lowered(self):
out = _capture(log.debug, "trace", env={"BOT_BOTTLE_LOG_LEVEL": "debug"})
self.assertEqual("bot-bottle: debug: trace\n", out)
def test_error_level_suppresses_info_and_warn(self):
env = {"BOT_BOTTLE_LOG_LEVEL": "error"}
self.assertEqual("", _capture(log.info, "i", env=env))
self.assertEqual("", _capture(log.warn, "w", env=env))
# error still surfaces — nothing sits above it
self.assertEqual(
"bot-bottle: error: e\n", _capture(log.error, "e", env=env)
)
def test_unknown_level_falls_back_to_default(self):
# garbage value → default INFO threshold, so info still prints
out = _capture(log.info, "i", env={"BOT_BOTTLE_LOG_LEVEL": "loud"})
self.assertEqual("bot-bottle: i\n", out)
def test_warning_alias_accepted(self):
env = {"BOT_BOTTLE_LOG_LEVEL": "warning"}
self.assertEqual("", _capture(log.info, "i", env=env))
self.assertEqual(
"bot-bottle: warning: w\n", _capture(log.warn, "w", env=env)
)
class TestDie(unittest.TestCase):
def test_die_still_raises_and_prints_error(self):
buf = io.StringIO()
with contextlib.redirect_stderr(buf):
with self.assertRaises(log.Die) as cm:
log.die("fatal thing")
self.assertEqual("fatal thing", cm.exception.message)
self.assertIn("bot-bottle: error: fatal thing", buf.getvalue())
def test_die_surfaces_even_at_error_level(self):
buf = io.StringIO()
with mock.patch.dict("os.environ", {"BOT_BOTTLE_LOG_LEVEL": "error"}):
with contextlib.redirect_stderr(buf):
with self.assertRaises(log.Die):
log.die("still fatal")
self.assertIn("bot-bottle: error: still fatal", buf.getvalue())
if __name__ == "__main__":
unittest.main()
+24 -1
View File
@@ -30,6 +30,7 @@ def _plan(
supervise: bool = False,
agent_git_gate_url: str = "",
agent_supervise_url: str = "",
canary: bool = False,
) -> MacosContainerBottlePlan:
routes_path = stage_dir / "routes.yaml"
routes_path.write_text("routes: []\n", encoding="utf-8")
@@ -42,6 +43,8 @@ def _plan(
routes_path=routes_path,
routes=("route",),
token_env_map={"EGRESS_TOKEN_0": "HOST_TOKEN"},
canary="fake-canary-value" if canary else "",
canary_env="CANON_ALPHA_SECRET" if canary else "",
)
if git:
key_path = stage_dir / "origin-key"
@@ -138,6 +141,26 @@ class TestMacosContainerLaunchArgv(unittest.TestCase):
argv,
)
def test_sidecar_argv_registers_canary_env_as_sensitive(self):
plan = _plan(stage_dir=self.stage_dir, canary=True)
argv = launch._sidecar_run_argv(
plan,
"bot-bottle-sidecars-dev-abc",
"bot-bottle-net-dev-abc",
"bot-bottle-egress-dev-abc",
)
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", argv)
self.assertIn("BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET", argv)
def test_agent_argv_receives_canary_env(self):
plan = _plan(stage_dir=self.stage_dir, canary=True)
argv = launch._agent_run_argv(
plan,
"bot-bottle-net-dev-abc",
"192.0.2.10",
)
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", argv)
def test_agent_env_points_proxy_at_sidecar_ip(self):
plan = _plan(
stage_dir=self.stage_dir,
@@ -271,7 +294,7 @@ def _build_plan(stage_dir: Path) -> MacosContainerBottlePlan:
manifest=_MANIFEST,
stage_dir=stage_dir,
git_gate_plan=cast(GitGatePlan, SimpleNamespace(upstreams=())),
egress_plan=cast(EgressPlan, SimpleNamespace()),
egress_plan=cast(EgressPlan, SimpleNamespace(canary="")),
supervise_plan=None,
agent_provision=AgentProvisionPlan(
template="claude",
+27
View File
@@ -73,6 +73,33 @@ resolver #2
)
self.assertTrue(run.call_args_list[-1].kwargs["check"])
def test_build_image_anchors_relative_dockerfile_to_context(self):
status = util.subprocess.CompletedProcess(
args=[],
returncode=0,
stdout=(
'[{"status":{"state":"running"},'
'"configuration":{"dns":{"nameservers":["9.9.9.9"]}}}]'
),
stderr="",
)
with patch.object(util.subprocess, "run", return_value=status) as run, \
patch.object(util.os, "environ", {
"BOT_BOTTLE_MACOS_CONTAINER_DNS": "9.9.9.9",
}):
util.build_image(
"bot-bottle-sidecars:latest",
"/repo",
dockerfile="Dockerfile.sidecars",
)
self.assertEqual(
[
"container", "build", "-t", "bot-bottle-sidecars:latest",
"--dns", "9.9.9.9", "-f", "/repo/Dockerfile.sidecars", "/repo",
],
run.call_args_list[-1].args[0],
)
def test_commit_container_execs_tar_and_builds_image(self):
# stderr is bytes because subprocess.run uses stderr=PIPE without text=True
completed = util.subprocess.CompletedProcess(
+46 -1
View File
@@ -167,13 +167,40 @@ class TestAgentProviderHostCredentials(unittest.TestCase):
},
})
def test_settings_rejected_for_claude(self):
def test_startup_args_allowed_for_claude(self):
b = _provider_config_bottle({
"template": "claude",
"settings": {"startup_args": ["--model", "opus"]},
})
self.assertEqual(
{"startup_args": ["--model", "opus"]},
b.agent_provider.settings,
)
def test_startup_args_allowed_for_codex(self):
b = _provider_config_bottle({
"template": "codex",
"settings": {"startup_args": ["--model", "gpt-5-codex"]},
})
self.assertEqual(
{"startup_args": ["--model", "gpt-5-codex"]},
b.agent_provider.settings,
)
def test_provider_specific_settings_still_rejected_for_claude(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
"template": "claude",
"settings": {"models": ["qwen2.5-coder:7b"]},
})
def test_startup_args_must_be_string_array(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
"template": "codex",
"settings": {"startup_args": ["--model", 42]},
})
def test_settings_models_must_be_non_empty_string_array(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
@@ -302,6 +329,24 @@ class TestDlp(unittest.TestCase):
"bogus": True,
}}])
def test_outbound_on_match_omitted_is_empty(self):
b = _bottle([{"host": "x.example"}])
self.assertEqual("", b.egress.routes[0].OutboundOnMatch)
def test_outbound_on_match_accepts_policies(self):
for policy in ("block", "redact", "supervise"):
with self.subTest(policy=policy):
b = _bottle([{"host": "x.example", "dlp": {
"outbound_on_match": policy,
}}])
self.assertEqual(policy, b.egress.routes[0].OutboundOnMatch)
def test_outbound_on_match_rejects_unknown_value(self):
with self.assertRaises(ManifestError):
_bottle([{"host": "x.example", "dlp": {
"outbound_on_match": "allow",
}}])
class TestGitPolicy(unittest.TestCase):
def test_omitted_means_https_git_fetch_disabled(self):
+176 -3
View File
@@ -423,9 +423,182 @@ class TestExtendsErrors(unittest.TestCase):
)
self.assertIn("extends cycle", msg)
def test_non_string_extends_dies(self):
msg = _error_message(_build, child={"extends": ["base"]})
self.assertIn("extends must be a string", msg)
def test_non_string_non_list_extends_dies(self):
msg = _error_message(_build, child={"extends": 123})
self.assertIn("extends must be a string or list of strings", msg)
def test_list_entry_non_string_dies(self):
msg = _error_message(_build, child={"extends": [123]})
self.assertIn("extends[0] must be a string", msg)
class TestExtendsMultiParent(unittest.TestCase):
"""extends: [p1, p2, ...] — multi-parent composition (issue #268)."""
_GIT_A = {"url": "ssh://git@host-a/a.git", "key": {"provider": "static", "path": "/k"}}
_GIT_B = {"url": "ssh://git@host-b/b.git", "key": {"provider": "static", "path": "/k"}}
def test_single_element_list_same_as_string(self):
m = _build(
base={"env": {"X": "1"}},
child={"extends": ["base"]},
)
self.assertEqual({"X": "1"}, dict(m.bottles["child"].env))
def test_two_parents_env_union(self):
m = _build(
p1={"env": {"A": "1"}},
p2={"env": {"B": "2"}},
child={"extends": ["p1", "p2"]},
)
self.assertEqual({"A": "1", "B": "2"}, dict(m.bottles["child"].env))
def test_two_parents_env_last_wins_on_collision(self):
m = _build(
p1={"env": {"X": "from-p1"}},
p2={"env": {"X": "from-p2"}},
child={"extends": ["p1", "p2"]},
)
self.assertEqual("from-p2", m.bottles["child"].env["X"])
def test_child_wins_over_all_parents(self):
m = _build(
p1={"env": {"X": "from-p1"}},
p2={"env": {"X": "from-p2"}},
child={"extends": ["p1", "p2"], "env": {"X": "from-child"}},
)
self.assertEqual("from-child", m.bottles["child"].env["X"])
def test_two_parents_supervise_last_wins(self):
m = _build(
p1={"supervise": False},
p2={"supervise": True},
child={"extends": ["p1", "p2"]},
)
self.assertTrue(m.bottles["child"].supervise)
def test_child_supervise_overrides_all_parents(self):
m = _build(
p1={"supervise": True},
p2={"supervise": True},
child={"extends": ["p1", "p2"], "supervise": False},
)
self.assertFalse(m.bottles["child"].supervise)
def test_two_parents_egress_routes_concatenated(self):
m = _build(
p1={"egress": {"routes": [{"host": "a.example.com"}]}},
p2={"egress": {"routes": [{"host": "b.example.com"}]}},
child={"extends": ["p1", "p2"]},
)
hosts = [r.Host for r in m.bottles["child"].egress.routes]
self.assertEqual(["a.example.com", "b.example.com"], hosts)
def test_child_egress_appends_after_combined_parents(self):
m = _build(
p1={"egress": {"routes": [{"host": "a.example.com"}]}},
p2={"egress": {"routes": [{"host": "b.example.com"}]}},
child={
"extends": ["p1", "p2"],
"egress": {"routes": [{"host": "c.example.com"}]},
},
)
hosts = [r.Host for r in m.bottles["child"].egress.routes]
self.assertEqual(["a.example.com", "b.example.com", "c.example.com"], hosts)
def test_two_parents_git_repos_union(self):
m = _build(
p1={"git-gate": {"repos": {"a": self._GIT_A}}},
p2={"git-gate": {"repos": {"b": self._GIT_B}}},
child={"extends": ["p1", "p2"]},
)
names = {e.Name for e in m.bottles["child"].git}
self.assertEqual({"a", "b"}, names)
def test_two_parents_git_same_name_later_wins_per_field(self):
# Both parents declare the same repo name. p2's `key` wins; p1's
# `host_key` is preserved because p2 doesn't override it.
p1_entry = {
"url": "ssh://git@host-a/repo.git",
"host_key": "ecdsa AAAA",
"key": {"provider": "static", "path": "/k1"},
}
p2_entry = {
"url": "ssh://git@host-a/repo.git", # required, same url
"key": {"provider": "gitea", "forge_token_env": "TOK"},
}
m = _build(
p1={"git-gate": {"repos": {"repo": p1_entry}}},
p2={"git-gate": {"repos": {"repo": p2_entry}}},
child={"extends": ["p1", "p2"]},
)
entries = m.bottles["child"].git
self.assertEqual(1, len(entries))
e = entries[0]
self.assertEqual("ssh://git@host-a/repo.git", e.Upstream)
self.assertEqual("ecdsa AAAA", e.KnownHostKey)
self.assertEqual("gitea", e.Key.provider)
def test_p1_repos_preserved_when_p2_has_none(self):
m = _build(
p1={"git-gate": {"repos": {"a": self._GIT_A}}},
p2={"env": {"X": "1"}},
child={"extends": ["p1", "p2"]},
)
names = [e.Name for e in m.bottles["child"].git]
self.assertEqual(["a"], names)
def test_diamond_shared_ancestor_resolved_once(self):
# a <- b, a <- c; child extends [b, c]
# `a` must be resolved once and cached.
m = _build(
a={"env": {"FROM_A": "1"}, "supervise": False},
b={"extends": "a", "env": {"FROM_B": "1"}},
c={"extends": "a", "env": {"FROM_C": "1"}},
child={"extends": ["b", "c"]},
)
child = m.bottles["child"]
self.assertEqual("1", child.env["FROM_A"])
self.assertEqual("1", child.env["FROM_B"])
self.assertEqual("1", child.env["FROM_C"])
# supervise=False from `a` threads through both b and c; c is the
# later parent so its effective supervise (False) wins.
self.assertFalse(child.supervise)
def test_three_parents_env_fold_order(self):
m = _build(
p1={"env": {"X": "1", "A": "a"}},
p2={"env": {"X": "2", "B": "b"}},
p3={"env": {"X": "3", "C": "c"}},
child={"extends": ["p1", "p2", "p3"]},
)
env = dict(m.bottles["child"].env)
self.assertEqual("3", env["X"])
self.assertEqual("a", env["A"])
self.assertEqual("b", env["B"])
self.assertEqual("c", env["C"])
def test_undefined_bottle_in_list_dies(self):
msg = _error_message(
_build,
base={"env": {}},
child={"extends": ["base", "ghost"]},
)
self.assertIn("extends 'ghost'", msg)
self.assertIn("not defined", msg)
def test_self_reference_in_list_dies(self):
msg = _error_message(_build, child={"extends": ["child"]})
self.assertIn("extends itself", msg)
def test_cycle_through_multi_parent_edge_dies(self):
msg = _error_message(
_build,
a={"extends": ["b", "c"]},
b={},
c={"extends": "a"},
)
self.assertIn("extends cycle", msg)
class TestExtendsAvailableInBottleKeys(unittest.TestCase):
+1 -1
View File
@@ -130,7 +130,7 @@ def _capture_print(plan: DockerBottlePlan | SmolmachinesBottlePlan) -> list[str]
orig = sys.stderr
sys.stderr = buf
try:
plan.print(remote_control=False)
plan.print()
finally:
sys.stderr = orig
return buf.getvalue().splitlines()
+38
View File
@@ -8,6 +8,7 @@ import unittest
from bot_bottle.git_gate import (
GIT_GATE_HOSTNAME,
_gitconfig_validate_value,
git_gate_render_gitconfig,
)
from bot_bottle.manifest import ManifestIndex
@@ -90,5 +91,42 @@ class TestGitGateGitconfigRender(unittest.TestCase):
self.assertNotIn("gitea.dideric.is", out)
class TestGitconfigValidateValue(unittest.TestCase):
"""_gitconfig_validate_value rejects values that would inject gitconfig keys."""
def test_normal_url_passes(self):
_gitconfig_validate_value("url", "ssh://git@github.com/owner/repo.git")
def test_newline_in_url_raises(self):
with self.assertRaises(ValueError):
_gitconfig_validate_value("url", "ssh://git@github.com/owner/\nrepo.git")
def test_carriage_return_in_url_raises(self):
with self.assertRaises(ValueError):
_gitconfig_validate_value("url", "ssh://git@github.com/\rrepo.git")
def test_error_message_names_field(self):
with self.assertRaises(ValueError, msg="error should name the field") as ctx:
_gitconfig_validate_value("repos['bad'].url", "ssh://host/\npath")
self.assertIn("repos['bad'].url", str(ctx.exception))
class TestGitconfigRenderRejectsNewlineInUpstream(unittest.TestCase):
"""git_gate_render_gitconfig raises on Upstream values with newlines."""
def test_newline_in_upstream_raises(self):
m = ManifestIndex.from_json_obj({
"bottles": {"dev": {"git-gate": {"repos": {
"evil": {
"url": "ssh://git@github.com/owner/\nfake-key = injected\nrepo.git",
"key": {"provider": "static", "path": "/dev/null"},
},
}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
with self.assertRaises(ValueError):
git_gate_render_gitconfig(m.bottles["dev"].git, GIT_GATE_HOSTNAME)
if __name__ == "__main__":
unittest.main()
-27
View File
@@ -1,27 +0,0 @@
"""Unit: Python package metadata for install script PRD."""
from __future__ import annotations
import tomllib
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
class TestPyproject(unittest.TestCase):
def test_console_script_and_no_runtime_dependencies(self):
data = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
project = data["project"]
self.assertEqual("bot-bottle", project["name"])
self.assertEqual(">=3.11", project["requires-python"])
self.assertEqual([], project["dependencies"])
self.assertEqual(
"bot_bottle.cli:main",
project["scripts"]["bot-bottle"],
)
if __name__ == "__main__":
unittest.main()
+29 -5
View File
@@ -26,9 +26,7 @@ from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
from bot_bottle.backend.smolmachines.bottle_plan import (
SmolmachinesBottlePlan,
)
# from bot_bottle.backend.smolmachines.provision import (
# workspace as _workspace,
# )
from bot_bottle.backend.smolmachines import launch as _launch
from bot_bottle.backend.smolmachines.launch import _bundle_launch_spec
from bot_bottle.backend.util import AGENT_CA_PATH
from bot_bottle.egress import EgressPlan, EgressRoute
@@ -44,7 +42,6 @@ class _Provider(AgentProvider):
return AgentProviderRuntime(
template="test", command="test", image="",
prompt_mode="append_file", bypass_args=(), resume_args=(),
remote_control_args=(),
)
def provision_plan(self, **kwargs): # type: ignore[override]
raise NotImplementedError
@@ -86,6 +83,7 @@ def _plan(
stage_dir: Path | None = None,
egress_routes: tuple[EgressRoute, ...] = (),
egress_ca_path: Path = Path(),
canary: bool = False,
supervise: bool = False,
bundle_ip: str = "192.168.50.2",
agent_git_gate_host: str = "127.0.0.1:55555",
@@ -132,7 +130,6 @@ def _plan(
supervise_plan = SupervisePlan(
slug="demo-abc12",
queue_dir=Path("/tmp/queue"),
current_config_dir=Path("/tmp/current-config"),
)
return SmolmachinesBottlePlan(
spec=spec,
@@ -156,6 +153,8 @@ def _plan(
routes=egress_routes,
token_env_map={},
mitmproxy_ca_cert_only_host_path=egress_ca_path,
canary="fake-canary-value" if canary else "",
canary_env="CANON_ALPHA_SECRET" if canary else "",
),
supervise_plan=supervise_plan,
agent_git_gate_host=agent_git_gate_host,
@@ -411,6 +410,31 @@ class TestBundleLaunchSpec(unittest.TestCase):
self.assertIn(9420, spec.ports_to_publish)
self.assertNotIn(9418, spec.ports_to_publish)
def test_canary_env_registered_as_sensitive_in_bundle(self):
plan = _plan(canary=True)
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", spec.environment)
self.assertIn(
"BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET",
spec.environment,
)
def test_canary_env_visible_to_smolvm_guest(self):
plan = _plan(canary=True)
with patch.object(
_launch._bundle,
"bundle_host_port",
return_value="65000",
):
stamped = _launch._discover_urls(plan, "127.0.0.16")
self.assertEqual(
"fake-canary-value",
stamped.guest_env["CANON_ALPHA_SECRET"],
)
class TestProvisionGitUser(unittest.TestCase):
"""`provision_git` runs `git config --global` inside the
+26 -20
View File
@@ -16,7 +16,7 @@ from bot_bottle.supervise import (
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOL_CAPABILITY_BLOCK,
TOOL_EGRESS_ALLOW,
TOOL_GITLEAKS_ALLOW,
archive_proposal,
audit_log_path,
@@ -37,9 +37,9 @@ FIXED_TS = datetime(2026, 5, 25, 12, 0, 0, tzinfo=timezone.utc)
def _proposal(
tool: str = TOOL_CAPABILITY_BLOCK,
proposed: str = "FROM python:3.13\n",
justification: str = "need a capability",
tool: str = TOOL_EGRESS_ALLOW,
proposed: str = "routes:\n - host: example.com\n",
justification: str = "need egress",
) -> Proposal:
return Proposal.new(
bottle_slug="dev",
@@ -57,7 +57,7 @@ class TestProposalRoundtrip(unittest.TestCase):
self.assertTrue(p.id)
self.assertEqual("2026-05-25T12:00:00+00:00", p.arrival_timestamp)
self.assertEqual("dev", p.bottle_slug)
self.assertEqual(TOOL_CAPABILITY_BLOCK, p.tool)
self.assertEqual(TOOL_EGRESS_ALLOW, p.tool)
def test_to_from_dict_roundtrip(self):
p = _proposal()
@@ -142,14 +142,14 @@ class TestQueueIO(unittest.TestCase):
def test_list_pending_sorted_by_arrival(self):
# Fabricate two with explicit timestamps.
a = Proposal.new(
bottle_slug="dev", tool=TOOL_CAPABILITY_BLOCK,
proposed_file="FROM python:3.13\n", justification="early",
bottle_slug="dev", tool=TOOL_EGRESS_ALLOW,
proposed_file="routes:\n - host: early.example.com\n", justification="early",
current_file_hash="x",
now=datetime(2026, 5, 25, 10, 0, 0, tzinfo=timezone.utc),
)
b = Proposal.new(
bottle_slug="dev", tool=TOOL_CAPABILITY_BLOCK,
proposed_file="FROM python:3.13\n", justification="late",
bottle_slug="dev", tool=TOOL_EGRESS_ALLOW,
proposed_file="routes:\n - host: late.example.com\n", justification="late",
current_file_hash="x",
now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc),
)
@@ -318,19 +318,29 @@ class TestToolConstants(unittest.TestCase):
def test_tools_tuple_matches_individual_constants(self):
self.assertEqual(
(
supervise.TOOL_ALLOW,
TOOL_CAPABILITY_BLOCK,
supervise.TOOL_EGRESS_ALLOW,
supervise.TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
supervise.TOOL_EGRESS_TOKEN_ALLOW,
supervise.TOOL_LIST_EGRESS_ROUTES,
),
supervise.TOOLS,
)
def test_token_allow_proposal_roundtrips(self):
p = Proposal.new(
bottle_slug="dev",
tool=supervise.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file="host: api.example.com\n",
justification="false positive",
current_file_hash="h",
)
self.assertEqual(p, Proposal.from_dict(p.to_dict()))
def test_component_map_has_egress_entries(self):
self.assertEqual(
{
supervise.TOOL_ALLOW: "egress",
supervise.TOOL_EGRESS_ALLOW: "egress",
supervise.TOOL_EGRESS_BLOCK: "egress",
},
supervise.COMPONENT_FOR_TOOL,
@@ -367,20 +377,16 @@ class TestSupervisePrepare(unittest.TestCase):
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
return lambda: setattr(supervise, "bot_bottle_root", original)
def test_prepare_creates_queue_and_current_config(self):
def test_prepare_creates_queue(self):
plan = _StubSupervise().prepare("dev", self.stage_dir)
self.assertTrue(plan.queue_dir.is_dir())
self.assertTrue(plan.current_config_dir.is_dir())
self.assertEqual("dev", plan.slug)
self.assertEqual("", plan.internal_network)
def test_prepare_writes_no_files_to_current_config(self):
# dockerfile_content is no longer accepted by prepare.
# routes.yaml + allowlist live behind the
# `list-egress-routes` MCP tool (PRD 0017 chunk 3).
def test_prepare_does_not_create_current_config_dir(self):
plan = _StubSupervise().prepare("dev", self.stage_dir)
files = sorted(p.name for p in plan.current_config_dir.iterdir())
self.assertEqual([], files)
self.assertFalse((self.stage_dir / "current-config").exists())
self.assertFalse(hasattr(plan, "current_config_dir"))
if __name__ == "__main__":
+58 -29
View File
@@ -18,8 +18,9 @@ from bot_bottle.supervise import (
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOL_CAPABILITY_BLOCK,
TOOL_EGRESS_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
read_audit_entries,
read_response,
sha256_hex,
@@ -29,12 +30,12 @@ from bot_bottle.supervise import (
FIXED = datetime(2026, 5, 25, 12, 0, 0, tzinfo=timezone.utc)
def _proposal(slug: str = "dev", tool: str = TOOL_CAPABILITY_BLOCK) -> Proposal:
def _proposal(slug: str = "dev", tool: str = TOOL_EGRESS_ALLOW) -> Proposal:
payloads = {
TOOL_CAPABILITY_BLOCK: "FROM python:3.13\n",
supervise.TOOL_ALLOW: "routes:\n - host: example.com\n",
supervise.TOOL_EGRESS_ALLOW: "routes:\n - host: example.com\n",
supervise.TOOL_EGRESS_BLOCK: "routes:\n - host: example.com\n",
TOOL_GITLEAKS_ALLOW: "file: tests/test_fixture.py\nline: 3\n",
TOOL_EGRESS_TOKEN_ALLOW: "host: api.example.com\ndetector: token\n",
}
payload = payloads.get(tool, "")
return Proposal.new(
@@ -84,14 +85,14 @@ class TestDiscoverPending(_FakeHomeMixin, unittest.TestCase):
def test_sorted_by_arrival_across_bottles(self):
early = Proposal.new(
bottle_slug="api", tool=TOOL_CAPABILITY_BLOCK,
proposed_file="FROM python:3.13\n", justification="early",
bottle_slug="api", tool=TOOL_EGRESS_ALLOW,
proposed_file="routes:\n - host: early.example.com\n", justification="early",
current_file_hash="h",
now=datetime(2026, 5, 25, 10, 0, 0, tzinfo=timezone.utc),
)
late = Proposal.new(
bottle_slug="dev", tool=TOOL_CAPABILITY_BLOCK,
proposed_file="FROM python:3.13\n", justification="late",
bottle_slug="dev", tool=TOOL_EGRESS_ALLOW,
proposed_file="routes:\n - host: late.example.com\n", justification="late",
current_file_hash="h",
now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc),
)
@@ -120,7 +121,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
def tearDown(self):
self._teardown_fake_home()
def _enqueue(self, tool: str = TOOL_CAPABILITY_BLOCK):
def _enqueue(self, tool: str = TOOL_EGRESS_ALLOW):
p = _proposal(tool=tool)
qdir = supervise.queue_dir_for_slug("dev")
qdir.mkdir(parents=True, exist_ok=True)
@@ -129,19 +130,29 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
def test_approve_writes_response(self):
qp = self._enqueue()
supervise_cli.approve(qp)
# capability-block is archived on approve, so the response file
# moves to processed/ before the caller can read it.
resp = read_response(qp.queue_dir / "processed", qp.proposal.id)
with patch(
"bot_bottle.cli.supervise.apply_routes_change",
return_value=("routes: []\n", "routes:\n - host: example.com\n"),
):
supervise_cli.approve(qp)
resp = read_response(qp.queue_dir, qp.proposal.id)
self.assertEqual(STATUS_APPROVED, resp.status)
self.assertIsNone(resp.final_file)
def test_approve_with_final_file_marks_modified(self):
qp = self._enqueue()
supervise_cli.approve(qp, final_file="FROM bookworm\n", notes="tweaked")
resp = read_response(qp.queue_dir / "processed", qp.proposal.id)
with patch(
"bot_bottle.cli.supervise.apply_routes_change",
return_value=("routes: []\n", "routes:\n - host: edited.example.com\n"),
):
supervise_cli.approve(
qp,
final_file="routes:\n - host: edited.example.com\n",
notes="tweaked",
)
resp = read_response(qp.queue_dir, qp.proposal.id)
self.assertEqual(STATUS_MODIFIED, resp.status)
self.assertEqual("FROM bookworm\n", resp.final_file)
self.assertEqual("routes:\n - host: edited.example.com\n", resp.final_file)
self.assertEqual("tweaked", resp.notes)
def test_reject_writes_rejection(self):
@@ -151,11 +162,6 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
self.assertEqual(STATUS_REJECTED, resp.status)
self.assertEqual("nope", resp.notes)
def test_no_audit_log_for_capability_block(self):
qp = self._enqueue(tool=TOOL_CAPABILITY_BLOCK)
supervise_cli.approve(qp)
self.assertEqual([], read_audit_entries("egress", "dev"))
def test_approve_egress_block_writes_audit_log(self):
qp = self._enqueue(tool=supervise.TOOL_EGRESS_BLOCK)
with patch(
@@ -196,10 +202,38 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
resp = read_response(qp.queue_dir, qp.proposal.id)
self.assertEqual("test fixture", resp.notes)
def test_approve_token_allow_leaves_response_for_egress(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
supervise_cli.approve(qp, notes="false positive")
# The egress addon polls the queue dir for the response; the TUI must
# not archive it (the addon archives after reading).
resp = read_response(qp.queue_dir, qp.proposal.id)
self.assertEqual(STATUS_APPROVED, resp.status)
self.assertEqual("false positive", resp.notes)
self.assertFalse((qp.queue_dir / "processed").exists())
# class TestCapabilityApplyWiring(_FakeHomeMixin, unittest.TestCase):
# # DISABLED — capability_apply functionality is currently commented out.
# pass
def test_token_allow_writes_no_audit_log(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
supervise_cli.approve(qp, notes="false positive")
self.assertEqual([], read_audit_entries("egress", "dev"))
def test_tui_token_allow_requires_reason(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
with patch.object(supervise_cli, "_prompt", return_value=""):
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
self.assertEqual("approve aborted (empty reason)", status)
self.assertFalse((qp.queue_dir / "processed").exists())
def test_tui_token_allow_writes_reason(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
with patch.object(supervise_cli, "_prompt", return_value="legit"):
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
self.assertIn("approved egress-token-allow", status)
resp = read_response(qp.queue_dir, qp.proposal.id)
self.assertEqual("legit", resp.notes)
def test_suffix_for_token_allow_is_txt(self):
self.assertEqual(".txt", supervise_cli._suffix_for_tool(TOOL_EGRESS_TOKEN_ALLOW))
class TestEditInEditor(unittest.TestCase):
@@ -246,10 +280,5 @@ class TestEditInEditor(unittest.TestCase):
os.environ["EDITOR"] = original_editor
# class TestCapabilityBlockSmolmachinesGuard(_FakeHomeMixin, unittest.TestCase):
# # DISABLED — capability_apply functionality is currently commented out.
# pass
if __name__ == "__main__":
unittest.main()
+180 -26
View File
@@ -20,6 +20,7 @@ import supervise as _sv # noqa: E402 # type: ignore
from bot_bottle import supervise_server # noqa: E402
from bot_bottle.supervise_server import (
ERR_INTERNAL,
ERR_INVALID_PARAMS,
ERR_INVALID_REQUEST,
ERR_METHOD_NOT_FOUND,
@@ -29,7 +30,9 @@ from bot_bottle.supervise_server import (
PROPOSED_FILE_FIELD,
ServerConfig,
TOOL_DEFINITIONS,
_RpcClientError,
_RpcError,
_RpcInternalError,
_response_timeout_from_env,
format_response_text,
handle_initialize,
@@ -47,19 +50,19 @@ from bot_bottle.supervise_server import (
class TestValidation(unittest.TestCase):
def test_capability_block_accepts_anything_nonempty(self):
validate_proposed_file(
_sv.TOOL_CAPABILITY_BLOCK,
"FROM python:3.13\nRUN apk add git\n",
)
def test_empty_proposed_file_rejected_for_tools_with_file_field(self):
with self.assertRaises(_RpcError):
validate_proposed_file(_sv.TOOL_CAPABILITY_BLOCK, " \n\t")
validate_proposed_file(_sv.TOOL_EGRESS_ALLOW, " \n\t")
def test_capability_block_rejected_as_unknown_tool(self):
with self.assertRaises(_RpcError) as cm:
validate_proposed_file("capability-block", "FROM python:3.13\n")
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
self.assertIn("unknown tool", cm.exception.message)
def test_egress_routes_yaml_is_validated(self):
validate_proposed_file(
_sv.TOOL_ALLOW,
_sv.TOOL_EGRESS_ALLOW,
"routes:\n - host: example.com\n",
)
@@ -67,6 +70,74 @@ class TestValidation(unittest.TestCase):
with self.assertRaises(_RpcError):
validate_proposed_file(_sv.TOOL_EGRESS_BLOCK, "routes: nope\n")
def test_egress_routes_yaml_rejects_log_full(self):
with self.assertRaises(_RpcError) as cm:
validate_proposed_file(
_sv.TOOL_EGRESS_ALLOW,
"log: 2\nroutes:\n - host: example.com\n",
)
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
self.assertIn("must not change egress logging", cm.exception.message)
# --- Error taxonomy --------------------------------------------------------
class TestRpcErrorTaxonomy(unittest.TestCase):
def test_rpc_client_error_is_rpc_error(self):
e = _RpcClientError(ERR_INVALID_PARAMS, "bad param")
self.assertIsInstance(e, _RpcError)
self.assertEqual(ERR_INVALID_PARAMS, e.code)
self.assertEqual("bad param", e.message)
def test_rpc_internal_error_is_rpc_error(self):
e = _RpcInternalError("disk full")
self.assertIsInstance(e, _RpcError)
self.assertEqual(ERR_INTERNAL, e.code)
self.assertEqual("disk full", e.message)
def test_rpc_internal_error_preserves_cause(self):
cause = OSError("no space left on device")
try:
raise _RpcInternalError("failed to write") from cause
except _RpcInternalError as e:
self.assertIs(cause, e.__cause__)
def test_parse_error_is_client_error(self):
with self.assertRaises(_RpcClientError):
parse_jsonrpc(b"{bad json")
def test_validation_error_is_client_error(self):
with self.assertRaises(_RpcClientError):
validate_proposed_file(_sv.TOOL_EGRESS_ALLOW, "routes: nope\n")
def test_unknown_tool_in_tools_call_is_client_error(self):
config = ServerConfig(bottle_slug="dev", queue_dir=Path("/unused"))
with self.assertRaises(_RpcClientError) as cm:
handle_tools_call({"name": "no-such-tool", "arguments": {}}, config)
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
class TestRpcInternalErrorOnIoFailure(unittest.TestCase):
def test_write_proposal_os_error_raises_internal(self):
config = ServerConfig(
bottle_slug="dev",
queue_dir=Path("/dev/null/cannot-exist"),
)
with self.assertRaises(_RpcInternalError) as cm:
handle_tools_call(
{
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "x",
},
},
config,
)
self.assertEqual(ERR_INTERNAL, cm.exception.code)
self.assertIsNotNone(cm.exception.__cause__)
# --- JSON-RPC parsing ------------------------------------------------------
@@ -147,8 +218,7 @@ class TestHandleToolsList(unittest.TestCase):
names = [t["name"] for t in result["tools"]] # type: ignore[index]
self.assertEqual(
sorted([
_sv.TOOL_ALLOW,
_sv.TOOL_CAPABILITY_BLOCK,
_sv.TOOL_EGRESS_ALLOW,
_sv.TOOL_EGRESS_BLOCK,
_sv.TOOL_LIST_EGRESS_ROUTES,
]),
@@ -181,7 +251,7 @@ class TestHandleToolsList(unittest.TestCase):
self.assertNotIn("required", schema) # type: ignore[operator]
def test_egress_tools_take_routes_yaml_and_justification(self):
for tool_name in (_sv.TOOL_ALLOW, _sv.TOOL_EGRESS_BLOCK):
for tool_name in (_sv.TOOL_EGRESS_ALLOW, _sv.TOOL_EGRESS_BLOCK):
with self.subTest(tool_name=tool_name):
tool = next(t for t in TOOL_DEFINITIONS if t["name"] == tool_name)
schema = tool["inputSchema"]
@@ -224,10 +294,10 @@ class TestHandleToolsCall(unittest.TestCase):
try:
result = handle_tools_call(
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"name": _sv.TOOL_EGRESS_BLOCK,
"arguments": {
"dockerfile": "FROM python:3.13\n",
"justification": "need git",
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "need example.com",
},
},
self.config,
@@ -244,7 +314,7 @@ class TestHandleToolsCall(unittest.TestCase):
try:
result = handle_tools_call(
{
"name": _sv.TOOL_ALLOW,
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "need example.com",
@@ -264,9 +334,9 @@ class TestHandleToolsCall(unittest.TestCase):
try:
result = handle_tools_call(
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {
"dockerfile": "FROM python:3.13\n",
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "needed for tests",
},
},
@@ -288,20 +358,52 @@ class TestHandleToolsCall(unittest.TestCase):
with self.assertRaises(_RpcError):
handle_tools_call(
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"arguments": {"dockerfile": "FROM python:3.13\n"},
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {"routes_yaml": "routes:\n - host: example.com\n"},
},
self.config,
)
def test_missing_name_raises(self):
with self.assertRaises(_RpcError) as cm:
handle_tools_call({"arguments": {}}, self.config)
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
def test_arguments_must_be_object(self):
with self.assertRaises(_RpcError) as cm:
handle_tools_call(
{
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": [],
},
self.config,
)
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
self.assertIn("must be an object", cm.exception.message)
def test_capability_block_call_raises_unknown_tool(self):
with self.assertRaises(_RpcError) as cm:
handle_tools_call(
{
"name": "capability-block",
"arguments": {
"dockerfile": "FROM python:3.13\n",
"justification": "need git",
},
},
self.config,
)
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
self.assertIn("unknown tool", cm.exception.message)
def test_archives_proposal_after_response(self):
responder = self._respond_when_proposal_appears(_sv.STATUS_APPROVED)
try:
handle_tools_call(
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {
"dockerfile": "FROM python:3.13\n",
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "x",
},
},
@@ -323,10 +425,10 @@ class TestHandleToolsCall(unittest.TestCase):
)
result = handle_tools_call(
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {
"dockerfile": "FROM python:3.13\n",
"justification": "need a capability",
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "need egress",
},
},
config,
@@ -341,6 +443,31 @@ class TestHandleToolsCall(unittest.TestCase):
class TestHandleListEgressRoutes(unittest.TestCase):
def test_success_returns_body_text(self):
class _Resp:
def __enter__(self):
return self
def __exit__(self, exc_type: type[BaseException] | None, exc: BaseException | None, tb: object) -> bool:
return False
def read(self):
return b"[{\"host\": \"example.com\"}]"
class _Opener:
def open(self, *args, **kwargs): # noqa: ANN001, ANN002, ANN003 # type: ignore
return _Resp()
with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()):
result = handle_list_egress_routes(
{},
ServerConfig(bottle_slug="dev", queue_dir=Path("/unused")),
)
self.assertFalse(result["isError"]) # type: ignore[index]
text = result["content"][0]["text"] # type: ignore[index]
self.assertIn("example.com", text)
def test_url_error_returns_tool_error(self):
class _Opener:
def open(self, *args, **kwargs): # noqa: ANN001, ANN002, ANN003 # type: ignore
@@ -400,6 +527,13 @@ class TestFormatResponseText(unittest.TestCase):
self.assertIn("the operator modified", text.lower())
class TestFormatPendingResponseText(unittest.TestCase):
def test_formats_timeout_message(self):
text = supervise_server.format_pending_response_text(12.5)
self.assertIn("status: pending", text)
self.assertIn("12.5s", text)
# --- End-to-end HTTP sanity ------------------------------------------------
@@ -450,8 +584,8 @@ class TestHttpEndToEnd(unittest.TestCase):
self.assertEqual("2.0", result["jsonrpc"])
self.assertEqual(1, result["id"])
names = [t["name"] for t in result["result"]["tools"]] # type: ignore[index]
self.assertIn(_sv.TOOL_CAPABILITY_BLOCK, names)
self.assertIn(_sv.TOOL_ALLOW, names)
self.assertNotIn("capability-block", names)
self.assertIn(_sv.TOOL_EGRESS_ALLOW, names)
self.assertIn(_sv.TOOL_EGRESS_BLOCK, names)
def test_unknown_method_returns_jsonrpc_error(self):
@@ -460,6 +594,26 @@ class TestHttpEndToEnd(unittest.TestCase):
)
self.assertEqual(ERR_METHOD_NOT_FOUND, result["error"]["code"]) # type: ignore[index]
def test_internal_error_returns_err_internal_over_http(self):
with patch.object(
supervise_server._sv, "write_proposal",
side_effect=OSError("disk full"),
):
result = self._post_jsonrpc({
"jsonrpc": "2.0",
"id": 99,
"method": "tools/call",
"params": {
"name": _sv.TOOL_EGRESS_ALLOW,
"arguments": {
"routes_yaml": "routes:\n - host: example.com\n",
"justification": "x",
},
},
})
self.assertIn("error", result)
self.assertEqual(ERR_INTERNAL, result["error"]["code"]) # type: ignore[index]
def test_health_endpoint(self):
conn = http.client.HTTPConnection("127.0.0.1", self.port, timeout=5)
try: