Compare commits

...

200 Commits

Author SHA1 Message Date
didericis 49c2ed0b93 feat(smolmachines): run backend on Linux
lint / lint (push) Failing after 1m52s
test / unit (pull_request) Successful in 45s
test / integration (pull_request) Successful in 17s
Port the smolmachines backend so BOT_BOTTLE_BACKEND=smolmachines
works on Linux (KVM), not just macOS:

- Preflight gates /dev/kvm presence + accessibility on Linux with
  actionable remediation (kvm module, kvm group).
- smolvm state-DB path is platform-derived (XDG on Linux).
- force_allowlist runs on both platforms and is fail-closed: it
  verifies the persisted TSI allowlist and dies rather than booting
  a VM whose egress confinement it can't confirm. Previously it
  no-oped on Linux, failing OPEN.
- allocate() does per-bottle 127.0.0.<N> scoping on Linux too (no
  ifconfig needed — all of 127/8 is already loopback); only
  ensure_pool's lo0 aliasing stays macOS-only.
- README documents Linux + NixOS host setup.

Linux/KVM integration (the sandbox-escape acceptance gate) is
pending verification on a NixOS host; unit tests cover the new
platform branches.

Issue: #283

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
2026-06-25 17:08:22 -04:00
didericis a666f9fe54 docs(prd): add PRD for smolmachines backend on Linux
Design for porting the smolmachines backend off macOS-only: KVM
preflight, platform-aware smolvm state-DB path, fail-closed TSI
allowlist enforcement, and per-bottle loopback scoping on Linux.
NixOS is the primary validation target.

Issue: #283

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
2026-06-25 17:08:22 -04:00
github-actions[bot] eaf6b1f72e ci(prd): assign sequential numbers to new PRDs 2026-06-25 20:43:06 +00:00
didericis ca910f8f4f fix(start): show bottle lineage root-first with -> arrows
lint / lint (push) Successful in 1m51s
test / unit (push) Successful in 43s
test / integration (push) Successful in 17s
test / unit (pull_request) Successful in 43s
test / integration (pull_request) Successful in 18s
prd-number / assign-numbers (push) Successful in 21s
Update Quality Badges / update-badges (push) Failing after 1m47s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 16:13:19 -04:00
didericis-codex 338c08a243 test: fix cli selector typing 2026-06-25 16:13:19 -04:00
didericis-claude 6faa6f67aa feat(tui,start): space/enter split, bottle lineage, YAML preflight
Three UX improvements requested in #270 review:

- filter_multiselect: Space toggles selection, Enter confirms (was both)
- bottle picker: bottles with extends chains show ancestry labels
  (e.g. 'claude-dev <- bot-bottle-dev <- dev') for at-a-glance lineage
- preflight: replaces key-value summary with YAML of the resolved manifest
2026-06-25 16:13:19 -04:00
didericis-claude b6ae6af63a fix(types): resolve pyright errors introduced in #269 changes
- manifest.py: remove unused load_bottle_chain_from_dir import
- manifest_extends.py: drop redundant ManifestEgressRoute annotation
- test_cli_start_selector.py: remove unused call import
- test_cli_tui.py: move Optional/constants to top, annotate FakeScreen,
  remove unused curses import
- test_manifest_bottle_merge.py: add type args to dict, annotate **kwargs
2026-06-25 16:13:19 -04:00
didericis-claude ad72eeddc1 feat(tui): add reordering to filter_multiselect
Tab switches focus to the selected-order panel; K/J shift the
highlighted item up/down; Space/Enter removes it. The filter list dims
while the order panel is active. Help line updates per focus mode.
2026-06-25 16:13:19 -04:00
didericis-claude 61f89de2da docs(prd): activate PRD for separate agent/bottle selection 2026-06-25 16:13:19 -04:00
didericis-claude 1ba185d1e0 feat(#269): separate agent and bottle selection at launch time
- `bottle:` in agent frontmatter is now optional; agents without it
  are portable and require bottles to be selected at launch.
- Adds `filter_multiselect` to `tui.py`: multi-select picker with
  ordered selection list, Space/Enter to toggle, Ctrl-D to confirm.
- `ManifestIndex` gains `all_bottle_names` and `load_for_agent` accepts
  `bottle_names: tuple[str, ...]` to merge bottles in order at runtime.
- `merge_bottles_runtime` in `manifest_extends.py` applies the same
  field-merge rules as `extends:` to pre-resolved bottle objects.
- `BottleSpec` gains `bottle_names`; `_validate` and `write_launch_metadata`
  thread it through so `resume` replays the same bottle configuration.
- `cmd_start` shows the bottle multiselect after agent selection,
  pre-populated from the agent's `bottle:` field when present.
- Existing agents with `bottle:` declared continue to work unchanged.
2026-06-25 16:13:19 -04:00
didericis-claude e82dbaba09 docs(prd): draft PRD for separate agent/bottle selection
Closes #269.
2026-06-25 16:13:19 -04:00
Quality Badge Bot d7fbe8e8a9 chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors
- Coverage: 79%

[skip ci]
2026-06-25 20:11:29 +00:00
didericis 50f5b3aa7f ci(badges): add coverage percentage to quality badges
test / integration (pull_request) Successful in 16s
test / unit (pull_request) Successful in 43s
test / unit (push) Successful in 44s
lint / lint (push) Successful in 1m49s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m53s
The update-badges workflow only refreshed pylint and pyright. Add a
coverage step that runs the unit suite under coverage.py, extracts the
TOTAL percentage, and updates a new coverage badge in the README.
Also trigger the workflow on .coveragerc changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
2026-06-25 15:13:44 -04:00
didericis-claude 45a096413f fix: add type annotations to __exit__ context manager (pyright)
lint / lint (push) Successful in 1m47s
test / unit (pull_request) Successful in 43s
test / integration (pull_request) Successful in 16s
2026-06-25 15:03:06 -04:00
didericis c6479d62e4 test: add coverage for git gate and supervise server 2026-06-25 15:03:06 -04:00
didericis d0cad3a559 chore: ignore coverage data 2026-06-25 15:03:06 -04:00
didericis c2ddac1be5 test: fix integration coverage failures 2026-06-25 15:03:06 -04:00
didericis 446414144e test: tune coverage exclusions 2026-06-25 15:03:06 -04:00
didericis 8188d6304e ci: add coverage.py reporting 2026-06-25 15:03:06 -04:00
github-actions[bot] 9f7c067e85 ci(prd): assign sequential numbers to new PRDs 2026-06-25 11:42:07 +00:00
didericis-codex 90e84a52e6 fix: remove unused supervise import for pyright
test / unit (pull_request) Successful in 37s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
Update Quality Badges / update-badges (push) Failing after 1m18s
prd-number / assign-numbers (push) Successful in 23s
test / unit (push) Successful in 33s
test / integration (push) Successful in 17s
2026-06-25 05:45:55 -04:00
didericis-claude 75755a472f refactor: drop redundant single-parent fast path in _resolve_one_bottle
lint / lint (push) Failing after 1m50s
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 18s
_fold_parents with one name returns after the first resolve; the
single-element branch was a verbatim copy of the general path.
2026-06-25 05:10:03 -04:00
didericis-claude 2f3dc57fa9 fix: resolve pyright reportUnnecessaryIsInstance in _resolve_one_bottle
Validate list entries against object-typed raw_list before narrowing to
list[str], so the isinstance(pname, str) check is not redundant.
2026-06-25 05:10:03 -04:00
didericis-claude 302920e290 feat: support multiple parents in bottle extends:
Allow extends: to accept a list of bottle names in addition to a plain
string. Parents are resolved independently and folded left-to-right
into a single combined parent before the child is merged on top, so
orthogonal concerns (base env, networking, agent provider) can live in
separate bottles without forcing a linear chain.

Merge rules for the parent fold: env dict-merge with later winning on
collision; git-gate.user per-field overlay; git-gate.repos union by
name with later winning per-field on same name; egress.routes
concatenated; all scalar fields (supervise, agent_provider, egress.log)
use last-wins. The existing child-wins-over-all-parents rule is
unchanged. Cycle detection, diamond deduplication, and missing/invalid
parent errors all work across multi-parent graphs.

Closes #268
2026-06-25 05:10:03 -04:00
Quality Badge Bot ca1b4afaea chore: update quality badges
- Pylint: 9.93/10
- Pyright: 1 errors

[skip ci]
2026-06-25 09:06:44 +00:00
didericis-codex d2072b13be feat!: remove capability apply
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 18s
lint / lint (push) Failing after 1m53s
test / unit (push) Successful in 40s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m37s
2026-06-25 08:58:28 +00:00
didericis-codex 36c5b7025b feat: add ripgrep to agent images
lint / lint (push) Successful in 1m48s
2026-06-25 04:32:53 -04:00
didericis-claude 515a95a79d fix: escape quotes/newlines in YAML and gitconfig emitters
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
test / unit (push) Successful in 32s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m18s
Closes #258.

`egress_render_routes` and `_render_match_entry` now pass all manifest
strings (host, auth_scheme, token_env, path/header values) through
`_yaml_str_escape` before interpolating into double-quoted YAML scalars,
preventing stray `"` or newlines from corrupting routes.yaml.

`git_gate_render_gitconfig` now calls `_gitconfig_validate_value` on
each Upstream value (and the derived alias) before writing the
`insteadOf` line, rejecting any value containing a newline that would
inject arbitrary gitconfig keys.
2026-06-25 04:23:13 -04:00
didericis-claude 0bace7615a refactor: rename GIT_GATE_DAEMON_TIMEOUT_SECS to GIT_GATE_TIMEOUT_SECS
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
test / unit (push) Successful in 35s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m20s
The constant now covers the daemon path, the HTTP backend access-hook,
and the git http-backend CGI subprocess, so 'daemon' in the name was
too narrow. Updated the comment to list all three current uses.
2026-06-25 04:12:43 -04:00
didericis-claude c0d3f16519 refactor: import GIT_GATE_DAEMON_TIMEOUT_SECS instead of duplicating the value 2026-06-25 04:12:43 -04:00
didericis-claude 508c537deb fix: add explicit timeouts to subprocess and HTTP calls in git-gate paths
Closes #255. Without timeouts, a hung upstream during the access-hook
or git http-backend CGI call (git_http_backend.py) and a stalled Gitea
API during deploy-key provisioning (contrib/gitea/deploy_key_provisioner.py)
could wedge a sidecar indefinitely. Adds GIT_HTTP_BACKEND_TIMEOUT_SECS
(30s) to both subprocess.run calls in the HTTP backend, mirroring the
existing GIT_GATE_DAEMON_TIMEOUT_SECS on the daemon path. Adds
_API_TIMEOUT_SECS (30s) and _KEYGEN_TIMEOUT_SECS (10s) to the Gitea
provisioner's urlopen and ssh-keygen calls. Tests verify the timeout
values are forwarded in all four call sites.
2026-06-25 04:12:43 -04:00
didericis-claude d99dba037c feat(supervise): typed RPC error taxonomy for dispatch
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m48s
test / unit (push) Successful in 32s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m20s
Introduce _RpcClientError and _RpcInternalError as distinct subclasses
of _RpcError so the dispatcher can handle bad requests and server-side
faults differently — returning client errors verbatim and logging
internal faults with their cause before replying ERR_INTERNAL.

Wrap write_proposal and archive_proposal IO with _RpcInternalError
so OS failures surface through the typed path instead of the bare
Exception fallback. All existing raise _RpcError(...) call sites
converted to _RpcClientError.

Closes #253
2026-06-25 04:02:39 -04:00
didericis-claude 9a878bd885 fix: guard CGI Status-line parse in _write_cgi_response
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 16s
lint / lint (push) Successful in 1m47s
test / unit (push) Successful in 32s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m19s
An empty or non-numeric Status: header from git http-backend raised
ValueError/IndexError that escaped the handler thread. Wrap the parse
in a try/except and fall back to HTTP 500 instead.

Closes #254
2026-06-25 03:47:05 -04:00
didericis 0f72843150 fix(macos-container): anchor relative Dockerfile path to build context
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m49s
test / unit (push) Successful in 33s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m19s
`container build` resolves -f relative to the current working directory,
not the build context, so builds failed from any cwd other than the repo
root. Anchor a relative Dockerfile to the context before passing it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 03:27:46 -04:00
didericis fd6b14fb32 fix: route remote control through provider startup args
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 18s
lint / lint (push) Successful in 1m46s
test / unit (push) Successful in 30s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m23s
2026-06-25 03:08:47 -04:00
didericis-claude 9f9aa2e762 refactor: remove load_routes, use load_config(...).routes in tests
test / unit (pull_request) Successful in 48s
test / integration (pull_request) Successful in 26s
lint / lint (push) Successful in 1m45s
test / unit (push) Successful in 32s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m21s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 06:07:47 +00:00
didericis-codex 454baaf3a1 fix(egress): validate proposed full config
lint / lint (push) Successful in 2m23s
test / unit (pull_request) Successful in 47s
test / integration (pull_request) Successful in 28s
2026-06-25 05:25:42 +00:00
github-actions[bot] 8a092504b8 ci(prd): assign sequential numbers to new PRDs 2026-06-25 04:50:36 +00:00
didericis-codex e7dacf7d86 fix: satisfy pyright for log redaction tests
test / integration (pull_request) Successful in 29s
prd-number / assign-numbers (push) Successful in 1m6s
Update Quality Badges / update-badges (push) Successful in 1m40s
test / unit (pull_request) Successful in 52s
lint / lint (push) Successful in 2m20s
test / unit (push) Successful in 50s
test / integration (push) Successful in 27s
2026-06-25 00:32:42 -04:00
didericis-claude 9b929d0684 fix(egress): strip injected Authorization and redact bodies in LOG_FULL path
_log_request and _log_response wrote headers and bodies to stderr verbatim.
_log_request also included the sidecar-injected upstream Authorization value,
exposing live bearer tokens on every allowed request under LOG_FULL.

Apply redact_tokens to all header values and bodies in both log functions;
exclude the authorization header from _log_request entirely since its value
is always a live sidecar-injected credential by the time _log_request runs.

Closes #257
2026-06-25 00:32:42 -04:00
github-actions[bot] ec41f629a4 ci(prd): assign sequential numbers to new PRDs 2026-06-25 04:19:30 +00:00
didericis-codex d9a9eef276 docs: remove prd-new code citations
test / integration (pull_request) Successful in 46s
test / unit (pull_request) Successful in 1m4s
lint / lint (push) Successful in 2m36s
prd-number / assign-numbers (push) Successful in 1m24s
test / integration (push) Successful in 34s
test / unit (push) Successful in 52s
Update Quality Badges / update-badges (push) Successful in 2m11s
2026-06-25 03:57:41 +00:00
didericis-codex 5204b98777 refactor(egress): centralize launch env entries
lint / lint (push) Successful in 2m12s
test / unit (pull_request) Successful in 43s
test / integration (pull_request) Successful in 25s
2026-06-25 03:35:24 +00:00
didericis-codex 14ae89580a fix(egress): wire canary env for smolmachines
lint / lint (push) Successful in 2m16s
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 23s
2026-06-25 03:31:51 +00:00
didericis-codex 4808ef557a fix(egress): randomize canary secret env name
lint / lint (push) Successful in 2m15s
test / unit (pull_request) Successful in 45s
test / integration (pull_request) Successful in 26s
2026-06-25 03:25:37 +00:00
didericis-codex 0a7e166b35 fix(tests): remove unused dlp entropy import
lint / lint (push) Successful in 2m8s
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 23s
2026-06-24 23:09:11 -04:00
didericis-claude a920203730 fix(dlp): skip projection passes when exact variant is safe-listed
When a supervisor-approved safe-token exactly matched an env secret
(Pass 1), Passes 2 & 3 (alnum projection) still ran and re-blocked on
the same value.  Track whether any variant was found-and-approved and
skip the projection passes for that secret in that case.
2026-06-24 23:09:11 -04:00
didericis-claude e02fab15d0 docs(prd): flip prd-new-strengthen-outbound-exfil-detection Draft → Active
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis-claude 11cf12188d feat(egress): inject per-session canary token into sidecar and agent environments
EgressPlan gains a `canary: str` field (default "") populated in Egress.prepare()
using secrets.token_urlsafe(32).  Each launched bottle:

  - sidecar receives EGRESS_TOKEN_CANARY=<value> (literal env entry, scanned by
    existing known-secrets detector without any detector code changes)
  - agent receives BOT_BOTTLE_CANARY=<value> (visible fake secret that signals
    exfiltration with zero false positives if it appears in outbound traffic)

Docker compose and macos-container backends updated; smolmachines shares docker
compose and so picks this up automatically.  Unit tests cover canary uniqueness,
detection via scan_known_secrets, and EgressPlan backward-compat default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis-claude 701df6cb2f feat(dlp): fragmentation resistance, entropy detector, broadened known-value scan
- _alnum_projection(): strip non-alphanumeric chars for separator-injection detection
- scan_known_secrets() gains two extra passes per secret after exact-variant matching:
  alnum-projection exact match (catches hyphens/spaces between secret chars) and a
  sliding-window partial-match scan (catches chunked substrings ≥ PARTIAL_MATCH_MIN_LEN)
- scan_known_secrets() accepts sensitive_prefixes param (default ("EGRESS_TOKEN_",))
  so redact_tokens and call-sites can extend the scanned env-var prefix set
- scan_entropy() warn-only detector flagging windows with Shannon entropy ≥ 5.5 bits/char
- "entropy" added to OUTBOUND_DETECTOR_NAMES; scan_outbound opts it in only when
  explicitly listed in dlp.outbound_detectors (never part of the default "all" set)
- scan_outbound reads BOT_BOTTLE_SENSITIVE_PREFIXES from environ to extend
  scan_known_secrets beyond EGRESS_TOKEN_* without schema changes
- Binary bodies decoded via latin-1 fallback (bijective byte↔codepoint) instead
  of utf-8 errors=replace, preserving ASCII secret strings in binary payloads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis-claude ea6bc5a170 docs: draft PRD prd-new for strengthen-outbound-exfil-detection
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 23:09:11 -04:00
didericis ecaae708f7 feat(provider): support startup args settings
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 26s
lint / lint (push) Successful in 2m12s
test / unit (push) Successful in 41s
test / integration (push) Successful in 26s
Update Quality Badges / update-badges (push) Successful in 2m9s
2026-06-24 22:51:27 -04:00
didericis-claude 2e790268b0 fix(deploy-key): raise DeployKeyCollisionError on 422 key conflicts
lint / lint (push) Successful in 2m7s
test / unit (push) Successful in 46s
test / integration (push) Successful in 25s
Update Quality Badges / update-badges (push) Successful in 2m7s
Gitea returns HTTP 422 when a deploy key title or public key content
already exists on the repo. The provisioner previously surfaced this
as a generic RuntimeError with the raw status code. Introduce
DeployKeyCollisionError (a RuntimeError subclass) in the base module
and detect 422 in GiteaDeployKeyProvisioner.create so callers can
catch collisions explicitly and the error message names the repo and
title involved.
2026-06-25 02:23:12 +00:00
didericis-claude a421d1d688 Rename TOOL_ALLOW to TOOL_EGRESS_ALLOW
lint / lint (push) Successful in 3m50s
test / unit (push) Successful in 38s
test / integration (push) Successful in 23s
Update Quality Badges / update-badges (push) Successful in 1m38s
The constant and its MCP tool name ("allow" → "egress-allow") were the
only supervise tools without an egress-scoped identifier, despite the
tool being egress-only (routes.yaml payload, COMPONENT_FOR_TOOL maps
it to "egress", always grouped with TOOL_EGRESS_BLOCK). The rename
brings it in line with TOOL_EGRESS_BLOCK and TOOL_EGRESS_TOKEN_ALLOW,
and adds TOOL_EGRESS_ALLOW and TOOL_EGRESS_BLOCK to __all__ (both were
previously absent).
2026-06-25 01:23:10 +00:00
didericis d2d50be65a Restructure PRD 0062 to the init-prd template
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 25s
lint / lint (push) Successful in 2m8s
test / unit (push) Successful in 41s
test / integration (push) Successful in 25s
Update Quality Badges / update-badges (push) Successful in 1m32s
Conform the PRD to the standard PRD-new skeleton: add a Scope section
(In scope / Out of scope), rename Design -> Proposed Design and split
its prose into New services / Existing code touched / Data model
changes / External dependencies, fold the old Implementation chunks
into In scope, and add a References section. No change in substance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 21:10:31 -04:00
didericis 1ad710a041 Default agent-provider routes to the redact on-match policy
lint / lint (push) Successful in 1m42s
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 16s
Provider routes (the agent talking to its own LLM API — api.anthropic.com,
the Codex backend, etc.) carry the whole conversation payload, which is the
worst source of token-shaped false positives. egress_routes_for_bottle now
fills outbound_on_match=redact on any provider route that doesn't set it
explicitly, so a match there is scrubbed and forwarded rather than blocked
or queued for the operator. A provider that sets the policy keeps its
choice; manifest routes still default to supervise.

Tests: provider route gets redact default, explicit provider policy
preserved, manifest route unaffected. README + PRD 0062 updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 20:40:36 -04:00
didericis b411577e76 Stop scanning the request body for CRLF injection
lint / lint (push) Successful in 1m41s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 18s
A 403 "egress DLP: URL-encoded CRLF (%0d%0a)" was firing on legitimate
requests (e.g. the Claude Code login flow) and bypassing the on-match
policy entirely, because CRLF blocks carry no matched value and were
routed straight to a hard 403.

Root cause: CRLF injection is only an attack in the request line and
headers. An HTTP body is delimited by Content-Length, so CRLF bytes in
the body cannot split the request — but the scan flattened the body into
the same blob it checked, so form-encoded / multi-line body content
(which legitimately contains %0d%0a) tripped it.

Fix:
- scan_outbound takes a crlf_text param; the addon scans CRLF only over
  the body-excluded request line + headers. crlf_text=None keeps the
  old full-blob behavior for host-side callers/tests; the websocket path
  passes "" since a data frame is not a request line.
- The redact policy now also scrubs CRLF (new strip_crlf helper) from the
  path and headers, so redact is a complete escape hatch and structural
  CRLF in the URL/headers can be forwarded when a route opts into it.

Tests: strip_crlf unit tests; scan_outbound crlf_text body-exclusion and
backward-compat tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 20:37:26 -04:00
didericis cdfaaa3de8 Add dlp.outbound_on_match policy (block | redact | supervise)
lint / lint (push) Successful in 1m41s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 18s
Give each egress route a policy for what the proxy does when an outbound
DLP detector matches a token, defaulting to the supervise flow added in
the previous commit. The goal is cutting false-positive friction without
weakening default-deny.

- redact: scrub the matched value(s) from the body, non-host headers, and
  path/query via redact_tokens, then re-scan. Forward if clean; fail
  closed with a 403 if a match remains on a surface redaction can't
  rewrite (the hostname, or a unicode-evasion token). For routes where a
  token-shaped value is noise the upstream doesn't need.
- block: the original hard 403, never overridable.
- supervise (default, unset): hold the request for operator approval.

Structural blocks (CRLF, no safelist-able value) stay hard 403s under
every policy.

Threads outbound_on_match from the bottle manifest (manifest_egress)
through the resolved EgressRoute and rendered routes.yaml (egress.py) to
the addon's Route (egress_addon_core), and round-trips it via the
list-egress-routes introspection endpoint. The allow/egress-block tool
descriptions document the new key.

Tests: manifest parse/validation, core parse/validation, full
manifest->render->addon round-trip for redact. README + PRD 0062 updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 16:50:13 -04:00
didericis 7f2352287e PRD 0062: supervisor override for egress token blocks
lint / lint (push) Successful in 1m42s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
When the outbound DLP catches a token, route the block through the
existing supervisor approval queue instead of returning 403 outright.
The egress proxy holds the request open until the operator answers, then
remembers an approved value for the life of the proxy so the request --
and later ones carrying it -- flow through. Fails closed on rejection,
timeout, malformed response, or when supervise is disabled.

- ScanResult.matched carries the raw matched substring (sidecar-only;
  never logged or written to the proposal). scan_outbound and the token
  detectors take a safe_tokens set and skip approved values, continuing
  past a safelisted match so a second secret in the same request is
  still caught.
- New egress-token-allow proposal tool, written directly to the queue by
  the addon (the gitleaks-allow pattern from PRD 0061). build_token_allow
  _payload renders host/method/path/detector reason + redacted context.
- Async request hook polls the queue without stalling the proxy event
  loop; EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS (default 300) bounds the wait.
- Supervisor TUI renders egress-token-allow like gitleaks-allow: report
  only, modify unavailable, approval requires a recorded reason.
- Unit tests for the matched/safe-tokens plumbing, payload builder, tool
  constant round-trip, and TUI paths; README + PRD 0062.

Closes #261.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
2026-06-24 16:12:50 -04:00
didericis 7cb967770e feat(log): add leveled severity and structured context to log wrappers
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
lint / lint (push) Successful in 1m39s
test / unit (push) Successful in 31s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m32s
log.py was bare print-to-stderr wrappers with no levels or attributable
context (issue #252). Add:

- Ordered severities (debug/info/warn/error) gated by
  BOT_BOTTLE_LOG_LEVEL (default info). debug is silent by default;
  error always surfaces (nothing sits above it), so the fatal die path
  stays visible regardless of configured level.
- An optional `context` mapping on every wrapper, rendered as a
  parseable ` [k=v ...]` suffix (keys sorted; whitespace/quoted values
  quoted) so failures can be filtered and correlated.

Default output with no context is byte-identical to the original lines,
so the 100+ existing single-string call sites are unaffected. Wires the
supervise crash path (the example the issue names) to attach error_type
and crash_log context. Adds test_log.py (backward-compat, context
rendering, level gating, die surfacing).

Closes #252.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-24 15:37:57 -04:00
didericis 80eca740d6 docs(research): replace unsourced "20% malicious skills" with cited empirical figures
The "~20% of ClawHub skills malicious" claim had no traceable source and
is contradicted by the empirical literature. Replace with the Jan 2026
large-scale study (98,380-skill snapshot: 157 confirmed malicious, ~71%
credential harvesters, exfiltration overwhelmingly naive) and add the
arXiv citation. The corrected figures still support the supply-chain
threat point and are defensible under scrutiny.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-24 09:32:19 -04:00
didericis 369d332204 Default the supervise flag to true
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m40s
test / unit (push) Successful in 30s
test / integration (push) Successful in 15s
Update Quality Badges / update-badges (push) Successful in 1m44s
Issue #249: bottles should be supervised by default. Rather than
remove the flag (which would make supervision mandatory and is the
wrong plane for cost-control enforcement — see #251), keep the
opt-out and flip the default. Bottles that omit `supervise:` now get
the stuck-recovery sidecar; `supervise: false` still skips it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-23 20:48:04 -04:00
didericis 31cde11b0d docs: correct stale role field and claude provider auth example
lint / lint (push) Successful in 1m53s
The egress route fields table described `role` as a functional field
that wires built-in auth flows. PRD 0029 removed the
`claude_code_oauth` role; the manifest parser now rejects any `role`
value as reserved-for-future-use. Provider auth routes are injected
from `agent_provider.auth_token`.

- README: fix the `role` row to state it is reserved and any value is
  rejected at load.
- examples/bottles/claude.md: the manual `api.anthropic.com` route used
  the rejected `role` key and, even without it, would be silently
  dropped (provider-injected routes win for a provisioned host) — so its
  auth never took effect and the dlp comments described a route that
  never exists in the plan. Replace it with the canonical
  `agent_provider.auth_token` shape.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
2026-06-23 17:53:18 -04:00
didericis-claude c41751f3b9 docs: add role and git.fetch to egress route fields table
Both fields were missing from the reference table added in the preceding
commit — `role` is visible in examples/bottles/claude.md and `git.fetch`
is documented in PRD 0052 but neither appeared in the README table.
2026-06-23 17:48:19 -04:00
didericis e2422c20a0 docs: document egress matches, dlp fields, and detector defaults 2026-06-23 17:48:19 -04:00
github-actions[bot] de71533a17 ci(prd): assign sequential numbers to new PRDs 2026-06-23 21:47:01 +00:00
didericis-claude 88c4f61901 fix: don't archive gitleaks-allow response before gate reads it
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 18s
lint / lint (push) Successful in 1m52s
prd-number / assign-numbers (push) Successful in 45s
test / unit (push) Successful in 36s
test / integration (push) Successful in 21s
Update Quality Badges / update-badges (push) Successful in 1m19s
The TUI was calling archive_proposal for gitleaks-allow immediately
after write_response, moving the response file to processed/ within
microseconds. The git-gate shell loop polls queue_dir for the response
file every second — it never sees it and hangs until timeout.

capability-block is handled by the MCP sidecar which archives after
reading; gitleaks-allow is handled by the shell gate which archives
after processing. Let the gate own the archive step.
2026-06-23 17:37:01 -04:00
didericis-claude c666eaa63f fix: add TOOL_GITLEAKS_ALLOW to __all__ in supervise.py 2026-06-23 17:36:08 -04:00
didericis-codex 83eb9e4041 docs(prd): add gitleaks allow supervision 2026-06-23 17:36:08 -04:00
didericis-codex 33333ac4d9 Supervise gitleaks inline allow exceptions 2026-06-23 17:36:08 -04:00
github-actions[bot] 4d56f515bc ci(prd): assign sequential numbers to new PRDs 2026-06-23 21:32:54 +00:00
didericis-claude c48c3688b8 fix(smolmachines): exclude /tmp+/var/tmp from snapshot; mkdir -p on boot
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 23s
lint / lint (push) Successful in 1m59s
prd-number / assign-numbers (push) Successful in 1m8s
test / unit (push) Successful in 35s
test / integration (push) Successful in 21s
Update Quality Badges / update-badges (push) Successful in 1m22s
On resume from a committed snapshot, smolvm's pack process remaps all
file uids to the host uid (501 on macOS). Files in /tmp that were
created during the session (e.g. /tmp/claude-1000 owned by node=uid
1000) get remapped to 501. Claude Code then refuses to use the temp
directory because it's owned by a different uid.

Two-part fix:
- Exclude ./tmp and ./var/tmp from the tar in _exec_tar_to_file.
  Both directories are ephemeral; a resumed VM should start with clean
  temp directories identical to a fresh VM.
- Add mkdir -p /tmp /var/tmp to _init_vm before chown/chmod, so the
  directories are created if the committed snapshot omitted them.
2026-06-23 16:53:41 -04:00
didericis-claude 6040b20e6e fix(smolmachines): write tar to VM file then machine_cp to host
Replace the Popen/stdout=PIPE approach with a write-then-copy
strategy that avoids binary-stdout piping through the smolvm exec
channel entirely:

1. Probe connectivity with `machine_exec(machine, ["true"])` first.
   If this fails while an interactive session is running, the error
   now says "concurrent exec not available" instead of the opaque
   "<no stderr>".

2. Run `tar --create --gzip --file=/var/tmp/.bot-bottle-commit.tar.gz`
   inside the VM via machine_exec (same mechanism used during
   provisioning). tar writes to a file in the VM, not stdout, so
   smolvm never has to transmit binary data over the exec channel.

3. Copy the compressed archive to the host with machine_cp.

4. Dockerfile switches to ADD rootfs.tar.gz / — Docker decompresses
   gzip tarballs automatically.
2026-06-23 16:53:41 -04:00
didericis-claude f2775101a0 fix(smolmachines): pipe tar stdout via PIPE not file fd
smolvm machine exec requires stdout to be a pipe, not a regular
file descriptor. Passing stdout=file caused smolvm to return
non-zero with no stderr (the error was silently swallowed or went
to the regular-file fd instead of reaching us).

Switch _snapshot_running_vm to a new _exec_tar_to_file helper that
uses Popen with stdout=PIPE and streams the tar to disk via
shutil.copyfileobj. A background thread drains stderr concurrently
to prevent deadlock when the stderr pipe buffer fills while we are
writing stdout data.
2026-06-23 16:53:41 -04:00
didericis-claude dd99c495f4 fix(smolmachines): use sh -c not sh -lc in exec_agent
The terminal-decoration wrapper script is invoked with sh -lc, which
sources login-shell init files (/etc/profile, ~/.profile) rather than
interactive-shell files (~/.zshrc). smolvm is typically installed via
homebrew whose PATH setup lands in ~/.zprofile or ~/.zshrc — not picked
up by sh -l — so pty_resize.py's Popen(["smolvm", ...]) raises
FileNotFoundError, pty_resize exits non-zero, and the trailing reset-
printf makes sh exit 0. The caller sees "session ended (exit 0)"
immediately with no agent output.

Use sh -c instead. The calling process (./cli.py) inherits the user's
interactive shell PATH where smolvm is present, confirmed by the
provision steps (machine_exec) succeeding before exec_agent is reached.
2026-06-23 16:53:41 -04:00
didericis-claude eb64a52ffa fix(smolmachines): commit via exec-tar instead of stop→pack
smolvm pack create --from-vm requires the VM to be stopped, and stopping
a smolmachines VM terminates any running interactive session.

Instead, mirror the macos-container approach: exec into the running VM as
root and stream the root filesystem via tar (smolvm machine exec -- tar),
build a Docker image from the archive, push to an ephemeral local registry,
and run smolvm pack create --image to produce the .smolmachine artifact.
The VM stays running throughout the commit.

Remove the stop-confirm prompt and machine_is_running check that were
added in the previous commit — neither is needed when we no longer stop.
2026-06-23 16:53:41 -04:00
didericis-claude d11e3940fa fix(smolmachines): stop VM before pack commit, with confirm prompt
smolvm pack create --from-vm requires the VM to be stopped. Add
machine_is_running() to smolvm.py (via machine ls --json state field),
and add the same confirm-stop flow to SmolmachinesFreezer that was
originally designed for macos-container: if running, prompt the user,
stop the VM, then pack. Already-stopped VMs are packed directly.
2026-06-23 16:53:41 -04:00
didericis-claude a32c0c7865 test: update macos-container tests for exec-tar commit approach
- Rename export test to reflect new exec-tar mechanism; update argv
  assertions to match the new `container exec ... tar` command shape
- Change mock stderr from str to bytes (subprocess.PIPE without text=True)
- Add type annotation to capture_freeze closure to satisfy pyright
2026-06-23 16:53:41 -04:00
didericis-claude ccb2956562 fix(macos-container): commit via exec-tar instead of stop→export
Apple Container removes containers when they stop, making the
stop-then-export flow impossible regardless of the --rm flag.

Replace `container export` (requires stopped container) with
`container exec --user root <name> tar --create ... --file=- --directory=/ .`
streamed to a temp file, then build the committed image from that archive
as before. The bottle stays running after commit, which is better UX.

Drop the stop-confirm prompt from MacosContainerFreezer since we no longer
need to stop the container at all.
2026-06-23 16:53:41 -04:00
didericis-claude c6362fda7b fix(macos-container): remove --rm from agent run so commit can export
container stop was removing the container immediately (due to --rm)
before container export could run. The force_remove_container teardown
callback on the ExitStack already handles cleanup on normal exit, so
--rm was redundant. Without it, the stopped container stays available
for container export to snapshot.
2026-06-23 16:53:41 -04:00
didericis-claude cb321f7ad4 refactor(freezer): drop Bottle from commit signature
Freezer._freeze only ever used bottle.name, which is always
f"bot-bottle-{agent.slug}". Remove the Bottle parameter from
commit() and _freeze(), derive the container name from agent.slug
directly in each subclass, and delete the _NamedBottle stub that
existed solely to paper over this.
2026-06-23 16:53:41 -04:00
didericis-claude 311cd46185 refactor(commit): introduce Freezer class hierarchy across backends
Adds a Freezer ABC (backend/freeze.py) that encapsulates the
stop-commit-mark-preserved flow for all backends, following the same
pattern as BottleBackend. Each backend gets its own Freezer subclass:

  DockerFreezer           — docker commit
  MacosContainerFreezer   — container export + image rebuild; prompts
                            to stop if the container is running
  SmolmachinesFreezer     — smolvm pack create --from-vm

The base class owns write_committed_image, mark_preserved, and the
resume hint. Subclasses implement _freeze() and optionally override
_export_hint() for migration instructions.

Freezer.commit(agent, bottle) is the primary entry point for use
within a live launch context. Freezer.commit_slug(slug) is a
convenience wrapper for cmd_commit, which no longer branches on
backend names itself.

get_freezer(backend_name) is the factory, analogous to
get_bottle_backend(). CommitCancelled is raised by MacosContainerFreezer
when the user declines the stop prompt; cmd_commit catches it and
returns 0.
2026-06-23 16:53:41 -04:00
didericis-claude 28335f453f fix(commit): stop running macos-container bottle before committing
`container export` requires the container to be stopped first. When a
running bottle is detected, prompt the user to confirm, stop the
container, then commit. Adds `container_is_running` and
`stop_container` helpers to the macos-container util.

Addresses #240 (comment)
2026-06-23 16:53:41 -04:00
didericis-claude a1aa8feb85 fix: correct Manifest/ManifestIndex usage and add missing type annotations in tests
- test_docker_launch_committed_image: replace Manifest.from_json_obj
  (nonexistent) with ManifestIndex.from_json_obj; pass manifest= arg
  to DockerBottlePlan constructor (required by BottlePlan base class)
- test_macos_container_launch: cast SimpleNamespace stubs to their
  expected types (BottleSpec, GitGatePlan, EgressPlan) in _build_plan;
  add str type annotations to fake_build parameter signatures
- test_macos_container_util: add str type annotations to fake_build_image
  parameter signatures
2026-06-23 16:53:41 -04:00
didericis cb3bb209d6 feat: support macos-container bottle commits 2026-06-23 16:53:41 -04:00
didericis-codex 6e73cc4d86 feat: support smolmachines bottle commit 2026-06-23 16:53:41 -04:00
didericis-claude 64fac71025 docs(prd): mark commit-bottle-state PRD as Active 2026-06-23 16:53:41 -04:00
didericis-claude f8ac22c316 feat(cli): add commit command to snapshot running bottle state
Adds `./cli.py commit [<slug>]` which runs `docker commit` on the
active agent container and stores the resulting image tag in per-bottle
state. The next `./cli.py resume <slug>` automatically boots from the
committed snapshot instead of rebuilding from the Dockerfile, preserving
all in-container state across restarts and migrations.

- bottle_state: add write_committed_image / read_committed_image helpers
- docker/util: add commit_container wrapper around `docker commit`
- docker/launch: check for a committed image before the Dockerfile build
  step; fall back to normal build if the image is absent from the daemon
- cli/commit: new command with interactive slug picker; errors clearly on
  non-Docker backends
- 50 new unit tests covering all paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 16:53:41 -04:00
Quality Badge Bot 9465857a99 chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors

[skip ci]
2026-06-23 20:46:17 +00:00
didericis-claude 200306f1cf refactor: export applicator singletons from egress_apply backends
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 21s
lint / lint (push) Successful in 1m44s
test / unit (push) Successful in 32s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Successful in 1m17s
Replace module-level apply_routes_change wrappers with a public
applicator singleton in each backend. Callers now work with the
EgressApplicator instance directly (applicator.apply_routes_change)
rather than through a function shim.
2026-06-23 20:39:05 +00:00
didericis-claude 77bdaf0a96 refactor: extract EgressApplicator base class shared between backends
lint / lint (push) Successful in 1m56s
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 20s
Pulls the duplicated apply_routes_change / validate_routes_content /
_routes_path logic into EgressApplicator (ABC) in backend/egress_apply.py.
DockerEgressApplicator and MacOSContainerEgressApplicator override the
single abstract _signal_bundle_reload method with their respective kill
commands. Module-level shims preserve the existing public API.
2026-06-23 20:33:43 +00:00
didericis 7e344bbb53 fix: add lowercase proxy env vars, route_to_yaml_dict, and richer tool descriptions
lint / lint (push) Successful in 1m51s
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 18s
- Set http_proxy/https_proxy (lowercase) alongside uppercase variants in smolmachines guest env for tools that only check lowercase
- Replace dataclasses.asdict with route_to_yaml_dict in /allowlist introspection so returned routes use YAML-schema-compatible keys
- Expand routes_yaml tool description in supervise_server to document all accepted route keys, making the round-trip from list-egress-routes to propose/apply explicit

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 16:13:07 -04:00
didericis-claude 5eb27cd9a8 fix: mount egress dir (not file) for docker and smolmachines backends
lint / lint (push) Successful in 1m37s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
Mirrors the fix already applied to the macos-container backend in
eb3e64e: bind-mount the parent egress directory instead of the
routes file itself, so the live routes update is visible inside the
running sidecar bundle when the host overwrites the file.
2026-06-23 09:05:44 +00:00
didericis-claude 5808d0b828 feat: add smolmachines/egress_apply proxying docker backend
lint / lint (push) Successful in 1m40s
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 16s
2026-06-23 06:53:56 +00:00
didericis-claude 7a991e1f5e refactor: split _signal_bundle_reload per backend, move macos egress to macos_container
lint / lint (push) Successful in 1m47s
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 19s
2026-06-23 05:57:07 +00:00
didericis-claude 5606797ac2 refactor: drop legacy routes path fallback from _routes_path
lint / lint (push) Successful in 1m37s
test / unit (pull_request) Failing after 29s
test / integration (pull_request) Successful in 18s
2026-06-23 05:48:50 +00:00
didericis-claude ebbb4053cf fix: add type annotations to fake_run in test_egress_apply
lint / lint (push) Successful in 1m40s
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 18s
2026-06-23 05:47:11 +00:00
didericis eb3e64ea8f fix(macos-container): mount live egress routes dir
lint / lint (push) Failing after 1m35s
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 16s
2026-06-23 01:39:29 -04:00
didericis 0ec1085238 fix(supervise): apply egress approvals
lint / lint (push) Failing after 1m34s
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 15s
2026-06-23 01:33:35 -04:00
didericis 4c39b45e34 fix(supervise): restore egress proposal tools
lint / lint (push) Successful in 1m35s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 16s
2026-06-23 01:24:28 -04:00
didericis-codex 3ea35ba5d2 fix: update codex supervise mcp registration
lint / lint (push) Successful in 1m54s
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 22s
2026-06-23 04:06:21 +00:00
Quality Badge Bot 7c6ab62e26 chore: update quality badges
- Pylint: 9.92/10
- Pyright: 0 errors

[skip ci]
2026-06-23 04:05:16 +00:00
didericis-claude da42740156 refactor(types): move loaded manifest from BottleSpec to BottlePlan
test / integration (pull_request) Successful in 21s
test / unit (pull_request) Successful in 49s
lint / lint (push) Successful in 2m15s
test / unit (push) Successful in 56s
test / integration (push) Successful in 27s
Update Quality Badges / update-badges (push) Successful in 2m37s
BottleSpec.manifest was ManifestIndex | Manifest — a union encoding
two lifecycle stages in one field. The union was unjustifiable:
it forced a type-narrowing workaround (loaded_manifest property)
on every consumer.

Clean split:
- BottleSpec.manifest: ManifestIndex (always; CLI-supplied intent)
- BottlePlan.manifest: Manifest (always; loaded by _validate())

_validate() returns the loaded Manifest directly. prepare() passes
it to _resolve_plan(), which stores it on the plan. All provisioner
code now reads plan.manifest.agent / plan.manifest.bottle — no
union, no asserts, no type: ignore.
2026-06-22 23:54:02 -04:00
didericis-claude 56ef71060a fix(types): add BottleSpec.loaded_manifest to satisfy pyright on union type
BottleSpec.manifest is ManifestIndex | Manifest (pre/post _validate()).
Downstream code always runs post-validate so it needs Manifest, but
pyright flagged every .agent/.bottle access. The new loaded_manifest
property asserts isinstance and returns Manifest, giving pyright a
narrowed type without scattering type: ignore everywhere.

Also remove unused Manifest imports from test files and annotate the
_index() helper in test_manifest_agent_git_user.
2026-06-22 23:54:02 -04:00
didericis-claude 294a6ed023 refactor(manifest): split Manifest into ManifestIndex + Manifest single-value type
Manifest now holds exactly one agent and one effective bottle (with
git_user overlay already applied). The old multi-agent/bottle
collection is renamed ManifestIndex. BottleSpec.manifest starts as
ManifestIndex from the CLI and becomes Manifest after _validate()
calls load_for_agent(); all provisioning code downstream reads
spec.manifest.agent / spec.manifest.bottle instead of indexing by name.
2026-06-22 23:54:02 -04:00
didericis-claude 468ab8c290 docs: clarify load_for_agent invariant in docstring 2026-06-22 23:54:02 -04:00
didericis-claude 2596c18954 fix: load_for_agent always returns single-agent manifest
Filter to exactly one agent and one bottle in both the lazy (md-dirs)
and eager (from_json_obj) paths so the returned manifest invariant
holds regardless of how the manifest was constructed.
2026-06-22 23:54:02 -04:00
didericis-claude 3ccd09ed0d refactor: scan filenames at resolve, parse only selected agent at preflight
Manifest.resolve() now returns an empty-dict manifest with only directory
paths recorded (home_md, cwd_md). No content is read from any .md file
until load_for_agent() is called for a specific agent at preflight.

- Manifest.from_md_dirs: scan-only, no frontmatter parsing
- Manifest.load_for_agent: parses the selected agent file and its bottle
  chain; works on eager (from_json_obj) manifests too by returning self
- Manifest.all_agent_names: scans filenames in lazy mode
- backend._validate: calls load_for_agent and propagates upgraded spec
- cli/info.py, cli/list.py, cli/start.py: use load_for_agent / all_agent_names
- manifest_extends.py: reverted to original (no partial-resolve helpers)
- manifest_loader.py: only scan_agent_names + load_bottle_chain_from_dir
- Tests updated to call load_for_agent before accessing agents/bottles;
  test_md_agent_repos_deferred renamed to test_md_agent_repos_fails_at_preflight
2026-06-22 23:54:02 -04:00
didericis-claude 996a260a98 fix: resolve pyright reportUnusedImport in manifest_extends
Import ManifestError at module level from manifest_util (no circular
dep) and remove the redundant local imports from function bodies that
were shadowing it. ManifestBottle retains its local import pattern to
avoid the circular manifest ↔ manifest_extends dependency.
2026-06-22 23:54:02 -04:00
didericis-claude 3375df3f52 feat: defer broken manifest parse errors to preflight
Broken bottle/agent files no longer block the agent selector or prevent
unrelated agents from loading. Per-file parse errors are collected in
`Manifest.broken_agents`; the CLI selector includes them via
`all_agent_names`, and the error surfaces only when the specific agent
is selected and launch is attempted (in `require_agent`/`bottle_for`).

Closes #236
2026-06-22 23:54:02 -04:00
Quality Badge Bot c9842ce831 chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors

[skip ci]
2026-06-23 03:46:30 +00:00
didericis-codex d314ccf455 test(macos-container): satisfy pyright mock typing
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 18s
lint / lint (push) Successful in 1m35s
test / unit (push) Successful in 31s
test / integration (push) Successful in 15s
Update Quality Badges / update-badges (push) Successful in 1m20s
2026-06-23 03:02:03 +00:00
didericis 31b29631b6 fix(macos-container): forward terminal capability env
lint / lint (push) Failing after 1m48s
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 20s
2026-06-22 22:57:16 -04:00
didericis-claude 1c11110da5 fix(macos-container): set host terminal to raw mode for container exec
lint / lint (push) Failing after 1m42s
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 19s
Apple's container exec --interactive --tty does not put the host
terminal into raw mode before starting its I/O relay.  In cooked
(canonical) mode the kernel line discipline buffers modifier-key
escape sequences — e.g. Shift+Enter in modifyOtherKeys mode generates
\x1b[13;2~ — until a carriage-return arrives, so they never reach
Claude Code inside the container.

Add pty_forward.py, a stdlib-only wrapper (modelled on the existing
smolmachines pty_resize.py) that sets the host terminal to raw mode
via tty.setraw(), spawns the container exec command, and restores the
original terminal attributes on exit.  Falls back to a bare
subprocess.run when stdin is not a TTY (piped invocations, CI) or
when termios operations fail.

Also retain the --env TERM=<host> forwarding from the previous commit:
without TERM inside the container session, Claude Code cannot determine
which modifier-key protocol to enable even with raw mode correctly set.

Non-TTY exec paths (bottle.exec, cp_in) are unaffected.
2026-06-23 02:30:46 +00:00
didericis-claude 25ca14a8a2 fix(macos-container): forward TERM env var in container exec --tty
lint / lint (push) Successful in 1m41s
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 21s
Without TERM, Claude Code inside the container cannot determine which
modifier-key protocol to enable (modifyOtherKeys / kitty). The inner
PTY session has no terminal-type context, so Shift+Enter and Enter
produce identical byte sequences (\r), making them indistinguishable.

Pass the host TERM via --env TERM=<value> on every container exec
--interactive --tty call, falling back to xterm-256color when TERM
is not set on the host. Non-TTY exec paths are unaffected.

Closes #245
2026-06-23 01:53:14 +00:00
Quality Badge Bot b5b7f15ef9 chore: update quality badges
- Pylint: 9.92/10
- Pyright: 0 errors

[skip ci]
2026-06-23 00:57:24 +00:00
didericis-claude 85e64b5134 feat: display agent name alongside label in terminal title and list output
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m42s
test / unit (push) Successful in 41s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m26s
When a label is set (e.g. "bob"), the display becomes "bob (claude-implementer)"
so the agent type is always visible. Affects all three backends (docker,
macos-container, smolmachines) and the `cli.py list active` output.

Closes #243
2026-06-23 00:28:16 +00:00
didericis-claude 1a5b6e25f8 fix: add type annotations to _modal stub for pyright
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 19s
lint / lint (push) Successful in 1m36s
test / unit (push) Successful in 30s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m19s
2026-06-22 19:30:53 +00:00
didericis-claude 54760964cf fix: label becomes the full slug; re-prompt on collision
lint / lint (push) Failing after 1m44s
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 17s
When a label is given it is now used verbatim as the slug (no random
suffix), so two launches with the same label collide by design.  The
CLI re-prompts via the TUI name modal with a disclaimer when the
candidate slug is already in use among running bottles.
2026-06-22 19:26:39 +00:00
didericis-claude e463670649 feat: use label as container slug prefix when provided
lint / lint (push) Successful in 1m45s
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 19s
When a user names a bottle via the TUI label field, that name is now
used as the slug prefix for the container identity instead of always
falling back to the agent name.
2026-06-22 19:16:53 +00:00
didericis-claude 6e6890ebd9 feat: remove cyan and white from color palette
lint / lint (push) Successful in 1m41s
test / unit (push) Successful in 33s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m16s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-22 19:09:22 +00:00
didericis-claude 609b3ed090 feat: drop dim colors, keep only bright variants renamed to base names
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 21s
lint / lint (push) Successful in 1m40s
test / unit (push) Successful in 34s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Successful in 1m19s
Remove the 8 non-bright and 1 bright-black colors from all color maps.
Rename the remaining 7 bright-* colors to their base names (e.g.
bright-green → green) so the palette is smaller and always vibrant.

Update _init_color_pairs in tui.py to always apply A_BOLD (all palette
entries are now bright variants), and fix all tests to match.
2026-06-22 18:59:51 +00:00
didericis 65faa40b9a refactor(backend): remove _validate_git_entries host key-file check
test / unit (pull_request) Successful in 37s
test / integration (pull_request) Successful in 18s
lint / lint (push) Successful in 1m39s
test / unit (push) Successful in 37s
test / integration (push) Successful in 18s
Update Quality Badges / update-badges (push) Successful in 1m38s
The git-gate copies the identity file at start time and surfaces a
clear failure then; the pre-launch presence check was redundant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 14:44:46 -04:00
didericis 9f97de115b fix(git-gate): skip host key-file check for gitea-provider repos
lint / lint (push) Successful in 1m39s
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 18s
_validate_git_entries was written for static keys (PRD 0008) and ran
os.path.isfile() on every entry's IdentityFile. gitea-provider repos
(PRD 0047/0048) create their deploy key at provision time, so
IdentityFile is empty at parse — tripping the check with an empty path
("git upstream key file not found for '<name>': "). Gate the host-file
check on the static provider; gitea entries have nothing to verify here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 13:21:11 -04:00
didericis 8f21f4df19 refactor(manifest-extends): thread resolved repos through recursion
lint / lint (push) Successful in 1m32s
test / unit (pull_request) Successful in 28s
test / integration (pull_request) Successful in 16s
Replace the lossy _entry_to_raw round-trip with a repos_cache threaded
alongside the ManifestBottle cache in _resolve_one_bottle. Each bottle's
effective git-gate.repos is stored as raw dicts keyed by name, so a child
field-merges directly against its parent's raw repos instead of
reconstructing them from parsed ManifestGitEntry objects.

_resolve_repos_raw now owns the union/clear/inherit semantics on plain
dicts; _merge_bottles just injects the precomputed merged set before
parsing. Drops _entry_to_raw entirely, removing the maintenance hazard
where a new ManifestGitEntry field would silently vanish from inherited
repos.

Addresses review feedback on #238.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NgEFTXcWZjA8n7ntq2zHQQ
2026-06-19 22:53:27 -04:00
didericis-claude ff7a52c1d2 refactor(manifest-extends): simplify git-gate repo merge to union + dict unpack
lint / lint (push) Failing after 1m31s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
Replace the bespoke _pre_merge_git_repos loop and _merge_git_remotes
with a single _merge_git_repos_raw that does a name-keyed union merge
at the raw dict level: build parent_repos from _entry_to_raw, then
for each name in set(child) | set(parent) produce {**parent.get(n,{}),
**child.get(n,{})}. child.git after from_dict already has the full
merged set, so _merge_git_remotes is no longer needed.
2026-06-20 02:25:09 +00:00
didericis-claude 4ed6b84863 feat(manifest-extends): field-merge same-name git-gate repos on extends
lint / lint (push) Successful in 1m34s
test / unit (pull_request) Successful in 27s
test / integration (pull_request) Successful in 15s
When a child bottle declares a git-gate repo with the same name as a
parent repo, merge field-by-field (child wins, parent provides fallback)
instead of letting the child entry silently replace the parent entry.
This lets a child override only `key:` without repeating `url:` and
`host_key:`. Change the merge key in _merge_git_remotes from UpstreamHost
to Name, which is the natural unique identity for a repo entry.

Closes #237
2026-06-20 02:02:12 +00:00
didericis-claude 7a124d7d25 refactor: make static the default branch in _parse_key_config
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 16s
lint / lint (push) Successful in 1m31s
test / unit (push) Successful in 28s
test / integration (push) Successful in 15s
Update Quality Badges / update-badges (push) Successful in 1m10s
2026-06-19 22:25:14 +00:00
didericis-claude f00c567469 rename: provisioner_token -> forge_token_env
lint / lint (push) Successful in 2m6s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
2026-06-19 22:21:37 +00:00
didericis-claude 6f0e5b4589 refactor: extract _resolve_identity_file from prepare loop
lint / lint (push) Successful in 1m33s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
2026-06-19 22:14:15 +00:00
didericis-claude 5da4d05bf2 fix: remove unused Optional import flagged by pyright
lint / lint (push) Successful in 1m33s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
2026-06-19 22:09:52 +00:00
didericis-claude 1a8718ca9d refactor: unify identity/provisioned_key into key block
lint / lint (push) Failing after 1m45s
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 17s
Replace the two mutually-exclusive repo keys (identity and
provisioned_key) with a single required key block. key.provider
is "static" (path to host SSH key) or "gitea" (deploy-key lifecycle
via provisioner_token env var, replacing token_env).

Internal fields: ManifestProvisionedKeyConfig → ManifestKeyConfig;
ProvisionedKey field removed from ManifestGitEntry; Key field added.
git_gate.py checks entry.Key.provider == "gitea" instead of
entry.ProvisionedKey is not None.
2026-06-19 22:01:43 +00:00
didericis-claude c1c225aa05 docs(gitea-provisioner): document required GITEA_DEPLOY_TOKEN permissions
lint / lint (push) Successful in 1m46s
test / unit (push) Successful in 34s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m21s
2026-06-11 03:43:13 +00:00
didericis dc7c10d6fe fix(macos-container): use correct system status probe in preflight
lint / lint (push) Successful in 1m41s
test / unit (push) Successful in 34s
test / integration (push) Successful in 21s
Update Quality Badges / update-badges (push) Successful in 1m23s
`container system info` is not a valid subcommand and always returned
non-zero, causing a false-positive on the service check. Switch to
`container system status` which is the correct command.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 23:31:32 -04:00
Quality Badge Bot a827b0841e chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors

[skip ci]
2026-06-11 03:28:32 +00:00
didericis a9c93ea9df fix(macos-container): preflight check for container system service
lint / lint (push) Successful in 1m43s
test / unit (push) Successful in 34s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Successful in 1m27s
Fail early with a clear message when the Apple Container system service
isn't running, instead of surfacing an opaque XPC connection error mid-build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 23:24:09 -04:00
didericis bb69af31f8 chore(claude): bump claude-code to 2.1.170
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 22:44:46 -04:00
didericis 7644da4280 docs: add Apple Container transparent egress spike 2026-06-10 22:36:55 -04:00
didericis 13e4af421d docs: add Apple Container networking spike 2026-06-10 22:36:55 -04:00
github-actions[bot] f2d5307573 ci(prd): assign sequential numbers to new PRDs 2026-06-11 02:36:07 +00:00
didericis bc9a22b46a fix(macos-container): support git-gate launch
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 20s
lint / lint (push) Successful in 1m45s
prd-number / assign-numbers (push) Successful in 25s
test / unit (push) Successful in 32s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Failing after 1m23s
2026-06-10 22:25:00 -04:00
didericis 932e71c0bf fix(macos-container): make backend the macos default 2026-06-10 22:25:00 -04:00
didericis d3b0b330aa fix(macos-container): preserve working builder dns 2026-06-10 22:25:00 -04:00
didericis 5e927bcd13 fix(macos-container): start builder with dns 2026-06-10 22:25:00 -04:00
didericis 890a146413 test(macos-container): add launch integration smoke 2026-06-10 22:25:00 -04:00
didericis afdf0779a1 feat(macos-container): launch explicit-proxy bottles 2026-06-10 22:25:00 -04:00
didericis-codex eb7cae1fea docs: link macos container prd to review comment 2026-06-10 22:25:00 -04:00
didericis-codex fe82dc7f2b feat: add macos container backend scaffold 2026-06-10 22:25:00 -04:00
didericis-claude b00b0ba4aa fix(git-gate): forward force push as +refspec to upstream
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 22s
lint / lint (push) Successful in 1m47s
test / unit (push) Successful in 34s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m21s
When $old != zero and $new is not a descendant of $old (detected via
git merge-base --is-ancestor), the hook now forwards +$new:$ref so the
upstream accepts the force push instead of rejecting it as a
non-fast-forward.

Closes #233
2026-06-11 02:17:27 +00:00
didericis-codex 3f04567290 egress: require opt-in for HTTPS git fetch
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 27s
lint / lint (push) Successful in 1m53s
test / unit (push) Successful in 41s
test / integration (push) Successful in 23s
Update Quality Badges / update-badges (push) Successful in 1m35s
2026-06-10 07:00:01 +00:00
didericis-codex acb9cd67c6 fix(git-gate): forward push options
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m28s
test / unit (push) Successful in 29s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m7s
2026-06-10 02:36:47 -04:00
didericis-codex d90ab7e646 ci: enforce pylint threshold
lint / lint (push) Successful in 1m37s
test / unit (push) Successful in 34s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m13s
2026-06-10 06:30:03 +00:00
didericis-codex 8ea90adcaf fix: raise git http body cap 2026-06-10 06:29:46 +00:00
Quality Badge Bot de803e1e76 chore: update quality badges
- Pylint: 9.94/10
- Pyright: 0 errors

[skip ci]
2026-06-10 05:34:38 +00:00
didericis-codex 019efab804 fix: narrow pi numeric settings types
test / unit (pull_request) Successful in 49s
test / integration (pull_request) Successful in 27s
lint / lint (push) Successful in 1m37s
test / unit (push) Successful in 31s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Successful in 1m20s
2026-06-10 01:13:00 -04:00
didericis-codex 957d37f51f fix: merge egress routes across extends 2026-06-10 01:13:00 -04:00
Quality Badge Bot 8e084262a0 chore: update quality badges
- Pylint: 9.94/10
- Pyright: 4 errors

[skip ci]
2026-06-10 04:07:25 +00:00
didericis 504144eb9c fix(pi): prepare runtime state and agent workdir
lint / lint (push) Failing after 1m58s
test / unit (push) Successful in 41s
test / integration (push) Successful in 24s
Update Quality Badges / update-badges (push) Successful in 1m27s
2026-06-10 00:02:28 -04:00
didericis 86374ab293 fix(pi): select configured startup models
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m37s
test / unit (push) Successful in 33s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m6s
2026-06-09 06:57:33 -04:00
didericis 199edb228c feat(pi): add fd and ripgrep to image
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 18s
2026-06-09 06:26:19 -04:00
didericis 598a20a3f0 fix(pi): keep interactive sessions open
lint / lint (push) Successful in 1m29s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
2026-06-09 06:00:40 -04:00
didericis c8b5ba3812 feat(pi): support egress injected api keys
lint / lint (push) Successful in 1m38s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 17s
2026-06-09 05:56:39 -04:00
didericis-codex 5ea9fda69b docs: activate pi provider prd
lint / lint (push) Successful in 1m43s
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 18s
2026-06-09 08:32:09 +00:00
didericis-codex 4f7cfc0418 feat: add pi agent provider 2026-06-09 08:31:48 +00:00
didericis-codex 1f38a96561 docs: add pi provider prd 2026-06-09 08:23:00 +00:00
Quality Badge Bot 660b9b3810 chore: update quality badges
- Pylint: 9.94/10
- Pyright: 0 errors

[skip ci]
2026-06-09 05:46:00 +00:00
didericis 328069809b fix(pyright): remove unused shlex imports from bottle backends
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 15s
lint / lint (push) Successful in 1m35s
test / unit (push) Successful in 31s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m30s
shlex is now only used in terminal.py after the exec_shell_script refactor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:48:46 -04:00
didericis b1551045dc feat(terminal): tint terminal background per agent color
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 17s
Add backend-agnostic terminal color support via OSC escape sequences:
- New backend/terminal.py with palette_printf() and exec_shell_script()
  shared by both Docker and smolmachines bottle backends
- Emits OSC 4 (indexed palette) + OSC 11 (default background tint)
  before launching; resets both on agent exit via OSC 104/111
- OSC 11 background tint is visible even when the TUI uses true/24-bit
  colors (which bypass the palette), as Codex does for its chrome
- Fix Codex [tui] config: status_line=["model-with-reasoning"],
  theme="ansi" (dark-ansi and cwd/directory were invalid identifiers)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:47:55 -04:00
didericis d02226aab9 feat: forward agent style via native CLI config and terminal title
Replace prompt-injection for display identity with native UI wiring:
- Claude: writes a statusline shell script + custom theme JSON, wired up
  via settings.json so label/color show in the status bar and theme
- Codex: writes [tui] block into codex-config.toml (status_line,
  terminal_title, dark-ansi theme)
- Both backends set the terminal title via ANSI OSC 0 escape before
  exec-ing the agent when a label is present

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:47:55 -04:00
didericis-codex 39811c9b32 feat: forward agent display identity to prompts 2026-06-09 00:47:55 -04:00
github-actions[bot] f7f161e60f ci(prd): assign sequential numbers to new PRDs 2026-06-09 03:37:10 +00:00
didericis-codex e6040fc824 fix(start): skip backend selector
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 24s
lint / lint (push) Successful in 1m47s
prd-number / assign-numbers (push) Successful in 29s
test / unit (push) Successful in 38s
test / integration (push) Successful in 27s
Update Quality Badges / update-badges (push) Failing after 1m20s
2026-06-09 03:31:26 +00:00
didericis-codex 17fc44d0d8 complete(prd): mark smolmachines default active
lint / lint (push) Successful in 1m46s
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 22s
2026-06-09 03:27:58 +00:00
didericis-codex 1bebb7467f feat(backend): default to smolmachines 2026-06-09 03:27:31 +00:00
didericis cc1d986a74 test: fix smolmachines proxy assertions
lint / lint (push) Successful in 1m52s
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 25s
2026-06-08 23:22:47 -04:00
didericis-codex fabcd026af test(smolmachines): verify TSI egress proxy path
lint / lint (push) Successful in 1m47s
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 23s
2026-06-09 03:14:58 +00:00
didericis aff042855a ci(prd): rename PRD to prd-new placeholder per new convention
lint / lint (push) Successful in 1m47s
2026-06-08 23:10:09 -04:00
didericis 39b0c4f720 docs(prd): renumber PRD 0055 → 0058 (0055 slot taken by extended-outbound-scan) 2026-06-08 23:10:09 -04:00
didericis 43a5700ae6 docs(prd): PRD 0055 - promote smolmachines to default backend 2026-06-08 23:10:09 -04:00
didericis-codex 7acdabaf96 test: narrow metadata assertions for pyright 2026-06-08 23:05:14 -04:00
didericis-codex dfd2d5f620 fix: restore runtime workspace provisioning 2026-06-08 23:05:14 -04:00
didericis-codex f24e2857ab fix: restore backend prepare wiring 2026-06-08 23:05:14 -04:00
didericis-codex d38432f640 fix: resolve pyright strict errors 2026-06-08 23:05:14 -04:00
didericis 4e570e3e2b fix(egress): ignore stripped auth header in DLP scan 2026-06-08 23:05:14 -04:00
didericis-claude a64e3170cd refactor: make AgentProvisionPlan the source of truth for instance_name, prompt_file, image, dockerfile, guest_home
Drop the parallel fields passed through prepare() → _resolve_plan and
read everything from agent_provision instead. The provider plugin now
declares its own guest_home (so the backend stops hardcoding
"/home/node") and the wrapper that builds the provision plan accepts
instance_name and prompt_file, which providers store on the plan.

DockerBottlePlan and SmolmachinesBottlePlan expose container_name /
machine_name, image / agent_image, dockerfile_path /
agent_dockerfile_path, and prompt_file as properties that delegate to
agent_provision so existing call sites keep working unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 23:05:14 -04:00
didericis-claude 4da4babcf4 fix: fall back to provider's bundled Dockerfile when manifest doesn't override
BottleBackend.prepare was calling resolve_manifest_dockerfile("", spec)
for every bottle where the manifest did not set agent_provider.dockerfile.
That resolves an empty string against user_cwd, returning the cwd
itself — which docker then tried to read as a Dockerfile, giving
"is a directory" errors during image build.

When the manifest doesn't override, use the provider plugin's bundled
Dockerfile path (next to its agent_provider.py module) — mirroring
the pre-refactor behavior.
2026-06-08 23:05:14 -04:00
didericis-claude 384e496a1b fix: thread slug + resolved_env from prepare to each backend's _resolve_plan
BottleBackend.prepare computed slug and resolved_env but never passed
them to _resolve_plan. The concrete docker/smolmachines _resolve_plan
methods still had the old (spec, *, stage_dir) signature too, so
prepare's kwargs blew up with "unexpected keyword argument
'instance_name'" the moment cli.py start was invoked.

Update the abstract _resolve_plan signature and both backend
implementations to accept the full kwarg set prepare passes, and
forward to resolve_plan.resolve_plan() with everything.
2026-06-08 23:05:14 -04:00
didericis-claude b38c6110f2 chore: comment out workspace + capability_apply, fix circular imports
The recent refactor partially removed workspace planning and
capability-apply logic. This commit finishes the cleanup so the
test suite imports cleanly:

- Comment out workspace_plan field/property on BottlePlan and the
  provision_workspace dispatch.
- Comment out workspace usages in docker.util (build_image_with_cwd),
  smolmachines.provision.workspace, agent_provider.provision_git,
  smolmachines.backend.
- Comment out capability_apply imports in cli.start and cli.supervise;
  add a local CapabilityApplyError placeholder so the supervise CLI
  module still imports.
- Break the bottle_state → backend.docker → backend circular import
  by lazy-loading docker_mod inside bottle_identity, and by moving the
  resolve_common import inside BottleBackend.prepare.
- Delete tests for workspace and capability_apply (unit + integration).
- Update test fixtures to drop removed kwargs (container_name_pinned,
  derived_image, env_file, workspace_plan, agent_image_ref) from
  DockerBottlePlan / SmolmachinesBottlePlan constructors.
- Delete the obsolete test_smolmachines_prepare.py (tested the old
  resolve_plan signature; the shared prepare flow now lives in
  BottleBackend.prepare).
- Adjust test_supervise.py for the new Supervise.prepare signature
  (dockerfile_content arg removed).

925 → 897 tests, all passing.
2026-06-08 23:05:14 -04:00
didericis 74efb1c143 chore: sketch out desired refactor
Manual refactor into the rough shape we want/how we want the
resolve_plan logic to be consolidated. Needs subsequent fixes.
2026-06-08 23:05:14 -04:00
didericis-claude f23b2b9683 refactor: move guest_home onto AgentProvisionPlan as source of truth
guest_home is now a field on AgentProvisionPlan (set by each provider's
provision_plan() method). BottlePlan.guest_home becomes a read-only
property delegating to agent_provision.guest_home so existing callers
(provision_git, provision_skills, provision_prompt) are unchanged.

Both resolve_plan.py files drop guest_home from the plan constructor
call; the local variable still exists as an intermediary for the
workspace_plan call that precedes agent_provision_plan.
2026-06-08 23:05:14 -04:00
didericis-claude 423003aa05 refactor: extract shared resolve_plan helpers into backend/resolve_common.py
Both docker and smolmachines resolve_plan.py duplicated: slug minting,
metadata writing, agent state dir setup, git gate / egress / supervise
preparation, env_vars merge, and manifest dockerfile path resolution.

These are now consolidated in bot_bottle/backend/resolve_common.py.
Each backend's resolve_plan retains only its own logic (container name
resolution + env-file for docker; subnet allocation + guest_env build
for smolmachines).
2026-06-08 23:05:14 -04:00
didericis-claude af82f2ba20 refactor: move bottle_state.py to top-level bot_bottle package
Both docker and smolmachines backends use bottle state helpers.
Moving to bot_bottle/ makes the sharing explicit and removes the
cross-backend dependency (smolmachines importing from ..docker).

All callers updated: docker backend, smolmachines backend, cli
modules, and tests.
2026-06-08 23:05:14 -04:00
didericis-claude fe8e15d211 refactor: rename prepare.py → resolve_plan.py in both backends 2026-06-08 23:05:14 -04:00
didericis-claude b098556757 refactor: prefix all manifest data classes with Manifest
Avoids name collisions with same-named runtime/plugin classes
(e.g. manifest AgentProvider vs plugin AgentProvider ABC,
manifest EgressRoute vs runtime EgressRoute). Renamed:

  AgentProvider        → ManifestAgentProvider   (manifest_agent.py)
  Agent                → ManifestAgent            (manifest_agent.py)
  EgressRoute          → ManifestEgressRoute      (manifest_egress.py)
  PathMatch            → ManifestPathMatch        (manifest_egress.py)
  HeaderMatch          → ManifestHeaderMatch      (manifest_egress.py)
  MatchEntry           → ManifestMatchEntry       (manifest_egress.py)
  EgressConfig         → ManifestEgressConfig     (manifest_egress.py)
  Bottle               → ManifestBottle           (manifest.py)
  ProvisionedKeyConfig → ManifestProvisionedKeyConfig (manifest_git.py)
  GitEntry             → ManifestGitEntry         (manifest_git.py)
  GitUser              → ManifestGitUser          (manifest_git.py)
2026-06-08 23:05:14 -04:00
didericis-claude 5c5f277d6d refactor: set image/dockerfile from provider default first, override after
Since every provider always has a dockerfile, establish the default
image and dockerfile_path from the provider up front and override for
per-bottle or manifest-specified cases. Removes the image_default
intermediate variable and the trailing else branch.
2026-06-08 23:05:14 -04:00
didericis-claude 2fa5229695 refactor: AgentProvider.dockerfile always returns Path, never None
The convention is that every provider declares a Dockerfile location;
callers that care whether the file actually exists check .is_file().
Drops all `is not None` guards on the property result.
2026-06-08 23:05:14 -04:00
didericis-claude c3caa3ea94 refactor: remove BOT_BOTTLE_IMAGE env override
Unused in tests, docs, or examples. Can be added back if/when merited.
2026-06-08 23:05:14 -04:00
didericis-claude ee0607f022 refactor: replace runtime.dockerfile with AgentProvider.dockerfile property
Drop the `dockerfile` field from `AgentProviderRuntime` and replace it
with a convention-based `dockerfile` property on `AgentProvider`: the
base class looks for a `Dockerfile` file next to the provider's own
`agent_provider.py` module (via `inspect.getfile`), returning its path
or None. Built-in providers inherit the default automatically; custom
user providers work the same way by dropping a Dockerfile next to their
plugin file; any provider needing a non-standard path can override.

All callers (`docker/prepare.py`, `smolmachines/prepare.py`,
`capability_apply.py`) now resolve the provider object once and call
`.dockerfile` directly instead of reading `runtime.dockerfile`.
2026-06-08 23:05:14 -04:00
didericis-claude afe5d43a9a refactor: move agent Dockerfiles into their contrib directories
Dockerfile.claude and Dockerfile.codex move from the repo root into
bot_bottle/contrib/claude/Dockerfile and bot_bottle/contrib/codex/Dockerfile
respectively, so all per-provider assets live alongside the provider code.

Closes #215
2026-06-08 23:05:14 -04:00
didericis dd332a5759 chore: Replace die with YamlSubsetError 2026-06-08 23:05:11 -04:00
177 changed files with 16256 additions and 3430 deletions
+9
View File
@@ -0,0 +1,9 @@
[run]
branch = True
source = .
[report]
omit =
bot_bottle/egress_addon.py
bot_bottle/cli/tui.py
tests/*
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
- name: Run pylint
run: |
# Run pylint on all Python files in the repo
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0 || true
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0
- name: Run pyright
run: |
+7 -1
View File
@@ -39,8 +39,14 @@ jobs:
with:
python-version: "3.12"
- name: Install dev requirements
run: python3 -m pip install -r requirements-dev.txt
- name: Run unit tests
run: python3 -m unittest discover -t . -s tests/unit -v
run: python3 -m coverage run -m unittest discover -t . -s tests/unit -v
- name: Report unit coverage
run: python3 -m coverage report -m
integration:
runs-on: ubuntu-latest
+15 -2
View File
@@ -8,6 +8,7 @@ on:
- '**.py'
- '.pylintrc'
- 'pyrightconfig.json'
- '.coveragerc'
workflow_dispatch:
jobs:
@@ -45,10 +46,19 @@ jobs:
echo "errors=$ERRORS" >> $GITHUB_OUTPUT
echo "Pyright errors: $ERRORS"
- name: Run coverage and extract percentage
id: coverage
run: |
python -m coverage run -m unittest discover -t . -s tests/unit > /dev/null 2>&1 || true
PERCENT=$(python -m coverage report 2>/dev/null | grep '^TOTAL' | grep -oP '\d+(?=%)' | tail -1)
echo "percent=$PERCENT" >> $GITHUB_OUTPUT
echo "Coverage: $PERCENT%"
- name: Update badges in README
run: |
PYLINT_SCORE="${{ steps.pylint.outputs.score }}"
PYRIGHT_ERRORS="${{ steps.pyright.outputs.errors }}"
COVERAGE_PERCENT="${{ steps.coverage.outputs.percent }}"
PYLINT_SCORE_ENCODED=$(echo "$PYLINT_SCORE" | sed 's|/|%2F|g')
@@ -58,9 +68,12 @@ jobs:
if [ -n "$PYRIGHT_ERRORS" ]; then
sed -i "s|/badge/pyright-[^)]*|/badge/pyright-${PYRIGHT_ERRORS}%20errors-brightgreen|" README.md
fi
if [ -n "$COVERAGE_PERCENT" ]; then
sed -i "s|/badge/coverage-[^)]*|/badge/coverage-${COVERAGE_PERCENT}%25-brightgreen|" README.md
fi
echo "Updated badges:"
grep -E "pylint|pyright" README.md | head -2
grep -E "pylint|pyright|coverage" README.md | head -3
- name: Commit and push badge updates
run: |
@@ -73,7 +86,7 @@ jobs:
else
echo "Badge changes detected, committing..."
git add README.md
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n\n'"[skip ci]"
MSG="chore: update quality badges"$'\n\n'"- Pylint: ${{ steps.pylint.outputs.score }}"$'\n'"- Pyright: ${{ steps.pyright.outputs.errors }} errors"$'\n'"- Coverage: ${{ steps.coverage.outputs.percent }}%"$'\n\n'"[skip ci]"
git commit -m "$MSG"
git push
fi
+1
View File
@@ -22,3 +22,4 @@ venv/
.pytest_cache/
.mypy_cache/
.ruff_cache/
.coverage
+13 -6
View File
@@ -2,11 +2,18 @@
## What this is
bot-bottle spins up an isolated container for running AI coding agents with a
curated set of skills and env vars. The point is to run agents with broad
permissions inside a sandbox, so a misbehaving agent cannot reach the host.
A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the container lifecycle and the copying of skills and env vars into it.
bot-bottle spins up an isolated backend runtime for running AI coding agents
with a curated set of skills and env vars. The point is to run agents with
broad permissions inside a sandbox, so a misbehaving agent cannot reach the
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the runtime lifecycle and the copying of skills and env vars into it.
The default backend on compatible macOS hosts is macos-container:
agents and sidecar bundles run through Apple's `container` CLI without
requiring Docker. The smolmachines backend remains available with
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
or `--backend=docker`.
## Goals
@@ -17,7 +24,7 @@ the container lifecycle and the copying of skills and env vars into it.
## Non-goals
- Communicating between agents directly
- Self hosted VMs (v1 uses local Docker containers, not VMs)
- Removing the Docker backend
- Advanced agent auditing (lean on git history for auditing)
## Repository layout
+62 -7
View File
@@ -5,8 +5,9 @@
# bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![pylint](https://img.shields.io/badge/pylint-9.95%2F10-brightgreen)](https://github.com/PyCQA/pylint)
[![pylint](https://img.shields.io/badge/pylint-9.93%2F10-brightgreen)](https://github.com/PyCQA/pylint)
[![pyright](https://img.shields.io/badge/pyright-0%20errors-brightgreen)](https://github.com/microsoft/pyright)
[![coverage](https://img.shields.io/badge/coverage-79%25-brightgreen)](https://coverage.readthedocs.io/)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -14,20 +15,29 @@
## Features
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default.
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other.
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker.
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. Runs on macOS (Hypervisor.framework) and Linux (KVM, `/dev/kvm`).
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
## Architecture
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles egress + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
The Docker topology looks like this:
```
host ( ./cli.py )
@@ -62,7 +72,25 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
## Quickstart
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus `smolvm` (macOS or Linux). The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
### smolmachines on Linux
The smolmachines backend runs on Linux as well as macOS. On Linux, `smolvm`/libkrun use KVM, so the host needs:
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
- **`smolvm`** on `PATH`: `curl -sSL https://smolmachines.com/install.sh | sh`.
- **Docker** for the sidecar bundle and image build, same as macOS.
Per-bottle isolation works the same as macOS without any `ifconfig`/sudo step — all of `127.0.0.0/8` is already loopback on Linux, so each bottle's sidecar bundle is published on its own `127.0.0.<N>` and TSI's allowlist is scoped to that `/32`.
```sh
BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>
```
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. If you run bottles from a Gitea Actions runner, use a `host`-label runner so Docker, `smolvm`, and `/dev/kvm` are all reachable from the job. `smolvm` isn't in nixpkgs — install the release binary (pin the version) and put it on the runner's `PATH`.
```sh
./cli.py start <agent> # builds the image on first run, drops you into claude
@@ -96,8 +124,15 @@ egress:
routes:
- host: gitea.dideric.is
auth:
scheme: token
scheme: token # Bearer | token
token_ref: BOT_BOTTLE_GITEA_TOKEN
matches: # optional — restrict to specific paths/methods/headers
- paths:
- {type: prefix, value: /api/v1/}
methods: [GET, POST, PATCH, DELETE]
dlp: # optional — per-route detector overrides (default: all on)
outbound_detectors: [token_patterns, known_secrets]
inbound_detectors: false # disable response scanning for this host
---
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
@@ -116,6 +151,26 @@ skills:
You help maintain Gitea-hosted projects.
````
**Egress route fields:**
| Field | Required | Description |
|---|---|---|
| `host` | yes | Hostname to allowlist. One entry per host. |
| `role` | no | Reserved for future use. The key is recognised but any value is currently rejected at load. Provider auth routes (e.g. Claude's `api.anthropic.com`) are injected automatically from `agent_provider.auth_token`, not via `role`. |
| `auth.scheme` | when `auth` present | `Bearer` or `token`. Injected by the proxy; the agent never sees the value. |
| `auth.token_ref` | when `auth` present | Env-var name holding the secret on the host. |
| `matches` | no | Array of `{paths, methods, headers}` filters. A request must match at least one entry (if any are given) to be forwarded. |
| `matches[].paths` | no | Array of `{type, value}`. `type` is `prefix` (default), `exact`, or `regex`. |
| `matches[].methods` | no | Array of HTTP method strings, e.g. `[GET, POST]`. |
| `matches[].headers` | no | Array of `{name, value, type}`. `type` is `exact` (default) or `regex`. |
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default for manifest routes — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). Agent-provider routes (e.g. `api.anthropic.com`) default to `redact`. |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Trademarks
+62 -26
View File
@@ -20,6 +20,7 @@ Per PRD 0050 the per-provider implementations live under
from __future__ import annotations
import importlib.util
import inspect
import os
import shlex
import tempfile
@@ -37,13 +38,19 @@ if TYPE_CHECKING:
PROVIDER_CLAUDE = "claude"
PROVIDER_CODEX = "codex"
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
PROVIDER_PI = "pi"
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_PI})
# Hosts that egress injects the host ChatGPT bearer on when Codex
# forward_host_credentials is enabled. Pipelock must pass these through
# (no TLS MITM) or its header DLP blocks the injected JWT.
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
PromptMode = Literal["append_file", "read_prompt_file"]
PromptMode = Literal[
"append_file",
"read_prompt_file",
"print_read_prompt_file",
"append_system_prompt",
]
@dataclass(frozen=True)
@@ -51,11 +58,9 @@ class AgentProviderRuntime:
template: str
command: str
image: str
dockerfile: str
prompt_mode: PromptMode
bypass_args: tuple[str, ...]
resume_args: tuple[str, ...]
remote_control_args: tuple[str, ...]
@dataclass(frozen=True)
@@ -103,7 +108,12 @@ class AgentProvisionPlan:
prompt_mode: PromptMode
image: str
dockerfile: str
guest_home: str
instance_name: str
prompt_file: Path
guest_env: dict[str, str]
has_prompt: bool = False
startup_args: tuple[str, ...] = ()
env_vars: dict[str, str] = field(default_factory=dict)
dirs: tuple[AgentProvisionDir, ...] = ()
files: tuple[AgentProvisionFile, ...] = ()
@@ -127,13 +137,31 @@ class AgentProvider(ABC):
"""The static command / image / prompt-mode table for this
template."""
@property
def guest_home(self) -> str:
"""In-guest home directory for the agent user. Defaults to
`/home/node` to match the Debian-based bot-bottle-* images
(USER node). Override for plugins whose image runs as a
different user."""
return "/home/node"
@property
def dockerfile(self) -> Path:
"""Path to the provider's Dockerfile.
Default: the `Dockerfile` file next to this provider's
`agent_provider.py` module. Override to point at a non-standard
path."""
return Path(inspect.getfile(type(self))).parent / "Dockerfile"
@abstractmethod
def provision_plan(
self,
*,
dockerfile: str,
state_dir: Path,
guest_home: str,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
@@ -141,6 +169,7 @@ class AgentProvider(ABC):
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
"""Build the declarative AgentProvisionPlan for one launch.
Backends call this during `prepare` and consume the result as
@@ -205,25 +234,12 @@ class AgentProvider(ABC):
def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None:
"""Configure git inside the agent container.
Default: Debian/node — copies .git when --cwd is set, writes the
git-gate insteadOf gitconfig, sets user.name/email as node.
Override for images that run as a different user or use a
non-standard home directory."""
Default: Debian/node — writes the git-gate insteadOf gitconfig
and sets user.name/email as node. Workspace copy runs through
BottleBackend.provision_workspace against the running bottle."""
from .log import info
workspace = plan.workspace_plan
if workspace.enabled and workspace.copy_git and workspace.has_host_git_dir:
guest_workspace_git = f"{workspace.guest_path}/.git"
host_git = str(workspace.host_path / ".git")
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
bottle.cp_in(host_git, guest_workspace_git)
bottle.exec(
f"chown -R {shlex.quote(workspace.owner)} "
f"{shlex.quote(guest_workspace_git)}",
user="root",
)
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
manifest_bottle = plan.manifest.bottle
if manifest_bottle.git:
from .git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
gate_host = getattr(plan, "git_gate_insteadof_host", GIT_GATE_HOSTNAME)
@@ -310,6 +326,9 @@ def get_provider(template: str) -> AgentProvider:
if template == PROVIDER_CODEX:
from .contrib.codex.agent_provider import CodexAgentProvider
return CodexAgentProvider()
if template == PROVIDER_PI:
from .contrib.pi.agent_provider import PiAgentProvider
return PiAgentProvider()
raise ValueError(f"unknown agent provider template: {template!r}")
@@ -317,12 +336,13 @@ def runtime_for(template: str) -> AgentProviderRuntime:
return get_provider(template).runtime
def agent_provision_plan(
def build_agent_provision_plan(
*,
template: str,
dockerfile: str,
state_dir: Path,
guest_home: str,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
@@ -330,13 +350,15 @@ def agent_provision_plan(
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
"""Back-compat shim — `prepare` callers stay the same; the work
now lives on the provider plugin."""
return get_provider(template).provision_plan(
dockerfile=dockerfile,
state_dir=state_dir,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
guest_env=guest_env,
auth_token=auth_token,
forward_host_credentials=forward_host_credentials,
@@ -344,9 +366,19 @@ def agent_provision_plan(
trusted_project_path=trusted_project_path,
label=label,
color=color,
provider_settings=provider_settings,
)
def provider_startup_args(
provider_settings: dict[str, object] | None,
) -> tuple[str, ...]:
raw = (provider_settings or {}).get("startup_args", ())
if not isinstance(raw, (list, tuple)):
return ()
return tuple(arg for arg in raw if isinstance(arg, str))
def prompt_args(
prompt_mode: PromptMode,
prompt_path: str | None,
@@ -358,7 +390,11 @@ def prompt_args(
if prompt_mode == "append_file":
return ["--append-system-prompt-file", prompt_path]
if prompt_mode == "read_prompt_file":
if argv and "resume" in argv:
if argv and ("resume" in argv or "remote-control" in argv):
return []
return [f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "print_read_prompt_file":
return ["-p", f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "append_system_prompt":
return ["--append-system-prompt", prompt_path]
raise ValueError(f"unknown provider prompt mode: {prompt_mode}")
+185 -52
View File
@@ -24,14 +24,16 @@ backend exposes five methods:
enough metadata for callers (CLI `list active`, dashboard
agents pane) to render a row.
Selection is driven by `--backend` on `start` or
BOT_BOTTLE_BACKEND (env var; default "docker"). Per PRD 0003 the
manifest does not carry a backend field; the host picks.
Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
(env var). When neither is set, compatible macOS hosts default to
`macos-container`; other hosts default to `smolmachines`. Per PRD 0003
the manifest does not carry a backend field; the host picks.
"""
from __future__ import annotations
import os
import shlex
import sys
from abc import ABC, abstractmethod
from contextlib import AbstractContextManager
@@ -39,14 +41,15 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Any, Generic, Sequence, TypeVar
from ..agent_provider import AgentProvisionPlan, get_provider
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
from ..egress import EgressPlan
from ..git_gate import GitGatePlan
from ..log import die, info
from ..manifest import GitEntry, Manifest
from ..manifest import Manifest, ManifestIndex
from ..supervise import SupervisePlan
from ..util import expand_tilde
from ..workspace import WorkspacePlan
from ..env import resolve_env, ResolvedEnv
from ..workspace import WorkspacePlan, workspace_plan
from .print_util import print_multi, visible_agent_env_names
from .util import host_skill_dir
@@ -58,7 +61,7 @@ class BottleSpec:
Resolved values (image names, container name, scratch paths, runsc
availability) live on the plan, not the spec."""
manifest: Manifest
manifest: ManifestIndex
agent_name: str
copy_cwd: bool
user_cwd: str
@@ -69,6 +72,9 @@ class BottleSpec:
identity: str = ""
label: str = ""
color: str = ""
# Ordered bottle names selected at launch (issue #269). When non-empty
# they are merged in order and replace the agent's `bottle:` field.
bottle_names: tuple[str, ...] = ()
@dataclass(frozen=True)
@@ -77,10 +83,14 @@ class BottlePlan(ABC):
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
spec: BottleSpec
manifest: Manifest
stage_dir: Path
guest_home: str
git_gate_plan: GitGatePlan
@property
def guest_home(self) -> str:
return self.agent_provision.guest_home
@property
def git_gate_insteadof_host(self) -> str:
"""Host (and optional port) used in git-gate insteadOf URLs.
@@ -97,15 +107,17 @@ class BottlePlan(ABC):
egress_plan: EgressPlan
supervise_plan: SupervisePlan | None
agent_provision: AgentProvisionPlan
workspace_plan: WorkspacePlan
def print(self, *, remote_control: bool) -> None:
@property
def workspace_plan(self) -> WorkspacePlan:
return workspace_plan(self.spec, guest_home=self.guest_home)
def print(self) -> None:
"""Render the y/N preflight summary to stderr."""
del remote_control
spec = self.spec
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
manifest = self.manifest
agent = manifest.agent
bottle = manifest.bottle
env_names = visible_agent_env_names(
sorted(
@@ -120,9 +132,13 @@ class BottlePlan(ABC):
info(f"provider : {self.agent_provision.template}")
print_multi("env ", env_names)
print_multi("skills ", list(agent.skills))
info(f"bottle : {agent.bottle}")
effective_bottles = (
list(spec.bottle_names) if spec.bottle_names
else ([agent.bottle] if agent.bottle else [])
)
print_multi("bottle ", effective_bottles)
identity = manifest.git_identity_summary(spec.agent_name)
identity = manifest.git_identity_summary()
if identity:
info(f" git identity : {identity}")
@@ -182,7 +198,7 @@ class ActiveAgent:
of sidecar daemons currently up for this bottle (`egress`,
`git-gate`, `supervise`); the dashboard uses it to
gate edit verbs. `backend_name` is the matching key in
`_BACKENDS` (`docker` / `smolmachines`) — used by the active-
`_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
list rendering to disambiguate and by the dashboard's
re-attach path."""
@@ -263,27 +279,101 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
name: str
def prepare(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
def prepare(self, spec: BottleSpec, stage_dir: Path) -> PlanT:
"""Template method: run cross-backend host-side validation, then
delegate to the subclass's `_resolve_plan` for the
backend-specific resolution (names, scratch files, etc.). The
validation step is enforced here so a future backend cannot
accidentally skip it. No remote/runtime resources are created."""
self._validate(spec)
return self._resolve_plan(spec, stage_dir=stage_dir)
from .resolve_common import (
merge_provision_env_vars,
mint_slug,
prepare_agent_state_dir,
prepare_egress,
prepare_git_gate,
prepare_supervise,
resolve_manifest_dockerfile,
write_launch_metadata,
)
def _validate(self, spec: BottleSpec) -> None:
"""Cross-backend pre-launch checks. Confirms the agent exists,
the named skills are present on the host, and every git
IdentityFile resolves. Subclasses with additional preconditions
should override and call `super()._validate(spec)` first."""
manifest = spec.manifest
manifest.require_agent(spec.agent_name)
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
self._validate_skills(agent.skills)
self._validate_git_entries(bottle.git)
self._validate_agent_provider_dockerfile(spec)
manifest = self._validate(spec)
self._preflight()
manifest_bottle = manifest.bottle
manifest_agent_provider = manifest_bottle.agent_provider
agent_provider = get_provider(manifest_agent_provider.template)
resolved_env = resolve_env(manifest)
workspace = workspace_plan(spec, guest_home=agent_provider.guest_home)
slug = mint_slug(spec)
write_launch_metadata(slug, spec, compose_project="", backend=self.name)
# Manifest may override the Dockerfile per-bottle; otherwise fall
# back to the provider plugin's bundled Dockerfile (next to its
# agent_provider.py module).
if manifest_agent_provider.dockerfile:
agent_dockerfile_path = resolve_manifest_dockerfile(
manifest_agent_provider.dockerfile, spec,
)
else:
agent_dockerfile_path = str(agent_provider.dockerfile)
agent_dir, prompt_file = prepare_agent_state_dir(slug, manifest)
agent_provision_plan = build_agent_provision_plan(
template=manifest_agent_provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
instance_name=f"bot-bottle-{slug}",
prompt_file=prompt_file,
guest_env=self._build_guest_env(resolved_env),
forward_host_credentials=manifest_agent_provider.forward_host_credentials,
auth_token=manifest_agent_provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace.workdir,
label=spec.label,
color=spec.color,
provider_settings=manifest_agent_provider.settings,
)
agent_provision_plan = merge_provision_env_vars(agent_provision_plan)
egress_plan = prepare_egress(manifest_bottle, slug, agent_provision_plan)
supervise_plan = prepare_supervise(manifest_bottle, slug)
git_gate_plan = prepare_git_gate(manifest_bottle, slug)
return self._resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return {}
def _preflight(self) -> None:
"""
tasks to do before resolving a plan
"""
pass
def _validate(self, spec: BottleSpec) -> Manifest:
"""Cross-backend pre-launch checks. Parses the selected agent and
its bottle (raising ManifestError on invalid content), confirms
skills are present on the host, and every git IdentityFile resolves.
Returns the loaded Manifest for the selected agent. Subclasses with
additional preconditions should override and call
`super()._validate(spec)` first."""
manifest = spec.manifest.load_for_agent(spec.agent_name, spec.bottle_names)
self._validate_skills(manifest.agent.skills)
self._validate_agent_provider_dockerfile(spec, manifest)
return manifest
def _validate_skills(self, skills: Sequence[str]) -> None:
"""Each named skill must be a directory under the host's
@@ -297,18 +387,8 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
f"Create it under ~/.claude/skills/, then re-run."
)
def _validate_git_entries(self, entries: Sequence[GitEntry]) -> None:
"""Each entry's IdentityFile must exist on the host (after
expanding leading ~) — the git-gate copies it in at start time
to authenticate the upstream push (PRD 0008). Shape is already
enforced by Manifest validation; this only checks presence."""
for entry in entries:
key = expand_tilde(entry.IdentityFile)
if not os.path.isfile(key):
die(f"git upstream key file not found for '{entry.Name}': {key}")
def _validate_agent_provider_dockerfile(self, spec: BottleSpec) -> None:
bottle = spec.manifest.bottle_for(spec.agent_name)
def _validate_agent_provider_dockerfile(self, spec: BottleSpec, manifest: Manifest) -> None:
bottle = manifest.bottle
dockerfile = bottle.agent_provider.dockerfile
if not dockerfile:
return
@@ -316,16 +396,31 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
if not path.is_absolute():
path = Path(spec.user_cwd) / path
if not path.is_file():
effective = (
", ".join(spec.bottle_names) if spec.bottle_names else manifest.agent.bottle
)
die(
f"agent_provider.dockerfile for bottle "
f"'{spec.manifest.agents[spec.agent_name].bottle}' not found: {path}"
f"'{effective}' not found: {path}"
)
@abstractmethod
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
def _resolve_plan(self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path) -> PlanT:
"""Backend-specific plan resolution: image/container names,
env-file, prompt-file, proxy plan, runtime detection. Called by
`prepare` after `_validate` succeeds."""
`prepare` after `_validate` succeeds. Instance name, image,
prompt file, Dockerfile path, and guest home all live on
`agent_provision_plan` — the source of truth."""
@abstractmethod
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
@@ -369,9 +464,30 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
return prompt_path
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
"""Copy the operator workspace into the running bottle when
the backend cannot bake it into the agent image. Default is
no-op for backends like Docker that handle this before launch."""
"""Copy the operator workspace into the running bottle.
This is the only supported workspace-provisioning path: Docker
does not build a derived image containing the current
workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path = shlex.quote(workspace.guest_path)
guest_parent = shlex.quote(guest_parent)
owner = shlex.quote(workspace.owner)
mode = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
bottle.exec(
f"rm -rf {guest_path} && mkdir -p {guest_parent}",
user="root",
)
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
bottle.exec(
f"chown -R {owner} {guest_path} && chmod {mode} {guest_path}",
user="root",
)
def supervise_mcp_url(self, plan: PlanT) -> str:
"""Return the agent-side URL of the per-bottle supervise
@@ -416,8 +532,14 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
# via `from . import ...` without hitting a partially-initialized module.
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
# Freezer is imported after the backend classes for the same reason:
# Freezer.commit_slug constructs ActiveAgent, which must be fully
# defined first.
from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylint: disable=wrong-import-position
# The dict is heterogeneous: each value is a BottleBackend specialized
# over its own plan type. Concrete plan types are erased here because
@@ -425,6 +547,7 @@ from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: dis
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
"docker": DockerBottleBackend(),
"macos-container": MacosContainerBottleBackend(),
"smolmachines": SmolmachinesBottleBackend(),
}
@@ -437,17 +560,24 @@ def get_bottle_backend(
`name` precedence:
1. explicit arg (CLI `--backend=<name>` passes through here)
2. BOT_BOTTLE_BACKEND env var
3. default `docker`
3. `macos-container` on compatible macOS hosts
4. default `smolmachines`
Dies with a pointer at the known backends if the chosen name
isn't implemented."""
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or _default_backend_name()
if resolved not in _BACKENDS:
known = ", ".join(sorted(_BACKENDS))
die(f"unknown backend {resolved!r}; known backends: {known}")
return _BACKENDS[resolved]
def _default_backend_name() -> str:
if has_backend("macos-container"):
return "macos-container"
return "smolmachines"
def known_backend_names() -> tuple[str, ...]:
"""Sorted tuple of all backend keys in `_BACKENDS`. Used by
argparse (`--backend` choices) and the dashboard's backend
@@ -497,9 +627,12 @@ __all__ = [
"BottleCleanupPlan",
"BottlePlan",
"BottleSpec",
"CommitCancelled",
"ExecResult",
"Freezer",
"enumerate_active_agents",
"get_bottle_backend",
"get_freezer",
"has_backend",
"known_backend_names",
]
+42 -8
View File
@@ -2,10 +2,10 @@
This module is a thin façade. The real work lives in four siblings:
- prepare.py — host-side resolution into a DockerBottlePlan
- launch.py — bring-up + teardown context manager
- cleanup.py — orphan enumeration + removal
- enumerate.py — active-agent listing
- resolve_plan.py — Docker-specific resolution into a DockerBottlePlan
- launch.py — bring-up + teardown context manager
- cleanup.py — orphan enumeration + removal
- enumerate.py — active-agent listing
The base class's `prepare` template runs cross-backend host-side
validation before calling `_resolve_plan` here.
@@ -25,17 +25,23 @@ from pathlib import Path
from typing import Generator, Sequence
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import prepare as _prepare
from . import resolve_plan as _resolve_plan
from .bottle import DockerBottle
from .bottle_cleanup_plan import DockerBottleCleanupPlan
from .bottle_plan import DockerBottlePlan
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
(default)."""
when set to `docker`; retained as a legacy/example backend."""
name = "docker"
@@ -48,8 +54,36 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
launch."""
return shutil.which("docker") is not None
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> DockerBottlePlan:
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> DockerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
+16 -6
View File
@@ -9,6 +9,7 @@ from typing import cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
class DockerBottle(Bottle):
@@ -22,15 +23,20 @@ class DockerBottle(Bottle):
*,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
):
self.name = container
self._teardown = teardown
self.prompt_path = prompt_path_in_container
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.agent_provider_template = (
"codex" if agent_command == "codex" else "claude"
)
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self._closed = False
def agent_argv(
@@ -43,13 +49,17 @@ class DockerBottle(Bottle):
cmd = ["docker", "exec"]
if tty:
cmd.append("-it")
if self.agent_workdir and self.agent_workdir != "/home/node":
cmd.extend(["-w", self.agent_workdir])
cmd.extend([self.name, self.agent_command, *full_argv])
return cmd
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
return subprocess.run(
self.agent_argv(argv, tty=tty), check=False,
).returncode
agent_argv = self.agent_argv(argv, tty=tty)
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
# Pipe via stdin to `sh -s` so the caller never has to worry
+19 -12
View File
@@ -22,25 +22,32 @@ class DockerBottlePlan(BottlePlan):
`agent_provision` from BottlePlan."""
slug: str
container_name: str
container_name_pinned: bool
image: str
derived_image: str # "" -> no derived image
runtime_image: str # image actually launched (derived or base)
# Absolute path to the Dockerfile that builds `image`. Empty means
# use the repo's default Dockerfile. Populated to a per-bottle
# state file (~/.bot-bottle/state/<slug>/Dockerfile) after a
# capability-block remediation (PRD 0016).
dockerfile_path: str
env_file: Path # docker --env-file: NAME=VALUE literals
# name -> value for vars forwarded into the docker-run child process
# via subprocess env (so values never land on argv or in a file).
# repr=False keeps secret/interpolated/OAuth values out of any
# accidental log of the plan dataclass.
forwarded_env: dict[str, str] = field(repr=False)
prompt_file: Path
use_runsc: bool
@property
def container_name(self) -> str:
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
"""Absolute path to the Dockerfile that builds `image`. Sourced
from the agent provision plan — the manifest may override per
bottle; otherwise the provider plugin's bundled Dockerfile."""
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property
def agent_command(self) -> str:
return self.agent_provision.command
@@ -1,218 +0,0 @@
"""capability_apply — host-side orchestrator for capability-block
remediation (PRD 0016).
On approval of a capability-block proposal, the dashboard calls
apply_capability_change(slug, new_dockerfile) which:
1. Snapshots the agent's transcript dir to
~/.bot-bottle/state/<slug>/transcript/ (best-effort).
2. Pushes the agent's working tree via `git push` (best-effort —
no upstream / no commits / no git repo all skip with a log).
3. Writes the new Dockerfile to
~/.bot-bottle/state/<slug>/Dockerfile (PRD 0016 Phase 1
state). The next `cli.py start <agent>` picks it up.
4. Force-removes the agent container + all sidecars + the
per-bottle networks. Idempotent — missing resources are not
errors.
Returns (before, after) Dockerfile contents so the dashboard can
record / render the diff. (capability-block has no audit log per
PRD 0013 — the per-bottle Dockerfile state is its own record.)
This is "fire-and-forget" from the agent's perspective: by the time
the dashboard writes the response file the supervise sidecar is
gone, so the agent's tool call connection drops without ever
receiving the response. The replacement agent (next manual
`cli.py start`) sees the new Dockerfile and starts from there.
v1 does not auto-relaunch — see PRD 0016's capability-block return
semantics open question.
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
from ...log import info, warn
from .bottle_state import (
mark_preserved,
per_bottle_dockerfile,
transcript_snapshot_dir,
write_per_bottle_dockerfile,
)
from .sidecar_bundle import sidecar_bundle_container_name
# Agent home inside the container (per the repo Dockerfile's
# `USER node` + `WORKDIR /home/node`). Used to locate the transcript
# dir + the workspace dir for git push.
_AGENT_HOME_IN_CONTAINER = "/home/node"
_AGENT_TRANSCRIPT_IN_CONTAINER = f"{_AGENT_HOME_IN_CONTAINER}/.claude"
_AGENT_WORKSPACE_IN_CONTAINER = f"{_AGENT_HOME_IN_CONTAINER}/workspace"
# Per-bottle resource name patterns (mirroring prepare.py).
def _agent_container_name(slug: str) -> str:
return f"bot-bottle-{slug}"
def _per_bottle_container_names(slug: str) -> list[str]:
"""All container names that belong to this bottle. Missing
containers are silently skipped by the teardown helper, so it's
fine to include names that don't exist for a given bottle."""
return [
_agent_container_name(slug),
sidecar_bundle_container_name(slug),
]
def _per_bottle_network_names(slug: str) -> list[str]:
return [
f"bot-bottle-net-{slug}",
f"bot-bottle-egress-{slug}",
]
class CapabilityApplyError(RuntimeError):
"""Raised when the apply fails in a way that should keep the
proposal pending (so the operator can retry). Best-effort
failures (transcript snapshot, git push) do not raise — they
just log and proceed."""
# --- Public helpers --------------------------------------------------------
def fetch_current_dockerfile(slug: str) -> str:
"""Return the Dockerfile content the next `cli.py start <agent>`
would use for this bottle. If a per-bottle override exists, that
one; otherwise the repo's Dockerfile.
Used by the operator-edit verb to show the current source of
truth, and by apply_capability_change for the before-diff."""
override = per_bottle_dockerfile(slug)
if override is not None:
return override
repo_dockerfile = _repo_dockerfile_path()
if repo_dockerfile.is_file():
return repo_dockerfile.read_text()
raise CapabilityApplyError(
f"no per-bottle Dockerfile for {slug} and no repo Dockerfile at "
f"{repo_dockerfile}"
)
def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
"""End-to-end capability-block remediation. See module docstring
for the sequence. Returns (before, after) Dockerfile content."""
if not new_dockerfile.strip():
raise CapabilityApplyError("proposed Dockerfile is empty")
before = fetch_current_dockerfile(slug)
snapshot_transcript(slug)
_push_working_tree(slug)
write_per_bottle_dockerfile(slug, new_dockerfile)
# Set the preserve marker BEFORE teardown so cli.py's session-end
# cleanup sees it and keeps the state dir intact for the
# operator's `cli.py resume <identity>`. Without the marker the
# state dir would be deleted as part of normal session end.
mark_preserved(slug)
_teardown_bottle(slug)
return before, new_dockerfile
# --- Internals -------------------------------------------------------------
def _repo_dockerfile_path() -> Path:
"""Path to the repo's Claude Dockerfile (one dir above this module's
package root). Resolved at call time so the path is correct
regardless of where this module is imported from."""
# bot_bottle/backend/docker/capability_apply.py -> repo root
return Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
def snapshot_transcript(slug: str) -> None:
"""`docker cp` /home/node/.claude out of the agent container into
~/.bot-bottle/state/<slug>/transcript/. Best-effort: missing
container, missing dir, or cp error all log a warning and return.
The transcript is what `claude --resume` reads to pick up where
the agent left off.
Called from two places:
- capability-apply, before tearing the bottle down.
- cli.py's session-end path, before the launch context closes,
so a crash or normal exit also leaves a transcript on disk
(deleted along with the state dir on clean exit, kept on
crash or capability-block per the preserve marker)."""
container = _agent_container_name(slug)
dest = transcript_snapshot_dir(slug)
if dest.exists():
# Remove any prior snapshot so the new one is a clean copy.
shutil.rmtree(dest, ignore_errors=True)
dest.parent.mkdir(parents=True, exist_ok=True)
r = subprocess.run(
["docker", "cp", f"{container}:{_AGENT_TRANSCRIPT_IN_CONTAINER}", str(dest)],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"transcript snapshot skipped "
f"({(r.stderr or '').strip() or 'no transcript dir in container?'})"
)
return
info(f"transcript snapshotted to {dest}")
def _push_working_tree(slug: str) -> None:
"""`docker exec <agent> git push` from /home/node/workspace.
Best-effort: not-a-git-repo, no upstream, nothing-to-push, no
network all log a warning and return. The replacement bottle
will pick up whatever's actually upstream."""
container = _agent_container_name(slug)
r = subprocess.run(
[
"docker", "exec", container, "sh", "-c",
f"cd {_AGENT_WORKSPACE_IN_CONTAINER} && "
f"git rev-parse --is-inside-work-tree >/dev/null 2>&1 && "
f"git push origin HEAD 2>&1 || true",
],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"capability-apply: git push skipped "
f"({(r.stderr or '').strip() or 'docker exec failed'})"
)
return
output = (r.stdout or "").strip()
if output:
info(f"capability-apply: git push: {output}")
else:
info("capability-apply: git push ran (no output — likely not a git workspace)")
def _teardown_bottle(slug: str) -> None:
"""Force-remove all per-bottle docker resources. Idempotent —
`docker rm -f` / `docker network rm` silently ignore missing
names, so this can be called even mid-rebuild."""
info(f"capability-apply: tearing down bottle {slug}")
for name in _per_bottle_container_names(slug):
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
for net in _per_bottle_network_names(slug):
subprocess.run(
["docker", "network", "rm", net],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
__all__ = [
"CapabilityApplyError",
"apply_capability_change",
"fetch_current_dockerfile",
"snapshot_transcript",
]
+1 -1
View File
@@ -31,7 +31,7 @@ from ... import supervise as _supervise
from ...log import info, warn
from . import util as docker_mod
from .bottle_cleanup_plan import DockerBottleCleanupPlan
from .bottle_state import bottle_state_dir, is_preserved
from ...bottle_state import bottle_state_dir, is_preserved
from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
+6 -16
View File
@@ -28,11 +28,12 @@ from typing import Any
from ...egress import (
EGRESS_HOSTNAME,
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_sidecar_env_entries,
)
from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn
from ...supervise import (
CURRENT_CONFIG_DIR_IN_AGENT,
QUEUE_DIR_IN_CONTAINER,
SUPERVISE_HOSTNAME,
SUPERVISE_PORT,
@@ -134,9 +135,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
ep = plan.egress_plan
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
if ep.routes:
volumes.append(_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER))
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
volumes.append(_bind(ep.routes_path.parent, str(Path(EGRESS_ROUTES_IN_CONTAINER).parent)))
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate -----------------------------------------------------
gp = plan.git_gate_plan
@@ -220,9 +220,10 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
# never lands on argv or in the compose file.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
service: dict[str, Any] = {
"image": plan.runtime_image,
"image": plan.image,
"container_name": plan.container_name,
"command": ["sleep", "infinity"],
"networks": {"internal": None},
@@ -230,17 +231,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
}
if plan.use_runsc:
service["runtime"] = "runsc"
if plan.env_file and plan.env_file.exists() and plan.env_file.stat().st_size > 0:
service["env_file"] = [str(plan.env_file)]
volumes: list[dict[str, Any]] = []
if plan.supervise_plan is not None:
volumes.append(_bind(
plan.supervise_plan.current_config_dir,
CURRENT_CONFIG_DIR_IN_AGENT,
))
if volumes:
service["volumes"] = volumes
# The init supervisor inside the bundle owns intra-bundle
# daemon ordering, so the agent only waits for the bundle
+29 -18
View File
@@ -1,24 +1,21 @@
"""Host-side helper for egress sidecar inspection (issue #198).
"""Host-side helper for egress sidecar inspection and live updates.
`_merge_single_route`, `add_route`, and `apply_routes_change` were
removed when the egress-block MCP tool was dropped. The remaining
helpers support runtime inspection and validation of the routes file
without modifying it at runtime.
The approve path uses this module to validate a proposed routes file,
write it to the bottle's live egress state dir, and signal the sidecar
bundle so the mitmproxy addon reloads it.
"""
from __future__ import annotations
import os
import subprocess
from ...egress import EGRESS_ROUTES_IN_CONTAINER
from ...egress_addon_core import load_routes
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError
from .sidecar_bundle import sidecar_bundle_container_name
class EgressApplyError(RuntimeError):
pass
def fetch_current_routes(slug: str) -> str:
container = sidecar_bundle_container_name(slug)
r = subprocess.run(
@@ -33,17 +30,31 @@ def fetch_current_routes(slug: str) -> str:
return r.stdout
def validate_routes_content(content: str) -> None:
try:
load_routes(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
class DockerEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None:
container = sidecar_bundle_container_name(slug)
result = subprocess.run(
["docker", "kill", "--signal", "HUP", container],
capture_output=True, text=True, check=False, env=os.environ,
)
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'docker kill failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {container}: "
f"{last_error or 'docker kill failed'}"
)
applicator = DockerEgressApplicator()
__all__ = [
"DockerEgressApplicator",
"EgressApplyError",
"applicator",
"fetch_current_routes",
"validate_routes_content",
]
+1 -1
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import subprocess
from .. import ActiveAgent
from .bottle_state import read_metadata
from ...bottle_state import read_metadata
from .compose import compose_project_name, list_active_slugs
+23
View File
@@ -0,0 +1,23 @@
"""DockerFreezer — snapshot a Docker bottle via `docker commit`."""
from __future__ import annotations
from .. import ActiveAgent
from ..freeze import Freezer
from .util import commit_container
from ...log import info
class DockerFreezer(Freezer):
"""Freezes a Docker bottle by running `docker commit`."""
backend_name = "docker"
def _freeze(self, agent: ActiveAgent) -> str:
container = f"bot-bottle-{agent.slug}"
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
commit_container(container, image_tag)
return image_tag
def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: docker save {image_ref} -o {slug}.tar")
+26 -15
View File
@@ -4,8 +4,8 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
The flow is:
1. Build the agent's base + derived image (compose builds the
sidecar images via the `build:` directive on first up).
1. Build the agent image from the provider Dockerfile (compose
builds the sidecar images via the `build:` directive on first up).
2. Mint the per-bottle egress CA (chunk 2 writes it under
state/<slug>/egress/).
3. Populate the inner plans with launch-time fields so the
@@ -15,8 +15,8 @@ The flow is:
7. `docker compose up -d` (token + OAuth values flow into the
compose subprocess env so `environment: [NAME]` bare-name
entries inherit without rendering values into the file).
8. Provision (CA install, prompt copy, skills, git, supervise
config) — unchanged, uses `docker exec`.
8. Provision (CA install, prompt copy, skills, workspace, git,
supervise config) — unchanged, uses `docker exec` / `docker cp`.
9. Yield a DockerBottle handle. `exec_agent` runs claude via
`docker exec -it` exactly like the pre-compose world.
@@ -43,10 +43,11 @@ from . import network as network_mod
from . import util as docker_mod
from .bottle import DockerBottle
from .bottle_plan import DockerBottlePlan
from .bottle_state import (
from ...bottle_state import (
bottle_state_dir,
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from .compose import (
bottle_plan_to_compose,
@@ -75,7 +76,7 @@ def launch(
Teardown on exit."""
stack = ExitStack()
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
_bottle_for_revoke = plan.manifest.bottle
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
def teardown() -> None:
@@ -91,15 +92,21 @@ def launch(
)
try:
# Step 1: agent image build. Sidecar images get built lazily by
# `docker compose up` via the renderer's `build:` directives.
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
if plan.derived_image:
docker_mod.build_image_with_cwd(
plan.derived_image, plan.image, plan.workspace_plan
# Step 1: agent image. Use a committed snapshot when one exists
# and is present in the local daemon; otherwise build from the
# Dockerfile. Sidecar images get built lazily by `docker compose
# up` via the renderer's `build:` directives.
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
else:
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
internal_network = network_mod.network_name_for_slug(plan.slug)
@@ -179,6 +186,10 @@ def launch(
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
-280
View File
@@ -1,280 +0,0 @@
"""Prepare step for the Docker bottle backend.
`resolve_plan` does all host-side resolution (image and container
names, env-file, prompt-file, proxy plan, runtime detection) and
returns a frozen DockerBottlePlan. No Docker resources are created;
the only side effects are scratch files under `stage_dir` and a probe
of `docker info`. Cross-backend host-side validation has already run
via the base class's `prepare` template before this is called.
"""
from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import PROVIDER_TEMPLATES, agent_provision_plan, runtime_for
from ...egress import Egress
from ...env import ResolvedEnv, resolve_env
from ...git_gate import GitGate
from ...log import die
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .. import BottleSpec
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
from .bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
clear_preserve_marker,
egress_state_dir,
git_gate_state_dir,
per_bottle_dockerfile,
per_bottle_dockerfile_path,
per_bottle_image_tag,
supervise_state_dir,
write_metadata,
)
from .sidecar_bundle import sidecar_bundle_container_name
def resolve_plan(
spec: BottleSpec,
*,
stage_dir: Path,
) -> DockerBottlePlan:
"""Resolve Docker-specific names and write scratch files. Trusts
that the agent and its skills/git-gate keys are present —
validation already ran in the base class."""
docker_mod.require_docker()
git_gate = GitGate()
egress = Egress()
supervise = Supervise()
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = "/home/node"
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
# mints a random-suffixed identity (so parallel runs of the same
# agent in the same cwd don't collide on container/network
# names); a `resume` passes the recorded identity in via
# spec.identity to continue an existing bottle's state.
slug = spec.identity or bottle_identity(spec.agent_name)
# Record the launch metadata so `cli.py resume <identity>` can
# reconstruct the spec. Idempotent — re-writes on resume with a
# refreshed started_at.
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=f"bot-bottle-{slug}",
backend="docker",
label=spec.label,
color=spec.color,
))
# Clear any leftover preserve marker from a prior capability-block
# so this fresh launch can be cleaned up at session-end unless
# the agent triggers another capability-block.
clear_preserve_marker(slug)
# PRD 0016 capability-block: if a per-bottle Dockerfile has been
# written (via apply_capability_change), the base image becomes
# per_bottle_image_tag(slug) built from that file. --cwd still
# layers a derived image on top.
dockerfile_path = ""
if per_bottle_dockerfile(slug) is not None:
image_default = per_bottle_image_tag(slug)
dockerfile_path = str(per_bottle_dockerfile_path(slug))
elif provider.dockerfile:
image_default = f"bot-bottle-{provider.template}:{slug}"
dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
elif provider_runtime.dockerfile:
image_default = provider_runtime.image
dockerfile_path = provider_runtime.dockerfile
elif provider.template not in PROVIDER_TEMPLATES:
user_dockerfile = (
Path.home() / ".bot-bottle" / "contrib" / provider.template / "Dockerfile"
)
if user_dockerfile.is_file():
image_default = f"bot-bottle-{provider.template}:{slug}"
dockerfile_path = str(user_dockerfile)
else:
image_default = provider_runtime.image
else:
image_default = provider_runtime.image
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
derived_image = ""
runtime_image = image
if spec.copy_cwd:
derived_image = os.environ.get(
"BOT_BOTTLE_DERIVED_IMAGE", f"bot-bottle-cwd:{slug}"
)
runtime_image = derived_image
default_container = f"bot-bottle-{slug}"
pinned_container = os.environ.get("BOT_BOTTLE_CONTAINER", "")
container_name_pinned = bool(pinned_container)
if container_name_pinned:
container_name = pinned_container
if docker_mod.container_exists(container_name):
die(
f"container '{container_name}' already exists "
f"(pinned via BOT_BOTTLE_CONTAINER). "
f"Remove it with 'docker rm -f {container_name}' or unset the override."
)
else:
container_name = ""
for candidate in docker_mod.container_name_candidates(default_container):
if not docker_mod.container_exists(candidate):
container_name = candidate
break
if not container_name:
die(
f"could not find a free container name after "
f"{default_container}-{docker_mod.MAX_CONTAINER_SUFFIX}; "
f"clean up old containers with 'docker rm -f <name>'"
)
# Probe the sidecar-bundle container name for an orphan from a
# previous run. Otherwise a stale bundle surfaces as a
# docker-create conflict deep inside launch() with no actionable
# hint; failing fast here points at the cleanup command.
bundle_name = sidecar_bundle_container_name(slug)
if docker_mod.container_exists(bundle_name):
die(
f"sidecar bundle container '{bundle_name}' already exists. "
f"This is an orphan from a previous run; clean it up with "
f"'./cli.py cleanup' (or 'docker rm -f {bundle_name}') and "
f"retry."
)
# PRD 0018 chunk 2: prepare-time scratch files live under
# ~/.bot-bottle/state/<slug>/<service>/ so chunk 3's compose
# bind-mounts can point at stable paths. The state subdirs are
# cleaned up by start.py's session-end teardown unless something
# explicitly preserves the state dir (capability-block, crash).
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
env_file = agent_dir / "agent.env"
prompt_file = agent_dir / "prompt.txt"
prompt_file.write_text("")
prompt_file.chmod(0o600)
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
resolved = resolve_env(manifest, spec.agent_name)
# Everything that should reach the bottle by-name (so its value
# never lands on argv or in env_file) goes into one dict. Nothing
# mutates the host os.environ.
forwarded_env: dict[str, str] = dict(resolved.forwarded)
_write_env_file(resolved, env_file)
prompt_file.write_text(agent.prompt)
use_runsc = docker_mod.runsc_available()
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
label=spec.label,
color=spec.color,
)
guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=guest_env)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = egress.prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
# Current Dockerfile for the agent image. Read from the repo
# root; for `--cwd` derived images the base Dockerfile is what
# the agent should propose changes against (the derived layer
# is just a workspace copy).
# (routes.yaml used to land here too but PRD 0017 chunk 3
# moved it behind the `list-egress-routes` MCP tool so the
# agent gets live state rather than a launch-time snapshot.)
supervise_dockerfile_path = (
Path(dockerfile_path)
if dockerfile_path
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
)
dockerfile_content = (
supervise_dockerfile_path.read_text(encoding="utf-8")
if supervise_dockerfile_path.is_file()
else ""
)
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = supervise.prepare(
slug, supervise_dir,
dockerfile_content=dockerfile_content,
)
return DockerBottlePlan(
spec=spec,
stage_dir=stage_dir,
guest_home=guest_home,
slug=slug,
container_name=container_name,
container_name_pinned=container_name_pinned,
image=image,
derived_image=derived_image,
runtime_image=runtime_image,
dockerfile_path=dockerfile_path,
env_file=env_file,
forwarded_env=forwarded_env,
prompt_file=prompt_file,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
def _write_env_file(resolved: ResolvedEnv, env_file: Path) -> None:
"""Serialize the literal portion of a ResolvedEnv into docker's
`--env-file` syntax (NAME=VALUE per line, mode 600 since the file
may carry verbatim values from the manifest). Forwarded names ride
on the plan as a structured tuple instead."""
env_lines: list[str] = []
for name, value in resolved.literals.items():
if "\n" in value:
die(
f"env entry {name} (literal) contains a newline; "
f"docker --env-file cannot represent multi-line values."
)
env_lines.append(f"{name}={value}")
env_file.write_text("\n".join(env_lines) + ("\n" if env_lines else ""))
env_file.chmod(0o600)
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
+62
View File
@@ -0,0 +1,62 @@
"""Prepare step for the Docker bottle backend.
`resolve_plan` does all host-side resolution (image and container
names, prompt-file, proxy plan, runtime detection) and returns a
frozen DockerBottlePlan. No Docker resources are created; the only
side effects are scratch files under `stage_dir` and a probe of
`docker info`. Cross-backend host-side validation has already run
via the base class's `prepare` template before this is called.
"""
from __future__ import annotations
from pathlib import Path
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
from .. import BottleSpec
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...manifest import Manifest
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
def preflight() -> None:
docker_mod.require_docker()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> DockerBottlePlan:
"""Resolve Docker-specific names and write scratch files. Trusts
that the agent and its skills/git-gate keys are present —
validation already ran in the base class."""
# ==== docker specific setup ====
use_runsc = docker_mod.runsc_available()
return DockerBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_provision=agent_provision_plan,
)
+47 -33
View File
@@ -7,11 +7,10 @@ from __future__ import annotations
import re
import shutil
import subprocess
import tempfile
from typing import Iterable, Iterator
from ...log import die, info
from ...workspace import WorkspacePlan
# from ...workspace import WorkspacePlan
# Cap on the suffix the container-name conflict logic will try before
@@ -118,39 +117,54 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
subprocess.run(args, check=True)
def build_image_with_cwd(
derived: str,
base: str,
workspace: WorkspacePlan,
) -> None:
"""Build a thin derived image that copies the workspace into
the plan's guest path and sets the plan's workdir."""
import os
# def build_image_with_cwd(
# derived: str,
# base: str,
# workspace: "WorkspacePlan",
# ) -> None:
# """Build a thin derived image that copies the workspace into
# the plan's guest path and sets the plan's workdir."""
# import os
#
# cwd = str(workspace.host_path)
# if not os.path.isdir(cwd):
# die(f"cwd not found at {cwd}")
# info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
# with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
# context_dir = os.path.join(tmp, "context")
# staged_workspace = os.path.join(context_dir, "workspace")
# shutil.copytree(
# cwd,
# staged_workspace,
# symlinks=True,
# ignore=shutil.ignore_patterns(".git"),
# )
# dockerfile = (
# f"FROM {base}\n"
# f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
# f"WORKDIR {workspace.workdir}\n"
# )
# subprocess.run(
# ["docker", "build", "-t", derived, "-f", "-", context_dir],
# input=dockerfile,
# text=True,
# check=True,
# )
cwd = str(workspace.host_path)
if not os.path.isdir(cwd):
die(f"cwd not found at {cwd}")
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
context_dir = os.path.join(tmp, "context")
staged_workspace = os.path.join(context_dir, "workspace")
shutil.copytree(
cwd,
staged_workspace,
symlinks=True,
ignore=shutil.ignore_patterns(".git"),
)
dockerfile = (
f"FROM {base}\n"
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
f"WORKDIR {workspace.workdir}\n"
)
subprocess.run(
["docker", "build", "-t", derived, "-f", "-", context_dir],
input=dockerfile,
text=True,
check=True,
def commit_container(container_name: str, image_tag: str) -> None:
"""Run `docker commit <container_name> <image_tag>` to snapshot the
running container's filesystem state as a local Docker image."""
result = subprocess.run(
["docker", "commit", container_name, image_tag],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(
f"docker commit {container_name!r}{image_tag!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
info(f"committed {container_name!r}{image_tag!r}")
def image_id(ref: str) -> str:
+54
View File
@@ -0,0 +1,54 @@
"""Shared base class for host-side egress apply across backends.
Each backend subclasses EgressApplicator and overrides _signal_bundle_reload
with the backend-specific kill command.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from pathlib import Path
from ..bottle_state import egress_state_dir
from ..egress import EGRESS_ROUTES_FILENAME
from ..egress_addon_core import LOG_OFF, load_config
class EgressApplyError(RuntimeError):
pass
class EgressApplicator(ABC):
def apply_routes_change(self, slug: str, content: str) -> tuple[str, str]:
"""Persist `content` to the live routes file and reload egress."""
self.validate_routes_content(content)
routes_path = self._routes_path(slug)
routes_path.parent.mkdir(parents=True, exist_ok=True)
before = routes_path.read_text(encoding="utf-8") if routes_path.exists() else ""
routes_path.write_text(content, encoding="utf-8")
routes_path.chmod(0o600)
self._signal_bundle_reload(slug)
return before, content
@staticmethod
def validate_routes_content(content: str) -> None:
try:
config = load_config(content)
except ValueError as e:
raise EgressApplyError(
f"proposed routes.yaml is not valid: {e}"
) from e
if config.log != LOG_OFF:
raise EgressApplyError(
"proposed routes.yaml must not change egress logging"
)
@staticmethod
def _routes_path(slug: str) -> Path:
return egress_state_dir(slug) / EGRESS_ROUTES_FILENAME
@abstractmethod
def _signal_bundle_reload(self, slug: str) -> None: ...
__all__ = ["EgressApplicator", "EgressApplyError"]
+100
View File
@@ -0,0 +1,100 @@
"""Freezer — snapshot a running bottle to a resumable artifact.
Follows the same pattern as BottleBackend: a shared base class with
common post-freeze steps (write committed-image path, mark preserved,
print resume hint) and backend-specific subclasses in their respective
backend directories.
Entry points:
Freezer.commit(agent) — freeze by ActiveAgent
Freezer.commit_slug(slug) — convenience wrapper for cmd_commit
get_freezer(backend_name) — factory
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from . import ActiveAgent
from ..bottle_state import mark_preserved, write_committed_image
from ..log import die, info
class CommitCancelled(Exception):
"""Raised by Freezer._freeze when the user declines a confirmation prompt."""
class Freezer(ABC):
"""Freezes a running bottle to a resumable artifact.
The base class owns the shared post-commit steps:
- write_committed_image — records the artifact path in per-bottle state
- mark_preserved — prevents teardown from removing the state dir
- resume hint — printed to stderr after the snapshot
Subclasses implement _freeze with the backend-specific snapshot
operation and optionally override _export_hint for migration hints.
"""
backend_name: str
def commit(self, agent: ActiveAgent) -> None:
"""Freeze the bottle for `agent` to a resumable artifact.
Calls _freeze for the backend-specific snapshot, then writes the
committed image reference to per-bottle state and marks the bottle
preserved so the next `./cli.py resume` boots from the snapshot.
Raises CommitCancelled if the user declines an interactive
confirmation prompt (e.g. the macos-container stop prompt).
"""
image_ref = self._freeze(agent)
write_committed_image(agent.slug, image_ref)
mark_preserved(agent.slug)
info(f"to resume from this snapshot: ./cli.py resume {agent.slug}")
self._export_hint(agent.slug, image_ref)
@abstractmethod
def _freeze(self, agent: ActiveAgent) -> str:
"""Backend-specific snapshot. Returns the image tag or artifact path
stored by write_committed_image. Raises CommitCancelled if the user
declines a stop-confirmation prompt."""
def _export_hint(self, slug: str, image_ref: str) -> None:
"""Optionally print an export-for-migration hint after committing.
Overridden by backends that provide a meaningful export command."""
def commit_slug(self, slug: str) -> None:
"""Convenience entry for cmd_commit when only a slug is available."""
from ..bottle_state import read_metadata
metadata = read_metadata(slug)
agent = ActiveAgent(
backend_name=self.backend_name,
slug=slug,
agent_name=metadata.agent_name if metadata else "",
started_at=metadata.started_at if metadata else "",
services=(),
)
self.commit(agent)
def get_freezer(backend_name: str) -> Freezer:
"""Return the Freezer for the named backend.
backend_name "" is treated as "docker" for backward compatibility
with state dirs written before the backend field was added."""
resolved = backend_name or "docker"
if resolved == "docker":
from .docker.freezer import DockerFreezer
return DockerFreezer()
if resolved == "macos-container":
from .macos_container.freezer import MacosContainerFreezer
return MacosContainerFreezer()
if resolved == "smolmachines":
from .smolmachines.freezer import SmolmachinesFreezer
return SmolmachinesFreezer()
die(
f"commit is only supported for docker, macos-container, and "
f"smolmachines; backend {backend_name!r} has no freezer"
)
raise AssertionError("unreachable")
@@ -0,0 +1,10 @@
"""macOS Apple Container backend.
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
the Apple `container` CLI integration; launch remains gated until the
sidecar network enforcement shape is implemented.
"""
from .backend import MacosContainerBottleBackend
__all__ = ["MacosContainerBottleBackend"]
@@ -0,0 +1,87 @@
"""MacosContainerBottleBackend — Apple Container implementation."""
from __future__ import annotations
from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import util as _container
from .bottle import MacosContainerBottle
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
from .bottle_plan import MacosContainerBottlePlan
class MacosContainerBottleBackend(
BottleBackend["MacosContainerBottlePlan", "MacosContainerBottleCleanupPlan"]
):
"""Apple Container backend. Selected by
`BOT_BOTTLE_BACKEND=macos-container` or
`--backend=macos-container`."""
name = "macos-container"
@classmethod
def is_available(cls) -> bool:
return _container.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> MacosContainerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager
def launch(
self, plan: MacosContainerBottlePlan
) -> Generator[MacosContainerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
return _cleanup.prepare_cleanup()
def cleanup(self, plan: MacosContainerBottleCleanupPlan) -> None:
_cleanup.cleanup(plan)
def enumerate_active(self) -> Sequence[ActiveAgent]:
return _enumerate.enumerate_active()
def supervise_mcp_url(self, plan: MacosContainerBottlePlan) -> str:
return plan.agent_supervise_url
@@ -0,0 +1,131 @@
"""Bottle handle for Apple's `container` CLI."""
from __future__ import annotations
import os
import subprocess
import sys
from typing import Callable, cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import pty_forward as _pty_forward
_PTY_FORWARD_SCRIPT = _pty_forward.__file__
_TERMINAL_ENV_NAMES = (
"TERM",
"COLORTERM",
"TERM_PROGRAM",
"TERM_PROGRAM_VERSION",
"KITTY_WINDOW_ID",
"KITTY_PID",
"WEZTERM_PANE",
"WEZTERM_UNIX_SOCKET",
"GHOSTTY_BIN_DIR",
"GHOSTTY_RESOURCES_DIR",
"ITERM_SESSION_ID",
"VTE_VERSION",
"KONSOLE_VERSION",
"ALACRITTY_WINDOW_ID",
)
def _terminal_env_names() -> tuple[str, ...]:
return tuple(
name for name in _TERMINAL_ENV_NAMES
if name == "TERM" or os.environ.get(name)
)
class MacosContainerBottle(Bottle):
def __init__(
self,
container: str,
teardown: Callable[[], None],
prompt_path_in_container: str | None,
*,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
):
self.name = container
self._teardown = teardown
self.prompt_path = prompt_path_in_container
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self._closed = False
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
full_argv = list(argv)
full_argv.extend(
prompt_args(
cast(PromptMode, self._agent_prompt_mode),
self.prompt_path,
argv=full_argv,
)
)
container_exec = ["container", "exec"]
if tty:
container_exec.extend(["--interactive", "--tty"])
# Forward terminal capability hints so TUIs can enable modified-key
# protocols. Use bare env names: values stay in the child env, not
# on argv, and pty_forward supplies a TERM fallback when needed.
for name in _terminal_env_names():
container_exec.extend(["--env", name])
if self.agent_workdir and self.agent_workdir != "/home/node":
container_exec.extend(["--workdir", self.agent_workdir])
container_exec.extend([self.name, self.agent_command, *full_argv])
if tty:
# Wrap with the raw-mode forwarder: container exec does not put
# the host terminal into raw mode itself, so the line discipline
# buffers modifier-key sequences until CR. The wrapper sets raw
# mode before exec and restores it on exit.
return [sys.executable, _PTY_FORWARD_SCRIPT, "--", *container_exec]
return container_exec
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
agent_argv = self.agent_argv(argv, tty=tty)
script = (
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
if tty else None
)
if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
result = subprocess.run(
["container", "exec", "--user", user, "--interactive",
self.name, "sh", "-s"],
input=script,
capture_output=True,
text=True,
check=False,
)
return ExecResult(
returncode=result.returncode,
stdout=result.stdout,
stderr=result.stderr,
)
def cp_in(self, host_path: str, container_path: str) -> None:
subprocess.run(
["container", "cp", host_path, f"{self.name}:{container_path}"],
stdout=subprocess.DEVNULL,
check=True,
)
def close(self) -> None:
if self._closed:
return
self._closed = True
self._teardown()
@@ -0,0 +1,27 @@
"""Cleanup plan for the macOS Apple Container backend."""
from __future__ import annotations
from dataclasses import dataclass
from ...log import info
from .. import BottleCleanupPlan
@dataclass(frozen=True)
class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
containers: tuple[str, ...] = ()
networks: tuple[str, ...] = ()
def print(self) -> None:
if not self.containers and not self.networks:
info("macos-container cleanup: nothing to remove")
return
for name in self.containers:
info(f"macos-container container: {name}")
for name in self.networks:
info(f"macos-container network: {name}")
@property
def empty(self) -> bool:
return not self.containers and not self.networks
@@ -0,0 +1,58 @@
"""Plan type for the macOS Apple Container backend."""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from ...agent_provider import PromptMode
from .. import BottlePlan
@dataclass(frozen=True)
class MacosContainerBottlePlan(BottlePlan):
slug: str
forwarded_env: dict[str, str] = field(repr=False)
agent_proxy_url: str = ""
agent_git_gate_url: str = ""
agent_supervise_url: str = ""
@property
def container_name(self) -> str:
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property
def agent_command(self) -> str:
return self.agent_provision.command
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@property
def git_gate_insteadof_host(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
return super().git_gate_insteadof_host
@property
def git_gate_insteadof_scheme(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return "http"
return super().git_gate_insteadof_scheme
@@ -0,0 +1,70 @@
"""Cleanup for the macOS Apple Container backend."""
from __future__ import annotations
import subprocess
from ...log import info, warn
from . import util as container_mod
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
_PREFIX = "bot-bottle-"
_BUNDLE_PREFIX = "bot-bottle-sidecars-"
def _list_prefixed_containers() -> list[str]:
result = subprocess.run(
["container", "list", "--all", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
warn(f"container list failed: {result.stderr.strip()}")
return []
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX) or name.startswith(_BUNDLE_PREFIX)
)
def _list_prefixed_networks() -> list[str]:
result = subprocess.run(
["container", "network", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX)
)
def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
container_mod.require_container()
return MacosContainerBottleCleanupPlan(
containers=tuple(_list_prefixed_containers()),
networks=tuple(_list_prefixed_networks()),
)
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
for name in plan.containers:
info(f"container delete --force {name}")
subprocess.run(
["container", "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
for name in plan.networks:
info(f"container network delete {name}")
subprocess.run(
["container", "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
@@ -0,0 +1,39 @@
"""Host-side egress apply for the macos-container backend.
Uses `container kill --signal HUP` (Apple Container framework) instead
of `docker kill` to signal the sidecar bundle.
"""
from __future__ import annotations
import os
import subprocess
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError
from .launch import sidecar_container_name
class MacOSContainerEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None:
container = sidecar_container_name(slug)
result = subprocess.run(
["container", "kill", "--signal", "HUP", container],
capture_output=True, text=True, check=False, env=os.environ,
)
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'container kill failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {container}: "
f"{last_error or 'container kill failed'}"
)
applicator = MacOSContainerEgressApplicator()
__all__ = ["MacOSContainerEgressApplicator", "EgressApplyError", "applicator"]
@@ -0,0 +1,40 @@
"""Active-agent enumeration for the macOS Apple Container backend."""
from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent
_PREFIX = "bot-bottle-"
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def enumerate_active() -> list[ActiveAgent]:
result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
out: list[ActiveAgent] = []
for name in sorted(line.strip() for line in result.stdout.splitlines()):
if not name.startswith(_PREFIX):
continue
if name.startswith(_SIDECAR_PREFIX):
continue
slug = name[len(_PREFIX):]
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="macos-container",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=(),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
@@ -0,0 +1,31 @@
"""MacosContainerFreezer — snapshot a macOS container bottle.
Apple Container removes containers when they stop, making stop-then-export
impossible. Instead, commit_container execs into the running container and
streams the root filesystem via tar. The bottle continues running after commit.
"""
from __future__ import annotations
from .. import ActiveAgent
from ..freeze import Freezer
from .util import commit_container
from ...log import info
class MacosContainerFreezer(Freezer):
"""Freezes a macOS-container bottle via exec-tar + image rebuild."""
backend_name = "macos-container"
def _freeze(self, agent: ActiveAgent) -> str:
container = f"bot-bottle-{agent.slug}"
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
commit_container(container, image_tag)
return image_tag
def _export_hint(self, slug: str, image_ref: str) -> None:
info(
f"to export for migration: "
f"container image save {image_ref} -o {slug}.tar"
)
@@ -0,0 +1,432 @@
"""Launch flow for the macOS Apple Container backend.
This backend keeps the explicit proxy-env enforcement model for v1:
the agent container is attached only to a host-only Apple Container
network, while the sidecar bundle is attached to a NAT network first
and the host-only network second. The sidecar's host-only IP is
discovered from `container inspect` and stamped into the agent's
HTTP_PROXY / HTTPS_PROXY env vars.
"""
from __future__ import annotations
import dataclasses
import os
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import die, info, warn
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from ..docker.egress import egress_tls_init
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import util as container_mod
from .bottle import MacosContainerBottle
from .bottle_plan import MacosContainerBottlePlan
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_AGENT_SLEEP_SECONDS = "2147483647"
_GIT_HTTP_PORT = 9420
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
def internal_network_name(slug: str) -> str:
return f"bot-bottle-net-{slug}"
def egress_network_name(slug: str) -> str:
return f"bot-bottle-egress-{slug}"
def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
@contextmanager
def launch(
plan: MacosContainerBottlePlan,
*,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]:
"""Build, run, provision, and yield an Apple Container bottle."""
stack = ExitStack()
bottle_for_revoke = plan.manifest.bottle
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"macos-container teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
plan = _mint_certs(plan)
plan = _build_images(plan)
internal_network = internal_network_name(plan.slug)
egress_network = egress_network_name(plan.slug)
_create_networks(internal_network, egress_network, stack)
sidecar_name = sidecar_container_name(plan.slug)
container_mod.force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
stack.callback(container_mod.force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
sidecar_ip = container_mod.container_ipv4_on_network(
sidecar_name, internal_network,
)
plan = _stamp_agent_urls(plan, sidecar_ip)
container_mod.force_remove_container(plan.container_name)
_start_agent(plan, internal_network, sidecar_ip)
stack.callback(container_mod.force_remove_container, plan.container_name)
bottle = MacosContainerBottle(
plan.container_name,
teardown,
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
)
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
container_mod.build_image(
SIDECAR_BUNDLE_IMAGE,
_REPO_DIR,
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
)
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(
plan.agent_provision,
image=committed,
),
)
container_mod.build_image(
plan.image,
_REPO_DIR,
dockerfile=plan.dockerfile_path,
)
return plan
def _create_networks(
internal_network: str,
egress_network: str,
stack: ExitStack,
) -> None:
container_mod.create_network(internal_network, internal=True)
stack.callback(container_mod.remove_network, internal_network)
container_mod.create_network(egress_network)
stack.callback(container_mod.remove_network, egress_network)
def _start_sidecar_bundle(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> None:
argv = _sidecar_run_argv(plan, sidecar_name, internal_network, egress_network)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
env = {**os.environ, **token_values}
info(f"container run sidecar bundle {sidecar_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for sidecar bundle {sidecar_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _start_agent(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> None:
argv = _agent_run_argv(plan, internal_network, sidecar_ip)
env = {
**os.environ,
**plan.forwarded_env,
}
info(f"container run agent {plan.container_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for agent {plan.container_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _stamp_agent_urls(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> MacosContainerBottlePlan:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
supervise_url = ""
if plan.supervise_plan is not None:
supervise_url = f"http://{sidecar_ip}:{SUPERVISE_PORT}/"
git_gate_url = ""
if plan.git_gate_plan.upstreams:
git_gate_url = f"http://{sidecar_ip}:{_GIT_HTTP_PORT}"
return dataclasses.replace(
plan,
agent_proxy_url=proxy_url,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
container_mod.exec_container(
sidecar_name,
[
"mkdir",
"-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER,
"/git",
str(Path(_GIT_GATE_READY_FILE).parent),
],
)
for host_path, container_path in _git_gate_files(plan):
container_mod.copy_into_container(
sidecar_name, host_path, container_path,
)
container_mod.exec_container(
sidecar_name,
[
"sh",
"-c",
"chmod 755 "
f"{GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} "
f"{GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
f"touch {_GIT_GATE_READY_FILE}",
],
)
def _git_gate_files(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str], ...]:
gp = plan.git_gate_plan
files: list[tuple[str, str]] = [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for upstream in gp.upstreams:
files.append((
expand_tilde(upstream.identity_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key",
))
if upstream.known_hosts_file:
files.append((
str(upstream.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts",
))
return tuple(files)
def _sidecar_run_argv(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> list[str]:
argv = [
"container", "run",
"--name", sidecar_name,
"--detach",
"--rm",
"--network", egress_network,
"--network", internal_network,
"--dns", _sidecar_dns(),
"--env", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}",
]
for entry in _sidecar_env_entries(plan):
argv += ["--env", entry]
for host_path, container_path, read_only in _sidecar_mounts(plan):
argv += ["--mount", _mount_spec(host_path, container_path, read_only)]
argv.append(SIDECAR_BUNDLE_IMAGE)
return argv
def _agent_run_argv(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> list[str]:
argv = [
"container", "run",
"--name", plan.container_name,
"--detach",
"--network", internal_network,
]
for entry in _agent_env_entries(plan, sidecar_ip):
argv += ["--env", entry]
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
return argv
def _sidecar_dns() -> str:
return container_mod.dns_server()
def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
daemons = ["egress"]
if plan.git_gate_plan.upstreams:
daemons += ["git-gate", "git-http"]
if plan.supervise_plan is not None:
daemons.append("supervise")
return tuple(daemons)
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
return tuple(env)
def _sidecar_mounts(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str, bool], ...]:
mounts: list[tuple[str, str, bool]] = []
ep = plan.egress_plan
mounts.append((
str(ep.mitmproxy_ca_host_path.parent),
str(Path(EGRESS_CA_IN_CONTAINER).parent),
False,
))
if ep.routes:
mounts.append((
str(ep.routes_path.parent),
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent),
True,
))
sp = plan.supervise_plan
if sp is not None:
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
return tuple(mounts)
def _mount_spec(host_path: str, container_path: str, read_only: bool) -> str:
spec = f"type=bind,source={host_path},target={container_path}"
if read_only:
spec += ",readonly"
return spec
def _agent_env_entries(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> tuple[str, ...]:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
no_proxy = _agent_no_proxy(plan, sidecar_ip)
env = [
f"HTTPS_PROXY={proxy_url}",
f"HTTP_PROXY={proxy_url}",
f"https_proxy={proxy_url}",
f"http_proxy={proxy_url}",
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
if plan.agent_git_gate_url:
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
if plan.agent_supervise_url:
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
return tuple(env)
def _agent_no_proxy(plan: MacosContainerBottlePlan, sidecar_ip: str) -> str:
hosts = ["localhost", "127.0.0.1", sidecar_ip]
return ",".join(hosts)
@@ -0,0 +1,70 @@
"""Host-side raw-mode wrapper for `container exec --interactive --tty`.
Apple's `container exec --interactive --tty` does not set the host terminal to
raw mode before starting its I/O relay. Without raw mode the kernel line
discipline buffers modifier-key escape sequences (e.g. Shift+Enter in
modifyOtherKeys mode produces \\x1b[13;2~) until a carriage-return arrives, so
they never reach Claude Code inside the container.
This module sets the host terminal to raw mode, spawns the inner argv (the
container exec command), and restores the original terminal attributes on
exit. When stdin is not a TTY (piped invocations, CI) it falls through to a
bare subprocess.run so callers do not need to special-case non-interactive
contexts.
Usage (the `--` separator is the API contract — everything after it is the
inner command):
python pty_forward.py -- container exec --interactive --tty <name> <cmd>
"""
from __future__ import annotations
import os
import subprocess
import sys
import termios
import tty
def _inner_env() -> dict[str, str]:
env = dict(os.environ)
env.setdefault("TERM", "xterm-256color")
return env
def _run_inner(inner: list[str]) -> int:
return subprocess.run(inner, check=False, env=_inner_env()).returncode
def main(argv: list[str]) -> int:
"""Entry point. ``argv`` shape: ``-- <inner-argv...>``."""
if len(argv) < 2 or argv[0] != "--":
sys.stderr.write(
"usage: python pty_forward.py -- <container-exec-argv...>\n"
)
return 2
inner = argv[1:]
try:
fd = sys.stdin.fileno()
except OSError:
return _run_inner(inner)
if not os.isatty(fd):
return _run_inner(inner)
try:
old = termios.tcgetattr(fd)
except termios.error:
return _run_inner(inner)
try:
tty.setraw(fd)
return _run_inner(inner)
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
@@ -0,0 +1,47 @@
"""Prepare step for the macOS Apple Container backend."""
from __future__ import annotations
from pathlib import Path
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import BottleSpec
from . import util as container_mod
from .bottle_plan import MacosContainerBottlePlan
def preflight() -> None:
container_mod.require_container()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> MacosContainerBottlePlan:
return MacosContainerBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
+471
View File
@@ -0,0 +1,471 @@
"""Host-side primitives for Apple's `container` CLI."""
from __future__ import annotations
import json
import os
import ipaddress
import platform
import shutil
import subprocess
import tempfile
import time
from typing import Iterable
from ...log import die, info
_CONTAINER = "container"
_DEFAULT_DNS = "1.1.1.1"
def is_macos() -> bool:
return platform.system() == "Darwin"
def is_available() -> bool:
return is_macos() and shutil.which(_CONTAINER) is not None
def require_container() -> None:
"""Fail with an install pointer if Apple Container is unavailable."""
if not is_macos():
info("BOT_BOTTLE_BACKEND=macos-container requires macOS.")
die("macos-container backend is only supported on macOS")
if shutil.which(_CONTAINER) is None:
info("Apple Container is required but was not found on PATH.")
info("Install: https://github.com/apple/container/releases")
die("container not found")
_require_container_service()
def _require_container_service() -> None:
result = subprocess.run(
[_CONTAINER, "system", "status"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode != 0:
info("Apple Container system service is not running.")
info("Start it with: container system start")
die("container system service not running")
def dns_server() -> str:
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
if override:
return override
return _host_ipv4_dns() or _DEFAULT_DNS
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
"""Build an OCI image with Apple's BuildKit-backed `container build`."""
info(
f"building image {ref} from {context} with Apple Container "
"(layer cache keeps repeat builds fast)"
)
_ensure_builder_dns()
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
if dockerfile:
# `container build` resolves -f relative to the current working
# directory, not the build context. Anchor a relative Dockerfile to
# the context so builds work from any cwd.
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args.extend(["-f", dockerfile])
args.append(context)
subprocess.run(args, check=True)
def commit_container(container_name: str, image_tag: str) -> None:
"""Snapshot a running Apple Container as a local image.
`container export` requires a stopped container, but Apple Container
removes containers when they stop, making stop-then-export impossible.
Instead, exec into the running container as root and stream the root
filesystem out via tar, then build a new image from that archive.
The bottle continues running after commit.
"""
with tempfile.TemporaryDirectory(prefix="bot-bottle-container-commit.") as tmp:
rootfs_tar = os.path.join(tmp, "rootfs.tar")
dockerfile = os.path.join(tmp, "Dockerfile")
with open(rootfs_tar, "wb") as tar_out:
result = subprocess.run(
[
_CONTAINER, "exec",
"--user", "root",
container_name,
"tar", "--create",
"--exclude=./proc",
"--exclude=./sys",
"--exclude=./dev",
"--exclude=./run",
"--file=-",
"--directory=/",
".",
],
stdout=tar_out,
stderr=subprocess.PIPE,
check=False,
)
if result.returncode != 0:
die(
f"container exec tar {container_name!r} failed: "
f"{(result.stderr or b'').decode().strip() or '<no stderr>'}"
)
with open(dockerfile, "w", encoding="utf-8") as f:
f.write(
"FROM scratch\n"
"ADD rootfs.tar /\n"
"USER node\n"
"WORKDIR /home/node\n"
)
build_image(image_tag, tmp, dockerfile=dockerfile)
info(f"committed {container_name!r}{image_tag!r}")
def _ensure_builder_dns() -> None:
dns = dns_server()
status = _builder_status()
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
if _builder_running(status) and _builder_resolves_build_hosts():
if override and not _builder_has_dns(status, dns):
_restart_builder_with_dns(dns)
return
_restart_builder_with_dns(dns)
def _restart_builder_with_dns(dns: str) -> None:
subprocess.run(
[_CONTAINER, "builder", "stop"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
subprocess.run(
[_CONTAINER, "builder", "start", "--dns", dns],
check=True,
)
def _host_ipv4_dns() -> str:
if not is_macos():
return ""
result = subprocess.run(
["scutil", "--dns"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return ""
blocks: list[list[str]] = []
current: list[str] = []
for line in result.stdout.splitlines():
if line.startswith("resolver #") and current:
blocks.append(current)
current = []
current.append(line)
if current:
blocks.append(current)
for direct_only in (True, False):
for block in blocks:
text = "\n".join(block)
if direct_only and "Directly Reachable Address" not in text:
continue
for line in block:
if "nameserver[" not in line or ":" not in line:
continue
candidate = line.split(":", 1)[1].strip()
if _usable_ipv4(candidate):
return candidate
return ""
def _usable_ipv4(value: str) -> bool:
try:
address = ipaddress.ip_address(value)
except ValueError:
return False
return (
address.version == 4
and not address.is_loopback
and not address.is_link_local
and not address.is_multicast
and not address.is_unspecified
)
def _builder_status() -> list[dict[str, object]]:
result = subprocess.run(
[_CONTAINER, "builder", "status", "--format", "json"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError:
return []
if isinstance(data, list):
return [entry for entry in data if isinstance(entry, dict)]
if isinstance(data, dict):
return [data]
return []
def _builder_running(status: list[dict[str, object]]) -> bool:
for entry in status:
entry_status = entry.get("status")
if isinstance(entry_status, dict) and entry_status.get("state") == "running":
return True
return False
def _builder_dns_nameservers(status: list[dict[str, object]]) -> list[str]:
out: list[str] = []
for entry in status:
config = entry.get("configuration")
config_dns = config.get("dns") if isinstance(config, dict) else None
nameservers = (
config_dns.get("nameservers")
if isinstance(config_dns, dict)
else None
)
if not isinstance(nameservers, list):
continue
out.extend(name for name in nameservers if isinstance(name, str))
return out
def _builder_has_dns(status: list[dict[str, object]], dns: str) -> bool:
return dns in _builder_dns_nameservers(status)
def _builder_resolves_build_hosts() -> bool:
result = subprocess.run(
[_CONTAINER, "exec", "buildkit", "getent", "hosts", "deb.debian.org"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
return result.returncode == 0
def image_exists(ref: str) -> bool:
return _silent_run([_CONTAINER, "image", "inspect", ref]) == 0
def container_exists(name: str) -> bool:
result = subprocess.run(
[_CONTAINER, "list", "--all", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return False
return name in {line.strip() for line in result.stdout.splitlines()}
def container_is_running(name: str) -> bool:
"""Return True if the named container is currently running.
`container list` without `--all` lists only running containers."""
result = subprocess.run(
[_CONTAINER, "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return False
return name in {line.strip() for line in result.stdout.splitlines()}
def stop_container(name: str) -> None:
"""Stop the named container without deleting it."""
result = subprocess.run(
[_CONTAINER, "stop", name],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container stop {name!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def force_remove_container(name: str) -> None:
if container_exists(name):
subprocess.run(
[_CONTAINER, "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
def copy_into_container(name: str, host_path: str, container_path: str) -> None:
cmd = [_CONTAINER, "cp", host_path, f"{name}:{container_path}"]
result = _run_container_op(cmd)
if result.returncode != 0:
die(
f"container cp into {name}:{container_path} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def exec_container(name: str, argv: list[str]) -> None:
result = _run_container_op([_CONTAINER, "exec", name, *argv])
if result.returncode != 0:
die(
f"container exec in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]:
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
)
for _ in range(19):
if result.returncode == 0:
return result
time.sleep(0.1)
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
)
return result
def create_network(name: str, *, internal: bool = False) -> None:
args = [
_CONTAINER, "network", "create",
"--label", "bot-bottle.backend=macos-container",
]
if internal:
args.append("--internal")
args.append(name)
result = subprocess.run(
args, capture_output=True, text=True, check=False,
)
if result.returncode == 0:
return
if "already exists" in (result.stderr or "").lower():
return
die(
f"container network create {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def remove_network(name: str) -> None:
result = subprocess.run(
[_CONTAINER, "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode != 0:
return
def inspect_container(name: str) -> dict[str, object]:
result = subprocess.run(
[_CONTAINER, "inspect", name],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container inspect {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError as exc:
die(f"container inspect {name} returned malformed JSON: {exc}")
if isinstance(data, list) and data and isinstance(data[0], dict):
return data[0]
if isinstance(data, dict):
return data
die(f"container inspect {name} returned an unexpected shape")
raise AssertionError("unreachable")
def container_ipv4_on_network(name: str, network: str) -> str:
data = inspect_container(name)
status = data.get("status")
networks = status.get("networks") if isinstance(status, dict) else None
if not isinstance(networks, list):
die(f"container inspect {name} did not include status.networks")
for entry in networks:
if not isinstance(entry, dict):
continue
if entry.get("network") != network:
continue
raw = entry.get("ipv4Address")
if not isinstance(raw, str) or not raw:
die(f"container {name} has no IPv4 address on {network}")
return raw.split("/", 1)[0]
die(f"container {name} is not attached to network {network}")
raise AssertionError("unreachable")
def image_id(ref: str) -> str:
"""Return the image digest/ID from `container image inspect`.
The command returns JSON on current Apple Container releases. Keep
parsing narrow and fatal so callers do not cache on an empty key.
"""
import json
result = subprocess.run(
[_CONTAINER, "image", "inspect", ref],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container image inspect for {ref!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "{}")
except json.JSONDecodeError as exc:
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
if isinstance(data, list) and data:
data = data[0]
if isinstance(data, dict):
value = data.get("id") or data.get("digest") or data.get("ID")
if value:
return str(value)
die(f"container image inspect for {ref!r} did not include an image id")
raise AssertionError("unreachable")
def save(ref: str, output: str) -> None:
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
def _silent_run(cmd: Iterable[str]) -> int:
return subprocess.run(
list(cmd),
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
).returncode
+132
View File
@@ -0,0 +1,132 @@
"""Shared helpers used by both backends' resolve_plan steps.
Each helper owns one well-defined step of the per-bottle plan
resolution so docker and smolmachines don't repeat the same logic.
Backend-specific steps (container names, env-file, per-bottle
Dockerfile overrides, subnet allocation) stay in the backend's own
resolve_plan.py.
"""
from __future__ import annotations
import os
from dataclasses import replace
from datetime import datetime, timezone
from pathlib import Path
from ..agent_provider import AgentProvisionPlan
from ..bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
egress_state_dir,
git_gate_state_dir,
supervise_state_dir,
write_metadata,
)
from ..egress import Egress, EgressPlan
from ..git_gate import GitGate, GitGatePlan
from ..manifest import Manifest, ManifestBottle
from ..supervise import Supervise, SupervisePlan
from . import BottleSpec
def mint_slug(spec: BottleSpec) -> str:
"""Return the bottle identity: the recorded identity for a resume,
or a freshly minted one for a new start.
When a label is provided it becomes the full slug (no random suffix),
so two launches with the same label collide by design. When no label
is given the identity is minted with a random suffix to avoid
collisions between anonymous launches of the same agent."""
if spec.identity:
return spec.identity
if spec.label:
from .docker import util as docker_mod
return docker_mod.slugify(spec.label)
return bottle_identity(spec.agent_name)
def write_launch_metadata(
slug: str, spec: BottleSpec, *, compose_project: str, backend: str,
) -> None:
"""Persist launch metadata so `cli.py resume <identity>` can
reconstruct the spec. Idempotent — re-writes on resume with a
refreshed started_at."""
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=compose_project,
backend=backend,
label=spec.label,
color=spec.color,
bottle_names=spec.bottle_names,
))
def prepare_agent_state_dir(slug: str, manifest: Manifest) -> tuple[Path, Path]:
"""Create the agent state subdir, write the prompt file.
Returns (agent_dir, prompt_file)."""
agent = manifest.agent
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
prompt_file = agent_dir / "prompt.txt"
prompt_file.write_text(agent.prompt or "")
prompt_file.chmod(0o600)
return agent_dir, prompt_file
def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
return GitGate().prepare(bottle, slug, git_gate_dir)
def prepare_egress(
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
) -> EgressPlan:
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
"""Prepare the supervise sidecar state dir. Returns None when
bottle.supervise is falsy."""
if not bottle.supervise:
return None
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
return Supervise().prepare(slug, supervise_dir)
def merge_provision_env_vars(provision: AgentProvisionPlan) -> AgentProvisionPlan:
"""Fold provision.env_vars into guest_env (setdefault semantics)
and return a new plan with the merged guest_env."""
merged = dict(provision.guest_env)
for key, val in provision.env_vars.items():
merged.setdefault(key, val)
return replace(provision, guest_env=merged)
def resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
"""Resolve a manifest-supplied dockerfile path relative to user_cwd."""
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
__all__ = [
"merge_provision_env_vars",
"mint_slug",
"prepare_agent_state_dir",
"prepare_egress",
"prepare_git_gate",
"prepare_supervise",
"resolve_manifest_dockerfile",
"write_launch_metadata",
]
+36 -10
View File
@@ -13,16 +13,21 @@ from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import prepare as _prepare
from . import resolve_plan as _resolve_plan
from . import smolvm as _smolvm
from .bottle import SmolmachinesBottle
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
from .bottle_plan import SmolmachinesBottlePlan
from .provision import workspace as _workspace
class SmolmachinesBottleBackend(
@@ -41,10 +46,36 @@ class SmolmachinesBottleBackend(
runtime check happens at `prepare`."""
return _smolvm.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self, spec: BottleSpec, *, stage_dir: Path
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager
def launch(
@@ -53,11 +84,6 @@ class SmolmachinesBottleBackend(
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def provision_workspace(
self, plan: SmolmachinesBottlePlan, bottle: Bottle
) -> None:
_workspace.provision_workspace(plan, bottle)
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
"""The smolmachines guest reaches the supervise sidecar via a
host-published random port the launch step pinned earlier
+28 -8
View File
@@ -20,10 +20,12 @@ from __future__ import annotations
import subprocess
import sys
import time
import shlex
from typing import Mapping, cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import pty_resize as _pty_resize
from . import smolvm as _smolvm
@@ -68,6 +70,10 @@ class SmolmachinesBottle(Bottle):
guest_env: Mapping[str, str] | None = None,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
) -> None:
self.name = machine_name
# In-VM path to the agent's prompt file. None when the
@@ -81,9 +87,10 @@ class SmolmachinesBottle(Bottle):
self._guest_env = dict(guest_env or {})
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.agent_provider_template = (
"codex" if agent_command == "codex" else "claude"
)
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
def agent_argv(
self, argv: list[str], *, tty: bool = True,
@@ -91,8 +98,14 @@ class SmolmachinesBottle(Bottle):
flags = ["smolvm", "machine", "exec", "--name", self.name]
if tty:
flags += ["-i", "-t"]
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
self.agent_command]
agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
agent_tail += [
"sh", "-lc",
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
"bot-bottle-agent",
]
agent_tail.append(self.agent_command)
provider_prompt_args = prompt_args(
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
)
@@ -128,9 +141,16 @@ class SmolmachinesBottle(Bottle):
UID switches via `runuser -u node --` (not `-l`) so we
avoid login-shell wiring. HOME / USER come from `smolvm
-e` instead, which sets them on the process env."""
return subprocess.run(
self.agent_argv(argv, tty=tty), check=False,
).returncode
agent_argv = self.agent_argv(argv, tty=tty)
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
if script is None:
return subprocess.run(agent_argv, check=False).returncode
# Use sh -c (not -lc) so the script inherits PATH from the calling
# process. sh -l sources login-shell init files (e.g. /etc/profile)
# which may NOT include smolvm's location when it was installed via
# homebrew. The calling process (./cli.py) already has smolvm on PATH
# (provision steps succeed), so -c is sufficient.
return subprocess.run(["sh", "-c", script], check=False).returncode
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
# early-VM provisioning. Retry once after a short settle so
+28 -26
View File
@@ -29,27 +29,6 @@ class SmolmachinesBottlePlan(BottlePlan):
bundle_subnet: str
bundle_gateway: str
bundle_ip: str
# smolvm machine name + agent image source. machine_create
# boots from a packed `.smolmachine` artifact (pre-baked at
# prepare time via `smolvm pack create`); using `--from`
# instead of `--image` avoids the registry-pull race we hit
# when machine_start tried to fetch on-demand and the libkrun
# agent's network attempt got refused by macOS.
#
# Chunk 2d ships with a public placeholder image (alpine)
# since bot-bottle-claude:latest lives in the operator's local
# docker daemon and smolvm's crane backend can't read from
# there; chunk 4 resolves the agent-image-conversion gap
# (push to a registry first, or smolvm grows a docker-daemon
# transport).
machine_name: str
# Agent image ref (docker tag). `launch` runs the
# build → save → registry push → smolvm pack pipeline against
# this and feeds the resulting `.smolmachine` artifact to
# `machine_create --from`. The pipeline runs at launch time
# (not prepare time) so the docker build output doesn't garble
# the dashboard's preflight modal.
agent_image_ref: str
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
# the guest has no DNS resolver inside the TSI allowlist.
# Passed to `smolvm machine create` as `-e K=V` flags.
@@ -57,11 +36,6 @@ class SmolmachinesBottlePlan(BottlePlan):
# `--smolfile` is mutually exclusive with `--from`, and
# `--from` is the path that avoids the registry-pull race).
guest_env: dict[str, str]
# Path to the agent's prompt file on the host. Always written
# (mode 0o600) so the in-VM path always exists; the file is
# empty when the agent has no prompt — claude-code reads it
# via --append-system-prompt-file only when non-empty.
prompt_file: Path
# Inner Plans for the sidecar bundle daemons. The same shape the
# docker backend uses — same `.prepare()` calls produced
# them — but our launch step doesn't populate the
@@ -82,6 +56,34 @@ class SmolmachinesBottlePlan(BottlePlan):
agent_git_gate_host: str = ""
agent_supervise_url: str = ""
@property
def machine_name(self) -> str:
"""smolvm machine name. `machine_create` boots from a packed
`.smolmachine` artifact (pre-baked at prepare time via
`smolvm pack create`); using `--from` instead of `--image`
avoids the registry-pull race we hit when machine_start tried
to fetch on-demand and the libkrun agent's network attempt
got refused by macOS."""
return self.agent_provision.instance_name
@property
def agent_image(self) -> str:
"""Agent image ref (docker tag). `launch` runs the
build → save → registry push → smolvm pack pipeline against
this and feeds the resulting `.smolmachine` artifact to
`machine_create --from`. The pipeline runs at launch time
(not prepare time) so the docker build output doesn't garble
the dashboard's preflight modal."""
return self.agent_provision.image
@property
def prompt_file(self) -> Path:
"""Path to the agent's prompt file on the host. Always written
(mode 0o600) so the in-VM path always exists; the file is
empty when the agent has no prompt — claude-code reads it
via --append-system-prompt-file only when non-empty."""
return self.agent_provision.prompt_file
@property
def git_gate_insteadof_host(self) -> str:
return self.agent_git_gate_host
@@ -0,0 +1,21 @@
"""Egress apply for the smolmachines backend.
The smolmachines sidecar bundle runs as a host-side Docker container,
so egress signalling is identical to the docker backend.
"""
from __future__ import annotations
from ..docker.egress_apply import ( # noqa: F401
DockerEgressApplicator,
EgressApplyError,
applicator,
fetch_current_routes,
)
__all__ = [
"DockerEgressApplicator",
"EgressApplyError",
"applicator",
"fetch_current_routes",
]
+1 -1
View File
@@ -23,7 +23,7 @@ import json
import subprocess
from .. import ActiveAgent
from ..docker.bottle_state import read_metadata
from ...bottle_state import read_metadata
from . import sidecar_bundle as _bundle
+145
View File
@@ -0,0 +1,145 @@
"""SmolmachinesFreezer — snapshot a smolmachines bottle.
`smolvm pack create --from-vm` requires the VM to be stopped, and smolvm
removes VMs when stopped (same issue as Apple Container). Instead, exec
into the running VM as root to write a gzip-compressed tar of the root
filesystem to /var/tmp, then copy it to the host with `smolvm machine cp`,
build a Docker image from the archive, convert it to a smolmachine artifact
via the existing registry pipeline, and record the sidecar path. The VM
stays running throughout."""
from __future__ import annotations
import tempfile
from pathlib import Path
from .. import ActiveAgent
from ..freeze import Freezer
from ..docker import util as docker_mod
from .local_registry import crane_push_tarball, ephemeral_registry
from .smolvm import machine_cp, machine_exec, pack_create
from ...bottle_state import bottle_state_dir
from ...log import die, info
# Temp file written inside the VM during commit. Lives in /var/tmp
# (on-disk, unlike tmpfs /tmp) to survive for machine_cp.
_VM_COMMIT_TAR = "/var/tmp/.bot-bottle-commit.tar.gz"
class SmolmachinesFreezer(Freezer):
"""Freezes a smolmachines bottle via exec-tar + Docker image + smolmachine pack.
The VM is NOT stopped. We exec into the running VM to write a compressed
tar of the root filesystem to /var/tmp, copy it to the host with
machine_cp, build a Docker image (Docker's ADD decompresses .tar.gz
automatically), then run the same image→registry→pack_create pipeline
that _ensure_smolmachine uses for fresh builds."""
backend_name = "smolmachines"
def _freeze(self, agent: ActiveAgent) -> str:
machine = f"bot-bottle-{agent.slug}"
image_ref = f"bot-bottle-committed-{agent.slug}:latest"
output_dir = bottle_state_dir(agent.slug)
output_dir.mkdir(parents=True, exist_ok=True)
binary = output_dir / "committed-smolmachine"
sidecar = output_dir / "committed-smolmachine.smolmachine"
_snapshot_running_vm(machine, image_ref, binary)
return str(sidecar)
def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: cp {image_ref} {slug}.smolmachine")
def _snapshot_running_vm(machine: str, image_ref: str, binary: Path) -> None:
"""Exec-tar the running VM, build a Docker image, and pack to a smolmachine.
binary: destination for the launcher (sibling .smolmachine is the artifact
that machine_create --from consumes, same convention as pack_create).
"""
with tempfile.TemporaryDirectory(prefix="bot-bottle-vm-commit.") as tmp:
tmp_path = Path(tmp)
# Use .tar.gz — Docker ADD decompresses automatically and the
# compressed archive fits in the VM's /var/tmp more easily.
rootfs_tar_gz = tmp_path / "rootfs.tar.gz"
dockerfile = tmp_path / "Dockerfile"
_exec_tar_to_file(machine, rootfs_tar_gz)
dockerfile.write_text(
"FROM scratch\n"
"ADD rootfs.tar.gz /\n"
"USER node\n"
"WORKDIR /home/node\n"
)
docker_mod.build_image(image_ref, str(tmp_path), dockerfile=str(dockerfile))
image_tarball = binary.parent / "committed.image.tar"
docker_mod.save(image_ref, str(image_tarball))
try:
with ephemeral_registry() as handle:
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
push_ref = f"{handle.push_endpoint}/bot-bottle-committed:{digest}"
pack_ref = f"{handle.pull_endpoint}/bot-bottle-committed:{digest}"
crane_push_tarball(handle, str(image_tarball), push_ref)
pack_create(pack_ref, binary)
finally:
image_tarball.unlink(missing_ok=True)
def _exec_tar_to_file(machine: str, dest: Path) -> None:
"""Snapshot the running VM's root filesystem to dest (.tar.gz).
Writes a gzip-compressed tar to _VM_COMMIT_TAR inside the VM via
machine_exec (same mechanism as provisioning), then copies it to the
host with machine_cp. This avoids binary-stdout piping through the
smolvm exec channel, which does not reliably handle large binary output.
A connectivity probe (machine_exec true) runs first so a concurrent-exec
limitation (smolvm may reject a second exec while -i -t is active) is
reported clearly rather than as a silent failure."""
# Connectivity probe — if smolvm rejects concurrent exec while an
# interactive session is running, fail clearly here.
probe = machine_exec(machine, ["true"])
if probe.returncode != 0:
die(
f"smolvm exec is not available for {machine!r} "
f"(exit {probe.returncode}: {probe.stderr.strip() or probe.stdout.strip() or '<no output>'}). "
f"If an interactive session is active, smolvm may not support concurrent exec."
)
# Create the compressed tar inside the VM.
# tar exits 1 when files change during archiving (normal for a live
# filesystem); only treat exit > 1 as fatal.
tar_result = machine_exec(
machine,
[
"tar", "--create", "--gzip",
"--exclude=./proc",
"--exclude=./sys",
"--exclude=./dev",
"--exclude=./run",
# /tmp and /var/tmp are ephemeral. Their stale contents
# (e.g. /tmp/claude-<uid>) have uid remapped by smolvm's
# pack process, causing Claude Code to refuse to use them
# on resume. Exclude both; _init_vm recreates them with
# mkdir -p + correct ownership on every boot.
"--exclude=./tmp",
"--exclude=./var/tmp",
f"--file={_VM_COMMIT_TAR}",
"--directory=/",
".",
],
)
if tar_result.returncode > 1:
die(
f"smolvm exec tar {machine!r} failed (exit {tar_result.returncode}): "
f"{tar_result.stderr.strip() or tar_result.stdout.strip() or '<no output>'}"
)
# Copy from VM to host, then clean up.
try:
machine_cp(f"{machine}:{_VM_COMMIT_TAR}", str(dest))
finally:
machine_exec(machine, ["rm", "-f", _VM_COMMIT_TAR])
+77 -33
View File
@@ -23,7 +23,9 @@ from typing import Callable, Generator
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
@@ -40,8 +42,12 @@ from ..docker.git_gate import (
GIT_GATE_HOOK_IN_CONTAINER,
)
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import warn
from ..docker.bottle_state import egress_state_dir, git_gate_state_dir
from ...log import info, warn
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from . import loopback_alias as _loopback
from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm
@@ -85,14 +91,7 @@ def launch(
plan = _start_bundle(plan, network, loopback_ip, stack)
plan = _discover_urls(plan, loopback_ip)
# Build the agent image and pack it into a `.smolmachine`
# artifact (or hit the per-Dockerfile-digest cache). Runs
# here, not in prepare, so the docker-build output doesn't
# garble the dashboard's preflight modal.
agent_from_path = _ensure_smolmachine(
plan.agent_image_ref,
dockerfile=plan.agent_dockerfile_path,
)
agent_from_path = _agent_from_path(plan)
_launch_vm(plan, agent_from_path, loopback_ip, stack)
_init_vm(plan)
@@ -103,6 +102,10 @@ def launch(
guest_env=plan.guest_env,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
@@ -126,7 +129,7 @@ def _teardown_smolmachines(
except BaseException as exc: # noqa: W0718 — teardown must not fail
teardown_exc = exc
warn(f"smolmachines teardown failed: {exc!r}")
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
bottle = plan.manifest.bottle
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
if teardown_exc is not None:
raise teardown_exc
@@ -138,10 +141,12 @@ def _allocate_resources(
) -> tuple[str, str]:
"""Reserve a loopback alias and create the per-bottle docker bridge.
macOS only routes 127.0.0.1 by default; the per-bottle alias
scopes TSI's allowlist to this bottle's published ports so the
agent can't reach other bottles' or host services' ports on
loopback. No-op on Linux."""
The per-bottle alias scopes TSI's allowlist to this bottle's
published ports so the agent can't reach other bottles' or host
services' ports on loopback. On macOS `ensure_pool` first
sudo-aliases the pool on `lo0`; on Linux that's a no-op since
all of 127.0.0.0/8 is already loopback, but the per-bottle
allocation runs on both."""
_loopback.ensure_pool()
loopback_ip = _loopback.allocate(plan.slug)
network = _bundle.bundle_network_name(plan.slug)
@@ -187,9 +192,11 @@ def _discover_urls(
return the plan with URLs + guest_env stamped in.
Docker container IPs (192.168.x.x in the daemon's bridge)
aren't reachable from the smolvm guest on macOS — TSI uses
macOS networking, and macOS sees the daemon's bridge via the
published-port loopback forward only.
aren't reachable from the smolvm guest — TSI proxies the
guest's connects through the host, and the host reaches the
bundle only via its published-port loopback forward (the
daemon's bridge isn't on the TSI allowlist). The agent dials
the published port on the per-bottle loopback alias.
NO_PROXY includes the per-bottle loopback alias so the
supervise + git-gate URLs bypass HTTPS_PROXY."""
@@ -213,16 +220,23 @@ def _discover_urls(
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
no_proxy = f"{existing_no_proxy},{loopback_ip}"
guest_env = {
**plan.guest_env,
"HTTPS_PROXY": agent_proxy_url,
"HTTP_PROXY": agent_proxy_url,
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
"https_proxy": agent_proxy_url,
"http_proxy": agent_proxy_url,
"NO_PROXY": no_proxy,
"no_proxy": no_proxy,
}
if agent_git_gate_host:
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
name, value = entry.split("=", 1)
guest_env[name] = value
return dataclasses.replace(
plan,
@@ -242,10 +256,11 @@ def _launch_vm(
"""Create, patch, and start the smolvm VM; register teardown.
--allow-cidr is the per-bottle loopback alias so the guest can
only reach this bottle's bundle ports. force_allowlist patches
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
and --smolfile mutually exclusive."""
only reach this bottle's bundle ports. force_allowlist then
confirms the allowlist persisted (patching smolvm 0.8.0's
silent-drop of --allow-cidr when combined with --from) and
fails closed if it can't. Smolfile isn't usable here — smolvm
0.8.0 makes --from and --smolfile mutually exclusive."""
_smolvm.machine_create(
plan.machine_name,
from_path=agent_from_path,
@@ -253,9 +268,10 @@ def _launch_vm(
env=plan.guest_env,
)
stack.callback(_smolvm.machine_delete, plan.machine_name)
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
# when combined with `--from`. Patch the persisted state DB
# before start so the booted VM's TSI actually enforces.
# Confirm the booted VM's TSI allowlist will actually enforce the
# /32 before start (smolvm 0.8.0 silently drops `--allow-cidr`
# with `--from`, so the persisted state DB is patched if needed).
# Fails closed if enforcement can't be confirmed.
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
_smolvm.machine_start(plan.machine_name)
stack.callback(_smolvm.machine_stop, plan.machine_name)
@@ -265,16 +281,24 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
"""Repair filesystem ownership and wait for exec channel readiness.
Ownership repair: smolvm's pack process remaps files to the host
invoker's uid (501 on macOS). /home/node must be node:node so
invoker's uid (e.g. 501 on macOS, 1000 on Linux). The chowns use
names not numbers so they're correct on either. /home/node must
be node:node so
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
mode 1777 so non-root processes can create per-uid scratch dirs.
All folded into one sh -c to avoid back-to-back exec calls
immediately after machine_start (libkrun exec-channel race).
mkdir -p guards: when booting from a committed snapshot, /tmp and
/var/tmp are excluded from the archive (they're ephemeral and their
stale contents would have wrong uid after smolvm's uid remap). The
directories must be created before chown/chmod can set permissions.
wait_exec_ready polls until the exec channel is ready for the
subsequent provision calls, replacing the empirical sleep."""
_smolvm.machine_exec(plan.machine_name, [
"sh", "-c",
"mkdir -p /tmp /var/tmp && "
"chown -R node:node /home/node && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp",
@@ -304,12 +328,8 @@ def _bundle_launch_spec(
ep = plan.egress_plan
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
if ep.routes:
volumes.append((str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True))
# Bare-name entries for upstream-token slots. Their values
# come from the docker-run subprocess env (inherited from
# the operator's shell), never landing on argv.
for token_env in sorted(ep.token_env_map.keys()):
env.append(token_env)
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate ---------------------------------------------
gp = plan.git_gate_plan
@@ -378,6 +398,30 @@ def _resolve_token_env(
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
"""Return the `.smolmachine` artifact used for `machine create --from`.
Prefer a committed VM artifact when one is recorded and still
present. If the file was removed, fall back to the normal image
build + pack cache path.
"""
committed = read_committed_image(plan.slug)
if committed:
committed_path = Path(committed)
if committed_path.is_file():
info(f"using committed smolmachine {str(committed_path)!r}")
return committed_path
# Build the agent image and pack it into a `.smolmachine`
# artifact (or hit the per-Dockerfile-digest cache). Runs here,
# not in prepare, so the docker-build output doesn't garble the
# dashboard's preflight modal.
return _ensure_smolmachine(
plan.agent_image,
dockerfile=plan.agent_dockerfile_path,
)
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
"""Build the agent docker image and convert it into a
`.smolmachine` artifact, caching the result under
@@ -33,10 +33,13 @@ sudo-add the missing pool on first use per boot — the aliases
persist on `lo0` until reboot, so subsequent launches don't
prompt.
Linux native daemons share the host's network namespace; the
whole `127.0.0.0/8` is reachable by default and aliases are
unnecessary. The pool logic detects native-Linux and skips sudo
entirely; the DB patch is also gated on macOS.
On Linux the whole `127.0.0.0/8` is already routed to `lo`, so
docker can publish a bundle's ports directly on `127.0.0.<N>`
with no `ifconfig`/sudo step. `ensure_pool` is therefore a no-op
on Linux, but per-bottle alias *allocation* and the TSI allowlist
DB patch run on both platforms — the isolation property is
identical, it's just cheaper to set up on Linux. The state-DB
path differs per platform (see `_smolvm_db_path`).
Allocation is coordinated by inspecting running bundle
containers' published host IPs — each bottle's bundle owns the
@@ -47,6 +50,7 @@ from __future__ import annotations
import fcntl
import json
import os
import platform
import re
import sqlite3
@@ -57,20 +61,34 @@ from typing import Iterable
from ...log import die, info
# smolvm's persistent VM state on macOS — a SQLite DB whose `vms`
# table holds one JSON BLOB per machine. The Linux path is
# different, but smolmachines is macOS-only in v1 (PRD 0023) so
# we hard-code this. If the file moves under us we'll see a
# clear FileNotFoundError; not worth defensive cross-platform
# detection until the backend actually needs Linux.
_SMOLVM_DB_PATH = (
Path.home()
/ "Library"
/ "Application Support"
/ "smolvm"
/ "server"
/ "smolvm.db"
)
def _smolvm_db_path() -> Path:
"""smolvm's persistent VM state — a SQLite DB whose `vms` table
holds one JSON BLOB per machine. macOS stores it under
`Application Support`; Linux follows the XDG base-dir spec
(`$XDG_DATA_HOME`, default `~/.local/share`).
NOTE: the Linux location is inferred from smolvm's documented
`~/.local/share` install layout and must be confirmed against a
real Linux smolvm install. If it's wrong, `force_allowlist`'s
fail-closed check turns it into a clear launch-time error rather
than a silent escape."""
if platform.system() == "Darwin":
return (
Path.home()
/ "Library"
/ "Application Support"
/ "smolvm"
/ "server"
/ "smolvm.db"
)
xdg_data = os.environ.get("XDG_DATA_HOME")
base = Path(xdg_data) if xdg_data else Path.home() / ".local" / "share"
return base / "smolvm" / "server" / "smolvm.db"
# Resolved once at import: the host platform doesn't change within a
# process. Tests patch this attribute directly.
_SMOLVM_DB_PATH = _smolvm_db_path()
# Sixteen aliases by default. Tunable for hosts that want more
@@ -131,51 +149,74 @@ def ensure_pool() -> None:
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
"""Patch smolvm's persistent VM-state DB to set the machine's
`allowed_cidrs` to the given list. Workaround for smolvm
0.8.0's silent-drop of `--allow-cidr` when used with `--from`.
"""Ensure the machine's persisted TSI allowlist equals
`allowed_cidrs`, failing **closed** if that can't be confirmed.
Must run AFTER `smolvm machine create` (the row has to
exist) and BEFORE `smolvm machine start` (smolvm reads the
row on start; in-flight VMs don't pick up changes). Once
smolvm honors the CLI flag upstream this whole function is
redundant — flag-respecting create + remove this call from
launch.
Runs on both macOS and Linux. It exists because smolvm 0.8.0
silently drops `--allow-cidr` when combined with `--from`, so
the allowlist has to be written into smolvm's persistent state
DB before `machine start`. Rather than assume the flag was
dropped, we read the persisted row and only patch when it
doesn't already match — so a newer smolvm that honors the flag
is left untouched.
No-op on non-macOS — the DB path differs and the Linux
smolmachines code path isn't exercised in v1."""
if not _is_macos():
return
Must run AFTER `smolvm machine create` (the row has to exist)
and BEFORE `smolvm machine start` (smolvm reads the row on
start; in-flight VMs don't pick up changes).
Fail-closed: if the state DB is missing, the row is missing, or
the allowlist still doesn't match after patching, we `die()`
rather than boot a VM whose egress confinement we can't verify
— an unconfirmed allowlist is a sandbox-escape risk (the agent
VM could reach all of host loopback)."""
want = list(allowed_cidrs)
if not _SMOLVM_DB_PATH.is_file():
die(
f"smolvm state DB not found at {_SMOLVM_DB_PATH}. "
f"smolvm 0.8.0 expected? `smolvm --version` to check."
f"smolvm state DB not found at {_SMOLVM_DB_PATH}; cannot "
f"confirm the TSI allowlist is enforced. Refusing to launch "
f"(fail-closed). Check `smolvm --version` and the DB "
f"location for your platform."
)
con = sqlite3.connect(str(_SMOLVM_DB_PATH))
try:
cur = con.cursor()
row = cur.execute(
"SELECT data FROM vms WHERE name = ?", (machine_name,),
).fetchone()
if row is None:
die(
f"smolvm DB has no row for machine {machine_name!r}"
f"machine_create must run before force_allowlist."
cfg = _read_machine_cfg(con, machine_name)
if cfg.get("allowed_cidrs") != want:
cfg["allowed_cidrs"] = want
# Write as BLOB (the column type smolvm uses) — passing a
# plain str makes sqlite store it as Text and smolvm then
# fails to read it.
con.execute(
"UPDATE vms SET data = ? WHERE name = ?",
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
)
con.commit()
cfg = _read_machine_cfg(con, machine_name)
if cfg.get("allowed_cidrs") != want:
die(
f"could not enforce TSI allowlist {want!r} for machine "
f"{machine_name!r} (persisted value is "
f"{cfg.get('allowed_cidrs')!r}). Refusing to launch "
f"(fail-closed)."
)
cfg = json.loads(row[0])
cfg["allowed_cidrs"] = list(allowed_cidrs)
# Write as BLOB (the column type smolvm uses) — passing a
# plain str makes sqlite store it as Text and smolvm then
# fails to read it.
cur.execute(
"UPDATE vms SET data = ? WHERE name = ?",
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
)
con.commit()
finally:
con.close()
def _read_machine_cfg(con: sqlite3.Connection, machine_name: str) -> dict[str, object]:
"""Read + JSON-decode a machine's `data` BLOB from the smolvm
state DB. Dies (fail-closed) if the row is missing — the caller
can't confirm enforcement without it."""
row = con.execute(
"SELECT data FROM vms WHERE name = ?", (machine_name,),
).fetchone()
if row is None:
die(
f"smolvm DB has no row for machine {machine_name!r}"
f"machine_create must run before force_allowlist."
)
return json.loads(row[0])
def allocate(_slug: str) -> str:
"""Pick the lowest-numbered alias from the pool not already
in use by a running smolmachines bundle. Bails when the pool
@@ -184,16 +225,17 @@ def allocate(_slug: str) -> str:
used (no on-disk reservation, allocation is purely
docker-state-driven).
On non-macOS the whole `127.0.0.0/8` is loopback by default;
`127.0.0.1` is fine to share and we skip the alias dance.
This still returns a deterministic address so launch.py's
callers don't have to branch on platform.
Runs on both platforms: the allocation logic (docker-state
inspection + the file lock) is platform-independent. macOS
needs `ensure_pool` to have aliased the addresses on `lo0`
first; on Linux all of `127.0.0.0/8` is already loopback, so
docker can publish on the chosen `127.0.0.<N>` with no setup.
Per-bottle scoping (so the agent can't reach other bottles' or
host services' loopback ports) therefore holds on both.
An exclusive file lock serialises concurrent calls so two
simultaneous launches don't read the same docker state and
claim the same alias."""
if not _is_macos():
return "127.0.0.1"
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
fcntl.flock(lf, fcntl.LOCK_EX)
-185
View File
@@ -1,185 +0,0 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup — that's `launch.launch`'s job."""
from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import agent_provision_plan, runtime_for
from ...backend import BottleSpec
from ...backend.docker.bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
egress_state_dir,
git_gate_state_dir,
supervise_state_dir,
write_metadata,
)
from ...egress import Egress
from ...env import resolve_env
from ...git_gate import GitGate
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
# Gateway ports the bundle exposes inside its container — git-gate's
# git-daemon, supervise's MCP. The agent inside the smolvm guest
# dials these on the bundle's pinned IP.
_BUNDLE_GIT_GATE_PORT = 9418
_BUNDLE_SUPERVISE_PORT = 9100
def resolve_plan(
spec: BottleSpec, *, stage_dir: Path
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
smolmachines_preflight()
manifest = spec.manifest
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = "/home/node"
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
slug = spec.identity or bottle_identity(spec.agent_name)
# Record minimal metadata so `cli.py resume` can recover the
# slug. Same schema as the docker backend.
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project="",
backend="smolmachines",
label=spec.label,
color=spec.color,
))
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
resolved = resolve_env(manifest, spec.agent_name)
guest_env: dict[str, str] = {
**resolved.literals,
**resolved.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
# Prompt file is always written (mode 0o600) so the in-VM
# path always exists. Content is the agent's `prompt`
# field (markdown body) — empty for agents with no prompt.
# claude-code reads it via --append-system-prompt-file only
# when non-empty, but the file must exist either way to
# match the docker backend's contract.
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
prompt_file = agent_dir / "prompt.txt"
agent = manifest.agents[spec.agent_name]
prompt_file.write_text(agent.prompt or "")
prompt_file.chmod(0o600)
machine_name = f"bot-bottle-{slug}"
# Stash the agent image ref — `launch.launch` runs the
# build → pack pipeline at bringup. Honors BOT_BOTTLE_IMAGE
# to match the docker backend's `resolve_plan` default.
agent_dockerfile_path = ""
if provider.dockerfile:
agent_dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
image_default = f"bot-bottle-{provider.template}:{slug}"
elif provider_runtime.dockerfile:
agent_dockerfile_path = provider_runtime.dockerfile
image_default = provider_runtime.image
else:
image_default = provider_runtime.image
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
guest_env=guest_env,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
label=spec.label,
color=spec.color,
)
merged_guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
merged_guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = Egress().prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = Supervise().prepare(slug, supervise_dir)
return SmolmachinesBottlePlan(
spec=spec,
stage_dir=stage_dir,
guest_home=guest_home,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
machine_name=machine_name,
agent_image_ref=agent_image_ref,
guest_env=agent_provision.guest_env,
prompt_file=prompt_file,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
@@ -6,8 +6,7 @@ the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
provisioning also moved to the AgentProvider ABC (with Debian/node
defaults); user plugins override them for non-standard images.
The module left in this subpackage handles the remaining backend-
specific step:
- workspace.py — copy the operator workspace into the guest
No modules remain in this subpackage. Workspace copying now runs
through `BottleBackend.provision_workspace` against the running
bottle for every backend.
"""
@@ -1,32 +0,0 @@
"""Copy the operator workspace into a smolmachines guest."""
from __future__ import annotations
import shlex
from ....log import info
from ... import Bottle
from ..bottle_plan import SmolmachinesBottlePlan
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
"""Copy host cwd contents to the planned guest workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path_q = shlex.quote(workspace.guest_path)
guest_parent_q = shlex.quote(guest_parent)
owner_q = shlex.quote(workspace.owner)
mode_q = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
bottle.exec(
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
user="root",
)
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
bottle.exec(
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
user="root",
)
@@ -0,0 +1,83 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup — that's `launch.launch`'s job."""
from __future__ import annotations
from pathlib import Path
from .. import BottleSpec
from ...manifest import Manifest
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
def preflight() -> None:
smolmachines_preflight()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
return {
**resolved_env.literals,
**resolved_env.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
# ==== smolmachines specific setup ====
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
return SmolmachinesBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
guest_env=agent_provision_plan.guest_env,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
+26
View File
@@ -25,6 +25,7 @@ smolvm binary."""
from __future__ import annotations
import json
import shutil
import subprocess
import time
@@ -94,6 +95,16 @@ def pack_create(image: str, output: Path) -> None:
_smolvm("pack", "create", "--image", image, "-o", str(output))
def pack_create_from_vm(name: str, output: Path) -> None:
"""`smolvm pack create --from-vm <name> -o <output>`.
Snapshots an existing persistent VM into a pack artifact. As
with `pack_create`, smolvm writes a launcher at `output` and the
bootable sidecar at `output.smolmachine`.
"""
_smolvm("pack", "create", "--from-vm", name, "-o", str(output))
# --- Machine lifecycle ---------------------------------------------------
@@ -143,6 +154,21 @@ def machine_create(
_smolvm(*args)
def machine_is_running(name: str) -> bool:
"""Return True if the named VM is in the 'running' state."""
result = _smolvm("machine", "ls", "--json", check=False)
if result.returncode != 0:
return False
try:
machines = json.loads(result.stdout or "[]")
except ValueError:
return False
return any(
isinstance(m, dict) and m.get("name") == name and m.get("state") == "running"
for m in machines
)
def machine_start(name: str) -> None:
"""`smolvm machine start --name NAME`."""
_smolvm("machine", "start", "--name", name)
+46 -12
View File
@@ -5,24 +5,58 @@ unit-tested without importing the docker subprocess paths."""
from __future__ import annotations
import hashlib
import os
import platform
import shutil
from ...log import die
# libkrun's Linux backend drives the guest through KVM, so the host
# must expose `/dev/kvm` and the invoking user must be able to open
# it. macOS uses Hypervisor.framework and needs no device node.
_KVM_DEVICE = "/dev/kvm"
def smolmachines_preflight() -> None:
"""Ensure `smolvm` is on PATH before the launch flow runs.
Called from `_resolve_plan`; gives the operator a clear
install pointer rather than a cryptic FileNotFoundError
later. `gvproxy` is no longer required see the PRD's design
pivot section."""
if shutil.which("smolvm") is not None:
return
die(
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
"PATH. Install with: "
"curl -sSL https://smolmachines.com/install.sh | sh"
)
"""Ensure the host can run the smolmachines backend before the
launch flow starts. Called from `_resolve_plan`; surfaces a
clear, actionable error instead of a cryptic `smolvm` failure
deep in launch.
Checks `smolvm` is on PATH (both platforms) and, on Linux,
that `/dev/kvm` exists and is accessible. `gvproxy` is no
longer required see the PRD's design pivot section."""
if shutil.which("smolvm") is None:
die(
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
"PATH. Install with: "
"curl -sSL https://smolmachines.com/install.sh | sh. "
"To use the legacy Docker backend instead, set "
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
)
if platform.system() == "Linux":
_preflight_kvm()
def _preflight_kvm() -> None:
"""Linux-only: libkrun needs `/dev/kvm`. Distinguish 'KVM not
enabled' from 'no permission' so the operator knows which to
fix."""
if not os.path.exists(_KVM_DEVICE):
die(
f"BOT_BOTTLE_BACKEND=smolmachines needs {_KVM_DEVICE} on "
"Linux but it is missing. Enable KVM: load the kvm-intel "
"or kvm-amd kernel module (and confirm virtualization is "
"enabled in BIOS/firmware). To use the legacy Docker "
"backend instead, set BOT_BOTTLE_BACKEND=docker."
)
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
die(
f"{_KVM_DEVICE} exists but is not readable/writable by the "
"current user. Add your user to the `kvm` group "
"(`sudo usermod -aG kvm \"$USER\"`) and re-login, or run "
"with access to the device."
)
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
+71
View File
@@ -0,0 +1,71 @@
"""Terminal escape-sequence helpers shared across all bottle backends."""
from __future__ import annotations
import shlex
# color name → (normal_idx, normal_hex, bright_idx, bright_hex, dark_bg_hex)
# OSC 4 sets indexed palette entries (affects syntax-highlighted code and any
# TUI content that uses indexed colors). dark_bg_hex is used for OSC 11
# (default background) — a very dark tint that's visible even when the TUI
# uses true/24-bit colors for its own chrome, which would otherwise bypass
# the palette entirely.
_COLORS: dict[str, tuple[int, str, int, str, str]] = {
"red": (9, "#e74c3c", 1, "#c0392b", "#200808"),
"green": (10, "#2ecc71", 2, "#27ae60", "#082008"),
"yellow": (11, "#f1c40f", 3, "#d4ac0d", "#201808"),
"blue": (12, "#3498db", 4, "#2471a3", "#080820"),
"magenta": (13, "#9b59b6", 5, "#7d3c98", "#160820"),
}
# OSC 104 resets all indexed palette entries; OSC 111 resets default background.
_RESET_PRINTF = "printf '\\033]104\\007\\033]111\\007'"
def palette_printf(color: str) -> str:
"""Shell `printf` command that emits OSC 4 + OSC 11 to tint the terminal
for *color*: sets the normal/bright palette entries AND the default
background to a dark shade of that color. Returns '' if unknown."""
entry = _COLORS.get(color)
if not entry:
return ""
n_idx, n_hex, b_idx, b_hex, bg_hex = entry
seq = (
f"\\033]4;{n_idx};{n_hex}\\007"
f"\\033]4;{b_idx};{b_hex}\\007"
f"\\033]11;{bg_hex}\\007"
)
return f"printf '{seq}'"
def exec_shell_script(
agent_argv: list[str],
terminal_title: str = "",
terminal_color: str = "",
) -> str | None:
"""Build a shell script string that optionally sets the terminal
title and/or palette before running *agent_argv*, and resets the
palette + background on exit. Returns None when no decoration is
needed callers should run *agent_argv* directly in that case."""
title_cmd = (
f"printf '\\033]0;%s\\007' {shlex.quote(terminal_title)}"
if terminal_title else ""
)
pal_cmd = palette_printf(terminal_color)
if not title_cmd and not pal_cmd:
return None
parts: list[str] = []
if title_cmd:
parts.append(title_cmd)
if pal_cmd:
parts.append(pal_cmd)
parts.append(shlex.join(agent_argv))
parts.append(_RESET_PRINTF)
else:
# No palette change — exec so the agent replaces the shell.
parts.append(f"exec {shlex.join(agent_argv)}")
return "; ".join(parts)
@@ -1,8 +1,7 @@
"""Per-bottle persistent state (PRD 0016).
"""Per-bottle persistent state.
Holds the per-bottle Dockerfile override that capability-block
remediation writes, the transcript snapshot the state-preservation
helper saves before teardown, and the launch metadata that lets
Holds optional per-bottle Dockerfile overrides, the transcript snapshot
the state-preservation helper saves before teardown, and the launch metadata that lets
`cli.py resume <identity>` reconstruct a bottle's spec. State
lives at:
@@ -37,13 +36,13 @@ from dataclasses import dataclass
from pathlib import Path
from typing import cast
from ... import supervise as _supervise
from . import util as docker_mod
from . import supervise as _supervise
# Directory layout: ~/.bot-bottle/state/<identity>/...
_STATE_SUBDIR = "state"
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
_COMMITTED_IMAGE_NAME = "committed-image"
_TRANSCRIPT_SUBDIR = "transcript"
# Per-sidecar scratch subdirs. PRD 0018 chunk 2: bind-mount sources
# live here so chunk 3's `docker compose up` can find them at stable
@@ -61,7 +60,7 @@ _METADATA_NAME = "metadata.json"
_LIVE_CONFIG_SUBDIR = "live-config"
LIVE_CONFIG_ROUTES_NAME = "routes.yaml"
LIVE_CONFIG_ALLOWLIST_NAME = "allowlist"
# Empty marker file. capability_apply writes it before teardown so
# Empty marker file. Session preservation writes it before teardown so
# cli.py's session-end cleanup knows to preserve the state dir for
# `cli.py resume <identity>`. Absent = clean up.
_PRESERVE_MARKER = ".preserve"
@@ -82,6 +81,7 @@ def bottle_identity(agent_name: str) -> str:
To continue an existing bottle's state, use the recorded
identity from BottleMetadata via `cli.py resume <identity>`,
not this function."""
from .backend.docker import util as docker_mod
slug = docker_mod.slugify(agent_name)
suffix = "".join(secrets.choice(_SUFFIX_ALPHABET) for _ in range(_RANDOM_SUFFIX_LEN))
return f"{slug}-{suffix}"
@@ -111,6 +111,10 @@ class BottleMetadata:
backend: str = ""
label: str = ""
color: str = ""
# Ordered bottle names selected at launch (issue #269). Empty tuple
# for state dirs written before this change; resume falls back to
# the agent's `bottle:` field in that case.
bottle_names: tuple[str, ...] = ()
def metadata_path(identity: str) -> Path:
@@ -138,6 +142,10 @@ def read_metadata(identity: str) -> BottleMetadata | None:
if not isinstance(raw, dict):
return None
raw_typed = cast(dict[str, object], raw)
raw_bottle_names = raw_typed.get("bottle_names", [])
bottle_names: tuple[str, ...] = ()
if isinstance(raw_bottle_names, list):
bottle_names = tuple(str(n) for n in raw_bottle_names if isinstance(n, str))
return BottleMetadata(
identity=str(raw_typed.get("identity", identity)),
agent_name=str(raw_typed.get("agent_name", "")),
@@ -148,6 +156,7 @@ def read_metadata(identity: str) -> BottleMetadata | None:
backend=str(raw_typed.get("backend", "")),
label=str(raw_typed.get("label", "")),
color=str(raw_typed.get("color", "")),
bottle_names=bottle_names,
)
@@ -163,8 +172,7 @@ def per_bottle_dockerfile_path(identity: str) -> Path:
def per_bottle_dockerfile(identity: str) -> str | None:
"""Return the per-bottle Dockerfile content if present, else
None. None means: use the repo's Dockerfile (the original
pre-capability-block behavior)."""
None. None means: use the provider or manifest Dockerfile."""
p = per_bottle_dockerfile_path(identity)
if p.is_file():
return p.read_text()
@@ -179,6 +187,32 @@ def write_per_bottle_dockerfile(identity: str, content: str) -> Path:
return p
def committed_image_path(identity: str) -> Path:
return bottle_state_dir(identity) / _COMMITTED_IMAGE_NAME
def write_committed_image(identity: str, image_tag: str) -> Path:
"""Persist the committed image tag for `identity`. The next
`cli.py resume <identity>` will boot from this image instead of
rebuilding from the Dockerfile."""
path = committed_image_path(identity)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(image_tag.strip() + "\n")
path.chmod(0o644)
return path
def read_committed_image(identity: str) -> str | None:
"""Return the committed image tag for `identity`, or None if no
commit has been recorded. Used by the Docker launch step to skip
the Dockerfile build when a committed snapshot exists."""
path = committed_image_path(identity)
if not path.is_file():
return None
tag = path.read_text().strip()
return tag or None
def per_bottle_image_tag(identity: str) -> str:
"""Image tag for a rebuilt bottle. Distinct from the base
bot-bottle-claude:latest so per-bottle rebuilds don't collide in
@@ -222,9 +256,7 @@ def write_live_config(
def transcript_snapshot_dir(identity: str) -> Path:
"""Where capability_apply stashes the agent's transcript before
teardown, so the next `cli.py start <agent>` can offer to
resume from it."""
"""Where agent session snapshots are kept for resume flows."""
return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR
@@ -251,8 +283,7 @@ def git_gate_state_dir(identity: str) -> Path:
def supervise_state_dir(identity: str) -> Path:
"""State subdir for the supervise sidecar's current-config dir
(bind-mounted into the agent at /etc/bot-bottle/current-config).
"""State subdir reserved for supervise sidecar bind-mount sources.
The queue dir is intentionally NOT under here it lives at
~/.bot-bottle/queue/<slug>/ alongside the audit logs, so it
survives state-dir cleanup."""
@@ -274,9 +305,8 @@ def preserve_marker_path(identity: str) -> Path:
def mark_preserved(identity: str) -> Path:
"""Mark this bottle's state for preservation across session
teardown. Written by capability_apply.apply_capability_change so
cli.py's session-end cleanup leaves the state dir intact for a
subsequent `cli.py resume`."""
teardown so cli.py's session-end cleanup leaves the state dir
intact for a subsequent `cli.py resume`."""
path = preserve_marker_path(identity)
path.parent.mkdir(parents=True, exist_ok=True)
path.touch()
@@ -289,7 +319,7 @@ def is_preserved(identity: str) -> bool:
def clear_preserve_marker(identity: str) -> None:
"""Idempotent removal. Called at fresh launch (start or resume)
so a marker left from a prior capability-block doesn't keep
so a marker left from a prior preserved session doesn't keep
state alive past the next normal session-end."""
try:
preserve_marker_path(identity).unlink()
@@ -314,6 +344,7 @@ __all__ = [
"bottle_state_dir",
"cleanup_state",
"clear_preserve_marker",
"committed_image_path",
"egress_state_dir",
"git_gate_state_dir",
"is_preserved",
@@ -323,9 +354,11 @@ __all__ = [
"per_bottle_dockerfile_path",
"per_bottle_image_tag",
"preserve_marker_path",
"read_committed_image",
"read_metadata",
"supervise_state_dir",
"transcript_snapshot_dir",
"write_committed_image",
"write_metadata",
"write_per_bottle_dockerfile",
]
+4 -1
View File
@@ -1,6 +1,6 @@
"""Main CLI dispatcher.
Commands: cleanup, edit, info, init, list, resume, start, supervise
Commands: cleanup, commit, edit, info, init, list, resume, start, supervise
"""
from __future__ import annotations
@@ -12,6 +12,7 @@ from ..manifest import ManifestError
from ._common import PROG
from . import list as _list_mod
from .cleanup import cmd_cleanup
from .commit import cmd_commit
from .edit import cmd_edit
from .info import cmd_info
from .init import cmd_init
@@ -23,6 +24,7 @@ cmd_list = _list_mod.cmd_list
COMMANDS = {
"cleanup": cmd_cleanup,
"commit": cmd_commit,
"edit": cmd_edit,
"info": cmd_info,
"init": cmd_init,
@@ -37,6 +39,7 @@ def usage() -> None:
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
sys.stderr.write("Commands:\n")
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
sys.stderr.write(" edit open an agent in vim for editing\n")
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
+2 -3
View File
@@ -13,9 +13,8 @@ dirs are shared layout, so docker is the single owner of that
bucket.
State dirs with `.preserve` are intentionally never touched they
hold capability-block rebuilds or crash snapshots the operator may
want to `resume`. Manual `rm -rf ~/.bot-bottle/state/<identity>`
is the path for those.
hold preserved sessions the operator may want to `resume`. Manual
`rm -rf ~/.bot-bottle/state/<identity>` is the path for those.
"""
from __future__ import annotations
+53
View File
@@ -0,0 +1,53 @@
"""commit: freeze a running bottle's state to a resumable artifact.
Docker bottles are committed to a local Docker image. Macos-container
bottles are exported and rebuilt as a local Apple Container image.
Smolmachines bottles are packed from the running VM into a
`.smolmachine` artifact. The resulting reference is stored in
per-bottle state so the next `./cli.py resume <slug>` boots from the
snapshot instead of rebuilding from the Dockerfile.
"""
from __future__ import annotations
import argparse
from ..backend import enumerate_active_agents
from ..backend.freeze import CommitCancelled, get_freezer
from ..bottle_state import read_metadata
from ..log import die
from ._common import PROG
from . import tui
def cmd_commit(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} commit", add_help=True)
parser.add_argument(
"slug",
nargs="?",
default=None,
help=(
"bottle slug from `cli.py list active` "
"(omit to pick interactively)"
),
)
args = parser.parse_args(argv)
slug = args.slug
if slug is None:
active = enumerate_active_agents()
if not active:
die("no active bottles; start one with `./cli.py start`")
choices = [a.slug for a in active]
slug = tui.filter_select(choices, title="Select bottle to commit")
if slug is None:
return 0
metadata = read_metadata(slug)
backend = metadata.backend if metadata else ""
try:
get_freezer(backend).commit_slug(slug)
except CommitCancelled:
return 0
return 0
+7 -6
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
import argparse
from ..log import info
from ..manifest import Manifest
from ..manifest import ManifestIndex
from ._common import PROG, USER_CWD
@@ -14,11 +14,12 @@ def cmd_info(argv: list[str]) -> int:
parser.add_argument("name", help="agent name defined in bot-bottle.json")
args = parser.parse_args(argv)
manifest = Manifest.resolve(USER_CWD)
manifest.require_agent(args.name)
names = ManifestIndex.resolve(USER_CWD)
names.require_agent(args.name)
manifest = names.load_for_agent(args.name)
agent = manifest.agents[args.name]
bottle = manifest.bottle_for(args.name)
agent = manifest.agent
bottle = manifest.bottle
env_names = list(bottle.env.keys())
prompt_first_line = agent.prompt.splitlines()[0] if agent.prompt else ""
@@ -31,7 +32,7 @@ def cmd_info(argv: list[str]) -> int:
f"first line: {prompt_first_line or '(empty)'}"
)
info(f"bottle : {agent.bottle}")
identity = manifest.git_identity_summary(args.name)
identity = manifest.git_identity_summary()
if identity:
info(f" git identity : {identity}")
if bottle.git:
+9 -20
View File
@@ -7,26 +7,15 @@ import os
import sys
from ..backend import enumerate_active_agents
from ..manifest import Manifest
from ..manifest import ManifestIndex
from ._common import PROG, USER_CWD
_ANSI_COLOR_CODES: dict[str, str] = {
"black": "\033[30m",
"red": "\033[31m",
"green": "\033[32m",
"yellow": "\033[33m",
"blue": "\033[34m",
"magenta": "\033[35m",
"cyan": "\033[36m",
"white": "\033[37m",
"bright-black": "\033[90m",
"bright-red": "\033[91m",
"bright-green": "\033[92m",
"bright-yellow": "\033[93m",
"bright-blue": "\033[94m",
"bright-magenta": "\033[95m",
"bright-cyan": "\033[96m",
"bright-white": "\033[97m",
"red": "\033[91m",
"green": "\033[92m",
"yellow": "\033[93m",
"blue": "\033[94m",
"magenta": "\033[95m",
}
_ANSI_RESET = "\033[0m"
@@ -51,8 +40,8 @@ def cmd_list(argv: list[str]) -> int:
args = parser.parse_args(argv)
if args.scope == "available":
manifest = Manifest.resolve(USER_CWD)
for name in manifest.agents.keys():
manifest = ManifestIndex.resolve(USER_CWD)
for name in manifest.all_agent_names:
print(name)
return 0
@@ -66,7 +55,7 @@ def cmd_list(argv: list[str]) -> int:
# Tab-separated keeps the format stable for shell pipelines.
for b in active:
services = ",".join(b.services) if b.services else "-"
display_name = b.label if b.label else b.agent_name
display_name = f"{b.label} ({b.agent_name})" if b.label else b.agent_name
colored_name = _ansi_label(display_name, b.color)
print(f"{b.backend_name}\t{b.slug}\t{colored_name}\t{services}")
return 0
+8 -10
View File
@@ -4,13 +4,12 @@ Reads ~/.bot-bottle/state/<identity>/metadata.json to recover the
(agent_name, cwd, copy_cwd) the bottle was originally started with,
then runs the same launch core as `start` but pinned to the
recorded identity so the new bottle picks up any per-bottle Dockerfile
(from capability-block apply) and transcript snapshot under the same
state dir.
override and transcript snapshot under the same state dir.
Use case: an agent calls capability-block, the dashboard approves
and tears down the bottle, the operator runs
Use case: an interrupted or preserved bottle needs to be relaunched;
the operator runs
./cli.py resume <identity>
to bring up the replacement with the new capabilities baked in.
to bring up the replacement from the recorded state.
"""
from __future__ import annotations
@@ -18,9 +17,9 @@ from __future__ import annotations
import argparse
from ..backend import BottleSpec
from ..backend.docker.bottle_state import read_metadata
from ..bottle_state import read_metadata
from ..log import die
from ..manifest import Manifest
from ..manifest import ManifestIndex
from ._common import PROG, USER_CWD
from .start import _launch_bottle
@@ -28,7 +27,6 @@ from .start import _launch_bottle
def cmd_resume(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} resume", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument(
"identity",
help="bottle identity from a prior `start` (see its session-end output)",
@@ -42,7 +40,7 @@ def cmd_resume(argv: list[str]) -> int:
f"check ~/.bot-bottle/state/ or run `cli.py start` to create a new bottle"
)
manifest = Manifest.resolve(USER_CWD)
manifest = ManifestIndex.resolve(USER_CWD)
manifest.require_agent(metadata.agent_name)
spec = BottleSpec(
@@ -51,11 +49,11 @@ def cmd_resume(argv: list[str]) -> int:
copy_cwd=metadata.copy_cwd,
user_cwd=metadata.cwd or USER_CWD,
identity=metadata.identity,
bottle_names=tuple(metadata.bottle_names),
)
backend_name = metadata.backend or None
return _launch_bottle(
spec,
dry_run=args.dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
+188 -33
View File
@@ -20,18 +20,19 @@ from ..agent_provider import runtime_for
from ..backend import (
Bottle,
BottleSpec,
enumerate_active_agents,
get_bottle_backend,
known_backend_names,
)
from ..backend.docker import util as docker_mod
from ..backend.docker.bottle_plan import DockerBottlePlan
from ..backend.docker.bottle_state import (
from ..bottle_state import (
cleanup_state,
is_preserved,
mark_preserved,
)
from ..backend.docker.capability_apply import snapshot_transcript
from ..log import info
from ..manifest import Manifest
from ..manifest import Manifest, ManifestIndex
from ._common import PROG, USER_CWD, read_tty_line
from . import tui
@@ -39,15 +40,14 @@ from . import tui
def cmd_start(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into a derived image")
parser.add_argument("--remote-control", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
parser.add_argument(
"--backend",
choices=known_backend_names(),
default=None,
help=(
"backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND "
"or 'docker'). Overrides the env var when set."
"or host auto-selection). Overrides the env var when set."
),
)
parser.add_argument(
@@ -60,27 +60,38 @@ def cmd_start(argv: list[str]) -> int:
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
manifest = Manifest.resolve(USER_CWD)
manifest = ManifestIndex.resolve(USER_CWD)
agent_name: str | None = args.name
if agent_name is None:
agent_name = tui.filter_select(
sorted(manifest.agents.keys()),
manifest.all_agent_names,
title="Select agent",
)
if agent_name is None:
return 0
backend_name: str | None = args.backend
if backend_name is None and "BOT_BOTTLE_BACKEND" not in os.environ:
backend_name = tui.filter_select(
list(known_backend_names()),
title="Select backend",
)
if backend_name is None:
return 0
# Bottle multiselect: always show after agent selection so operators
# can compose bottles at launch time without editing agent manifests.
available_bottles = manifest.all_bottle_names
lineage_map = _bottle_lineage(manifest)
display_labels = [lineage_map.get(n, n) for n in available_bottles]
label_to_name = {lineage_map.get(n, n): n for n in available_bottles}
initial_bottle = _peek_agent_bottle(manifest, agent_name)
initial_labels = [lineage_map.get(initial_bottle, initial_bottle)] if initial_bottle else []
selected_labels = tui.filter_multiselect(
display_labels,
title="Select bottles",
initial=initial_labels,
)
if selected_labels is None:
return 0
bottle_names = tuple(label_to_name.get(lbl, lbl) for lbl in selected_labels)
label, color = tui.name_color_modal(default_label=agent_name)
label, color = _resolve_unique_label(label, color)
spec = BottleSpec(
manifest=manifest,
@@ -89,11 +100,11 @@ def cmd_start(argv: list[str]) -> int:
user_cwd=USER_CWD,
label=label,
color=color,
bottle_names=bottle_names,
)
return _launch_bottle(
spec,
dry_run=dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
@@ -114,8 +125,8 @@ def prepare_with_preflight(
injected callable, prompt y/N via the injected callable.
`backend_name` selects which backend prepares the plan
(`None` `$BOT_BOTTLE_BACKEND` `docker`). The CLI passes
whatever `--backend` resolved to.
(`None` `$BOT_BOTTLE_BACKEND` host auto-selection). The CLI
passes whatever `--backend` resolved to.
Returns `(plan, identity)`. `plan` is None on dry-run or
operator-N, but `identity` is set as soon as `backend.prepare`
@@ -138,8 +149,9 @@ def prepare_with_preflight(
def attach_agent(
bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
bottle: Bottle, *, resume: bool = False,
agent_provider_template: str = "claude",
startup_args: tuple[str, ...] = (),
) -> int:
"""Run the selected provider CLI inside `bottle` as an
interactive session. Blocks until the session ends; returns the
@@ -156,8 +168,7 @@ def attach_agent(
"(Ctrl-D or 'exit' to leave; container will be removed)"
)
agent_args = list(runtime.bypass_args)
if remote_control:
agent_args.extend(runtime.remote_control_args)
agent_args.extend(startup_args)
if resume:
agent_args.extend(runtime.resume_args)
return bottle.exec_agent(agent_args, tty=True)
@@ -172,7 +183,7 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
# instead of relying on each agent's transcript layout.
if not identity:
return
snapshot_transcript(identity)
# snapshot_transcript(identity)
if exit_code != 0:
mark_preserved(identity)
@@ -196,6 +207,53 @@ def _identity_from_plan(plan: object) -> str:
return getattr(plan, "slug", "")
def _peek_agent_bottle(manifest: ManifestIndex, agent_name: str) -> str:
"""Return the `bottle:` value from the named agent's frontmatter without
fully parsing the agent file, or "" when absent or unreadable.
Used to pre-populate the bottle multiselect with the agent's default
bottle so operators who haven't removed `bottle:` from their manifests
don't need to re-select it every time."""
if manifest.home_md is None:
# Eager mode (from_json_obj): agent is pre-parsed.
if agent_name in manifest.agents:
return manifest.agents[agent_name].bottle
return ""
from ..manifest_loader import scan_agent_names
from ..yaml_subset import YamlSubsetError, parse_frontmatter
home_agents = scan_agent_names(manifest.home_md / "agents")
cwd_agents: dict[str, Path] = {}
if manifest.cwd_md is not None:
cwd_agents = scan_agent_names(manifest.cwd_md / "agents")
merged = {**home_agents, **cwd_agents}
path = merged.get(agent_name)
if path is None:
return ""
try:
fm, _ = parse_frontmatter(path.read_text())
bottle = fm.get("bottle", "")
return str(bottle) if isinstance(bottle, str) else ""
except (OSError, YamlSubsetError):
return ""
def _resolve_unique_label(label: str, color: str) -> tuple[str, str]:
"""Re-prompt with a disclaimer until the label's slug is not already
in use among running bottles. Passes through unchanged when no
collision is found on the first check."""
while True:
slug_candidate = docker_mod.slugify(label)
active_slugs = {a.slug for a in enumerate_active_agents()}
if slug_candidate not in active_slugs:
return label, color
label, color = tui.name_color_modal(
default_label=label,
disclaimer=f'"{label}" is already in use',
)
def _text_prompt_yes() -> bool:
"""Default `prompt_yes` for CLI use: reads y/N from the
controlling tty via stderr prompt + tty-line read."""
@@ -205,17 +263,118 @@ def _text_prompt_yes() -> bool:
return reply in ("y", "Y", "yes", "YES")
def _text_render_preflight(*, remote_control: bool):
def _text_render_preflight():
def _render(plan: DockerBottlePlan) -> None:
plan.print(remote_control=remote_control)
print(file=sys.stderr)
print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
return _render
def _bottle_lineage(manifest: ManifestIndex) -> dict[str, str]:
"""Return {bottle_name: lineage_label} for bottles that have an extends chain.
Bottles without a parent are omitted (the caller falls back to the bare name).
Labels show the chain root-first: e.g. 'dev -> bot-bottle-dev -> claude-dev'."""
if manifest.home_md is None:
return {}
bottles_dir = manifest.home_md / "bottles"
if not bottles_dir.is_dir():
return {}
from ..yaml_subset import YamlSubsetError, parse_frontmatter
extends_of: dict[str, str] = {}
for path in bottles_dir.glob("*.md"):
try:
fm, _ = parse_frontmatter(path.read_text())
parent = fm.get("extends", "")
if isinstance(parent, str) and parent:
extends_of[path.stem] = parent
except (OSError, YamlSubsetError):
pass
labels: dict[str, str] = {}
for name in extends_of:
chain = [name]
seen = {name}
cur = name
while cur in extends_of:
par = extends_of[cur]
if par in seen:
break
chain.append(par)
seen.add(par)
cur = par
labels[name] = " -> ".join(reversed(chain))
return labels
def _manifest_to_yaml(manifest: Manifest) -> str:
"""Serialize the resolved Manifest to a YAML string for preflight display."""
lines: list[str] = []
agent = manifest.agent
lines.append("agent:")
if agent.skills:
lines.append(" skills:")
for s in agent.skills:
lines.append(f" - {s}")
if not agent.git_user.is_empty():
lines.append(" git-gate:")
lines.append(" user:")
if agent.git_user.name:
lines.append(f" name: {agent.git_user.name}")
if agent.git_user.email:
lines.append(f" email: {agent.git_user.email}")
bottle = manifest.bottle
lines.append("bottle:")
if bottle.agent_provider.template != "claude" or bottle.agent_provider.dockerfile:
lines.append(" agent_provider:")
lines.append(f" template: {bottle.agent_provider.template}")
if bottle.agent_provider.dockerfile:
lines.append(f" dockerfile: {bottle.agent_provider.dockerfile}")
if bottle.env:
lines.append(" env:")
for k, v in sorted(bottle.env.items()):
lines.append(f" {k}: {v}")
has_git_gate = not bottle.git_user.is_empty() or bottle.git
if has_git_gate:
lines.append(" git-gate:")
if not bottle.git_user.is_empty():
lines.append(" user:")
if bottle.git_user.name:
lines.append(f" name: {bottle.git_user.name}")
if bottle.git_user.email:
lines.append(f" email: {bottle.git_user.email}")
if bottle.git:
lines.append(" repos:")
for entry in bottle.git:
lines.append(f" {entry.Name}:")
lines.append(f" url: {entry.Upstream}")
if bottle.egress.routes:
lines.append(" egress:")
lines.append(" routes:")
for r in bottle.egress.routes:
lines.append(f" - host: {r.Host}")
if r.AuthScheme:
lines.append(f" auth:")
lines.append(f" scheme: {r.AuthScheme}")
lines.append(f" supervise: {'true' if bottle.supervise else 'false'}")
return "\n".join(lines)
def _launch_bottle(
spec: BottleSpec,
*,
dry_run: bool,
remote_control: bool,
backend_name: str | None = None,
) -> int:
"""Shared launch core for `start` and `resume`. Builds the plan,
@@ -227,7 +386,7 @@ def _launch_bottle(
plan, identity = prepare_with_preflight(
spec,
stage_dir=stage_dir,
render_preflight=_text_render_preflight(remote_control=remote_control),
render_preflight=_text_render_preflight(),
prompt_yes=_text_prompt_yes,
dry_run=dry_run,
backend_name=backend_name,
@@ -240,8 +399,8 @@ def _launch_bottle(
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
exit_code = attach_agent(
bottle,
remote_control=remote_control,
agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args,
)
info(
f"session ended (exit {exit_code}); "
@@ -249,12 +408,8 @@ def _launch_bottle(
)
# While the container is still alive: always snapshot the
# transcript and — if the agent exited non-zero — mark
# the state for preservation. Capability-block already
# did both before triggering teardown from the dashboard;
# this picks up crashes / Ctrl-Cs / OOM kills the same
# way. snapshot_transcript is best-effort so the
# capability-block path's prior snapshot isn't clobbered
# when the container is already gone.
# the state for preservation. This picks up crashes /
# Ctrl-Cs / OOM kills before cleanup removes the state dir.
if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code)
return 0
+83 -37
View File
@@ -2,8 +2,8 @@
act on them (approve / modify / reject).
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
approval handler wires to PRD 0016 (capability-block), which rebuilds
the bottle Dockerfile. The egress-block tool was removed in issue #198.
Egress proposals are queued for operator review as full routes.yaml
updates.
"""
from __future__ import annotations
@@ -20,12 +20,19 @@ from datetime import datetime, timezone
from pathlib import Path
from .. import supervise as _supervise
from ..backend.docker.bottle_state import read_metadata
from ..backend.docker.capability_apply import (
CapabilityApplyError,
apply_capability_change,
from ..bottle_state import read_metadata
from ..backend.docker.egress_apply import (
EgressApplyError,
applicator as _docker_applicator,
)
from ..backend.macos_container.egress_apply import (
applicator as _macos_applicator,
)
from ..backend.smolmachines.egress_apply import (
applicator as _smolmachines_applicator,
)
from ..log import Die, error, info
from ..supervise import (
COMPONENT_FOR_TOOL,
AuditEntry,
@@ -34,8 +41,10 @@ from ..supervise import (
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOL_CAPABILITY_BLOCK,
archive_proposal,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
list_pending_proposals,
render_diff,
write_audit_entry,
@@ -46,6 +55,11 @@ from ._common import PROG
_REFRESH_INTERVAL_MS = 1000
# Proposal tools whose payload is a read-only report, not a file the operator
# edits: modify is unavailable and approval requires a recorded reason for the
# audit trail.
_REPORT_ONLY_TOOLS: tuple[str, ...] = (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW)
@dataclass(frozen=True)
class QueuedProposal:
@@ -58,7 +72,17 @@ class QueuedProposal:
# Errors any remediation engine may raise. Caught by the TUI key
# handlers and surfaced in the status line so a failed apply keeps
# the proposal pending rather than crashing curses.
ApplyError = (CapabilityApplyError,)
ApplyError = (EgressApplyError,)
def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
meta = read_metadata(slug)
backend = meta.backend if meta is not None else ""
if backend == "macos-container":
return _macos_applicator.apply_routes_change(slug, content)
if backend == "smolmachines":
return _smolmachines_applicator.apply_routes_change(slug, content)
return _docker_applicator.apply_routes_change(slug, content)
def discover_pending() -> list[QueuedProposal]:
@@ -108,8 +132,10 @@ def _detail_lines(
def _suffix_for_tool(tool: str) -> str:
if tool == TOOL_CAPABILITY_BLOCK:
return ".dockerfile"
if tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
return ".yaml"
if tool in (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_ALLOW):
return ".txt"
return ".txt"
@@ -127,16 +153,10 @@ def approve(
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
diff_before, diff_after = "", ""
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
_meta = read_metadata(qp.proposal.bottle_slug)
if _meta is not None and not _meta.compose_project:
raise CapabilityApplyError(
"capability-block remediation is not supported for smolmachines "
"bottles. Reject this proposal or handle the capability change "
"manually, then restart the bottle."
)
diff_before, diff_after = apply_capability_change(
qp.proposal.bottle_slug, file_to_apply,
if qp.proposal.tool in (TOOL_EGRESS_ALLOW, TOOL_EGRESS_BLOCK):
diff_before, diff_after = apply_routes_change(
qp.proposal.bottle_slug,
file_to_apply,
)
response = Response(
@@ -150,9 +170,6 @@ def approve(
qp, action=status, notes=notes,
diff_before=diff_before, diff_after=diff_after,
)
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
archive_proposal(qp.queue_dir, qp.proposal.id)
def reject(qp: QueuedProposal, *, reason: str) -> None:
"""Write a rejection response and an audit entry."""
@@ -166,6 +183,23 @@ def reject(qp: QueuedProposal, *, reason: str) -> None:
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
def _approve_from_tui(
stdscr: "curses._CursesWindow", # type: ignore
qp: QueuedProposal,
*,
final_file: str | None = None,
notes: str = "",
) -> str:
"""Approve from curses, prompting for any tool-specific audit note."""
if qp.proposal.tool in _REPORT_ONLY_TOOLS and final_file is None:
notes = _prompt(stdscr, "allow reason (false positive / legitimately needed): ")
if not notes:
return "approve aborted (empty reason)"
approve(qp, final_file=final_file, notes=notes)
verb = "modified+approved" if final_file is not None else "approved"
return _approval_status(qp, verb)
def _write_audit(
qp: QueuedProposal,
*,
@@ -237,7 +271,10 @@ def cmd_supervise(argv: list[str]) -> int:
return e.code if isinstance(e.code, int) else 1
except Exception as e: # noqa: W0718 — catch supervise crash for logging
log_path = _write_crash_log(e)
error(f"supervise crashed: {type(e).__name__}: {e}")
error(
f"supervise crashed: {type(e).__name__}: {e}",
context={"error_type": type(e).__name__, "crash_log": str(log_path)},
)
error(f"full traceback written to {log_path}")
return 1
return 0
@@ -282,7 +319,7 @@ def _list_once() -> int:
return 0
def _try_init_green() -> int:
def _try_init_green() -> int: # pragma: no cover
"""Initialise a green color pair and return its attr, or 0."""
try:
curses.start_color()
@@ -293,7 +330,7 @@ def _try_init_green() -> int:
return 0
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore # pragma: no cover
curses.curs_set(0)
stdscr.timeout(_REFRESH_INTERVAL_MS)
green_attr = _try_init_green()
@@ -349,18 +386,22 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
_detail_view(stdscr, qp, green_attr=green_attr)
elif key == ord("a"):
try:
approve(qp)
status_line = _approval_status(qp, "approved")
status_line = _approve_from_tui(stdscr, qp)
except ApplyError as e:
status_line = f"apply failed: {e}"
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
status_line = f"modify unavailable for {qp.proposal.tool}"
continue
edited = _modify(stdscr, qp)
if edited is None:
status_line = "modify aborted (no change)"
else:
try:
approve(qp, final_file=edited, notes="operator modified before approving")
status_line = _approval_status(qp, "modified+approved")
status_line = _approve_from_tui(
stdscr, qp, final_file=edited,
notes="operator modified before approving",
)
except ApplyError as e:
status_line = f"apply failed: {e}"
elif key == ord("r"):
@@ -379,7 +420,7 @@ def _render(
status_line: str,
*,
green_attr: int = 0, # noqa: F841 — unused, but required by interface
) -> None:
) -> None: # pragma: no cover
stdscr.erase()
h, w = stdscr.getmaxyx()
header = f"bot-bottle supervise ({len(pending)} pending)"
@@ -430,7 +471,7 @@ def _detail_view(
qp: QueuedProposal,
*,
green_attr: int = 0,
) -> None:
) -> None: # pragma: no cover
"""Render the full proposal. Scrollable. Press q to return."""
lines = _detail_lines(qp, green_attr=green_attr)
offset = 0
@@ -458,15 +499,20 @@ def _detail_view(
offset = max(0, len(lines) - 1)
elif key == ord("a"):
try:
approve(qp)
_approve_from_tui(stdscr, qp)
except ApplyError:
pass
return
elif key == ord("m"):
if qp.proposal.tool in _REPORT_ONLY_TOOLS:
return
edited = _modify(stdscr, qp)
if edited is not None:
try:
approve(qp, final_file=edited, notes="operator modified before approving")
_approve_from_tui(
stdscr, qp, final_file=edited,
notes="operator modified before approving",
)
except ApplyError:
pass
return
@@ -477,7 +523,7 @@ def _detail_view(
return
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None: # type: ignore # pragma: no cover
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
suffix = _suffix_for_tool(qp.proposal.tool)
curses.endwin()
@@ -488,7 +534,7 @@ def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
return edited
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str: # type: ignore # pragma: no cover
"""One-line input at the bottom of the screen."""
curses.curs_set(1)
h, _ = stdscr.getmaxyx()
+311 -19
View File
@@ -17,6 +17,43 @@ import sys
from typing import Any, Optional
def filter_multiselect(
items: list[str],
*,
title: str = "",
initial: Optional[list[str]] = None,
tty_path: str = "/dev/tty",
) -> Optional[list[str]]:
"""Render a multi-select picker over *items*.
Returns the ordered list of selected items, or ``None`` if the user
cancelled (Esc / ``q`` / Ctrl-C / Ctrl-D with no items).
Press Space to toggle the item under the cursor.
Press Enter to confirm the current selection.
Press Ctrl-D to confirm the current selection (returns even if empty).
Press Esc/q to cancel (returns None).
*initial* pre-populates the selection in insertion order. Items
added are appended; removed items leave the remaining order unchanged.
"""
if not items:
return []
try:
tty_fd = open(tty_path, "r+b", buffering=0)
except OSError:
return None
try:
fd_dup = os.dup(tty_fd.fileno())
return _run_multiselect(
items, title=title, initial=list(initial or []), tty_fd=fd_dup
)
finally:
tty_fd.close()
def filter_select(
items: list[str],
*,
@@ -221,25 +258,275 @@ def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.
pass
# ---------------------------------------------------------------------------
# filter_multiselect internals
# ---------------------------------------------------------------------------
_KEY_SPACE = 32
def _run_multiselect(
items: list[str], *, title: str, initial: list[str], tty_fd: int
) -> Optional[list[str]]:
"""Drive a curses multi-select session on *tty_fd*."""
os.environ.setdefault("TERM", "xterm-256color")
orig_stdin = sys.__stdin__
orig_stdout = sys.__stdout__
try:
import io
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode='r+'), write_through=True)
sys.__stdin__ = tty_text # type: ignore[assignment]
sys.__stdout__ = tty_text # type: ignore[assignment]
screen = curses.initscr()
curses.noecho()
curses.cbreak()
screen.keypad(True)
try:
result = _multiselect_loop(screen, items, title=title, initial=initial)
finally:
screen.keypad(False)
curses.nocbreak()
curses.echo()
curses.endwin()
except Exception: # noqa: W0718
return None
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
sys.__stdout__ = orig_stdout # type: ignore[assignment]
return result
def _multiselect_loop(
screen: Any, items: list[str], *, title: str, initial: list[str]
) -> Optional[list[str]]:
query = ""
cursor = 0
selected: list[str] = [s for s in initial if s in items]
# focus = "filter": navigate + toggle items in the filterable list
# focus = "order": navigate + reorder items in the selected list
focus = "filter"
order_cursor = 0
while True:
filtered = _filter_items(items, query)
if not filtered:
cursor = 0
elif cursor >= len(filtered):
cursor = len(filtered) - 1
if not selected:
order_cursor = 0
if focus == "order":
focus = "filter"
elif order_cursor >= len(selected):
order_cursor = len(selected) - 1
try:
_render_multiselect(
screen, filtered, cursor,
query=query, title=title, selected=selected,
focus=focus, order_cursor=order_cursor,
)
except curses.error:
return None
try:
key = screen.getch()
except KeyboardInterrupt:
return None
if key in (_KEY_ESC, _KEY_CTRL_C, ord("q")):
return None
if key == _KEY_CTRL_D:
return list(selected)
# Tab toggles between filter and order focus.
if key == ord("\t"):
if focus == "filter" and selected:
focus = "order"
order_cursor = 0
else:
focus = "filter"
continue
if focus == "filter":
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
return list(selected)
elif key == _KEY_SPACE:
if filtered:
item = filtered[cursor]
if item in selected:
selected.remove(item)
else:
selected.append(item)
elif key in (curses.KEY_UP, ord("k")):
if cursor > 0:
cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")):
if cursor < len(filtered) - 1:
cursor += 1
elif key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
query = query[:-1]
new_filtered = _filter_items(items, query)
if cursor >= len(new_filtered):
cursor = max(0, len(new_filtered) - 1)
elif 32 <= key <= 126 and key != _KEY_SPACE:
query += chr(key)
cursor = 0
else: # focus == "order"
if key in (curses.KEY_UP, ord("k")):
if order_cursor > 0:
order_cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")):
if order_cursor < len(selected) - 1:
order_cursor += 1
elif key == ord("K"):
# Move selected item up (earlier in order).
if order_cursor > 0:
i = order_cursor
selected[i - 1], selected[i] = selected[i], selected[i - 1]
order_cursor -= 1
elif key == ord("J"):
# Move selected item down (later in order).
if order_cursor < len(selected) - 1:
i = order_cursor
selected[i], selected[i + 1] = selected[i + 1], selected[i]
order_cursor += 1
elif key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r"), _KEY_SPACE):
# Remove item from selection while in order mode.
del selected[order_cursor]
if order_cursor >= len(selected) and order_cursor > 0:
order_cursor -= 1
def _render_multiselect(
screen: Any,
filtered: list[str],
cursor: int,
*,
query: str,
title: str,
selected: list[str],
focus: str = "filter",
order_cursor: int = 0,
) -> None:
screen.erase()
rows, cols = screen.getmaxyx()
min_rows = 7
if rows < min_rows:
raise curses.error("terminal too small")
sep = "" * min(cols - 1, 40)
row = 0
if title and row < rows - 1:
_addstr_safe(screen, row, 0, title[:cols - 1], curses.A_BOLD)
row += 1
# Filter line — dim when focus is on the order panel.
filter_label = f"Filter: {query}"
filter_hint = " [Tab: reorder]" if focus == "filter" and selected else ""
filter_attr = curses.A_DIM if focus == "order" else curses.A_NORMAL
if row < rows - 1:
_addstr_safe(screen, row, 0, (filter_label + filter_hint)[:cols - 1], filter_attr)
row += 1
if row < rows - 1:
_addstr_safe(screen, row, 0, sep)
row += 1
# Compute how many rows the bottom order panel needs.
# Cap the visible selected list to keep the filter list legible.
order_rows = min(len(selected), max(1, (rows - row) // 3)) if selected else 0
# Bottom reserved: sep + order_rows + sep + help = order_rows + 3
bottom_reserved = order_rows + 3
list_start = row
list_rows = rows - list_start - bottom_reserved
if list_rows < 1:
list_rows = 1
selected_set = set(selected)
filter_dim = focus == "order"
scroll = max(0, cursor - list_rows + 1)
visible = filtered[scroll: scroll + list_rows]
for idx, item in enumerate(visible):
abs_idx = scroll + idx
mark = "[*]" if item in selected_set else "[ ]"
prefix = "> " if (abs_idx == cursor and focus == "filter") else " "
line = (prefix + mark + " " + item)[:cols - 1]
item_attr = curses.A_DIM if filter_dim else (
curses.A_REVERSE if abs_idx == cursor else curses.A_NORMAL
)
if row < rows - bottom_reserved:
_addstr_safe(screen, row, 0, line, item_attr)
row += 1
# Separator before the order panel.
if row < rows - (order_rows + 2):
_addstr_safe(screen, row, 0, sep)
row += 1
# Order panel.
order_scroll = max(0, order_cursor - order_rows + 1)
order_visible = selected[order_scroll: order_scroll + order_rows]
for idx, item in enumerate(order_visible):
abs_idx = order_scroll + idx
is_active = focus == "order" and abs_idx == order_cursor
prefix = "> " if is_active else " "
line = (prefix + item)[:cols - 1]
attr = curses.A_REVERSE if is_active else curses.A_NORMAL
if row < rows - 2:
_addstr_safe(screen, row, 0, line, attr)
row += 1
if row < rows - 1:
_addstr_safe(screen, row, 0, sep)
row += 1
if focus == "filter":
help_line = "[↑↓/jk] move [Space] toggle [Enter] confirm [Tab] reorder [Esc/q] cancel"
else:
help_line = "[↑↓/jk] cursor [K/J] reorder [Space/Enter] remove [Tab] back [Ctrl-D] done"
if row < rows:
_addstr_safe(screen, min(rows - 1, row), 0, help_line[:cols - 1])
screen.refresh()
# ---------------------------------------------------------------------------
# name_color_modal — two-step label + color picker
# ---------------------------------------------------------------------------
_ANSI_COLORS = [
"red", "green", "blue", "yellow", "magenta", "cyan", "white", "black",
"bright-red", "bright-green", "bright-blue", "bright-yellow",
"bright-magenta", "bright-cyan", "bright-white", "bright-black",
"red", "green", "yellow", "blue", "magenta",
]
_CURSES_COLOR_MAP: dict[str, int] = {
"black": curses.COLOR_BLACK,
"red": curses.COLOR_RED,
"green": curses.COLOR_GREEN,
"yellow": curses.COLOR_YELLOW,
"blue": curses.COLOR_BLUE,
"magenta": curses.COLOR_MAGENTA,
"cyan": curses.COLOR_CYAN,
"white": curses.COLOR_WHITE,
}
_COLOR_NONE = "(none)"
@@ -248,11 +535,15 @@ _COLOR_NONE = "(none)"
def name_color_modal(
default_label: str,
*,
disclaimer: str = "",
tty_path: str = "/dev/tty",
) -> tuple[str, str]:
"""Present a two-step curses modal: first edit the agent label,
then optionally pick a color.
``disclaimer`` is shown below the input field use it to surface
an error from a previous attempt (e.g. name already in use).
Returns ``(label, color)`` where ``color`` is one of the 16 ANSI
color name strings or ``""`` for no color. Falls back to
``(default_label, "")`` on any error (terminal too small, not a tty).
@@ -264,14 +555,14 @@ def name_color_modal(
try:
fd_dup = os.dup(tty_fd.fileno())
return _run_name_color(default_label, tty_fd=fd_dup)
return _run_name_color(default_label, tty_fd=fd_dup, disclaimer=disclaimer)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return default_label, ""
finally:
tty_fd.close()
def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
def _run_name_color(default_label: str, *, tty_fd: int, disclaimer: str = "") -> tuple[str, str]:
import io
orig_stdin = sys.__stdin__
orig_stdout = sys.__stdout__
@@ -286,7 +577,7 @@ def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
curses.cbreak()
screen.keypad(True)
try:
label = _label_step(screen, default_label)
label = _label_step(screen, default_label, disclaimer=disclaimer)
color = _color_step(screen, label)
finally:
screen.keypad(False)
@@ -299,14 +590,14 @@ def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
return label, color
def _label_step(screen: Any, default_label: str) -> str:
def _label_step(screen: Any, default_label: str, *, disclaimer: str = "") -> str:
"""Step 1: edit the label. First printable key replaces the
pre-fill; subsequent keys append. Enter confirms."""
text = default_label
replaced = False # True once the user has typed their first char
while True:
_render_label(screen, text)
_render_label(screen, text, disclaimer=disclaimer)
try:
key = screen.getch()
except KeyboardInterrupt:
@@ -330,7 +621,7 @@ def _label_step(screen: Any, default_label: str) -> str:
text += chr(key)
def _render_label(screen: Any, text: str) -> None:
def _render_label(screen: Any, text: str, *, disclaimer: str = "") -> None:
screen.erase()
rows, cols = screen.getmaxyx()
sep = "" * min(cols - 1, 40)
@@ -338,8 +629,12 @@ def _render_label(screen: Any, text: str) -> None:
_addstr_safe(screen, 1, 0, sep)
_addstr_safe(screen, 2, 0, text[:cols - 1], curses.A_REVERSE)
_addstr_safe(screen, 3, 0, sep)
if rows > 5:
_addstr_safe(screen, 5, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
row = 4
if disclaimer and rows > row + 1:
_addstr_safe(screen, row, 0, disclaimer[:cols - 1], curses.A_BOLD)
row += 1
if rows > row + 1:
_addstr_safe(screen, row, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
screen.refresh()
@@ -379,13 +674,10 @@ def _init_color_pairs() -> dict[str, int]:
curses.use_default_colors()
pair_idx = 2 # pair 1 reserved for other uses
for name in _ANSI_COLORS:
base = name.replace("bright-", "")
fg = _CURSES_COLOR_MAP.get(base, curses.COLOR_WHITE)
fg = _CURSES_COLOR_MAP.get(name, curses.COLOR_WHITE)
try:
curses.init_pair(pair_idx, fg, -1)
attr = curses.color_pair(pair_idx)
if name.startswith("bright-"):
attr |= curses.A_BOLD
attr = curses.color_pair(pair_idx) | curses.A_BOLD
attrs[name] = attr
pair_idx += 1
except curses.error:
@@ -21,7 +21,7 @@ FROM node:22-slim
# to it) works against egress's bumped TLS without the agent needing
# local DNS.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl \
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by claude-code itself
@@ -36,7 +36,7 @@ RUN apt-get update \
# build (`claude --version` returns 2.1.126). Bump deliberately when
# rolling forward; an unpinned install would mean rebuilds silently pick
# up new behavior.
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.126 \
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
&& npm cache clean --force
# Run as a non-root user. The node image already provides a `node` user
+103 -16
View File
@@ -17,9 +17,12 @@ from typing import TYPE_CHECKING
from ...agent_provider import (
AgentProvider,
AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...backend.docker import util as docker_mod
from ...egress import EgressRoute
from ...log import die, info, warn
@@ -28,8 +31,6 @@ if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_REPO_ROOT = Path(__file__).resolve().parents[3]
_SUPERVISE_MCP_NAME = "supervise"
@@ -40,15 +41,56 @@ def _skills_dir(guest_home: str) -> str:
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
_STATUS_LINE_COLORS = {
"red": "\033[91m",
"green": "\033[92m",
"yellow": "\033[93m",
"blue": "\033[94m",
"magenta": "\033[95m",
}
_CLAUDE_THEME_COLORS = {
"red": "redBright",
"green": "greenBright",
"yellow": "yellowBright",
"blue": "blueBright",
"magenta": "magentaBright",
}
def _status_line_script(label: str, color: str) -> str:
if not label:
return "#!/bin/sh\nprintf '\\n'\n"
label_q = shlex.quote(label)
if color and color in _STATUS_LINE_COLORS:
return (
"#!/bin/sh\n"
f"printf '%b%s%b\\n' '{_STATUS_LINE_COLORS[color]}' {label_q} '\\033[0m'\n"
)
return f"#!/bin/sh\nprintf '%s\\n' {label_q}\n"
def _custom_theme_payload(color: str) -> dict[str, object] | None:
theme_color = _CLAUDE_THEME_COLORS.get(color)
if not theme_color:
return None
return {
"name": f"Bot-bottle {color}",
"base": "dark",
"overrides": {
"claude": f"ansi:{theme_color}",
},
}
_RUNTIME = AgentProviderRuntime(
template="claude",
command="claude",
image="bot-bottle-claude:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",),
remote_control_args=("--remote-control",),
)
@@ -62,7 +104,8 @@ class ClaudeAgentProvider(AgentProvider):
*,
dockerfile: str,
state_dir: Path,
guest_home: str,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
@@ -70,15 +113,22 @@ class ClaudeAgentProvider(AgentProvider):
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del forward_host_credentials, host_env # Codex-only knobs
del forward_host_credentials, host_env
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = {
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
"DISABLE_ERROR_REPORTING": "1",
}
dirs = (
AgentProvisionDir(f"{guest_home}/.claude"),
AgentProvisionDir(f"{guest_home}/.claude/themes"),
)
claude_config = state_dir / "claude.json"
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
@@ -88,15 +138,45 @@ class ClaudeAgentProvider(AgentProvider):
"bypassPermissionsModeAccepted": True,
"projects": claude_projects,
}
if label:
payload["name"] = label
if color:
payload["color"] = color
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
claude_config.chmod(0o600)
files = (
files = [
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
)
]
claude_settings = state_dir / "claude-settings.json"
claude_settings_payload: dict[str, object] = {}
if label or color:
statusline_script = state_dir / "claude-statusline.sh"
statusline_script.write_text(_status_line_script(label, color))
statusline_script.chmod(0o755)
files.append(AgentProvisionFile(
statusline_script,
f"{guest_home}/.claude/statusline.sh",
mode="755",
))
claude_settings_payload["statusLine"] = {
"type": "command",
"command": "~/.claude/statusline.sh",
}
theme_payload = _custom_theme_payload(color)
if theme_payload is not None:
theme_name = f"bot-bottle-{docker_mod.slugify(label or color)}"
theme_file = state_dir / f"{theme_name}.json"
theme_file.write_text(json.dumps(theme_payload, indent=2) + "\n")
theme_file.chmod(0o644)
files.append(AgentProvisionFile(
theme_file,
f"{guest_home}/.claude/themes/{theme_name}.json",
))
claude_settings_payload["theme"] = f"custom:{theme_name}"
if claude_settings_payload:
claude_settings.write_text(json.dumps(claude_settings_payload, indent=2) + "\n")
claude_settings.chmod(0o600)
files.append(AgentProvisionFile(
claude_settings,
f"{guest_home}/.claude/settings.json",
))
egress_routes = (EgressRoute(
host="api.anthropic.com",
auth_scheme="Bearer" if auth_token else "",
@@ -107,15 +187,22 @@ class ClaudeAgentProvider(AgentProvider):
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
env_vars=env_vars,
guest_env=resolved_guest_env,
files=files,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=dirs,
files=tuple(files),
egress_routes=egress_routes,
hidden_env_names=hidden_env_names,
)
@@ -126,7 +213,7 @@ class ClaudeAgentProvider(AgentProvider):
when the agent has no skills."""
from ...backend.util import host_skill_dir
agent = plan.spec.manifest.agents[plan.spec.agent_name]
agent = plan.manifest.agent
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
@@ -155,8 +242,8 @@ class ClaudeAgentProvider(AgentProvider):
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
user="root",
)
agent = plan.spec.manifest.agents[plan.spec.agent_name]
return prompt_path if agent.prompt else None
agent = plan.manifest.agent
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
"""Apply the claude-side declarative provision steps from
@@ -1,12 +1,12 @@
# bot-bottle Codex provider image.
#
# Mirrors the default Claude image shape: Node LTS, git/network tooling,
# non-root node user, and the provider CLI installed globally.
# non-root node user, and the provider CLI installed for that user.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl \
&& apt-get install -y --no-install-recommends git ca-certificates curl procps ripgrep \
&& rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by codex itself
@@ -17,12 +17,15 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
&& npm cache clean --force
USER node
WORKDIR /home/node
RUN mkdir -p /home/node/.codex
ENV PATH="/home/node/.local/bin:${PATH}"
# Remote-control support requires the standalone Codex install layout
# under ~/.codex/packages/standalone/current. The npm package can run
# the TUI, but remote-control commands expect this installer-owned path.
RUN mkdir -p /home/node/.codex \
&& curl -fsSL https://chatgpt.com/codex/install.sh | sh
CMD ["codex"]
+26 -13
View File
@@ -18,10 +18,11 @@ from ...agent_provider import (
CODEX_HOST_CREDENTIAL_HOSTS,
AgentProvider,
AgentProviderRuntime,
AgentProvisionCommand,
AgentProvisionDir,
AgentProvisionCommand,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
@@ -32,8 +33,6 @@ if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_REPO_ROOT = Path(__file__).resolve().parents[3]
_SUPERVISE_MCP_NAME = "supervise"
@@ -48,15 +47,14 @@ def _skills_dir(guest_home: str) -> str:
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
_RUNTIME = AgentProviderRuntime(
template="codex",
command="codex",
image="bot-bottle-codex:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"),
remote_control_args=(),
)
@@ -70,7 +68,8 @@ class CodexAgentProvider(AgentProvider):
*,
dockerfile: str,
state_dir: Path,
guest_home: str,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
@@ -78,9 +77,12 @@ class CodexAgentProvider(AgentProvider):
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, label, color # Claude-only knobs
del auth_token, label, color
resolved_guest_env = dict(guest_env or {})
startup_args = provider_startup_args(provider_settings)
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = {
@@ -102,6 +104,11 @@ class CodexAgentProvider(AgentProvider):
config_file.write_text(
f'[projects."{toml_path}"]\n'
'trust_level = "trusted"\n'
"\n"
"[tui]\n"
'status_line = ["model-with-reasoning"]\n'
'terminal_title = ["spinner", "project"]\n'
'theme = "ansi"\n'
)
config_file.chmod(0o600)
files.append(AgentProvisionFile(config_file, config_path))
@@ -144,14 +151,20 @@ class CodexAgentProvider(AgentProvider):
"guest, but Codex did not accept it"
)))
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
env_vars=env_vars,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=startup_args,
dirs=tuple(dirs),
files=tuple(files),
pre_copy=tuple(pre_copy),
@@ -166,7 +179,7 @@ class CodexAgentProvider(AgentProvider):
skills."""
from ...backend.util import host_skill_dir
agent = plan.spec.manifest.agents[plan.spec.agent_name]
agent = plan.manifest.agent
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
@@ -195,8 +208,8 @@ class CodexAgentProvider(AgentProvider):
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
user="root",
)
agent = plan.spec.manifest.agents[plan.spec.agent_name]
return prompt_path if agent.prompt else None
agent = plan.manifest.agent
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
"""Apply the codex-side declarative provision steps from
@@ -250,8 +263,8 @@ class CodexAgentProvider(AgentProvider):
return
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
r = bottle.exec(
f"codex mcp add --transport http "
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
f"codex mcp add {_SUPERVISE_MCP_NAME} --url "
f"{shlex.quote(supervise_url)}",
user="node",
)
if r.returncode != 0:
@@ -259,7 +272,7 @@ class CodexAgentProvider(AgentProvider):
f"`codex mcp add supervise` failed (exit {r.returncode}): "
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
f"register manually with: "
f"codex mcp add --transport http supervise {supervise_url}"
f"codex mcp add supervise --url {shlex.quote(supervise_url)}"
)
@@ -2,7 +2,13 @@
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
them using the Gitea deploy-key HTTP API. No new Python dependencies
only stdlib `urllib.request` and `subprocess`."""
only stdlib `urllib.request` and `subprocess`.
Required token permissions (Gitea "Applications" "Generate Token"):
- Repository: Read & Write
Grants POST /api/v1/repos/{owner}/{repo}/keys (create deploy key)
and DELETE /api/v1/repos/{owner}/{repo}/keys/{id} (revoke deploy key).
No other scopes are needed."""
from __future__ import annotations
@@ -13,7 +19,12 @@ import urllib.error
import urllib.request
from pathlib import Path
from ...deploy_key_provisioner import DeployKeyProvisioner
from ...deploy_key_provisioner import DeployKeyCollisionError, DeployKeyProvisioner
# Timeout for ssh-keygen and Gitea API HTTP calls. A hung Gitea instance at
# prepare time would stall bottle launch indefinitely without this bound.
_API_TIMEOUT_SECS = 30
_KEYGEN_TIMEOUT_SECS = 10
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
@@ -40,6 +51,7 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=_KEYGEN_TIMEOUT_SECS,
)
private_key = key_path.read_bytes()
public_key = key_path.with_suffix(".pub").read_text().strip()
@@ -61,10 +73,15 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
method="POST",
)
try:
with urllib.request.urlopen(req) as resp:
with urllib.request.urlopen(req, timeout=_API_TIMEOUT_SECS) as resp:
body = json.loads(resp.read())
except urllib.error.HTTPError as exc:
_body = _read_error_body(exc)
if exc.code == 422:
raise DeployKeyCollisionError(
f"deploy key collision for {owner_repo!r} "
f"(title={title!r}): key title or content already registered — {_body}"
) from exc
raise RuntimeError(
f"failed to create deploy key for {owner_repo}: "
f"HTTP {exc.code}{_body}"
@@ -87,7 +104,7 @@ class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
method="DELETE",
)
try:
with urllib.request.urlopen(req):
with urllib.request.urlopen(req, timeout=_API_TIMEOUT_SECS):
pass
except urllib.error.HTTPError as exc:
if exc.code == 404:
+41
View File
@@ -0,0 +1,41 @@
# bot-bottle Pi provider image.
#
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
fd-find \
ripgrep \
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
&& npm cache clean --force
RUN mkdir -p /home/node/.pi/agent \
/home/node/.pi/context-mode/sessions \
/tmp/pi-subagents-uid-1000 \
&& chown -R node:node /home/node/.pi /tmp \
&& chmod -R u+rwX /tmp \
&& chown root:root /tmp /var/tmp \
&& chmod 1777 /tmp /var/tmp
USER node
WORKDIR /home/node
RUN pi install npm:@harms-haus/pi-cwd \
&& pi install npm:pi-web-access \
&& pi install npm:context-mode \
&& pi install npm:pi-subagents \
&& pi install npm:pi-mcp-adapter
CMD ["pi"]
+1
View File
@@ -0,0 +1 @@
"""Pi agent provider package."""
+321
View File
@@ -0,0 +1,321 @@
"""Pi agent provider plugin (PRD 0058, contrib).
Pi uses ~/.pi/agent/models.json for custom provider/model settings.
This provider writes an Ollama-compatible default configuration and
lets bottles override the model endpoint and model ids via
agent_provider.settings.
"""
from __future__ import annotations
import json
import os
import shlex
from pathlib import Path
from typing import TYPE_CHECKING
from urllib.parse import urlparse
from ...agent_provider import (
AgentProvider,
AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
provider_startup_args,
)
from ...egress import EgressRoute
from ...log import die, info
if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_DEFAULT_BASE_URL = "http://ollama:11434/v1"
_DEFAULT_MODEL = "qwen2.5-coder:7b"
_DEFAULT_PROVIDER_NAME = "ollama"
_DEFAULT_CONTEXT_WINDOW = 4096
_DEFAULT_MAX_TOKENS = 1024
def _skills_dir(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/skills"
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
def _append_system_path(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/APPEND_SYSTEM.md"
def _models_path(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/models.json"
def _runtime_state_repair_script(guest_home: str) -> str:
home = shlex.quote(guest_home)
pi_home = shlex.quote(f"{guest_home}/.pi")
context_sessions = shlex.quote(f"{guest_home}/.pi/context-mode/sessions")
return (
f"mkdir -p {context_sessions} /tmp/pi-subagents-uid-1000 && "
f"chown node:node {home} && "
f"chown -R node:node {pi_home} /tmp && "
"chmod -R u+rwX /tmp && "
f"chmod 755 {home} && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp"
)
def _settings_value(
settings: dict[str, object],
key: str,
default: object,
) -> object:
value = settings.get(key)
return default if value is None else value
def _settings_int(
settings: dict[str, object],
key: str,
default: int,
) -> int:
value = _settings_value(settings, key, default)
if isinstance(value, bool):
return default
if isinstance(value, (int, str)):
return int(value)
return default
def _pi_models_json(
settings: dict[str, object],
) -> tuple[dict[str, object], str, str, list[str], str]:
provider_name = str(
_settings_value(settings, "provider", _DEFAULT_PROVIDER_NAME)
)
base_url = str(_settings_value(settings, "base_url", _DEFAULT_BASE_URL))
api = str(_settings_value(settings, "api", "openai-completions"))
api_key = settings.get("api_key")
api_key_env = str(settings.get("api_key_env", ""))
models_raw = _settings_value(settings, "models", [_DEFAULT_MODEL])
models = [str(model) for model in models_raw] # type: ignore[union-attr]
supports_developer_role = bool(
_settings_value(settings, "supports_developer_role", False)
)
supports_reasoning_effort = bool(
_settings_value(settings, "supports_reasoning_effort", False)
)
max_tokens_field = str(
_settings_value(settings, "max_tokens_field", "max_tokens")
)
context_window = _settings_int(
settings, "context_window", _DEFAULT_CONTEXT_WINDOW,
)
max_tokens = _settings_int(settings, "max_tokens", _DEFAULT_MAX_TOKENS)
input_context_window = max(1, context_window - max_tokens)
provider: dict[str, object] = {
"baseUrl": base_url,
"api": api,
"compat": {
"supportsDeveloperRole": supports_developer_role,
"supportsReasoningEffort": supports_reasoning_effort,
"maxTokensField": max_tokens_field,
},
"models": [
{
"id": model,
"name": model,
"contextWindow": input_context_window,
"maxTokens": max_tokens,
}
for model in models
],
}
if api_key is not None:
provider["apiKey"] = str(api_key)
elif api_key_env:
provider["apiKey"] = "egress-placeholder"
elif provider_name == _DEFAULT_PROVIDER_NAME:
provider["apiKey"] = "ollama"
payload: dict[str, object] = {
"providers": {
provider_name: provider,
}
}
return payload, base_url, api_key_env, models, provider_name
def _route_host(base_url: str) -> str:
parsed = urlparse(base_url)
if not parsed.scheme or not parsed.hostname:
die(
"agent provider provisioning: pi settings base_url must be an "
f"absolute URL (was {base_url!r})"
)
return parsed.hostname
_RUNTIME = AgentProviderRuntime(
template="pi",
command="pi",
image="bot-bottle-pi:latest",
prompt_mode="append_system_prompt",
bypass_args=(),
resume_args=(),
)
class PiAgentProvider(AgentProvider):
@property
def runtime(self) -> AgentProviderRuntime:
return _RUNTIME
def provision_plan(
self,
*,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, forward_host_credentials, host_env, trusted_project_path
del label, color
resolved_guest_env = dict(guest_env or {})
guest_home = self.guest_home
settings = dict(provider_settings or {})
models_payload, base_url, api_key_env, models, provider_name = (
_pi_models_json(settings)
)
extra_startup_args = provider_startup_args(provider_settings)
models_file = state_dir / "pi-models.json"
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
models_file.chmod(0o600)
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
auth_scheme = "Bearer" if api_key_env else ""
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=(
"--models",
",".join(f"{provider_name}/{model}" for model in models),
*extra_startup_args,
),
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
egress_routes=(EgressRoute(
host=_route_host(base_url),
auth_scheme=auth_scheme,
token_ref=api_key_env,
),),
)
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
from ...backend.util import host_skill_dir
agent = plan.manifest.agent
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
bottle.exec(f"mkdir -p {skills_dir}", user="root")
for name in agent.skills:
src = host_skill_dir(name)
if not os.path.isdir(src):
die(
f"skill {name!r} disappeared from host between "
f"validation and copy at {src}."
)
dst = f"{skills_dir}/{name}"
info(f"copying skill {name} into {bottle.name}:{dst}")
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
bottle.cp_in(f"{src}/.", f"{dst}/")
bottle.exec(f"chown -R node:node {dst}", user="root")
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
prompt_path = _prompt_path(plan.guest_home)
append_system_path = _append_system_path(plan.guest_home)
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
bottle.exec(
f"mkdir -p {shlex.quote(plan.guest_home)}/.pi/agent && "
f"cp {shlex.quote(prompt_path)} {shlex.quote(append_system_path)} && "
f"chown node:node {shlex.quote(prompt_path)} "
f"{shlex.quote(append_system_path)} && "
f"chmod 600 {shlex.quote(prompt_path)} "
f"{shlex.quote(append_system_path)}",
user="root",
)
# Pi's `--append-system-prompt` takes literal text, not a file path.
# Use its documented APPEND_SYSTEM.md discovery path instead.
return None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
provision = plan.agent_provision
_exec(
bottle,
_runtime_state_repair_script(plan.guest_home),
"could not prepare pi runtime state",
)
for d in provision.dirs:
path = shlex.quote(d.guest_path)
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
_exec(
bottle,
f"chown {shlex.quote(d.owner)} {path}",
f"could not chown {d.guest_path}",
)
_exec(
bottle,
f"chmod {shlex.quote(d.mode)} {path}",
f"could not chmod {d.guest_path}",
)
for f in provision.files:
bottle.cp_in(str(f.host_path), f.guest_path)
path = shlex.quote(f.guest_path)
_exec(
bottle,
f"chown {shlex.quote(f.owner)} {path}",
f"could not chown {f.guest_path}",
)
_exec(
bottle,
f"chmod {shlex.quote(f.mode)} {path}",
f"could not chmod {f.guest_path}",
)
def provision_supervise_mcp(
self,
plan: "BottlePlan",
bottle: "Bottle",
supervise_url: str,
) -> None:
del plan, bottle, supervise_url
def _exec(bottle: "Bottle", script: str, error: str) -> None:
result = bottle.exec(script, user="root")
if result.returncode != 0:
detail = (result.stderr or result.stdout).strip()
if detail:
detail = f": {detail}"
die(f"agent provider provisioning: {error}{detail}")
+4
View File
@@ -11,6 +11,10 @@ from __future__ import annotations
from abc import ABC, abstractmethod
class DeployKeyCollisionError(RuntimeError):
"""Raised when a deploy key title or public key already exists on the repo."""
class DeployKeyProvisioner(ABC):
"""Manages a single deploy-key lifecycle on a remote forge."""
+190 -7
View File
@@ -15,6 +15,8 @@ import gzip
import re
import typing
import unicodedata
from math import log2
from collections import Counter
from urllib.parse import quote as url_quote
try:
@@ -78,16 +80,27 @@ TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
)
def scan_token_patterns(text: str, *, location: str = "body") -> ScanResult | None:
def scan_token_patterns(
text: str,
*,
location: str = "body",
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
normalized = _normalize_text(text)
for name, pattern in TOKEN_PATTERNS:
m = pattern.search(normalized)
if m is not None:
for m in pattern.finditer(normalized):
value = m.group(0)
# A value the supervisor has approved (PRD 0062) is no longer a
# block — keep scanning so a second, un-approved token in the
# same request is still caught.
if safe_tokens is not None and value in safe_tokens:
continue
return ScanResult(
severity="block",
reason=f"{name} found in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
context=_snippet(normalized, m.start(), m.end()),
matched=value,
)
return None
@@ -96,20 +109,21 @@ def redact_tokens(
text: str,
*,
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> str:
"""Replace token pattern matches and (if env given) provisioned secrets with REDACT."""
for _, pattern in TOKEN_PATTERNS:
text = pattern.sub(REDACT, text)
if env is not None:
for key, value in env.items():
if key.startswith("EGRESS_TOKEN_") and value:
if any(key.startswith(p) for p in sensitive_prefixes) and value:
for variant in _encoded_variants(value):
text = text.replace(variant, REDACT)
return text
# ---------------------------------------------------------------------------
# Known secrets detector (Phase 1b)
# Known secrets detector
# ---------------------------------------------------------------------------
def _encoded_variants(secret: str) -> list[str]:
@@ -150,26 +164,179 @@ def _encoded_variants(secret: str) -> list[str]:
return variants
# ---------------------------------------------------------------------------
# Fragmentation-resistant helpers
# ---------------------------------------------------------------------------
# Minimum length of alnum projection for projection-based checks to run.
# Short secrets produce too many false positives in projection space.
_ALNUM_MIN_LEN = 8
# Minimum window length for the partial-substring sliding scan.
PARTIAL_MATCH_MIN_LEN = 12
def _alnum_projection(text: str) -> str:
"""Return text with every non-alphanumeric character stripped.
Used for fragmentation-resistant matching: separator-injected secrets
(spaces, hyphens, dots inserted between characters) are identical to
their originals in alnum projection space.
"""
return "".join(c for c in text if c.isalnum())
def _find_partial_window(secret_alnum: str, text_alnum: str, min_len: int) -> int | None:
"""Return the position in text_alnum where any min_len-char window of
secret_alnum first appears, or None.
Slides a window of width min_len across secret_alnum and searches for
each window in text_alnum. The first hit position is returned.
"""
if len(secret_alnum) < min_len or len(text_alnum) < min_len:
return None
for i in range(len(secret_alnum) - min_len + 1):
window = secret_alnum[i:i + min_len]
pos = text_alnum.find(window)
if pos >= 0:
return pos
return None
def scan_known_secrets(
text: str,
*,
location: str = "body",
env: typing.Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
safe_tokens: typing.AbstractSet[str] | None = None,
) -> ScanResult | None:
if env is None:
return None
# Pre-compute alnum projection of the scan text once; reused per secret.
text_alnum: str | None = None
for key, value in env.items():
if not key.startswith("EGRESS_TOKEN_") or not value:
if not any(key.startswith(p) for p in sensitive_prefixes) or not value:
continue
# Pass 1: exact match across encoded variants (original behaviour).
approved_exact = False
for variant in _encoded_variants(value):
pos = text.find(variant)
if pos >= 0:
# The supervisor approves the exact encoded variant found
# (PRD 0062); a different encoding of the same secret is a
# fresh block.
if safe_tokens is not None and variant in safe_tokens:
approved_exact = True
continue
return ScanResult(
severity="block",
reason=f"provisioned secret from {key} found in {location}",
location=location,
context=_snippet(text, pos, pos + len(variant)),
matched=variant,
)
if approved_exact:
# Exact match was found and approved; projection passes would
# fire on the same value, so skip them for this secret.
continue
# Pass 2 & 3: fragmentation-resistant projection checks.
secret_alnum = _alnum_projection(value)
if len(secret_alnum) < _ALNUM_MIN_LEN:
continue
if text_alnum is None:
text_alnum = _alnum_projection(text)
# Pass 2: full alnum-projection exact match (catches separator injection).
pos2 = text_alnum.find(secret_alnum)
if pos2 >= 0:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(fragmented match — separator injection)"
),
location=location,
context=_snippet(text_alnum, pos2, pos2 + len(secret_alnum)),
)
# Pass 3: sliding-window partial match (catches chunked-substring leaks).
pos3 = _find_partial_window(secret_alnum, text_alnum, PARTIAL_MATCH_MIN_LEN)
if pos3 is not None:
return ScanResult(
severity="block",
reason=(
f"provisioned secret from {key} found in {location} "
f"(partial match — at least {PARTIAL_MATCH_MIN_LEN} consecutive "
f"alphanumeric chars)"
),
location=location,
context=_snippet(text_alnum, pos3, pos3 + PARTIAL_MATCH_MIN_LEN),
)
return None
# ---------------------------------------------------------------------------
# Entropy detector (warn-only)
# ---------------------------------------------------------------------------
# Sliding window size and step for the entropy scan.
ENTROPY_WINDOW = 64
ENTROPY_STEP = 32
# Bits-per-character threshold. Random ASCII printable ≈ 6.6 bits; random
# lowercase hex ≈ 4 bits; random base64url ≈ 6 bits. 5.5 sits above
# typical structured data (JSON, URLs) while staying below truly random
# content.
ENTROPY_BLOCK_THRESHOLD = 5.5
def _shannon_entropy(text: str) -> float:
if not text:
return 0.0
counts = Counter(text)
n = len(text)
return -sum((c / n) * log2(c / n) for c in counts.values())
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW,
threshold: float = ENTROPY_BLOCK_THRESHOLD,
) -> ScanResult | None:
"""Warn-only detector: flag windows of `window` chars with Shannon entropy
above `threshold` bits per character.
Never blocks; always returns severity='warn'. Disabled by default
routes must opt in via dlp.outbound_detectors=['entropy'].
"""
if not text:
return None
step = max(1, window // 2)
end = len(text)
# Scan overlapping windows; also check the final tail if shorter than window.
positions = list(range(0, end - window + 1, step))
if end < window:
positions = [0]
elif (end - window) % step != 0:
positions.append(end - window)
for i in positions:
chunk = text[i:i + window]
if _shannon_entropy(chunk) >= threshold:
return ScanResult(
severity="warn",
reason=f"high-entropy content in {location} (possible encrypted exfil)",
location=location,
context=_snippet(text, i, i + len(chunk)),
)
return None
@@ -265,6 +432,14 @@ _CRLF_ENCODED_RE = re.compile(r"%0[dD]%0[aA]", re.ASCII)
_CRLF_HEADER_INJECT_RE = re.compile(r"\r\n[A-Za-z][A-Za-z0-9\-]+\s*:", re.ASCII)
def strip_crlf(text: str) -> str:
"""Remove URL-encoded and literal CRLF injection sequences from a request
surface (PRD 0062 redact policy). Used to scrub the request line / headers
so the request can be forwarded instead of hard-blocked."""
text = _CRLF_ENCODED_RE.sub("", text)
return _CRLF_HEADER_INJECT_RE.sub(lambda m: m.group(0)[2:], text)
def scan_crlf_injection(text: str) -> ScanResult | None:
if _CRLF_ENCODED_RE.search(text):
return ScanResult(
@@ -280,12 +455,20 @@ def scan_crlf_injection(text: str) -> ScanResult | None:
__all__ = [
"ENTROPY_BLOCK_THRESHOLD",
"ENTROPY_WINDOW",
"ENTROPY_STEP",
"PARTIAL_MATCH_MIN_LEN",
"REDACT",
"SNIPPET_CONTEXT",
"TOKEN_PATTERNS",
"_alnum_projection",
"_shannon_entropy",
"redact_tokens",
"scan_crlf_injection",
"scan_entropy",
"scan_known_secrets",
"scan_naive_injection",
"scan_token_patterns",
"strip_crlf",
]
+117 -16
View File
@@ -10,12 +10,14 @@ specific and lives on concrete subclasses (see
from __future__ import annotations
import dataclasses
import secrets
from abc import ABC
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
from .egress_addon_core import (
ON_MATCH_REDACT,
HeaderMatch as CoreHeaderMatch,
MatchEntry as CoreMatchEntry,
PathMatch as CorePathMatch,
@@ -24,13 +26,58 @@ from .egress_addon_core import (
from .log import die
if TYPE_CHECKING:
from .manifest import Bottle
from .manifest import ManifestBottle
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
EGRESS_HOSTNAME = "egress"
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
EGRESS_ROUTES_FILENAME = Path(EGRESS_ROUTES_IN_CONTAINER).name
_CANARY_ENV_WORDS = (
"ACCORD",
"ANCHOR",
"ATLAS",
"CANON",
"CIPHER",
"EMBER",
"FALCON",
"HARBOR",
"LANTERN",
"MARBLE",
"NOVA",
"ORBIT",
"PIVOT",
"RADIUS",
"SUMMIT",
"VECTOR",
)
def _random_canary_env() -> str:
first = secrets.choice(_CANARY_ENV_WORDS)
remaining = tuple(word for word in _CANARY_ENV_WORDS if word != first)
second = secrets.choice(remaining)
return f"{first}_{second}_SECRET"
def egress_sidecar_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return sidecar env entries needed by egress across all backends."""
env: list[str] = []
if plan.routes:
env.extend(sorted(plan.token_env_map.keys()))
if plan.canary and plan.canary_env:
env.append(f"{plan.canary_env}={plan.canary}")
env.append(f"BOT_BOTTLE_SENSITIVE_PREFIXES={plan.canary_env}")
return tuple(env)
def egress_agent_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return agent-visible egress env entries shared by all backends."""
if plan.canary and plan.canary_env:
return (f"{plan.canary_env}={plan.canary}",)
return ()
@dataclass(frozen=True)
@@ -63,10 +110,12 @@ class EgressPlan:
mitmproxy_ca_host_path: Path = Path()
mitmproxy_ca_cert_only_host_path: Path = Path()
log: int = 0
canary: str = ""
canary_env: str = ""
def egress_manifest_routes(
bottle: Bottle,
bottle: ManifestBottle,
) -> tuple[EgressRoute, ...]:
out: list[EgressRoute] = []
for r in bottle.egress.routes:
@@ -91,24 +140,41 @@ def egress_manifest_routes(
auth_scheme=r.AuthScheme,
token_ref=r.TokenRef,
roles=r.Role,
git_fetch=r.GitFetch,
outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors,
outbound_on_match=r.OutboundOnMatch,
))
return tuple(out)
def egress_routes_for_bottle(
bottle: Bottle,
bottle: ManifestBottle,
provider_routes: tuple[EgressRoute, ...] = (),
) -> tuple[EgressRoute, ...]:
manifest = egress_manifest_routes(bottle)
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
merged = list(provider_routes) + [
merged = list(_default_provider_on_match(provider_routes)) + [
r for r in manifest if r.host.lower() not in provisioned_hosts
]
return _assign_token_slots(merged)
def _default_provider_on_match(
provider_routes: tuple[EgressRoute, ...],
) -> tuple[EgressRoute, ...]:
"""Provider routes (the agent talking to its own LLM API) default to the
`redact` on-match policy (PRD 0062): high-volume conversation payloads are
the worst source of token-shaped false positives, so a match is scrubbed
and forwarded rather than hard-blocked or queued for the operator. A
provider that sets `outbound_on_match` explicitly keeps its choice."""
return tuple(
r if r.outbound_on_match
else dataclasses.replace(r, outbound_on_match=ON_MATCH_REDACT)
for r in provider_routes
)
def _assign_token_slots(
routes: list[EgressRoute],
) -> tuple[EgressRoute, ...]:
@@ -144,6 +210,17 @@ def egress_token_env_map(
return out
def _yaml_str_escape(s: str) -> str:
"""Escape a string for use inside a YAML double-quoted scalar."""
return (
s.replace("\\", "\\\\")
.replace('"', '\\"')
.replace("\n", "\\n")
.replace("\r", "\\r")
.replace("\t", "\\t")
)
def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields: dict[str, object] = {"host": r.host}
if r.auth_scheme and r.token_env:
@@ -173,7 +250,13 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
entry_data["headers"] = headers_data
matches_data.append(entry_data)
fields["matches"] = matches_data
if r.outbound_detectors is not None or r.inbound_detectors is not None:
if r.git_fetch:
fields["git"] = {"fetch": True}
if (
r.outbound_detectors is not None
or r.inbound_detectors is not None
or r.outbound_on_match
):
dlp: dict[str, object] = {}
if r.outbound_detectors is not None:
dlp["outbound_detectors"] = (
@@ -185,6 +268,8 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
False if not r.inbound_detectors
else list(r.inbound_detectors)
)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
fields["dlp"] = dlp
return fields
@@ -198,12 +283,12 @@ def _render_match_entry(entry: dict[str, object]) -> list[str]:
for pd in entry["paths"]: # type: ignore[union-attr]
pd_dict: dict[str, str] = pd # type: ignore[assignment]
if "type" in pd_dict:
lines.append(f' - type: "{pd_dict["type"]}"')
lines.append(f' value: "{pd_dict["value"]}"')
lines.append(f' - type: "{_yaml_str_escape(pd_dict["type"])}"')
lines.append(f' value: "{_yaml_str_escape(pd_dict["value"])}"')
else:
lines.append(f' - value: "{pd_dict["value"]}"')
lines.append(f' - value: "{_yaml_str_escape(pd_dict["value"])}"')
if "methods" in entry:
methods_str = ", ".join(f'"{m}"' for m in entry["methods"]) # type: ignore[union-attr]
methods_str = ", ".join(f'"{_yaml_str_escape(m)}"' for m in entry["methods"]) # type: ignore[union-attr]
prefix = " - " if first_key else " "
lines.append(f'{prefix}methods: [{methods_str}]')
first_key = False
@@ -213,8 +298,8 @@ def _render_match_entry(entry: dict[str, object]) -> list[str]:
first_key = False
for hd in entry["headers"]: # type: ignore[union-attr]
hd_dict: dict[str, str] = hd # type: ignore[assignment]
lines.append(f' - name: "{hd_dict["name"]}"')
lines.append(f' value: "{hd_dict["value"]}"')
lines.append(f' - name: "{_yaml_str_escape(hd_dict["name"])}"')
lines.append(f' value: "{_yaml_str_escape(hd_dict["value"])}"')
if first_key:
lines.append(" - {}")
return lines
@@ -234,14 +319,19 @@ def egress_render_routes(
return "\n".join(lines) + "\n"
for r in routes:
f = _route_to_yaml_fields(r)
lines.append(f' - host: "{f["host"]}"')
lines.append(f' - host: "{_yaml_str_escape(str(f["host"]))}"')
if "auth_scheme" in f:
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
lines.append(f' token_env: "{f["token_env"]}"')
lines.append(f' auth_scheme: "{_yaml_str_escape(str(f["auth_scheme"]))}"')
lines.append(f' token_env: "{_yaml_str_escape(str(f["token_env"]))}"')
if "matches" in f:
lines.append(" matches:")
for entry in f["matches"]: # type: ignore[union-attr]
lines.extend(_render_match_entry(entry)) # type: ignore[arg-type]
if "git" in f:
git_dict: dict[str, object] = f["git"] # type: ignore
lines.append(" git:")
if git_dict.get("fetch") is True:
lines.append(" fetch: true")
if "dlp" in f:
dlp_dict: dict[str, object] = f["dlp"] # type: ignore
lines.append(" dlp:")
@@ -251,6 +341,8 @@ def egress_render_routes(
elif isinstance(dv, list):
items_str = ", ".join(f'"{x}"' for x in dv)
lines.append(f" {dk}: [{items_str}]")
elif isinstance(dv, str):
lines.append(f' {dk}: "{_yaml_str_escape(dv)}"')
return "\n".join(lines) + "\n"
@@ -280,27 +372,34 @@ def egress_resolve_token_values(
class Egress(ABC):
def prepare(
self,
bottle: Bottle,
bottle: ManifestBottle,
slug: str,
stage_dir: Path,
provider_routes: tuple[EgressRoute, ...] = (),
) -> EgressPlan:
routes = egress_routes_for_bottle(bottle, provider_routes)
log = bottle.egress.Log
routes_path = stage_dir / "egress_routes.yaml"
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
routes_path.write_text(egress_render_routes(routes, log=log))
routes_path.chmod(0o600)
# Generate a per-session fake secret under a plausible random env name.
# The sidecar marks that exact env name as sensitive for known-secret
# scanning; the agent receives the same name/value as exfil bait.
canary = secrets.token_urlsafe(32)
return EgressPlan(
slug=slug,
routes_path=routes_path,
routes=routes,
token_env_map=egress_token_env_map(routes),
log=log,
canary=canary,
canary_env=_random_canary_env(),
)
__all__ = [
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
"EGRESS_HOSTNAME",
"EGRESS_ROUTES_FILENAME",
"EGRESS_ROUTES_IN_CONTAINER",
"Egress",
"EgressPlan",
@@ -309,5 +408,7 @@ __all__ = [
"egress_render_routes",
"egress_resolve_token_values",
"egress_routes_for_bottle",
"egress_agent_env_entries",
"egress_sidecar_env_entries",
"egress_token_env_map",
]
+299 -24
View File
@@ -5,7 +5,7 @@ egress container."""
from __future__ import annotations
import dataclasses
import asyncio
import json
import os
import signal
@@ -17,35 +17,77 @@ from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
LOG_BLOCKS,
LOG_FULL,
DEFAULT_OUTBOUND_ON_MATCH,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
Config,
Route,
ScanResult,
build_inbound_scan_text,
build_outbound_scan_text,
build_token_allow_payload,
decide,
decide_git_fetch,
is_git_fetch_request,
is_git_push_request,
load_config,
match_route,
outbound_scan_headers,
route_to_yaml_dict,
scan_inbound,
scan_outbound,
)
try:
from dlp_detectors import redact_tokens # type: ignore[import-not-found]
from dlp_detectors import redact_tokens, strip_crlf # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.dlp_detectors import redact_tokens # type: ignore[import-not-found]
from bot_bottle.dlp_detectors import ( # type: ignore[import-not-found]
redact_tokens,
strip_crlf,
)
try:
import supervise as _sv # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
INTROSPECT_HOST = "_egress.local"
# Seconds the egress proxy holds a token-blocked request open waiting for the
# operator's supervisor decision (PRD 0062), overridable via env.
DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS = 300.0
# Filesystem poll cadence while awaiting the operator's response.
TOKEN_ALLOW_POLL_INTERVAL_SECONDS = 0.5
# Fixed operator guidance attached to every token-allow proposal.
_TOKEN_ALLOW_JUSTIFICATION = (
"egress DLP blocked an outbound request carrying a detected token. "
"Approve only if this value is a false positive or a credential this "
"request legitimately needs; the value is then allowed for the life of "
"this bottle's egress proxy."
)
class EgressAddon:
def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.config: Config = Config(routes=())
# Tokens the operator has approved this session (PRD 0062). In-memory
# only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed.
self.safe_tokens: set[str] = set()
self._supervise_queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "").strip()
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True)
self._install_sighup()
def _supervise_available(self) -> bool:
return bool(self._supervise_queue_dir and self._supervise_slug)
def _reload(self, *, initial: bool = False) -> None:
try:
text = Path(self.routes_path).read_text(encoding="utf-8")
@@ -79,7 +121,7 @@ class EgressAddon:
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
if path == "/allowlist":
payload = json.dumps(
{"routes": [dataclasses.asdict(r) for r in self.config.routes]},
{"routes": [route_to_yaml_dict(r) for r in self.config.routes]},
indent=2,
).encode("utf-8")
flow.response = http.Response.make(
@@ -118,31 +160,42 @@ class EgressAddon:
)
def _log_request(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.request.headers.items()
if k.lower() != "authorization"
}
body = redact_tokens(flow.request.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": dict(flow.request.headers),
"body": flow.request.get_text(strict=False) or "",
"headers": headers,
"body": body,
})
+ "\n"
)
def _log_response(self, flow: http.HTTPFlow) -> None:
headers = {
k: redact_tokens(v, env=os.environ)
for k, v in flow.response.headers.items()
}
body = redact_tokens(flow.response.get_text(strict=False) or "", env=os.environ)
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": dict(flow.response.headers),
"body": flow.response.get_text(strict=False) or "",
"headers": headers,
"body": body,
})
+ "\n"
)
def request(self, flow: http.HTTPFlow) -> None:
async def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?")
if flow.request.pretty_host == INTROSPECT_HOST:
@@ -154,21 +207,11 @@ class EgressAddon:
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
if route is not None:
body = flow.request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
dict(flow.request.headers),
body,
)
dlp_result = scan_outbound(route, scan_text, os.environ)
if dlp_result is not None and dlp_result.severity == "block":
ctx = self._req_ctx(flow)
if dlp_result.context:
ctx = {**ctx, "context": dlp_result.context}
self._block(flow, f"egress DLP: {dlp_result.reason}", ctx=ctx)
if not await self._handle_outbound_dlp(flow, route):
return
# The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on.
request_path, _, query = flow.request.path.partition("?")
if is_git_push_request(request_path, query):
self._block(
@@ -180,6 +223,18 @@ class EgressAddon:
)
return
if is_git_fetch_request(request_path, query):
git_decision = decide_git_fetch(
self.config.routes, flow.request.pretty_host,
)
if git_decision.action == "block":
self._block(
flow,
git_decision.reason,
ctx=self._req_ctx(flow),
)
return
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject sidecar-owned auth below.
flow.request.headers.pop("authorization", None)
@@ -206,6 +261,202 @@ class EgressAddon:
if self.config.log >= LOG_FULL:
self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
ctx = self._req_ctx(flow)
if result.context:
ctx = {**ctx, "context": result.context}
self._block(flow, f"egress DLP: {result.reason}", ctx=ctx)
async def _handle_outbound_dlp(
self,
flow: http.HTTPFlow,
route: Route,
) -> bool:
"""Scan the outbound request and apply the route's on-match policy
(PRD 0062). Returns True if the request may be forwarded, False if a
403 response has been written to `flow`.
Loops so the supervise policy can re-scan after each approval a
second, un-approved token in the same request is still caught."""
while True:
request_path, _, query = flow.request.path.partition("?")
body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, body,
)
# CRLF is scanned only over the request line + headers, never the
# body (see scan_outbound) — a body is not an injection vector.
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(
route, scan_text, os.environ,
safe_tokens=self.safe_tokens, crlf_text=crlf_text,
)
if result is None or result.severity != "block":
return True
policy = route.outbound_on_match or DEFAULT_OUTBOUND_ON_MATCH
# redact scrubs every detection (tokens and structural CRLF) and
# forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route):
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_redacted",
"reason": f"egress DLP: {result.reason}",
**self._req_ctx(flow),
}) + "\n")
return True
self._block(
flow,
f"egress DLP: {result.reason}; redaction could not remove "
"all matches (e.g. a match in the hostname)",
ctx=self._req_ctx(flow),
)
return False
# Structural blocks (CRLF, no safelist-able value) cannot be
# supervised — there is nothing to approve and remember — so under
# block/supervise they are a hard 403.
if policy == ON_MATCH_BLOCK or not result.matched:
self._block_dlp(flow, result)
return False
# supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available():
self._block_dlp(flow, result)
return False
approved = await self._supervise_token_block(flow, request_path, result)
if not approved:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound(self, flow: http.HTTPFlow, route: Route) -> bool:
"""Scrub detected tokens (and CRLF injection sequences) from the mutable
request surfaces (body, headers, path/query) and re-scan. Returns True
if the request is now clean; False if a block-severity match remains on
a surface redaction cannot rewrite (the hostname) so the caller fails
closed."""
body = flow.request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=os.environ)
if redacted_body != body:
flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()):
if name.lower() == "host":
continue # routing-critical; never a legitimate token
redacted = strip_crlf(redact_tokens(value, env=os.environ))
if redacted != value:
flow.request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=os.environ))
if redacted_path != flow.request.path:
flow.request.path = redacted_path
request_path, _, query = flow.request.path.partition("?")
new_body = flow.request.get_text(strict=False) or ""
headers = outbound_scan_headers(route, dict(flow.request.headers))
scan_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, new_body,
)
crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "",
)
result = scan_outbound(route, scan_text, os.environ, crlf_text=crlf_text)
return result is None or result.severity != "block"
async def _supervise_token_block(
self,
flow: http.HTTPFlow,
request_path: str,
result: ScanResult,
) -> bool:
"""Route a token DLP block to the operator's supervisor queue and wait.
Returns True if the operator approved (the matched value is added to
`self.safe_tokens` and the caller re-scans); False if the request must
be blocked (a 403 response has been written to `flow`)."""
host = flow.request.pretty_host
payload = build_token_allow_payload(
redact_tokens(host, env=os.environ),
flow.request.method,
redact_tokens(request_path, env=os.environ),
result,
)
proposal = _sv.Proposal.new(
bottle_slug=self._supervise_slug,
tool=_sv.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file=payload,
justification=_TOKEN_ALLOW_JUSTIFICATION,
current_file_hash=_sv.sha256_hex(payload),
)
queue_dir = Path(self._supervise_queue_dir)
try:
_sv.write_proposal(queue_dir, proposal)
except OSError as e:
sys.stderr.write(
f"egress: could not queue token-allow proposal: {e}; "
"blocking request\n"
)
self._block(flow, f"egress DLP: {result.reason}", ctx=self._req_ctx(flow))
return False
sys.stderr.write(json.dumps({
"event": "egress_token_supervise",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
response = await self._await_token_response(queue_dir, proposal.id)
_sv.archive_proposal(queue_dir, proposal.id)
if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
):
self.safe_tokens.add(result.matched)
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_token_allowed",
"reason": f"egress DLP: {result.reason}",
"proposal": proposal.id,
**self._req_ctx(flow),
}) + "\n")
return True
if response is None:
reason = (
f"egress DLP: {result.reason}; supervisor approval timed out "
f"after {self._token_allow_timeout:g}s"
)
else:
reason = f"egress DLP: {result.reason}; supervisor rejected the request"
self._block(flow, reason, ctx=self._req_ctx(flow))
return False
async def _await_token_response(
self,
queue_dir: Path,
proposal_id: str,
) -> "_sv.Response | None":
"""Poll the queue dir for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout."""
loop = asyncio.get_running_loop()
deadline = loop.time() + self._token_allow_timeout
while True:
try:
return _sv.read_response(queue_dir, proposal_id)
except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed.
pass
if loop.time() >= deadline:
return None
await asyncio.sleep(TOKEN_ALLOW_POLL_INTERVAL_SECONDS)
def response(self, flow: http.HTTPFlow) -> None:
"""DLP inbound scan on response headers and body."""
route = match_route(self.config.routes, flow.request.pretty_host)
@@ -257,7 +508,12 @@ class EgressAddon:
message = flow.websocket.messages[-1] # type: ignore[union-attr]
content = message.content.decode("utf-8", errors="replace")
if message.from_client:
result = scan_outbound(route, content, os.environ)
# A WebSocket data frame is not an HTTP request line, so CRLF is
# not an injection vector here — scan only for credential leakage.
result = scan_outbound(
route, content, os.environ,
safe_tokens=self.safe_tokens, crlf_text="",
)
if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
@@ -271,4 +527,23 @@ class EgressAddon:
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
def _token_allow_timeout_from_env(env: "os._Environ[str]") -> float:
"""Read EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS; fall back to the default on an
unset or invalid value (a bad value should not wedge egress at boot)."""
raw = env.get("EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS", "").strip()
if not raw:
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
try:
value = float(raw)
except ValueError:
value = 0.0
if value <= 0:
sys.stderr.write(
"egress: invalid EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS="
f"{raw!r}; using default {DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS:g}s\n"
)
return DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS
return value
addons = [EgressAddon()]
+235 -24
View File
@@ -34,9 +34,18 @@ VALID_METHODS = frozenset({
"CONNECT",
})
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets", "entropy"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# Per-route policy for what the proxy does when an outbound DLP detector
# matches a token (PRD 0062).
ON_MATCH_BLOCK = "block" # hard 403, never overridable
ON_MATCH_REDACT = "redact" # scrub the matched value, forward the request
ON_MATCH_SUPERVISE = "supervise" # queue for operator approval, hold the request
OUTBOUND_ON_MATCH_VALUES = (ON_MATCH_BLOCK, ON_MATCH_REDACT, ON_MATCH_SUPERVISE)
# Unset resolves to supervise (fall back to block when supervise is not wired).
DEFAULT_OUTBOUND_ON_MATCH = ON_MATCH_SUPERVISE
@dataclass(frozen=True)
class PathMatch:
@@ -66,8 +75,11 @@ class Route:
matches: tuple[MatchEntry, ...] = ()
auth_scheme: str = ""
token_env: str = ""
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
# "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES.
outbound_on_match: str = ""
LOG_OFF = 0 # no logging
@@ -94,6 +106,11 @@ class ScanResult:
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
matched: str = ""
# ---------------------------------------------------------------------------
@@ -217,12 +234,12 @@ def _parse_detectors(
idx: int,
host: str,
raw_dict: dict[str, object],
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
"""Parse the optional `dlp` block on a route, returning
(outbound_detectors, inbound_detectors)."""
(outbound_detectors, inbound_detectors, outbound_on_match)."""
dlp_raw = raw_dict.get("dlp")
if dlp_raw is None:
return None, None
return None, None, ""
label = f"route[{idx}] ({host})"
if not isinstance(dlp_raw, dict):
raise ValueError(f"{label}: 'dlp' must be an object")
@@ -259,13 +276,24 @@ def _parse_detectors(
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = dlp.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ValueError(
f"{label}: dlp.outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in dlp:
if k not in ("outbound_detectors", "inbound_detectors"):
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ValueError(
f"{label}: dlp has unknown key {k!r}; accepted keys "
f"are 'outbound_detectors', 'inbound_detectors'"
f"are 'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
)
return outbound, inbound
return outbound, inbound, on_match
def parse_routes(payload: object) -> tuple[Route, ...]:
@@ -316,16 +344,35 @@ def _parse_one(idx: int, raw: object) -> Route:
f"token_env={token_env!r})"
)
# git-over-HTTPS policy
git_fetch = False
git_raw = raw_dict.get("git")
if git_raw is not None:
if not isinstance(git_raw, dict):
raise ValueError(f"{label} ({host}): 'git' must be an object")
git_dict: dict[str, object] = typing.cast(dict[str, object], git_raw)
fetch_raw = git_dict.get("fetch", False)
if fetch_raw is True or fetch_raw is False:
git_fetch = fetch_raw
else:
raise ValueError(f"{label} ({host}): 'git.fetch' must be a boolean")
for k in git_dict:
if k != "fetch":
raise ValueError(
f"{label} ({host}): git has unknown key {k!r}; "
"accepted key is 'fetch'"
)
# dlp detectors
outbound_detectors, inbound_detectors = _parse_detectors(
outbound_detectors, inbound_detectors, outbound_on_match = _parse_detectors(
idx, host, raw_dict,
)
for k in raw_dict:
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp"):
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git"):
raise ValueError(
f"{label} ({host}): unknown key {k!r}; accepted keys "
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp'"
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git'"
)
return Route(
@@ -333,18 +380,63 @@ def _parse_one(idx: int, raw: object) -> Route:
matches=matches,
auth_scheme=auth_scheme,
token_env=token_env,
git_fetch=git_fetch,
outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match,
)
def load_routes(text: str) -> tuple[Route, ...]:
"""Parse YAML text → routes."""
try:
payload = parse_yaml_subset(text)
except YamlSubsetError as e:
raise ValueError(f"routes payload: invalid YAML: {e}") from e
return parse_routes(payload)
def _path_match_to_dict(pm: PathMatch) -> dict[str, object]:
d: dict[str, object] = {"value": pm.value}
if pm.type != "prefix":
d["type"] = pm.type
return d
def _header_match_to_dict(hm: HeaderMatch) -> dict[str, object]:
d: dict[str, object] = {"name": hm.name, "value": hm.value}
if hm.type != "exact":
d["type"] = hm.type
return d
def _match_entry_to_dict(me: MatchEntry) -> dict[str, object]:
d: dict[str, object] = {}
if me.paths:
d["paths"] = [_path_match_to_dict(p) for p in me.paths]
if me.methods:
d["methods"] = list(me.methods)
if me.headers:
d["headers"] = [_header_match_to_dict(h) for h in me.headers]
return d
def route_to_yaml_dict(r: Route) -> dict[str, object]:
"""Serialize a Route to YAML-schema-compatible dict.
Uses the same field names the YAML parser accepts, so the output
can be round-tripped directly into an `allow` or `egress-block`
proposal without translation. Fields that are empty/default are
omitted so the agent doesn't copy irrelevant keys."""
d: dict[str, object] = {"host": r.host}
if r.auth_scheme:
d["auth_scheme"] = r.auth_scheme
d["token_env"] = r.token_env
if r.matches:
d["matches"] = [_match_entry_to_dict(m) for m in r.matches]
if r.git_fetch:
d["git"] = {"fetch": True}
dlp: dict[str, object] = {}
if r.outbound_detectors is not None:
dlp["outbound_detectors"] = list(r.outbound_detectors)
if r.inbound_detectors is not None:
dlp["inbound_detectors"] = list(r.inbound_detectors)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
if dlp:
d["dlp"] = dlp
return d
def parse_config(payload: object) -> "Config":
@@ -450,6 +542,17 @@ def is_git_push_request(path: str, query: str) -> bool:
return False
def is_git_fetch_request(path: str, query: str) -> bool:
if path.endswith("/git-upload-pack"):
return True
if path.endswith("/info/refs"):
for pair in query.split("&"):
k, _, v = pair.partition("=")
if k == "service" and v == "git-upload-pack":
return True
return False
# ---------------------------------------------------------------------------
# Route lookup + decision
# ---------------------------------------------------------------------------
@@ -513,6 +616,24 @@ def decide(
return Decision(action="forward")
def decide_git_fetch(
routes: typing.Sequence[Route],
request_host: str,
) -> Decision:
route = match_route(routes, request_host)
if route is not None and route.git_fetch:
return Decision(action="forward")
return Decision(
action="block",
reason=(
"egress: git fetch/clone over HTTPS is not allowed by default; "
"use git-gate for declared repos or set "
"egress.routes[].git.fetch=true for explicit read-only "
"HTTPS Git access."
),
)
# ---------------------------------------------------------------------------
# DLP scan dispatch (PRD 0053)
# ---------------------------------------------------------------------------
@@ -538,6 +659,27 @@ def build_outbound_scan_text(
return "\n".join(parts)
def outbound_scan_headers(
route: Route,
headers: typing.Mapping[str, str],
) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP.
Routes that inject sidecar-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what
reaches the upstream.
"""
out: dict[str, str] = {}
skip_auth = bool(route.auth_scheme and route.token_env)
for name, value in headers.items():
if skip_auth and name.lower() == "authorization":
continue
out[name] = value
return out
def build_inbound_scan_text(
headers: typing.Mapping[str, str],
body: str,
@@ -569,43 +711,103 @@ def scan_outbound(
route: Route,
body: str | bytes,
environ: typing.Mapping[str, str],
*,
safe_tokens: typing.AbstractSet[str] | None = None,
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
except ImportError: # pragma: no cover - host-side path
from .dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_entropy,
scan_known_secrets,
scan_token_patterns,
)
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
# Binary bodies: latin-1 is a bijective byte↔codepoint mapping that
# preserves every byte value, so ASCII-range secret strings remain
# findable by str.find / regex. Prefer strict UTF-8 for valid text bodies.
if isinstance(body, bytes):
try:
text = body.decode("utf-8")
except UnicodeDecodeError:
text = body.decode("latin-1")
else:
text = body
# CRLF injection is never legitimate — runs unconditionally, not gated
# by outbound_detectors config.
result = scan_crlf_injection(text)
# CRLF injection is only an attack in the request line + headers, never the
# body: an HTTP body is delimited by Content-Length, so CRLF bytes there
# cannot split the request. Scanning the body produces false positives on
# legitimate form-encoded / multi-line content. Callers pass the
# body-excluded surfaces as `crlf_text`; `None` falls back to the full text
# for backward-compatible callers (host-side tests, websocket frames).
crlf_target = text if crlf_text is None else crlf_text
result = scan_crlf_injection(crlf_target)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "token_patterns"):
result = scan_token_patterns(text, location="body")
result = scan_token_patterns(text, location="body", safe_tokens=safe_tokens)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "known_secrets"):
result = scan_known_secrets(text, location="body", env=environ)
# BOT_BOTTLE_SENSITIVE_PREFIXES lets operators add extra env prefixes
# beyond EGRESS_TOKEN_* without changing the manifest schema.
extra_raw = environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "")
extra = tuple(p for p in extra_raw.split(",") if p)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
result = scan_known_secrets(
text, location="body", env=environ,
sensitive_prefixes=sensitive_prefixes, safe_tokens=safe_tokens,
)
if result is not None:
return result
# Entropy scanning requires explicit opt-in: it is NOT part of the
# default "all detectors" set because it produces false positives on
# legitimate base64 / binary payloads. Routes must list "entropy" in
# dlp.outbound_detectors to enable it.
if (
route.outbound_detectors is not None
and "entropy" in route.outbound_detectors
):
result = scan_entropy(text, location="body")
if result is not None:
return result
return None
def build_token_allow_payload(
host: str,
method: str,
path: str,
result: ScanResult,
) -> str:
"""Render the human-readable supervisor proposal body for an outbound
token block (PRD 0062). Carries the host/method/path, the detector
reason, and the redacted context snippet never the raw token value."""
lines = [
"egress blocked an outbound request carrying a detected token",
f"host: {host}",
f"method: {method}",
f"path: {path}",
f"detector: {result.reason}",
]
if result.context:
lines.append(f"context: {result.context}")
return "\n".join(lines) + "\n"
def scan_inbound(
route: Route,
body: str | bytes,
@@ -627,8 +829,14 @@ def scan_inbound(
__all__ = [
"LOG_BLOCKS",
"route_to_yaml_dict",
"LOG_FULL",
"LOG_OFF",
"ON_MATCH_BLOCK",
"ON_MATCH_REDACT",
"ON_MATCH_SUPERVISE",
"OUTBOUND_ON_MATCH_VALUES",
"DEFAULT_OUTBOUND_ON_MATCH",
"Config",
"Decision",
"HeaderMatch",
@@ -638,12 +846,15 @@ __all__ = [
"ScanResult",
"build_inbound_scan_text",
"build_outbound_scan_text",
"build_token_allow_payload",
"decide",
"decide_git_fetch",
"evaluate_matches",
"is_git_push_request",
"is_git_fetch_request",
"load_config",
"load_routes",
"match_route",
"outbound_scan_headers",
"parse_config",
"parse_routes",
"scan_inbound",
+2 -2
View File
@@ -114,7 +114,7 @@ def _read_secret_silent(name: str, prompt_body: str) -> str:
return value
def resolve_env(manifest: Manifest, agent: str) -> ResolvedEnv:
def resolve_env(manifest: Manifest) -> ResolvedEnv:
"""Iterate the agent's env entries:
- secret: prompt at runtime; carry value in forwarded
- interpolated: read $HOST_VAR from os.environ; carry value in forwarded
@@ -124,7 +124,7 @@ def resolve_env(manifest: Manifest, agent: str) -> ResolvedEnv:
backend injects forwarded values via its launcher's env parameter."""
forwarded: dict[str, str] = {}
literals: dict[str, str] = {}
bottle = manifest.bottle_for(agent)
bottle = manifest.bottle
for name, raw in bottle.env.items():
if not name:
continue
+226 -30
View File
@@ -37,16 +37,16 @@ from dataclasses import dataclass
from pathlib import Path
from .log import info
from .manifest import Bottle, GitEntry
from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the sidecar bundle. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate"
# Bound half-open git client sessions. If an agent/tool runner is
# interrupted during push, git daemon should reap the receive-pack
# child instead of keeping the gate wedged indefinitely.
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
# git daemon (--timeout/--init-timeout), the access-hook subprocess in
# git_http_backend, and the git http-backend CGI subprocess.
GIT_GATE_TIMEOUT_SECS = 15
@dataclass(frozen=True)
@@ -96,9 +96,9 @@ class GitGatePlan:
egress_network: str = ""
def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]:
def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstream, ...]:
"""Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name
validation already ran in `manifest.Bottle.from_dict`."""
validation already ran in `manifest.ManifestBottle.from_dict`."""
return tuple(
GitGateUpstream(
name=e.Name,
@@ -112,8 +112,17 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
)
def _gitconfig_validate_value(field: str, value: str) -> None:
"""Raise ValueError if value contains characters that break gitconfig line syntax."""
if "\n" in value or "\r" in value:
raise ValueError(
f"git-gate: {field} contains a newline, which would inject "
f"arbitrary gitconfig keys; rejecting manifest entry"
)
def git_gate_render_gitconfig(
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
"""Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
@@ -136,6 +145,7 @@ def git_gate_render_gitconfig(
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
]
for entry in entries:
_gitconfig_validate_value(f"repos[{entry.Name!r}].url", entry.Upstream)
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
out.append(f"\tinsteadOf = {entry.Upstream}\n")
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
@@ -148,6 +158,7 @@ def git_gate_render_gitconfig(
f"ssh://{entry.UpstreamUser}@{entry.RemoteKey}{port}/"
f"{entry.UpstreamPath}"
)
_gitconfig_validate_value(f"repos[{entry.Name!r}].url (resolved alias)", alias)
out.append(f"\tinsteadOf = {alias}\n")
return "".join(out)
@@ -204,6 +215,7 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
" git -C \"$repo\" config receive.advertisePushOptions true",
" git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}",
@@ -216,8 +228,8 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
"",
"exec git daemon \\",
" --reuseaddr \\",
f" --timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
f" --timeout={GIT_GATE_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_TIMEOUT_SECS} \\",
" --base-path=/git \\",
" --export-all \\",
" --enable=receive-pack \\",
@@ -246,6 +258,164 @@ cat > "$refs_file"
zero=0000000000000000000000000000000000000000
supervise_gitleaks_allow() {
log_opts=$1
ref=$2
report_file=$(mktemp)
if ! gitleaks git \
--log-opts="$log_opts" \
--no-banner \
--redact \
--ignore-gitleaks-allow \
--report-format=json \
--report-path="$report_file" \
--exit-code 0 \
1>&2; then
rm -f "$report_file"
echo "git-gate: gitleaks inline-suppression scan failed for $ref" >&2
return 1
fi
proposal_id=$(
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
import datetime
import hashlib
import json
import os
import sys
import uuid
from pathlib import Path
report_path = Path(sys.argv[1])
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
if not queue_dir or not slug:
sys.exit(2)
try:
raw = json.loads(report_path.read_text() or "[]")
except json.JSONDecodeError:
sys.exit(3)
if not isinstance(raw, list):
sys.exit(3)
if not raw:
sys.exit(0)
ref = os.environ.get("GITLEAKS_ALLOW_REF", "")
lines = [
"gitleaks inline suppression requires supervisor approval",
f"ref: {ref}",
"",
]
for i, finding in enumerate(raw, 1):
if not isinstance(finding, dict):
continue
file_path = finding.get("File", "")
line_no = finding.get("StartLine", finding.get("Line", ""))
rule_id = finding.get("RuleID", "")
commit = finding.get("Commit", "")
line = finding.get("Line", "")
lines.extend([
f"finding {i}:",
f" file: {file_path}",
f" line: {line_no}",
f" rule: {rule_id}",
f" commit: {commit}",
f" code: {line}",
"",
])
payload = "\n".join(lines).rstrip() + "\n"
proposal_id = str(uuid.uuid4())
proposal = {
"id": proposal_id,
"bottle_slug": slug,
"tool": "gitleaks-allow",
"proposed_file": payload,
"justification": (
"git-gate found gitleaks findings hidden by # gitleaks:allow; "
"approve only for dummy test fixtures or confirmed false positives"
),
"arrival_timestamp": datetime.datetime.now(
datetime.timezone.utc
).isoformat(),
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(),
}
queue = Path(queue_dir)
queue.mkdir(parents=True, exist_ok=True)
path = queue / f"{proposal_id}.proposal.json"
tmp = path.with_suffix(path.suffix + ".tmp")
with tmp.open("w", encoding="utf-8") as f:
json.dump(proposal, f, indent=2)
f.write("\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
print(proposal_id)
PY
)
rc=$?
rm -f "$report_file"
if [ "$rc" -eq 0 ] && [ -z "$proposal_id" ]; then
return 0
fi
if [ "$rc" -ne 0 ]; then
echo "git-gate: cannot route # gitleaks:allow finding to supervisor; refusing push" >&2
return 1
fi
queue_dir=${SUPERVISE_QUEUE_DIR:-}
response_file="$queue_dir/${proposal_id}.response.json"
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
case "$timeout" in
''|*[!0-9]*)
echo "git-gate: invalid SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS=$timeout" >&2
return 1
;;
esac
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
waited=0
while [ "$waited" -lt "$timeout" ]; do
if [ -f "$response_file" ]; then
status=$(python3 - "$response_file" <<'PY'
import json
import sys
try:
with open(sys.argv[1], encoding="utf-8") as f:
raw = json.load(f)
except (OSError, json.JSONDecodeError):
sys.exit(1)
status = raw.get("status")
if not isinstance(status, str):
sys.exit(1)
print(status)
PY
) || status=""
case "$status" in
approved|modified)
mkdir -p "$queue_dir/processed"
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
return 0
;;
rejected)
echo "git-gate: supervisor rejected # gitleaks:allow for $ref" >&2
return 1
;;
*)
echo "git-gate: invalid supervisor response for # gitleaks:allow" >&2
return 1
;;
esac
fi
sleep 1
waited=$((waited + 1))
done
echo "git-gate: supervisor approval timed out for # gitleaks:allow; refusing push" >&2
return 1
}
# Phase 1: gitleaks scan each ref's incoming commits.
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
@@ -267,6 +437,9 @@ while IFS=' ' read -r old new ref; do
echo "git-gate: gitleaks rejected push to $ref" >&2
exit 1
fi
if ! supervise_gitleaks_allow "$log_opts" "$ref"; then
exit 1
fi
done < "$refs_file"
# Phase 2: forward each ref to the upstream (`origin`, configured
@@ -280,15 +453,32 @@ if [ ! -f "$hostsfile" ]; then
fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
push_option_count=${GIT_PUSH_OPTION_COUNT:-0}
case "$push_option_count" in
''|*[!0-9]*)
echo "git-gate: invalid GIT_PUSH_OPTION_COUNT=$push_option_count" >&2
exit 1
;;
esac
set --
i=0
while [ "$i" -lt "$push_option_count" ]; do
opt=$(printenv "GIT_PUSH_OPTION_$i" || :)
set -- "$@" --push-option="$opt"
i=$((i + 1))
done
while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue
if [ "$new" = "$zero" ]; then
refspec=":$ref"
elif [ "$old" != "$zero" ] && ! git merge-base --is-ancestor "$old" "$new" 2>/dev/null; then
refspec="+$new:$ref"
else
refspec="$new:$ref"
fi
echo "git-gate: forwarding $ref to origin" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git push origin "$refspec" 1>&2; then
if ! GIT_SSH_COMMAND="$ssh_cmd" git push "$@" origin "$refspec" 1>&2; then
echo "git-gate: upstream push failed for $ref" >&2
exit 1
fi
@@ -361,7 +551,7 @@ exit 0
def _provision_dynamic_key(
entry: GitEntry,
entry: ManifestGitEntry,
slug: str,
stage_dir: Path,
) -> str:
@@ -371,13 +561,12 @@ def _provision_dynamic_key(
Returns the host-side path to the private key file so the caller
can inject it into the GitGateUpstream as `identity_file`."""
from .deploy_key_provisioner import get_provisioner
pk = entry.ProvisionedKey
assert pk is not None
token = os.environ.get(pk.token_env)
pk = entry.Key
token = os.environ.get(pk.forge_token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
f" = {pk.token_env!r}: env var is not set"
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url)
@@ -402,7 +591,7 @@ def _provision_dynamic_key(
return str(key_file)
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) -> None:
"""Revoke all deploy keys provisioned for `bottle` during prepare.
Called at teardown after containers stop. Raises if any revocation
@@ -410,18 +599,18 @@ def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
address manually."""
from .deploy_key_provisioner import get_provisioner
for entry in bottle.git:
if entry.ProvisionedKey is None:
if entry.Key.provider != "gitea":
continue
pk = entry.ProvisionedKey
pk = entry.Key
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
if not id_file.exists():
continue
key_id = id_file.read_text().strip()
token = os.environ.get(pk.token_env)
token = os.environ.get(pk.forge_token_env)
if token is None:
raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
f" = {pk.token_env!r}: env var is not set;"
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set;"
f" cannot revoke deploy key {key_id}"
)
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
@@ -434,18 +623,26 @@ def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path) -> str:
"""Return the host-side SSH identity file path for this entry.
For gitea entries, provisions a fresh deploy key first."""
if entry.Key.provider == "gitea":
return _provision_dynamic_key(entry, slug, stage_dir)
return entry.IdentityFile
class GitGate(ABC):
"""The per-agent git-gate. Encapsulates the host-side prepare
(upstream lift + entrypoint/hook render); the sidecar's
start/stop lifecycle is backend-specific and lives on concrete
subclasses."""
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> GitGatePlan:
def prepare(self, bottle: ManifestBottle, slug: str, stage_dir: Path) -> GitGatePlan:
"""Compute the upstream table from `bottle.git` and write the
entrypoint, pre-receive hook, and access-hook scripts (mode
600) under `stage_dir`. Pure host-side, no docker subprocess.
For `provisioned_key` entries, also generates and registers
For `gitea` key entries, also generates and registers
a fresh deploy key via the forge API and writes the private key
+ key ID to `stage_dir`.
@@ -454,11 +651,10 @@ class GitGate(ABC):
before passing the plan to `.start`."""
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
for i, entry in enumerate(bottle.git):
if entry.ProvisionedKey is not None:
key_file = _provision_dynamic_key(entry, slug, stage_dir)
upstreams_list[i] = dataclasses.replace(
upstreams_list[i], identity_file=key_file
)
upstreams_list[i] = dataclasses.replace(
upstreams_list[i],
identity_file=_resolve_identity_file(entry, slug, stage_dir),
)
upstreams = tuple(upstreams_list)
entrypoint = stage_dir / "git_gate_entrypoint.sh"
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
+13 -3
View File
@@ -16,11 +16,13 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlsplit
from .git_gate import GIT_GATE_TIMEOUT_SECS
DEFAULT_PORT = 9420
# Body-size cap matching supervise_server.py's 1 MiB limit.
MAX_BODY_BYTES = 1 * 1024 * 1024
# Bound memory use while still allowing ordinary git push packfiles.
MAX_BODY_BYTES = 100 * 1024 * 1024
class GitHttpHandler(BaseHTTPRequestHandler):
@@ -47,6 +49,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
[hook_path, "upload-pack", str(repo_dir), peer, peer],
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
if hook.returncode != 0:
detail = (hook.stderr or hook.stdout).decode(
@@ -110,6 +113,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
env=env,
capture_output=True,
check=False,
timeout=GIT_GATE_TIMEOUT_SECS,
)
self._write_cgi_response(proc.stdout)
@@ -148,7 +152,13 @@ class GitHttpHandler(BaseHTTPRequestHandler):
key, _, value = line.decode("latin1").partition(":")
value = value.strip()
if key.lower() == "status":
status = int(value.split()[0])
try:
status = int(value.split()[0])
except (ValueError, IndexError):
self.log_message(
"malformed CGI Status header %r; using 500", value,
)
status = 500
else:
headers.append((key, value))
self.send_response(status)
+96 -10
View File
@@ -1,21 +1,107 @@
"""Tiny logging wrappers. All output goes to stderr."""
"""Tiny logging wrappers. All output goes to stderr.
Two capabilities layer onto the bare wrappers (issue #252):
- **Levels.** `debug` / `info` / `warn` / `error` carry an ordered
severity. Output is gated by `BOT_BOTTLE_LOG_LEVEL` (debug | info |
warn | error; default `info`). A message emits when its severity is
at or above the threshold, so `debug` is silent by default and
`error` always surfaces (nothing sits above it) which keeps the
fatal `die` path visible regardless of the configured level.
- **Context.** Every wrapper takes an optional `context` mapping that
renders as a parseable ` [k=v ...]` suffix (keys sorted; values with
whitespace/quotes are quoted), so failures can be filtered and
correlated instead of being flat strings.
With no `context` and the default level, output is byte-identical to the
original `bot-bottle: <msg>` / `bot-bottle: warning: <msg>` /
`bot-bottle: error: <msg>` lines the 100+ existing call sites are
unaffected.
"""
from __future__ import annotations
import os
import sys
from typing import NoReturn
from typing import Mapping, NoReturn
# Ordered severities. Gaps left between values so intermediate levels
# can be added later without renumbering.
DEBUG = 10
INFO = 20
WARN = 30
ERROR = 40
_LEVEL_NAMES: dict[str, int] = {
"debug": DEBUG,
"info": INFO,
"warn": WARN,
"warning": WARN,
"error": ERROR,
}
# Default threshold when BOT_BOTTLE_LOG_LEVEL is unset or unrecognised.
_DEFAULT_THRESHOLD = INFO
_LOG_LEVEL_ENV = "BOT_BOTTLE_LOG_LEVEL"
def info(msg: str) -> None:
print(f"bot-bottle: {msg}", file=sys.stderr)
def _threshold() -> int:
"""Resolve the active level threshold from the environment.
Read per-call (not cached) so the level can be changed at runtime
and so tests can patch `os.environ` without a reload. Unknown values
fall back to the default rather than raising logging must never be
the thing that crashes the process."""
raw = os.environ.get(_LOG_LEVEL_ENV, "")
return _LEVEL_NAMES.get(raw.strip().lower(), _DEFAULT_THRESHOLD)
def warn(msg: str) -> None:
print(f"bot-bottle: warning: {msg}", file=sys.stderr)
def _format_context(context: Mapping[str, object] | None) -> str:
"""Render a context mapping as a ` [k=v k2=v2]` suffix.
Keys are sorted for stable, diffable output. Values that are empty or
contain whitespace or a quote are wrapped in double quotes (with inner
quotes escaped) so each `k=v` pair stays parseable. Empty/None context
renders as the empty string."""
if not context:
return ""
parts: list[str] = []
for key in sorted(context):
value = str(context[key])
if value == "" or any(ch.isspace() for ch in value) or '"' in value:
value = '"' + value.replace('"', '\\"') + '"'
parts.append(f"{key}={value}")
return " [" + " ".join(parts) + "]"
def error(msg: str) -> None:
print(f"bot-bottle: error: {msg}", file=sys.stderr)
def _emit(
level: int,
label: str,
msg: str,
context: Mapping[str, object] | None,
) -> None:
if level < _threshold():
return
prefix = f"{label}: " if label else ""
sys.stderr.write(f"bot-bottle: {prefix}{msg}{_format_context(context)}\n")
def debug(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(DEBUG, "debug", msg, context)
def info(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(INFO, "", msg, context)
def warn(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(WARN, "warning", msg, context)
def error(msg: str, *, context: Mapping[str, object] | None = None) -> None:
_emit(ERROR, "error", msg, context)
class Die(SystemExit):
@@ -31,6 +117,6 @@ class Die(SystemExit):
self.message = message
def die(msg: str) -> NoReturn:
error(msg)
def die(msg: str, *, context: Mapping[str, object] | None = None) -> NoReturn:
error(msg, context=context)
raise Die(1, msg)
+316 -133
View File
@@ -19,7 +19,7 @@ Bottle schema (frontmatter):
repos: { <name>: <git-gate-entry>, ... } # optional
egress: { routes: [ <egress-route>, ... ] }
# route keys: host, matches, auth, role, dlp
supervise: <bool> # optional
supervise: <bool> # optional (default true)
Agent schema (frontmatter):
bottle: <bottle-name> # required
@@ -36,10 +36,23 @@ Bottles can ONLY live under $HOME. A bottles/ dir under $CWD is a
warn at load time and contributes nothing. The trust boundary is
expressed as filesystem layout rather than resolver logic.
Validation runs once at load. Manifest.from_json_obj is preserved
as a programmatic entry point (used by tests) that takes a dict
with the same field names useful for building manifests without
on-disk files.
Two types are exported:
ManifestIndex the multi-agent/bottle collection returned by
resolve() and from_json_obj(). Used for agent
selection (all_agent_names), validation
(require_agent), and lazy loading (load_for_agent).
This is the pre-preflight form.
Manifest a single-agent/bottle value type holding exactly
one agent: ManifestAgent and one bottle:
ManifestBottle (with the agent's git-gate.user
already overlaid). Returned by load_for_agent().
This is the post-preflight form passed to backends.
ManifestIndex.from_json_obj is preserved as a programmatic entry
point (used by tests) that takes a dict with the same field names
useful for building manifests without on-disk files.
"""
from __future__ import annotations
@@ -50,26 +63,28 @@ from pathlib import Path
from typing import Mapping
from .manifest_util import ManifestError, as_json_object
from .manifest_agent import Agent, AgentProvider
from .manifest_agent import ManifestAgent, ManifestAgentProvider
from .manifest_egress import (
EGRESS_AUTH_SCHEMES,
EgressConfig,
EgressRoute,
ManifestEgressConfig,
ManifestEgressRoute,
)
from .manifest_git import GitEntry, GitUser, parse_git_gate_config
from .manifest_git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig, parse_git_gate_config
from .manifest_schema import BOTTLE_KEYS
# Re-export everything that callers currently import from this module.
__all__ = [
"ManifestError",
"GitEntry",
"GitUser",
"AgentProvider",
"ManifestGitEntry",
"ManifestGitUser",
"ManifestKeyConfig",
"ManifestAgentProvider",
"EGRESS_AUTH_SCHEMES",
"EgressRoute",
"EgressConfig",
"Agent",
"Bottle",
"ManifestEgressRoute",
"ManifestEgressConfig",
"ManifestAgent",
"ManifestBottle",
"ManifestIndex",
"Manifest",
]
@@ -86,26 +101,24 @@ def _section_dict(value: object, label: str) -> dict[str, object]:
@dataclass(frozen=True)
class Bottle:
class ManifestBottle:
env: Mapping[str, str] = field(default_factory=_empty_str_dict)
agent_provider: AgentProvider = field(default_factory=AgentProvider)
git: tuple[GitEntry, ...] = ()
agent_provider: ManifestAgentProvider = field(default_factory=ManifestAgentProvider)
git: tuple[ManifestGitEntry, ...] = ()
# Per-bottle git identity (issue #86). Empty default — bottles
# that don't set `git-gate.user:` in the manifest skip the
# `git config --global` step entirely. A bottle can declare a user
# identity without any git-gate.repos upstreams, and vice versa.
git_user: GitUser = field(default_factory=GitUser)
egress: EgressConfig = field(default_factory=EgressConfig)
# Opt-in per-bottle stuck-recovery sidecar (PRD 0013). When true,
# the launch step brings up a supervise sidecar that exposes MCP
# tools to the agent (egress-block, capability-block) plus mounts
# the current-config dir read-only into the agent at
# /etc/bot-bottle/current-config. False (the default) skips the
# sidecar and mount.
supervise: bool = False
git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
# Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
# default, issue #249), the launch step brings up a supervise
# sidecar that exposes egress MCP tools to the agent. Set
# `supervise: false` to skip the sidecar.
supervise: bool = True
@classmethod
def from_dict(cls, name: str, raw: object) -> "Bottle":
def from_dict(cls, name: str, raw: object) -> "ManifestBottle":
d = as_json_object(raw, f"bottle '{name}'")
if "runtime" in d:
@@ -157,25 +170,25 @@ class Bottle:
)
env[var] = value
git: tuple[GitEntry, ...] = ()
git_user = GitUser()
git: tuple[ManifestGitEntry, ...] = ()
git_user = ManifestGitUser()
git_raw = d.get("git-gate")
if git_raw is not None:
git, git_user = parse_git_gate_config(name, git_raw)
agent_provider = (
AgentProvider.from_dict(name, d["agent_provider"])
ManifestAgentProvider.from_dict(name, d["agent_provider"])
if "agent_provider" in d
else AgentProvider()
else ManifestAgentProvider()
)
egress = (
EgressConfig.from_dict(name, d["egress"])
ManifestEgressConfig.from_dict(name, d["egress"])
if "egress" in d
else EgressConfig()
else ManifestEgressConfig()
)
supervise_raw = d.get("supervise", False)
supervise_raw = d.get("supervise", True)
if not isinstance(supervise_raw, bool):
raise ManifestError(
f"bottle '{name}' supervise must be a boolean "
@@ -188,14 +201,123 @@ class Bottle:
)
def _merge_git_user(
agent_user: ManifestGitUser, base_user: ManifestGitUser
) -> ManifestGitUser:
"""Merge the agent's git.user over the bottle's, agent-wins-on-non-empty."""
if agent_user.is_empty():
return base_user
return ManifestGitUser(
name=agent_user.name or base_user.name,
email=agent_user.email or base_user.email,
)
def _resolve_effective_bottle_eager(
agent_name: str,
agent: "ManifestAgent",
bottle_names: "tuple[str, ...]",
bottles: "Mapping[str, ManifestBottle]",
) -> "ManifestBottle":
"""Return the effective ManifestBottle for the eager (from_json_obj) path.
When bottle_names is non-empty they are merged in order. When empty, falls
back to agent.bottle. Raises ManifestError when neither is set."""
from .manifest_extends import merge_bottles_runtime
if bottle_names:
resolved: list[ManifestBottle] = []
for bn in bottle_names:
if bn not in bottles:
available = ", ".join(sorted(bottles.keys())) or "(none)"
raise ManifestError(
f"bottle '{bn}' not defined. Available: {available}"
)
resolved.append(bottles[bn])
return merge_bottles_runtime(resolved)
if not agent.bottle:
raise ManifestError(
f"agent '{agent_name}' has no 'bottle' field and no bottles were "
f"selected at launch. Select at least one bottle or add "
f"'bottle: <name>' to the agent manifest."
)
return bottles[agent.bottle]
def _resolve_effective_bottle_lazy(
agent_name: str,
agent_bottle: str,
bottle_names: "tuple[str, ...]",
bottles_dir: "Path",
) -> "ManifestBottle":
"""Return the effective ManifestBottle for the lazy (from_md_dirs) path.
When bottle_names is non-empty they are resolved from disk and merged in
order. When empty, falls back to agent_bottle. Raises ManifestError when
neither is set."""
from .manifest_extends import merge_bottles_runtime
from .manifest_loader import load_bottle_chain_from_dir
if bottle_names:
resolved = [load_bottle_chain_from_dir(bn, bottles_dir) for bn in bottle_names]
return merge_bottles_runtime(resolved)
if not agent_bottle:
raise ManifestError(
f"agent '{agent_name}' has no 'bottle' field and no bottles were "
f"selected at launch. Select at least one bottle or add "
f"'bottle: <name>' to the agent manifest."
)
return load_bottle_chain_from_dir(agent_bottle, bottles_dir)
@dataclass(frozen=True)
class Manifest:
bottles: Mapping[str, Bottle]
agents: Mapping[str, Agent]
"""Single-agent/bottle value type. Returned by ManifestIndex.load_for_agent().
`bottle` is the effective bottle with the agent's git-gate.user already
overlaid per-field (agent wins on non-empty). Backends and provisioners
use this directly no agent_name lookup needed."""
agent: ManifestAgent
bottle: ManifestBottle
def git_identity_summary(self) -> str | None:
"""One-line effective git identity with per-field provenance, e.g.
`name=claude (agent), email=eric@dideric.is (bottle)`.
Returns None when neither agent nor bottle sets an identity."""
over = self.agent.git_user # agent's declared git_user (pre-merge)
merged = self.bottle.git_user # effective git_user (post-merge)
if merged.is_empty():
return None
parts: list[str] = []
if merged.name:
parts.append(f"name={merged.name} ({'agent' if over.name else 'bottle'})")
if merged.email:
parts.append(f"email={merged.email} ({'agent' if over.email else 'bottle'})")
return ", ".join(parts)
@dataclass(frozen=True)
class ManifestIndex:
"""Multi-agent/bottle collection. The pre-preflight form.
In lazy mode (from resolve()/from_md_dirs()) only filenames are scanned;
no file content is read. In eager mode (from from_json_obj()) all agents
and bottles are pre-parsed. Call load_for_agent() to get a single-value
Manifest ready for backend use."""
bottles: Mapping[str, ManifestBottle]
agents: Mapping[str, ManifestAgent]
# Set by from_md_dirs; None in from_json_obj (test/programmatic) mode.
# Stores the manifest root dirs so load_for_agent can locate files later.
home_md: Path | None = field(default=None)
cwd_md: Path | None = field(default=None)
@classmethod
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "Manifest":
"""Walk the per-file manifest tree and build a Manifest.
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "ManifestIndex":
"""Walk the per-file manifest tree and build a ManifestIndex.
Layout (PRD 0011):
$HOME/.bot-bottle/bottles/<name>.md bottles (home-only)
@@ -208,7 +330,7 @@ class Manifest:
boundary.
If `missing_ok` is true, a missing `$HOME/.bot-bottle/`
returns an empty manifest instead of dying. This is for
returns an empty index instead of dying. This is for
passive UI surfaces like the dashboard, which can still
monitor already-running agents without launch config.
@@ -247,25 +369,16 @@ class Manifest:
cls,
home_dir: Path,
cwd_dir: Path | None,
) -> "Manifest":
"""Programmatic entry point. Loads bottles from
`<home_dir>/bottles/`, home agents from `<home_dir>/agents/`,
and (if `cwd_dir` is passed) cwd agents from
`<cwd_dir>/agents/`. Cwd agents override home agents on
name collision. A `bottles/` subdir under `cwd_dir` is
logged as a warning and ignored.
) -> "ManifestIndex":
"""Return a names-only ManifestIndex. No file content is read; only
filenames are scanned for the agent selector. Full parsing happens
later, per-agent, via `load_for_agent`.
Used by tests to build a Manifest from fixture directories
A `bottles/` subdir under `cwd_dir` is logged as a warning and
ignored the filesystem layout IS the trust boundary.
Used by tests to build a ManifestIndex from fixture directories
without touching `os.environ`."""
bottles_dir = home_dir / "bottles"
from .manifest_loader import load_agents_from_dir, load_bottles_from_dir
bottles = load_bottles_from_dir(bottles_dir)
bottle_names = set(bottles.keys())
agents_dir = home_dir / "agents"
agents = load_agents_from_dir(agents_dir, bottle_names, source="$HOME")
if cwd_dir is not None:
stale_bottles = cwd_dir / "bottles"
if stale_bottles.is_dir():
@@ -279,17 +392,11 @@ class Manifest:
f"live under $HOME/.bot-bottle/bottles/ "
f"(PRD 0011). Move them or delete."
)
cwd_agents_dir = cwd_dir / "agents"
cwd_agents = load_agents_from_dir(
cwd_agents_dir, bottle_names, source="$CWD"
)
agents = {**agents, **cwd_agents}
return cls(bottles=bottles, agents=agents)
return cls(bottles={}, agents={}, home_md=home_dir, cwd_md=cwd_dir)
@classmethod
def from_json_obj(cls, obj: object) -> "Manifest":
"""Validate and build a Manifest from a raw JSON-like dict."""
def from_json_obj(cls, obj: object) -> "ManifestIndex":
"""Validate and build a ManifestIndex from a raw JSON-like dict."""
d = as_json_object(obj, "manifest")
raw_bottles_obj = _section_dict(d.get("bottles"), "manifest 'bottles'")
raw_agents = _section_dict(d.get("agents"), "manifest 'agents'")
@@ -305,80 +412,156 @@ class Manifest:
bottles = resolve_bottles(raw_bottles)
bottle_names = set(bottles.keys())
agents: dict[str, Agent] = {
n: Agent.from_dict(n, a, bottle_names) for n, a in raw_agents.items()
agents: dict[str, ManifestAgent] = {
n: ManifestAgent.from_dict(n, a, bottle_names) for n, a in raw_agents.items()
}
return cls(bottles=bottles, agents=agents)
@property
def all_bottle_names(self) -> list[str]:
"""Sorted list of all discoverable bottle names.
In names-only mode (from resolve/from_md_dirs) this scans bottle
filenames without reading their content. In eager mode (from
from_json_obj) it returns the pre-parsed bottles' names."""
if self.home_md is not None:
from .manifest_loader import scan_bottle_names
return scan_bottle_names(self.home_md / "bottles")
return sorted(self.bottles.keys())
@property
def all_agent_names(self) -> list[str]:
"""Sorted list of all discoverable agent names.
In names-only mode (from resolve/from_md_dirs) this scans agent
filenames without reading their content. In eager mode (from
from_json_obj) it returns the pre-parsed agents' names."""
if self.home_md is not None:
from .manifest_loader import scan_agent_names
home_names = set(scan_agent_names(self.home_md / "agents").keys())
cwd_names: set[str] = set()
if self.cwd_md is not None:
cwd_names = set(scan_agent_names(self.cwd_md / "agents").keys())
return sorted(home_names | cwd_names)
return sorted(self.agents.keys())
def load_for_agent(
self,
agent_name: str,
bottle_names: "tuple[str, ...] | None" = None,
) -> "Manifest":
"""Parse the named agent and its bottle; return a single-value Manifest.
`bottle_names` is an ordered list of bottles selected at launch time.
When non-empty they are resolved and merged in order (index 0 = base;
later entries override). When empty or None, falls back to the agent's
own `bottle:` field. Raises ManifestError when neither is set.
In lazy mode (from resolve/from_md_dirs) the agent file and its
bottle chain are read from disk for the first time here. In eager
mode (from_json_obj) the data is already parsed; this just filters
down to the requested agent and its bottle.
The returned Manifest.bottle has the agent's git-gate.user already
overlaid (agent wins on non-empty, per-field).
Always raises ManifestError if the agent is unknown or invalid.
Backends call this at preflight inside _validate."""
effective_bottle_names: tuple[str, ...] = bottle_names or ()
if self.home_md is None:
# Eager manifest (from_json_obj): data already parsed; filter to
# the one requested agent and its bottle so the returned Manifest
# always holds exactly one agent and one bottle regardless of path.
if agent_name not in self.agents:
available = ", ".join(sorted(self.agents.keys())) or "(none)"
raise ManifestError(
f"agent '{agent_name}' not defined. Available: {available}"
)
agent = self.agents[agent_name]
raw_bottle = _resolve_effective_bottle_eager(
agent_name, agent, effective_bottle_names, self.bottles
)
merged = _merge_git_user(agent.git_user, raw_bottle.git_user)
bottle = raw_bottle if merged == raw_bottle.git_user else replace(raw_bottle, git_user=merged)
return Manifest(agent=agent, bottle=bottle)
from .manifest_loader import scan_agent_names
from .manifest_schema import validate_agent_frontmatter_keys
from .yaml_subset import YamlSubsetError, parse_frontmatter
# Locate the agent file; cwd wins over home on name collision.
home_agents = scan_agent_names(self.home_md / "agents")
cwd_agents: dict[str, Path] = {}
if self.cwd_md is not None:
cwd_agents = scan_agent_names(self.cwd_md / "agents")
merged_agents = {**home_agents, **cwd_agents}
if agent_name not in merged_agents:
available = ", ".join(sorted(merged_agents.keys())) or "(none)"
raise ManifestError(
f"agent '{agent_name}' not defined. Available: {available}"
)
agent_path = merged_agents[agent_name]
try:
fm, body = parse_frontmatter(agent_path.read_text())
except OSError as e:
raise ManifestError(f"could not read {agent_path}: {e}") from e
except YamlSubsetError as e:
raise ManifestError(f"{agent_path}: {e}") from e
validate_agent_frontmatter_keys(agent_path, fm.keys())
# Determine the effective bottle name(s).
agent_bottle = fm.get("bottle") or ""
bottles_dir = self.home_md / "bottles"
raw_bottle = _resolve_effective_bottle_lazy(
agent_name, str(agent_bottle), effective_bottle_names, bottles_dir
)
effective_bottle_name = (
effective_bottle_names[-1] if effective_bottle_names
else str(agent_bottle)
)
# Build and validate the full ManifestAgent.
agent_dict: dict[str, object] = {
"skills": fm.get("skills", []),
"prompt": body.strip(),
}
if agent_bottle:
agent_dict["bottle"] = agent_bottle
if "git-gate" in fm:
agent_dict["git-gate"] = fm["git-gate"]
# Pass the effective bottle name as the known-bottles set so agents
# that have bottle: set are validated; agents without bottle: pass {}
# since bottle_names were already resolved above.
known = {effective_bottle_name} if effective_bottle_name else set()
agent = ManifestAgent.from_dict(agent_name, agent_dict, known)
merged_user = _merge_git_user(agent.git_user, raw_bottle.git_user)
bottle = raw_bottle if merged_user == raw_bottle.git_user else replace(raw_bottle, git_user=merged_user)
return Manifest(agent=agent, bottle=bottle)
def has_agent(self, name: str) -> bool:
return name in self.agents
def require_agent(self, name: str) -> None:
"""Check that `name` is a discoverable agent. In names-only mode
this checks whether the .md file exists; in eager mode it checks
the pre-parsed agents dict. Does NOT parse file content."""
if self.has_agent(name):
return
available = ", ".join(self.agents.keys())
if available:
msg = f"agent '{name}' not defined in bot-bottle.json. Available: {available}"
raise ManifestError(msg)
raise ManifestError(
f"agent '{name}' not defined in bot-bottle.json (manifest is empty)."
)
def has_bottle(self, name: str) -> bool:
return name in self.bottles
def require_bottle(self, name: str) -> None:
if self.has_bottle(name):
return
available = ", ".join(self.bottles.keys())
if available:
raise ManifestError(
f"bottle '{name}' not defined in bot-bottle.json. "
f"Available bottles: {available}"
if self.home_md is not None:
# Names-only mode: check file existence without parsing.
home_path = self.home_md / "agents" / f"{name}.md"
cwd_path = (
self.cwd_md / "agents" / f"{name}.md"
if self.cwd_md else None
)
raise ManifestError(f"bottle '{name}' not defined in bot-bottle.json (no bottles defined).")
def _effective_git_user(self, agent_name: str) -> GitUser:
"""Merge the agent's git.user over the referenced bottle's,
per-field, agent-wins-on-non-empty (issue #94). Same overlay
the `extends:` resolver applies between bottles
(`_merge_bottles`)."""
agent = self.agents[agent_name]
base = self.bottles[agent.bottle].git_user
over = agent.git_user
if over.is_empty():
return base
return GitUser(
name=over.name or base.name,
email=over.email or base.email,
if home_path.is_file() or (cwd_path and cwd_path.is_file()):
return
available = ", ".join(self.all_agent_names) or "(none)"
raise ManifestError(
f"agent '{name}' not defined. Available: {available}"
)
def bottle_for(self, agent_name: str) -> Bottle:
"""Resolve the Bottle the named agent references, with the
agent's git.user overlaid on top. The validator guarantees both
lookups succeed for a manifest built via from_json_obj.
The overlay lives here, the single point both backends call to
resolve an agent's bottle, so the docker / smolmachines git
provisioners pick up the merged identity unchanged."""
bottle = self.bottles[self.agents[agent_name].bottle]
merged = self._effective_git_user(agent_name)
if merged == bottle.git_user:
return bottle
return replace(bottle, git_user=merged)
def git_identity_summary(self, agent_name: str) -> str | None:
"""One-line effective git identity with per-field provenance
for launch summaries, e.g.
`name=claude (agent), email=eric@dideric.is (bottle)`.
Returns None when neither agent nor bottle sets an identity."""
over = self.agents[agent_name].git_user
merged = self._effective_git_user(agent_name)
if merged.is_empty():
return None
parts: list[str] = []
if merged.name:
parts.append(f"name={merged.name} ({'agent' if over.name else 'bottle'})")
if merged.email:
parts.append(f"email={merged.email} ({'agent' if over.email else 'bottle'})")
return ", ".join(parts)
+143 -24
View File
@@ -2,17 +2,17 @@
from __future__ import annotations
from dataclasses import dataclass
from dataclasses import dataclass, field
from typing import cast
from .agent_provider import PROVIDER_TEMPLATES
from .manifest_util import ManifestError, as_json_object
from .manifest_git import GitUser
from .manifest_git import ManifestGitUser
from .manifest_schema import AGENT_MODEL_KEYS
@dataclass(frozen=True)
class AgentProvider:
class ManifestAgentProvider:
"""Provider/template for the agent process inside a bottle.
`template` selects a built-in launch/runtime contract. `dockerfile`
@@ -33,15 +33,23 @@ class AgentProvider:
dockerfile: str = ""
auth_token: str = ""
forward_host_credentials: bool = False
settings: dict[str, object] = field(default_factory=dict)
@classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider":
def from_dict(cls, bottle_name: str, raw: object) -> "ManifestAgentProvider":
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
for k in d:
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}:
if k not in {
"template",
"dockerfile",
"auth_token",
"forward_host_credentials",
"settings",
}:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
f"allowed: template, dockerfile, auth_token, forward_host_credentials"
"allowed: template, dockerfile, auth_token, "
"forward_host_credentials, settings"
)
template = d.get("template", "claude")
if not isinstance(template, str) or not template:
@@ -89,27 +97,30 @@ class AgentProvider:
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
"is currently only supported for template 'codex'"
)
settings = _parse_provider_settings(bottle_name, template, d.get("settings"))
return cls(
template=template,
dockerfile=dockerfile,
auth_token=auth_token,
forward_host_credentials=forward_host_credentials,
settings=settings,
)
@dataclass(frozen=True)
class Agent:
bottle: str
class ManifestAgent:
# Optional: when empty the operator selects bottles at launch time.
bottle: str = ""
skills: tuple[str, ...] = ()
prompt: str = ""
# Per-agent git identity (issue #94). Overlays the referenced
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
# `user` is allowed at the agent level; `repos` stays bottle-only
# because it carries credentials and host trust.
git_user: GitUser = GitUser()
git_user: ManifestGitUser = ManifestGitUser()
@classmethod
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent":
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "ManifestAgent":
d = as_json_object(raw, f"agent '{name}'")
unknown = set(d.keys()) - AGENT_MODEL_KEYS
if unknown:
@@ -119,18 +130,20 @@ class Agent:
f"allowed keys are {allowed}."
)
bottle = d.get("bottle")
if not isinstance(bottle, str) or not bottle:
raise ManifestError(
f"agent '{name}' must declare a 'bottle' field naming a "
f"defined bottle"
)
if bottle not in bottle_names:
available = ", ".join(sorted(bottle_names)) or "(none defined)"
raise ManifestError(
f"agent '{name}' references bottle '{bottle}', which is not defined. "
f"Available: {available}"
)
bottle_raw = d.get("bottle")
bottle = ""
if bottle_raw is not None:
if not isinstance(bottle_raw, str) or not bottle_raw:
raise ManifestError(
f"agent '{name}' bottle must be a non-empty string when declared"
)
if bottle_raw not in bottle_names:
available = ", ".join(sorted(bottle_names)) or "(none defined)"
raise ManifestError(
f"agent '{name}' references bottle '{bottle_raw}', which is not defined. "
f"Available: {available}"
)
bottle = bottle_raw
skills: tuple[str, ...] = ()
skills_raw = d.get("skills")
@@ -164,7 +177,7 @@ class Agent:
# git-gate: agents may declare only `git-gate.user` (name/email).
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
git_user = GitUser()
git_user = ManifestGitUser()
git_raw = d.get("git-gate")
if git_raw is not None:
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
@@ -177,6 +190,112 @@ class Agent:
f"(it carries credentials and host trust)."
)
if "user" in gd:
git_user = GitUser.from_dict(name, gd["user"])
git_user = ManifestGitUser.from_dict(name, gd["user"])
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
def _parse_provider_settings(
bottle_name: str,
template: str,
raw: object,
) -> dict[str, object]:
if raw is None:
return {}
settings = as_json_object(raw, f"bottle '{bottle_name}' agent_provider.settings")
common_allowed = {"startup_args"}
pi_allowed = {
"provider",
"base_url",
"api",
"api_key",
"api_key_env",
"models",
"context_window",
"max_tokens_field",
"max_tokens",
"supports_developer_role",
"supports_reasoning_effort",
}
if template == "pi":
allowed = common_allowed | pi_allowed
elif template in ("claude", "codex"):
allowed = common_allowed
elif template not in PROVIDER_TEMPLATES:
return dict(settings)
else:
allowed = common_allowed
for key in settings:
if key not in allowed:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings has unknown "
f"key {key!r}; allowed: {', '.join(sorted(allowed))}"
)
startup_args = settings.get("startup_args")
if startup_args is not None:
if not isinstance(startup_args, list):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.startup_args "
f"must be an array of strings"
)
for i, arg in enumerate(startup_args):
if not isinstance(arg, str) or not arg:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings."
f"startup_args[{i}] must be a non-empty string"
)
if template != "pi":
return dict(settings)
for key in ("provider", "base_url", "api", "api_key", "api_key_env"):
value = settings.get(key)
if value is not None and (not isinstance(value, str) or not value):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
"be a non-empty string"
)
max_tokens_field = settings.get("max_tokens_field")
if max_tokens_field is not None and max_tokens_field not in (
"max_tokens", "max_completion_tokens",
):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.max_tokens_field "
"must be 'max_tokens' or 'max_completion_tokens'"
)
if settings.get("api_key") is not None and settings.get("api_key_env") is not None:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings may set either "
"api_key or api_key_env, not both"
)
models = settings.get("models")
if models is not None:
if not isinstance(models, list) or not models:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.models must "
"be a non-empty array of strings"
)
for i, model in enumerate(models):
if not isinstance(model, str) or not model:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.models[{i}] "
"must be a non-empty string"
)
for key in ("supports_developer_role", "supports_reasoning_effort"):
value = settings.get(key)
if value is not None and not isinstance(value, bool):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
f"be a boolean (was {type(value).__name__})"
)
for key in ("context_window", "max_tokens"):
value = settings.get(key)
if value is not None and (
not isinstance(value, int) or isinstance(value, bool) or value <= 0
):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
f"be a positive integer (was {type(value).__name__})"
)
return dict(settings)
+71 -33
View File
@@ -21,10 +21,13 @@ VALID_METHODS = frozenset({
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# What the proxy does on an outbound token match (PRD 0062).
OUTBOUND_ON_MATCH_VALUES = ("block", "redact", "supervise")
def validate_egress_routes(
bottle_name: str,
routes: tuple[EgressRoute, ...],
routes: tuple[ManifestEgressRoute, ...],
) -> None:
seen_hosts: dict[str, None] = {}
for r in routes:
@@ -38,37 +41,39 @@ def validate_egress_routes(
@dataclass(frozen=True)
class PathMatch:
class ManifestPathMatch:
Type: str = "prefix"
Value: str = ""
@dataclass(frozen=True)
class HeaderMatch:
class ManifestHeaderMatch:
Name: str = ""
Value: str = ""
Type: str = "exact"
@dataclass(frozen=True)
class MatchEntry:
Paths: tuple[PathMatch, ...] = ()
class ManifestMatchEntry:
Paths: tuple[ManifestPathMatch, ...] = ()
Methods: tuple[str, ...] = ()
Headers: tuple[HeaderMatch, ...] = ()
Headers: tuple[ManifestHeaderMatch, ...] = ()
@dataclass(frozen=True)
class EgressRoute:
class ManifestEgressRoute:
Host: str
Matches: tuple[MatchEntry, ...] = ()
Matches: tuple[ManifestMatchEntry, ...] = ()
AuthScheme: str = ""
TokenRef: str = ""
Role: tuple[str, ...] = ()
GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None
InboundDetectors: tuple[str, ...] | None = None
OutboundOnMatch: str = ""
@classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
label = f"bottle '{bottle_name}' egress.routes[{idx}]"
d = as_json_object(raw, label)
host = d.get("host")
@@ -76,7 +81,7 @@ class EgressRoute:
raise ManifestError(f"{label} missing required string field 'host'")
# --- matches ---
matches: tuple[MatchEntry, ...] = ()
matches: tuple[ManifestMatchEntry, ...] = ()
matches_raw = d.get("matches")
if matches_raw is not None:
if not isinstance(matches_raw, list):
@@ -85,7 +90,7 @@ class EgressRoute:
f"(was {type(matches_raw).__name__})"
)
matches_list = cast(list[object], matches_raw)
entries: list[MatchEntry] = []
entries: list[ManifestMatchEntry] = []
for k, entry_raw in enumerate(matches_list):
entries.append(
_parse_match_entry(label, k, entry_raw)
@@ -160,16 +165,36 @@ class EgressRoute:
# --- dlp ---
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
outbound_on_match = ""
if "dlp" in d:
outbound_detectors, inbound_detectors = _parse_dlp_block(
outbound_detectors, inbound_detectors, outbound_on_match = _parse_dlp_block(
label, d.get("dlp"),
)
# --- git-over-HTTPS policy ---
git_fetch = False
if "git" in d:
git_d = as_json_object(d.get("git"), f"{label} git")
raw_fetch = git_d.get("fetch", False)
if isinstance(raw_fetch, bool):
git_fetch = raw_fetch
else:
raise ManifestError(
f"{label} git.fetch must be a boolean "
f"(was {type(raw_fetch).__name__})"
)
for k in git_d:
if k != "fetch":
raise ManifestError(
f"{label} git has unknown key {k!r}; "
f"only 'fetch' is accepted"
)
for k in d:
if k not in ("host", "matches", "auth", "role", "dlp"):
if k not in ("host", "matches", "auth", "role", "dlp", "git"):
raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are "
f"'host', 'matches', 'auth', 'role', 'dlp'"
f"'host', 'matches', 'auth', 'role', 'dlp', 'git'"
)
return cls(
@@ -178,24 +203,26 @@ class EgressRoute:
AuthScheme=auth_scheme,
TokenRef=token_ref,
Role=roles,
GitFetch=git_fetch,
OutboundDetectors=outbound_detectors,
InboundDetectors=inbound_detectors,
OutboundOnMatch=outbound_on_match,
)
def _parse_match_entry(
route_label: str, k: int, raw: object,
) -> MatchEntry:
) -> ManifestMatchEntry:
label = f"{route_label} matches[{k}]"
d = as_json_object(raw, label)
paths: tuple[PathMatch, ...] = ()
paths: tuple[ManifestPathMatch, ...] = ()
paths_raw = d.get("paths")
if paths_raw is not None:
if not isinstance(paths_raw, list):
raise ManifestError(f"{label} paths must be an array")
paths_list = cast(list[object], paths_raw)
parsed_paths: list[PathMatch] = []
parsed_paths: list[ManifestPathMatch] = []
for j, p_raw in enumerate(paths_list):
parsed_paths.append(_parse_path_match(label, j, p_raw))
paths = tuple(parsed_paths)
@@ -220,13 +247,13 @@ def _parse_match_entry(
normalised.append(upper)
methods = tuple(normalised)
headers: tuple[HeaderMatch, ...] = ()
headers: tuple[ManifestHeaderMatch, ...] = ()
headers_raw = d.get("headers")
if headers_raw is not None:
if not isinstance(headers_raw, list):
raise ManifestError(f"{label} headers must be an array")
headers_list = cast(list[object], headers_raw)
parsed_headers: list[HeaderMatch] = []
parsed_headers: list[ManifestHeaderMatch] = []
for j, h_raw in enumerate(headers_list):
parsed_headers.append(_parse_header_match(label, j, h_raw))
headers = tuple(parsed_headers)
@@ -235,12 +262,12 @@ def _parse_match_entry(
if key not in ("paths", "methods", "headers"):
raise ManifestError(f"{label} has unknown key {key!r}")
return MatchEntry(Paths=paths, Methods=methods, Headers=headers)
return ManifestMatchEntry(Paths=paths, Methods=methods, Headers=headers)
def _parse_path_match(
entry_label: str, j: int, raw: object,
) -> PathMatch:
) -> ManifestPathMatch:
label = f"{entry_label} paths[{j}]"
d = as_json_object(raw, label)
ptype = d.get("type", "prefix")
@@ -266,12 +293,12 @@ def _parse_path_match(
for k in d:
if k not in ("type", "value"):
raise ManifestError(f"{label} has unknown key {k!r}")
return PathMatch(Type=ptype, Value=value)
return ManifestPathMatch(Type=ptype, Value=value)
def _parse_header_match(
entry_label: str, j: int, raw: object,
) -> HeaderMatch:
) -> ManifestHeaderMatch:
label = f"{entry_label} headers[{j}]"
d = as_json_object(raw, label)
name = d.get("name")
@@ -296,13 +323,13 @@ def _parse_header_match(
for k in d:
if k not in ("name", "value", "type"):
raise ManifestError(f"{label} has unknown key {k!r}")
return HeaderMatch(Name=name, Value=value, Type=htype)
return ManifestHeaderMatch(Name=name, Value=value, Type=htype)
def _parse_dlp_block(
route_label: str,
raw: object,
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]:
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
label = f"{route_label} dlp"
d = as_json_object(raw, label)
@@ -337,28 +364,39 @@ def _parse_dlp_block(
outbound = _parse_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = d.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ManifestError(
f"{label} outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in d:
if k not in ("outbound_detectors", "inbound_detectors"):
if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are "
f"'outbound_detectors', 'inbound_detectors'"
f"'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
)
return outbound, inbound
return outbound, inbound, on_match
LOG_LEVELS = frozenset({0, 1, 2})
@dataclass(frozen=True)
class EgressConfig:
routes: tuple[EgressRoute, ...] = ()
class ManifestEgressConfig:
routes: tuple[ManifestEgressRoute, ...] = ()
Log: int = 0
@classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig":
def from_dict(cls, bottle_name: str, raw: object) -> "ManifestEgressConfig":
d = as_json_object(raw, f"bottle '{bottle_name}' egress")
routes_raw = d.get("routes")
routes: tuple[EgressRoute, ...] = ()
routes: tuple[ManifestEgressRoute, ...] = ()
if routes_raw is not None:
if not isinstance(routes_raw, list):
raise ManifestError(
@@ -367,7 +405,7 @@ class EgressConfig:
)
routes_list = cast(list[object], routes_raw)
routes = tuple(
EgressRoute.from_dict(bottle_name, i, entry)
ManifestEgressRoute.from_dict(bottle_name, i, entry)
for i, entry in enumerate(routes_list)
)
validate_egress_routes(bottle_name, routes)
+269 -55
View File
@@ -5,25 +5,83 @@ from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .manifest import Bottle, GitEntry
from .manifest import ManifestBottle
from .manifest_egress import ManifestEgressConfig
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
"""Apply `extends:` chains and return resolved Bottle objects."""
cache: dict[str, Bottle] = {}
def merge_bottles_runtime(bottles: "list[ManifestBottle]") -> "ManifestBottle":
"""Merge an ordered list of pre-resolved ManifestBottle objects.
Index 0 is the base; each subsequent entry is applied on top using
the same field-merge rules as the file-based extends machinery:
env: dict merge, later wins; git_user: per-field overlay, later
wins on non-empty; git (repos): union by name, later wins; egress
routes: concatenate; agent_provider, supervise: later replaces.
"""
if not bottles:
raise ValueError("merge_bottles_runtime requires at least one bottle")
result = bottles[0]
for override in bottles[1:]:
result = _merge_two_bottles_runtime(result, override)
return result
def _merge_two_bottles_runtime(base: "ManifestBottle", override: "ManifestBottle") -> "ManifestBottle":
from .manifest import ManifestBottle, ManifestGitUser
from .manifest_egress import ManifestEgressConfig
merged_env = {**base.env, **override.env}
merged_git_user = ManifestGitUser(
name=override.git_user.name or base.git_user.name,
email=override.git_user.email or base.git_user.email,
)
# git repos: union keyed by Name, override wins per-name.
base_repos_by_name = {entry.Name: entry for entry in base.git}
override_repos_by_name = {entry.Name: entry for entry in override.git}
merged_repos_names = list(base_repos_by_name) + [
n for n in override_repos_by_name if n not in base_repos_by_name
]
merged_git = tuple(
override_repos_by_name.get(n, base_repos_by_name[n])
for n in merged_repos_names
)
merged_routes = base.egress.routes + override.egress.routes
merged_egress = ManifestEgressConfig(routes=merged_routes, Log=override.egress.Log)
return ManifestBottle(
env=merged_env,
agent_provider=override.agent_provider,
git=merged_git,
git_user=merged_git_user,
egress=merged_egress,
supervise=override.supervise,
)
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, ManifestBottle]:
"""Apply `extends:` chains and return resolved ManifestBottle objects."""
cache: dict[str, ManifestBottle] = {}
# Per-bottle effective git-gate.repos, as raw dicts keyed by repo name.
# Threaded alongside `cache` so a child can field-merge against its
# parent's repos without reconstructing them from parsed entries.
repos_cache: dict[str, dict[str, object]] = {}
for name in raws:
if name not in cache:
_resolve_one_bottle(name, raws, cache, ())
_resolve_one_bottle(name, raws, cache, repos_cache, ())
return cache
def _resolve_one_bottle(
name: str,
raws: dict[str, dict[str, object]],
cache: dict[str, Bottle],
cache: dict[str, ManifestBottle],
repos_cache: dict[str, dict[str, object]],
seen: tuple[str, ...],
) -> Bottle:
from .manifest import Bottle, ManifestError
) -> ManifestBottle:
from .manifest import ManifestBottle, ManifestError
if name in cache:
return cache[name]
@@ -32,76 +90,189 @@ def _resolve_one_bottle(
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
raw = raws[name]
parent_name_raw = raw.get("extends")
# Strip `extends:` before passing to Bottle.from_dict so it
# is not accidentally treated as a real Bottle field by future
# Strip `extends:` before passing to ManifestBottle.from_dict so it
# is not accidentally treated as a real ManifestBottle field by future
# schema additions. It is only meaningful here.
child_raw = {k: v for k, v in raw.items() if k != "extends"}
if parent_name_raw is None:
bottle = Bottle.from_dict(name, child_raw)
bottle = ManifestBottle.from_dict(name, child_raw)
cache[name] = bottle
repos_cache[name] = _resolve_repos_raw({}, child_raw)
return bottle
if not isinstance(parent_name_raw, str):
# Normalize to list, accepting both str and list[str].
raw_list: list[object]
if isinstance(parent_name_raw, str):
raw_list = [parent_name_raw]
elif isinstance(parent_name_raw, list):
raw_list = parent_name_raw
else:
raise ManifestError(
f"bottle '{name}' extends must be a string "
f"bottle '{name}' extends must be a string or list of strings "
f"(was {type(parent_name_raw).__name__})"
)
parent_name: str = parent_name_raw
if parent_name == name:
raise ManifestError(
f"bottle '{name}' extends itself; remove the "
f"self-reference"
)
if parent_name not in raws:
avail = ", ".join(sorted(raws.keys())) or "(none)"
raise ManifestError(
f"bottle '{name}' extends '{parent_name}' which is not "
f"defined. Available bottles: {avail}"
)
parent = _resolve_one_bottle(parent_name, raws, cache, seen + (name,))
bottle = _merge_bottles(parent, child_raw, name)
# Validate each entry before resolving any of them.
parent_names: list[str] = []
for i, pname in enumerate(raw_list):
if not isinstance(pname, str):
raise ManifestError(
f"bottle '{name}' extends[{i}] must be a string "
f"(was {type(pname).__name__})"
)
parent_names.append(pname)
if pname == name:
raise ManifestError(
f"bottle '{name}' extends itself; remove the self-reference"
)
if pname not in raws:
avail = ", ".join(sorted(raws.keys())) or "(none)"
raise ManifestError(
f"bottle '{name}' extends '{pname}' which is not "
f"defined. Available bottles: {avail}"
)
combined_parent, combined_repos_raw = _fold_parents(
parent_names, raws, cache, repos_cache, seen + (name,)
)
merged_repos_raw = _resolve_repos_raw(combined_repos_raw, child_raw)
bottle = _merge_bottles(combined_parent, child_raw, merged_repos_raw, name)
cache[name] = bottle
repos_cache[name] = merged_repos_raw
return bottle
def _merge_bottles(
parent: Bottle,
child_raw: dict[str, object],
name: str,
) -> Bottle:
"""Apply PRD 0025 merge rules."""
from .manifest import Bottle, GitUser
from .manifest_egress import validate_egress_routes
def _fold_parents(
parent_names: list[str],
raws: dict[str, dict[str, object]],
cache: dict[str, ManifestBottle],
repos_cache: dict[str, dict[str, object]],
seen: tuple[str, ...],
) -> tuple[ManifestBottle, dict[str, object]]:
"""Resolve each parent and fold them left-to-right.
# Parse the child's declared fields into a Bottle (with the
Later parents win over earlier ones on conflict. The `seen` tuple
carries the current bottle's name so cycle detection works across
every parent edge in the multi-parent graph."""
first = parent_names[0]
effective = _resolve_one_bottle(first, raws, cache, repos_cache, seen)
effective_repos_raw = repos_cache[first]
for pname in parent_names[1:]:
later = _resolve_one_bottle(pname, raws, cache, repos_cache, seen)
later_repos_raw = repos_cache[pname]
effective, effective_repos_raw = _fold_two_bottles(
effective, effective_repos_raw, later, later_repos_raw
)
return effective, effective_repos_raw
def _fold_two_bottles(
earlier: ManifestBottle,
earlier_repos_raw: dict[str, object],
later: ManifestBottle,
later_repos_raw: dict[str, object],
) -> tuple[ManifestBottle, dict[str, object]]:
"""Combine two resolved parent bottles; later wins over earlier."""
from .manifest import ManifestBottle, ManifestGitUser
from .manifest_egress import ManifestEgressConfig
from .manifest_git import parse_git_gate_config
from .manifest_util import as_json_object
merged_env = {**earlier.env, **later.env}
merged_git_user = ManifestGitUser(
name=later.git_user.name or earlier.git_user.name,
email=later.git_user.email or earlier.git_user.email,
)
# Repos: union by name; for same-name entries, later wins per-field.
# Unlike _resolve_repos_raw, an empty later_repos_raw means "no repos
# declared" — it does NOT clear the earlier parent's repos.
names = list(earlier_repos_raw) + [
n for n in later_repos_raw if n not in earlier_repos_raw
]
merged_repos_raw: dict[str, object] = {
n: {
**as_json_object(earlier_repos_raw.get(n, {}), "earlier parent repo"),
**as_json_object(later_repos_raw.get(n, {}), "later parent repo"),
}
for n in names
}
if merged_repos_raw:
merged_git, _ = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
else:
merged_git = ()
# Egress: routes concatenate; scalar fields use last-wins.
merged_egress = ManifestEgressConfig(
routes=earlier.egress.routes + later.egress.routes,
Log=later.egress.Log,
)
return ManifestBottle(
env=merged_env,
agent_provider=later.agent_provider,
git=merged_git,
git_user=merged_git_user,
egress=merged_egress,
supervise=later.supervise,
), merged_repos_raw
def _merge_bottles(
parent: ManifestBottle,
child_raw: dict[str, object],
merged_repos_raw: dict[str, object],
name: str,
) -> ManifestBottle:
"""Apply PRD 0025 merge rules."""
from .manifest import ManifestBottle, ManifestGitUser
from .manifest_egress import validate_egress_routes
from .manifest_util import as_json_object
# git-gate.repos: when the child declares repos, inject the already
# name-merged repo set (computed by _resolve_repos_raw) so the child
# parses with the full inherited+overridden list (issue #237).
if _child_declares_git_gate_repos(child_raw):
git_raw = as_json_object(child_raw.get("git-gate", {}), "child git-gate")
child_raw = {**child_raw, "git-gate": {**git_raw, "repos": merged_repos_raw}}
# Parse the child's declared fields into a ManifestBottle (with the
# usual defaults for anything missing). Validation runs the same
# way it would for a leaf bottle: typos / wrong types die here.
child = Bottle.from_dict(name, child_raw)
child = ManifestBottle.from_dict(name, child_raw)
# env: dict merge, child wins on collision.
merged_env = {**parent.env, **child.env}
# git-gate.user: per-field overlay. Each non-empty field on child
# wins; empties fall through to parent. The default GitUser()
# wins; empties fall through to parent. The default ManifestGitUser()
# is two empty strings, so a child that omits git-gate.user
# inherits the parent's user verbatim.
merged_git_user = GitUser(
merged_git_user = ManifestGitUser(
name=child.git_user.name or parent.git_user.name,
email=child.git_user.email or parent.git_user.email,
)
# git-gate.repos: missing means inherit; an explicit empty object
# clears; otherwise parent and child merge by UpstreamHost with
# child entries replacing duplicate hosts.
# git-gate.repos: when declared, child.git already holds the merged
# set (an explicit empty dict clears parent, leaving child.git empty).
# When omitted, the parent's entries are inherited verbatim.
if _child_declares_git_gate_repos(child_raw):
merged_git = _merge_git_remotes(parent.git, child.git) if child.git else ()
merged_git = child.git
else:
merged_git = parent.git
# Presence-driven full-replace for the remaining list-valued +
# scalar fields.
merged_egress = child.egress if "egress" in child_raw else parent.egress
# egress.routes: missing means inherit; otherwise parent and child
# route lists concatenate. Other egress scalar fields remain
# presence-driven overlays.
merged_egress = (
_merge_egress(parent.egress, child.egress, child_raw)
if "egress" in child_raw
else parent.egress
)
# Presence-driven full-replace for the remaining scalar fields.
merged_agent_provider = (
child.agent_provider
if "agent_provider" in child_raw
@@ -112,7 +283,7 @@ def _merge_bottles(
)
validate_egress_routes(name, merged_egress.routes)
return Bottle(
return ManifestBottle(
env=merged_env,
agent_provider=merged_agent_provider,
git=merged_git,
@@ -122,6 +293,45 @@ def _merge_bottles(
)
def _resolve_repos_raw(
parent_repos: dict[str, object],
child_raw: dict[str, object],
) -> dict[str, object]:
"""Compute a bottle's effective git-gate.repos as raw dicts.
Repos are keyed by name. When the child omits git-gate.repos it
inherits the parent's set verbatim; an explicit empty dict clears it.
Otherwise parent and child unite by name, with same-name entries
field-merged (parent fields are defaults, child fields win)."""
from .manifest_util import as_json_object
if not _child_declares_git_gate_repos(child_raw):
return parent_repos
child_repos = _declared_repos_raw(child_raw)
if not child_repos:
return {}
# Parent entries keep their order; child-only names are appended.
names = list(parent_repos) + [n for n in child_repos if n not in parent_repos]
return {
name: {
**as_json_object(parent_repos.get(name, {}), "parent git-gate repo"),
**as_json_object(child_repos.get(name, {}), "child git-gate repo"),
}
for name in names
}
def _declared_repos_raw(child_raw: dict[str, object]) -> dict[str, object]:
"""Return the child's explicitly declared git-gate.repos as raw dicts,
or an empty dict when none are declared."""
from .manifest_util import as_json_object
if not _child_declares_git_gate_repos(child_raw):
return {}
git_raw = as_json_object(child_raw.get("git-gate", {}), "child git-gate")
return as_json_object(git_raw.get("repos", {}), "child git-gate.repos")
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
from .manifest_util import as_json_object
@@ -132,11 +342,15 @@ def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
return "repos" in git_obj
def _merge_git_remotes(
parent: tuple[GitEntry, ...],
child: tuple[GitEntry, ...],
) -> tuple[GitEntry, ...]:
by_host = {entry.UpstreamHost: entry for entry in parent}
for entry in child:
by_host[entry.UpstreamHost] = entry
return tuple(by_host.values())
def _merge_egress(
parent: ManifestEgressConfig,
child: ManifestEgressConfig,
child_raw: dict[str, object],
) -> ManifestEgressConfig:
from .manifest_egress import ManifestEgressConfig
from .manifest_util import as_json_object
child_egress_raw = as_json_object(child_raw.get("egress"), "child egress")
routes = parent.routes + child.routes
log = child.Log if "log" in child_egress_raw else parent.Log
return ManifestEgressConfig(routes=routes, Log=log)
+83 -76
View File
@@ -4,7 +4,6 @@ from __future__ import annotations
import re
from dataclasses import dataclass
from typing import Optional
from .manifest_util import ManifestError, as_json_object
@@ -13,6 +12,8 @@ from .manifest_util import ManifestError, as_json_object
# defence; this regex is belt-and-suspenders and documents intent).
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
_KEY_PROVIDERS = {"static", "gitea"}
def _opt_str(value: object, label: str) -> str:
if value is None:
@@ -57,7 +58,7 @@ def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
return (user, host, port, path)
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None:
def validate_unique_git_names(bottle_name: str, git: tuple[ManifestGitEntry, ...]) -> None:
seen: dict[str, None] = {}
for g in git:
if g.Name in seen:
@@ -69,25 +70,27 @@ def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> No
@dataclass(frozen=True)
class ProvisionedKeyConfig:
"""Configuration for automatic deploy-key lifecycle management
(PRD 0048). Used when a git-gate.repos entry opts out of a
static identity file and instead wants a fresh SSH keypair
generated at spin-up and revoked at teardown.
class ManifestKeyConfig:
"""Configuration for a repo's SSH key in git-gate.repos.
`provider` names the contrib sub-package to load (e.g. `gitea`).
`token_env` is the name of a host-side env var carrying the API
token; the value is read at provision time, never stored on the
plan. `api_url` is the forge's HTTP API root; if empty, it is
derived from the upstream URL's host at provision time."""
`provider` is either `"static"` (a pre-existing key on the host) or
`"gitea"` (automatic deploy-key lifecycle via the Gitea API).
For `static`: `path` is the host-side absolute path to the SSH private key.
For `gitea`: `forge_token_env` is the name of a host-side env var
carrying the Gitea API token; the value is read at provision time,
never stored on the plan. `api_url` is the forge's HTTP API root; if
empty, it is derived from the upstream URL's host at provision time."""
provider: str
token_env: str
path: str = ""
forge_token_env: str = ""
api_url: str = ""
@dataclass(frozen=True)
class GitEntry:
class ManifestGitEntry:
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
talk to. `Upstream` is the real remote URL the agent would push to
if there were no gate; the gate hosts a bare repo at /git/<Name>.git
@@ -99,15 +102,16 @@ class GitEntry:
stashed in the `Upstream*` fields so the git-gate render step
doesn't have to re-parse.
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). Exactly
one of `identity` (static key path) or `provisioned_key` (automatic
lifecycle) must be present. The internal field names are stable."""
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). A `key`
block is required; `key.provider` is `"static"` or `"gitea"`. For
`static`, `IdentityFile` is populated at parse time from `key.path`.
For `gitea`, `IdentityFile` is populated at provision time."""
Name: str
Upstream: str
Key: ManifestKeyConfig = ManifestKeyConfig(provider="")
IdentityFile: str = ""
KnownHostKey: str = ""
ProvisionedKey: Optional[ProvisionedKeyConfig] = None
RemoteKey: str = ""
UpstreamUser: str = ""
UpstreamHost: str = ""
@@ -117,11 +121,11 @@ class GitEntry:
@classmethod
def from_repos_entry(
cls, bottle_name: str, repo_name: str, raw: object
) -> "GitEntry":
) -> "ManifestGitEntry":
"""Parse one entry from `git-gate.repos.<repo_name>`.
YAML keys: `url` (required), exactly one of `identity` or
`provisioned_key` (required), `host_key` (optional).
YAML keys: `url` (required), `key` (required object with
`provider`, and provider-specific fields), `host_key` (optional).
The repo_name becomes `Name`."""
if not repo_name:
raise ManifestError(
@@ -135,10 +139,10 @@ class GitEntry:
label = f"git-gate.repos[{repo_name!r}]"
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
for k in d:
if k not in {"url", "identity", "provisioned_key", "host_key"}:
if k not in {"url", "key", "host_key"}:
raise ManifestError(
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
f"allowed: url, identity, provisioned_key, host_key"
f"allowed: url, key, host_key"
)
upstream = d.get("url")
if not isinstance(upstream, str) or not upstream:
@@ -146,32 +150,13 @@ class GitEntry:
f"bottle '{bottle_name}' {label} missing required string field 'url'"
)
has_identity = "identity" in d
has_provisioned = "provisioned_key" in d
if has_identity and has_provisioned:
if "key" not in d:
raise ManifestError(
f"bottle '{bottle_name}' {label} must set exactly one of "
f"'identity' or 'provisioned_key'; got both."
)
if not has_identity and not has_provisioned:
raise ManifestError(
f"bottle '{bottle_name}' {label} must set exactly one of "
f"'identity' or 'provisioned_key'; got neither."
f"bottle '{bottle_name}' {label} missing required 'key' block"
)
key_config = _parse_key_config(bottle_name, label, d["key"])
ident = ""
provisioned_key: Optional[ProvisionedKeyConfig] = None
if has_identity:
raw_ident = d.get("identity")
if not isinstance(raw_ident, str) or not raw_ident:
raise ManifestError(
f"bottle '{bottle_name}' {label} 'identity' must be a non-empty string"
)
ident = raw_ident
else:
provisioned_key = _parse_provisioned_key_config(
bottle_name, label, d["provisioned_key"]
)
ident = key_config.path if key_config.provider == "static" else ""
khk = _opt_str(
d.get("host_key"),
@@ -183,9 +168,9 @@ class GitEntry:
return cls(
Name=repo_name,
Upstream=upstream,
Key=key_config,
IdentityFile=ident,
KnownHostKey=khk,
ProvisionedKey=provisioned_key,
RemoteKey=host,
UpstreamUser=user,
UpstreamHost=host,
@@ -194,42 +179,64 @@ class GitEntry:
)
def _parse_provisioned_key_config(
def _parse_key_config(
bottle_name: str, label: str, raw: object
) -> ProvisionedKeyConfig:
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.provisioned_key")
for k in d:
if k not in {"provider", "token_env", "api_url"}:
raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key has unknown key {k!r}; "
f"allowed: provider, token_env, api_url"
)
) -> ManifestKeyConfig:
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.key")
provider = d.get("provider")
if not isinstance(provider, str) or not provider:
raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
f"bottle '{bottle_name}' {label}.key missing required "
f"string field 'provider'"
)
token_env = d.get("token_env")
if not isinstance(token_env, str) or not token_env:
if provider not in _KEY_PROVIDERS:
raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
f"string field 'token_env'"
f"bottle '{bottle_name}' {label}.key provider {provider!r} is unknown; "
f"allowed: {', '.join(sorted(_KEY_PROVIDERS))}"
)
api_url_raw = d.get("api_url", "")
if not isinstance(api_url_raw, str):
if provider == "gitea":
for k in d:
if k not in {"provider", "forge_token_env", "api_url"}:
raise ManifestError(
f"bottle '{bottle_name}' {label}.key has unknown key {k!r} "
f"for provider 'gitea'; allowed: provider, forge_token_env, api_url"
)
forge_token_env = d.get("forge_token_env")
if not isinstance(forge_token_env, str) or not forge_token_env:
raise ManifestError(
f"bottle '{bottle_name}' {label}.key missing required "
f"string field 'forge_token_env' for provider 'gitea'"
)
api_url_raw = d.get("api_url", "")
if not isinstance(api_url_raw, str):
raise ManifestError(
f"bottle '{bottle_name}' {label}.key 'api_url' must be a string"
)
return ManifestKeyConfig(
provider=provider,
forge_token_env=forge_token_env,
api_url=api_url_raw,
)
# provider == "static"
for k in d:
if k not in {"provider", "path"}:
raise ManifestError(
f"bottle '{bottle_name}' {label}.key has unknown key {k!r} "
f"for provider 'static'; allowed: provider, path"
)
path = d.get("path")
if not isinstance(path, str) or not path:
raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key 'api_url' must be a string"
f"bottle '{bottle_name}' {label}.key missing required "
f"string field 'path' for provider 'static'"
)
return ProvisionedKeyConfig(
provider=provider,
token_env=token_env,
api_url=api_url_raw,
)
return ManifestKeyConfig(provider=provider, path=path)
@dataclass(frozen=True)
class GitUser:
class ManifestGitUser:
"""Per-bottle `git config --global user.name` / `user.email`
pair (issue #86). The agent's commits inside the bottle are
attributed to this identity rather than the agent image's
@@ -244,7 +251,7 @@ class GitUser:
email: str = ""
@classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
def from_dict(cls, bottle_name: str, raw: object) -> "ManifestGitUser":
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
for k in d:
if k not in {"name", "email"}:
@@ -279,7 +286,7 @@ class GitUser:
def parse_git_gate_config(
bottle_name: str,
raw: object,
) -> tuple[tuple[GitEntry, ...], GitUser]:
) -> tuple[tuple[ManifestGitEntry, ...], ManifestGitUser]:
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
for k in d:
if k not in {"user", "repos"}:
@@ -289,17 +296,17 @@ def parse_git_gate_config(
)
git_user = (
GitUser.from_dict(bottle_name, d["user"])
ManifestGitUser.from_dict(bottle_name, d["user"])
if "user" in d
else GitUser()
else ManifestGitUser()
)
git: tuple[GitEntry, ...] = ()
git: tuple[ManifestGitEntry, ...] = ()
repos_raw = d.get("repos")
if repos_raw is not None:
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
git = tuple(
GitEntry.from_repos_entry(bottle_name, name, entry)
ManifestGitEntry.from_repos_entry(bottle_name, name, entry)
for name, entry in repos.items()
)
validate_unique_git_names(bottle_name, git)
+53 -46
View File
@@ -8,21 +8,19 @@ from typing import TYPE_CHECKING
from .log import warn
from .manifest_schema import (
entity_name_from_path,
validate_agent_frontmatter_keys,
validate_bottle_frontmatter_keys,
)
from .manifest_util import ManifestError
from .yaml_subset import YamlSubsetError, parse_frontmatter
if TYPE_CHECKING:
from .manifest import Agent, Bottle
from .manifest import ManifestBottle
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
not. The manifest format changed in PRD 0011 and we do not want
to silently leave the JSON content unused."""
from .manifest import ManifestError
legacy = dir_path / "bot-bottle.json"
if legacy.is_file() and not md_dir.exists():
raise ManifestError(
@@ -34,15 +32,13 @@ def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
)
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
`{name: Bottle}`. Missing dir returns an empty dict."""
from .manifest import ManifestError
from .manifest_extends import resolve_bottles
def scan_bottle_names(bottles_dir: Path) -> list[str]:
"""Scan `<bottles_dir>/*.md` for valid filenames and return sorted bottle names.
raws: dict[str, dict[str, object]] = {}
No file content is read. Invalid filenames are skipped with a warning."""
result: list[str] = []
if not bottles_dir.is_dir():
return {}
return result
for path in sorted(bottles_dir.glob("*.md")):
name = entity_name_from_path(path)
if name is None:
@@ -51,31 +47,17 @@ def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
)
continue
try:
fm, _body = parse_frontmatter(path.read_text())
except OSError as e:
raise ManifestError(f"could not read {path}: {e}") from e
except YamlSubsetError as e:
raise ManifestError(f"{path}: {e}") from e
validate_bottle_frontmatter_keys(path, fm.keys())
raws[name] = fm
return resolve_bottles(raws)
result.append(name)
return result
def load_agents_from_dir(
agents_dir: Path,
bottle_names: set[str],
*,
source: str, # noqa: F841 — unused, but required by interface
) -> dict[str, Agent]:
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
`{name: Agent}`. The Markdown body becomes the agent's prompt.
Missing dir returns an empty dict."""
from .manifest import Agent, ManifestError
def scan_agent_names(agents_dir: Path) -> dict[str, Path]:
"""Scan `<agents_dir>/*.md` for valid filenames and return `{name: path}`.
out: dict[str, Agent] = {}
No file content is read. Invalid filenames are skipped with a warning."""
result: dict[str, Path] = {}
if not agents_dir.is_dir():
return out
return result
for path in sorted(agents_dir.glob("*.md")):
name = entity_name_from_path(path)
if name is None:
@@ -84,22 +66,47 @@ def load_agents_from_dir(
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
)
continue
result[name] = path
return result
def load_bottle_chain_from_dir(
bottle_name: str, bottles_dir: Path
) -> ManifestBottle:
"""Load `bottle_name` and its full `extends:` chain from `bottles_dir`,
returning the resolved ManifestBottle.
Only the files in the extends chain are read unrelated bottle files
are never touched. Raises ManifestError on parse or validation failure."""
from .manifest_extends import resolve_bottles
raws: dict[str, dict[str, object]] = {}
to_load = [bottle_name]
while to_load:
name = to_load.pop()
if name in raws:
continue
path = bottles_dir / f"{name}.md"
if not path.is_file():
avail = ", ".join(
p.stem for p in sorted(bottles_dir.glob("*.md")) if p.is_file()
) or "(none)"
raise ManifestError(
f"bottle '{name}' not found at {path}. "
f"Available: {avail}"
)
try:
fm, body = parse_frontmatter(path.read_text())
fm, _body = parse_frontmatter(path.read_text())
except OSError as e:
raise ManifestError(f"could not read {path}: {e}") from e
except YamlSubsetError as e:
raise ManifestError(f"{path}: {e}") from e
validate_agent_frontmatter_keys(path, fm.keys())
# Build the dict Agent.from_dict expects. The body becomes
# prompt; Claude Code passthrough fields stay in fm and get
# ignored by Agent.from_dict (reads bottle/skills/git-gate/prompt).
agent_dict: dict[str, object] = {
"bottle": fm.get("bottle"),
"skills": fm.get("skills", []),
"prompt": body.strip(),
}
if "git-gate" in fm:
agent_dict["git-gate"] = fm["git-gate"]
out[name] = Agent.from_dict(name, agent_dict, bottle_names)
return out
validate_bottle_frontmatter_keys(path, fm.keys())
raws[name] = dict(fm)
parent = fm.get("extends")
if isinstance(parent, str):
to_load.append(parent)
elif isinstance(parent, list):
to_load.extend(p for p in parent if isinstance(p, str))
return resolve_bottles(raws)[bottle_name]
+2 -2
View File
@@ -18,8 +18,8 @@ _FILENAME_RX = re.compile(r"^[a-z][a-z0-9-]*$")
BOTTLE_KEYS = frozenset(
{"env", "extends", "agent_provider", "git-gate", "egress", "supervise"}
)
AGENT_KEYS_REQUIRED = frozenset({"bottle"})
AGENT_KEYS_OPTIONAL = frozenset({"skills", "git-gate"})
AGENT_KEYS_REQUIRED: frozenset[str] = frozenset()
AGENT_KEYS_OPTIONAL = frozenset({"bottle", "skills", "git-gate"})
# Claude Code subagent fields bot-bottle ignores at launch but does
# not reject. This lets the same file double as
+20 -2
View File
@@ -59,6 +59,7 @@ class _DaemonSpec:
# reads to inject `Authorization` headers on configured routes;
# no other daemon in the bundle should see these values.
_EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",)
_READY_GATED_DAEMONS: tuple[str, ...] = ("git-gate", "git-http")
def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]:
@@ -82,6 +83,22 @@ _DAEMONS: tuple[_DaemonSpec, ...] = (
)
def _argv_for_daemon(name: str, argv: Sequence[str], env: dict[str, str]) -> list[str]:
ready_file = env.get("BOT_BOTTLE_GIT_GATE_READY_FILE", "").strip()
if name not in _READY_GATED_DAEMONS or not ready_file:
return list(argv)
return [
"/bin/sh",
"-c",
"while [ ! -f \"$BOT_BOTTLE_GIT_GATE_READY_FILE\" ]; do "
"sleep 0.1; "
"done; "
"exec \"$@\"",
name,
*argv,
]
def _selected_daemons(
env: dict[str, str],
all_daemons: Sequence[_DaemonSpec] | None = None,
@@ -118,12 +135,13 @@ def _pump(name: str, stream: IO[bytes]) -> None:
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
env = _env_for_daemon(spec.name, dict(os.environ))
proc = subprocess.Popen(
list(spec.argv),
_argv_for_daemon(spec.name, spec.argv, env),
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=0,
env=_env_for_daemon(spec.name, dict(os.environ)),
env=env,
)
threading.Thread(
target=_pump, args=(spec.name, proc.stdout), daemon=True
+34 -54
View File
@@ -2,11 +2,10 @@
The supervise plane is the per-bottle MCP sidecar plus its host-side
queue/audit support. The sidecar (bot_bottle.supervise_server)
sits on the bottle's internal network and exposes three MCP tools the
agent calls when it hits a stuck-recovery category:
sits on the bottle's internal network and exposes MCP tools the agent
calls when it needs an operator-reviewed egress change:
* egress-block agent proposes a new routes.yaml
* capability-block agent proposes a new agent Dockerfile
* egress-block / allow agent proposes a new routes.yaml
Each tool call: the agent passes the full proposed file plus a
justification text. The sidecar validates the proposal syntactically,
@@ -48,28 +47,35 @@ from pathlib import Path
SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100
TOOL_CAPABILITY_BLOCK = "capability-block"
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
# Written directly by the egress addon (not an agent-facing MCP tool) when an
# outbound DLP token block is routed to the operator for override (PRD 0062).
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
TOOLS: tuple[str, ...] = (
TOOL_CAPABILITY_BLOCK,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
# The supervise sidecar uses these to query egress's
# introspection endpoint for the `list-egress-routes` MCP
# tool. The hostname + port match egress's docker network
# alias + listen port (see bot_bottle.egress.EGRESS_HOSTNAME
# and backend.docker.egress.EGRESS_PORT — the values
# are inlined here so the in-container supervise_server doesn't
# need to import the egress package).
EGRESS_FORWARD_PROXY = "http://egress:9099"
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise
# daemon runs inside the sidecar bundle alongside egress, so loopback
# is the stable address across docker, smolmachines, and Apple
# Container backends.
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
# capability-block has no on-disk config the operator edits in place
# (the Dockerfile is rebuilt, not patched), so it has no audit log
# here — those changes are captured by git history + the rebuild
# record laid down in PRD 0016. egress-block was removed in issue #198.
COMPONENT_FOR_TOOL: dict[str, str] = {}
COMPONENT_FOR_TOOL: dict[str, str] = {
TOOL_EGRESS_ALLOW: "egress",
TOOL_EGRESS_BLOCK: "egress",
}
STATUS_APPROVED = "approved"
STATUS_MODIFIED = "modified"
@@ -81,8 +87,6 @@ STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
ACTION_OPERATOR_EDIT = "operator-edit"
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
CURRENT_CONFIG_DIR_IN_AGENT = "/etc/bot-bottle/current-config"
DEFAULT_POLL_INTERVAL_SEC = 0.5
@@ -425,64 +429,39 @@ def sha256_hex(content: str) -> str:
# --- Sidecar plan + abstract lifecycle -------------------------------------
# Filename of the staged Dockerfile inside the agent's read-only
# current-config mount. The capability-block tool's description
# points the agent at this exact path so it can read the current
# Dockerfile and propose modifications.
#
# routes.yaml + allowlist used to live here too; PRD 0017 chunk 3
# moved them behind the `list-egress-routes` MCP tool (live
# state from egress's introspection endpoint) so the agent
# always sees current data rather than a launch-time snapshot.
CURRENT_CONFIG_DOCKERFILE = "Dockerfile"
@dataclass(frozen=True)
class SupervisePlan:
"""Output of Supervise.prepare; consumed by .start.
`queue_dir` is the host directory bind-mounted into the sidecar
at /run/supervise/queue. `current_config_dir` is the host
directory bind-mounted (read-only) into the *agent* container
at /etc/bot-bottle/current-config currently holds only the
Dockerfile snapshot (routes.yaml + allowlist moved to the
`list-egress-routes` MCP tool). `internal_network` is
empty at prepare time; the backend's launch step fills it via
dataclasses.replace before calling .start."""
at /run/supervise/queue. `internal_network` is empty at prepare
time; the backend's launch step fills it via dataclasses.replace
before calling .start."""
slug: str
queue_dir: Path
current_config_dir: Path
internal_network: str = ""
class Supervise(ABC):
"""Per-bottle supervise sidecar. Encapsulates the host-side
prepare (queue dir + current-config staging); the sidecar's
start/stop lifecycle is backend-specific."""
prepare (queue dir staging); the sidecar's start/stop lifecycle
is backend-specific."""
def prepare(
self,
slug: str,
stage_dir: Path,
*,
dockerfile_content: str = "",
) -> SupervisePlan:
"""Stage the per-bottle queue dir on the host and the
current-config dir under `stage_dir`. Returns the plan;
`internal_network` must be set by the launch step before
"""Stage the per-bottle queue dir on the host. Returns the
plan; `internal_network` must be set by the launch step before
.start runs."""
del stage_dir
queue_dir = queue_dir_for_slug(slug)
queue_dir.mkdir(parents=True, exist_ok=True)
current_config_dir = stage_dir / "current-config"
current_config_dir.mkdir(parents=True, exist_ok=True)
dockerfile_path = current_config_dir / CURRENT_CONFIG_DOCKERFILE
dockerfile_path.write_text(dockerfile_content)
dockerfile_path.chmod(0o644)
return SupervisePlan(
slug=slug,
queue_dir=queue_dir,
current_config_dir=current_config_dir,
)
# --- Helpers ---------------------------------------------------------------
@@ -533,8 +512,6 @@ __all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"COMPONENT_FOR_TOOL",
"CURRENT_CONFIG_DIR_IN_AGENT",
"CURRENT_CONFIG_DOCKERFILE",
"DEFAULT_POLL_INTERVAL_SEC",
"Proposal",
"QUEUE_DIR_IN_CONTAINER",
@@ -550,7 +527,10 @@ __all__ = [
"TOOLS",
"EGRESS_FORWARD_PROXY",
"EGRESS_INTROSPECT_URL",
"TOOL_CAPABILITY_BLOCK",
"TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK",
"TOOL_GITLEAKS_ALLOW",
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"archive_proposal",
"audit_dir",
+148 -52
View File
@@ -1,8 +1,8 @@
"""Supervise sidecar HTTP server (PRD 0013).
Per-bottle MCP server exposing tools the agent calls to propose config
changes when stuck. The egress-block tool was removed in issue #198;
the remaining tools are `capability-block` and `list-egress-routes`.
Per-bottle MCP server exposing tools the agent calls to propose egress
config changes when stuck. The tools are `egress-allow`,
`egress-block`, and `list-egress-routes`.
Each queued tool call:
@@ -44,9 +44,15 @@ import urllib.request
from dataclasses import dataclass
from pathlib import Path
# Same-directory import inside the bundle container; `supervise.py`
# is COPYed alongside this file by Dockerfile.sidecars.
import supervise as _sv
try:
# Same-directory imports inside the bundle container; these files are
# COPYed flat under /app by Dockerfile.sidecars.
from egress_addon_core import LOG_OFF, load_config
import supervise as _sv
except ModuleNotFoundError:
# Package imports for host-side tests and tooling.
from .egress_addon_core import LOG_OFF, load_config
from . import supervise as _sv
# --- JSON-RPC / MCP plumbing ----------------------------------------------
@@ -84,19 +90,19 @@ def parse_jsonrpc(body: bytes) -> JsonRpcRequest:
try:
raw = json.loads(body)
except json.JSONDecodeError as e:
raise _RpcError(ERR_PARSE, f"parse error: {e}") from e
raise _RpcClientError(ERR_PARSE, f"parse error: {e}") from e
if not isinstance(raw, dict):
raise _RpcError(ERR_INVALID_REQUEST, "request must be a JSON object")
raise _RpcClientError(ERR_INVALID_REQUEST, "request must be a JSON object")
if raw.get("jsonrpc") != JSONRPC_VERSION:
raise _RpcError(ERR_INVALID_REQUEST, "jsonrpc field must be '2.0'")
raise _RpcClientError(ERR_INVALID_REQUEST, "jsonrpc field must be '2.0'")
method = raw.get("method")
if not isinstance(method, str):
raise _RpcError(ERR_INVALID_REQUEST, "method must be a string")
raise _RpcClientError(ERR_INVALID_REQUEST, "method must be a string")
params = raw.get("params", {})
if params is None:
params = {}
if not isinstance(params, dict):
raise _RpcError(ERR_INVALID_PARAMS, "params must be an object")
raise _RpcClientError(ERR_INVALID_PARAMS, "params must be an object")
rpc_id = raw.get("id", _NO_ID)
is_notification = rpc_id is _NO_ID
return JsonRpcRequest(
@@ -111,12 +117,23 @@ _NO_ID = object()
class _RpcError(Exception):
"""Base class for all typed RPC errors that surface as JSON-RPC error responses."""
def __init__(self, code: int, message: str):
super().__init__(message)
self.code = code
self.message = message
class _RpcClientError(_RpcError):
"""Caller sent a bad request; returned verbatim, no server-side logging."""
class _RpcInternalError(_RpcError):
"""Server-side fault; logged at ERROR with cause, always returns ERR_INTERNAL."""
def __init__(self, message: str) -> None:
super().__init__(ERR_INTERNAL, message)
def jsonrpc_result(request_id: object, result: object) -> bytes:
payload = {"jsonrpc": JSONRPC_VERSION, "id": request_id, "result": result}
return (json.dumps(payload) + "\n").encode("utf-8")
@@ -142,8 +159,9 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
"allowlist. Returns JSON with one entry per allowed host, "
"each carrying its matches rules (if any) and whether "
"the proxy injects Authorization for the route. Use this "
"before composing an `egress-block` proposal so the new "
"routes file extends the live one rather than replacing it."
"before composing an `egress-allow` or `egress-block` proposal so "
"the new routes file extends the live one rather than "
"replacing it."
),
"inputSchema": {
"type": "object",
@@ -152,41 +170,97 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
},
},
{
"name": _sv.TOOL_CAPABILITY_BLOCK,
"name": _sv.TOOL_EGRESS_ALLOW,
"description": (
"Call when the bottle is missing a tool, skill, permission, "
"or env var you need — something that lives in the agent "
"Dockerfile rather than in the egress routes. "
"Read the current Dockerfile from "
"/etc/bot-bottle/current-config/Dockerfile, compose a "
"modified version, and pass the full new file plus a "
"justification. On approval the supervisor rebuilds the "
"bottle from the new Dockerfile and starts a replacement on "
"the same branch (wired in PRD 0016; v1 acknowledges only)."
"Request operator approval to change the bottle's egress "
"allowlist. Pass the full proposed routes.yaml content, not "
"just the new host, plus a justification. Use "
"`list-egress-routes` first so the proposal preserves existing "
"routes."
),
"inputSchema": {
"type": "object",
"properties": {
"dockerfile": {
"routes_yaml": {
"type": "string",
"description": "Full proposed Dockerfile content.",
"description": (
"Full proposed /etc/egress/routes.yaml content. "
"Each route entry accepts these keys:\n"
" host: <hostname> (required)\n"
" auth_scheme: Bearer|token (must pair with token_env)\n"
" token_env: <ENV_VAR_NAME> (must pair with auth_scheme)\n"
" matches: (optional list of match entries)\n"
" - paths: [{type: prefix|exact|regex, value: /...}]\n"
" methods: [GET, POST, ...]\n"
" headers: [{name: X-Hdr, value: val, type: exact|regex}]\n"
" git: (optional; omit to block git clone/fetch)\n"
" fetch: true\n"
" dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format."
),
},
"justification": {
"type": "string",
"description": "Why this capability is needed.",
"description": "Why this egress route is needed.",
},
},
"required": ["dockerfile", "justification"],
"required": ["routes_yaml", "justification"],
},
},
{
"name": _sv.TOOL_EGRESS_BLOCK,
"description": (
"Request operator approval to change the bottle's egress "
"allowlist after a blocked outbound request. Pass the full "
"proposed routes.yaml content plus a justification. Use "
"`list-egress-routes` first so the proposal preserves existing "
"routes."
),
"inputSchema": {
"type": "object",
"properties": {
"routes_yaml": {
"type": "string",
"description": (
"Full proposed /etc/egress/routes.yaml content. "
"Each route entry accepts these keys:\n"
" host: <hostname> (required)\n"
" auth_scheme: Bearer|token (must pair with token_env)\n"
" token_env: <ENV_VAR_NAME> (must pair with auth_scheme)\n"
" matches: (optional list of match entries)\n"
" - paths: [{type: prefix|exact|regex, value: /...}]\n"
" methods: [GET, POST, ...]\n"
" headers: [{name: X-Hdr, value: val, type: exact|regex}]\n"
" git: (optional; omit to block git clone/fetch)\n"
" fetch: true\n"
" dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format."
),
},
"justification": {
"type": "string",
"description": "Why this egress route is needed.",
},
},
"required": ["routes_yaml", "justification"],
},
},
]
# Map each non-egress tool to the input field that carries the agent's
# payload (stored in Proposal.proposed_file). egress-block builds its
# payload from structured input fields in `handle_egress_block`.
# Map each proposal tool to the input field that carries the agent's
# payload (stored in Proposal.proposed_file).
PROPOSED_FILE_FIELD: dict[str, str] = {
_sv.TOOL_CAPABILITY_BLOCK: "dockerfile",
_sv.TOOL_EGRESS_ALLOW: "routes_yaml",
_sv.TOOL_EGRESS_BLOCK: "routes_yaml",
}
@@ -198,13 +272,22 @@ def validate_proposed_file(tool: str, content: str) -> None:
catches obvious paste-errors / wrong-tool selections before they
enter the queue."""
if not content.strip():
raise _RpcError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
if tool == _sv.TOOL_CAPABILITY_BLOCK:
# Dockerfiles are too varied to validate syntactically beyond
# non-empty. The operator reads the diff in the TUI.
pass
raise _RpcClientError(ERR_INVALID_PARAMS, f"{tool}: proposed file is empty")
if tool in (_sv.TOOL_EGRESS_ALLOW, _sv.TOOL_EGRESS_BLOCK):
try:
config = load_config(content)
except ValueError as e:
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{tool}: proposed routes.yaml is not valid: {e}",
) from e
if config.log != LOG_OFF:
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{tool}: proposed routes.yaml must not change egress logging",
)
else:
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
raise _RpcClientError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
# --- MCP handlers ----------------------------------------------------------
@@ -277,17 +360,17 @@ def handle_tools_call(
doesn't need operator approval."""
name = params.get("name")
if not isinstance(name, str):
raise _RpcError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call missing 'name'")
if name == _sv.TOOL_LIST_EGRESS_ROUTES:
return handle_list_egress_routes(typing.cast(dict[str, object], params.get("arguments", {})), config)
args_raw = params.get("arguments", {})
if not isinstance(args_raw, dict):
raise _RpcError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
raise _RpcClientError(ERR_INVALID_PARAMS, "tools/call 'arguments' must be an object")
justification = args_raw.get("justification")
if not isinstance(justification, str) or not justification.strip():
raise _RpcError(
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{name}: 'justification' is required and must be a non-empty string",
)
@@ -296,13 +379,13 @@ def handle_tools_call(
file_field = PROPOSED_FILE_FIELD[name]
proposed_file = args_raw.get(file_field)
if not isinstance(proposed_file, str):
raise _RpcError(
raise _RpcClientError(
ERR_INVALID_PARAMS,
f"{name}: '{file_field}' is required and must be a string",
)
validate_proposed_file(name, proposed_file)
else:
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {name!r}")
raise _RpcClientError(ERR_INVALID_PARAMS, f"unknown tool {name!r}")
proposal = _sv.Proposal.new(
bottle_slug=config.bottle_slug,
@@ -311,7 +394,10 @@ def handle_tools_call(
justification=justification,
current_file_hash=_sv.sha256_hex(proposed_file),
)
_sv.write_proposal(config.queue_dir, proposal)
try:
_sv.write_proposal(config.queue_dir, proposal)
except OSError as e:
raise _RpcInternalError(f"failed to write proposal to queue: {e}") from e
sys.stderr.write(
f"supervise: queued proposal {proposal.id} ({name}) "
f"for bottle {config.bottle_slug}; waiting for operator...\n"
@@ -331,7 +417,10 @@ def handle_tools_call(
"content": [{"type": "text", "text": text}],
"isError": False,
}
_sv.archive_proposal(config.queue_dir, proposal.id)
try:
_sv.archive_proposal(config.queue_dir, proposal.id)
except OSError as e:
raise _RpcInternalError(f"failed to archive proposal: {e}") from e
text = format_response_text(response)
return {
@@ -365,9 +454,8 @@ def format_pending_response_text(timeout_seconds: float) -> str:
# --- HTTP transport --------------------------------------------------------
# Max request body the server accepts. Generous because Dockerfile
# proposals can be a few KB; routes.json is small. 1 MB is well above
# any realistic config file.
# Max request body the server accepts. 1 MB is well above any realistic
# routes.yaml proposal.
MAX_BODY_BYTES = 1 * 1024 * 1024
@@ -407,7 +495,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
req = parse_jsonrpc(body)
except _RpcError as e:
except _RpcClientError as e:
self._write_jsonrpc(jsonrpc_error(None, e.code, e.message))
return
@@ -415,11 +503,19 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
try:
result = self._dispatch(req, config)
except _RpcError as e:
except _RpcClientError as e:
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
return
except Exception as e: # noqa: W0718 — catch-all for RPC dispatch errors
sys.stderr.write(f"supervise: internal error: {e}\n")
except _RpcInternalError as e:
cause = e.__cause__
detail = f": {cause}" if cause else ""
sys.stderr.write(f"supervise: internal error: {e.message}{detail}\n")
sys.stderr.flush()
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
return
except Exception as e: # noqa: W0718 — unexpected errors
sys.stderr.write(f"supervise: unexpected error: {type(e).__name__}: {e}\n")
sys.stderr.flush()
self._write_jsonrpc(jsonrpc_error(req.id, ERR_INTERNAL, "internal error"))
return
@@ -438,7 +534,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
return handle_tools_list(req.params)
if method == "tools/call":
return handle_tools_call(req.params, config)
raise _RpcError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def _write_jsonrpc(self, body: bytes) -> None:
self.send_response(200)
+30 -36
View File
@@ -69,12 +69,6 @@ class YamlSubsetError(ValueError):
egress sidecar's addon) handle it as a normal exception."""
def die(msg: str) -> None:
"""Module-local helper so the parser body reads cleanly. Just
raises YamlSubsetError the `bot-bottle: error: ` prefix
is added by the boundary `die` in `bot_bottle.log`."""
raise YamlSubsetError(msg)
# --- Tokenizer / line preprocessing ----------------------------------------
@@ -119,7 +113,7 @@ def _tokenize(text: str) -> list[_Line]:
# editors render them differently and the spec says spaces.
leading = len(raw) - len(raw.lstrip(" \t"))
if "\t" in raw[:leading]:
die(f"yaml-subset: tab character in indent on line {n}")
raise YamlSubsetError(f"yaml-subset: tab character in indent on line {n}")
stripped = raw.strip()
if not stripped:
continue
@@ -169,14 +163,14 @@ def _parse_scalar(s: str, lineno: int) -> object:
s.startswith("'") and s.endswith("'")
):
if len(s) < 2:
die(f"yaml-subset: unterminated quoted string on line {lineno}")
raise YamlSubsetError(f"yaml-subset: unterminated quoted string on line {lineno}")
body = s[1:-1]
if s.startswith('"'):
# JSON-style escapes for double quotes.
try:
return body.encode("utf-8").decode("unicode_escape")
except UnicodeDecodeError as e:
die(f"yaml-subset: bad escape on line {lineno}: {e}")
raise YamlSubsetError(f"yaml-subset: bad escape on line {lineno}: {e}")
else:
# Single quotes: only '' → ' (standard YAML); no other escapes.
return body.replace("''", "'")
@@ -186,7 +180,7 @@ def _parse_scalar(s: str, lineno: int) -> object:
if s in _RESERVED_BOOL_LIKE:
if s in ("true", "false"):
return s == "true"
die(
raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} is ambiguous "
f"(use literal `true` / `false`, or quote it as a string)"
)
@@ -203,22 +197,22 @@ def _parse_scalar(s: str, lineno: int) -> object:
# Look-alikes that we reject to keep the user in control.
if _DATE_RX.match(s):
die(
raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} looks like a "
f"date — quote it as a string or use an explicit int"
)
if _OCTAL_RX.match(s):
die(
raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} looks like an "
f"octal/0-prefixed integer — quote it as a string"
)
if _HEX_RX.match(s):
die(
raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} looks like a "
f"hex integer — quote it as a string"
)
if _FLOAT_RX.match(s):
die(
raise YamlSubsetError(
f"yaml-subset: floats not supported (line {lineno}, "
f"value {s!r}); use an int or quote as a string"
)
@@ -241,7 +235,7 @@ def _parse_inline(s: str, lineno: int) -> object:
s = s.strip()
if s.startswith("["):
if not s.endswith("]"):
die(f"yaml-subset: unterminated `[` on line {lineno}")
raise YamlSubsetError(f"yaml-subset: unterminated `[` on line {lineno}")
body = s[1:-1].strip()
if not body:
return []
@@ -252,21 +246,21 @@ def _parse_inline(s: str, lineno: int) -> object:
return items
if s.startswith("{"):
if not s.endswith("}"):
die(f"yaml-subset: unterminated `{{` on line {lineno}")
raise YamlSubsetError(f"yaml-subset: unterminated `{{` on line {lineno}")
body = s[1:-1].strip()
if not body:
return {}
out: dict[str, object] = {}
for raw in _split_flow(body, lineno, "dict"):
if ":" not in raw:
die(
raise YamlSubsetError(
f"yaml-subset: inline dict entry on line {lineno} "
f"missing `:` ({raw!r})"
)
k, _, v = raw.partition(":")
k = k.strip()
if not _BARE_RX.match(k):
die(
raise YamlSubsetError(
f"yaml-subset: inline dict key on line {lineno} "
f"must be a bare identifier ({k!r})"
)
@@ -296,7 +290,7 @@ def _split_flow(body: str, lineno: int, kind: str) -> list[str]:
elif ch in "]}":
depth_b -= 1
if depth_b > 0:
die(
raise YamlSubsetError(
f"yaml-subset: nested flow {kind} on line "
f"{lineno} (only one level of flow allowed)"
)
@@ -330,7 +324,7 @@ def _split_key_value(content: str, lineno: int) -> tuple[str, str]:
# ambiguous with URLs etc.).
if i + 1 >= len(content) or content[i + 1] in (" ", "\t"):
return content[:i].strip(), content[i + 1:].lstrip()
die(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
raise YamlSubsetError(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
return "", "" # unreachable, but needed for type checker
@@ -341,15 +335,15 @@ def _parse_block(
to live at `base_indent`. Returns (value, new_idx) where
`new_idx` is the index of the first unconsumed line."""
if idx >= len(lines):
die("yaml-subset: unexpected end of document")
raise YamlSubsetError("yaml-subset: unexpected end of document")
first = lines[idx]
if first.indent < base_indent:
die(
raise YamlSubsetError(
f"yaml-subset: line {first.lineno} indented less than "
f"expected (got {first.indent}, expected >= {base_indent})"
)
if first.indent > base_indent:
die(
raise YamlSubsetError(
f"yaml-subset: line {first.lineno} indented more than "
f"expected (got {first.indent}, expected {base_indent})"
)
@@ -366,18 +360,18 @@ def _parse_block_mapping(
while idx < len(lines) and lines[idx].indent == base_indent:
line = lines[idx]
if line.content.startswith("- "):
die(
raise YamlSubsetError(
f"yaml-subset: line {line.lineno} unexpected list "
f"item at mapping indent (got `-`, expected `key:`)"
)
key, value_text = _split_key_value(line.content, line.lineno)
if not _BARE_RX.match(key):
die(
raise YamlSubsetError(
f"yaml-subset: line {line.lineno} key {key!r} is not "
f"a bare identifier"
)
if key in out:
die(
raise YamlSubsetError(
f"yaml-subset: line {line.lineno} duplicate key {key!r}"
)
if value_text:
@@ -417,7 +411,7 @@ def _parse_block_list(
content_col = base_indent + 2
first_key, first_value_text = _split_key_value(rest, line.lineno)
if not _BARE_RX.match(first_key):
die(
raise YamlSubsetError(
f"yaml-subset: line {line.lineno} key {first_key!r} "
f"is not a bare identifier"
)
@@ -440,12 +434,12 @@ def _parse_block_list(
break # next list item, not a sibling key
k, v_text = _split_key_value(ln.content, ln.lineno)
if not _BARE_RX.match(k):
die(
raise YamlSubsetError(
f"yaml-subset: line {ln.lineno} key {k!r} is "
f"not a bare identifier"
)
if k in item:
die(f"yaml-subset: line {ln.lineno} duplicate key {k!r}")
raise YamlSubsetError(f"yaml-subset: line {ln.lineno} duplicate key {k!r}")
if v_text:
item[k] = _parse_inline(v_text, ln.lineno)
idx += 1
@@ -501,7 +495,7 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
for n, raw in enumerate(text.splitlines(), start=1):
s = raw.strip()
if s.startswith("|") or s.startswith(">") or s.startswith("- |") or s.startswith("- >"):
die(
raise YamlSubsetError(
f"yaml-subset: line {n} uses a multi-line block "
f"scalar (`|` / `>`) — not supported. Use a quoted "
f"single-line string instead."
@@ -511,12 +505,12 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
# not when it's inside a quoted string. Cheap check: any
# bare `&foo:` / `*foo` at the start of a value position.
if re.search(r"(^|\s)[&*][A-Za-z0-9_]+", s):
die(
raise YamlSubsetError(
f"yaml-subset: line {n} uses anchors / aliases "
f"(`&` / `*`) — not supported."
)
if "!!" in s and not (s.count("'") % 2 or s.count('"') % 2):
die(
raise YamlSubsetError(
f"yaml-subset: line {n} uses a YAML tag (`!!`) — not "
f"supported."
)
@@ -526,18 +520,18 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
return {}
base_indent = lines[0].indent
if base_indent != 0:
die(
raise YamlSubsetError(
f"yaml-subset: top-level content must start in column 0 "
f"(got column {base_indent} on line {lines[0].lineno})"
)
value, consumed = _parse_block(lines, 0, 0)
if consumed < len(lines):
die(
raise YamlSubsetError(
f"yaml-subset: trailing content starting on line "
f"{lines[consumed].lineno}"
)
if not isinstance(value, dict):
die("yaml-subset: top-level value must be a mapping")
raise YamlSubsetError("yaml-subset: top-level value must be a mapping")
return cast(dict[str, object], value)
@@ -576,7 +570,7 @@ def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
fm_end_lineno = line_idx
break
if body_start < 0:
die("frontmatter: opening `---` has no matching closing `---`")
raise YamlSubsetError("frontmatter: opening `---` has no matching closing `---`")
fm_text = text[line_starts[1]:line_starts[fm_end_lineno]] if fm_end_lineno > 1 else ""
fm = parse_yaml_subset(fm_text)
+4 -4
View File
@@ -13,13 +13,13 @@ Add Content-Length validation and a body-size cap to `git_http_backend.py` so ma
`bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler.
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use. For comparison, `supervise_server.py` caps request bodies at 1 MiB.
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use.
## Goals / Success Criteria
- A missing or non-numeric Content-Length returns HTTP 400.
- A negative Content-Length returns HTTP 400.
- A body larger than the cap (1 MiB, matching `supervise_server.py`) returns HTTP 413.
- A body larger than the cap (100 MiB) returns HTTP 413.
- Valid Git smart-HTTP pushes and fetches continue to work.
- Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough.
@@ -43,12 +43,12 @@ Out of scope:
## Design
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 1 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 100 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
## Testing Strategy
- Unit tests using `unittest.mock` to drive the handler with crafted headers.
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, `Content-Length: 2097152` (over cap), and a normal small POST body.
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, a declared length above `MAX_BODY_BYTES`, and a normal small POST body.
Run:
+21
View File
@@ -199,6 +199,25 @@ Named inbound detectors: `naive_injection_detection`.
The manifest parser (`manifest_egress.py`) validates the `dlp` block and
rejects unknown detector names.
### Manifest schema — `git` block
HTTPS Git clone/fetch traffic is not implied by a host-level egress route.
Smart HTTP Git fetch uses `git-upload-pack`, which can transfer large repo
packfiles and bypass the git-gate mirror path. It is therefore blocked by
default and must be explicitly enabled per route:
```yaml
egress:
routes:
- host: github.com
git:
fetch: true
```
`git.fetch: true` permits read-only smart HTTP clone/fetch requests
(`git-upload-pack`) after the normal host and `matches` checks pass. HTTPS
Git push (`git-receive-pack`) remains blocked by the egress addon.
### `EgressRoute` changes
`EgressRoute` replaces `PathAllowlist` with `Matches` and gains two new
@@ -232,6 +251,7 @@ class EgressRoute:
AuthScheme: str = ""
TokenRef: str = ""
Role: tuple[str, ...] = ()
GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None # None = all enabled
InboundDetectors: tuple[str, ...] | None = None # None = all enabled
```
@@ -252,6 +272,7 @@ class Route:
matches: tuple[MatchEntry, ...] = ()
auth_scheme: str = ""
token_env: str = ""
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None
```
+79
View File
@@ -0,0 +1,79 @@
# PRD 0057: Promote smolmachines to default backend; convert Docker to example-only
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-06
- **Issue:** #206
## Summary
Make smolmachines the default bot-bottle backend and demote Docker to an example-only configuration. This closes the DNS sinkhole gap that exists in the Docker backend: the mitmproxy egress addon intercepts HTTP(S) but cannot see raw UDP port-53 DNS queries, so an agent can exfiltrate data via DNS tunnelling without the egress guard seeing it. The smolmachines backend eliminates this gap at the VMM layer — DNS filtering is built in and the agent container cannot bypass it.
## Problem
The current default backend is Docker. The egress addon (PRDs 0052/0053) intercepts HTTPS and scans request/response surfaces, but it is an HTTP proxy: raw UDP/TCP port-53 DNS queries go to the OS resolver and never pass through it. An agent can encode secrets as base32 or hex subdomains in a DNS query (`<encoded>.attacker.com`) and exfiltrate them silently.
The smolmachines backend already solves this: its Transport Socket Interface (TSI) enforces a CIDR allowlist at the VMM layer, and DNS is handled via vsock port 6002 — the guest's `/etc/resolv.conf` points at `127.0.0.1`, and a guest-side DNS proxy tunnels queries over vsock to the host, which returns NXDOMAIN for anything not on the allowlist. The agent cannot bypass this by hardcoding IPs or by configuring an alternate resolver, because both mechanisms are enforced below the guest OS.
Docker has no equivalent. Adding dnsmasq to the Docker backend would close the gap at some cost (dnsmasq sidecar, iptables `NET_ADMIN`, per-launch config generation), but it is the wrong direction if smolmachines supersedes Docker anyway.
## Goals / Success Criteria
- `BOT_BOTTLE_BACKEND` defaults to `smolmachines` when not set.
- The existing Docker backend remains functional (not removed) but is no longer the default and is documented as legacy/example-only.
- Example bottles (`examples/bottles/`) reference smolmachines, not Docker.
- `AGENTS.md` documents the backend choice and the DNS gap closure.
- Existing Docker-backed integration tests continue to pass; they select Docker explicitly via `BOT_BOTTLE_BACKEND=docker` rather than relying on the default.
## Non-goals
- Removing the Docker backend or its tests.
- Implementing a dnsmasq layer for the Docker backend (closed by this change; not needed on the default path).
- Iptables / `NET_ADMIN` work for Docker (deferred).
- Subdomain-depth filtering for allowlisted zones (documented residual gap; tracked separately per the issue).
## Design
### Default backend change
`bot_bottle/backend/__init__.py`, line ~440:
```python
# Before
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
# After
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "smolmachines"
```
### DNS gap closure (how smolmachines handles it)
When the smolmachines backend launches an agent VM:
1. The VM's network device uses TSI (`--allow-host` / `--allow-cidr` flags), which enforces a CIDR allowlist at the VMM layer. The guest cannot dial IPs outside the allowlist even with raw sockets.
2. The guest's `/etc/resolv.conf` is set to `127.0.0.1`; a guest-side DNS proxy relays queries over vsock port 6002 to the host.
3. The host-side DNS filter returns NXDOMAIN for any hostname not in the allowlist derived from `egress.routes` in the bottle manifest.
This means DNS exfiltration via unknown subdomains is blocked by NXDOMAIN before the query leaves the host, and even if the agent hardcoded the IP of an attacker-controlled server, TSI would drop the packet at the VMM layer.
**Residual gap:** if the attacker controls a subdomain of an allowlisted zone (e.g., a legitimate zone like `api.anthropic.com` that the attacker can inject into via a separate compromise), DNS queries for that subdomain would be forwarded. This is accepted and documented.
### Example bottles
Update `examples/bottles/dev.md` and `examples/bottles/claude.md` to remove Docker-specific notes and reference smolmachines as the runtime.
### Integration test migration
Tests that exercise the Docker backend explicitly should set `BOT_BOTTLE_BACKEND=docker` rather than relying on the default. Tests that are backend-agnostic continue to use whatever `BOT_BOTTLE_BACKEND` is set to (defaulting to smolmachines in the test environment if available).
## Resolved questions
- **TSI + egress proxy loopback.** The implementation uses a per-bottle loopback alias rather than broad `127.0.0.1` passthrough. The smolmachines launch integration test now asserts that the guest receives proxy env vars on a `127.x` alias, can reach an allowlisted host through the proxy, cannot reach the same host directly with proxy vars unset, and cannot reach a non-allowlisted host through the proxy.
- **smolmachines availability check.** The smolmachines preflight error points operators at the smolvm installer and explicitly suggests `BOT_BOTTLE_BACKEND=docker` / `--backend=docker` for legacy Docker-backed runs.
## References
- `docs/research/smolmachines-as-vm-backend.md` — smolmachines evaluation
- `docs/research/network-egress-guard.md` — Approach 4 (DNS-based egress control)
- `docs/research/secret-exfil-tripwire-encodings.md` — DNS exfil discussion
- PRD 0052, PRD 0053 — egress DLP addon (HTTP-level; partial mitigation only)
+123
View File
@@ -0,0 +1,123 @@
# PRD 0058: Add built-in Pi agent provider
- **Status:** Active
- **Author:** codex
- **Created:** 2026-06-09
- **Issue:** #221
## Summary
Add `pi` as a built-in `agent_provider.template`. The provider runs the Pi
coding-agent CLI, provisions its agent config under `~/.pi/agent`, and writes a
provider settings file that targets an unauthenticated Ollama-compatible server.
The default settings assume an Ollama server at `http://ollama:11434/v1`, using
the `openai-completions` API with a dummy API key because Ollama ignores it.
Users can override the provider id, base URL, model list, API key, API-key env
reference, API type, and compatibility flags through a new
`agent_provider.settings` object.
## Problem
bot-bottle currently ships Claude and Codex as built-in agent providers. Pi is a
useful third harness, but using it today requires a custom provider plugin and a
custom image. That repeats boilerplate for prompt copying, skill copying,
provider config, and runtime registration.
Pi's local-model path is also easy to misconfigure: its custom-model docs require
`~/.pi/agent/models.json`, an API entry, at least one model id, and a dummy
`apiKey` for Ollama even though the server does not authenticate. bot-bottle
should generate that shape consistently.
## Goals / Success Criteria
- `agent_provider.template: pi` is accepted as a built-in provider.
- `bot_bottle/contrib/pi/` provides a Pi image and `PiAgentProvider`.
- Pi receives the bot-bottle prompt at `~/.bot-bottle-prompt.txt` and starts in
print-mode prompt delivery like Codex.
- Pi skills are copied into `~/.pi/agent/skills/<name>/`.
- Pi provider settings are configurable from the bottle manifest via
`agent_provider.settings`.
- The default Pi provider settings configure an unauthenticated Ollama-compatible
server.
- Unit tests cover manifest parsing, runtime selection, plan generation, prompt,
skills, and provider provisioning.
## Non-goals
- Managing or launching an Ollama server.
- Authenticating to Ollama or any remote Pi provider.
- Forwarding host Pi credentials.
- Implementing Pi extensions or MCP registration.
- Changing Claude or Codex provider behavior.
## Design
### Manifest
Extend `agent_provider` with an optional `settings` object. It is currently only
supported for built-in `pi`.
Supported keys:
- `base_url`: string, defaults to `http://ollama:11434/v1`
- `provider`: string, defaults to `ollama`
- `api`: string, defaults to `openai-completions`
- `api_key`: string, defaults to `ollama`
- `api_key_env`: string, optional host env var name for egress auth injection
- `models`: non-empty array of strings, defaults to `["qwen2.5-coder:7b"]`
- `context_window`: positive integer, defaults to `4096`; this is the Ollama
runtime context, and bot-bottle subtracts `max_tokens` before writing Pi's
`contextWindow` so output space is reserved
- `max_tokens`: positive integer, defaults to `1024`
- `max_tokens_field`: `max_tokens` or `max_completion_tokens`, defaults to
`max_tokens`
- `supports_developer_role`: boolean, defaults to `false`
- `supports_reasoning_effort`: boolean, defaults to `false`
The snake-case manifest keys are converted into Pi's JSON field names:
`baseUrl`, `apiKey`, `contextWindow`, `maxTokens`,
`supportsDeveloperRole`, and `supportsReasoningEffort`. `context_window`
describes the server's total context; Pi's `contextWindow` receives
`context_window - max_tokens` because Pi uses it as an input compaction target.
`api_key` and `api_key_env` are mutually exclusive. When targeting a hosted
provider through bot-bottle's egress sidecar, omit `api_key` and set
`api_key_env` to the host env var that holds the API key. The generated
`models.json` receives only an `egress-placeholder` API key, and the egress
route injects the real `Authorization` header from the sidecar env. For example,
OpenRouter can use provider id `openrouter` with
`api_key_env: OPENROUTER_API_KEY`, keeping the key out of the agent env and
`models.json`.
### Provider
`PiAgentProvider.provision_plan` writes `models.json` into the per-launch state
directory and returns an `AgentProvisionPlan` that copies it to
`~/.pi/agent/models.json`. The provider also declares an unauthenticated egress
route for the configured base URL host so the egress layer can allow the Ollama
endpoint.
The Pi runtime uses:
- `command="pi"`
- `prompt_mode="append_system_prompt"`
- `image="bot-bottle-pi:latest"`
- `bypass_args=()`
- `resume_args=()`
- `remote_control_args=()`
The Dockerfile installs `@earendil-works/pi-coding-agent` globally from npm and
keeps the same Debian/node base shape as the existing provider images.
### Supervise MCP
Pi does not have built-in MCP support in the current public docs, so
`provision_supervise_mcp` is a no-op. This keeps Pi bottles launchable with
`supervise: true` while preserving the explicit non-goal of implementing Pi
extensions.
## Merge rule(s)
This PR can merge when the focused unit tests pass and the PRD status is flipped
from Draft to Active in the final implementation commit.
+190
View File
@@ -0,0 +1,190 @@
# PRD 0059: macOS Container backend
- **Status:** Active
- **Author:** Codex
- **Created:** 2026-06-10
- **Issue:** #220
## Summary
Add a `macos-container` backend that integrates Apple's `container`
CLI as a host runtime on macOS. The shipped slices register the
backend, implement reusable host primitives (`build`, `exec`, `cp`,
image inspection, cleanup, active enumeration), make launch runnable
with the proven two-network sidecar topology, and add real-runtime
coverage without weakening bot-bottle's sidecar egress model.
## Problem
bot-bottle currently has two local execution paths:
- `docker`, which runs the whole bottle topology through Docker
Compose.
- `smolmachines`, which runs the agent in smolvm but still depends on
Docker for the sidecar bundle and image-building pipeline.
Issue #220 explored removing Docker as a host dependency. A follow-up
review comment verified that smolvm can publish guest ports back to
host loopback and that another smolvm guest can reach that service
through the existing per-bottle loopback alias plus `--allow-cidr`
path. That keeps the VM-contained sidecar direction viable and rejects
the host-process sidecar fallback.
Apple's `container` CLI is another macOS-native way to run OCI images
as lightweight Linux VMs. Its current command surface includes
Docker-like `build`, `run`, `exec`, `cp`, port publishing, image
inspection, and user-defined networks. That makes it a plausible local
backend, but it does not remove the need to preserve bot-bottle's
sidecar enforcement property: the agent must not have a direct egress
path around the egress sidecar.
## Goals / Success Criteria
- `--backend=macos-container` and
`BOT_BOTTLE_BACKEND=macos-container` are accepted by the existing
backend selector.
- Compatible macOS hosts default to `macos-container` when
`BOT_BOTTLE_BACKEND` and `--backend` are both unset.
- Backend availability is true only on macOS hosts with `container` on
`PATH`.
- The backend has tested wrappers for Apple Container image build,
image inspection, container `exec`, container `cp`, cleanup, and
active-agent enumeration.
- Full launch uses a host-only internal network for the agent and a
separate NAT egress network for the sidecar bundle.
- The agent container does not attach to the egress network. It reaches
allowed outbound hosts through HTTP(S)_PROXY pointing at the
sidecar's internal-network IP.
- `bottle.git` / git-gate bottles fail loudly on this backend until a
safe Apple Container key-delivery path exists.
- Real-runtime integration coverage is present and guarded by macOS and
Apple Container availability.
## Non-goals
- Do not remove or deprecate the Docker backend.
- Do not remove or deprecate the smolmachines backend.
- Do not run sidecar daemons as host processes.
- Do not launch a degraded backend where the agent can bypass the
egress sidecar through direct network access.
- Do not require Docker Desktop as part of the macOS Container backend.
## Design
### Backend name
The selectable backend name is `macos-container`. The Python package
uses `bot_bottle.backend.macos_container` because module names cannot
contain hyphens.
### Availability and preflight
`MacosContainerBottleBackend.is_available()` returns true only when:
- `platform.system() == "Darwin"`
- `container` is discoverable on `PATH`
`prepare()` calls `require_container()`, which produces a concrete
install pointer and rejects non-macOS hosts.
### Implemented primitives
The backend owns an Apple Container wrapper module instead of reusing
Docker wrappers. The wrapper maps bot-bottle's backend needs to
Apple's CLI:
| bot-bottle need | Apple Container command |
|---|---|
| Build provider image | `container build -t <ref> [-f Dockerfile] <context>` |
| Run agent commands | `container exec [--interactive --tty] <id> ...` |
| Copy files into guest | `container cp <host> <id>:<path>` |
| Inspect image identity | `container image inspect <ref>` |
| Cleanup stale containers | `container delete --force <id>` |
| Cleanup stale networks | `container network delete <name>` |
| Active enumeration | `container list --quiet` |
The bottle handle mirrors `DockerBottle`: it builds a host argv for
foreground agent execution, pipes shell snippets through stdin for
`Bottle.exec`, and exposes `cp_in` for provisioning.
### Launch topology
`launch()` uses Apple Container's two-network topology:
- create a host-only internal network for the bottle;
- create a normal NAT egress network for the sidecar bundle;
- start the sidecar bundle attached to the egress network first and the
internal network second;
- discover the sidecar's internal-network IPv4 address from
`container inspect`;
- start the agent attached only to the internal network, with
HTTP_PROXY / HTTPS_PROXY / lowercase proxy vars pointing at the
sidecar IP and egress port.
This keeps the agent off the outbound network while preserving the
proxy-env contract that existing agent tooling already honors. The
integration smoke also removes the proxy env in-guest and confirms
direct egress fails.
### Deferred git-gate support
Apple Container currently rejects single-file bind mounts, and
`container cp` into a stopped container is not available. Starting the
container earlier would allow `container cp` into a running container,
but it would also mean delivering SSH private key material into a live
sidecar before the git-gate daemon is ready to own it. Mounting broad
host SSH directories is not acceptable.
For this PRD, `bottle.git` / git-gate support is explicitly deferred on
the `macos-container` backend. Bottles with git-gate upstreams fail
loudly and should use `docker` or `smolmachines` until a narrower key
delivery design lands.
## Implementation chunks
1. Register `macos-container`, add availability/preflight, bottle
handle, utility wrappers, cleanup, active enumeration, unit tests,
and this PRD.
2. Spike Apple Container networking against real macOS 26 hosts:
repeated `--network`, internal network egress behavior, published
loopback reachability from another container, DNS behavior, and
labels/JSON output stability.
3. Implement launch once the enforcement shape is proven. Reuse the
existing sidecar bundle image and daemon subset env contract where
possible.
4. Add real-runtime integration tests guarded by `container` presence
and macOS version.
5. Consider moving smolmachines sidecar/image-building work to
VM-contained or Apple Container-backed execution only after the
`macos-container` launch path is trustworthy.
## Testing Strategy
- Unit tests cover backend registration through `known_backend_names`.
- Unit tests cover availability/preflight behavior without requiring
macOS.
- Unit tests cover `MacosContainerBottle` command construction and
stdin-based shell execution.
- Unit tests cover cleanup and active enumeration parsing.
- Unit tests cover launch argv/env construction, sidecar mount
staging, sidecar IP parsing, and git-gate rejection.
- Integration tests run on macOS hosts with Apple Container installed
and verify that egress cannot bypass the sidecar. They also preflight
Apple Container BuildKit DNS because image builds must resolve
package mirrors before a launch smoke can be meaningful. The backend
probes the running builder before image builds and leaves it alone
when its current resolver works. If the probe fails, or if the
operator explicitly sets `BOT_BOTTLE_MACOS_CONTAINER_DNS`, the backend
restarts the Apple Container builder with the configured DNS server.
Without an explicit override, that server is discovered from the
host's directly reachable IPv4 resolver before falling back to a
public resolver.
## References
- [Issue #220 review comment](https://gitea.dideric.is/didericis/bot-bottle/issues/220#issuecomment-1980):
smolvm `--port/-p` can expose a guest service to host loopback, and
another smolvm guest can reach it through the existing per-bottle
loopback alias path.
- Apple Container command reference: `container run`, `build`, `exec`,
port publishing, and network commands.
+159
View File
@@ -0,0 +1,159 @@
# PRD 0060: Commit bottle state to an image
- **Status:** Active
- **Author:** Claude
- **Created:** 2026-06-20
- **Issue:** #194
## Summary
Add a `commit` CLI command that freezes a running bottle's state to a
resumable local artifact. Docker bottles are stored as Docker images;
smolmachines bottles are stored as `.smolmachine` artifacts. Operators
can then resume the bottle from that exact filesystem snapshot, or
export the artifact to migrate work to a different host.
## Problem
When a long-running agent session is interrupted — by a host reboot, a
network failure, or a planned infrastructure migration — the in-progress
container state is lost. `cli.py resume` rebuilds the agent image from
the Dockerfile and reprovi-sions the bottle, but that returns the guest
to its initial state, not to wherever the agent was mid-task.
There is no mechanism today to capture "what's installed / configured
inside the running container right now" and make it reproducible. The
`capability-block` flow writes a new Dockerfile and marks the bottle for
resume, but that only applies when the agent itself has requested a
capability change; it doesn't help the operator who wants to take a
snapshot before a planned host reboot or hardware migration.
## Goals / Success Criteria
- `./cli.py commit [<slug>]` takes a snapshot of the running agent and
stores it as a local artifact.
- Without a slug argument the command shows the same interactive picker
as `start` (the list of active slugs).
- The committed artifact reference is stored in per-bottle state so
that the next `./cli.py resume <slug>` automatically uses the
snapshot instead of rebuilding from the Dockerfile.
- `mark_preserved` is called so the state dir survives the normal
session-end cleanup.
- A backend-specific export hint is printed so operators know how to
migrate the snapshot.
- The command errors clearly on unsupported backends.
## Non-goals
- macOS-container backend support.
- Automatic commit on agent exit.
- Image push to a remote registry.
- Storing the image tag in the manifest or sharing it between operators.
## Design
### Docker image tag
`bot-bottle-committed-<slug>:latest` — namespaced under `bot-bottle-`
to match existing image naming conventions; `committed` distinguishes it
from the build-time image (`bot-bottle-claude:latest`) and the
capability-block rebuild image (`bot-bottle-rebuilt-<identity>:latest`).
### State storage
A new plain-text file `committed-image` is added to the per-bottle state
directory:
```
~/.bot-bottle/state/<identity>/
metadata.json
Dockerfile (capability-block override; optional)
committed-image (committed artifact reference; optional)
transcript/
```
`bottle_state.committed_image_path(identity)` returns the path.
`write_committed_image` / `read_committed_image` are the read/write
helpers, matching the existing `per_bottle_dockerfile` pattern. Docker
stores a Docker tag in this file; smolmachines stores the absolute path
to the committed `.smolmachine` artifact.
### `commit` command
```
./cli.py commit [<slug>]
```
1. Resolve slug (arg or interactive picker from `enumerate_active_agents`).
2. Check metadata and branch by backend.
3. For Docker, derive container name `bot-bottle-<slug>` and run
`docker commit <container> bot-bottle-committed-<slug>:latest`.
4. For smolmachines, derive machine name `bot-bottle-<slug>` and run
`smolvm pack create --from-vm <machine> -o ~/.bot-bottle/state/<slug>/committed-smolmachine`.
5. Write the Docker image tag or smolmachine artifact path to
`~/.bot-bottle/state/<slug>/committed-image`.
6. Call `mark_preserved(<slug>)` so the state dir survives session-end.
7. Print the resume hint and a backend-specific export example.
### Resume from committed image
`bot_bottle/backend/docker/launch.py` already rebuilds the agent image
at the top of the `launch` context manager. The change is a check
immediately before that step:
```python
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
else:
docker_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
```
Replacing `agent_provision.image` propagates to `plan.image` (a
property) and from there to the Compose spec renderer's `_agent_service`
`image:` field, so the container boots from the committed snapshot.
The build step is skipped entirely when a committed image is found and
exists locally.
If the committed image has been deleted from the local daemon (e.g.
after `docker rmi` or a `docker system prune`), the launch falls back
to a normal Dockerfile build, matching the pre-commit behavior.
### Resume from committed smolmachine
`bot_bottle/backend/smolmachines/launch.py` checks the committed
reference before the normal Docker build -> pack cache path:
```python
committed = read_committed_image(plan.slug)
if committed and Path(committed).is_file():
return Path(committed)
return _ensure_smolmachine(plan.agent_image, dockerfile=plan.agent_dockerfile_path)
```
The returned path is passed to `smolvm machine create --from`, so the
resumed VM boots from the committed snapshot. If the artifact has been
deleted, launch falls back to the normal build and pack flow.
## Testing strategy
- Unit tests for `write_committed_image` / `read_committed_image` in
`tests/unit/test_bottle_state.py`, using the existing `_FakeHomeMixin`
pattern.
- Unit tests for `commit_container` in `tests/unit/test_docker_util_image.py`,
mocking `subprocess.run` and asserting on the `docker commit` argv.
- Unit tests for `cmd_commit` argument parsing, Docker commit,
smolmachines pack, and the unsupported backend error path, mocking
`enumerate_active_agents`, `commit_container`, and
`pack_create_from_vm`.
- Unit tests for the launch-step committed-image branch: patch
`read_committed_image` to return a tag, patch `image_exists` to return
True, and assert that `build_image` is not called and `plan.image` is
overridden.
- Unit tests for the smolmachines launch-step committed-artifact branch:
patch `read_committed_image` to return an existing path and assert the
normal `_ensure_smolmachine` path is skipped.
@@ -0,0 +1,101 @@
# PRD 0061: Gitleaks inline suppression supervision
- **Status:** Active
- **Author:** codex
- **Created:** 2026-06-10
- **Issue:** #208
## Summary
Route every `# gitleaks:allow` inline suppression through the existing
supervisor approval queue before git-gate forwards a push upstream. The
annotation remains available for test fixtures and confirmed false positives,
but it no longer lets an agent silently bypass the gate's secret scan.
## Problem
Gitleaks supports inline suppression comments. With the default behaviour,
a pushed line that contains a secret-like value and `# gitleaks:allow` is
ignored by gitleaks and accepted by git-gate. That is convenient for dummy
credentials in tests, but it also gives an agent a one-line bypass for a real
secret. Once the commit lands, the diff alone cannot prove whether the
suppression was legitimate.
## Goals / Success Criteria
1. Git-gate continues to run the normal gitleaks scan for every incoming ref.
2. After the normal scan passes, git-gate runs a second scan with
`--ignore-gitleaks-allow` and a JSON report so suppressed findings become
visible.
3. If that second scan reports no suppressed findings, the push proceeds
unchanged.
4. If it reports suppressed findings, git-gate creates a `gitleaks-allow`
supervisor proposal containing the ref, file path, line number, rule,
commit, and flagged line for each finding.
5. The push proceeds only when the supervisor explicitly approves the
proposal; rejection, malformed responses, missing supervisor configuration,
and timeout all refuse the push.
6. The supervisor TUI requires a reason when approving a `gitleaks-allow`
proposal, so the audit trail records whether the approval was for a test
fixture or a false positive.
## Non-goals
- Replacing gitleaks or changing the main secret-detection rule set.
- Removing support for `# gitleaks:allow`.
- Automatically classifying fixture files or false positives.
- Adding new supervisor transport or authentication mechanisms.
## Design
### Git-gate flow
`git_gate_render_hook()` emits a `supervise_gitleaks_allow` shell helper.
For each incoming ref, git-gate first runs the existing gitleaks command. If
that scan passes, it runs:
```sh
gitleaks git \
--log-opts="$log_opts" \
--no-banner \
--redact \
--ignore-gitleaks-allow \
--report-format=json \
--report-path="$report_file" \
--exit-code 0
```
The second pass keeps the push path non-interactive while producing a report
of findings that would otherwise have been hidden by inline suppression.
### Supervisor proposal
When the JSON report contains findings, an embedded Python helper writes a
proposal into `SUPERVISE_QUEUE_DIR` using the existing proposal schema. The
proposal uses:
- `tool: "gitleaks-allow"`
- a text payload with the ref and each finding's file, line, rule, commit,
and redacted code line
- a justification that tells the operator to approve only dummy test fixtures
or confirmed false positives
Git-gate then waits for `<proposal-id>.response.json` for
`SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS`, defaulting to 300 seconds.
`approved` and `modified` responses allow the push; `rejected`, invalid
responses, invalid timeout configuration, or timeout refuse it.
### Supervisor UI
`TOOL_GITLEAKS_ALLOW` is added to the supervisor tool registry. The curses
supervisor renders the proposal as text and allows approval or rejection.
Modification is unavailable for this proposal type because there is no file
patch to apply. Approval from the TUI prompts for a non-empty reason and
writes that reason to the response/audit path.
### Tests
Unit tests assert that the rendered git-gate hook includes the second gitleaks
pass, supervisor queue fields, and fail-closed messages. Supervisor tests cover
the new tool constant, proposal archiving, and the required TUI approval
reason.
@@ -0,0 +1,210 @@
# PRD 0062: Supervisor override for egress token blocks
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-24
- **Issue:** #261
## Summary
Give each egress route a policy for what happens when an outbound DLP detector
matches a token, via `dlp.outbound_on_match: block | redact | supervise`
(default `supervise`):
- **`supervise`** (default) — route the block through the existing supervisor
approval queue instead of returning `403` immediately. The proxy holds the
request open until the operator approves or rejects it. On approval the
matched token is added to an in-memory "safe tokens" set so the request — and
any later request carrying the same token — flows through without
re-prompting.
- **`redact`** — scrub the matched value(s) from the request and forward it,
no operator in the loop. For routes where a token-shaped value is noise the
upstream doesn't need (telemetry/log sinks). Fails closed if a match lands on
a surface redaction can't rewrite (the hostname).
- **`block`** — the original hard `403`; never overridable. For routes where a
detected token must always stop.
The motivating goal is reducing friction from false positives without weakening
the default-deny posture: supervise keeps a human in the loop, redact is an
explicit per-route opt-in, and block stays available for sensitive routes.
## Problem
The outbound DLP detectors (`token_patterns`, `known_secrets`) are
deliberately aggressive: any string that looks like a credential is blocked
before it leaves the bottle. That is the right default, but it produces false
positives — a token-shaped value that is not actually a secret, or a credential
the agent legitimately needs to send to a declared host. Today the only
recovery is for the operator to notice the `egress DLP` 403 in the logs and
hand-edit the route's `dlp.outbound_detectors`, which disables the detector for
the whole route rather than allowing the one value.
The operator has no in-the-loop signal that a token block happened and no
fine-grained way to say "this specific value is fine."
## Goals / Success Criteria
1. An outbound DLP **token** block (a `ScanResult` carrying a matched secret
value) creates a supervisor proposal instead of an immediate `403`.
2. The egress proxy holds the blocked request open, polling for the operator's
response up to a bounded timeout.
3. The proposal shows the operator the host, method, path, the detector reason,
and a **redacted** context snippet — never the raw token value.
4. On `approved`/`modified`, the matched token value is added to an in-memory
safe-tokens set and the request proceeds normally; later requests carrying
the same value skip the block.
5. On `rejected`, timeout, malformed response, or missing supervisor wiring,
the request fails closed with the same `403` as today.
6. Structural blocks that carry no token value (CRLF injection) and the
route-not-allowlisted / git blocks are unchanged — they stay hard `403`s and
keep their existing agent-driven `allow` / `egress-block` MCP path.
7. The proxy event loop is not stalled while waiting: the wait is asynchronous,
so other flows keep being served.
## Non-goals
- Persisting the safe-tokens set across egress restarts. It lives in process
memory only; a restart re-prompts. (The issue explicitly defers persistence.)
- Supervising inbound (prompt-injection) blocks or WebSocket frame blocks.
WebSocket frames still honour the safe-tokens set for already-approved values
but cannot wait for approval (there is no response surface after upgrade).
- Generalising an approved secret across encodings. The safe-tokens set matches
the exact value the detector found.
- Replacing the per-route `dlp.outbound_detectors` override. That remains the
way to turn a detector off wholesale.
- Making `redact` the default. Silent redaction of a true false positive
corrupts legitimate data, so it is opt-in per route; `supervise` (human in
the loop) stays the default.
## Scope
### In scope
The minimum cut that ships, in build order:
1. **Core**`ScanResult.matched`; thread `safe_tokens` through
`scan_outbound` / the token detectors; `build_token_allow_payload`.
2. **Supervise + TUI**`TOOL_EGRESS_TOKEN_ALLOW`; TUI suffix, modify guard,
required approval reason.
3. **Addon glue** — async `request`, safe-tokens set, proposal write + async
poll, allow/block decision; pass `safe_tokens` into the WebSocket path.
4. **On-match policy**`dlp.outbound_on_match` through manifest → render →
addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in
the addon's outbound handler.
5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README
egress + supervisor notes.
### Out of scope
The deferrals enumerated under **Non-goals** — restart persistence, inbound /
WebSocket-frame supervision, cross-encoding generalisation, replacing
`dlp.outbound_detectors`, and making `redact` the default.
## Proposed Design
### New services / components
A new proposal tool constant `egress-token-allow` (`TOOL_EGRESS_TOKEN_ALLOW`)
is added to `supervise.TOOLS`, and the egress addon gains an in-memory
safe-tokens set plus the policy-dispatch path that drives it.
On an outbound block the addon dispatches on the resolved policy:
- **Structural blocks always 403.** A `ScanResult` with no `matched` value
(CRLF injection) is a hard `403` regardless of policy — there is nothing to
redact or safelist.
- **`redact`** runs `redact_tokens` over the body, non-`host` header values,
and path/query, then re-scans. If the re-scan is clean the (rewritten)
request is forwarded; if a block-severity match remains (e.g. in the
hostname, or a unicode-evasion token redaction can't reach) it fails closed
with a `403`.
- **`block`** writes the `403` immediately.
- **`supervise`** runs the queue-and-wait loop, falling back to `block` when
supervise isn't wired for the bottle.
For `supervise`, the addon writes the proposal directly to
`SUPERVISE_QUEUE_DIR` (the queue is bind-mounted into the sidecar bundle and
shared by every daemon, exactly as git-gate's `gitleaks-allow` proposal in PRD
0061 does). The proposal's `proposed_file` is a human-readable text payload
built by `build_token_allow_payload`:
```
egress blocked an outbound request carrying a detected token
host: api.example.com
method: POST
path: /v1/ingest
detector: OpenAI API key found in body
context: ...before ******** after...
```
The justification tells the operator to approve only if the value is a false
positive or a credential the request legitimately needs. The addon then polls
`<proposal-id>.response.json` for `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default
300). `approved`/`modified` allow the request and add the value to the
safe-tokens set; `rejected`, malformed responses, and timeout fail the request
closed. The proposal + response are archived to `processed/` after a decision.
Because the wait happens inside mitmproxy's asyncio loop, the addon's `request`
hook is async and polls with `asyncio.sleep`, so concurrent flows are
unaffected.
### Existing code touched
- **Policy threading.** `dlp.outbound_on_match` is a per-route enum threaded
from the bottle manifest (`manifest_egress`) through the resolved route
(`egress.EgressRoute`), the rendered `routes.yaml` (`egress_render_routes`),
and the addon's `Route` (`egress_addon_core`). Unset renders nothing and
resolves to `supervise` at request time. The `list-egress-routes`
introspection endpoint round-trips it so the agent's proposals preserve it.
- **Provider-route default.** Agent-provider routes (the agent talking to its
own LLM API — `api.anthropic.com`, the Codex backend, etc.) are the worst
source of token-shaped false positives because the whole conversation payload
flows through them. `egress_routes_for_bottle` fills `outbound_on_match=redact`
on any provider route that doesn't set it explicitly; a provider that sets the
policy keeps its choice, and manifest routes are unaffected (they default to
`supervise`).
- **Scanners.** `scan_outbound` (and the token detectors `scan_token_patterns`
/ `scan_known_secrets` it calls) accept a `safe_tokens` set. A match whose
value is in `safe_tokens` is skipped, so an approved token no longer blocks;
the scanners keep searching past a safelisted match so a second, un-approved
secret in the same request is still caught. The WebSocket path is passed the
same `safe_tokens` set.
- **Supervisor UI.** `cli/supervise.py` renders `egress-token-allow` like
`gitleaks-allow`: the text payload is shown, modify is unavailable (there is
no file patch to edit), and approval prompts for a non-empty reason recorded
in the response notes. There is no on-disk config diff, so — like
`gitleaks-allow` and `capability-block` — it writes no egress audit-log entry.
- **Failure handling.** If `SUPERVISE_QUEUE_DIR` / `SUPERVISE_BOTTLE_SLUG` are
unset (supervise disabled for the bottle), the addon skips the queue and
returns the existing `403`. Any error writing the proposal or reading the
response also fails closed.
### Data model changes
- New per-route manifest field `dlp.outbound_on_match: block | redact |
supervise`, rendered into `routes.yaml` (omitted when unset).
- `ScanResult` gains a `matched: str = ""` field carrying the raw substring the
detector matched. The token detectors populate it; the structural CRLF
detector leaves it empty. The value stays inside the egress sidecar process —
never written to a log line (logs use the redacted `context`) nor to the
proposal file.
- Proposal text payload (above) plus `<proposal-id>.response.json` in
`SUPERVISE_QUEUE_DIR`, archived to `processed/` after a decision.
- New env var `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default 300).
### External dependencies
None. Reuses the existing supervisor queue (`SUPERVISE_QUEUE_DIR`) and the
mitmproxy addon framework already in the egress sidecar.
## Open questions
- Should `known_secrets` (provisioned `EGRESS_TOKEN_*` exfiltration) be
override-able at all, or only `token_patterns`? This PRD allows both —
approval is an explicit operator decision and the safe-tokens set matches the
exact found value — but a future revision could restrict `known_secrets` to
reject-only.
## References
- Issue #261
- PRD 0061 — `gitleaks-allow` supervisor proposal pattern this reuses.
@@ -0,0 +1,189 @@
# PRD 0063: Strengthen outbound exfiltration detection
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-25
- **Issue:** #259
## Summary
The current DLP system has two detection tiers: weak (vendor token regexes)
and strong (exact matching of provisioned `EGRESS_TOKEN_*` values in multiple
encodings). This PRD adds four complementary hardening layers in ranked order
of impact:
1. **Canary tokens** — inject a per-session fake secret into the agent's
environment; any outbound appearance is a zero-false-positive exfil signal.
2. **Broadened known-value scanning** — extend the strong tier beyond
`EGRESS_TOKEN_*` via a `BOT_BOTTLE_SENSITIVE_PREFIXES` sidecar env var.
3. **Fragmentation-resistant matching** — catch separator injection and
long-substring leaks via alnum projection and sliding-window substring
search.
4. **Entropy scoring** — warn-only flag on high-entropy blobs as a
complementary signal for encrypted exfil.
## Problem
After PRD 0056 the outbound scan covers all HTTP request surfaces (host, path,
query, headers, body). But the content-based strong tier only matches
`EGRESS_TOKEN_*` values and their encodings. Three gaps remain:
| Gap | How an agent exploits it |
|-----|--------------------------|
| No canary | There is no "can't be false-positive" signal |
| Narrow prefix filter | Sensitive values provisioned under other env prefixes (MCP keys, API keys injected via the agent provider) are invisible to `scan_known_secrets` |
| Fragmentation blindness | Inserting `-`, space, or `\n` between secret characters turns an exact-match into a miss |
## Goals / Success Criteria
1. Each launched bottle has a unique canary token in the agent's environment
under a randomized `WORD_WORD_SECRET` env var name. The egress sidecar gets
the same env var and registers that exact name through
`BOT_BOTTLE_SENSITIVE_PREFIXES`. Any outbound appearance of the canary
blocks the request as a known-secret match.
2. `scan_known_secrets` accepts a `sensitive_prefixes` parameter (default:
`("EGRESS_TOKEN_",)`). `scan_outbound` reads
`BOT_BOTTLE_SENSITIVE_PREFIXES` from `environ` and merges those prefixes
in, so operators can mark additional env vars as scanned values without
changing the manifest schema.
3. For every secret that passes exact-match, a secondary alnum-projection pass
checks for the secret with all non-alphanumeric characters stripped. This
catches separator-injection evasion (`MY-SECRET` → body contains
`MY SECRET`).
4. A sliding-window partial-match pass checks for long-enough contiguous
substrings of the secret's alnum projection in the text's alnum projection.
Any match ≥ `PARTIAL_MATCH_MIN_LEN` (12 chars) blocks with reason
`"partial match"`.
5. A new `scan_entropy` detector flags outbound text windows with Shannon
entropy ≥ `ENTROPY_BLOCK_THRESHOLD` (5.5 bits/char) at **warn** severity
only. It is registered under the new detector name `"entropy"` in
`OUTBOUND_DETECTOR_NAMES` and disabled by default (routes must opt in).
6. Binary request bodies are decoded via `latin-1` instead of
`utf-8 errors="replace"`, preserving every byte value and allowing
ASCII-range secrets to be found within binary payloads.
7. All new behaviour is unit-tested; existing tests pass unchanged.
## Non-goals
- Rolling per-host buffer for split-across-requests detection (state in the
stateless addon is complex; deferred).
- Additional vendor regexes.
- ML / embedding-based detection.
- Entropy-based hard blocks (warn only per the issue).
## Design
### Canary token flow
```
Egress.prepare()
canary = secrets.token_urlsafe(32)
canary_env = <random WORD_WORD_SECRET>
EgressPlan(canary=canary, canary_env=canary_env, ...)
Docker compose render:
sidecar env: <canary_env>=<canary>
sidecar env: BOT_BOTTLE_SENSITIVE_PREFIXES=<canary_env>
agent env: <canary_env>=<canary> ← visible to agent as a "secret"
macos-container launch: same literals added to sidecar + agent env entries
```
The sidecar uses `BOT_BOTTLE_SENSITIVE_PREFIXES` to make the random canary env
name part of the existing `scan_known_secrets` detector without adding a
manifest schema field.
### Broadened known-value scanning
`scan_known_secrets` gains a `sensitive_prefixes` parameter:
```python
def scan_known_secrets(
text: str,
*,
location: str = "body",
env: Mapping[str, str] | None = None,
sensitive_prefixes: tuple[str, ...] = ("EGRESS_TOKEN_",),
) -> ScanResult | None:
```
`scan_outbound` reads `BOT_BOTTLE_SENSITIVE_PREFIXES` (comma-separated list
of additional prefixes) from `environ` and appends them:
```python
extra = tuple(
p for p in environ.get("BOT_BOTTLE_SENSITIVE_PREFIXES", "").split(",") if p
)
sensitive_prefixes = ("EGRESS_TOKEN_",) + extra
```
`redact_tokens` receives the same treatment for consistent redaction.
### Fragmentation-resistant matching
A new helper `_alnum_projection(text)` strips all non-alphanumeric characters.
`scan_known_secrets` runs two passes per secret:
1. **Exact pass** — existing encoded-variant loop (unchanged).
2. **Alnum-projection pass** — if the secret's alnum projection has ≥ 8 chars,
check if it appears in the text's alnum projection. Match → block with
`"fragmented match (separator injection)"` reason.
3. **Partial-substring pass** — if the secret's alnum projection has ≥
`PARTIAL_MATCH_MIN_LEN` chars (12), slide a window of that length across the
secret's projection and look for each window in the text's alnum projection.
First match → block with `"partial match"` reason.
All three passes run only for the `"known_secrets"` detector; the token-pattern
and entropy detectors are unchanged.
### Entropy scoring
New public function:
```python
def scan_entropy(
text: str,
*,
location: str = "body",
window: int = ENTROPY_WINDOW, # 64
threshold: float = ENTROPY_BLOCK_THRESHOLD, # 5.5
) -> ScanResult | None:
```
Slides a window of `window` characters across `text` in steps of `window // 2`.
If any window's Shannon entropy exceeds `threshold`, returns a **warn**-severity
`ScanResult`. Never blocks.
`OUTBOUND_DETECTOR_NAMES` gains `"entropy"`. Routes opt in via their `dlp`
block; entropy scanning is **off by default** to avoid false-positive noise on
legitimate binary payloads.
### Binary body handling
In `scan_outbound`, the bytes → str decoding changes from:
```python
body.decode("utf-8", errors="replace")
```
to:
```python
body.decode("utf-8") if body is str else body.decode("latin-1")
```
`latin-1` is a bijective byte↔codepoint mapping; every byte value is preserved
as its corresponding Latin-1 code point, so ASCII-range secret strings remain
intact and `str.find` / regex still locate them correctly. The fallback from
strict UTF-8 is tried first so valid UTF-8 bodies are decoded faithfully.
## Implementation
Delivered in three commits on the same branch:
1. **DLP detector changes**`_alnum_projection`, fragmentation passes,
`scan_entropy`, broadened `scan_known_secrets`, updated `scan_outbound` and
`redact_tokens`; all accompanying unit tests.
2. **Canary injection**`EgressPlan.canary`, `Egress.prepare()`,
Docker compose + macos-container backend injection.
3. **PRD flip**`Status: Draft → Active`.

Some files were not shown because too many files have changed in this diff Show More