Compare commits

...

103 Commits

Author SHA1 Message Date
didericis-claude df469b2f47 docs: add role and git.fetch to egress route fields table
Both fields were missing from the reference table added in the preceding
commit — `role` is visible in examples/bottles/claude.md and `git.fetch`
is documented in PRD 0052 but neither appeared in the README table.
2026-06-22 18:31:32 +00:00
didericis d1d9e7a105 docs: document egress matches, dlp fields, and detector defaults
lint / lint (push) Successful in 1m32s
2026-06-19 21:58:20 -04:00
didericis-claude 7a124d7d25 refactor: make static the default branch in _parse_key_config
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 16s
lint / lint (push) Successful in 1m31s
test / unit (push) Successful in 28s
test / integration (push) Successful in 15s
Update Quality Badges / update-badges (push) Successful in 1m10s
2026-06-19 22:25:14 +00:00
didericis-claude f00c567469 rename: provisioner_token -> forge_token_env
lint / lint (push) Successful in 2m6s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
2026-06-19 22:21:37 +00:00
didericis-claude 6f0e5b4589 refactor: extract _resolve_identity_file from prepare loop
lint / lint (push) Successful in 1m33s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 16s
2026-06-19 22:14:15 +00:00
didericis-claude 5da4d05bf2 fix: remove unused Optional import flagged by pyright
lint / lint (push) Successful in 1m33s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
2026-06-19 22:09:52 +00:00
didericis-claude 1a8718ca9d refactor: unify identity/provisioned_key into key block
lint / lint (push) Failing after 1m45s
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 17s
Replace the two mutually-exclusive repo keys (identity and
provisioned_key) with a single required key block. key.provider
is "static" (path to host SSH key) or "gitea" (deploy-key lifecycle
via provisioner_token env var, replacing token_env).

Internal fields: ManifestProvisionedKeyConfig → ManifestKeyConfig;
ProvisionedKey field removed from ManifestGitEntry; Key field added.
git_gate.py checks entry.Key.provider == "gitea" instead of
entry.ProvisionedKey is not None.
2026-06-19 22:01:43 +00:00
didericis-claude c1c225aa05 docs(gitea-provisioner): document required GITEA_DEPLOY_TOKEN permissions
lint / lint (push) Successful in 1m46s
test / unit (push) Successful in 34s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m21s
2026-06-11 03:43:13 +00:00
didericis dc7c10d6fe fix(macos-container): use correct system status probe in preflight
lint / lint (push) Successful in 1m41s
test / unit (push) Successful in 34s
test / integration (push) Successful in 21s
Update Quality Badges / update-badges (push) Successful in 1m23s
`container system info` is not a valid subcommand and always returned
non-zero, causing a false-positive on the service check. Switch to
`container system status` which is the correct command.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 23:31:32 -04:00
Quality Badge Bot a827b0841e chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors

[skip ci]
2026-06-11 03:28:32 +00:00
didericis a9c93ea9df fix(macos-container): preflight check for container system service
lint / lint (push) Successful in 1m43s
test / unit (push) Successful in 34s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Successful in 1m27s
Fail early with a clear message when the Apple Container system service
isn't running, instead of surfacing an opaque XPC connection error mid-build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 23:24:09 -04:00
didericis bb69af31f8 chore(claude): bump claude-code to 2.1.170
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 22:44:46 -04:00
didericis 7644da4280 docs: add Apple Container transparent egress spike 2026-06-10 22:36:55 -04:00
didericis 13e4af421d docs: add Apple Container networking spike 2026-06-10 22:36:55 -04:00
github-actions[bot] f2d5307573 ci(prd): assign sequential numbers to new PRDs 2026-06-11 02:36:07 +00:00
didericis bc9a22b46a fix(macos-container): support git-gate launch
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 20s
lint / lint (push) Successful in 1m45s
prd-number / assign-numbers (push) Successful in 25s
test / unit (push) Successful in 32s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Failing after 1m23s
2026-06-10 22:25:00 -04:00
didericis 932e71c0bf fix(macos-container): make backend the macos default 2026-06-10 22:25:00 -04:00
didericis d3b0b330aa fix(macos-container): preserve working builder dns 2026-06-10 22:25:00 -04:00
didericis 5e927bcd13 fix(macos-container): start builder with dns 2026-06-10 22:25:00 -04:00
didericis 890a146413 test(macos-container): add launch integration smoke 2026-06-10 22:25:00 -04:00
didericis afdf0779a1 feat(macos-container): launch explicit-proxy bottles 2026-06-10 22:25:00 -04:00
didericis-codex eb7cae1fea docs: link macos container prd to review comment 2026-06-10 22:25:00 -04:00
didericis-codex fe82dc7f2b feat: add macos container backend scaffold 2026-06-10 22:25:00 -04:00
didericis-claude b00b0ba4aa fix(git-gate): forward force push as +refspec to upstream
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 22s
lint / lint (push) Successful in 1m47s
test / unit (push) Successful in 34s
test / integration (push) Successful in 20s
Update Quality Badges / update-badges (push) Successful in 1m21s
When $old != zero and $new is not a descendant of $old (detected via
git merge-base --is-ancestor), the hook now forwards +$new:$ref so the
upstream accepts the force push instead of rejecting it as a
non-fast-forward.

Closes #233
2026-06-11 02:17:27 +00:00
didericis-codex 3f04567290 egress: require opt-in for HTTPS git fetch
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 27s
lint / lint (push) Successful in 1m53s
test / unit (push) Successful in 41s
test / integration (push) Successful in 23s
Update Quality Badges / update-badges (push) Successful in 1m35s
2026-06-10 07:00:01 +00:00
didericis-codex acb9cd67c6 fix(git-gate): forward push options
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m28s
test / unit (push) Successful in 29s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m7s
2026-06-10 02:36:47 -04:00
didericis-codex d90ab7e646 ci: enforce pylint threshold
lint / lint (push) Successful in 1m37s
test / unit (push) Successful in 34s
test / integration (push) Successful in 16s
Update Quality Badges / update-badges (push) Successful in 1m13s
2026-06-10 06:30:03 +00:00
didericis-codex 8ea90adcaf fix: raise git http body cap 2026-06-10 06:29:46 +00:00
Quality Badge Bot de803e1e76 chore: update quality badges
- Pylint: 9.94/10
- Pyright: 0 errors

[skip ci]
2026-06-10 05:34:38 +00:00
didericis-codex 019efab804 fix: narrow pi numeric settings types
test / unit (pull_request) Successful in 49s
test / integration (pull_request) Successful in 27s
lint / lint (push) Successful in 1m37s
test / unit (push) Successful in 31s
test / integration (push) Successful in 19s
Update Quality Badges / update-badges (push) Successful in 1m20s
2026-06-10 01:13:00 -04:00
didericis-codex 957d37f51f fix: merge egress routes across extends 2026-06-10 01:13:00 -04:00
Quality Badge Bot 8e084262a0 chore: update quality badges
- Pylint: 9.94/10
- Pyright: 4 errors

[skip ci]
2026-06-10 04:07:25 +00:00
didericis 504144eb9c fix(pi): prepare runtime state and agent workdir
lint / lint (push) Failing after 1m58s
test / unit (push) Successful in 41s
test / integration (push) Successful in 24s
Update Quality Badges / update-badges (push) Successful in 1m27s
2026-06-10 00:02:28 -04:00
didericis 86374ab293 fix(pi): select configured startup models
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
lint / lint (push) Successful in 1m37s
test / unit (push) Successful in 33s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m6s
2026-06-09 06:57:33 -04:00
didericis 199edb228c feat(pi): add fd and ripgrep to image
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 18s
2026-06-09 06:26:19 -04:00
didericis 598a20a3f0 fix(pi): keep interactive sessions open
lint / lint (push) Successful in 1m29s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 17s
2026-06-09 06:00:40 -04:00
didericis c8b5ba3812 feat(pi): support egress injected api keys
lint / lint (push) Successful in 1m38s
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 17s
2026-06-09 05:56:39 -04:00
didericis-codex 5ea9fda69b docs: activate pi provider prd
lint / lint (push) Successful in 1m43s
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 18s
2026-06-09 08:32:09 +00:00
didericis-codex 4f7cfc0418 feat: add pi agent provider 2026-06-09 08:31:48 +00:00
didericis-codex 1f38a96561 docs: add pi provider prd 2026-06-09 08:23:00 +00:00
Quality Badge Bot 660b9b3810 chore: update quality badges
- Pylint: 9.94/10
- Pyright: 0 errors

[skip ci]
2026-06-09 05:46:00 +00:00
didericis 328069809b fix(pyright): remove unused shlex imports from bottle backends
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 15s
lint / lint (push) Successful in 1m35s
test / unit (push) Successful in 31s
test / integration (push) Successful in 17s
Update Quality Badges / update-badges (push) Successful in 1m30s
shlex is now only used in terminal.py after the exec_shell_script refactor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:48:46 -04:00
didericis b1551045dc feat(terminal): tint terminal background per agent color
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 17s
Add backend-agnostic terminal color support via OSC escape sequences:
- New backend/terminal.py with palette_printf() and exec_shell_script()
  shared by both Docker and smolmachines bottle backends
- Emits OSC 4 (indexed palette) + OSC 11 (default background tint)
  before launching; resets both on agent exit via OSC 104/111
- OSC 11 background tint is visible even when the TUI uses true/24-bit
  colors (which bypass the palette), as Codex does for its chrome
- Fix Codex [tui] config: status_line=["model-with-reasoning"],
  theme="ansi" (dark-ansi and cwd/directory were invalid identifiers)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:47:55 -04:00
didericis d02226aab9 feat: forward agent style via native CLI config and terminal title
Replace prompt-injection for display identity with native UI wiring:
- Claude: writes a statusline shell script + custom theme JSON, wired up
  via settings.json so label/color show in the status bar and theme
- Codex: writes [tui] block into codex-config.toml (status_line,
  terminal_title, dark-ansi theme)
- Both backends set the terminal title via ANSI OSC 0 escape before
  exec-ing the agent when a label is present

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:47:55 -04:00
didericis-codex 39811c9b32 feat: forward agent display identity to prompts 2026-06-09 00:47:55 -04:00
github-actions[bot] f7f161e60f ci(prd): assign sequential numbers to new PRDs 2026-06-09 03:37:10 +00:00
didericis-codex e6040fc824 fix(start): skip backend selector
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 24s
lint / lint (push) Successful in 1m47s
prd-number / assign-numbers (push) Successful in 29s
test / unit (push) Successful in 38s
test / integration (push) Successful in 27s
Update Quality Badges / update-badges (push) Failing after 1m20s
2026-06-09 03:31:26 +00:00
didericis-codex 17fc44d0d8 complete(prd): mark smolmachines default active
lint / lint (push) Successful in 1m46s
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 22s
2026-06-09 03:27:58 +00:00
didericis-codex 1bebb7467f feat(backend): default to smolmachines 2026-06-09 03:27:31 +00:00
didericis cc1d986a74 test: fix smolmachines proxy assertions
lint / lint (push) Successful in 1m52s
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 25s
2026-06-08 23:22:47 -04:00
didericis-codex fabcd026af test(smolmachines): verify TSI egress proxy path
lint / lint (push) Successful in 1m47s
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 23s
2026-06-09 03:14:58 +00:00
didericis aff042855a ci(prd): rename PRD to prd-new placeholder per new convention
lint / lint (push) Successful in 1m47s
2026-06-08 23:10:09 -04:00
didericis 39b0c4f720 docs(prd): renumber PRD 0055 → 0058 (0055 slot taken by extended-outbound-scan) 2026-06-08 23:10:09 -04:00
didericis 43a5700ae6 docs(prd): PRD 0055 - promote smolmachines to default backend 2026-06-08 23:10:09 -04:00
didericis-codex 7acdabaf96 test: narrow metadata assertions for pyright 2026-06-08 23:05:14 -04:00
didericis-codex dfd2d5f620 fix: restore runtime workspace provisioning 2026-06-08 23:05:14 -04:00
didericis-codex f24e2857ab fix: restore backend prepare wiring 2026-06-08 23:05:14 -04:00
didericis-codex d38432f640 fix: resolve pyright strict errors 2026-06-08 23:05:14 -04:00
didericis 4e570e3e2b fix(egress): ignore stripped auth header in DLP scan 2026-06-08 23:05:14 -04:00
didericis-claude a64e3170cd refactor: make AgentProvisionPlan the source of truth for instance_name, prompt_file, image, dockerfile, guest_home
Drop the parallel fields passed through prepare() → _resolve_plan and
read everything from agent_provision instead. The provider plugin now
declares its own guest_home (so the backend stops hardcoding
"/home/node") and the wrapper that builds the provision plan accepts
instance_name and prompt_file, which providers store on the plan.

DockerBottlePlan and SmolmachinesBottlePlan expose container_name /
machine_name, image / agent_image, dockerfile_path /
agent_dockerfile_path, and prompt_file as properties that delegate to
agent_provision so existing call sites keep working unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 23:05:14 -04:00
didericis-claude 4da4babcf4 fix: fall back to provider's bundled Dockerfile when manifest doesn't override
BottleBackend.prepare was calling resolve_manifest_dockerfile("", spec)
for every bottle where the manifest did not set agent_provider.dockerfile.
That resolves an empty string against user_cwd, returning the cwd
itself — which docker then tried to read as a Dockerfile, giving
"is a directory" errors during image build.

When the manifest doesn't override, use the provider plugin's bundled
Dockerfile path (next to its agent_provider.py module) — mirroring
the pre-refactor behavior.
2026-06-08 23:05:14 -04:00
didericis-claude 384e496a1b fix: thread slug + resolved_env from prepare to each backend's _resolve_plan
BottleBackend.prepare computed slug and resolved_env but never passed
them to _resolve_plan. The concrete docker/smolmachines _resolve_plan
methods still had the old (spec, *, stage_dir) signature too, so
prepare's kwargs blew up with "unexpected keyword argument
'instance_name'" the moment cli.py start was invoked.

Update the abstract _resolve_plan signature and both backend
implementations to accept the full kwarg set prepare passes, and
forward to resolve_plan.resolve_plan() with everything.
2026-06-08 23:05:14 -04:00
didericis-claude b38c6110f2 chore: comment out workspace + capability_apply, fix circular imports
The recent refactor partially removed workspace planning and
capability-apply logic. This commit finishes the cleanup so the
test suite imports cleanly:

- Comment out workspace_plan field/property on BottlePlan and the
  provision_workspace dispatch.
- Comment out workspace usages in docker.util (build_image_with_cwd),
  smolmachines.provision.workspace, agent_provider.provision_git,
  smolmachines.backend.
- Comment out capability_apply imports in cli.start and cli.supervise;
  add a local CapabilityApplyError placeholder so the supervise CLI
  module still imports.
- Break the bottle_state → backend.docker → backend circular import
  by lazy-loading docker_mod inside bottle_identity, and by moving the
  resolve_common import inside BottleBackend.prepare.
- Delete tests for workspace and capability_apply (unit + integration).
- Update test fixtures to drop removed kwargs (container_name_pinned,
  derived_image, env_file, workspace_plan, agent_image_ref) from
  DockerBottlePlan / SmolmachinesBottlePlan constructors.
- Delete the obsolete test_smolmachines_prepare.py (tested the old
  resolve_plan signature; the shared prepare flow now lives in
  BottleBackend.prepare).
- Adjust test_supervise.py for the new Supervise.prepare signature
  (dockerfile_content arg removed).

925 → 897 tests, all passing.
2026-06-08 23:05:14 -04:00
didericis 74efb1c143 chore: sketch out desired refactor
Manual refactor into the rough shape we want/how we want the
resolve_plan logic to be consolidated. Needs subsequent fixes.
2026-06-08 23:05:14 -04:00
didericis-claude f23b2b9683 refactor: move guest_home onto AgentProvisionPlan as source of truth
guest_home is now a field on AgentProvisionPlan (set by each provider's
provision_plan() method). BottlePlan.guest_home becomes a read-only
property delegating to agent_provision.guest_home so existing callers
(provision_git, provision_skills, provision_prompt) are unchanged.

Both resolve_plan.py files drop guest_home from the plan constructor
call; the local variable still exists as an intermediary for the
workspace_plan call that precedes agent_provision_plan.
2026-06-08 23:05:14 -04:00
didericis-claude 423003aa05 refactor: extract shared resolve_plan helpers into backend/resolve_common.py
Both docker and smolmachines resolve_plan.py duplicated: slug minting,
metadata writing, agent state dir setup, git gate / egress / supervise
preparation, env_vars merge, and manifest dockerfile path resolution.

These are now consolidated in bot_bottle/backend/resolve_common.py.
Each backend's resolve_plan retains only its own logic (container name
resolution + env-file for docker; subnet allocation + guest_env build
for smolmachines).
2026-06-08 23:05:14 -04:00
didericis-claude af82f2ba20 refactor: move bottle_state.py to top-level bot_bottle package
Both docker and smolmachines backends use bottle state helpers.
Moving to bot_bottle/ makes the sharing explicit and removes the
cross-backend dependency (smolmachines importing from ..docker).

All callers updated: docker backend, smolmachines backend, cli
modules, and tests.
2026-06-08 23:05:14 -04:00
didericis-claude fe8e15d211 refactor: rename prepare.py → resolve_plan.py in both backends 2026-06-08 23:05:14 -04:00
didericis-claude b098556757 refactor: prefix all manifest data classes with Manifest
Avoids name collisions with same-named runtime/plugin classes
(e.g. manifest AgentProvider vs plugin AgentProvider ABC,
manifest EgressRoute vs runtime EgressRoute). Renamed:

  AgentProvider        → ManifestAgentProvider   (manifest_agent.py)
  Agent                → ManifestAgent            (manifest_agent.py)
  EgressRoute          → ManifestEgressRoute      (manifest_egress.py)
  PathMatch            → ManifestPathMatch        (manifest_egress.py)
  HeaderMatch          → ManifestHeaderMatch      (manifest_egress.py)
  MatchEntry           → ManifestMatchEntry       (manifest_egress.py)
  EgressConfig         → ManifestEgressConfig     (manifest_egress.py)
  Bottle               → ManifestBottle           (manifest.py)
  ProvisionedKeyConfig → ManifestProvisionedKeyConfig (manifest_git.py)
  GitEntry             → ManifestGitEntry         (manifest_git.py)
  GitUser              → ManifestGitUser          (manifest_git.py)
2026-06-08 23:05:14 -04:00
didericis-claude 5c5f277d6d refactor: set image/dockerfile from provider default first, override after
Since every provider always has a dockerfile, establish the default
image and dockerfile_path from the provider up front and override for
per-bottle or manifest-specified cases. Removes the image_default
intermediate variable and the trailing else branch.
2026-06-08 23:05:14 -04:00
didericis-claude 2fa5229695 refactor: AgentProvider.dockerfile always returns Path, never None
The convention is that every provider declares a Dockerfile location;
callers that care whether the file actually exists check .is_file().
Drops all `is not None` guards on the property result.
2026-06-08 23:05:14 -04:00
didericis-claude c3caa3ea94 refactor: remove BOT_BOTTLE_IMAGE env override
Unused in tests, docs, or examples. Can be added back if/when merited.
2026-06-08 23:05:14 -04:00
didericis-claude ee0607f022 refactor: replace runtime.dockerfile with AgentProvider.dockerfile property
Drop the `dockerfile` field from `AgentProviderRuntime` and replace it
with a convention-based `dockerfile` property on `AgentProvider`: the
base class looks for a `Dockerfile` file next to the provider's own
`agent_provider.py` module (via `inspect.getfile`), returning its path
or None. Built-in providers inherit the default automatically; custom
user providers work the same way by dropping a Dockerfile next to their
plugin file; any provider needing a non-standard path can override.

All callers (`docker/prepare.py`, `smolmachines/prepare.py`,
`capability_apply.py`) now resolve the provider object once and call
`.dockerfile` directly instead of reading `runtime.dockerfile`.
2026-06-08 23:05:14 -04:00
didericis-claude afe5d43a9a refactor: move agent Dockerfiles into their contrib directories
Dockerfile.claude and Dockerfile.codex move from the repo root into
bot_bottle/contrib/claude/Dockerfile and bot_bottle/contrib/codex/Dockerfile
respectively, so all per-provider assets live alongside the provider code.

Closes #215
2026-06-08 23:05:14 -04:00
didericis dd332a5759 chore: Replace die with YamlSubsetError 2026-06-08 23:05:11 -04:00
github-actions[bot] 103f9adcfd ci(prd): assign sequential numbers to new PRDs 2026-06-08 03:26:08 +00:00
didericis 652c8cb5a7 ci(prd): rename PRD to prd-new placeholder per new convention
test / unit (pull_request) Successful in 37s
test / integration (pull_request) Successful in 49s
lint / lint (push) Successful in 1m30s
prd-number / assign-numbers (push) Successful in 32s
test / unit (push) Successful in 31s
test / integration (push) Successful in 42s
Update Quality Badges / update-badges (push) Successful in 1m11s
2026-06-07 23:19:11 -04:00
didericis 11a8f3ba99 docs(prd): renumber PRD 0053 → 0055 (0053 slot claimed by user-provider-plugins) 2026-06-07 23:19:11 -04:00
didericis-claude 451e6fc2fc feat(dlp): add 7 token patterns, Unicode normalization, CRLF injection detection (PRD 0053)
Token patterns: HuggingFace (hf_), Databricks (dapi), Slack (xox[baprs]-),
npm (npm_), SendGrid (SG.x.y), PyPI (pypi-), HashiCorp Vault (hvs.).

Unicode normalization (_normalize_text) applies NFKD + strips combining
marks and control chars before pattern matching, defeating fullwidth-char
and combining-mark evasion.

CRLF injection (scan_crlf_injection) detects %0d%0a in URLs and literal
\r\n header-injection patterns; runs unconditionally in scan_outbound
regardless of outbound_detectors config.
2026-06-07 23:19:11 -04:00
didericis-claude 1ecef55fea feat(dlp): websocket scanning, response headers, extended encoding variants, sk-proj pattern (PRD 0053) 2026-06-07 23:19:11 -04:00
didericis-claude 76e38b24e6 fix(types): resolve pyright errors in test_egress_addon_core 2026-06-07 23:19:11 -04:00
didericis-claude b1283a0e7b feat(egress): extend outbound DLP scan to headers, query params, path, and hostname (PRD 0053) 2026-06-07 23:19:11 -04:00
didericis-claude 2c51bc47e8 docs(prd): PRD 0053 extended outbound DLP scan surfaces 2026-06-07 23:19:11 -04:00
Quality Badge Bot ff495c1521 chore: update quality badges
- Pylint: 9.95/10
- Pyright: 0 errors

[skip ci]
2026-06-08 02:40:06 +00:00
didericis a04aed098d fix(egress): strip Authorization before DLP scan; remove auth_header param from scan_outbound
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 46s
lint / lint (push) Successful in 1m27s
test / unit (push) Successful in 35s
test / integration (push) Successful in 42s
Update Quality Badges / update-badges (push) Successful in 1m20s
2026-06-07 22:30:10 -04:00
github-actions[bot] 916b70c595 ci(prd): assign sequential numbers to new PRDs 2026-06-08 00:34:45 +00:00
didericis 55cb3429d4 fix(lint): add parse_config tests to satisfy pyright unused-import
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 43s
lint / lint (push) Successful in 1m26s
prd-number / assign-numbers (push) Successful in 35s
test / unit (push) Successful in 28s
test / integration (push) Successful in 44s
Update Quality Badges / update-badges (push) Failing after 1m8s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 20:25:59 -04:00
didericis 545ff3582f fix(lint): resolve pylint and pyright issues on egress-log-option
lint / lint (push) Failing after 1m34s
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 44s
- egress.py: extract _render_match_entry helper to reduce nesting depth
- egress_addon_core.py: make request_method/request_headers keyword-only
  to satisfy too-many-positional-arguments; wrap long lazy import lines
- egress_addon.py: remove unused Route import; add pylint disable for
  import-error on sidecar-only mitmproxy/egress_addon_core imports
- dlp_detectors.py: remove dead _min_distance function (superseded by
  _closest_pair)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 20:10:32 -04:00
didericis 8743299226 ci(prd): rename PRD to prd-new placeholder per new convention
lint / lint (push) Failing after 1m29s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 44s
2026-06-07 14:41:27 -04:00
didericis 205e94f960 docs(prd): renumber PRD 0053 → 0056 (0053 slot claimed by user-provider-plugins) 2026-06-07 14:41:27 -04:00
didericis 86b0a4d285 feat(egress): add location, context snippets, and token redaction to DLP logging
Each DLP block/warn now reports where the match was found (body,
authorization header, response body) and includes a context snippet:
SNIPPET_CONTEXT chars before and after the match, with the matched
value replaced by REDACT ("********").

scan_token_patterns/scan_known_secrets/scan_naive_injection all gain
`location` and `context` fields on their ScanResult returns. The
outbound scanner takes `auth_header` as a separate kwarg so the two
locations are scanned and reported independently.

redact_tokens() is added to dlp_detectors and used in egress_addon.py
to scrub token patterns and provisioned secrets from host/path fields
before they appear in any log output (level 1 and 2).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:41:27 -04:00
didericis 79212481c9 feat(egress): replace log bool with integer log levels (0/1/2)
Level 0 (off, default): no stderr output beyond boot line.
Level 1 (blocks): each block/warn emitted as JSON with reason and
request context (host, method, path, response_status for inbound).
Level 2 (full): level-1 events + egress_request and egress_response
JSON lines for every forwarded connection.

Block logging at level 1+ replaces the previous plain-text stderr write.
DLP warn logging is also gated on level 1+. All block call sites now pass
_req_ctx(flow) so the blocked request is visible in the log entry.
Boot message shows log level label (off/blocks/full).

Adds PRD 0053 documenting wire format, manifest format, and all log event
shapes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:41:27 -04:00
didericis 76dd153760 feat(egress): add global log option for full request/response logging
Adds a top-level `log: true` option to the egress config that logs the
full request (method, path, headers, body) and response (status, headers,
body) for every forwarded connection as JSON lines on stderr.

Wire format: `log: true` at the root of routes.yaml, parsed into the new
`Config` dataclass alongside `routes`. The sidecar addon switches from
`self.routes` to `self.config` and writes `_log_request` / `_log_response`
JSON lines when `self.config.log` is set.

Manifest: `egress.log: true` in bottle YAML flows through `EgressConfig.Log`
→ `Egress.prepare()` → `egress_render_routes(..., log=)` → routes.yaml.
`EgressPlan` also carries the flag for introspection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:41:27 -04:00
didericis b8d10abec9 fix(ci): scan working tree for prd-new files instead of HEAD~1..HEAD
The workflow was silently skipping prd-new-*.md files added in earlier
commits of a multi-commit PR. The final push commit is just the
implementation; the PRD rename to prd-new- happens in an earlier commit
on the branch, so git diff HEAD~1 HEAD never saw it.

Fix: glob the working tree for prd-new-*.md directly. Also switch the
non-PRD-changed check to use GITHUB_EVENT_BEFORE..HEAD so it covers the
full push range rather than just the last commit. Increase fetch-depth
to 0 so the before-SHA is always reachable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:40:02 -04:00
didericis 7ebddf7792 ci(prd): assign sequential numbers to new PRDs
prd-number / assign-numbers (push) Successful in 20s
prd-new-user-provider-plugins → 0053-user-provider-plugins
prd-new-named-labelled-agents → 0054-named-labelled-agents

Both PRDs ship with their implementations so Status flips Draft → Active.
Manual fix: the prd-number workflow did not fire on these merges.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 14:23:56 -04:00
didericis 04d7ca2e6a feat(agents): named and labelled agents with optional ANSI color
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 43s
lint / lint (push) Successful in 1m32s
prd-number / assign-numbers (push) Successful in 17s
test / unit (push) Successful in 29s
Update Quality Badges / update-badges (push) Successful in 1m18s
test / integration (push) Successful in 45s
Chunk 1 (schema + storage): BottleSpec, ActiveAgent, and BottleMetadata
gain label and color fields. Both docker and smolmachines backends
persist them to metadata.json on prepare and surface them in
enumerate_active_agents(). AgentProvider.provision_plan() passes
label/color through to the Claude provider, which injects them into
claude.json so claude-code displays the session name and color in its
header. Codex provider accepts and ignores the knobs.

Chunk 2 (curses modal + display): cmd_start presents a two-step curses
modal — first edit the label (first keystroke replaces the pre-fill),
then optionally pick a color. cli list active renders label with ANSI
escape codes when the terminal supports it, falling back to agent_name
when no label is set.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:12:32 -04:00
didericis f6f47c2f23 docs(prd): remove dashboard references, align with current codebase
- Dashboard no longer exists; remove all references to it
- Active agent display surface is cli list active, not a TUI pane
- Label/color rendered with ANSI escape codes in list output
- Modal called from cmd_start only, no supervisor _new_agent_flow
- Remove _format_agent_row/_color_pair_for curses design (list is
  plain text); add _ansi_color() helper design instead
- Clarify slug-suffix caveat: modal appears before prepare() mints
  the slug so default label falls back to agent_name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:04:17 -04:00
didericis 39e0976ace docs(prd): redesign label+color prompt as a curses modal window
lint / lint (push) Successful in 1m44s
- Single modal with two steps (label then color) instead of
  bare text prompts dropped to terminal
- Default label is <agent_name>-<slug_suffix>; first keystroke
  replaces the pre-fill rather than appending to it
- Color step shows a navigable list with live color preview;
  (none) selected by default; Esc skips
- Modal lives in tui.py and is shared between supervisor flow
  and cmd_start

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:01:11 -04:00
didericis 299579ab7b ci(prd): rename PRD to prd-new placeholder per new convention 2026-06-07 11:59:53 -04:00
didericis 3a10c38511 docs(prd): renumber PRD 0051 → 0054 (0051 slot taken by launch-selector on main) 2026-06-07 11:59:53 -04:00
didericis-claude db54f3d0b4 docs(prd): add PRD 0051 (named/labelled agents, renumbered from 0049) 2026-06-07 11:59:53 -04:00
Quality Badge Bot 8105e93031 chore: update quality badges
- Pylint: 9.93/10
- Pyright: 0 errors

[skip ci]
2026-06-07 15:57:03 +00:00
didericis 0d5c2f1a2e chore(ci): remove prd-check workflow
test / unit (pull_request) Successful in 28s
test / integration (pull_request) Successful in 42s
lint / lint (push) Successful in 1m42s
prd-number / assign-numbers (push) Successful in 23s
test / unit (push) Successful in 35s
test / integration (push) Successful in 52s
Update Quality Badges / update-badges (push) Successful in 1m40s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 11:43:42 -04:00
134 changed files with 9057 additions and 2274 deletions
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
- name: Run pylint - name: Run pylint
run: | run: |
# Run pylint on all Python files in the repo # Run pylint on all Python files in the repo
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0 || true find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0
- name: Run pyright - name: Run pyright
run: | run: |
-39
View File
@@ -1,39 +0,0 @@
# Block PRs that add prd-new-*.md files directly to main.
#
# prd-new-*.md files are placeholders — they must go through a PR so
# the post-merge prd-number workflow can assign a sequential number and
# rename the file. A direct push or a PR that slips through without
# triggering the check would leave an un-numbered PRD on main.
name: prd-check
on:
pull_request:
branches:
- main
paths:
- 'docs/prds/prd-new-*.md'
jobs:
no-prd-new-on-main:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Fail if prd-new-*.md files are present in the diff
run: |
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
new_prds=$(git diff --name-only --diff-filter=A "$base" "$head" \
| grep -E '^docs/prds/prd-new-.+\.md$' || true)
if [ -n "$new_prds" ]; then
echo "ERROR: PRs to main must not add prd-new-*.md files directly."
echo "These files must be merged via a feature branch so the"
echo "prd-number workflow can assign a sequential number on merge:"
echo "$new_prds"
exit 1
fi
echo "OK: no prd-new-*.md files added in this PR."
+18 -16
View File
@@ -4,12 +4,16 @@
# 1. Finds the next available NNNN number by scanning existing PRDs. # 1. Finds the next available NNNN number by scanning existing PRDs.
# 2. Renames each prd-new-*.md to NNNN-<slug>.md. # 2. Renames each prd-new-*.md to NNNN-<slug>.md.
# 3. Updates the title header (# PRD prd-new: → # PRD NNNN:). # 3. Updates the title header (# PRD prd-new: → # PRD NNNN:).
# 4. Flips Status: Draft → Active when the merge commit also touched # 4. Flips Status: Draft → Active when the push touched files outside
# files outside docs/prds/ (i.e. the implementation shipped together # docs/prds/ anywhere in its commit range (i.e. the implementation
# with the PRD). # shipped together with the PRD).
# 5. Commits the renaming back to main. # 5. Commits the renaming back to main.
# #
# No-op if the push contained no prd-new-*.md files. # No-op if the working tree contains no prd-new-*.md files.
#
# NOTE: The workflow scans the working tree (not just HEAD~1..HEAD) because
# PRs land as multi-commit pushes and the prd-new file is often added in an
# earlier commit on the branch, not in the final squash/merge commit.
name: prd-number name: prd-number
@@ -30,7 +34,7 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
fetch-depth: 2 fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Python - name: Set up Python
@@ -54,22 +58,20 @@ jobs:
prds_dir = Path("docs/prds") prds_dir = Path("docs/prds")
# Files added in the latest commit (HEAD vs HEAD~1). # Scan the working tree — prd-new files may have landed in any
result = subprocess.run( # commit of a multi-commit push, not just HEAD.
["git", "diff", "--name-only", "--diff-filter=A", "HEAD~1", "HEAD"], new_prds = sorted(prds_dir.glob("prd-new-*.md"))
capture_output=True, text=True, check=True,
)
added = [Path(p) for p in result.stdout.splitlines()]
new_prds = [p for p in added if p.parent == prds_dir
and re.match(r"prd-new-.+\.md$", p.name)]
if not new_prds: if not new_prds:
print("No prd-new-*.md files added in this commit — nothing to do.") print("No prd-new-*.md files found — nothing to do.")
sys.exit(0) sys.exit(0)
# Determine whether non-PRD files were also changed (for Status flip). # Determine whether non-PRD files were also changed anywhere in
# the push range (BEFORE_SHA → HEAD). Falls back to HEAD~1 when
# the env var isn't set (e.g. local act runs).
before_sha = os.environ.get("GITHUB_EVENT_BEFORE", "HEAD~1")
all_changed = subprocess.run( all_changed = subprocess.run(
["git", "diff", "--name-only", "HEAD~1", "HEAD"], ["git", "diff", "--name-only", before_sha, "HEAD"],
capture_output=True, text=True, check=True, capture_output=True, text=True, check=True,
).stdout.splitlines() ).stdout.splitlines()
non_prd_changed = any( non_prd_changed = any(
+13 -6
View File
@@ -2,11 +2,18 @@
## What this is ## What this is
bot-bottle spins up an isolated container for running AI coding agents with a bot-bottle spins up an isolated backend runtime for running AI coding agents
curated set of skills and env vars. The point is to run agents with broad with a curated set of skills and env vars. The point is to run agents with
permissions inside a sandbox, so a misbehaving agent cannot reach the host. broad permissions inside a sandbox, so a misbehaving agent cannot reach the
A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the container lifecycle and the copying of skills and env vars into it. the runtime lifecycle and the copying of skills and env vars into it.
The default backend on compatible macOS hosts is macos-container:
agents and sidecar bundles run through Apple's `container` CLI without
requiring Docker. The smolmachines backend remains available with
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
or `--backend=docker`.
## Goals ## Goals
@@ -17,7 +24,7 @@ the container lifecycle and the copying of skills and env vars into it.
## Non-goals ## Non-goals
- Communicating between agents directly - Communicating between agents directly
- Self hosted VMs (v1 uses local Docker containers, not VMs) - Removing the Docker backend
- Advanced agent auditing (lean on git history for auditing) - Advanced agent auditing (lean on git history for auditing)
## Repository layout ## Repository layout
+41 -7
View File
@@ -5,7 +5,7 @@
# bot-bottle # bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml) [![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![pylint](https://img.shields.io/badge/pylint-9.92%2F10-brightgreen)](https://github.com/PyCQA/pylint) [![pylint](https://img.shields.io/badge/pylint-9.93%2F10-brightgreen)](https://github.com/PyCQA/pylint)
[![pyright](https://img.shields.io/badge/pyright-0%20errors-brightgreen)](https://github.com/microsoft/pyright) [![pyright](https://img.shields.io/badge/pyright-0%20errors-brightgreen)](https://github.com/microsoft/pyright)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data. **Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -14,20 +14,28 @@
## Features ## Features
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default. - **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only. - **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential. - **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load. - **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host. - **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top. - **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other. - **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding. - **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required. - **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker. - **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
## Architecture ## Architecture
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles egress + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box. On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
The Docker topology looks like this:
``` ```
host ( ./cli.py ) host ( ./cli.py )
@@ -62,7 +70,9 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
## Quickstart ## Quickstart
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
```sh ```sh
./cli.py start <agent> # builds the image on first run, drops you into claude ./cli.py start <agent> # builds the image on first run, drops you into claude
@@ -96,8 +106,15 @@ egress:
routes: routes:
- host: gitea.dideric.is - host: gitea.dideric.is
auth: auth:
scheme: token scheme: token # Bearer | token
token_ref: BOT_BOTTLE_GITEA_TOKEN token_ref: BOT_BOTTLE_GITEA_TOKEN
matches: # optional — restrict to specific paths/methods/headers
- paths:
- {type: prefix, value: /api/v1/}
methods: [GET, POST, PATCH, DELETE]
dlp: # optional — per-route detector overrides (default: all on)
outbound_detectors: [token_patterns, known_secrets]
inbound_detectors: false # disable response scanning for this host
--- ---
The `gitea-dev` bottle. Provider auth via the inherited Claude route; The `gitea-dev` bottle. Provider auth via the inherited Claude route;
@@ -116,6 +133,23 @@ skills:
You help maintain Gitea-hosted projects. You help maintain Gitea-hosted projects.
```` ````
**Egress route fields:**
| Field | Required | Description |
|---|---|---|
| `host` | yes | Hostname to allowlist. One entry per host. |
| `role` | no | Provider-specific role string (e.g. `claude_code_oauth`). Wires built-in auth flows; set by provider templates, not manually. |
| `auth.scheme` | when `auth` present | `Bearer` or `token`. Injected by the proxy; the agent never sees the value. |
| `auth.token_ref` | when `auth` present | Env-var name holding the secret on the host. |
| `matches` | no | Array of `{paths, methods, headers}` filters. A request must match at least one entry (if any are given) to be forwarded. |
| `matches[].paths` | no | Array of `{type, value}`. `type` is `prefix` (default), `exact`, or `regex`. |
| `matches[].methods` | no | Array of HTTP method strings, e.g. `[GET, POST]`. |
| `matches[].headers` | no | Array of `{name, value, type}`. `type` is `exact` (default) or `regex`. |
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`. More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
## Trademarks ## Trademarks
+57 -23
View File
@@ -20,6 +20,7 @@ Per PRD 0050 the per-provider implementations live under
from __future__ import annotations from __future__ import annotations
import importlib.util import importlib.util
import inspect
import os import os
import shlex import shlex
import tempfile import tempfile
@@ -37,13 +38,19 @@ if TYPE_CHECKING:
PROVIDER_CLAUDE = "claude" PROVIDER_CLAUDE = "claude"
PROVIDER_CODEX = "codex" PROVIDER_CODEX = "codex"
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX}) PROVIDER_PI = "pi"
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_PI})
# Hosts that egress injects the host ChatGPT bearer on when Codex # Hosts that egress injects the host ChatGPT bearer on when Codex
# forward_host_credentials is enabled. Pipelock must pass these through # forward_host_credentials is enabled. Pipelock must pass these through
# (no TLS MITM) or its header DLP blocks the injected JWT. # (no TLS MITM) or its header DLP blocks the injected JWT.
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com") CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
PromptMode = Literal["append_file", "read_prompt_file"] PromptMode = Literal[
"append_file",
"read_prompt_file",
"print_read_prompt_file",
"append_system_prompt",
]
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -51,7 +58,6 @@ class AgentProviderRuntime:
template: str template: str
command: str command: str
image: str image: str
dockerfile: str
prompt_mode: PromptMode prompt_mode: PromptMode
bypass_args: tuple[str, ...] bypass_args: tuple[str, ...]
resume_args: tuple[str, ...] resume_args: tuple[str, ...]
@@ -103,7 +109,12 @@ class AgentProvisionPlan:
prompt_mode: PromptMode prompt_mode: PromptMode
image: str image: str
dockerfile: str dockerfile: str
guest_home: str
instance_name: str
prompt_file: Path
guest_env: dict[str, str] guest_env: dict[str, str]
has_prompt: bool = False
startup_args: tuple[str, ...] = ()
env_vars: dict[str, str] = field(default_factory=dict) env_vars: dict[str, str] = field(default_factory=dict)
dirs: tuple[AgentProvisionDir, ...] = () dirs: tuple[AgentProvisionDir, ...] = ()
files: tuple[AgentProvisionFile, ...] = () files: tuple[AgentProvisionFile, ...] = ()
@@ -127,18 +138,39 @@ class AgentProvider(ABC):
"""The static command / image / prompt-mode table for this """The static command / image / prompt-mode table for this
template.""" template."""
@property
def guest_home(self) -> str:
"""In-guest home directory for the agent user. Defaults to
`/home/node` to match the Debian-based bot-bottle-* images
(USER node). Override for plugins whose image runs as a
different user."""
return "/home/node"
@property
def dockerfile(self) -> Path:
"""Path to the provider's Dockerfile.
Default: the `Dockerfile` file next to this provider's
`agent_provider.py` module. Override to point at a non-standard
path."""
return Path(inspect.getfile(type(self))).parent / "Dockerfile"
@abstractmethod @abstractmethod
def provision_plan( def provision_plan(
self, self,
*, *,
dockerfile: str, dockerfile: str,
state_dir: Path, state_dir: Path,
guest_home: str, instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None, guest_env: dict[str, str] | None = None,
auth_token: str = "", auth_token: str = "",
forward_host_credentials: bool = False, forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None, host_env: dict[str, str] | None = None,
trusted_project_path: str = "", trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan: ) -> AgentProvisionPlan:
"""Build the declarative AgentProvisionPlan for one launch. """Build the declarative AgentProvisionPlan for one launch.
Backends call this during `prepare` and consume the result as Backends call this during `prepare` and consume the result as
@@ -203,23 +235,10 @@ class AgentProvider(ABC):
def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None: def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None:
"""Configure git inside the agent container. """Configure git inside the agent container.
Default: Debian/node — copies .git when --cwd is set, writes the Default: Debian/node — writes the git-gate insteadOf gitconfig
git-gate insteadOf gitconfig, sets user.name/email as node. and sets user.name/email as node. Workspace copy runs through
Override for images that run as a different user or use a BottleBackend.provision_workspace against the running bottle."""
non-standard home directory."""
from .log import info from .log import info
workspace = plan.workspace_plan
if workspace.enabled and workspace.copy_git and workspace.has_host_git_dir:
guest_workspace_git = f"{workspace.guest_path}/.git"
host_git = str(workspace.host_path / ".git")
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
bottle.cp_in(host_git, guest_workspace_git)
bottle.exec(
f"chown -R {shlex.quote(workspace.owner)} "
f"{shlex.quote(guest_workspace_git)}",
user="root",
)
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name) manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
if manifest_bottle.git: if manifest_bottle.git:
@@ -308,6 +327,9 @@ def get_provider(template: str) -> AgentProvider:
if template == PROVIDER_CODEX: if template == PROVIDER_CODEX:
from .contrib.codex.agent_provider import CodexAgentProvider from .contrib.codex.agent_provider import CodexAgentProvider
return CodexAgentProvider() return CodexAgentProvider()
if template == PROVIDER_PI:
from .contrib.pi.agent_provider import PiAgentProvider
return PiAgentProvider()
raise ValueError(f"unknown agent provider template: {template!r}") raise ValueError(f"unknown agent provider template: {template!r}")
@@ -315,29 +337,37 @@ def runtime_for(template: str) -> AgentProviderRuntime:
return get_provider(template).runtime return get_provider(template).runtime
def agent_provision_plan( def build_agent_provision_plan(
*, *,
template: str, template: str,
dockerfile: str, dockerfile: str,
state_dir: Path, state_dir: Path,
guest_home: str, instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None, guest_env: dict[str, str] | None = None,
auth_token: str = "", auth_token: str = "",
forward_host_credentials: bool = False, forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None, host_env: dict[str, str] | None = None,
trusted_project_path: str = "", trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan: ) -> AgentProvisionPlan:
"""Back-compat shim — `prepare` callers stay the same; the work """Back-compat shim — `prepare` callers stay the same; the work
now lives on the provider plugin.""" now lives on the provider plugin."""
return get_provider(template).provision_plan( return get_provider(template).provision_plan(
dockerfile=dockerfile, dockerfile=dockerfile,
state_dir=state_dir, state_dir=state_dir,
guest_home=guest_home, instance_name=instance_name,
prompt_file=prompt_file,
guest_env=guest_env, guest_env=guest_env,
auth_token=auth_token, auth_token=auth_token,
forward_host_credentials=forward_host_credentials, forward_host_credentials=forward_host_credentials,
host_env=host_env, host_env=host_env,
trusted_project_path=trusted_project_path, trusted_project_path=trusted_project_path,
label=label,
color=color,
provider_settings=provider_settings,
) )
@@ -355,4 +385,8 @@ def prompt_args(
if argv and "resume" in argv: if argv and "resume" in argv:
return [] return []
return [f"Read and follow the instructions in {prompt_path}."] return [f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "print_read_prompt_file":
return ["-p", f"Read and follow the instructions in {prompt_path}."]
if prompt_mode == "append_system_prompt":
return ["--append-system-prompt", prompt_path]
raise ValueError(f"unknown provider prompt mode: {prompt_mode}") raise ValueError(f"unknown provider prompt mode: {prompt_mode}")
+147 -19
View File
@@ -24,14 +24,16 @@ backend exposes five methods:
enough metadata for callers (CLI `list active`, dashboard enough metadata for callers (CLI `list active`, dashboard
agents pane) to render a row. agents pane) to render a row.
Selection is driven by `--backend` on `start` or Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
BOT_BOTTLE_BACKEND (env var; default "docker"). Per PRD 0003 the (env var). When neither is set, compatible macOS hosts default to
manifest does not carry a backend field; the host picks. `macos-container`; other hosts default to `smolmachines`. Per PRD 0003
the manifest does not carry a backend field; the host picks.
""" """
from __future__ import annotations from __future__ import annotations
import os import os
import shlex
import sys import sys
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from contextlib import AbstractContextManager from contextlib import AbstractContextManager
@@ -39,14 +41,15 @@ from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any, Generic, Sequence, TypeVar from typing import Any, Generic, Sequence, TypeVar
from ..agent_provider import AgentProvisionPlan, get_provider from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
from ..egress import EgressPlan from ..egress import EgressPlan
from ..git_gate import GitGatePlan from ..git_gate import GitGatePlan
from ..log import die, info from ..log import die, info
from ..manifest import GitEntry, Manifest from ..manifest import ManifestGitEntry, Manifest
from ..supervise import SupervisePlan from ..supervise import SupervisePlan
from ..util import expand_tilde from ..util import expand_tilde
from ..workspace import WorkspacePlan from ..env import resolve_env, ResolvedEnv
from ..workspace import WorkspacePlan, workspace_plan
from .print_util import print_multi, visible_agent_env_names from .print_util import print_multi, visible_agent_env_names
from .util import host_skill_dir from .util import host_skill_dir
@@ -67,6 +70,8 @@ class BottleSpec:
# (`cli.py resume <identity>`) sets this to continue an existing # (`cli.py resume <identity>`) sets this to continue an existing
# bottle's state. Empty string for a fresh `start`. # bottle's state. Empty string for a fresh `start`.
identity: str = "" identity: str = ""
label: str = ""
color: str = ""
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -76,9 +81,12 @@ class BottlePlan(ABC):
spec: BottleSpec spec: BottleSpec
stage_dir: Path stage_dir: Path
guest_home: str
git_gate_plan: GitGatePlan git_gate_plan: GitGatePlan
@property
def guest_home(self) -> str:
return self.agent_provision.guest_home
@property @property
def git_gate_insteadof_host(self) -> str: def git_gate_insteadof_host(self) -> str:
"""Host (and optional port) used in git-gate insteadOf URLs. """Host (and optional port) used in git-gate insteadOf URLs.
@@ -95,7 +103,10 @@ class BottlePlan(ABC):
egress_plan: EgressPlan egress_plan: EgressPlan
supervise_plan: SupervisePlan | None supervise_plan: SupervisePlan | None
agent_provision: AgentProvisionPlan agent_provision: AgentProvisionPlan
workspace_plan: WorkspacePlan
@property
def workspace_plan(self) -> WorkspacePlan:
return workspace_plan(self.spec, guest_home=self.guest_home)
def print(self, *, remote_control: bool) -> None: def print(self, *, remote_control: bool) -> None:
"""Render the y/N preflight summary to stderr.""" """Render the y/N preflight summary to stderr."""
@@ -180,7 +191,7 @@ class ActiveAgent:
of sidecar daemons currently up for this bottle (`egress`, of sidecar daemons currently up for this bottle (`egress`,
`git-gate`, `supervise`); the dashboard uses it to `git-gate`, `supervise`); the dashboard uses it to
gate edit verbs. `backend_name` is the matching key in gate edit verbs. `backend_name` is the matching key in
`_BACKENDS` (`docker` / `smolmachines`) — used by the active- `_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
list rendering to disambiguate and by the dashboard's list rendering to disambiguate and by the dashboard's
re-attach path.""" re-attach path."""
@@ -189,6 +200,8 @@ class ActiveAgent:
agent_name: str # from metadata.json; "?" if missing agent_name: str # from metadata.json; "?" if missing
started_at: str # ISO 8601 from metadata.json; "" if missing started_at: str # ISO 8601 from metadata.json; "" if missing
services: tuple[str, ...] # alphabetical services: tuple[str, ...] # alphabetical
label: str = ""
color: str = ""
class Bottle(ABC): class Bottle(ABC):
@@ -259,14 +272,88 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
name: str name: str
def prepare(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT: def prepare(self, spec: BottleSpec, stage_dir: Path) -> PlanT:
"""Template method: run cross-backend host-side validation, then """Template method: run cross-backend host-side validation, then
delegate to the subclass's `_resolve_plan` for the delegate to the subclass's `_resolve_plan` for the
backend-specific resolution (names, scratch files, etc.). The backend-specific resolution (names, scratch files, etc.). The
validation step is enforced here so a future backend cannot validation step is enforced here so a future backend cannot
accidentally skip it. No remote/runtime resources are created.""" accidentally skip it. No remote/runtime resources are created."""
from .resolve_common import (
merge_provision_env_vars,
mint_slug,
prepare_agent_state_dir,
prepare_egress,
prepare_git_gate,
prepare_supervise,
resolve_manifest_dockerfile,
write_launch_metadata,
)
self._validate(spec) self._validate(spec)
return self._resolve_plan(spec, stage_dir=stage_dir)
self._preflight()
manifest = spec.manifest
manifest_bottle = manifest.bottle_for(spec.agent_name)
manifest_agent_provider = manifest_bottle.agent_provider
agent_provider = get_provider(manifest_agent_provider.template)
resolved_env = resolve_env(manifest, spec.agent_name)
workspace = workspace_plan(spec, guest_home=agent_provider.guest_home)
slug = mint_slug(spec)
write_launch_metadata(slug, spec, compose_project="", backend=self.name)
# Manifest may override the Dockerfile per-bottle; otherwise fall
# back to the provider plugin's bundled Dockerfile (next to its
# agent_provider.py module).
if manifest_agent_provider.dockerfile:
agent_dockerfile_path = resolve_manifest_dockerfile(
manifest_agent_provider.dockerfile, spec,
)
else:
agent_dockerfile_path = str(agent_provider.dockerfile)
agent_dir, prompt_file = prepare_agent_state_dir(slug, spec)
agent_provision_plan = build_agent_provision_plan(
template=manifest_agent_provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
instance_name=f"bot-bottle-{slug}",
prompt_file=prompt_file,
guest_env=self._build_guest_env(resolved_env),
forward_host_credentials=manifest_agent_provider.forward_host_credentials,
auth_token=manifest_agent_provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace.workdir,
label=spec.label,
color=spec.color,
provider_settings=manifest_agent_provider.settings,
)
agent_provision_plan = merge_provision_env_vars(agent_provision_plan)
egress_plan = prepare_egress(manifest_bottle, slug, agent_provision_plan)
supervise_plan = prepare_supervise(manifest_bottle, slug)
git_gate_plan = prepare_git_gate(manifest_bottle, slug)
return self._resolve_plan(
spec,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return {}
def _preflight(self) -> None:
"""
tasks to do before resolving a plan
"""
pass
def _validate(self, spec: BottleSpec) -> None: def _validate(self, spec: BottleSpec) -> None:
"""Cross-backend pre-launch checks. Confirms the agent exists, """Cross-backend pre-launch checks. Confirms the agent exists,
@@ -293,7 +380,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
f"Create it under ~/.claude/skills/, then re-run." f"Create it under ~/.claude/skills/, then re-run."
) )
def _validate_git_entries(self, entries: Sequence[GitEntry]) -> None: def _validate_git_entries(self, entries: Sequence[ManifestGitEntry]) -> None:
"""Each entry's IdentityFile must exist on the host (after """Each entry's IdentityFile must exist on the host (after
expanding leading ~) — the git-gate copies it in at start time expanding leading ~) — the git-gate copies it in at start time
to authenticate the upstream push (PRD 0008). Shape is already to authenticate the upstream push (PRD 0008). Shape is already
@@ -318,10 +405,21 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
) )
@abstractmethod @abstractmethod
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT: def _resolve_plan(self,
spec: BottleSpec,
*,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path) -> PlanT:
"""Backend-specific plan resolution: image/container names, """Backend-specific plan resolution: image/container names,
env-file, prompt-file, proxy plan, runtime detection. Called by env-file, prompt-file, proxy plan, runtime detection. Called by
`prepare` after `_validate` succeeds.""" `prepare` after `_validate` succeeds. Instance name, image,
prompt file, Dockerfile path, and guest home all live on
`agent_provision_plan` — the source of truth."""
@abstractmethod @abstractmethod
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]: def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
@@ -365,9 +463,30 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
return prompt_path return prompt_path
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None: def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
"""Copy the operator workspace into the running bottle when """Copy the operator workspace into the running bottle.
the backend cannot bake it into the agent image. Default is
no-op for backends like Docker that handle this before launch.""" This is the only supported workspace-provisioning path: Docker
does not build a derived image containing the current
workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path = shlex.quote(workspace.guest_path)
guest_parent = shlex.quote(guest_parent)
owner = shlex.quote(workspace.owner)
mode = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
bottle.exec(
f"rm -rf {guest_path} && mkdir -p {guest_parent}",
user="root",
)
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
bottle.exec(
f"chown -R {owner} {guest_path} && chmod {mode} {guest_path}",
user="root",
)
def supervise_mcp_url(self, plan: PlanT) -> str: def supervise_mcp_url(self, plan: PlanT) -> str:
"""Return the agent-side URL of the per-bottle supervise """Return the agent-side URL of the per-bottle supervise
@@ -412,6 +531,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
# each backend module can pull BottleSpec / BottlePlan / BottleBackend # each backend module can pull BottleSpec / BottlePlan / BottleBackend
# via `from . import ...` without hitting a partially-initialized module. # via `from . import ...` without hitting a partially-initialized module.
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
@@ -421,6 +541,7 @@ from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: dis
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.). # unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
_BACKENDS: dict[str, BottleBackend[Any, Any]] = { _BACKENDS: dict[str, BottleBackend[Any, Any]] = {
"docker": DockerBottleBackend(), "docker": DockerBottleBackend(),
"macos-container": MacosContainerBottleBackend(),
"smolmachines": SmolmachinesBottleBackend(), "smolmachines": SmolmachinesBottleBackend(),
} }
@@ -433,17 +554,24 @@ def get_bottle_backend(
`name` precedence: `name` precedence:
1. explicit arg (CLI `--backend=<name>` passes through here) 1. explicit arg (CLI `--backend=<name>` passes through here)
2. BOT_BOTTLE_BACKEND env var 2. BOT_BOTTLE_BACKEND env var
3. default `docker` 3. `macos-container` on compatible macOS hosts
4. default `smolmachines`
Dies with a pointer at the known backends if the chosen name Dies with a pointer at the known backends if the chosen name
isn't implemented.""" isn't implemented."""
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker" resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or _default_backend_name()
if resolved not in _BACKENDS: if resolved not in _BACKENDS:
known = ", ".join(sorted(_BACKENDS)) known = ", ".join(sorted(_BACKENDS))
die(f"unknown backend {resolved!r}; known backends: {known}") die(f"unknown backend {resolved!r}; known backends: {known}")
return _BACKENDS[resolved] return _BACKENDS[resolved]
def _default_backend_name() -> str:
if has_backend("macos-container"):
return "macos-container"
return "smolmachines"
def known_backend_names() -> tuple[str, ...]: def known_backend_names() -> tuple[str, ...]:
"""Sorted tuple of all backend keys in `_BACKENDS`. Used by """Sorted tuple of all backend keys in `_BACKENDS`. Used by
argparse (`--backend` choices) and the dashboard's backend argparse (`--backend` choices) and the dashboard's backend
+39 -8
View File
@@ -2,10 +2,10 @@
This module is a thin façade. The real work lives in four siblings: This module is a thin façade. The real work lives in four siblings:
- prepare.py — host-side resolution into a DockerBottlePlan - resolve_plan.py — Docker-specific resolution into a DockerBottlePlan
- launch.py — bring-up + teardown context manager - launch.py — bring-up + teardown context manager
- cleanup.py — orphan enumeration + removal - cleanup.py — orphan enumeration + removal
- enumerate.py — active-agent listing - enumerate.py — active-agent listing
The base class's `prepare` template runs cross-backend host-side The base class's `prepare` template runs cross-backend host-side
validation before calling `_resolve_plan` here. validation before calling `_resolve_plan` here.
@@ -25,17 +25,22 @@ from pathlib import Path
from typing import Generator, Sequence from typing import Generator, Sequence
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from .. import ActiveAgent, BottleBackend, BottleSpec from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup from . import cleanup as _cleanup
from . import enumerate as _enumerate from . import enumerate as _enumerate
from . import launch as _launch from . import launch as _launch
from . import prepare as _prepare from . import resolve_plan as _resolve_plan
from .bottle import DockerBottle from .bottle import DockerBottle
from .bottle_cleanup_plan import DockerBottleCleanupPlan from .bottle_cleanup_plan import DockerBottleCleanupPlan
from .bottle_plan import DockerBottlePlan from .bottle_plan import DockerBottlePlan
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]): class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND """Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
(default).""" when set to `docker`; retained as a legacy/example backend."""
name = "docker" name = "docker"
@@ -48,8 +53,34 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
launch.""" launch."""
return shutil.which("docker") is not None return shutil.which("docker") is not None
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> DockerBottlePlan: def _preflight(self) -> None:
return _prepare.resolve_plan(spec, stage_dir=stage_dir) _resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> DockerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager @contextmanager
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]: def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
+16 -6
View File
@@ -9,6 +9,7 @@ from typing import cast
from ...agent_provider import PromptMode, prompt_args from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
class DockerBottle(Bottle): class DockerBottle(Bottle):
@@ -22,15 +23,20 @@ class DockerBottle(Bottle):
*, *,
agent_command: str = "claude", agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file", agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
): ):
self.name = container self.name = container
self._teardown = teardown self._teardown = teardown
self.prompt_path = prompt_path_in_container self.prompt_path = prompt_path_in_container
self._agent_prompt_mode = agent_prompt_mode self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command self.agent_command = agent_command
self.agent_provider_template = ( self.terminal_title = terminal_title
"codex" if agent_command == "codex" else "claude" self.terminal_color = terminal_color
) self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self._closed = False self._closed = False
def agent_argv( def agent_argv(
@@ -43,13 +49,17 @@ class DockerBottle(Bottle):
cmd = ["docker", "exec"] cmd = ["docker", "exec"]
if tty: if tty:
cmd.append("-it") cmd.append("-it")
if self.agent_workdir and self.agent_workdir != "/home/node":
cmd.extend(["-w", self.agent_workdir])
cmd.extend([self.name, self.agent_command, *full_argv]) cmd.extend([self.name, self.agent_command, *full_argv])
return cmd return cmd
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int: def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
return subprocess.run( agent_argv = self.agent_argv(argv, tty=tty)
self.agent_argv(argv, tty=tty), check=False, script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
).returncode if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult: def exec(self, script: str, *, user: str = "node") -> ExecResult:
# Pipe via stdin to `sh -s` so the caller never has to worry # Pipe via stdin to `sh -s` so the caller never has to worry
+19 -12
View File
@@ -22,25 +22,32 @@ class DockerBottlePlan(BottlePlan):
`agent_provision` from BottlePlan.""" `agent_provision` from BottlePlan."""
slug: str slug: str
container_name: str
container_name_pinned: bool
image: str
derived_image: str # "" -> no derived image
runtime_image: str # image actually launched (derived or base)
# Absolute path to the Dockerfile that builds `image`. Empty means
# use the repo's default Dockerfile. Populated to a per-bottle
# state file (~/.bot-bottle/state/<slug>/Dockerfile) after a
# capability-block remediation (PRD 0016).
dockerfile_path: str
env_file: Path # docker --env-file: NAME=VALUE literals
# name -> value for vars forwarded into the docker-run child process # name -> value for vars forwarded into the docker-run child process
# via subprocess env (so values never land on argv or in a file). # via subprocess env (so values never land on argv or in a file).
# repr=False keeps secret/interpolated/OAuth values out of any # repr=False keeps secret/interpolated/OAuth values out of any
# accidental log of the plan dataclass. # accidental log of the plan dataclass.
forwarded_env: dict[str, str] = field(repr=False) forwarded_env: dict[str, str] = field(repr=False)
prompt_file: Path
use_runsc: bool use_runsc: bool
@property
def container_name(self) -> str:
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
"""Absolute path to the Dockerfile that builds `image`. Sourced
from the agent provision plan — the manifest may override per
bottle; otherwise the provider plugin's bundled Dockerfile."""
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property @property
def agent_command(self) -> str: def agent_command(self) -> str:
return self.agent_provision.command return self.agent_provision.command
+4 -11
View File
@@ -32,10 +32,10 @@ from __future__ import annotations
import shutil import shutil
import subprocess import subprocess
from pathlib import Path
from ...agent_provider import get_provider
from ...log import info, warn from ...log import info, warn
from .bottle_state import ( from ...bottle_state import (
mark_preserved, mark_preserved,
per_bottle_dockerfile, per_bottle_dockerfile,
transcript_snapshot_dir, transcript_snapshot_dir,
@@ -93,11 +93,11 @@ def fetch_current_dockerfile(slug: str) -> str:
override = per_bottle_dockerfile(slug) override = per_bottle_dockerfile(slug)
if override is not None: if override is not None:
return override return override
repo_dockerfile = _repo_dockerfile_path() repo_dockerfile = get_provider("claude").dockerfile
if repo_dockerfile.is_file(): if repo_dockerfile.is_file():
return repo_dockerfile.read_text() return repo_dockerfile.read_text()
raise CapabilityApplyError( raise CapabilityApplyError(
f"no per-bottle Dockerfile for {slug} and no repo Dockerfile at " f"no per-bottle Dockerfile for {slug} and no provider Dockerfile at "
f"{repo_dockerfile}" f"{repo_dockerfile}"
) )
@@ -125,13 +125,6 @@ def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
# --- Internals ------------------------------------------------------------- # --- Internals -------------------------------------------------------------
def _repo_dockerfile_path() -> Path:
"""Path to the repo's Claude Dockerfile (one dir above this module's
package root). Resolved at call time so the path is correct
regardless of where this module is imported from."""
# bot_bottle/backend/docker/capability_apply.py -> repo root
return Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
def snapshot_transcript(slug: str) -> None: def snapshot_transcript(slug: str) -> None:
"""`docker cp` /home/node/.claude out of the agent container into """`docker cp` /home/node/.claude out of the agent container into
+1 -1
View File
@@ -31,7 +31,7 @@ from ... import supervise as _supervise
from ...log import info, warn from ...log import info, warn
from . import util as docker_mod from . import util as docker_mod
from .bottle_cleanup_plan import DockerBottleCleanupPlan from .bottle_cleanup_plan import DockerBottleCleanupPlan
from .bottle_state import bottle_state_dir, is_preserved from ...bottle_state import bottle_state_dir, is_preserved
from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
+1 -3
View File
@@ -222,7 +222,7 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
env.append(name) env.append(name)
service: dict[str, Any] = { service: dict[str, Any] = {
"image": plan.runtime_image, "image": plan.image,
"container_name": plan.container_name, "container_name": plan.container_name,
"command": ["sleep", "infinity"], "command": ["sleep", "infinity"],
"networks": {"internal": None}, "networks": {"internal": None},
@@ -230,8 +230,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
} }
if plan.use_runsc: if plan.use_runsc:
service["runtime"] = "runsc" service["runtime"] = "runsc"
if plan.env_file and plan.env_file.exists() and plan.env_file.stat().st_size > 0:
service["env_file"] = [str(plan.env_file)]
volumes: list[dict[str, Any]] = [] volumes: list[dict[str, Any]] = []
if plan.supervise_plan is not None: if plan.supervise_plan is not None:
+3 -1
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import subprocess import subprocess
from .. import ActiveAgent from .. import ActiveAgent
from .bottle_state import read_metadata from ...bottle_state import read_metadata
from .compose import compose_project_name, list_active_slugs from .compose import compose_project_name, list_active_slugs
@@ -39,6 +39,8 @@ def enumerate_active() -> list[ActiveAgent]:
agent_name=metadata.agent_name if metadata else "?", agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "", started_at=metadata.started_at if metadata else "",
services=tuple(sorted(services)), services=tuple(sorted(services)),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
)) ))
return out return out
+9 -9
View File
@@ -4,8 +4,8 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
The flow is: The flow is:
1. Build the agent's base + derived image (compose builds the 1. Build the agent image from the provider Dockerfile (compose
sidecar images via the `build:` directive on first up). builds the sidecar images via the `build:` directive on first up).
2. Mint the per-bottle egress CA (chunk 2 writes it under 2. Mint the per-bottle egress CA (chunk 2 writes it under
state/<slug>/egress/). state/<slug>/egress/).
3. Populate the inner plans with launch-time fields so the 3. Populate the inner plans with launch-time fields so the
@@ -15,8 +15,8 @@ The flow is:
7. `docker compose up -d` (token + OAuth values flow into the 7. `docker compose up -d` (token + OAuth values flow into the
compose subprocess env so `environment: [NAME]` bare-name compose subprocess env so `environment: [NAME]` bare-name
entries inherit without rendering values into the file). entries inherit without rendering values into the file).
8. Provision (CA install, prompt copy, skills, git, supervise 8. Provision (CA install, prompt copy, skills, workspace, git,
config) — unchanged, uses `docker exec`. supervise config) — unchanged, uses `docker exec` / `docker cp`.
9. Yield a DockerBottle handle. `exec_agent` runs claude via 9. Yield a DockerBottle handle. `exec_agent` runs claude via
`docker exec -it` exactly like the pre-compose world. `docker exec -it` exactly like the pre-compose world.
@@ -43,7 +43,7 @@ from . import network as network_mod
from . import util as docker_mod from . import util as docker_mod
from .bottle import DockerBottle from .bottle import DockerBottle
from .bottle_plan import DockerBottlePlan from .bottle_plan import DockerBottlePlan
from .bottle_state import ( from ...bottle_state import (
bottle_state_dir, bottle_state_dir,
egress_state_dir, egress_state_dir,
git_gate_state_dir, git_gate_state_dir,
@@ -97,10 +97,6 @@ def launch(
plan.image, _REPO_DIR, plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path, dockerfile=plan.dockerfile_path,
) )
if plan.derived_image:
docker_mod.build_image_with_cwd(
plan.derived_image, plan.image, plan.workspace_plan
)
internal_network = network_mod.network_name_for_slug(plan.slug) internal_network = network_mod.network_name_for_slug(plan.slug)
egress_network = network_mod.network_egress_name_for_slug(plan.slug) egress_network = network_mod.network_egress_name_for_slug(plan.slug)
@@ -179,6 +175,10 @@ def launch(
None, None,
agent_command=plan.agent_command, agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode, agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=plan.spec.label or plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
) )
bottle.prompt_path = provision(plan, bottle) bottle.prompt_path = provision(plan, bottle)
-276
View File
@@ -1,276 +0,0 @@
"""Prepare step for the Docker bottle backend.
`resolve_plan` does all host-side resolution (image and container
names, env-file, prompt-file, proxy plan, runtime detection) and
returns a frozen DockerBottlePlan. No Docker resources are created;
the only side effects are scratch files under `stage_dir` and a probe
of `docker info`. Cross-backend host-side validation has already run
via the base class's `prepare` template before this is called.
"""
from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import PROVIDER_TEMPLATES, agent_provision_plan, runtime_for
from ...egress import Egress
from ...env import ResolvedEnv, resolve_env
from ...git_gate import GitGate
from ...log import die
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .. import BottleSpec
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
from .bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
clear_preserve_marker,
egress_state_dir,
git_gate_state_dir,
per_bottle_dockerfile,
per_bottle_dockerfile_path,
per_bottle_image_tag,
supervise_state_dir,
write_metadata,
)
from .sidecar_bundle import sidecar_bundle_container_name
def resolve_plan(
spec: BottleSpec,
*,
stage_dir: Path,
) -> DockerBottlePlan:
"""Resolve Docker-specific names and write scratch files. Trusts
that the agent and its skills/git-gate keys are present —
validation already ran in the base class."""
docker_mod.require_docker()
git_gate = GitGate()
egress = Egress()
supervise = Supervise()
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = "/home/node"
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
# mints a random-suffixed identity (so parallel runs of the same
# agent in the same cwd don't collide on container/network
# names); a `resume` passes the recorded identity in via
# spec.identity to continue an existing bottle's state.
slug = spec.identity or bottle_identity(spec.agent_name)
# Record the launch metadata so `cli.py resume <identity>` can
# reconstruct the spec. Idempotent — re-writes on resume with a
# refreshed started_at.
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=f"bot-bottle-{slug}",
backend="docker",
))
# Clear any leftover preserve marker from a prior capability-block
# so this fresh launch can be cleaned up at session-end unless
# the agent triggers another capability-block.
clear_preserve_marker(slug)
# PRD 0016 capability-block: if a per-bottle Dockerfile has been
# written (via apply_capability_change), the base image becomes
# per_bottle_image_tag(slug) built from that file. --cwd still
# layers a derived image on top.
dockerfile_path = ""
if per_bottle_dockerfile(slug) is not None:
image_default = per_bottle_image_tag(slug)
dockerfile_path = str(per_bottle_dockerfile_path(slug))
elif provider.dockerfile:
image_default = f"bot-bottle-{provider.template}:{slug}"
dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
elif provider_runtime.dockerfile:
image_default = provider_runtime.image
dockerfile_path = provider_runtime.dockerfile
elif provider.template not in PROVIDER_TEMPLATES:
user_dockerfile = (
Path.home() / ".bot-bottle" / "contrib" / provider.template / "Dockerfile"
)
if user_dockerfile.is_file():
image_default = f"bot-bottle-{provider.template}:{slug}"
dockerfile_path = str(user_dockerfile)
else:
image_default = provider_runtime.image
else:
image_default = provider_runtime.image
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
derived_image = ""
runtime_image = image
if spec.copy_cwd:
derived_image = os.environ.get(
"BOT_BOTTLE_DERIVED_IMAGE", f"bot-bottle-cwd:{slug}"
)
runtime_image = derived_image
default_container = f"bot-bottle-{slug}"
pinned_container = os.environ.get("BOT_BOTTLE_CONTAINER", "")
container_name_pinned = bool(pinned_container)
if container_name_pinned:
container_name = pinned_container
if docker_mod.container_exists(container_name):
die(
f"container '{container_name}' already exists "
f"(pinned via BOT_BOTTLE_CONTAINER). "
f"Remove it with 'docker rm -f {container_name}' or unset the override."
)
else:
container_name = ""
for candidate in docker_mod.container_name_candidates(default_container):
if not docker_mod.container_exists(candidate):
container_name = candidate
break
if not container_name:
die(
f"could not find a free container name after "
f"{default_container}-{docker_mod.MAX_CONTAINER_SUFFIX}; "
f"clean up old containers with 'docker rm -f <name>'"
)
# Probe the sidecar-bundle container name for an orphan from a
# previous run. Otherwise a stale bundle surfaces as a
# docker-create conflict deep inside launch() with no actionable
# hint; failing fast here points at the cleanup command.
bundle_name = sidecar_bundle_container_name(slug)
if docker_mod.container_exists(bundle_name):
die(
f"sidecar bundle container '{bundle_name}' already exists. "
f"This is an orphan from a previous run; clean it up with "
f"'./cli.py cleanup' (or 'docker rm -f {bundle_name}') and "
f"retry."
)
# PRD 0018 chunk 2: prepare-time scratch files live under
# ~/.bot-bottle/state/<slug>/<service>/ so chunk 3's compose
# bind-mounts can point at stable paths. The state subdirs are
# cleaned up by start.py's session-end teardown unless something
# explicitly preserves the state dir (capability-block, crash).
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
env_file = agent_dir / "agent.env"
prompt_file = agent_dir / "prompt.txt"
prompt_file.write_text("")
prompt_file.chmod(0o600)
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
resolved = resolve_env(manifest, spec.agent_name)
# Everything that should reach the bottle by-name (so its value
# never lands on argv or in env_file) goes into one dict. Nothing
# mutates the host os.environ.
forwarded_env: dict[str, str] = dict(resolved.forwarded)
_write_env_file(resolved, env_file)
prompt_file.write_text(agent.prompt)
use_runsc = docker_mod.runsc_available()
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
)
guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=guest_env)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = egress.prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
# Current Dockerfile for the agent image. Read from the repo
# root; for `--cwd` derived images the base Dockerfile is what
# the agent should propose changes against (the derived layer
# is just a workspace copy).
# (routes.yaml used to land here too but PRD 0017 chunk 3
# moved it behind the `list-egress-routes` MCP tool so the
# agent gets live state rather than a launch-time snapshot.)
supervise_dockerfile_path = (
Path(dockerfile_path)
if dockerfile_path
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
)
dockerfile_content = (
supervise_dockerfile_path.read_text(encoding="utf-8")
if supervise_dockerfile_path.is_file()
else ""
)
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = supervise.prepare(
slug, supervise_dir,
dockerfile_content=dockerfile_content,
)
return DockerBottlePlan(
spec=spec,
stage_dir=stage_dir,
guest_home=guest_home,
slug=slug,
container_name=container_name,
container_name_pinned=container_name_pinned,
image=image,
derived_image=derived_image,
runtime_image=runtime_image,
dockerfile_path=dockerfile_path,
env_file=env_file,
forwarded_env=forwarded_env,
prompt_file=prompt_file,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
def _write_env_file(resolved: ResolvedEnv, env_file: Path) -> None:
"""Serialize the literal portion of a ResolvedEnv into docker's
`--env-file` syntax (NAME=VALUE per line, mode 600 since the file
may carry verbatim values from the manifest). Forwarded names ride
on the plan as a structured tuple instead."""
env_lines: list[str] = []
for name, value in resolved.literals.items():
if "\n" in value:
die(
f"env entry {name} (literal) contains a newline; "
f"docker --env-file cannot represent multi-line values."
)
env_lines.append(f"{name}={value}")
env_file.write_text("\n".join(env_lines) + ("\n" if env_lines else ""))
env_file.chmod(0o600)
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
+59
View File
@@ -0,0 +1,59 @@
"""Prepare step for the Docker bottle backend.
`resolve_plan` does all host-side resolution (image and container
names, prompt-file, proxy plan, runtime detection) and returns a
frozen DockerBottlePlan. No Docker resources are created; the only
side effects are scratch files under `stage_dir` and a probe of
`docker info`. Cross-backend host-side validation has already run
via the base class's `prepare` template before this is called.
"""
from __future__ import annotations
from pathlib import Path
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
from .. import BottleSpec
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
def preflight() -> None:
docker_mod.require_docker()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> DockerBottlePlan:
"""Resolve Docker-specific names and write scratch files. Trusts
that the agent and its skills/git-gate keys are present —
validation already ran in the base class."""
# ==== docker specific setup ====
use_runsc = docker_mod.runsc_available()
return DockerBottlePlan(
spec=spec,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_provision=agent_provision_plan,
)
+34 -35
View File
@@ -7,11 +7,10 @@ from __future__ import annotations
import re import re
import shutil import shutil
import subprocess import subprocess
import tempfile
from typing import Iterable, Iterator from typing import Iterable, Iterator
from ...log import die, info from ...log import die, info
from ...workspace import WorkspacePlan # from ...workspace import WorkspacePlan
# Cap on the suffix the container-name conflict logic will try before # Cap on the suffix the container-name conflict logic will try before
@@ -118,39 +117,39 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
subprocess.run(args, check=True) subprocess.run(args, check=True)
def build_image_with_cwd( # def build_image_with_cwd(
derived: str, # derived: str,
base: str, # base: str,
workspace: WorkspacePlan, # workspace: "WorkspacePlan",
) -> None: # ) -> None:
"""Build a thin derived image that copies the workspace into # """Build a thin derived image that copies the workspace into
the plan's guest path and sets the plan's workdir.""" # the plan's guest path and sets the plan's workdir."""
import os # import os
#
cwd = str(workspace.host_path) # cwd = str(workspace.host_path)
if not os.path.isdir(cwd): # if not os.path.isdir(cwd):
die(f"cwd not found at {cwd}") # die(f"cwd not found at {cwd}")
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}") # info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp: # with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
context_dir = os.path.join(tmp, "context") # context_dir = os.path.join(tmp, "context")
staged_workspace = os.path.join(context_dir, "workspace") # staged_workspace = os.path.join(context_dir, "workspace")
shutil.copytree( # shutil.copytree(
cwd, # cwd,
staged_workspace, # staged_workspace,
symlinks=True, # symlinks=True,
ignore=shutil.ignore_patterns(".git"), # ignore=shutil.ignore_patterns(".git"),
) # )
dockerfile = ( # dockerfile = (
f"FROM {base}\n" # f"FROM {base}\n"
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n" # f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
f"WORKDIR {workspace.workdir}\n" # f"WORKDIR {workspace.workdir}\n"
) # )
subprocess.run( # subprocess.run(
["docker", "build", "-t", derived, "-f", "-", context_dir], # ["docker", "build", "-t", derived, "-f", "-", context_dir],
input=dockerfile, # input=dockerfile,
text=True, # text=True,
check=True, # check=True,
) # )
def image_id(ref: str) -> str: def image_id(ref: str) -> str:
@@ -0,0 +1,10 @@
"""macOS Apple Container backend.
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
the Apple `container` CLI integration; launch remains gated until the
sidecar network enforcement shape is implemented.
"""
from .backend import MacosContainerBottleBackend
__all__ = ["MacosContainerBottleBackend"]
@@ -0,0 +1,84 @@
"""MacosContainerBottleBackend — Apple Container implementation."""
from __future__ import annotations
from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import util as _container
from .bottle import MacosContainerBottle
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
from .bottle_plan import MacosContainerBottlePlan
class MacosContainerBottleBackend(
BottleBackend["MacosContainerBottlePlan", "MacosContainerBottleCleanupPlan"]
):
"""Apple Container backend. Selected by
`BOT_BOTTLE_BACKEND=macos-container` or
`--backend=macos-container`."""
name = "macos-container"
@classmethod
def is_available(cls) -> bool:
return _container.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> MacosContainerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager
def launch(
self, plan: MacosContainerBottlePlan
) -> Generator[MacosContainerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
return _cleanup.prepare_cleanup()
def cleanup(self, plan: MacosContainerBottleCleanupPlan) -> None:
_cleanup.cleanup(plan)
def enumerate_active(self) -> Sequence[ActiveAgent]:
return _enumerate.enumerate_active()
def supervise_mcp_url(self, plan: MacosContainerBottlePlan) -> str:
return plan.agent_supervise_url
@@ -0,0 +1,91 @@
"""Bottle handle for Apple's `container` CLI."""
from __future__ import annotations
import subprocess
from typing import Callable, cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
class MacosContainerBottle(Bottle):
def __init__(
self,
container: str,
teardown: Callable[[], None],
prompt_path_in_container: str | None,
*,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
):
self.name = container
self._teardown = teardown
self.prompt_path = prompt_path_in_container
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
self._closed = False
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
full_argv = list(argv)
full_argv.extend(
prompt_args(
cast(PromptMode, self._agent_prompt_mode),
self.prompt_path,
argv=full_argv,
)
)
cmd = ["container", "exec"]
if tty:
cmd.extend(["--interactive", "--tty"])
if self.agent_workdir and self.agent_workdir != "/home/node":
cmd.extend(["--workdir", self.agent_workdir])
cmd.extend([self.name, self.agent_command, *full_argv])
return cmd
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
agent_argv = self.agent_argv(argv, tty=tty)
script = (
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
if tty else None
)
if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
result = subprocess.run(
["container", "exec", "--user", user, "--interactive",
self.name, "sh", "-s"],
input=script,
capture_output=True,
text=True,
check=False,
)
return ExecResult(
returncode=result.returncode,
stdout=result.stdout,
stderr=result.stderr,
)
def cp_in(self, host_path: str, container_path: str) -> None:
subprocess.run(
["container", "cp", host_path, f"{self.name}:{container_path}"],
stdout=subprocess.DEVNULL,
check=True,
)
def close(self) -> None:
if self._closed:
return
self._closed = True
self._teardown()
@@ -0,0 +1,27 @@
"""Cleanup plan for the macOS Apple Container backend."""
from __future__ import annotations
from dataclasses import dataclass
from ...log import info
from .. import BottleCleanupPlan
@dataclass(frozen=True)
class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
containers: tuple[str, ...] = ()
networks: tuple[str, ...] = ()
def print(self) -> None:
if not self.containers and not self.networks:
info("macos-container cleanup: nothing to remove")
return
for name in self.containers:
info(f"macos-container container: {name}")
for name in self.networks:
info(f"macos-container network: {name}")
@property
def empty(self) -> bool:
return not self.containers and not self.networks
@@ -0,0 +1,58 @@
"""Plan type for the macOS Apple Container backend."""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from ...agent_provider import PromptMode
from .. import BottlePlan
@dataclass(frozen=True)
class MacosContainerBottlePlan(BottlePlan):
slug: str
forwarded_env: dict[str, str] = field(repr=False)
agent_proxy_url: str = ""
agent_git_gate_url: str = ""
agent_supervise_url: str = ""
@property
def container_name(self) -> str:
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property
def agent_command(self) -> str:
return self.agent_provision.command
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@property
def git_gate_insteadof_host(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
return super().git_gate_insteadof_host
@property
def git_gate_insteadof_scheme(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return "http"
return super().git_gate_insteadof_scheme
@@ -0,0 +1,70 @@
"""Cleanup for the macOS Apple Container backend."""
from __future__ import annotations
import subprocess
from ...log import info, warn
from . import util as container_mod
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
_PREFIX = "bot-bottle-"
_BUNDLE_PREFIX = "bot-bottle-sidecars-"
def _list_prefixed_containers() -> list[str]:
result = subprocess.run(
["container", "list", "--all", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
warn(f"container list failed: {result.stderr.strip()}")
return []
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX) or name.startswith(_BUNDLE_PREFIX)
)
def _list_prefixed_networks() -> list[str]:
result = subprocess.run(
["container", "network", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
return sorted(
name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX)
)
def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
container_mod.require_container()
return MacosContainerBottleCleanupPlan(
containers=tuple(_list_prefixed_containers()),
networks=tuple(_list_prefixed_networks()),
)
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
for name in plan.containers:
info(f"container delete --force {name}")
subprocess.run(
["container", "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
for name in plan.networks:
info(f"container network delete {name}")
subprocess.run(
["container", "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
@@ -0,0 +1,40 @@
"""Active-agent enumeration for the macOS Apple Container backend."""
from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent
_PREFIX = "bot-bottle-"
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def enumerate_active() -> list[ActiveAgent]:
result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
out: list[ActiveAgent] = []
for name in sorted(line.strip() for line in result.stdout.splitlines()):
if not name.startswith(_PREFIX):
continue
if name.startswith(_SIDECAR_PREFIX):
continue
slug = name[len(_PREFIX):]
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="macos-container",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=(),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
@@ -0,0 +1,426 @@
"""Launch flow for the macOS Apple Container backend.
This backend keeps the explicit proxy-env enforcement model for v1:
the agent container is attached only to a host-only Apple Container
network, while the sidecar bundle is attached to a NAT network first
and the host-only network second. The sidecar's host-only IP is
discovered from `container inspect` and stamped into the agent's
HTTP_PROXY / HTTPS_PROXY env vars.
"""
from __future__ import annotations
import dataclasses
import os
import shutil
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...bottle_state import egress_state_dir, git_gate_state_dir
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import die, info, warn
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from ..docker.egress import egress_tls_init
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import util as container_mod
from .bottle import MacosContainerBottle
from .bottle_plan import MacosContainerBottlePlan
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_AGENT_SLEEP_SECONDS = "2147483647"
_GIT_HTTP_PORT = 9420
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
def internal_network_name(slug: str) -> str:
return f"bot-bottle-net-{slug}"
def egress_network_name(slug: str) -> str:
return f"bot-bottle-egress-{slug}"
def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
@contextmanager
def launch(
plan: MacosContainerBottlePlan,
*,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]:
"""Build, run, provision, and yield an Apple Container bottle."""
stack = ExitStack()
bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"macos-container teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
plan = _mint_certs(plan)
_build_images(plan)
internal_network = internal_network_name(plan.slug)
egress_network = egress_network_name(plan.slug)
_create_networks(internal_network, egress_network, stack)
sidecar_name = sidecar_container_name(plan.slug)
container_mod.force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
stack.callback(container_mod.force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
sidecar_ip = container_mod.container_ipv4_on_network(
sidecar_name, internal_network,
)
plan = _stamp_agent_urls(plan, sidecar_ip)
container_mod.force_remove_container(plan.container_name)
_start_agent(plan, internal_network, sidecar_ip)
stack.callback(container_mod.force_remove_container, plan.container_name)
bottle = MacosContainerBottle(
plan.container_name,
teardown,
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=plan.spec.label or plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
)
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_images(plan: MacosContainerBottlePlan) -> None:
container_mod.build_image(
SIDECAR_BUNDLE_IMAGE,
_REPO_DIR,
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
)
container_mod.build_image(
plan.image,
_REPO_DIR,
dockerfile=plan.dockerfile_path,
)
def _create_networks(
internal_network: str,
egress_network: str,
stack: ExitStack,
) -> None:
container_mod.create_network(internal_network, internal=True)
stack.callback(container_mod.remove_network, internal_network)
container_mod.create_network(egress_network)
stack.callback(container_mod.remove_network, egress_network)
def _start_sidecar_bundle(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> None:
argv = _sidecar_run_argv(plan, sidecar_name, internal_network, egress_network)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
env = {**os.environ, **token_values}
info(f"container run sidecar bundle {sidecar_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for sidecar bundle {sidecar_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _start_agent(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> None:
argv = _agent_run_argv(plan, internal_network, sidecar_ip)
env = {
**os.environ,
**plan.forwarded_env,
}
info(f"container run agent {plan.container_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for agent {plan.container_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _stamp_agent_urls(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> MacosContainerBottlePlan:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
supervise_url = ""
if plan.supervise_plan is not None:
supervise_url = f"http://{sidecar_ip}:{SUPERVISE_PORT}/"
git_gate_url = ""
if plan.git_gate_plan.upstreams:
git_gate_url = f"http://{sidecar_ip}:{_GIT_HTTP_PORT}"
return dataclasses.replace(
plan,
agent_proxy_url=proxy_url,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
container_mod.exec_container(
sidecar_name,
[
"mkdir",
"-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER,
"/git",
str(Path(_GIT_GATE_READY_FILE).parent),
],
)
for host_path, container_path in _git_gate_files(plan):
container_mod.copy_into_container(
sidecar_name, host_path, container_path,
)
container_mod.exec_container(
sidecar_name,
[
"sh",
"-c",
"chmod 755 "
f"{GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} "
f"{GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
f"touch {_GIT_GATE_READY_FILE}",
],
)
def _git_gate_files(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str], ...]:
gp = plan.git_gate_plan
files: list[tuple[str, str]] = [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for upstream in gp.upstreams:
files.append((
expand_tilde(upstream.identity_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key",
))
if upstream.known_hosts_file:
files.append((
str(upstream.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts",
))
return tuple(files)
def _sidecar_run_argv(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> list[str]:
argv = [
"container", "run",
"--name", sidecar_name,
"--detach",
"--rm",
"--network", egress_network,
"--network", internal_network,
"--dns", _sidecar_dns(),
"--env", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}",
]
for entry in _sidecar_env_entries(plan):
argv += ["--env", entry]
for host_path, container_path, read_only in _sidecar_mounts(plan):
argv += ["--mount", _mount_spec(host_path, container_path, read_only)]
argv.append(SIDECAR_BUNDLE_IMAGE)
return argv
def _agent_run_argv(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> list[str]:
argv = [
"container", "run",
"--name", plan.container_name,
"--detach",
"--rm",
"--network", internal_network,
]
for entry in _agent_env_entries(plan, sidecar_ip):
argv += ["--env", entry]
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
return argv
def _sidecar_dns() -> str:
return container_mod.dns_server()
def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
daemons = ["egress"]
if plan.git_gate_plan.upstreams:
daemons += ["git-gate", "git-http"]
if plan.supervise_plan is not None:
daemons.append("supervise")
return tuple(daemons)
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
env: list[str] = []
if plan.egress_plan.routes:
env.extend(sorted(plan.egress_plan.token_env_map.keys()))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
return tuple(env)
def _sidecar_mounts(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str, bool], ...]:
mounts: list[tuple[str, str, bool]] = []
ep = plan.egress_plan
mounts.append((
str(ep.mitmproxy_ca_host_path.parent),
str(Path(EGRESS_CA_IN_CONTAINER).parent),
False,
))
if ep.routes:
mounts.append((
str(_stage_routes_dir(plan)),
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent),
True,
))
sp = plan.supervise_plan
if sp is not None:
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
return tuple(mounts)
def _stage_routes_dir(plan: MacosContainerBottlePlan) -> Path:
routes_dir = plan.stage_dir / "macos-container-egress"
routes_dir.mkdir(parents=True, exist_ok=True)
shutil.copyfile(
plan.egress_plan.routes_path,
routes_dir / Path(EGRESS_ROUTES_IN_CONTAINER).name,
)
return routes_dir
def _mount_spec(host_path: str, container_path: str, read_only: bool) -> str:
spec = f"type=bind,source={host_path},target={container_path}"
if read_only:
spec += ",readonly"
return spec
def _agent_env_entries(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> tuple[str, ...]:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
no_proxy = _agent_no_proxy(plan, sidecar_ip)
env = [
f"HTTPS_PROXY={proxy_url}",
f"HTTP_PROXY={proxy_url}",
f"https_proxy={proxy_url}",
f"http_proxy={proxy_url}",
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
if plan.agent_git_gate_url:
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
if plan.agent_supervise_url:
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
return tuple(env)
def _agent_no_proxy(plan: MacosContainerBottlePlan, sidecar_ip: str) -> str:
hosts = ["localhost", "127.0.0.1", sidecar_ip]
return ",".join(hosts)
@@ -0,0 +1,44 @@
"""Prepare step for the macOS Apple Container backend."""
from __future__ import annotations
from pathlib import Path
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from .. import BottleSpec
from . import util as container_mod
from .bottle_plan import MacosContainerBottlePlan
def preflight() -> None:
container_mod.require_container()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> MacosContainerBottlePlan:
return MacosContainerBottlePlan(
spec=spec,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
+388
View File
@@ -0,0 +1,388 @@
"""Host-side primitives for Apple's `container` CLI."""
from __future__ import annotations
import json
import os
import ipaddress
import platform
import shutil
import subprocess
import time
from typing import Iterable
from ...log import die, info
_CONTAINER = "container"
_DEFAULT_DNS = "1.1.1.1"
def is_macos() -> bool:
return platform.system() == "Darwin"
def is_available() -> bool:
return is_macos() and shutil.which(_CONTAINER) is not None
def require_container() -> None:
"""Fail with an install pointer if Apple Container is unavailable."""
if not is_macos():
info("BOT_BOTTLE_BACKEND=macos-container requires macOS.")
die("macos-container backend is only supported on macOS")
if shutil.which(_CONTAINER) is None:
info("Apple Container is required but was not found on PATH.")
info("Install: https://github.com/apple/container/releases")
die("container not found")
_require_container_service()
def _require_container_service() -> None:
result = subprocess.run(
[_CONTAINER, "system", "status"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode != 0:
info("Apple Container system service is not running.")
info("Start it with: container system start")
die("container system service not running")
def dns_server() -> str:
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
if override:
return override
return _host_ipv4_dns() or _DEFAULT_DNS
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
"""Build an OCI image with Apple's BuildKit-backed `container build`."""
info(
f"building image {ref} from {context} with Apple Container "
"(layer cache keeps repeat builds fast)"
)
_ensure_builder_dns()
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
if dockerfile:
args.extend(["-f", dockerfile])
args.append(context)
subprocess.run(args, check=True)
def _ensure_builder_dns() -> None:
dns = dns_server()
status = _builder_status()
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
if _builder_running(status) and _builder_resolves_build_hosts():
if override and not _builder_has_dns(status, dns):
_restart_builder_with_dns(dns)
return
_restart_builder_with_dns(dns)
def _restart_builder_with_dns(dns: str) -> None:
subprocess.run(
[_CONTAINER, "builder", "stop"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
subprocess.run(
[_CONTAINER, "builder", "start", "--dns", dns],
check=True,
)
def _host_ipv4_dns() -> str:
if not is_macos():
return ""
result = subprocess.run(
["scutil", "--dns"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return ""
blocks: list[list[str]] = []
current: list[str] = []
for line in result.stdout.splitlines():
if line.startswith("resolver #") and current:
blocks.append(current)
current = []
current.append(line)
if current:
blocks.append(current)
for direct_only in (True, False):
for block in blocks:
text = "\n".join(block)
if direct_only and "Directly Reachable Address" not in text:
continue
for line in block:
if "nameserver[" not in line or ":" not in line:
continue
candidate = line.split(":", 1)[1].strip()
if _usable_ipv4(candidate):
return candidate
return ""
def _usable_ipv4(value: str) -> bool:
try:
address = ipaddress.ip_address(value)
except ValueError:
return False
return (
address.version == 4
and not address.is_loopback
and not address.is_link_local
and not address.is_multicast
and not address.is_unspecified
)
def _builder_status() -> list[dict[str, object]]:
result = subprocess.run(
[_CONTAINER, "builder", "status", "--format", "json"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError:
return []
if isinstance(data, list):
return [entry for entry in data if isinstance(entry, dict)]
if isinstance(data, dict):
return [data]
return []
def _builder_running(status: list[dict[str, object]]) -> bool:
for entry in status:
entry_status = entry.get("status")
if isinstance(entry_status, dict) and entry_status.get("state") == "running":
return True
return False
def _builder_dns_nameservers(status: list[dict[str, object]]) -> list[str]:
out: list[str] = []
for entry in status:
config = entry.get("configuration")
config_dns = config.get("dns") if isinstance(config, dict) else None
nameservers = (
config_dns.get("nameservers")
if isinstance(config_dns, dict)
else None
)
if not isinstance(nameservers, list):
continue
out.extend(name for name in nameservers if isinstance(name, str))
return out
def _builder_has_dns(status: list[dict[str, object]], dns: str) -> bool:
return dns in _builder_dns_nameservers(status)
def _builder_resolves_build_hosts() -> bool:
result = subprocess.run(
[_CONTAINER, "exec", "buildkit", "getent", "hosts", "deb.debian.org"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
return result.returncode == 0
def image_exists(ref: str) -> bool:
return _silent_run([_CONTAINER, "image", "inspect", ref]) == 0
def container_exists(name: str) -> bool:
result = subprocess.run(
[_CONTAINER, "list", "--all", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return False
return name in {line.strip() for line in result.stdout.splitlines()}
def force_remove_container(name: str) -> None:
if container_exists(name):
subprocess.run(
[_CONTAINER, "delete", "--force", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
def copy_into_container(name: str, host_path: str, container_path: str) -> None:
cmd = [_CONTAINER, "cp", host_path, f"{name}:{container_path}"]
result = _run_container_op(cmd)
if result.returncode != 0:
die(
f"container cp into {name}:{container_path} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def exec_container(name: str, argv: list[str]) -> None:
result = _run_container_op([_CONTAINER, "exec", name, *argv])
if result.returncode != 0:
die(
f"container exec in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]:
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
)
for _ in range(19):
if result.returncode == 0:
return result
time.sleep(0.1)
result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
)
return result
def create_network(name: str, *, internal: bool = False) -> None:
args = [
_CONTAINER, "network", "create",
"--label", "bot-bottle.backend=macos-container",
]
if internal:
args.append("--internal")
args.append(name)
result = subprocess.run(
args, capture_output=True, text=True, check=False,
)
if result.returncode == 0:
return
if "already exists" in (result.stderr or "").lower():
return
die(
f"container network create {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def remove_network(name: str) -> None:
result = subprocess.run(
[_CONTAINER, "network", "delete", name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode != 0:
return
def inspect_container(name: str) -> dict[str, object]:
result = subprocess.run(
[_CONTAINER, "inspect", name],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container inspect {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "[]")
except json.JSONDecodeError as exc:
die(f"container inspect {name} returned malformed JSON: {exc}")
if isinstance(data, list) and data and isinstance(data[0], dict):
return data[0]
if isinstance(data, dict):
return data
die(f"container inspect {name} returned an unexpected shape")
raise AssertionError("unreachable")
def container_ipv4_on_network(name: str, network: str) -> str:
data = inspect_container(name)
status = data.get("status")
networks = status.get("networks") if isinstance(status, dict) else None
if not isinstance(networks, list):
die(f"container inspect {name} did not include status.networks")
for entry in networks:
if not isinstance(entry, dict):
continue
if entry.get("network") != network:
continue
raw = entry.get("ipv4Address")
if not isinstance(raw, str) or not raw:
die(f"container {name} has no IPv4 address on {network}")
return raw.split("/", 1)[0]
die(f"container {name} is not attached to network {network}")
raise AssertionError("unreachable")
def image_id(ref: str) -> str:
"""Return the image digest/ID from `container image inspect`.
The command returns JSON on current Apple Container releases. Keep
parsing narrow and fatal so callers do not cache on an empty key.
"""
import json
result = subprocess.run(
[_CONTAINER, "image", "inspect", ref],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container image inspect for {ref!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "{}")
except json.JSONDecodeError as exc:
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
if isinstance(data, list) and data:
data = data[0]
if isinstance(data, dict):
value = data.get("id") or data.get("digest") or data.get("ID")
if value:
return str(value)
die(f"container image inspect for {ref!r} did not include an image id")
raise AssertionError("unreachable")
def save(ref: str, output: str) -> None:
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
def _silent_run(cmd: Iterable[str]) -> int:
return subprocess.run(
list(cmd),
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
).returncode
+122
View File
@@ -0,0 +1,122 @@
"""Shared helpers used by both backends' resolve_plan steps.
Each helper owns one well-defined step of the per-bottle plan
resolution so docker and smolmachines don't repeat the same logic.
Backend-specific steps (container names, env-file, per-bottle
Dockerfile overrides, subnet allocation) stay in the backend's own
resolve_plan.py.
"""
from __future__ import annotations
import os
from dataclasses import replace
from datetime import datetime, timezone
from pathlib import Path
from ..agent_provider import AgentProvisionPlan
from ..bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
egress_state_dir,
git_gate_state_dir,
supervise_state_dir,
write_metadata,
)
from ..egress import Egress, EgressPlan
from ..git_gate import GitGate, GitGatePlan
from ..manifest import ManifestBottle
from ..supervise import Supervise, SupervisePlan
from . import BottleSpec
def mint_slug(spec: BottleSpec) -> str:
"""Return the bottle identity: the recorded identity for a resume,
or a freshly minted one for a new start."""
return spec.identity or bottle_identity(spec.agent_name)
def write_launch_metadata(
slug: str, spec: BottleSpec, *, compose_project: str, backend: str,
) -> None:
"""Persist launch metadata so `cli.py resume <identity>` can
reconstruct the spec. Idempotent — re-writes on resume with a
refreshed started_at."""
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=compose_project,
backend=backend,
label=spec.label,
color=spec.color,
))
def prepare_agent_state_dir(slug: str, spec: BottleSpec) -> tuple[Path, Path]:
"""Create the agent state subdir, write the prompt file.
Returns (agent_dir, prompt_file)."""
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
prompt_file = agent_dir / "prompt.txt"
prompt_file.write_text(agent.prompt or "")
prompt_file.chmod(0o600)
return agent_dir, prompt_file
def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
return GitGate().prepare(bottle, slug, git_gate_dir)
def prepare_egress(
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
) -> EgressPlan:
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
"""Prepare the supervise sidecar state dir. Returns None when
bottle.supervise is falsy."""
if not bottle.supervise:
return None
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
return Supervise().prepare(slug, supervise_dir)
def merge_provision_env_vars(provision: AgentProvisionPlan) -> AgentProvisionPlan:
"""Fold provision.env_vars into guest_env (setdefault semantics)
and return a new plan with the merged guest_env."""
merged = dict(provision.guest_env)
for key, val in provision.env_vars.items():
merged.setdefault(key, val)
return replace(provision, guest_env=merged)
def resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
"""Resolve a manifest-supplied dockerfile path relative to user_cwd."""
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
__all__ = [
"merge_provision_env_vars",
"mint_slug",
"prepare_agent_state_dir",
"prepare_egress",
"prepare_git_gate",
"prepare_supervise",
"resolve_manifest_dockerfile",
"write_launch_metadata",
]
+33 -10
View File
@@ -13,16 +13,20 @@ from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from typing import Generator, Sequence from typing import Generator, Sequence
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup from . import cleanup as _cleanup
from . import enumerate as _enumerate from . import enumerate as _enumerate
from . import launch as _launch from . import launch as _launch
from . import prepare as _prepare from . import resolve_plan as _resolve_plan
from . import smolvm as _smolvm from . import smolvm as _smolvm
from .bottle import SmolmachinesBottle from .bottle import SmolmachinesBottle
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
from .bottle_plan import SmolmachinesBottlePlan from .bottle_plan import SmolmachinesBottlePlan
from .provision import workspace as _workspace
class SmolmachinesBottleBackend( class SmolmachinesBottleBackend(
@@ -41,10 +45,34 @@ class SmolmachinesBottleBackend(
runtime check happens at `prepare`.""" runtime check happens at `prepare`."""
return _smolvm.is_available() return _smolvm.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan( def _resolve_plan(
self, spec: BottleSpec, *, stage_dir: Path self,
spec: BottleSpec,
*,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> SmolmachinesBottlePlan: ) -> SmolmachinesBottlePlan:
return _prepare.resolve_plan(spec, stage_dir=stage_dir) return _resolve_plan.resolve_plan(
spec,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager @contextmanager
def launch( def launch(
@@ -53,11 +81,6 @@ class SmolmachinesBottleBackend(
with _launch.launch(plan, provision=self.provision) as bottle: with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle yield bottle
def provision_workspace(
self, plan: SmolmachinesBottlePlan, bottle: Bottle
) -> None:
_workspace.provision_workspace(plan, bottle)
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str: def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
"""The smolmachines guest reaches the supervise sidecar via a """The smolmachines guest reaches the supervise sidecar via a
host-published random port the launch step pinned earlier host-published random port the launch step pinned earlier
+23 -8
View File
@@ -20,10 +20,12 @@ from __future__ import annotations
import subprocess import subprocess
import sys import sys
import time import time
import shlex
from typing import Mapping, cast from typing import Mapping, cast
from ...agent_provider import PromptMode, prompt_args from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import pty_resize as _pty_resize from . import pty_resize as _pty_resize
from . import smolvm as _smolvm from . import smolvm as _smolvm
@@ -68,6 +70,10 @@ class SmolmachinesBottle(Bottle):
guest_env: Mapping[str, str] | None = None, guest_env: Mapping[str, str] | None = None,
agent_command: str = "claude", agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file", agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
) -> None: ) -> None:
self.name = machine_name self.name = machine_name
# In-VM path to the agent's prompt file. None when the # In-VM path to the agent's prompt file. None when the
@@ -81,9 +87,10 @@ class SmolmachinesBottle(Bottle):
self._guest_env = dict(guest_env or {}) self._guest_env = dict(guest_env or {})
self._agent_prompt_mode = agent_prompt_mode self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command self.agent_command = agent_command
self.agent_provider_template = ( self.terminal_title = terminal_title
"codex" if agent_command == "codex" else "claude" self.terminal_color = terminal_color
) self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
def agent_argv( def agent_argv(
self, argv: list[str], *, tty: bool = True, self, argv: list[str], *, tty: bool = True,
@@ -91,8 +98,14 @@ class SmolmachinesBottle(Bottle):
flags = ["smolvm", "machine", "exec", "--name", self.name] flags = ["smolvm", "machine", "exec", "--name", self.name]
if tty: if tty:
flags += ["-i", "-t"] flags += ["-i", "-t"]
agent_tail = ["env", *_env_assignments_for("node", self._guest_env), agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
self.agent_command] if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
agent_tail += [
"sh", "-lc",
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
"bot-bottle-agent",
]
agent_tail.append(self.agent_command)
provider_prompt_args = prompt_args( provider_prompt_args = prompt_args(
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv, cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
) )
@@ -128,9 +141,11 @@ class SmolmachinesBottle(Bottle):
UID switches via `runuser -u node --` (not `-l`) so we UID switches via `runuser -u node --` (not `-l`) so we
avoid login-shell wiring. HOME / USER come from `smolvm avoid login-shell wiring. HOME / USER come from `smolvm
-e` instead, which sets them on the process env.""" -e` instead, which sets them on the process env."""
return subprocess.run( agent_argv = self.agent_argv(argv, tty=tty)
self.agent_argv(argv, tty=tty), check=False, script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
).returncode if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
# smolvm/libkrun can SIGKILL an otherwise-normal exec during # smolvm/libkrun can SIGKILL an otherwise-normal exec during
# early-VM provisioning. Retry once after a short settle so # early-VM provisioning. Retry once after a short settle so
+28 -26
View File
@@ -29,27 +29,6 @@ class SmolmachinesBottlePlan(BottlePlan):
bundle_subnet: str bundle_subnet: str
bundle_gateway: str bundle_gateway: str
bundle_ip: str bundle_ip: str
# smolvm machine name + agent image source. machine_create
# boots from a packed `.smolmachine` artifact (pre-baked at
# prepare time via `smolvm pack create`); using `--from`
# instead of `--image` avoids the registry-pull race we hit
# when machine_start tried to fetch on-demand and the libkrun
# agent's network attempt got refused by macOS.
#
# Chunk 2d ships with a public placeholder image (alpine)
# since bot-bottle-claude:latest lives in the operator's local
# docker daemon and smolvm's crane backend can't read from
# there; chunk 4 resolves the agent-image-conversion gap
# (push to a registry first, or smolvm grows a docker-daemon
# transport).
machine_name: str
# Agent image ref (docker tag). `launch` runs the
# build → save → registry push → smolvm pack pipeline against
# this and feeds the resulting `.smolmachine` artifact to
# `machine_create --from`. The pipeline runs at launch time
# (not prepare time) so the docker build output doesn't garble
# the dashboard's preflight modal.
agent_image_ref: str
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since # In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
# the guest has no DNS resolver inside the TSI allowlist. # the guest has no DNS resolver inside the TSI allowlist.
# Passed to `smolvm machine create` as `-e K=V` flags. # Passed to `smolvm machine create` as `-e K=V` flags.
@@ -57,11 +36,6 @@ class SmolmachinesBottlePlan(BottlePlan):
# `--smolfile` is mutually exclusive with `--from`, and # `--smolfile` is mutually exclusive with `--from`, and
# `--from` is the path that avoids the registry-pull race). # `--from` is the path that avoids the registry-pull race).
guest_env: dict[str, str] guest_env: dict[str, str]
# Path to the agent's prompt file on the host. Always written
# (mode 0o600) so the in-VM path always exists; the file is
# empty when the agent has no prompt — claude-code reads it
# via --append-system-prompt-file only when non-empty.
prompt_file: Path
# Inner Plans for the sidecar bundle daemons. The same shape the # Inner Plans for the sidecar bundle daemons. The same shape the
# docker backend uses — same `.prepare()` calls produced # docker backend uses — same `.prepare()` calls produced
# them — but our launch step doesn't populate the # them — but our launch step doesn't populate the
@@ -82,6 +56,34 @@ class SmolmachinesBottlePlan(BottlePlan):
agent_git_gate_host: str = "" agent_git_gate_host: str = ""
agent_supervise_url: str = "" agent_supervise_url: str = ""
@property
def machine_name(self) -> str:
"""smolvm machine name. `machine_create` boots from a packed
`.smolmachine` artifact (pre-baked at prepare time via
`smolvm pack create`); using `--from` instead of `--image`
avoids the registry-pull race we hit when machine_start tried
to fetch on-demand and the libkrun agent's network attempt
got refused by macOS."""
return self.agent_provision.instance_name
@property
def agent_image(self) -> str:
"""Agent image ref (docker tag). `launch` runs the
build → save → registry push → smolvm pack pipeline against
this and feeds the resulting `.smolmachine` artifact to
`machine_create --from`. The pipeline runs at launch time
(not prepare time) so the docker build output doesn't garble
the dashboard's preflight modal."""
return self.agent_provision.image
@property
def prompt_file(self) -> Path:
"""Path to the agent's prompt file on the host. Always written
(mode 0o600) so the in-VM path always exists; the file is
empty when the agent has no prompt — claude-code reads it
via --append-system-prompt-file only when non-empty."""
return self.agent_provision.prompt_file
@property @property
def git_gate_insteadof_host(self) -> str: def git_gate_insteadof_host(self) -> str:
return self.agent_git_gate_host return self.agent_git_gate_host
+3 -1
View File
@@ -23,7 +23,7 @@ import json
import subprocess import subprocess
from .. import ActiveAgent from .. import ActiveAgent
from ..docker.bottle_state import read_metadata from ...bottle_state import read_metadata
from . import sidecar_bundle as _bundle from . import sidecar_bundle as _bundle
@@ -64,6 +64,8 @@ def enumerate_active() -> list[ActiveAgent]:
agent_name=metadata.agent_name if metadata else "?", agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "", started_at=metadata.started_at if metadata else "",
services=services_by_slug.get(slug, ()), services=services_by_slug.get(slug, ()),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
)) ))
return out return out
+6 -2
View File
@@ -41,7 +41,7 @@ from ..docker.git_gate import (
) )
from ...git_gate import revoke_git_gate_provisioned_keys from ...git_gate import revoke_git_gate_provisioned_keys
from ...log import warn from ...log import warn
from ..docker.bottle_state import egress_state_dir, git_gate_state_dir from ...bottle_state import egress_state_dir, git_gate_state_dir
from . import loopback_alias as _loopback from . import loopback_alias as _loopback
from . import sidecar_bundle as _bundle from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm from . import smolvm as _smolvm
@@ -90,7 +90,7 @@ def launch(
# here, not in prepare, so the docker-build output doesn't # here, not in prepare, so the docker-build output doesn't
# garble the dashboard's preflight modal. # garble the dashboard's preflight modal.
agent_from_path = _ensure_smolmachine( agent_from_path = _ensure_smolmachine(
plan.agent_image_ref, plan.agent_image,
dockerfile=plan.agent_dockerfile_path, dockerfile=plan.agent_dockerfile_path,
) )
@@ -103,6 +103,10 @@ def launch(
guest_env=plan.guest_env, guest_env=plan.guest_env,
agent_command=plan.agent_command, agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode, agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=plan.spec.label or plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
) )
bottle.prompt_path = provision(plan, bottle) bottle.prompt_path = provision(plan, bottle)
-181
View File
@@ -1,181 +0,0 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup — that's `launch.launch`'s job."""
from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import agent_provision_plan, runtime_for
from ...backend import BottleSpec
from ...backend.docker.bottle_state import (
BottleMetadata,
agent_state_dir,
bottle_identity,
egress_state_dir,
git_gate_state_dir,
supervise_state_dir,
write_metadata,
)
from ...egress import Egress
from ...env import resolve_env
from ...git_gate import GitGate
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
# Gateway ports the bundle exposes inside its container — git-gate's
# git-daemon, supervise's MCP. The agent inside the smolvm guest
# dials these on the bundle's pinned IP.
_BUNDLE_GIT_GATE_PORT = 9418
_BUNDLE_SUPERVISE_PORT = 9100
def resolve_plan(
spec: BottleSpec, *, stage_dir: Path
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
smolmachines_preflight()
manifest = spec.manifest
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = "/home/node"
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
slug = spec.identity or bottle_identity(spec.agent_name)
# Record minimal metadata so `cli.py resume` can recover the
# slug. Same schema as the docker backend.
write_metadata(BottleMetadata(
identity=slug,
agent_name=spec.agent_name,
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project="",
backend="smolmachines",
))
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
resolved = resolve_env(manifest, spec.agent_name)
guest_env: dict[str, str] = {
**resolved.literals,
**resolved.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
# Prompt file is always written (mode 0o600) so the in-VM
# path always exists. Content is the agent's `prompt`
# field (markdown body) — empty for agents with no prompt.
# claude-code reads it via --append-system-prompt-file only
# when non-empty, but the file must exist either way to
# match the docker backend's contract.
agent_dir = agent_state_dir(slug)
agent_dir.mkdir(parents=True, exist_ok=True)
prompt_file = agent_dir / "prompt.txt"
agent = manifest.agents[spec.agent_name]
prompt_file.write_text(agent.prompt or "")
prompt_file.chmod(0o600)
machine_name = f"bot-bottle-{slug}"
# Stash the agent image ref — `launch.launch` runs the
# build → pack pipeline at bringup. Honors BOT_BOTTLE_IMAGE
# to match the docker backend's `resolve_plan` default.
agent_dockerfile_path = ""
if provider.dockerfile:
agent_dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
image_default = f"bot-bottle-{provider.template}:{slug}"
elif provider_runtime.dockerfile:
agent_dockerfile_path = provider_runtime.dockerfile
image_default = provider_runtime.image
else:
image_default = provider_runtime.image
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
guest_env=guest_env,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
)
merged_guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
merged_guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = Egress().prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = Supervise().prepare(slug, supervise_dir)
return SmolmachinesBottlePlan(
spec=spec,
stage_dir=stage_dir,
guest_home=guest_home,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
machine_name=machine_name,
agent_image_ref=agent_image_ref,
guest_env=agent_provision.guest_env,
prompt_file=prompt_file,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
path = Path(os.path.expanduser(path_value))
if not path.is_absolute():
path = Path(spec.user_cwd) / path
return str(path)
@@ -6,8 +6,7 @@ the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
provisioning also moved to the AgentProvider ABC (with Debian/node provisioning also moved to the AgentProvider ABC (with Debian/node
defaults); user plugins override them for non-standard images. defaults); user plugins override them for non-standard images.
The module left in this subpackage handles the remaining backend- No modules remain in this subpackage. Workspace copying now runs
specific step: through `BottleBackend.provision_workspace` against the running
bottle for every backend.
- workspace.py — copy the operator workspace into the guest
""" """
@@ -1,32 +0,0 @@
"""Copy the operator workspace into a smolmachines guest."""
from __future__ import annotations
import shlex
from ....log import info
from ... import Bottle
from ..bottle_plan import SmolmachinesBottlePlan
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
"""Copy host cwd contents to the planned guest workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path_q = shlex.quote(workspace.guest_path)
guest_parent_q = shlex.quote(guest_parent)
owner_q = shlex.quote(workspace.owner)
mode_q = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
bottle.exec(
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
user="root",
)
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
bottle.exec(
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
user="root",
)
@@ -0,0 +1,80 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup — that's `launch.launch`'s job."""
from __future__ import annotations
from pathlib import Path
from .. import BottleSpec
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
def preflight() -> None:
smolmachines_preflight()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
return {
**resolved_env.literals,
**resolved_env.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
def resolve_plan(
spec: BottleSpec,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
# ==== smolmachines specific setup ====
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
return SmolmachinesBottlePlan(
spec=spec,
stage_dir=stage_dir,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
guest_env=agent_provision_plan.guest_env,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
+3 -1
View File
@@ -21,7 +21,9 @@ def smolmachines_preflight() -> None:
die( die(
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on " "BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
"PATH. Install with: " "PATH. Install with: "
"curl -sSL https://smolmachines.com/install.sh | sh" "curl -sSL https://smolmachines.com/install.sh | sh. "
"To use the legacy Docker backend instead, set "
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
) )
+82
View File
@@ -0,0 +1,82 @@
"""Terminal escape-sequence helpers shared across all bottle backends."""
from __future__ import annotations
import shlex
# color name → (normal_idx, normal_hex, bright_idx, bright_hex, dark_bg_hex)
# OSC 4 sets indexed palette entries (affects syntax-highlighted code and any
# TUI content that uses indexed colors). dark_bg_hex is used for OSC 11
# (default background) — a very dark tint that's visible even when the TUI
# uses true/24-bit colors for its own chrome, which would otherwise bypass
# the palette entirely.
_COLORS: dict[str, tuple[int, str, int, str, str]] = {
"black": (0, "#2d2d2d", 8, "#5c5c5c", "#0a0a0a"),
"red": (1, "#c0392b", 9, "#e74c3c", "#1a0707"),
"green": (2, "#27ae60", 10, "#2ecc71", "#071a09"),
"yellow": (3, "#d4ac0d", 11, "#f1c40f", "#1a1507"),
"blue": (4, "#2471a3", 12, "#3498db", "#07071a"),
"magenta": (5, "#7d3c98", 13, "#9b59b6", "#12071a"),
"cyan": (6, "#148f77", 14, "#1abc9c", "#071a1a"),
"white": (7, "#bdc3c7", 15, "#ecf0f1", "#111111"),
"bright-black": (8, "#5c5c5c", 0, "#2d2d2d", "#111111"),
"bright-red": (9, "#e74c3c", 1, "#c0392b", "#200808"),
"bright-green": (10, "#2ecc71", 2, "#27ae60", "#082008"),
"bright-yellow": (11, "#f1c40f", 3, "#d4ac0d", "#201808"),
"bright-blue": (12, "#3498db", 4, "#2471a3", "#080820"),
"bright-magenta": (13, "#9b59b6", 5, "#7d3c98", "#160820"),
"bright-cyan": (14, "#1abc9c", 6, "#148f77", "#082020"),
"bright-white": (15, "#ecf0f1", 7, "#bdc3c7", "#151515"),
}
# OSC 104 resets all indexed palette entries; OSC 111 resets default background.
_RESET_PRINTF = "printf '\\033]104\\007\\033]111\\007'"
def palette_printf(color: str) -> str:
"""Shell `printf` command that emits OSC 4 + OSC 11 to tint the terminal
for *color*: sets the normal/bright palette entries AND the default
background to a dark shade of that color. Returns '' if unknown."""
entry = _COLORS.get(color)
if not entry:
return ""
n_idx, n_hex, b_idx, b_hex, bg_hex = entry
seq = (
f"\\033]4;{n_idx};{n_hex}\\007"
f"\\033]4;{b_idx};{b_hex}\\007"
f"\\033]11;{bg_hex}\\007"
)
return f"printf '{seq}'"
def exec_shell_script(
agent_argv: list[str],
terminal_title: str = "",
terminal_color: str = "",
) -> str | None:
"""Build a shell script string that optionally sets the terminal
title and/or palette before running *agent_argv*, and resets the
palette + background on exit. Returns None when no decoration is
needed — callers should run *agent_argv* directly in that case."""
title_cmd = (
f"printf '\\033]0;%s\\007' {shlex.quote(terminal_title)}"
if terminal_title else ""
)
pal_cmd = palette_printf(terminal_color)
if not title_cmd and not pal_cmd:
return None
parts: list[str] = []
if title_cmd:
parts.append(title_cmd)
if pal_cmd:
parts.append(pal_cmd)
parts.append(shlex.join(agent_argv))
parts.append(_RESET_PRINTF)
else:
# No palette change — exec so the agent replaces the shell.
parts.append(f"exec {shlex.join(agent_argv)}")
return "; ".join(parts)
@@ -37,8 +37,7 @@ from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import cast from typing import cast
from ... import supervise as _supervise from . import supervise as _supervise
from . import util as docker_mod
# Directory layout: ~/.bot-bottle/state/<identity>/... # Directory layout: ~/.bot-bottle/state/<identity>/...
@@ -82,6 +81,7 @@ def bottle_identity(agent_name: str) -> str:
To continue an existing bottle's state, use the recorded To continue an existing bottle's state, use the recorded
identity from BottleMetadata via `cli.py resume <identity>`, identity from BottleMetadata via `cli.py resume <identity>`,
not this function.""" not this function."""
from .backend.docker import util as docker_mod
slug = docker_mod.slugify(agent_name) slug = docker_mod.slugify(agent_name)
suffix = "".join(secrets.choice(_SUFFIX_ALPHABET) for _ in range(_RANDOM_SUFFIX_LEN)) suffix = "".join(secrets.choice(_SUFFIX_ALPHABET) for _ in range(_RANDOM_SUFFIX_LEN))
return f"{slug}-{suffix}" return f"{slug}-{suffix}"
@@ -109,6 +109,8 @@ class BottleMetadata:
# for state dirs written before PRD 0040; callers default to "docker" # for state dirs written before PRD 0040; callers default to "docker"
# for backward compatibility. # for backward compatibility.
backend: str = "" backend: str = ""
label: str = ""
color: str = ""
def metadata_path(identity: str) -> Path: def metadata_path(identity: str) -> Path:
@@ -144,6 +146,8 @@ def read_metadata(identity: str) -> BottleMetadata | None:
started_at=str(raw_typed.get("started_at", "")), started_at=str(raw_typed.get("started_at", "")),
compose_project=str(raw_typed.get("compose_project", "")), compose_project=str(raw_typed.get("compose_project", "")),
backend=str(raw_typed.get("backend", "")), backend=str(raw_typed.get("backend", "")),
label=str(raw_typed.get("label", "")),
color=str(raw_typed.get("color", "")),
) )
+40 -5
View File
@@ -3,12 +3,47 @@
from __future__ import annotations from __future__ import annotations
import argparse import argparse
import os
import sys import sys
from ..backend import enumerate_active_agents from ..backend import enumerate_active_agents
from ..manifest import Manifest from ..manifest import Manifest
from ._common import PROG, USER_CWD from ._common import PROG, USER_CWD
_ANSI_COLOR_CODES: dict[str, str] = {
"black": "\033[30m",
"red": "\033[31m",
"green": "\033[32m",
"yellow": "\033[33m",
"blue": "\033[34m",
"magenta": "\033[35m",
"cyan": "\033[36m",
"white": "\033[37m",
"bright-black": "\033[90m",
"bright-red": "\033[91m",
"bright-green": "\033[92m",
"bright-yellow": "\033[93m",
"bright-blue": "\033[94m",
"bright-magenta": "\033[95m",
"bright-cyan": "\033[96m",
"bright-white": "\033[97m",
}
_ANSI_RESET = "\033[0m"
def _ansi_label(text: str, color: str) -> str:
if not color:
return text
if not sys.stdout.isatty():
return text
term = os.environ.get("TERM", "")
if term in ("dumb", ""):
return text
code = _ANSI_COLOR_CODES.get(color)
if not code:
return text
return f"{code}{text}{_ANSI_RESET}"
def cmd_list(argv: list[str]) -> int: def cmd_list(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} list", add_help=True) parser = argparse.ArgumentParser(prog=f"{PROG} list", add_help=True)
@@ -27,11 +62,11 @@ def cmd_list(argv: list[str]) -> int:
if not active: if not active:
print("no active bot-bottle bottles", file=sys.stderr) print("no active bot-bottle bottles", file=sys.stderr)
return 0 return 0
# One line per bottle: `<backend>\t<slug>\t<agent>\t<status>`. # One line per bottle: `<backend>\t<slug>\t<label>\t<services>`.
# Tab-separated keeps the format stable for shell pipelines; # Tab-separated keeps the format stable for shell pipelines.
# the dashboard renders the same data through its own
# formatter.
for b in active: for b in active:
services = ",".join(b.services) if b.services else "-" services = ",".join(b.services) if b.services else "-"
print(f"{b.backend_name}\t{b.slug}\t{b.agent_name}\t{services}") display_name = b.label if b.label else b.agent_name
colored_name = _ansi_label(display_name, b.color)
print(f"{b.backend_name}\t{b.slug}\t{colored_name}\t{services}")
return 0 return 0
+1 -1
View File
@@ -18,7 +18,7 @@ from __future__ import annotations
import argparse import argparse
from ..backend import BottleSpec from ..backend import BottleSpec
from ..backend.docker.bottle_state import read_metadata from ..bottle_state import read_metadata
from ..log import die from ..log import die
from ..manifest import Manifest from ..manifest import Manifest
from ._common import PROG, USER_CWD from ._common import PROG, USER_CWD
+14 -14
View File
@@ -24,12 +24,12 @@ from ..backend import (
known_backend_names, known_backend_names,
) )
from ..backend.docker.bottle_plan import DockerBottlePlan from ..backend.docker.bottle_plan import DockerBottlePlan
from ..backend.docker.bottle_state import ( from ..bottle_state import (
cleanup_state, cleanup_state,
is_preserved, is_preserved,
mark_preserved, mark_preserved,
) )
from ..backend.docker.capability_apply import snapshot_transcript # from ..backend.docker.capability_apply import snapshot_transcript
from ..log import info from ..log import info
from ..manifest import Manifest from ..manifest import Manifest
from ._common import PROG, USER_CWD, read_tty_line from ._common import PROG, USER_CWD, read_tty_line
@@ -39,7 +39,7 @@ from . import tui
def cmd_start(argv: list[str]) -> int: def cmd_start(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True) parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
parser.add_argument("--dry-run", action="store_true") parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into a derived image") parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
parser.add_argument("--remote-control", action="store_true") parser.add_argument("--remote-control", action="store_true")
parser.add_argument( parser.add_argument(
"--backend", "--backend",
@@ -47,7 +47,7 @@ def cmd_start(argv: list[str]) -> int:
default=None, default=None,
help=( help=(
"backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND " "backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND "
"or 'docker'). Overrides the env var when set." "or host auto-selection). Overrides the env var when set."
), ),
) )
parser.add_argument( parser.add_argument(
@@ -72,19 +72,16 @@ def cmd_start(argv: list[str]) -> int:
return 0 return 0
backend_name: str | None = args.backend backend_name: str | None = args.backend
if backend_name is None and "BOT_BOTTLE_BACKEND" not in os.environ:
backend_name = tui.filter_select( label, color = tui.name_color_modal(default_label=agent_name)
list(known_backend_names()),
title="Select backend",
)
if backend_name is None:
return 0
spec = BottleSpec( spec = BottleSpec(
manifest=manifest, manifest=manifest,
agent_name=agent_name, agent_name=agent_name,
copy_cwd=args.cwd, copy_cwd=args.cwd,
user_cwd=USER_CWD, user_cwd=USER_CWD,
label=label,
color=color,
) )
return _launch_bottle( return _launch_bottle(
spec, spec,
@@ -110,8 +107,8 @@ def prepare_with_preflight(
injected callable, prompt y/N via the injected callable. injected callable, prompt y/N via the injected callable.
`backend_name` selects which backend prepares the plan `backend_name` selects which backend prepares the plan
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). The CLI passes (`None` → `$BOT_BOTTLE_BACKEND` → host auto-selection). The CLI
whatever `--backend` resolved to. passes whatever `--backend` resolved to.
Returns `(plan, identity)`. `plan` is None on dry-run or Returns `(plan, identity)`. `plan` is None on dry-run or
operator-N, but `identity` is set as soon as `backend.prepare` operator-N, but `identity` is set as soon as `backend.prepare`
@@ -136,6 +133,7 @@ def prepare_with_preflight(
def attach_agent( def attach_agent(
bottle: Bottle, *, remote_control: bool = False, resume: bool = False, bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
agent_provider_template: str = "claude", agent_provider_template: str = "claude",
startup_args: tuple[str, ...] = (),
) -> int: ) -> int:
"""Run the selected provider CLI inside `bottle` as an """Run the selected provider CLI inside `bottle` as an
interactive session. Blocks until the session ends; returns the interactive session. Blocks until the session ends; returns the
@@ -154,6 +152,7 @@ def attach_agent(
agent_args = list(runtime.bypass_args) agent_args = list(runtime.bypass_args)
if remote_control: if remote_control:
agent_args.extend(runtime.remote_control_args) agent_args.extend(runtime.remote_control_args)
agent_args.extend(startup_args)
if resume: if resume:
agent_args.extend(runtime.resume_args) agent_args.extend(runtime.resume_args)
return bottle.exec_agent(agent_args, tty=True) return bottle.exec_agent(agent_args, tty=True)
@@ -168,7 +167,7 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
# instead of relying on each agent's transcript layout. # instead of relying on each agent's transcript layout.
if not identity: if not identity:
return return
snapshot_transcript(identity) # snapshot_transcript(identity)
if exit_code != 0: if exit_code != 0:
mark_preserved(identity) mark_preserved(identity)
@@ -238,6 +237,7 @@ def _launch_bottle(
bottle, bottle,
remote_control=remote_control, remote_control=remote_control,
agent_provider_template=agent_provider_template, agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args,
) )
info( info(
f"session ended (exit {exit_code}); " f"session ended (exit {exit_code}); "
+21 -17
View File
@@ -20,12 +20,17 @@ from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from .. import supervise as _supervise from .. import supervise as _supervise
from ..backend.docker.bottle_state import read_metadata # from ..bottle_state import read_metadata
from ..backend.docker.capability_apply import ( # from ..backend.docker.capability_apply import (
CapabilityApplyError, # CapabilityApplyError,
apply_capability_change, # apply_capability_change,
) # )
from ..log import Die, error, info from ..log import Die, error, info
class CapabilityApplyError(RuntimeError):
"""Placeholder while capability_apply is disabled."""
from ..supervise import ( from ..supervise import (
COMPONENT_FOR_TOOL, COMPONENT_FOR_TOOL,
AuditEntry, AuditEntry,
@@ -124,20 +129,19 @@ def approve(
) -> None: ) -> None:
"""Apply the proposal, write the waiting response, and audit it.""" """Apply the proposal, write the waiting response, and audit it."""
status = STATUS_MODIFIED if final_file is not None else STATUS_APPROVED status = STATUS_MODIFIED if final_file is not None else STATUS_APPROVED
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
diff_before, diff_after = "", "" diff_before, diff_after = "", ""
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK: # if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
_meta = read_metadata(qp.proposal.bottle_slug) # _meta = read_metadata(qp.proposal.bottle_slug)
if _meta is not None and not _meta.compose_project: # if _meta is not None and not _meta.compose_project:
raise CapabilityApplyError( # raise CapabilityApplyError(
"capability-block remediation is not supported for smolmachines " # "capability-block remediation is not supported for smolmachines "
"bottles. Reject this proposal or handle the capability change " # "bottles. Reject this proposal or handle the capability change "
"manually, then restart the bottle." # "manually, then restart the bottle."
) # )
diff_before, diff_after = apply_capability_change( # diff_before, diff_after = apply_capability_change(
qp.proposal.bottle_slug, file_to_apply, # qp.proposal.bottle_slug, file_to_apply,
) # )
response = Response( response = Response(
proposal_id=qp.proposal.id, proposal_id=qp.proposal.id,
+217
View File
@@ -3,6 +3,7 @@
Exposed surface: Exposed surface:
filter_select(items, *, title="", tty_path="/dev/tty") -> str | None filter_select(items, *, title="", tty_path="/dev/tty") -> str | None
name_color_modal(default_label, *, tty_path="/dev/tty") -> (str, str)
Opens /dev/tty directly so the picker works even when stdout/stdin are Opens /dev/tty directly so the picker works even when stdout/stdin are
redirected. Returns the selected item or None on cancel. redirected. Returns the selected item or None on cancel.
@@ -218,3 +219,219 @@ def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.
screen.addstr(row, col, text, attr) screen.addstr(row, col, text, attr)
except curses.error: except curses.error:
pass pass
# ---------------------------------------------------------------------------
# name_color_modal — two-step label + color picker
# ---------------------------------------------------------------------------
_ANSI_COLORS = [
"red", "green", "blue", "yellow", "magenta", "cyan", "white", "black",
"bright-red", "bright-green", "bright-blue", "bright-yellow",
"bright-magenta", "bright-cyan", "bright-white", "bright-black",
]
_CURSES_COLOR_MAP: dict[str, int] = {
"black": curses.COLOR_BLACK,
"red": curses.COLOR_RED,
"green": curses.COLOR_GREEN,
"yellow": curses.COLOR_YELLOW,
"blue": curses.COLOR_BLUE,
"magenta": curses.COLOR_MAGENTA,
"cyan": curses.COLOR_CYAN,
"white": curses.COLOR_WHITE,
}
_COLOR_NONE = "(none)"
def name_color_modal(
default_label: str,
*,
tty_path: str = "/dev/tty",
) -> tuple[str, str]:
"""Present a two-step curses modal: first edit the agent label,
then optionally pick a color.
Returns ``(label, color)`` where ``color`` is one of the 16 ANSI
color name strings or ``""`` for no color. Falls back to
``(default_label, "")`` on any error (terminal too small, not a tty).
"""
try:
tty_fd = open(tty_path, "r+b", buffering=0) # pylint: disable=consider-using-with
except OSError:
return default_label, ""
try:
fd_dup = os.dup(tty_fd.fileno())
return _run_name_color(default_label, tty_fd=fd_dup)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return default_label, ""
finally:
tty_fd.close()
def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
import io
orig_stdin = sys.__stdin__
orig_stdout = sys.__stdout__
try:
tty_text = io.TextIOWrapper(io.FileIO(tty_fd, mode="r+"), write_through=True)
sys.__stdin__ = tty_text # type: ignore[assignment]
sys.__stdout__ = tty_text # type: ignore[assignment]
os.environ.setdefault("TERM", "xterm-256color")
screen = curses.initscr()
curses.noecho()
curses.cbreak()
screen.keypad(True)
try:
label = _label_step(screen, default_label)
color = _color_step(screen, label)
finally:
screen.keypad(False)
curses.nocbreak()
curses.echo()
curses.endwin()
finally:
sys.__stdin__ = orig_stdin # type: ignore[assignment]
sys.__stdout__ = orig_stdout # type: ignore[assignment]
return label, color
def _label_step(screen: Any, default_label: str) -> str:
"""Step 1: edit the label. First printable key replaces the
pre-fill; subsequent keys append. Enter confirms."""
text = default_label
replaced = False # True once the user has typed their first char
while True:
_render_label(screen, text)
try:
key = screen.getch()
except KeyboardInterrupt:
return default_label
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
return text.strip() or default_label
if key in (curses.KEY_BACKSPACE, _KEY_BACKSPACE_WIN, 127):
if replaced:
text = text[:-1]
else:
text = ""
replaced = True
elif 32 <= key <= 126:
if not replaced:
text = chr(key)
replaced = True
else:
text += chr(key)
def _render_label(screen: Any, text: str) -> None:
screen.erase()
rows, cols = screen.getmaxyx()
sep = "" * min(cols - 1, 40)
_addstr_safe(screen, 0, 0, "Name agent", curses.A_BOLD)
_addstr_safe(screen, 1, 0, sep)
_addstr_safe(screen, 2, 0, text[:cols - 1], curses.A_REVERSE)
_addstr_safe(screen, 3, 0, sep)
if rows > 5:
_addstr_safe(screen, 5, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
screen.refresh()
def _color_step(screen: Any, confirmed_label: str) -> str:
"""Step 2: pick a color from the list, or skip."""
items = [_COLOR_NONE] + _ANSI_COLORS
cursor = 0
# Initialise color pairs once; index 0 = none, 1..16 = palette.
color_attrs = _init_color_pairs()
while True:
_render_color(screen, items, cursor, confirmed_label, color_attrs)
try:
key = screen.getch()
except KeyboardInterrupt:
return ""
if key in (ord("q"), _KEY_ESC):
return ""
if key in (curses.KEY_ENTER, _KEY_ENTER_ALT, ord("\r")):
chosen = items[cursor]
return "" if chosen == _COLOR_NONE else chosen
if key in (curses.KEY_UP, ord("k")) and cursor > 0:
cursor -= 1
elif key in (curses.KEY_DOWN, ord("j")) and cursor < len(items) - 1:
cursor += 1
def _init_color_pairs() -> dict[str, int]:
"""Return {color_name: curses_attr} for the palette items."""
attrs: dict[str, int] = {_COLOR_NONE: curses.A_NORMAL}
try:
curses.start_color()
curses.use_default_colors()
pair_idx = 2 # pair 1 reserved for other uses
for name in _ANSI_COLORS:
base = name.replace("bright-", "")
fg = _CURSES_COLOR_MAP.get(base, curses.COLOR_WHITE)
try:
curses.init_pair(pair_idx, fg, -1)
attr = curses.color_pair(pair_idx)
if name.startswith("bright-"):
attr |= curses.A_BOLD
attrs[name] = attr
pair_idx += 1
except curses.error:
attrs[name] = curses.A_NORMAL
except curses.error:
for name in _ANSI_COLORS:
attrs[name] = curses.A_NORMAL
return attrs
def _render_color(
screen: Any,
items: list[str],
cursor: int,
confirmed_label: str,
color_attrs: dict[str, int],
) -> None:
screen.erase()
rows, cols = screen.getmaxyx()
sep = "" * min(cols - 1, 40)
_addstr_safe(screen, 0, 0, "Name agent", curses.A_BOLD)
_addstr_safe(screen, 1, 0, sep)
_addstr_safe(screen, 2, 0, confirmed_label[:cols - 1])
_addstr_safe(screen, 3, 0, sep)
_addstr_safe(screen, 4, 0, "Color (optional)", curses.A_BOLD)
list_start = 5
list_rows = rows - list_start - 2
scroll = max(0, cursor - list_rows + 1)
visible = items[scroll: scroll + list_rows]
for idx, name in enumerate(visible):
abs_idx = scroll + idx
row = list_start + idx
if row >= rows - 2:
break
prefix = "> " if abs_idx == cursor else " "
attr = color_attrs.get(name, curses.A_NORMAL)
if abs_idx == cursor:
attr |= curses.A_REVERSE
_addstr_safe(screen, row, 0, (prefix + name)[:cols - 1], attr)
_addstr_safe(screen, rows - 2, 0, sep)
_addstr_safe(
screen, rows - 1, 0,
"[↑↓/jk] move [Enter] select [Esc/q] skip",
curses.A_DIM,
)
screen.refresh()
@@ -36,7 +36,7 @@ RUN apt-get update \
# build (`claude --version` returns 2.1.126). Bump deliberately when # build (`claude --version` returns 2.1.126). Bump deliberately when
# rolling forward; an unpinned install would mean rebuilds silently pick # rolling forward; an unpinned install would mean rebuilds silently pick
# up new behavior. # up new behavior.
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.126 \ RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
&& npm cache clean --force && npm cache clean --force
# Run as a non-root user. The node image already provides a `node` user # Run as a non-root user. The node image already provides a `node` user
+125 -11
View File
@@ -17,9 +17,11 @@ from typing import TYPE_CHECKING
from ...agent_provider import ( from ...agent_provider import (
AgentProvider, AgentProvider,
AgentProviderRuntime, AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionFile, AgentProvisionFile,
AgentProvisionPlan, AgentProvisionPlan,
) )
from ...backend.docker import util as docker_mod
from ...egress import EgressRoute from ...egress import EgressRoute
from ...log import die, info, warn from ...log import die, info, warn
@@ -28,8 +30,6 @@ if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan from ...backend import Bottle, BottlePlan
_REPO_ROOT = Path(__file__).resolve().parents[3]
_SUPERVISE_MCP_NAME = "supervise" _SUPERVISE_MCP_NAME = "supervise"
@@ -40,11 +40,75 @@ def _skills_dir(guest_home: str) -> str:
def _prompt_path(guest_home: str) -> str: def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt" return f"{guest_home}/.bot-bottle-prompt.txt"
_STATUS_LINE_COLORS = {
"black": "\033[30m",
"red": "\033[31m",
"green": "\033[32m",
"yellow": "\033[33m",
"blue": "\033[34m",
"magenta": "\033[35m",
"cyan": "\033[36m",
"white": "\033[37m",
"bright-black": "\033[90m",
"bright-red": "\033[91m",
"bright-green": "\033[92m",
"bright-yellow": "\033[93m",
"bright-blue": "\033[94m",
"bright-magenta": "\033[95m",
"bright-cyan": "\033[96m",
"bright-white": "\033[97m",
}
_CLAUDE_THEME_COLORS = {
"black": "black",
"red": "red",
"green": "green",
"yellow": "yellow",
"blue": "blue",
"magenta": "magenta",
"cyan": "cyan",
"white": "white",
"bright-black": "blackBright",
"bright-red": "redBright",
"bright-green": "greenBright",
"bright-yellow": "yellowBright",
"bright-blue": "blueBright",
"bright-magenta": "magentaBright",
"bright-cyan": "cyanBright",
"bright-white": "whiteBright",
}
def _status_line_script(label: str, color: str) -> str:
if not label:
return "#!/bin/sh\nprintf '\\n'\n"
label_q = shlex.quote(label)
if color and color in _STATUS_LINE_COLORS:
return (
"#!/bin/sh\n"
f"printf '%b%s%b\\n' '{_STATUS_LINE_COLORS[color]}' {label_q} '\\033[0m'\n"
)
return f"#!/bin/sh\nprintf '%s\\n' {label_q}\n"
def _custom_theme_payload(color: str) -> dict[str, object] | None:
theme_color = _CLAUDE_THEME_COLORS.get(color)
if not theme_color:
return None
return {
"name": f"Bot-bottle {color}",
"base": "dark",
"overrides": {
"claude": f"ansi:{theme_color}",
},
}
_RUNTIME = AgentProviderRuntime( _RUNTIME = AgentProviderRuntime(
template="claude", template="claude",
command="claude", command="claude",
image="bot-bottle-claude:latest", image="bot-bottle-claude:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
prompt_mode="append_file", prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",), bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",), resume_args=("--continue",),
@@ -62,34 +126,78 @@ class ClaudeAgentProvider(AgentProvider):
*, *,
dockerfile: str, dockerfile: str,
state_dir: Path, state_dir: Path,
guest_home: str, instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None, guest_env: dict[str, str] | None = None,
auth_token: str = "", auth_token: str = "",
forward_host_credentials: bool = False, forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None, host_env: dict[str, str] | None = None,
trusted_project_path: str = "", trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan: ) -> AgentProvisionPlan:
del forward_host_credentials, host_env # Codex-only knobs del forward_host_credentials, host_env, provider_settings
resolved_guest_env = dict(guest_env or {}) resolved_guest_env = dict(guest_env or {})
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = { env_vars: dict[str, str] = {
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1", "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
"DISABLE_ERROR_REPORTING": "1", "DISABLE_ERROR_REPORTING": "1",
} }
dirs = (
AgentProvisionDir(f"{guest_home}/.claude"),
AgentProvisionDir(f"{guest_home}/.claude/themes"),
)
claude_config = state_dir / "claude.json" claude_config = state_dir / "claude.json"
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}} claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True} claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
claude_config.write_text(json.dumps({ payload: dict[str, object] = {
"hasCompletedOnboarding": True, "hasCompletedOnboarding": True,
"theme": "dark", "theme": "dark",
"bypassPermissionsModeAccepted": True, "bypassPermissionsModeAccepted": True,
"projects": claude_projects, "projects": claude_projects,
}, indent=2) + "\n") }
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
claude_config.chmod(0o600) claude_config.chmod(0o600)
files = ( files = [
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"), AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
) ]
claude_settings = state_dir / "claude-settings.json"
claude_settings_payload: dict[str, object] = {}
if label or color:
statusline_script = state_dir / "claude-statusline.sh"
statusline_script.write_text(_status_line_script(label, color))
statusline_script.chmod(0o755)
files.append(AgentProvisionFile(
statusline_script,
f"{guest_home}/.claude/statusline.sh",
mode="755",
))
claude_settings_payload["statusLine"] = {
"type": "command",
"command": "~/.claude/statusline.sh",
}
theme_payload = _custom_theme_payload(color)
if theme_payload is not None:
theme_name = f"bot-bottle-{docker_mod.slugify(label or color)}"
theme_file = state_dir / f"{theme_name}.json"
theme_file.write_text(json.dumps(theme_payload, indent=2) + "\n")
theme_file.chmod(0o644)
files.append(AgentProvisionFile(
theme_file,
f"{guest_home}/.claude/themes/{theme_name}.json",
))
claude_settings_payload["theme"] = f"custom:{theme_name}"
if claude_settings_payload:
claude_settings.write_text(json.dumps(claude_settings_payload, indent=2) + "\n")
claude_settings.chmod(0o600)
files.append(AgentProvisionFile(
claude_settings,
f"{guest_home}/.claude/settings.json",
))
egress_routes = (EgressRoute( egress_routes = (EgressRoute(
host="api.anthropic.com", host="api.anthropic.com",
auth_scheme="Bearer" if auth_token else "", auth_scheme="Bearer" if auth_token else "",
@@ -100,15 +208,21 @@ class ClaudeAgentProvider(AgentProvider):
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder" env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"}) hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
return AgentProvisionPlan( return AgentProvisionPlan(
template=_RUNTIME.template, template=_RUNTIME.template,
command=_RUNTIME.command, command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode, prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image, image=_RUNTIME.image,
dockerfile=dockerfile, dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
env_vars=env_vars, env_vars=env_vars,
guest_env=resolved_guest_env, guest_env=resolved_guest_env,
files=files, has_prompt=has_prompt,
dirs=dirs,
files=tuple(files),
egress_routes=egress_routes, egress_routes=egress_routes,
hidden_env_names=hidden_env_names, hidden_env_names=hidden_env_names,
) )
@@ -149,7 +263,7 @@ class ClaudeAgentProvider(AgentProvider):
user="root", user="root",
) )
agent = plan.spec.manifest.agents[plan.spec.agent_name] agent = plan.spec.manifest.agents[plan.spec.agent_name]
return prompt_path if agent.prompt else None return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None: def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
"""Apply the claude-side declarative provision steps from """Apply the claude-side declarative provision steps from
+20 -7
View File
@@ -18,8 +18,8 @@ from ...agent_provider import (
CODEX_HOST_CREDENTIAL_HOSTS, CODEX_HOST_CREDENTIAL_HOSTS,
AgentProvider, AgentProvider,
AgentProviderRuntime, AgentProviderRuntime,
AgentProvisionCommand,
AgentProvisionDir, AgentProvisionDir,
AgentProvisionCommand,
AgentProvisionFile, AgentProvisionFile,
AgentProvisionPlan, AgentProvisionPlan,
) )
@@ -32,8 +32,6 @@ if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan from ...backend import Bottle, BottlePlan
_REPO_ROOT = Path(__file__).resolve().parents[3]
_SUPERVISE_MCP_NAME = "supervise" _SUPERVISE_MCP_NAME = "supervise"
@@ -48,11 +46,11 @@ def _skills_dir(guest_home: str) -> str:
def _prompt_path(guest_home: str) -> str: def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt" return f"{guest_home}/.bot-bottle-prompt.txt"
_RUNTIME = AgentProviderRuntime( _RUNTIME = AgentProviderRuntime(
template="codex", template="codex",
command="codex", command="codex",
image="bot-bottle-codex:latest", image="bot-bottle-codex:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
prompt_mode="read_prompt_file", prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",), bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"), resume_args=("resume", "--last"),
@@ -70,15 +68,20 @@ class CodexAgentProvider(AgentProvider):
*, *,
dockerfile: str, dockerfile: str,
state_dir: Path, state_dir: Path,
guest_home: str, instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None, guest_env: dict[str, str] | None = None,
auth_token: str = "", auth_token: str = "",
forward_host_credentials: bool = False, forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None, host_env: dict[str, str] | None = None,
trusted_project_path: str = "", trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan: ) -> AgentProvisionPlan:
del auth_token # Claude-only knob del auth_token, label, color, provider_settings
resolved_guest_env = dict(guest_env or {}) resolved_guest_env = dict(guest_env or {})
guest_home = self.guest_home
trusted_path = trusted_project_path or guest_home trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = { env_vars: dict[str, str] = {
@@ -100,6 +103,11 @@ class CodexAgentProvider(AgentProvider):
config_file.write_text( config_file.write_text(
f'[projects."{toml_path}"]\n' f'[projects."{toml_path}"]\n'
'trust_level = "trusted"\n' 'trust_level = "trusted"\n'
"\n"
"[tui]\n"
'status_line = ["model-with-reasoning"]\n'
'terminal_title = ["spinner", "project"]\n'
'theme = "ansi"\n'
) )
config_file.chmod(0o600) config_file.chmod(0o600)
files.append(AgentProvisionFile(config_file, config_path)) files.append(AgentProvisionFile(config_file, config_path))
@@ -142,14 +150,19 @@ class CodexAgentProvider(AgentProvider):
"guest, but Codex did not accept it" "guest, but Codex did not accept it"
))) )))
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
return AgentProvisionPlan( return AgentProvisionPlan(
template=_RUNTIME.template, template=_RUNTIME.template,
command=_RUNTIME.command, command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode, prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image, image=_RUNTIME.image,
dockerfile=dockerfile, dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
env_vars=env_vars, env_vars=env_vars,
guest_env=resolved_guest_env, guest_env=resolved_guest_env,
has_prompt=has_prompt,
dirs=tuple(dirs), dirs=tuple(dirs),
files=tuple(files), files=tuple(files),
pre_copy=tuple(pre_copy), pre_copy=tuple(pre_copy),
@@ -194,7 +207,7 @@ class CodexAgentProvider(AgentProvider):
user="root", user="root",
) )
agent = plan.spec.manifest.agents[plan.spec.agent_name] agent = plan.spec.manifest.agents[plan.spec.agent_name]
return prompt_path if agent.prompt else None return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None: def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
"""Apply the codex-side declarative provision steps from """Apply the codex-side declarative provision steps from
@@ -2,7 +2,13 @@
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
them using the Gitea deploy-key HTTP API. No new Python dependencies them using the Gitea deploy-key HTTP API. No new Python dependencies
only stdlib `urllib.request` and `subprocess`.""" only stdlib `urllib.request` and `subprocess`.
Required token permissions (Gitea "Applications" "Generate Token"):
- Repository: Read & Write
Grants POST /api/v1/repos/{owner}/{repo}/keys (create deploy key)
and DELETE /api/v1/repos/{owner}/{repo}/keys/{id} (revoke deploy key).
No other scopes are needed."""
from __future__ import annotations from __future__ import annotations
+41
View File
@@ -0,0 +1,41 @@
# bot-bottle Pi provider image.
#
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
fd-find \
ripgrep \
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
&& npm cache clean --force
RUN mkdir -p /home/node/.pi/agent \
/home/node/.pi/context-mode/sessions \
/tmp/pi-subagents-uid-1000 \
&& chown -R node:node /home/node/.pi /tmp \
&& chmod -R u+rwX /tmp \
&& chown root:root /tmp /var/tmp \
&& chmod 1777 /tmp /var/tmp
USER node
WORKDIR /home/node
RUN pi install npm:@harms-haus/pi-cwd \
&& pi install npm:pi-web-access \
&& pi install npm:context-mode \
&& pi install npm:pi-subagents \
&& pi install npm:pi-mcp-adapter
CMD ["pi"]
+1
View File
@@ -0,0 +1 @@
"""Pi agent provider package."""
+319
View File
@@ -0,0 +1,319 @@
"""Pi agent provider plugin (PRD 0058, contrib).
Pi uses ~/.pi/agent/models.json for custom provider/model settings.
This provider writes an Ollama-compatible default configuration and
lets bottles override the model endpoint and model ids via
agent_provider.settings.
"""
from __future__ import annotations
import json
import os
import shlex
from pathlib import Path
from typing import TYPE_CHECKING
from urllib.parse import urlparse
from ...agent_provider import (
AgentProvider,
AgentProviderRuntime,
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
)
from ...egress import EgressRoute
from ...log import die, info
if TYPE_CHECKING:
from ...backend import Bottle, BottlePlan
_DEFAULT_BASE_URL = "http://ollama:11434/v1"
_DEFAULT_MODEL = "qwen2.5-coder:7b"
_DEFAULT_PROVIDER_NAME = "ollama"
_DEFAULT_CONTEXT_WINDOW = 4096
_DEFAULT_MAX_TOKENS = 1024
def _skills_dir(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/skills"
def _prompt_path(guest_home: str) -> str:
return f"{guest_home}/.bot-bottle-prompt.txt"
def _append_system_path(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/APPEND_SYSTEM.md"
def _models_path(guest_home: str) -> str:
return f"{guest_home}/.pi/agent/models.json"
def _runtime_state_repair_script(guest_home: str) -> str:
home = shlex.quote(guest_home)
pi_home = shlex.quote(f"{guest_home}/.pi")
context_sessions = shlex.quote(f"{guest_home}/.pi/context-mode/sessions")
return (
f"mkdir -p {context_sessions} /tmp/pi-subagents-uid-1000 && "
f"chown node:node {home} && "
f"chown -R node:node {pi_home} /tmp && "
"chmod -R u+rwX /tmp && "
f"chmod 755 {home} && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp"
)
def _settings_value(
settings: dict[str, object],
key: str,
default: object,
) -> object:
value = settings.get(key)
return default if value is None else value
def _settings_int(
settings: dict[str, object],
key: str,
default: int,
) -> int:
value = _settings_value(settings, key, default)
if isinstance(value, bool):
return default
if isinstance(value, (int, str)):
return int(value)
return default
def _pi_models_json(
settings: dict[str, object],
) -> tuple[dict[str, object], str, str, list[str], str]:
provider_name = str(
_settings_value(settings, "provider", _DEFAULT_PROVIDER_NAME)
)
base_url = str(_settings_value(settings, "base_url", _DEFAULT_BASE_URL))
api = str(_settings_value(settings, "api", "openai-completions"))
api_key = settings.get("api_key")
api_key_env = str(settings.get("api_key_env", ""))
models_raw = _settings_value(settings, "models", [_DEFAULT_MODEL])
models = [str(model) for model in models_raw] # type: ignore[union-attr]
supports_developer_role = bool(
_settings_value(settings, "supports_developer_role", False)
)
supports_reasoning_effort = bool(
_settings_value(settings, "supports_reasoning_effort", False)
)
max_tokens_field = str(
_settings_value(settings, "max_tokens_field", "max_tokens")
)
context_window = _settings_int(
settings, "context_window", _DEFAULT_CONTEXT_WINDOW,
)
max_tokens = _settings_int(settings, "max_tokens", _DEFAULT_MAX_TOKENS)
input_context_window = max(1, context_window - max_tokens)
provider: dict[str, object] = {
"baseUrl": base_url,
"api": api,
"compat": {
"supportsDeveloperRole": supports_developer_role,
"supportsReasoningEffort": supports_reasoning_effort,
"maxTokensField": max_tokens_field,
},
"models": [
{
"id": model,
"name": model,
"contextWindow": input_context_window,
"maxTokens": max_tokens,
}
for model in models
],
}
if api_key is not None:
provider["apiKey"] = str(api_key)
elif api_key_env:
provider["apiKey"] = "egress-placeholder"
elif provider_name == _DEFAULT_PROVIDER_NAME:
provider["apiKey"] = "ollama"
payload: dict[str, object] = {
"providers": {
provider_name: provider,
}
}
return payload, base_url, api_key_env, models, provider_name
def _route_host(base_url: str) -> str:
parsed = urlparse(base_url)
if not parsed.scheme or not parsed.hostname:
die(
"agent provider provisioning: pi settings base_url must be an "
f"absolute URL (was {base_url!r})"
)
return parsed.hostname
_RUNTIME = AgentProviderRuntime(
template="pi",
command="pi",
image="bot-bottle-pi:latest",
prompt_mode="append_system_prompt",
bypass_args=(),
resume_args=(),
remote_control_args=(),
)
class PiAgentProvider(AgentProvider):
@property
def runtime(self) -> AgentProviderRuntime:
return _RUNTIME
def provision_plan(
self,
*,
dockerfile: str,
state_dir: Path,
instance_name: str,
prompt_file: Path,
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
label: str = "",
color: str = "",
provider_settings: dict[str, object] | None = None,
) -> AgentProvisionPlan:
del auth_token, forward_host_credentials, host_env, trusted_project_path
del label, color
resolved_guest_env = dict(guest_env or {})
guest_home = self.guest_home
settings = dict(provider_settings or {})
models_payload, base_url, api_key_env, models, provider_name = (
_pi_models_json(settings)
)
models_file = state_dir / "pi-models.json"
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
models_file.chmod(0o600)
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
auth_scheme = "Bearer" if api_key_env else ""
return AgentProvisionPlan(
template=_RUNTIME.template,
command=_RUNTIME.command,
prompt_mode=_RUNTIME.prompt_mode,
image=_RUNTIME.image,
dockerfile=dockerfile,
guest_home=guest_home,
instance_name=instance_name,
prompt_file=prompt_file,
guest_env=resolved_guest_env,
has_prompt=has_prompt,
startup_args=(
"--models",
",".join(f"{provider_name}/{model}" for model in models),
),
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
egress_routes=(EgressRoute(
host=_route_host(base_url),
auth_scheme=auth_scheme,
token_ref=api_key_env,
),),
)
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
from ...backend.util import host_skill_dir
agent = plan.spec.manifest.agents[plan.spec.agent_name]
if not agent.skills:
return
skills_dir = _skills_dir(plan.guest_home)
bottle.exec(f"mkdir -p {skills_dir}", user="root")
for name in agent.skills:
src = host_skill_dir(name)
if not os.path.isdir(src):
die(
f"skill {name!r} disappeared from host between "
f"validation and copy at {src}."
)
dst = f"{skills_dir}/{name}"
info(f"copying skill {name} into {bottle.name}:{dst}")
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
bottle.cp_in(f"{src}/.", f"{dst}/")
bottle.exec(f"chown -R node:node {dst}", user="root")
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
prompt_path = _prompt_path(plan.guest_home)
append_system_path = _append_system_path(plan.guest_home)
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
bottle.exec(
f"mkdir -p {shlex.quote(plan.guest_home)}/.pi/agent && "
f"cp {shlex.quote(prompt_path)} {shlex.quote(append_system_path)} && "
f"chown node:node {shlex.quote(prompt_path)} "
f"{shlex.quote(append_system_path)} && "
f"chmod 600 {shlex.quote(prompt_path)} "
f"{shlex.quote(append_system_path)}",
user="root",
)
# Pi's `--append-system-prompt` takes literal text, not a file path.
# Use its documented APPEND_SYSTEM.md discovery path instead.
return None
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
provision = plan.agent_provision
_exec(
bottle,
_runtime_state_repair_script(plan.guest_home),
"could not prepare pi runtime state",
)
for d in provision.dirs:
path = shlex.quote(d.guest_path)
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
_exec(
bottle,
f"chown {shlex.quote(d.owner)} {path}",
f"could not chown {d.guest_path}",
)
_exec(
bottle,
f"chmod {shlex.quote(d.mode)} {path}",
f"could not chmod {d.guest_path}",
)
for f in provision.files:
bottle.cp_in(str(f.host_path), f.guest_path)
path = shlex.quote(f.guest_path)
_exec(
bottle,
f"chown {shlex.quote(f.owner)} {path}",
f"could not chown {f.guest_path}",
)
_exec(
bottle,
f"chmod {shlex.quote(f.mode)} {path}",
f"could not chmod {f.guest_path}",
)
def provision_supervise_mcp(
self,
plan: "BottlePlan",
bottle: "Bottle",
supervise_url: str,
) -> None:
del plan, bottle, supervise_url
def _exec(bottle: "Bottle", script: str, error: str) -> None:
result = bottle.exec(script, user="root")
if result.returncode != 0:
detail = (result.stderr or result.stdout).strip()
if detail:
detail = f": {detail}"
die(f"agent provider provisioning: {error}{detail}")
+163 -38
View File
@@ -11,8 +11,10 @@ the same try/except import shim pattern.
from __future__ import annotations from __future__ import annotations
import base64 import base64
import gzip
import re import re
import typing import typing
import unicodedata
from urllib.parse import quote as url_quote from urllib.parse import quote as url_quote
try: try:
@@ -22,7 +24,39 @@ except ImportError: # pragma: no cover - host-side path
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Token patterns detector (Phase 1a) # Snippet helpers
# ---------------------------------------------------------------------------
SNIPPET_CONTEXT = 40 # chars of surrounding text to include on each side
REDACT = "********" # fixed-width replacement for the matched sensitive value
def _snippet(text: str, start: int, end: int) -> str:
"""Return context around a match with the matched span replaced by REDACT."""
before = text[max(0, start - SNIPPET_CONTEXT):start].replace("\n", " ").replace("\r", " ")
after = text[end:end + SNIPPET_CONTEXT].replace("\n", " ").replace("\r", " ")
return f"{before}{REDACT}{after}"
# ---------------------------------------------------------------------------
# Unicode normalization (defeats confusable-char and combining-mark evasion)
# ---------------------------------------------------------------------------
def _normalize_text(text: str) -> str:
# NFKD separates base characters from combining marks and resolves
# compatibility equivalents (fullwidth ASCII, ligatures, etc.)
decomposed = unicodedata.normalize("NFKD", text)
return "".join(
ch for ch in decomposed
# Strip combining marks inserted between chars to break patterns
if unicodedata.category(ch) != "Mn"
# Strip control chars; keep common whitespace (\n \r \t)
and (unicodedata.category(ch) != "Cc" or ch in "\n\r\t")
)
# ---------------------------------------------------------------------------
# Token patterns detector
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = ( TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
@@ -31,44 +65,95 @@ TOKEN_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
("GitHub fine-grained token", re.compile(r"github_pat_[A-Za-z0-9_]{82}")), ("GitHub fine-grained token", re.compile(r"github_pat_[A-Za-z0-9_]{82}")),
("Anthropic API key", re.compile(r"sk-ant-[A-Za-z0-9\-_]{93}")), ("Anthropic API key", re.compile(r"sk-ant-[A-Za-z0-9\-_]{93}")),
("OpenAI API key", re.compile(r"sk-[A-Za-z0-9]{48}")), ("OpenAI API key", re.compile(r"sk-[A-Za-z0-9]{48}")),
("OpenAI project API key", re.compile(r"sk-proj-[A-Za-z0-9_\-]{48,}")),
("Stripe live key", re.compile(r"sk_live_[A-Za-z0-9]{24}")), ("Stripe live key", re.compile(r"sk_live_[A-Za-z0-9]{24}")),
("Generic Bearer JWT", re.compile(r"Bearer\s+[A-Za-z0-9._\-]{50,}")), ("Generic Bearer JWT", re.compile(r"Bearer\s+[A-Za-z0-9._\-]{50,}")),
("HuggingFace token", re.compile(r"hf_[A-Za-z0-9]{34,}")),
("Databricks token", re.compile(r"dapi[A-Za-z0-9]{32}")),
("Slack token", re.compile(r"xox[baprs]-[A-Za-z0-9]+-[A-Za-z0-9]+-[A-Za-z0-9]{24,}")),
("npm token", re.compile(r"npm_[A-Za-z0-9]{36}")),
("SendGrid API key", re.compile(r"SG\.[A-Za-z0-9_\-]{22}\.[A-Za-z0-9_\-]{43}")),
("PyPI token", re.compile(r"pypi-[A-Za-z0-9_\-]{80,}")),
("HashiCorp Vault token", re.compile(r"hvs\.[A-Za-z0-9_\-]{24,}")),
) )
def scan_token_patterns(text: str) -> ScanResult | None: def scan_token_patterns(text: str, *, location: str = "body") -> ScanResult | None:
normalized = _normalize_text(text)
for name, pattern in TOKEN_PATTERNS: for name, pattern in TOKEN_PATTERNS:
if pattern.search(text): m = pattern.search(normalized)
if m is not None:
return ScanResult( return ScanResult(
severity="block", severity="block",
reason=f"outbound request contains {name}", reason=f"{name} found in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
) )
return None return None
def redact_tokens(
text: str,
*,
env: typing.Mapping[str, str] | None = None,
) -> str:
"""Replace token pattern matches and (if env given) provisioned secrets with REDACT."""
for _, pattern in TOKEN_PATTERNS:
text = pattern.sub(REDACT, text)
if env is not None:
for key, value in env.items():
if key.startswith("EGRESS_TOKEN_") and value:
for variant in _encoded_variants(value):
text = text.replace(variant, REDACT)
return text
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Known secrets detector (Phase 1b) # Known secrets detector (Phase 1b)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _encoded_variants(secret: str) -> list[str]: def _encoded_variants(secret: str) -> list[str]:
"""Return the secret plus base64, URL-encoded, and hex variants.""" """Return the secret plus common encoded variants for exfil detection."""
variants = [secret] seen: set[str] = {secret}
variants: list[str] = [secret]
def _add(v: str) -> None:
if v not in seen:
seen.add(v)
variants.append(v)
secret_bytes = secret.encode("utf-8") secret_bytes = secret.encode("utf-8")
# Standard base64 — with and without padding
b64 = base64.b64encode(secret_bytes).decode("ascii") b64 = base64.b64encode(secret_bytes).decode("ascii")
if b64 != secret: _add(b64)
variants.append(b64) _add(b64.rstrip("="))
url_enc = url_quote(secret, safe="")
if url_enc != secret: # URL-safe base64 (JWT/OAuth use -_ alphabet) — with and without padding
variants.append(url_enc) b64url = base64.urlsafe_b64encode(secret_bytes).decode("ascii")
hex_enc = secret_bytes.hex() _add(b64url)
if hex_enc != secret: _add(b64url.rstrip("="))
variants.append(hex_enc)
# URL percent-encoding
_add(url_quote(secret, safe=""))
# Hex — lowercase and uppercase
_add(secret_bytes.hex())
_add(secret_bytes.hex().upper())
# Base32 (TOTP seeds, some DNS-exfil channels)
_add(base64.b32encode(secret_bytes).decode("ascii"))
# gzip + base64 (deterministic: mtime=0); recognisable by H4sI prefix
_add(base64.b64encode(gzip.compress(secret_bytes, mtime=0)).decode("ascii"))
return variants return variants
def scan_known_secrets( def scan_known_secrets(
text: str, text: str,
*, *,
location: str = "body",
env: typing.Mapping[str, str] | None = None, env: typing.Mapping[str, str] | None = None,
) -> ScanResult | None: ) -> ScanResult | None:
if env is None: if env is None:
@@ -77,13 +162,13 @@ def scan_known_secrets(
if not key.startswith("EGRESS_TOKEN_") or not value: if not key.startswith("EGRESS_TOKEN_") or not value:
continue continue
for variant in _encoded_variants(value): for variant in _encoded_variants(value):
if variant in text: pos = text.find(variant)
if pos >= 0:
return ScanResult( return ScanResult(
severity="block", severity="block",
reason=( reason=f"provisioned secret from {key} found in {location}",
f"outbound request contains provisioned secret " location=location,
f"from {key}" context=_snippet(text, pos, pos + len(variant)),
),
) )
return None return None
@@ -112,54 +197,94 @@ JAILBREAK_PHRASES: tuple[re.Pattern[str], ...] = (
PROXIMITY_CHARS = 500 PROXIMITY_CHARS = 500
def _min_distance( def _closest_pair(
a_matches: list[re.Match[str]], a_matches: list[re.Match[str]],
b_matches: list[re.Match[str]], b_matches: list[re.Match[str]],
) -> int | None: ) -> tuple[re.Match[str], re.Match[str]] | None:
"""Smallest char distance between any pair of matches.""" """Return the pair (a, b) with the smallest character gap, or None."""
if not a_matches or not b_matches: best: tuple[re.Match[str], re.Match[str]] | None = None
return None best_gap: int | None = None
best = None
for a in a_matches: for a in a_matches:
for b in b_matches: for b in b_matches:
gap = max(0, max(a.start(), b.start()) - min(a.end(), b.end())) gap = max(0, max(a.start(), b.start()) - min(a.end(), b.end()))
if best is None or gap < best: if best_gap is None or gap < best_gap:
best = gap best_gap = gap
best = (a, b)
return best return best
def scan_naive_injection(text: str) -> ScanResult | None: def scan_naive_injection(text: str) -> ScanResult | None:
location = "response body"
disclosure_hits = [m for p in DISCLOSURE_PHRASES for m in p.finditer(text)] disclosure_hits = [m for p in DISCLOSURE_PHRASES for m in p.finditer(text)]
jailbreak_hits = [m for p in JAILBREAK_PHRASES for m in p.finditer(text)] jailbreak_hits = [m for p in JAILBREAK_PHRASES for m in p.finditer(text)]
if disclosure_hits and jailbreak_hits: if disclosure_hits and jailbreak_hits:
dist = _min_distance(disclosure_hits, jailbreak_hits) pair = _closest_pair(disclosure_hits, jailbreak_hits)
if dist is not None and dist <= PROXIMITY_CHARS: if pair is not None:
return ScanResult( dist = max(0, max(pair[0].start(), pair[1].start()) - min(pair[0].end(), pair[1].end()))
severity="block", if dist <= PROXIMITY_CHARS:
reason=( first = pair[0] if pair[0].start() <= pair[1].start() else pair[1]
f"disclosure and jailbreak phrases within " return ScanResult(
f"{dist} chars in response" severity="block",
), reason=(
) f"disclosure and jailbreak phrases within "
f"{dist} chars in {location}"
),
location=location,
context=_snippet(text, first.start(), first.end()),
)
if disclosure_hits: if disclosure_hits:
m = disclosure_hits[0]
return ScanResult( return ScanResult(
severity="warn", severity="warn",
reason="prompt disclosure phrase detected in response", reason=f"prompt disclosure phrase detected in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
) )
if jailbreak_hits: if jailbreak_hits:
m = jailbreak_hits[0]
return ScanResult( return ScanResult(
severity="warn", severity="warn",
reason="jailbreak phrase detected in response", reason=f"jailbreak phrase detected in {location}",
location=location,
context=_snippet(text, m.start(), m.end()),
) )
return None return None
# ---------------------------------------------------------------------------
# CRLF injection detector
# ---------------------------------------------------------------------------
# URL-encoded CRLF is never legitimate in a request URL or header value.
_CRLF_ENCODED_RE = re.compile(r"%0[dD]%0[aA]", re.ASCII)
# Literal CRLF followed by a header-name pattern indicates header injection.
_CRLF_HEADER_INJECT_RE = re.compile(r"\r\n[A-Za-z][A-Za-z0-9\-]+\s*:", re.ASCII)
def scan_crlf_injection(text: str) -> ScanResult | None:
if _CRLF_ENCODED_RE.search(text):
return ScanResult(
severity="block",
reason="URL-encoded CRLF (%0d%0a) in outbound request",
)
if _CRLF_HEADER_INJECT_RE.search(text):
return ScanResult(
severity="block",
reason="CRLF header injection pattern in outbound request",
)
return None
__all__ = [ __all__ = [
"REDACT",
"SNIPPET_CONTEXT",
"TOKEN_PATTERNS", "TOKEN_PATTERNS",
"redact_tokens",
"scan_crlf_injection",
"scan_known_secrets", "scan_known_secrets",
"scan_naive_injection", "scan_naive_injection",
"scan_token_patterns", "scan_token_patterns",
+56 -39
View File
@@ -24,7 +24,7 @@ from .egress_addon_core import (
from .log import die from .log import die
if TYPE_CHECKING: if TYPE_CHECKING:
from .manifest import Bottle from .manifest import ManifestBottle
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN" CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
@@ -62,10 +62,11 @@ class EgressPlan:
egress_network: str = "" egress_network: str = ""
mitmproxy_ca_host_path: Path = Path() mitmproxy_ca_host_path: Path = Path()
mitmproxy_ca_cert_only_host_path: Path = Path() mitmproxy_ca_cert_only_host_path: Path = Path()
log: int = 0
def egress_manifest_routes( def egress_manifest_routes(
bottle: Bottle, bottle: ManifestBottle,
) -> tuple[EgressRoute, ...]: ) -> tuple[EgressRoute, ...]:
out: list[EgressRoute] = [] out: list[EgressRoute] = []
for r in bottle.egress.routes: for r in bottle.egress.routes:
@@ -90,6 +91,7 @@ def egress_manifest_routes(
auth_scheme=r.AuthScheme, auth_scheme=r.AuthScheme,
token_ref=r.TokenRef, token_ref=r.TokenRef,
roles=r.Role, roles=r.Role,
git_fetch=r.GitFetch,
outbound_detectors=r.OutboundDetectors, outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors, inbound_detectors=r.InboundDetectors,
)) ))
@@ -97,7 +99,7 @@ def egress_manifest_routes(
def egress_routes_for_bottle( def egress_routes_for_bottle(
bottle: Bottle, bottle: ManifestBottle,
provider_routes: tuple[EgressRoute, ...] = (), provider_routes: tuple[EgressRoute, ...] = (),
) -> tuple[EgressRoute, ...]: ) -> tuple[EgressRoute, ...]:
manifest = egress_manifest_routes(bottle) manifest = egress_manifest_routes(bottle)
@@ -172,6 +174,8 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
entry_data["headers"] = headers_data entry_data["headers"] = headers_data
matches_data.append(entry_data) matches_data.append(entry_data)
fields["matches"] = matches_data fields["matches"] = matches_data
if r.git_fetch:
fields["git"] = {"fetch": True}
if r.outbound_detectors is not None or r.inbound_detectors is not None: if r.outbound_detectors is not None or r.inbound_detectors is not None:
dlp: dict[str, object] = {} dlp: dict[str, object] = {}
if r.outbound_detectors is not None: if r.outbound_detectors is not None:
@@ -188,12 +192,48 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
return fields return fields
def _render_match_entry(entry: dict[str, object]) -> list[str]:
lines: list[str] = []
first_key = True
if "paths" in entry:
lines.append(" - paths:")
first_key = False
for pd in entry["paths"]: # type: ignore[union-attr]
pd_dict: dict[str, str] = pd # type: ignore[assignment]
if "type" in pd_dict:
lines.append(f' - type: "{pd_dict["type"]}"')
lines.append(f' value: "{pd_dict["value"]}"')
else:
lines.append(f' - value: "{pd_dict["value"]}"')
if "methods" in entry:
methods_str = ", ".join(f'"{m}"' for m in entry["methods"]) # type: ignore[union-attr]
prefix = " - " if first_key else " "
lines.append(f'{prefix}methods: [{methods_str}]')
first_key = False
if "headers" in entry:
prefix = " - " if first_key else " "
lines.append(f"{prefix}headers:")
first_key = False
for hd in entry["headers"]: # type: ignore[union-attr]
hd_dict: dict[str, str] = hd # type: ignore[assignment]
lines.append(f' - name: "{hd_dict["name"]}"')
lines.append(f' value: "{hd_dict["value"]}"')
if first_key:
lines.append(" - {}")
return lines
def egress_render_routes( def egress_render_routes(
routes: tuple[EgressRoute, ...], routes: tuple[EgressRoute, ...],
*,
log: int = 0,
) -> str: ) -> str:
lines: list[str] = ["routes:"] lines: list[str] = []
if log:
lines.append(f"log: {log}")
lines.append("routes:")
if not routes: if not routes:
lines[0] = "routes: []" lines[-1] = "routes: []"
return "\n".join(lines) + "\n" return "\n".join(lines) + "\n"
for r in routes: for r in routes:
f = _route_to_yaml_fields(r) f = _route_to_yaml_fields(r)
@@ -203,38 +243,13 @@ def egress_render_routes(
lines.append(f' token_env: "{f["token_env"]}"') lines.append(f' token_env: "{f["token_env"]}"')
if "matches" in f: if "matches" in f:
lines.append(" matches:") lines.append(" matches:")
for entry in f["matches"]: # type: ignore for entry in f["matches"]: # type: ignore[union-attr]
entry_dict: dict[str, object] = entry # type: ignore lines.extend(_render_match_entry(entry)) # type: ignore[arg-type]
first_key = True if "git" in f:
if "paths" in entry_dict: git_dict: dict[str, object] = f["git"] # type: ignore
lines.append(" - paths:") lines.append(" git:")
first_key = False if git_dict.get("fetch") is True:
for pd in entry_dict["paths"]: # type: ignore lines.append(" fetch: true")
pd_dict: dict[str, str] = pd # type: ignore
if "type" in pd_dict:
lines.append(f' - type: "{pd_dict["type"]}"')
lines.append(f' value: "{pd_dict["value"]}"')
else:
lines.append(f' - value: "{pd_dict["value"]}"')
if "methods" in entry_dict:
methods_str = ", ".join(
f'"{m}"' for m in entry_dict["methods"] # type: ignore
)
prefix = " - " if first_key else " "
lines.append(f'{prefix}methods: [{methods_str}]')
first_key = False
if "headers" in entry_dict:
prefix = " - " if first_key else " "
lines.append(f"{prefix}headers:")
first_key = False
for hd in entry_dict["headers"]: # type: ignore
hd_dict: dict[str, str] = hd # type: ignore
lines.append(f' - name: "{hd_dict["name"]}"')
lines.append(f' value: "{hd_dict["value"]}"')
if "type" in hd_dict:
lines.append(f' type: "{hd_dict["type"]}"')
if first_key:
lines.append(" - {}")
if "dlp" in f: if "dlp" in f:
dlp_dict: dict[str, object] = f["dlp"] # type: ignore dlp_dict: dict[str, object] = f["dlp"] # type: ignore
lines.append(" dlp:") lines.append(" dlp:")
@@ -273,20 +288,22 @@ def egress_resolve_token_values(
class Egress(ABC): class Egress(ABC):
def prepare( def prepare(
self, self,
bottle: Bottle, bottle: ManifestBottle,
slug: str, slug: str,
stage_dir: Path, stage_dir: Path,
provider_routes: tuple[EgressRoute, ...] = (), provider_routes: tuple[EgressRoute, ...] = (),
) -> EgressPlan: ) -> EgressPlan:
routes = egress_routes_for_bottle(bottle, provider_routes) routes = egress_routes_for_bottle(bottle, provider_routes)
log = bottle.egress.Log
routes_path = stage_dir / "egress_routes.yaml" routes_path = stage_dir / "egress_routes.yaml"
routes_path.write_text(egress_render_routes(routes)) routes_path.write_text(egress_render_routes(routes, log=log))
routes_path.chmod(0o600) routes_path.chmod(0o600)
return EgressPlan( return EgressPlan(
slug=slug, slug=slug,
routes_path=routes_path, routes_path=routes_path,
routes=routes, routes=routes,
token_env_map=egress_token_env_map(routes), token_env_map=egress_token_env_map(routes),
log=log,
) )
__all__ = [ __all__ = [
+157 -32
View File
@@ -12,18 +12,30 @@ import signal
import sys import sys
from pathlib import Path from pathlib import Path
from mitmproxy import http # type: ignore[import-not-found] from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
from egress_addon_core import ( # type: ignore[import-not-found] from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
Route, LOG_BLOCKS,
LOG_FULL,
Config,
build_inbound_scan_text,
build_outbound_scan_text,
decide, decide,
decide_git_fetch,
is_git_fetch_request,
is_git_push_request, is_git_push_request,
load_routes, load_config,
match_route, match_route,
outbound_scan_headers,
scan_inbound, scan_inbound,
scan_outbound, scan_outbound,
) )
try:
from dlp_detectors import redact_tokens # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.dlp_detectors import redact_tokens # type: ignore[import-not-found]
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml" DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
@@ -33,26 +45,28 @@ INTROSPECT_HOST = "_egress.local"
class EgressAddon: class EgressAddon:
def __init__(self) -> None: def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH) self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.routes: tuple[Route, ...] = () self.config: Config = Config(routes=())
self._reload(initial=True) self._reload(initial=True)
self._install_sighup() self._install_sighup()
def _reload(self, *, initial: bool = False) -> None: def _reload(self, *, initial: bool = False) -> None:
try: try:
text = Path(self.routes_path).read_text(encoding="utf-8") text = Path(self.routes_path).read_text(encoding="utf-8")
new_routes = load_routes(text) new_config = load_config(text)
except (OSError, ValueError) as e: except (OSError, ValueError) as e:
tag = "boot" if initial else "SIGHUP" tag = "boot" if initial else "SIGHUP"
sys.stderr.write( sys.stderr.write(
f"egress: {tag} load failed: {e}\n" f"egress: {tag} load failed: {e}\n"
) )
if initial: if initial:
self.routes = () self.config = Config(routes=())
return return
self.routes = new_routes self.config = new_config
log_label = ("off", "blocks", "full")[self.config.log]
sys.stderr.write( sys.stderr.write(
f"egress: loaded {len(self.routes)} route(s): " f"egress: loaded {len(self.config.routes)} route(s): "
f"{', '.join(r.host for r in self.routes)}\n" f"{', '.join(r.host for r in self.config.routes)}"
f" [log={log_label}]\n"
) )
def _install_sighup(self) -> None: def _install_sighup(self) -> None:
@@ -68,7 +82,7 @@ class EgressAddon:
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None: def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
if path == "/allowlist": if path == "/allowlist":
payload = json.dumps( payload = json.dumps(
{"routes": [dataclasses.asdict(r) for r in self.routes]}, {"routes": [dataclasses.asdict(r) for r in self.config.routes]},
indent=2, indent=2,
).encode("utf-8") ).encode("utf-8")
flow.response = http.Response.make( flow.response = http.Response.make(
@@ -82,14 +96,55 @@ class EgressAddon:
{"Content-Type": "text/plain; charset=utf-8"}, {"Content-Type": "text/plain; charset=utf-8"},
) )
def _block(self, flow: http.HTTPFlow, reason: str) -> None: def _req_ctx(self, flow: http.HTTPFlow) -> dict[str, object]:
sys.stderr.write(f"{reason}\n") return {
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
}
def _block(
self,
flow: http.HTTPFlow,
reason: str,
ctx: dict[str, object] | None = None,
) -> None:
if self.config.log >= LOG_BLOCKS:
entry: dict[str, object] = {"event": "egress_block", "reason": reason}
if ctx:
entry.update(ctx)
sys.stderr.write(json.dumps(entry) + "\n")
flow.response = http.Response.make( flow.response = http.Response.make(
403, 403,
reason.encode("utf-8"), reason.encode("utf-8"),
{"Content-Type": "text/plain; charset=utf-8"}, {"Content-Type": "text/plain; charset=utf-8"},
) )
def _log_request(self, flow: http.HTTPFlow) -> None:
sys.stderr.write(
json.dumps({
"event": "egress_request",
"host": redact_tokens(flow.request.pretty_host, env=os.environ),
"method": flow.request.method,
"path": redact_tokens(flow.request.path, env=os.environ),
"headers": dict(flow.request.headers),
"body": flow.request.get_text(strict=False) or "",
})
+ "\n"
)
def _log_response(self, flow: http.HTTPFlow) -> None:
sys.stderr.write(
json.dumps({
"event": "egress_response",
"host": flow.request.pretty_host,
"status": flow.response.status_code,
"headers": dict(flow.response.headers),
"body": flow.response.get_text(strict=False) or "",
})
+ "\n"
)
def request(self, flow: http.HTTPFlow) -> None: def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?") request_path, _, query = flow.request.path.partition("?")
@@ -98,36 +153,57 @@ class EgressAddon:
return return
# DLP outbound scan BEFORE stripping auth — catches tokens the # DLP outbound scan BEFORE stripping auth — catches tokens the
# agent tried to smuggle in the Authorization header. # agent tried to smuggle in any header, path, query param, or body.
route = match_route(self.routes, flow.request.pretty_host) # Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
if route is not None: if route is not None:
body = flow.request.get_text(strict=False) or "" body = flow.request.get_text(strict=False) or ""
auth_header = flow.request.headers.get("authorization", "") scan_text = build_outbound_scan_text(
scan_text = body flow.request.pretty_host,
if auth_header: request_path,
scan_text = auth_header + "\n" + body query,
outbound_scan_headers(route, dict(flow.request.headers)),
body,
)
dlp_result = scan_outbound(route, scan_text, os.environ) dlp_result = scan_outbound(route, scan_text, os.environ)
if dlp_result is not None and dlp_result.severity == "block": if dlp_result is not None and dlp_result.severity == "block":
self._block(flow, f"egress DLP: {dlp_result.reason}") ctx = self._req_ctx(flow)
if dlp_result.context:
ctx = {**ctx, "context": dlp_result.context}
self._block(flow, f"egress DLP: {dlp_result.reason}", ctx=ctx)
return return
# Strip inbound Authorization — agent cannot smuggle tokens.
flow.request.headers.pop("authorization", None)
if is_git_push_request(request_path, query): if is_git_push_request(request_path, query):
self._block( self._block(
flow, flow,
"egress: git push over HTTPS is not supported; " "egress: git push over HTTPS is not supported; "
"use the bottle.git SSH path (gitleaks-scanned by " "use the bottle.git SSH path (gitleaks-scanned by "
"git-gate's pre-receive hook).", "git-gate's pre-receive hook).",
ctx=self._req_ctx(flow),
) )
return return
if is_git_fetch_request(request_path, query):
git_decision = decide_git_fetch(
self.config.routes, flow.request.pretty_host,
)
if git_decision.action == "block":
self._block(
flow,
git_decision.reason,
ctx=self._req_ctx(flow),
)
return
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject sidecar-owned auth below.
flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation # Build headers mapping for match evaluation
req_headers = {k.lower(): v for k, v in flow.request.headers.items()} req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
decision = decide( decision = decide(
self.routes, self.config.routes,
flow.request.pretty_host, flow.request.pretty_host,
request_path, request_path,
os.environ, os.environ,
@@ -136,29 +212,78 @@ class EgressAddon:
) )
if decision.action == "block": if decision.action == "block":
self._block(flow, decision.reason) self._block(flow, decision.reason, ctx=self._req_ctx(flow))
return return
if decision.inject_authorization is not None: if decision.inject_authorization is not None:
flow.request.headers["authorization"] = decision.inject_authorization flow.request.headers["authorization"] = decision.inject_authorization
if self.config.log >= LOG_FULL:
self._log_request(flow)
def response(self, flow: http.HTTPFlow) -> None: def response(self, flow: http.HTTPFlow) -> None:
"""DLP inbound scan on response bodies (PRD 0053).""" """DLP inbound scan on response headers and body."""
route = match_route(self.routes, flow.request.pretty_host) route = match_route(self.config.routes, flow.request.pretty_host)
if route is None: if route is None:
return return
if flow.response is None: if flow.response is None:
return return
if self.config.log >= LOG_FULL:
self._log_response(flow)
resp_headers = {k.lower(): v for k, v in flow.response.headers.items()}
body = flow.response.get_text(strict=False) or "" body = flow.response.get_text(strict=False) or ""
if not body: scan_text = build_inbound_scan_text(resp_headers, body)
if not scan_text:
return return
result = scan_inbound(route, body) result = scan_inbound(route, scan_text)
if result is None: if result is None:
return return
resp_ctx: dict[str, object] = {
**self._req_ctx(flow),
"response_status": flow.response.status_code,
}
if result.context:
resp_ctx = {**resp_ctx, "context": result.context}
if result.severity == "block": if result.severity == "block":
self._block(flow, f"egress DLP: {result.reason}") self._block(flow, f"egress DLP: {result.reason}", ctx=resp_ctx)
elif result.severity == "warn": elif result.severity == "warn" and self.config.log >= LOG_BLOCKS:
sys.stderr.write(f"egress DLP warn: {result.reason}\n") sys.stderr.write(
json.dumps({
"event": "egress_warn",
"reason": f"egress DLP: {result.reason}",
**resp_ctx,
})
+ "\n"
)
def websocket_message(self, flow: http.HTTPFlow) -> None:
"""DLP scan on WebSocket frames.
Outbound frames (from_client) are scanned for credential leakage;
inbound frames are scanned for prompt injection. On a block the
entire connection is killed there is no HTTP response surface to
write to after the upgrade.
"""
if flow.websocket is None: # type: ignore[union-attr]
return
route = match_route(self.config.routes, flow.request.pretty_host)
if route is None:
return
message = flow.websocket.messages[-1] # type: ignore[union-attr]
content = message.content.decode("utf-8", errors="replace")
if message.from_client:
result = scan_outbound(route, content, os.environ)
if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
else:
result = scan_inbound(route, content)
if result is not None:
if result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
flow.kill() # type: ignore[union-attr]
elif result.severity == "warn":
sys.stderr.write(f"egress DLP warn: {result.reason}\n")
addons = [EgressAddon()] addons = [EgressAddon()]
+179 -6
View File
@@ -66,10 +66,22 @@ class Route:
matches: tuple[MatchEntry, ...] = () matches: tuple[MatchEntry, ...] = ()
auth_scheme: str = "" auth_scheme: str = ""
token_env: str = "" token_env: str = ""
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None inbound_detectors: tuple[str, ...] | None = None
LOG_OFF = 0 # no logging
LOG_BLOCKS = 1 # log block/warn events with request context
LOG_FULL = 2 # log block/warn events + full request and response bodies
@dataclass(frozen=True)
class Config:
routes: tuple[Route, ...]
log: int = LOG_OFF
@dataclass(frozen=True) @dataclass(frozen=True)
class Decision: class Decision:
action: str # "forward" or "block" action: str # "forward" or "block"
@@ -81,6 +93,8 @@ class Decision:
class ScanResult: class ScanResult:
severity: str # "block" or "warn" severity: str # "block" or "warn"
reason: str reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -303,16 +317,35 @@ def _parse_one(idx: int, raw: object) -> Route:
f"token_env={token_env!r})" f"token_env={token_env!r})"
) )
# git-over-HTTPS policy
git_fetch = False
git_raw = raw_dict.get("git")
if git_raw is not None:
if not isinstance(git_raw, dict):
raise ValueError(f"{label} ({host}): 'git' must be an object")
git_dict: dict[str, object] = typing.cast(dict[str, object], git_raw)
fetch_raw = git_dict.get("fetch", False)
if fetch_raw is True or fetch_raw is False:
git_fetch = fetch_raw
else:
raise ValueError(f"{label} ({host}): 'git.fetch' must be a boolean")
for k in git_dict:
if k != "fetch":
raise ValueError(
f"{label} ({host}): git has unknown key {k!r}; "
"accepted key is 'fetch'"
)
# dlp detectors # dlp detectors
outbound_detectors, inbound_detectors = _parse_detectors( outbound_detectors, inbound_detectors = _parse_detectors(
idx, host, raw_dict, idx, host, raw_dict,
) )
for k in raw_dict: for k in raw_dict:
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp"): if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git"):
raise ValueError( raise ValueError(
f"{label} ({host}): unknown key {k!r}; accepted keys " f"{label} ({host}): unknown key {k!r}; accepted keys "
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp'" f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git'"
) )
return Route( return Route(
@@ -320,6 +353,7 @@ def _parse_one(idx: int, raw: object) -> Route:
matches=matches, matches=matches,
auth_scheme=auth_scheme, auth_scheme=auth_scheme,
token_env=token_env, token_env=token_env,
git_fetch=git_fetch,
outbound_detectors=outbound_detectors, outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors, inbound_detectors=inbound_detectors,
) )
@@ -334,6 +368,32 @@ def load_routes(text: str) -> tuple[Route, ...]:
return parse_routes(payload) return parse_routes(payload)
def parse_config(payload: object) -> "Config":
"""Parse a full egress config payload (top-level log level + routes)."""
if not isinstance(payload, dict):
raise ValueError("routes payload: top-level must be an object")
payload_dict: dict[str, object] = typing.cast(dict[str, object], payload)
log_raw: object = payload_dict.get("log", LOG_OFF)
if log_raw is True or log_raw is False or not isinstance(log_raw, int) \
or log_raw not in (LOG_OFF, LOG_BLOCKS, LOG_FULL):
raise ValueError(
f"routes payload: 'log' must be {LOG_OFF}, {LOG_BLOCKS}, or {LOG_FULL}"
)
routes = parse_routes(payload)
return Config(routes=routes, log=log_raw)
def load_config(text: str) -> "Config":
"""Parse YAML text → Config (routes + log flag)."""
try:
payload = parse_yaml_subset(text)
except YamlSubsetError as e:
raise ValueError(f"routes payload: invalid YAML: {e}") from e
return parse_config(payload)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Match evaluation # Match evaluation
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -411,6 +471,17 @@ def is_git_push_request(path: str, query: str) -> bool:
return False return False
def is_git_fetch_request(path: str, query: str) -> bool:
if path.endswith("/git-upload-pack"):
return True
if path.endswith("/info/refs"):
for pair in query.split("&"):
k, _, v = pair.partition("=")
if k == "service" and v == "git-upload-pack":
return True
return False
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Route lookup + decision # Route lookup + decision
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -431,6 +502,7 @@ def decide(
request_host: str, request_host: str,
request_path: str, request_path: str,
environ: typing.Mapping[str, str], environ: typing.Mapping[str, str],
*,
request_method: str = "GET", request_method: str = "GET",
request_headers: typing.Mapping[str, str] | None = None, request_headers: typing.Mapping[str, str] | None = None,
) -> Decision: ) -> Decision:
@@ -473,10 +545,86 @@ def decide(
return Decision(action="forward") return Decision(action="forward")
def decide_git_fetch(
routes: typing.Sequence[Route],
request_host: str,
) -> Decision:
route = match_route(routes, request_host)
if route is not None and route.git_fetch:
return Decision(action="forward")
return Decision(
action="block",
reason=(
"egress: git fetch/clone over HTTPS is not allowed by default; "
"use git-gate for declared repos or set "
"egress.routes[].git.fetch=true for explicit read-only "
"HTTPS Git access."
),
)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# DLP scan dispatch (PRD 0053) # DLP scan dispatch (PRD 0053)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def build_outbound_scan_text(
host: str,
path: str,
query: str,
headers: typing.Mapping[str, str],
body: str,
) -> str:
"""Assemble all outbound request surfaces into one string for DLP scanning.
Covers hostname (DNS tunnelling), path, query params, all headers, body.
"""
parts: list[str] = [host, path]
if query:
parts.append(query)
for name, value in headers.items():
parts.append(f"{name}: {value}")
if body:
parts.append(body)
return "\n".join(parts)
def outbound_scan_headers(
route: Route,
headers: typing.Mapping[str, str],
) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP.
Routes that inject sidecar-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what
reaches the upstream.
"""
out: dict[str, str] = {}
skip_auth = bool(route.auth_scheme and route.token_env)
for name, value in headers.items():
if skip_auth and name.lower() == "authorization":
continue
out[name] = value
return out
def build_inbound_scan_text(
headers: typing.Mapping[str, str],
body: str,
) -> str:
"""Assemble inbound response surfaces into one string for DLP scanning.
Covers all response headers plus body.
"""
parts: list[str] = []
for name, value in headers.items():
parts.append(f"{name}: {value}")
if body:
parts.append(body)
return "\n".join(parts)
def _detector_enabled( def _detector_enabled(
configured: tuple[str, ...] | None, configured: tuple[str, ...] | None,
name: str, name: str,
@@ -496,19 +644,33 @@ def scan_outbound(
# Lazy import to avoid circular deps and keep dlp_detectors optional # Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file). # at import time (the sidecar copies it flat alongside this file).
try: try:
from dlp_detectors import scan_token_patterns, scan_known_secrets # type: ignore[import-not-found] from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_known_secrets,
scan_token_patterns,
)
except ImportError: # pragma: no cover - host-side path except ImportError: # pragma: no cover - host-side path
from .dlp_detectors import scan_token_patterns, scan_known_secrets # type: ignore[import-not-found] from .dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
scan_known_secrets,
scan_token_patterns,
)
text = body if isinstance(body, str) else body.decode("utf-8", errors="replace") text = body if isinstance(body, str) else body.decode("utf-8", errors="replace")
# CRLF injection is never legitimate — runs unconditionally, not gated
# by outbound_detectors config.
result = scan_crlf_injection(text)
if result is not None:
return result
if _detector_enabled(route.outbound_detectors, "token_patterns"): if _detector_enabled(route.outbound_detectors, "token_patterns"):
result = scan_token_patterns(text) result = scan_token_patterns(text, location="body")
if result is not None: if result is not None:
return result return result
if _detector_enabled(route.outbound_detectors, "known_secrets"): if _detector_enabled(route.outbound_detectors, "known_secrets"):
result = scan_known_secrets(text, env=environ) result = scan_known_secrets(text, location="body", env=environ)
if result is not None: if result is not None:
return result return result
@@ -535,17 +697,28 @@ def scan_inbound(
__all__ = [ __all__ = [
"LOG_BLOCKS",
"LOG_FULL",
"LOG_OFF",
"Config",
"Decision", "Decision",
"HeaderMatch", "HeaderMatch",
"MatchEntry", "MatchEntry",
"PathMatch", "PathMatch",
"Route", "Route",
"ScanResult", "ScanResult",
"build_inbound_scan_text",
"build_outbound_scan_text",
"decide", "decide",
"decide_git_fetch",
"evaluate_matches", "evaluate_matches",
"is_git_push_request", "is_git_push_request",
"is_git_fetch_request",
"load_config",
"load_routes", "load_routes",
"match_route", "match_route",
"outbound_scan_headers",
"parse_config",
"parse_routes", "parse_routes",
"scan_inbound", "scan_inbound",
"scan_outbound", "scan_outbound",
+48 -24
View File
@@ -37,7 +37,7 @@ from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from .log import info from .log import info
from .manifest import Bottle, GitEntry from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the sidecar bundle. The # Short network alias for git-gate inside the sidecar bundle. The
@@ -96,9 +96,9 @@ class GitGatePlan:
egress_network: str = "" egress_network: str = ""
def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]: def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstream, ...]:
"""Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name """Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name
validation already ran in `manifest.Bottle.from_dict`.""" validation already ran in `manifest.ManifestBottle.from_dict`."""
return tuple( return tuple(
GitGateUpstream( GitGateUpstream(
name=e.Name, name=e.Name,
@@ -113,7 +113,7 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
def git_gate_render_gitconfig( def git_gate_render_gitconfig(
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git", entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str: ) -> str:
"""Render the agent's ~/.gitconfig content for git-gate """Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm; `insteadOf` rewrites. Pure host-side, no docker / smolvm;
@@ -204,6 +204,7 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"", " git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"", " git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
" git -C \"$repo\" config receive.denyCurrentBranch ignore", " git -C \"$repo\" config receive.denyCurrentBranch ignore",
" git -C \"$repo\" config receive.advertisePushOptions true",
" git -C \"$repo\" config http.receivepack true", " git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"", " install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}", "}",
@@ -280,15 +281,32 @@ if [ ! -f "$hostsfile" ]; then
fi fi
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10" ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
push_option_count=${GIT_PUSH_OPTION_COUNT:-0}
case "$push_option_count" in
''|*[!0-9]*)
echo "git-gate: invalid GIT_PUSH_OPTION_COUNT=$push_option_count" >&2
exit 1
;;
esac
set --
i=0
while [ "$i" -lt "$push_option_count" ]; do
opt=$(printenv "GIT_PUSH_OPTION_$i" || :)
set -- "$@" --push-option="$opt"
i=$((i + 1))
done
while IFS=' ' read -r old new ref; do while IFS=' ' read -r old new ref; do
[ -z "$ref" ] && continue [ -z "$ref" ] && continue
if [ "$new" = "$zero" ]; then if [ "$new" = "$zero" ]; then
refspec=":$ref" refspec=":$ref"
elif [ "$old" != "$zero" ] && ! git merge-base --is-ancestor "$old" "$new" 2>/dev/null; then
refspec="+$new:$ref"
else else
refspec="$new:$ref" refspec="$new:$ref"
fi fi
echo "git-gate: forwarding $ref to origin" >&2 echo "git-gate: forwarding $ref to origin" >&2
if ! GIT_SSH_COMMAND="$ssh_cmd" git push origin "$refspec" 1>&2; then if ! GIT_SSH_COMMAND="$ssh_cmd" git push "$@" origin "$refspec" 1>&2; then
echo "git-gate: upstream push failed for $ref" >&2 echo "git-gate: upstream push failed for $ref" >&2
exit 1 exit 1
fi fi
@@ -361,7 +379,7 @@ exit 0
def _provision_dynamic_key( def _provision_dynamic_key(
entry: GitEntry, entry: ManifestGitEntry,
slug: str, slug: str,
stage_dir: Path, stage_dir: Path,
) -> str: ) -> str:
@@ -371,13 +389,12 @@ def _provision_dynamic_key(
Returns the host-side path to the private key file so the caller Returns the host-side path to the private key file so the caller
can inject it into the GitGateUpstream as `identity_file`.""" can inject it into the GitGateUpstream as `identity_file`."""
from .deploy_key_provisioner import get_provisioner from .deploy_key_provisioner import get_provisioner
pk = entry.ProvisionedKey pk = entry.Key
assert pk is not None token = os.environ.get(pk.forge_token_env)
token = os.environ.get(pk.token_env)
if token is None: if token is None:
raise RuntimeError( raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env" f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.token_env!r}: env var is not set" f" = {pk.forge_token_env!r}: env var is not set"
) )
api_url = pk.api_url or f"https://{entry.UpstreamHost}" api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url) provisioner = get_provisioner(pk.provider, token, api_url)
@@ -402,7 +419,7 @@ def _provision_dynamic_key(
return str(key_file) return str(key_file)
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None: def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) -> None:
"""Revoke all deploy keys provisioned for `bottle` during prepare. """Revoke all deploy keys provisioned for `bottle` during prepare.
Called at teardown after containers stop. Raises if any revocation Called at teardown after containers stop. Raises if any revocation
@@ -410,18 +427,18 @@ def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
address manually.""" address manually."""
from .deploy_key_provisioner import get_provisioner from .deploy_key_provisioner import get_provisioner
for entry in bottle.git: for entry in bottle.git:
if entry.ProvisionedKey is None: if entry.Key.provider != "gitea":
continue continue
pk = entry.ProvisionedKey pk = entry.Key
id_file = stage_dir / f"{entry.Name}-deploy-key-id" id_file = stage_dir / f"{entry.Name}-deploy-key-id"
if not id_file.exists(): if not id_file.exists():
continue continue
key_id = id_file.read_text().strip() key_id = id_file.read_text().strip()
token = os.environ.get(pk.token_env) token = os.environ.get(pk.forge_token_env)
if token is None: if token is None:
raise RuntimeError( raise RuntimeError(
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env" f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.token_env!r}: env var is not set;" f" = {pk.forge_token_env!r}: env var is not set;"
f" cannot revoke deploy key {key_id}" f" cannot revoke deploy key {key_id}"
) )
api_url = pk.api_url or f"https://{entry.UpstreamHost}" api_url = pk.api_url or f"https://{entry.UpstreamHost}"
@@ -434,18 +451,26 @@ def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]") info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path) -> str:
"""Return the host-side SSH identity file path for this entry.
For gitea entries, provisions a fresh deploy key first."""
if entry.Key.provider == "gitea":
return _provision_dynamic_key(entry, slug, stage_dir)
return entry.IdentityFile
class GitGate(ABC): class GitGate(ABC):
"""The per-agent git-gate. Encapsulates the host-side prepare """The per-agent git-gate. Encapsulates the host-side prepare
(upstream lift + entrypoint/hook render); the sidecar's (upstream lift + entrypoint/hook render); the sidecar's
start/stop lifecycle is backend-specific and lives on concrete start/stop lifecycle is backend-specific and lives on concrete
subclasses.""" subclasses."""
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> GitGatePlan: def prepare(self, bottle: ManifestBottle, slug: str, stage_dir: Path) -> GitGatePlan:
"""Compute the upstream table from `bottle.git` and write the """Compute the upstream table from `bottle.git` and write the
entrypoint, pre-receive hook, and access-hook scripts (mode entrypoint, pre-receive hook, and access-hook scripts (mode
600) under `stage_dir`. Pure host-side, no docker subprocess. 600) under `stage_dir`. Pure host-side, no docker subprocess.
For `provisioned_key` entries, also generates and registers For `gitea` key entries, also generates and registers
a fresh deploy key via the forge API and writes the private key a fresh deploy key via the forge API and writes the private key
+ key ID to `stage_dir`. + key ID to `stage_dir`.
@@ -454,11 +479,10 @@ class GitGate(ABC):
before passing the plan to `.start`.""" before passing the plan to `.start`."""
upstreams_list = list(git_gate_upstreams_for_bottle(bottle)) upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
for i, entry in enumerate(bottle.git): for i, entry in enumerate(bottle.git):
if entry.ProvisionedKey is not None: upstreams_list[i] = dataclasses.replace(
key_file = _provision_dynamic_key(entry, slug, stage_dir) upstreams_list[i],
upstreams_list[i] = dataclasses.replace( identity_file=_resolve_identity_file(entry, slug, stage_dir),
upstreams_list[i], identity_file=key_file )
)
upstreams = tuple(upstreams_list) upstreams = tuple(upstreams_list)
entrypoint = stage_dir / "git_gate_entrypoint.sh" entrypoint = stage_dir / "git_gate_entrypoint.sh"
entrypoint.write_text(git_gate_render_entrypoint(upstreams)) entrypoint.write_text(git_gate_render_entrypoint(upstreams))
+2 -2
View File
@@ -19,8 +19,8 @@ from urllib.parse import urlsplit
DEFAULT_PORT = 9420 DEFAULT_PORT = 9420
# Body-size cap matching supervise_server.py's 1 MiB limit. # Bound memory use while still allowing ordinary git push packfiles.
MAX_BODY_BYTES = 1 * 1024 * 1024 MAX_BODY_BYTES = 100 * 1024 * 1024
class GitHttpHandler(BaseHTTPRequestHandler): class GitHttpHandler(BaseHTTPRequestHandler):
+31 -30
View File
@@ -50,26 +50,27 @@ from pathlib import Path
from typing import Mapping from typing import Mapping
from .manifest_util import ManifestError, as_json_object from .manifest_util import ManifestError, as_json_object
from .manifest_agent import Agent, AgentProvider from .manifest_agent import ManifestAgent, ManifestAgentProvider
from .manifest_egress import ( from .manifest_egress import (
EGRESS_AUTH_SCHEMES, EGRESS_AUTH_SCHEMES,
EgressConfig, ManifestEgressConfig,
EgressRoute, ManifestEgressRoute,
) )
from .manifest_git import GitEntry, GitUser, parse_git_gate_config from .manifest_git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig, parse_git_gate_config
from .manifest_schema import BOTTLE_KEYS from .manifest_schema import BOTTLE_KEYS
# Re-export everything that callers currently import from this module. # Re-export everything that callers currently import from this module.
__all__ = [ __all__ = [
"ManifestError", "ManifestError",
"GitEntry", "ManifestGitEntry",
"GitUser", "ManifestGitUser",
"AgentProvider", "ManifestKeyConfig",
"ManifestAgentProvider",
"EGRESS_AUTH_SCHEMES", "EGRESS_AUTH_SCHEMES",
"EgressRoute", "ManifestEgressRoute",
"EgressConfig", "ManifestEgressConfig",
"Agent", "ManifestAgent",
"Bottle", "ManifestBottle",
"Manifest", "Manifest",
] ]
@@ -86,16 +87,16 @@ def _section_dict(value: object, label: str) -> dict[str, object]:
@dataclass(frozen=True) @dataclass(frozen=True)
class Bottle: class ManifestBottle:
env: Mapping[str, str] = field(default_factory=_empty_str_dict) env: Mapping[str, str] = field(default_factory=_empty_str_dict)
agent_provider: AgentProvider = field(default_factory=AgentProvider) agent_provider: ManifestAgentProvider = field(default_factory=ManifestAgentProvider)
git: tuple[GitEntry, ...] = () git: tuple[ManifestGitEntry, ...] = ()
# Per-bottle git identity (issue #86). Empty default — bottles # Per-bottle git identity (issue #86). Empty default — bottles
# that don't set `git-gate.user:` in the manifest skip the # that don't set `git-gate.user:` in the manifest skip the
# `git config --global` step entirely. A bottle can declare a user # `git config --global` step entirely. A bottle can declare a user
# identity without any git-gate.repos upstreams, and vice versa. # identity without any git-gate.repos upstreams, and vice versa.
git_user: GitUser = field(default_factory=GitUser) git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
egress: EgressConfig = field(default_factory=EgressConfig) egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
# Opt-in per-bottle stuck-recovery sidecar (PRD 0013). When true, # Opt-in per-bottle stuck-recovery sidecar (PRD 0013). When true,
# the launch step brings up a supervise sidecar that exposes MCP # the launch step brings up a supervise sidecar that exposes MCP
# tools to the agent (egress-block, capability-block) plus mounts # tools to the agent (egress-block, capability-block) plus mounts
@@ -105,7 +106,7 @@ class Bottle:
supervise: bool = False supervise: bool = False
@classmethod @classmethod
def from_dict(cls, name: str, raw: object) -> "Bottle": def from_dict(cls, name: str, raw: object) -> "ManifestBottle":
d = as_json_object(raw, f"bottle '{name}'") d = as_json_object(raw, f"bottle '{name}'")
if "runtime" in d: if "runtime" in d:
@@ -157,22 +158,22 @@ class Bottle:
) )
env[var] = value env[var] = value
git: tuple[GitEntry, ...] = () git: tuple[ManifestGitEntry, ...] = ()
git_user = GitUser() git_user = ManifestGitUser()
git_raw = d.get("git-gate") git_raw = d.get("git-gate")
if git_raw is not None: if git_raw is not None:
git, git_user = parse_git_gate_config(name, git_raw) git, git_user = parse_git_gate_config(name, git_raw)
agent_provider = ( agent_provider = (
AgentProvider.from_dict(name, d["agent_provider"]) ManifestAgentProvider.from_dict(name, d["agent_provider"])
if "agent_provider" in d if "agent_provider" in d
else AgentProvider() else ManifestAgentProvider()
) )
egress = ( egress = (
EgressConfig.from_dict(name, d["egress"]) ManifestEgressConfig.from_dict(name, d["egress"])
if "egress" in d if "egress" in d
else EgressConfig() else ManifestEgressConfig()
) )
supervise_raw = d.get("supervise", False) supervise_raw = d.get("supervise", False)
@@ -190,8 +191,8 @@ class Bottle:
@dataclass(frozen=True) @dataclass(frozen=True)
class Manifest: class Manifest:
bottles: Mapping[str, Bottle] bottles: Mapping[str, ManifestBottle]
agents: Mapping[str, Agent] agents: Mapping[str, ManifestAgent]
@classmethod @classmethod
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "Manifest": def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "Manifest":
@@ -305,8 +306,8 @@ class Manifest:
bottles = resolve_bottles(raw_bottles) bottles = resolve_bottles(raw_bottles)
bottle_names = set(bottles.keys()) bottle_names = set(bottles.keys())
agents: dict[str, Agent] = { agents: dict[str, ManifestAgent] = {
n: Agent.from_dict(n, a, bottle_names) for n, a in raw_agents.items() n: ManifestAgent.from_dict(n, a, bottle_names) for n, a in raw_agents.items()
} }
return cls(bottles=bottles, agents=agents) return cls(bottles=bottles, agents=agents)
@@ -338,7 +339,7 @@ class Manifest:
) )
raise ManifestError(f"bottle '{name}' not defined in bot-bottle.json (no bottles defined).") raise ManifestError(f"bottle '{name}' not defined in bot-bottle.json (no bottles defined).")
def _effective_git_user(self, agent_name: str) -> GitUser: def _effective_git_user(self, agent_name: str) -> ManifestGitUser:
"""Merge the agent's git.user over the referenced bottle's, """Merge the agent's git.user over the referenced bottle's,
per-field, agent-wins-on-non-empty (issue #94). Same overlay per-field, agent-wins-on-non-empty (issue #94). Same overlay
the `extends:` resolver applies between bottles the `extends:` resolver applies between bottles
@@ -348,12 +349,12 @@ class Manifest:
over = agent.git_user over = agent.git_user
if over.is_empty(): if over.is_empty():
return base return base
return GitUser( return ManifestGitUser(
name=over.name or base.name, name=over.name or base.name,
email=over.email or base.email, email=over.email or base.email,
) )
def bottle_for(self, agent_name: str) -> Bottle: def bottle_for(self, agent_name: str) -> ManifestBottle:
"""Resolve the Bottle the named agent references, with the """Resolve the Bottle the named agent references, with the
agent's git.user overlaid on top. The validator guarantees both agent's git.user overlaid on top. The validator guarantees both
lookups succeed for a manifest built via from_json_obj. lookups succeed for a manifest built via from_json_obj.
+105 -11
View File
@@ -2,17 +2,17 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass, field
from typing import cast from typing import cast
from .agent_provider import PROVIDER_TEMPLATES from .agent_provider import PROVIDER_TEMPLATES
from .manifest_util import ManifestError, as_json_object from .manifest_util import ManifestError, as_json_object
from .manifest_git import GitUser from .manifest_git import ManifestGitUser
from .manifest_schema import AGENT_MODEL_KEYS from .manifest_schema import AGENT_MODEL_KEYS
@dataclass(frozen=True) @dataclass(frozen=True)
class AgentProvider: class ManifestAgentProvider:
"""Provider/template for the agent process inside a bottle. """Provider/template for the agent process inside a bottle.
`template` selects a built-in launch/runtime contract. `dockerfile` `template` selects a built-in launch/runtime contract. `dockerfile`
@@ -33,15 +33,23 @@ class AgentProvider:
dockerfile: str = "" dockerfile: str = ""
auth_token: str = "" auth_token: str = ""
forward_host_credentials: bool = False forward_host_credentials: bool = False
settings: dict[str, object] = field(default_factory=dict)
@classmethod @classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider": def from_dict(cls, bottle_name: str, raw: object) -> "ManifestAgentProvider":
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider") d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
for k in d: for k in d:
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}: if k not in {
"template",
"dockerfile",
"auth_token",
"forward_host_credentials",
"settings",
}:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; " f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
f"allowed: template, dockerfile, auth_token, forward_host_credentials" "allowed: template, dockerfile, auth_token, "
"forward_host_credentials, settings"
) )
template = d.get("template", "claude") template = d.get("template", "claude")
if not isinstance(template, str) or not template: if not isinstance(template, str) or not template:
@@ -89,16 +97,18 @@ class AgentProvider:
f"bottle '{bottle_name}' agent_provider.forward_host_credentials " f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
"is currently only supported for template 'codex'" "is currently only supported for template 'codex'"
) )
settings = _parse_provider_settings(bottle_name, template, d.get("settings"))
return cls( return cls(
template=template, template=template,
dockerfile=dockerfile, dockerfile=dockerfile,
auth_token=auth_token, auth_token=auth_token,
forward_host_credentials=forward_host_credentials, forward_host_credentials=forward_host_credentials,
settings=settings,
) )
@dataclass(frozen=True) @dataclass(frozen=True)
class Agent: class ManifestAgent:
bottle: str bottle: str
skills: tuple[str, ...] = () skills: tuple[str, ...] = ()
prompt: str = "" prompt: str = ""
@@ -106,10 +116,10 @@ class Agent:
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only # bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
# `user` is allowed at the agent level; `repos` stays bottle-only # `user` is allowed at the agent level; `repos` stays bottle-only
# because it carries credentials and host trust. # because it carries credentials and host trust.
git_user: GitUser = GitUser() git_user: ManifestGitUser = ManifestGitUser()
@classmethod @classmethod
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent": def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "ManifestAgent":
d = as_json_object(raw, f"agent '{name}'") d = as_json_object(raw, f"agent '{name}'")
unknown = set(d.keys()) - AGENT_MODEL_KEYS unknown = set(d.keys()) - AGENT_MODEL_KEYS
if unknown: if unknown:
@@ -164,7 +174,7 @@ class Agent:
# git-gate: agents may declare only `git-gate.user` (name/email). # git-gate: agents may declare only `git-gate.user` (name/email).
# `git-gate.repos` is bottle-only — it carries credentials and host trust. # `git-gate.repos` is bottle-only — it carries credentials and host trust.
git_user = GitUser() git_user = ManifestGitUser()
git_raw = d.get("git-gate") git_raw = d.get("git-gate")
if git_raw is not None: if git_raw is not None:
gd = as_json_object(git_raw, f"agent '{name}' git-gate") gd = as_json_object(git_raw, f"agent '{name}' git-gate")
@@ -177,6 +187,90 @@ class Agent:
f"(it carries credentials and host trust)." f"(it carries credentials and host trust)."
) )
if "user" in gd: if "user" in gd:
git_user = GitUser.from_dict(name, gd["user"]) git_user = ManifestGitUser.from_dict(name, gd["user"])
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user) return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
def _parse_provider_settings(
bottle_name: str,
template: str,
raw: object,
) -> dict[str, object]:
if raw is None:
return {}
if template != "pi":
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings is only "
"supported for template 'pi'"
)
settings = as_json_object(raw, f"bottle '{bottle_name}' agent_provider.settings")
allowed = {
"provider",
"base_url",
"api",
"api_key",
"api_key_env",
"models",
"context_window",
"max_tokens_field",
"max_tokens",
"supports_developer_role",
"supports_reasoning_effort",
}
for key in settings:
if key not in allowed:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings has unknown "
f"key {key!r}; allowed: {', '.join(sorted(allowed))}"
)
for key in ("provider", "base_url", "api", "api_key", "api_key_env"):
value = settings.get(key)
if value is not None and (not isinstance(value, str) or not value):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
"be a non-empty string"
)
max_tokens_field = settings.get("max_tokens_field")
if max_tokens_field is not None and max_tokens_field not in (
"max_tokens", "max_completion_tokens",
):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.max_tokens_field "
"must be 'max_tokens' or 'max_completion_tokens'"
)
if settings.get("api_key") is not None and settings.get("api_key_env") is not None:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings may set either "
"api_key or api_key_env, not both"
)
models = settings.get("models")
if models is not None:
if not isinstance(models, list) or not models:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.models must "
"be a non-empty array of strings"
)
for i, model in enumerate(models):
if not isinstance(model, str) or not model:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.models[{i}] "
"must be a non-empty string"
)
for key in ("supports_developer_role", "supports_reasoning_effort"):
value = settings.get(key)
if value is not None and not isinstance(value, bool):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
f"be a boolean (was {type(value).__name__})"
)
for key in ("context_window", "max_tokens"):
value = settings.get(key)
if value is not None and (
not isinstance(value, int) or isinstance(value, bool) or value <= 0
):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
f"be a positive integer (was {type(value).__name__})"
)
return dict(settings)
+62 -31
View File
@@ -24,7 +24,7 @@ INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
def validate_egress_routes( def validate_egress_routes(
bottle_name: str, bottle_name: str,
routes: tuple[EgressRoute, ...], routes: tuple[ManifestEgressRoute, ...],
) -> None: ) -> None:
seen_hosts: dict[str, None] = {} seen_hosts: dict[str, None] = {}
for r in routes: for r in routes:
@@ -38,37 +38,38 @@ def validate_egress_routes(
@dataclass(frozen=True) @dataclass(frozen=True)
class PathMatch: class ManifestPathMatch:
Type: str = "prefix" Type: str = "prefix"
Value: str = "" Value: str = ""
@dataclass(frozen=True) @dataclass(frozen=True)
class HeaderMatch: class ManifestHeaderMatch:
Name: str = "" Name: str = ""
Value: str = "" Value: str = ""
Type: str = "exact" Type: str = "exact"
@dataclass(frozen=True) @dataclass(frozen=True)
class MatchEntry: class ManifestMatchEntry:
Paths: tuple[PathMatch, ...] = () Paths: tuple[ManifestPathMatch, ...] = ()
Methods: tuple[str, ...] = () Methods: tuple[str, ...] = ()
Headers: tuple[HeaderMatch, ...] = () Headers: tuple[ManifestHeaderMatch, ...] = ()
@dataclass(frozen=True) @dataclass(frozen=True)
class EgressRoute: class ManifestEgressRoute:
Host: str Host: str
Matches: tuple[MatchEntry, ...] = () Matches: tuple[ManifestMatchEntry, ...] = ()
AuthScheme: str = "" AuthScheme: str = ""
TokenRef: str = "" TokenRef: str = ""
Role: tuple[str, ...] = () Role: tuple[str, ...] = ()
GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None OutboundDetectors: tuple[str, ...] | None = None
InboundDetectors: tuple[str, ...] | None = None InboundDetectors: tuple[str, ...] | None = None
@classmethod @classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute": def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
label = f"bottle '{bottle_name}' egress.routes[{idx}]" label = f"bottle '{bottle_name}' egress.routes[{idx}]"
d = as_json_object(raw, label) d = as_json_object(raw, label)
host = d.get("host") host = d.get("host")
@@ -76,7 +77,7 @@ class EgressRoute:
raise ManifestError(f"{label} missing required string field 'host'") raise ManifestError(f"{label} missing required string field 'host'")
# --- matches --- # --- matches ---
matches: tuple[MatchEntry, ...] = () matches: tuple[ManifestMatchEntry, ...] = ()
matches_raw = d.get("matches") matches_raw = d.get("matches")
if matches_raw is not None: if matches_raw is not None:
if not isinstance(matches_raw, list): if not isinstance(matches_raw, list):
@@ -85,7 +86,7 @@ class EgressRoute:
f"(was {type(matches_raw).__name__})" f"(was {type(matches_raw).__name__})"
) )
matches_list = cast(list[object], matches_raw) matches_list = cast(list[object], matches_raw)
entries: list[MatchEntry] = [] entries: list[ManifestMatchEntry] = []
for k, entry_raw in enumerate(matches_list): for k, entry_raw in enumerate(matches_list):
entries.append( entries.append(
_parse_match_entry(label, k, entry_raw) _parse_match_entry(label, k, entry_raw)
@@ -165,11 +166,30 @@ class EgressRoute:
label, d.get("dlp"), label, d.get("dlp"),
) )
# --- git-over-HTTPS policy ---
git_fetch = False
if "git" in d:
git_d = as_json_object(d.get("git"), f"{label} git")
raw_fetch = git_d.get("fetch", False)
if isinstance(raw_fetch, bool):
git_fetch = raw_fetch
else:
raise ManifestError(
f"{label} git.fetch must be a boolean "
f"(was {type(raw_fetch).__name__})"
)
for k in git_d:
if k != "fetch":
raise ManifestError(
f"{label} git has unknown key {k!r}; "
f"only 'fetch' is accepted"
)
for k in d: for k in d:
if k not in ("host", "matches", "auth", "role", "dlp"): if k not in ("host", "matches", "auth", "role", "dlp", "git"):
raise ManifestError( raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are " f"{label} has unknown key {k!r}; accepted keys are "
f"'host', 'matches', 'auth', 'role', 'dlp'" f"'host', 'matches', 'auth', 'role', 'dlp', 'git'"
) )
return cls( return cls(
@@ -178,6 +198,7 @@ class EgressRoute:
AuthScheme=auth_scheme, AuthScheme=auth_scheme,
TokenRef=token_ref, TokenRef=token_ref,
Role=roles, Role=roles,
GitFetch=git_fetch,
OutboundDetectors=outbound_detectors, OutboundDetectors=outbound_detectors,
InboundDetectors=inbound_detectors, InboundDetectors=inbound_detectors,
) )
@@ -185,17 +206,17 @@ class EgressRoute:
def _parse_match_entry( def _parse_match_entry(
route_label: str, k: int, raw: object, route_label: str, k: int, raw: object,
) -> MatchEntry: ) -> ManifestMatchEntry:
label = f"{route_label} matches[{k}]" label = f"{route_label} matches[{k}]"
d = as_json_object(raw, label) d = as_json_object(raw, label)
paths: tuple[PathMatch, ...] = () paths: tuple[ManifestPathMatch, ...] = ()
paths_raw = d.get("paths") paths_raw = d.get("paths")
if paths_raw is not None: if paths_raw is not None:
if not isinstance(paths_raw, list): if not isinstance(paths_raw, list):
raise ManifestError(f"{label} paths must be an array") raise ManifestError(f"{label} paths must be an array")
paths_list = cast(list[object], paths_raw) paths_list = cast(list[object], paths_raw)
parsed_paths: list[PathMatch] = [] parsed_paths: list[ManifestPathMatch] = []
for j, p_raw in enumerate(paths_list): for j, p_raw in enumerate(paths_list):
parsed_paths.append(_parse_path_match(label, j, p_raw)) parsed_paths.append(_parse_path_match(label, j, p_raw))
paths = tuple(parsed_paths) paths = tuple(parsed_paths)
@@ -220,13 +241,13 @@ def _parse_match_entry(
normalised.append(upper) normalised.append(upper)
methods = tuple(normalised) methods = tuple(normalised)
headers: tuple[HeaderMatch, ...] = () headers: tuple[ManifestHeaderMatch, ...] = ()
headers_raw = d.get("headers") headers_raw = d.get("headers")
if headers_raw is not None: if headers_raw is not None:
if not isinstance(headers_raw, list): if not isinstance(headers_raw, list):
raise ManifestError(f"{label} headers must be an array") raise ManifestError(f"{label} headers must be an array")
headers_list = cast(list[object], headers_raw) headers_list = cast(list[object], headers_raw)
parsed_headers: list[HeaderMatch] = [] parsed_headers: list[ManifestHeaderMatch] = []
for j, h_raw in enumerate(headers_list): for j, h_raw in enumerate(headers_list):
parsed_headers.append(_parse_header_match(label, j, h_raw)) parsed_headers.append(_parse_header_match(label, j, h_raw))
headers = tuple(parsed_headers) headers = tuple(parsed_headers)
@@ -235,12 +256,12 @@ def _parse_match_entry(
if key not in ("paths", "methods", "headers"): if key not in ("paths", "methods", "headers"):
raise ManifestError(f"{label} has unknown key {key!r}") raise ManifestError(f"{label} has unknown key {key!r}")
return MatchEntry(Paths=paths, Methods=methods, Headers=headers) return ManifestMatchEntry(Paths=paths, Methods=methods, Headers=headers)
def _parse_path_match( def _parse_path_match(
entry_label: str, j: int, raw: object, entry_label: str, j: int, raw: object,
) -> PathMatch: ) -> ManifestPathMatch:
label = f"{entry_label} paths[{j}]" label = f"{entry_label} paths[{j}]"
d = as_json_object(raw, label) d = as_json_object(raw, label)
ptype = d.get("type", "prefix") ptype = d.get("type", "prefix")
@@ -266,12 +287,12 @@ def _parse_path_match(
for k in d: for k in d:
if k not in ("type", "value"): if k not in ("type", "value"):
raise ManifestError(f"{label} has unknown key {k!r}") raise ManifestError(f"{label} has unknown key {k!r}")
return PathMatch(Type=ptype, Value=value) return ManifestPathMatch(Type=ptype, Value=value)
def _parse_header_match( def _parse_header_match(
entry_label: str, j: int, raw: object, entry_label: str, j: int, raw: object,
) -> HeaderMatch: ) -> ManifestHeaderMatch:
label = f"{entry_label} headers[{j}]" label = f"{entry_label} headers[{j}]"
d = as_json_object(raw, label) d = as_json_object(raw, label)
name = d.get("name") name = d.get("name")
@@ -296,7 +317,7 @@ def _parse_header_match(
for k in d: for k in d:
if k not in ("name", "value", "type"): if k not in ("name", "value", "type"):
raise ManifestError(f"{label} has unknown key {k!r}") raise ManifestError(f"{label} has unknown key {k!r}")
return HeaderMatch(Name=name, Value=value, Type=htype) return ManifestHeaderMatch(Name=name, Value=value, Type=htype)
def _parse_dlp_block( def _parse_dlp_block(
@@ -346,15 +367,19 @@ def _parse_dlp_block(
return outbound, inbound return outbound, inbound
LOG_LEVELS = frozenset({0, 1, 2})
@dataclass(frozen=True) @dataclass(frozen=True)
class EgressConfig: class ManifestEgressConfig:
routes: tuple[EgressRoute, ...] = () routes: tuple[ManifestEgressRoute, ...] = ()
Log: int = 0
@classmethod @classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig": def from_dict(cls, bottle_name: str, raw: object) -> "ManifestEgressConfig":
d = as_json_object(raw, f"bottle '{bottle_name}' egress") d = as_json_object(raw, f"bottle '{bottle_name}' egress")
routes_raw = d.get("routes") routes_raw = d.get("routes")
routes: tuple[EgressRoute, ...] = () routes: tuple[ManifestEgressRoute, ...] = ()
if routes_raw is not None: if routes_raw is not None:
if not isinstance(routes_raw, list): if not isinstance(routes_raw, list):
raise ManifestError( raise ManifestError(
@@ -363,14 +388,20 @@ class EgressConfig:
) )
routes_list = cast(list[object], routes_raw) routes_list = cast(list[object], routes_raw)
routes = tuple( routes = tuple(
EgressRoute.from_dict(bottle_name, i, entry) ManifestEgressRoute.from_dict(bottle_name, i, entry)
for i, entry in enumerate(routes_list) for i, entry in enumerate(routes_list)
) )
validate_egress_routes(bottle_name, routes) validate_egress_routes(bottle_name, routes)
log_raw = d.get("log", 0)
if isinstance(log_raw, bool) or not isinstance(log_raw, int) \
or log_raw not in LOG_LEVELS:
raise ManifestError(
f"bottle '{bottle_name}' egress.log must be 0, 1, or 2"
)
for k in d: for k in d:
if k != "routes": if k not in ("routes", "log"):
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' egress has unknown key {k!r}; " f"bottle '{bottle_name}' egress has unknown key {k!r}; "
f"only 'routes' is accepted" f"accepted keys are 'routes', 'log'"
) )
return cls(routes=routes) return cls(routes=routes, Log=log_raw)
+46 -24
View File
@@ -5,12 +5,13 @@ from __future__ import annotations
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
if TYPE_CHECKING: if TYPE_CHECKING:
from .manifest import Bottle, GitEntry from .manifest import ManifestBottle, ManifestGitEntry
from .manifest_egress import ManifestEgressConfig
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]: def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, ManifestBottle]:
"""Apply `extends:` chains and return resolved Bottle objects.""" """Apply `extends:` chains and return resolved ManifestBottle objects."""
cache: dict[str, Bottle] = {} cache: dict[str, ManifestBottle] = {}
for name in raws: for name in raws:
if name not in cache: if name not in cache:
_resolve_one_bottle(name, raws, cache, ()) _resolve_one_bottle(name, raws, cache, ())
@@ -20,10 +21,10 @@ def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
def _resolve_one_bottle( def _resolve_one_bottle(
name: str, name: str,
raws: dict[str, dict[str, object]], raws: dict[str, dict[str, object]],
cache: dict[str, Bottle], cache: dict[str, ManifestBottle],
seen: tuple[str, ...], seen: tuple[str, ...],
) -> Bottle: ) -> ManifestBottle:
from .manifest import Bottle, ManifestError from .manifest import ManifestBottle, ManifestError
if name in cache: if name in cache:
return cache[name] return cache[name]
@@ -32,13 +33,13 @@ def _resolve_one_bottle(
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}") raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
raw = raws[name] raw = raws[name]
parent_name_raw = raw.get("extends") parent_name_raw = raw.get("extends")
# Strip `extends:` before passing to Bottle.from_dict so it # Strip `extends:` before passing to ManifestBottle.from_dict so it
# is not accidentally treated as a real Bottle field by future # is not accidentally treated as a real ManifestBottle field by future
# schema additions. It is only meaningful here. # schema additions. It is only meaningful here.
child_raw = {k: v for k, v in raw.items() if k != "extends"} child_raw = {k: v for k, v in raw.items() if k != "extends"}
if parent_name_raw is None: if parent_name_raw is None:
bottle = Bottle.from_dict(name, child_raw) bottle = ManifestBottle.from_dict(name, child_raw)
cache[name] = bottle cache[name] = bottle
return bottle return bottle
@@ -66,27 +67,27 @@ def _resolve_one_bottle(
def _merge_bottles( def _merge_bottles(
parent: Bottle, parent: ManifestBottle,
child_raw: dict[str, object], child_raw: dict[str, object],
name: str, name: str,
) -> Bottle: ) -> ManifestBottle:
"""Apply PRD 0025 merge rules.""" """Apply PRD 0025 merge rules."""
from .manifest import Bottle, GitUser from .manifest import ManifestBottle, ManifestGitUser
from .manifest_egress import validate_egress_routes from .manifest_egress import validate_egress_routes
# Parse the child's declared fields into a Bottle (with the # Parse the child's declared fields into a ManifestBottle (with the
# usual defaults for anything missing). Validation runs the same # usual defaults for anything missing). Validation runs the same
# way it would for a leaf bottle: typos / wrong types die here. # way it would for a leaf bottle: typos / wrong types die here.
child = Bottle.from_dict(name, child_raw) child = ManifestBottle.from_dict(name, child_raw)
# env: dict merge, child wins on collision. # env: dict merge, child wins on collision.
merged_env = {**parent.env, **child.env} merged_env = {**parent.env, **child.env}
# git-gate.user: per-field overlay. Each non-empty field on child # git-gate.user: per-field overlay. Each non-empty field on child
# wins; empties fall through to parent. The default GitUser() # wins; empties fall through to parent. The default ManifestGitUser()
# is two empty strings, so a child that omits git-gate.user # is two empty strings, so a child that omits git-gate.user
# inherits the parent's user verbatim. # inherits the parent's user verbatim.
merged_git_user = GitUser( merged_git_user = ManifestGitUser(
name=child.git_user.name or parent.git_user.name, name=child.git_user.name or parent.git_user.name,
email=child.git_user.email or parent.git_user.email, email=child.git_user.email or parent.git_user.email,
) )
@@ -99,9 +100,16 @@ def _merge_bottles(
else: else:
merged_git = parent.git merged_git = parent.git
# Presence-driven full-replace for the remaining list-valued + # egress.routes: missing means inherit; otherwise parent and child
# scalar fields. # route lists concatenate. Other egress scalar fields remain
merged_egress = child.egress if "egress" in child_raw else parent.egress # presence-driven overlays.
merged_egress = (
_merge_egress(parent.egress, child.egress, child_raw)
if "egress" in child_raw
else parent.egress
)
# Presence-driven full-replace for the remaining scalar fields.
merged_agent_provider = ( merged_agent_provider = (
child.agent_provider child.agent_provider
if "agent_provider" in child_raw if "agent_provider" in child_raw
@@ -112,7 +120,7 @@ def _merge_bottles(
) )
validate_egress_routes(name, merged_egress.routes) validate_egress_routes(name, merged_egress.routes)
return Bottle( return ManifestBottle(
env=merged_env, env=merged_env,
agent_provider=merged_agent_provider, agent_provider=merged_agent_provider,
git=merged_git, git=merged_git,
@@ -133,10 +141,24 @@ def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
def _merge_git_remotes( def _merge_git_remotes(
parent: tuple[GitEntry, ...], parent: tuple[ManifestGitEntry, ...],
child: tuple[GitEntry, ...], child: tuple[ManifestGitEntry, ...],
) -> tuple[GitEntry, ...]: ) -> tuple[ManifestGitEntry, ...]:
by_host = {entry.UpstreamHost: entry for entry in parent} by_host = {entry.UpstreamHost: entry for entry in parent}
for entry in child: for entry in child:
by_host[entry.UpstreamHost] = entry by_host[entry.UpstreamHost] = entry
return tuple(by_host.values()) return tuple(by_host.values())
def _merge_egress(
parent: ManifestEgressConfig,
child: ManifestEgressConfig,
child_raw: dict[str, object],
) -> ManifestEgressConfig:
from .manifest_egress import ManifestEgressConfig
from .manifest_util import as_json_object
child_egress_raw = as_json_object(child_raw.get("egress"), "child egress")
routes = parent.routes + child.routes
log = child.Log if "log" in child_egress_raw else parent.Log
return ManifestEgressConfig(routes=routes, Log=log)
+83 -76
View File
@@ -4,7 +4,6 @@ from __future__ import annotations
import re import re
from dataclasses import dataclass from dataclasses import dataclass
from typing import Optional
from .manifest_util import ManifestError, as_json_object from .manifest_util import ManifestError, as_json_object
@@ -13,6 +12,8 @@ from .manifest_util import ManifestError, as_json_object
# defence; this regex is belt-and-suspenders and documents intent). # defence; this regex is belt-and-suspenders and documents intent).
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$") _GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
_KEY_PROVIDERS = {"static", "gitea"}
def _opt_str(value: object, label: str) -> str: def _opt_str(value: object, label: str) -> str:
if value is None: if value is None:
@@ -57,7 +58,7 @@ def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
return (user, host, port, path) return (user, host, port, path)
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None: def validate_unique_git_names(bottle_name: str, git: tuple[ManifestGitEntry, ...]) -> None:
seen: dict[str, None] = {} seen: dict[str, None] = {}
for g in git: for g in git:
if g.Name in seen: if g.Name in seen:
@@ -69,25 +70,27 @@ def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> No
@dataclass(frozen=True) @dataclass(frozen=True)
class ProvisionedKeyConfig: class ManifestKeyConfig:
"""Configuration for automatic deploy-key lifecycle management """Configuration for a repo's SSH key in git-gate.repos.
(PRD 0048). Used when a git-gate.repos entry opts out of a
static identity file and instead wants a fresh SSH keypair
generated at spin-up and revoked at teardown.
`provider` names the contrib sub-package to load (e.g. `gitea`). `provider` is either `"static"` (a pre-existing key on the host) or
`token_env` is the name of a host-side env var carrying the API `"gitea"` (automatic deploy-key lifecycle via the Gitea API).
token; the value is read at provision time, never stored on the
plan. `api_url` is the forge's HTTP API root; if empty, it is For `static`: `path` is the host-side absolute path to the SSH private key.
derived from the upstream URL's host at provision time."""
For `gitea`: `forge_token_env` is the name of a host-side env var
carrying the Gitea API token; the value is read at provision time,
never stored on the plan. `api_url` is the forge's HTTP API root; if
empty, it is derived from the upstream URL's host at provision time."""
provider: str provider: str
token_env: str path: str = ""
forge_token_env: str = ""
api_url: str = "" api_url: str = ""
@dataclass(frozen=True) @dataclass(frozen=True)
class GitEntry: class ManifestGitEntry:
"""One upstream the per-agent git-gate (PRD 0008) is allowed to """One upstream the per-agent git-gate (PRD 0008) is allowed to
talk to. `Upstream` is the real remote URL the agent would push to talk to. `Upstream` is the real remote URL the agent would push to
if there were no gate; the gate hosts a bare repo at /git/<Name>.git if there were no gate; the gate hosts a bare repo at /git/<Name>.git
@@ -99,15 +102,16 @@ class GitEntry:
stashed in the `Upstream*` fields so the git-gate render step stashed in the `Upstream*` fields so the git-gate render step
doesn't have to re-parse. doesn't have to re-parse.
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). Exactly Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). A `key`
one of `identity` (static key path) or `provisioned_key` (automatic block is required; `key.provider` is `"static"` or `"gitea"`. For
lifecycle) must be present. The internal field names are stable.""" `static`, `IdentityFile` is populated at parse time from `key.path`.
For `gitea`, `IdentityFile` is populated at provision time."""
Name: str Name: str
Upstream: str Upstream: str
Key: ManifestKeyConfig = ManifestKeyConfig(provider="")
IdentityFile: str = "" IdentityFile: str = ""
KnownHostKey: str = "" KnownHostKey: str = ""
ProvisionedKey: Optional[ProvisionedKeyConfig] = None
RemoteKey: str = "" RemoteKey: str = ""
UpstreamUser: str = "" UpstreamUser: str = ""
UpstreamHost: str = "" UpstreamHost: str = ""
@@ -117,11 +121,11 @@ class GitEntry:
@classmethod @classmethod
def from_repos_entry( def from_repos_entry(
cls, bottle_name: str, repo_name: str, raw: object cls, bottle_name: str, repo_name: str, raw: object
) -> "GitEntry": ) -> "ManifestGitEntry":
"""Parse one entry from `git-gate.repos.<repo_name>`. """Parse one entry from `git-gate.repos.<repo_name>`.
YAML keys: `url` (required), exactly one of `identity` or YAML keys: `url` (required), `key` (required object with
`provisioned_key` (required), `host_key` (optional). `provider`, and provider-specific fields), `host_key` (optional).
The repo_name becomes `Name`.""" The repo_name becomes `Name`."""
if not repo_name: if not repo_name:
raise ManifestError( raise ManifestError(
@@ -135,10 +139,10 @@ class GitEntry:
label = f"git-gate.repos[{repo_name!r}]" label = f"git-gate.repos[{repo_name!r}]"
d = as_json_object(raw, f"bottle '{bottle_name}' {label}") d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
for k in d: for k in d:
if k not in {"url", "identity", "provisioned_key", "host_key"}: if k not in {"url", "key", "host_key"}:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' {label} has unknown key {k!r}; " f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
f"allowed: url, identity, provisioned_key, host_key" f"allowed: url, key, host_key"
) )
upstream = d.get("url") upstream = d.get("url")
if not isinstance(upstream, str) or not upstream: if not isinstance(upstream, str) or not upstream:
@@ -146,32 +150,13 @@ class GitEntry:
f"bottle '{bottle_name}' {label} missing required string field 'url'" f"bottle '{bottle_name}' {label} missing required string field 'url'"
) )
has_identity = "identity" in d if "key" not in d:
has_provisioned = "provisioned_key" in d
if has_identity and has_provisioned:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' {label} must set exactly one of " f"bottle '{bottle_name}' {label} missing required 'key' block"
f"'identity' or 'provisioned_key'; got both."
)
if not has_identity and not has_provisioned:
raise ManifestError(
f"bottle '{bottle_name}' {label} must set exactly one of "
f"'identity' or 'provisioned_key'; got neither."
) )
key_config = _parse_key_config(bottle_name, label, d["key"])
ident = "" ident = key_config.path if key_config.provider == "static" else ""
provisioned_key: Optional[ProvisionedKeyConfig] = None
if has_identity:
raw_ident = d.get("identity")
if not isinstance(raw_ident, str) or not raw_ident:
raise ManifestError(
f"bottle '{bottle_name}' {label} 'identity' must be a non-empty string"
)
ident = raw_ident
else:
provisioned_key = _parse_provisioned_key_config(
bottle_name, label, d["provisioned_key"]
)
khk = _opt_str( khk = _opt_str(
d.get("host_key"), d.get("host_key"),
@@ -183,9 +168,9 @@ class GitEntry:
return cls( return cls(
Name=repo_name, Name=repo_name,
Upstream=upstream, Upstream=upstream,
Key=key_config,
IdentityFile=ident, IdentityFile=ident,
KnownHostKey=khk, KnownHostKey=khk,
ProvisionedKey=provisioned_key,
RemoteKey=host, RemoteKey=host,
UpstreamUser=user, UpstreamUser=user,
UpstreamHost=host, UpstreamHost=host,
@@ -194,42 +179,64 @@ class GitEntry:
) )
def _parse_provisioned_key_config( def _parse_key_config(
bottle_name: str, label: str, raw: object bottle_name: str, label: str, raw: object
) -> ProvisionedKeyConfig: ) -> ManifestKeyConfig:
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.provisioned_key") d = as_json_object(raw, f"bottle '{bottle_name}' {label}.key")
for k in d:
if k not in {"provider", "token_env", "api_url"}:
raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key has unknown key {k!r}; "
f"allowed: provider, token_env, api_url"
)
provider = d.get("provider") provider = d.get("provider")
if not isinstance(provider, str) or not provider: if not isinstance(provider, str) or not provider:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key missing required " f"bottle '{bottle_name}' {label}.key missing required "
f"string field 'provider'" f"string field 'provider'"
) )
token_env = d.get("token_env") if provider not in _KEY_PROVIDERS:
if not isinstance(token_env, str) or not token_env:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key missing required " f"bottle '{bottle_name}' {label}.key provider {provider!r} is unknown; "
f"string field 'token_env'" f"allowed: {', '.join(sorted(_KEY_PROVIDERS))}"
) )
api_url_raw = d.get("api_url", "")
if not isinstance(api_url_raw, str): if provider == "gitea":
for k in d:
if k not in {"provider", "forge_token_env", "api_url"}:
raise ManifestError(
f"bottle '{bottle_name}' {label}.key has unknown key {k!r} "
f"for provider 'gitea'; allowed: provider, forge_token_env, api_url"
)
forge_token_env = d.get("forge_token_env")
if not isinstance(forge_token_env, str) or not forge_token_env:
raise ManifestError(
f"bottle '{bottle_name}' {label}.key missing required "
f"string field 'forge_token_env' for provider 'gitea'"
)
api_url_raw = d.get("api_url", "")
if not isinstance(api_url_raw, str):
raise ManifestError(
f"bottle '{bottle_name}' {label}.key 'api_url' must be a string"
)
return ManifestKeyConfig(
provider=provider,
forge_token_env=forge_token_env,
api_url=api_url_raw,
)
# provider == "static"
for k in d:
if k not in {"provider", "path"}:
raise ManifestError(
f"bottle '{bottle_name}' {label}.key has unknown key {k!r} "
f"for provider 'static'; allowed: provider, path"
)
path = d.get("path")
if not isinstance(path, str) or not path:
raise ManifestError( raise ManifestError(
f"bottle '{bottle_name}' {label}.provisioned_key 'api_url' must be a string" f"bottle '{bottle_name}' {label}.key missing required "
f"string field 'path' for provider 'static'"
) )
return ProvisionedKeyConfig( return ManifestKeyConfig(provider=provider, path=path)
provider=provider,
token_env=token_env,
api_url=api_url_raw,
)
@dataclass(frozen=True) @dataclass(frozen=True)
class GitUser: class ManifestGitUser:
"""Per-bottle `git config --global user.name` / `user.email` """Per-bottle `git config --global user.name` / `user.email`
pair (issue #86). The agent's commits inside the bottle are pair (issue #86). The agent's commits inside the bottle are
attributed to this identity rather than the agent image's attributed to this identity rather than the agent image's
@@ -244,7 +251,7 @@ class GitUser:
email: str = "" email: str = ""
@classmethod @classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser": def from_dict(cls, bottle_name: str, raw: object) -> "ManifestGitUser":
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user") d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
for k in d: for k in d:
if k not in {"name", "email"}: if k not in {"name", "email"}:
@@ -279,7 +286,7 @@ class GitUser:
def parse_git_gate_config( def parse_git_gate_config(
bottle_name: str, bottle_name: str,
raw: object, raw: object,
) -> tuple[tuple[GitEntry, ...], GitUser]: ) -> tuple[tuple[ManifestGitEntry, ...], ManifestGitUser]:
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate") d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
for k in d: for k in d:
if k not in {"user", "repos"}: if k not in {"user", "repos"}:
@@ -289,17 +296,17 @@ def parse_git_gate_config(
) )
git_user = ( git_user = (
GitUser.from_dict(bottle_name, d["user"]) ManifestGitUser.from_dict(bottle_name, d["user"])
if "user" in d if "user" in d
else GitUser() else ManifestGitUser()
) )
git: tuple[GitEntry, ...] = () git: tuple[ManifestGitEntry, ...] = ()
repos_raw = d.get("repos") repos_raw = d.get("repos")
if repos_raw is not None: if repos_raw is not None:
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos") repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
git = tuple( git = tuple(
GitEntry.from_repos_entry(bottle_name, name, entry) ManifestGitEntry.from_repos_entry(bottle_name, name, entry)
for name, entry in repos.items() for name, entry in repos.items()
) )
validate_unique_git_names(bottle_name, git) validate_unique_git_names(bottle_name, git)
+6 -6
View File
@@ -14,7 +14,7 @@ from .manifest_schema import (
from .yaml_subset import YamlSubsetError, parse_frontmatter from .yaml_subset import YamlSubsetError, parse_frontmatter
if TYPE_CHECKING: if TYPE_CHECKING:
from .manifest import Agent, Bottle from .manifest import ManifestAgent, ManifestBottle
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None: def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
@@ -34,7 +34,7 @@ def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
) )
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]: def load_bottles_from_dir(bottles_dir: Path) -> dict[str, ManifestBottle]:
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return """Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
`{name: Bottle}`. Missing dir returns an empty dict.""" `{name: Bottle}`. Missing dir returns an empty dict."""
from .manifest import ManifestError from .manifest import ManifestError
@@ -67,13 +67,13 @@ def load_agents_from_dir(
bottle_names: set[str], bottle_names: set[str],
*, *,
source: str, # noqa: F841 — unused, but required by interface source: str, # noqa: F841 — unused, but required by interface
) -> dict[str, Agent]: ) -> dict[str, ManifestAgent]:
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return """Walk `<agents_dir>/*.md`, parse each as an agent, and return
`{name: Agent}`. The Markdown body becomes the agent's prompt. `{name: Agent}`. The Markdown body becomes the agent's prompt.
Missing dir returns an empty dict.""" Missing dir returns an empty dict."""
from .manifest import Agent, ManifestError from .manifest import ManifestAgent, ManifestError
out: dict[str, Agent] = {} out: dict[str, ManifestAgent] = {}
if not agents_dir.is_dir(): if not agents_dir.is_dir():
return out return out
for path in sorted(agents_dir.glob("*.md")): for path in sorted(agents_dir.glob("*.md")):
@@ -101,5 +101,5 @@ def load_agents_from_dir(
} }
if "git-gate" in fm: if "git-gate" in fm:
agent_dict["git-gate"] = fm["git-gate"] agent_dict["git-gate"] = fm["git-gate"]
out[name] = Agent.from_dict(name, agent_dict, bottle_names) out[name] = ManifestAgent.from_dict(name, agent_dict, bottle_names)
return out return out
+20 -2
View File
@@ -59,6 +59,7 @@ class _DaemonSpec:
# reads to inject `Authorization` headers on configured routes; # reads to inject `Authorization` headers on configured routes;
# no other daemon in the bundle should see these values. # no other daemon in the bundle should see these values.
_EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",) _EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",)
_READY_GATED_DAEMONS: tuple[str, ...] = ("git-gate", "git-http")
def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]: def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]:
@@ -82,6 +83,22 @@ _DAEMONS: tuple[_DaemonSpec, ...] = (
) )
def _argv_for_daemon(name: str, argv: Sequence[str], env: dict[str, str]) -> list[str]:
ready_file = env.get("BOT_BOTTLE_GIT_GATE_READY_FILE", "").strip()
if name not in _READY_GATED_DAEMONS or not ready_file:
return list(argv)
return [
"/bin/sh",
"-c",
"while [ ! -f \"$BOT_BOTTLE_GIT_GATE_READY_FILE\" ]; do "
"sleep 0.1; "
"done; "
"exec \"$@\"",
name,
*argv,
]
def _selected_daemons( def _selected_daemons(
env: dict[str, str], env: dict[str, str],
all_daemons: Sequence[_DaemonSpec] | None = None, all_daemons: Sequence[_DaemonSpec] | None = None,
@@ -118,12 +135,13 @@ def _pump(name: str, stream: IO[bytes]) -> None:
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]: def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
env = _env_for_daemon(spec.name, dict(os.environ))
proc = subprocess.Popen( proc = subprocess.Popen(
list(spec.argv), _argv_for_daemon(spec.name, spec.argv, env),
stdout=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, stderr=subprocess.STDOUT,
bufsize=0, bufsize=0,
env=_env_for_daemon(spec.name, dict(os.environ)), env=env,
) )
threading.Thread( threading.Thread(
target=_pump, args=(spec.name, proc.stdout), daemon=True target=_pump, args=(spec.name, proc.stdout), daemon=True
-5
View File
@@ -465,8 +465,6 @@ class Supervise(ABC):
self, self,
slug: str, slug: str,
stage_dir: Path, stage_dir: Path,
*,
dockerfile_content: str = "",
) -> SupervisePlan: ) -> SupervisePlan:
"""Stage the per-bottle queue dir on the host and the """Stage the per-bottle queue dir on the host and the
current-config dir under `stage_dir`. Returns the plan; current-config dir under `stage_dir`. Returns the plan;
@@ -476,9 +474,6 @@ class Supervise(ABC):
queue_dir.mkdir(parents=True, exist_ok=True) queue_dir.mkdir(parents=True, exist_ok=True)
current_config_dir = stage_dir / "current-config" current_config_dir = stage_dir / "current-config"
current_config_dir.mkdir(parents=True, exist_ok=True) current_config_dir.mkdir(parents=True, exist_ok=True)
dockerfile_path = current_config_dir / CURRENT_CONFIG_DOCKERFILE
dockerfile_path.write_text(dockerfile_content)
dockerfile_path.chmod(0o644)
return SupervisePlan( return SupervisePlan(
slug=slug, slug=slug,
queue_dir=queue_dir, queue_dir=queue_dir,
+30 -36
View File
@@ -69,12 +69,6 @@ class YamlSubsetError(ValueError):
egress sidecar's addon) handle it as a normal exception.""" egress sidecar's addon) handle it as a normal exception."""
def die(msg: str) -> None:
"""Module-local helper so the parser body reads cleanly. Just
raises YamlSubsetError the `bot-bottle: error: ` prefix
is added by the boundary `die` in `bot_bottle.log`."""
raise YamlSubsetError(msg)
# --- Tokenizer / line preprocessing ---------------------------------------- # --- Tokenizer / line preprocessing ----------------------------------------
@@ -119,7 +113,7 @@ def _tokenize(text: str) -> list[_Line]:
# editors render them differently and the spec says spaces. # editors render them differently and the spec says spaces.
leading = len(raw) - len(raw.lstrip(" \t")) leading = len(raw) - len(raw.lstrip(" \t"))
if "\t" in raw[:leading]: if "\t" in raw[:leading]:
die(f"yaml-subset: tab character in indent on line {n}") raise YamlSubsetError(f"yaml-subset: tab character in indent on line {n}")
stripped = raw.strip() stripped = raw.strip()
if not stripped: if not stripped:
continue continue
@@ -169,14 +163,14 @@ def _parse_scalar(s: str, lineno: int) -> object:
s.startswith("'") and s.endswith("'") s.startswith("'") and s.endswith("'")
): ):
if len(s) < 2: if len(s) < 2:
die(f"yaml-subset: unterminated quoted string on line {lineno}") raise YamlSubsetError(f"yaml-subset: unterminated quoted string on line {lineno}")
body = s[1:-1] body = s[1:-1]
if s.startswith('"'): if s.startswith('"'):
# JSON-style escapes for double quotes. # JSON-style escapes for double quotes.
try: try:
return body.encode("utf-8").decode("unicode_escape") return body.encode("utf-8").decode("unicode_escape")
except UnicodeDecodeError as e: except UnicodeDecodeError as e:
die(f"yaml-subset: bad escape on line {lineno}: {e}") raise YamlSubsetError(f"yaml-subset: bad escape on line {lineno}: {e}")
else: else:
# Single quotes: only '' → ' (standard YAML); no other escapes. # Single quotes: only '' → ' (standard YAML); no other escapes.
return body.replace("''", "'") return body.replace("''", "'")
@@ -186,7 +180,7 @@ def _parse_scalar(s: str, lineno: int) -> object:
if s in _RESERVED_BOOL_LIKE: if s in _RESERVED_BOOL_LIKE:
if s in ("true", "false"): if s in ("true", "false"):
return s == "true" return s == "true"
die( raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} is ambiguous " f"yaml-subset: bare {s!r} on line {lineno} is ambiguous "
f"(use literal `true` / `false`, or quote it as a string)" f"(use literal `true` / `false`, or quote it as a string)"
) )
@@ -203,22 +197,22 @@ def _parse_scalar(s: str, lineno: int) -> object:
# Look-alikes that we reject to keep the user in control. # Look-alikes that we reject to keep the user in control.
if _DATE_RX.match(s): if _DATE_RX.match(s):
die( raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} looks like a " f"yaml-subset: bare {s!r} on line {lineno} looks like a "
f"date — quote it as a string or use an explicit int" f"date — quote it as a string or use an explicit int"
) )
if _OCTAL_RX.match(s): if _OCTAL_RX.match(s):
die( raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} looks like an " f"yaml-subset: bare {s!r} on line {lineno} looks like an "
f"octal/0-prefixed integer — quote it as a string" f"octal/0-prefixed integer — quote it as a string"
) )
if _HEX_RX.match(s): if _HEX_RX.match(s):
die( raise YamlSubsetError(
f"yaml-subset: bare {s!r} on line {lineno} looks like a " f"yaml-subset: bare {s!r} on line {lineno} looks like a "
f"hex integer — quote it as a string" f"hex integer — quote it as a string"
) )
if _FLOAT_RX.match(s): if _FLOAT_RX.match(s):
die( raise YamlSubsetError(
f"yaml-subset: floats not supported (line {lineno}, " f"yaml-subset: floats not supported (line {lineno}, "
f"value {s!r}); use an int or quote as a string" f"value {s!r}); use an int or quote as a string"
) )
@@ -241,7 +235,7 @@ def _parse_inline(s: str, lineno: int) -> object:
s = s.strip() s = s.strip()
if s.startswith("["): if s.startswith("["):
if not s.endswith("]"): if not s.endswith("]"):
die(f"yaml-subset: unterminated `[` on line {lineno}") raise YamlSubsetError(f"yaml-subset: unterminated `[` on line {lineno}")
body = s[1:-1].strip() body = s[1:-1].strip()
if not body: if not body:
return [] return []
@@ -252,21 +246,21 @@ def _parse_inline(s: str, lineno: int) -> object:
return items return items
if s.startswith("{"): if s.startswith("{"):
if not s.endswith("}"): if not s.endswith("}"):
die(f"yaml-subset: unterminated `{{` on line {lineno}") raise YamlSubsetError(f"yaml-subset: unterminated `{{` on line {lineno}")
body = s[1:-1].strip() body = s[1:-1].strip()
if not body: if not body:
return {} return {}
out: dict[str, object] = {} out: dict[str, object] = {}
for raw in _split_flow(body, lineno, "dict"): for raw in _split_flow(body, lineno, "dict"):
if ":" not in raw: if ":" not in raw:
die( raise YamlSubsetError(
f"yaml-subset: inline dict entry on line {lineno} " f"yaml-subset: inline dict entry on line {lineno} "
f"missing `:` ({raw!r})" f"missing `:` ({raw!r})"
) )
k, _, v = raw.partition(":") k, _, v = raw.partition(":")
k = k.strip() k = k.strip()
if not _BARE_RX.match(k): if not _BARE_RX.match(k):
die( raise YamlSubsetError(
f"yaml-subset: inline dict key on line {lineno} " f"yaml-subset: inline dict key on line {lineno} "
f"must be a bare identifier ({k!r})" f"must be a bare identifier ({k!r})"
) )
@@ -296,7 +290,7 @@ def _split_flow(body: str, lineno: int, kind: str) -> list[str]:
elif ch in "]}": elif ch in "]}":
depth_b -= 1 depth_b -= 1
if depth_b > 0: if depth_b > 0:
die( raise YamlSubsetError(
f"yaml-subset: nested flow {kind} on line " f"yaml-subset: nested flow {kind} on line "
f"{lineno} (only one level of flow allowed)" f"{lineno} (only one level of flow allowed)"
) )
@@ -330,7 +324,7 @@ def _split_key_value(content: str, lineno: int) -> tuple[str, str]:
# ambiguous with URLs etc.). # ambiguous with URLs etc.).
if i + 1 >= len(content) or content[i + 1] in (" ", "\t"): if i + 1 >= len(content) or content[i + 1] in (" ", "\t"):
return content[:i].strip(), content[i + 1:].lstrip() return content[:i].strip(), content[i + 1:].lstrip()
die(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}") raise YamlSubsetError(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
return "", "" # unreachable, but needed for type checker return "", "" # unreachable, but needed for type checker
@@ -341,15 +335,15 @@ def _parse_block(
to live at `base_indent`. Returns (value, new_idx) where to live at `base_indent`. Returns (value, new_idx) where
`new_idx` is the index of the first unconsumed line.""" `new_idx` is the index of the first unconsumed line."""
if idx >= len(lines): if idx >= len(lines):
die("yaml-subset: unexpected end of document") raise YamlSubsetError("yaml-subset: unexpected end of document")
first = lines[idx] first = lines[idx]
if first.indent < base_indent: if first.indent < base_indent:
die( raise YamlSubsetError(
f"yaml-subset: line {first.lineno} indented less than " f"yaml-subset: line {first.lineno} indented less than "
f"expected (got {first.indent}, expected >= {base_indent})" f"expected (got {first.indent}, expected >= {base_indent})"
) )
if first.indent > base_indent: if first.indent > base_indent:
die( raise YamlSubsetError(
f"yaml-subset: line {first.lineno} indented more than " f"yaml-subset: line {first.lineno} indented more than "
f"expected (got {first.indent}, expected {base_indent})" f"expected (got {first.indent}, expected {base_indent})"
) )
@@ -366,18 +360,18 @@ def _parse_block_mapping(
while idx < len(lines) and lines[idx].indent == base_indent: while idx < len(lines) and lines[idx].indent == base_indent:
line = lines[idx] line = lines[idx]
if line.content.startswith("- "): if line.content.startswith("- "):
die( raise YamlSubsetError(
f"yaml-subset: line {line.lineno} unexpected list " f"yaml-subset: line {line.lineno} unexpected list "
f"item at mapping indent (got `-`, expected `key:`)" f"item at mapping indent (got `-`, expected `key:`)"
) )
key, value_text = _split_key_value(line.content, line.lineno) key, value_text = _split_key_value(line.content, line.lineno)
if not _BARE_RX.match(key): if not _BARE_RX.match(key):
die( raise YamlSubsetError(
f"yaml-subset: line {line.lineno} key {key!r} is not " f"yaml-subset: line {line.lineno} key {key!r} is not "
f"a bare identifier" f"a bare identifier"
) )
if key in out: if key in out:
die( raise YamlSubsetError(
f"yaml-subset: line {line.lineno} duplicate key {key!r}" f"yaml-subset: line {line.lineno} duplicate key {key!r}"
) )
if value_text: if value_text:
@@ -417,7 +411,7 @@ def _parse_block_list(
content_col = base_indent + 2 content_col = base_indent + 2
first_key, first_value_text = _split_key_value(rest, line.lineno) first_key, first_value_text = _split_key_value(rest, line.lineno)
if not _BARE_RX.match(first_key): if not _BARE_RX.match(first_key):
die( raise YamlSubsetError(
f"yaml-subset: line {line.lineno} key {first_key!r} " f"yaml-subset: line {line.lineno} key {first_key!r} "
f"is not a bare identifier" f"is not a bare identifier"
) )
@@ -440,12 +434,12 @@ def _parse_block_list(
break # next list item, not a sibling key break # next list item, not a sibling key
k, v_text = _split_key_value(ln.content, ln.lineno) k, v_text = _split_key_value(ln.content, ln.lineno)
if not _BARE_RX.match(k): if not _BARE_RX.match(k):
die( raise YamlSubsetError(
f"yaml-subset: line {ln.lineno} key {k!r} is " f"yaml-subset: line {ln.lineno} key {k!r} is "
f"not a bare identifier" f"not a bare identifier"
) )
if k in item: if k in item:
die(f"yaml-subset: line {ln.lineno} duplicate key {k!r}") raise YamlSubsetError(f"yaml-subset: line {ln.lineno} duplicate key {k!r}")
if v_text: if v_text:
item[k] = _parse_inline(v_text, ln.lineno) item[k] = _parse_inline(v_text, ln.lineno)
idx += 1 idx += 1
@@ -501,7 +495,7 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
for n, raw in enumerate(text.splitlines(), start=1): for n, raw in enumerate(text.splitlines(), start=1):
s = raw.strip() s = raw.strip()
if s.startswith("|") or s.startswith(">") or s.startswith("- |") or s.startswith("- >"): if s.startswith("|") or s.startswith(">") or s.startswith("- |") or s.startswith("- >"):
die( raise YamlSubsetError(
f"yaml-subset: line {n} uses a multi-line block " f"yaml-subset: line {n} uses a multi-line block "
f"scalar (`|` / `>`) — not supported. Use a quoted " f"scalar (`|` / `>`) — not supported. Use a quoted "
f"single-line string instead." f"single-line string instead."
@@ -511,12 +505,12 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
# not when it's inside a quoted string. Cheap check: any # not when it's inside a quoted string. Cheap check: any
# bare `&foo:` / `*foo` at the start of a value position. # bare `&foo:` / `*foo` at the start of a value position.
if re.search(r"(^|\s)[&*][A-Za-z0-9_]+", s): if re.search(r"(^|\s)[&*][A-Za-z0-9_]+", s):
die( raise YamlSubsetError(
f"yaml-subset: line {n} uses anchors / aliases " f"yaml-subset: line {n} uses anchors / aliases "
f"(`&` / `*`) — not supported." f"(`&` / `*`) — not supported."
) )
if "!!" in s and not (s.count("'") % 2 or s.count('"') % 2): if "!!" in s and not (s.count("'") % 2 or s.count('"') % 2):
die( raise YamlSubsetError(
f"yaml-subset: line {n} uses a YAML tag (`!!`) — not " f"yaml-subset: line {n} uses a YAML tag (`!!`) — not "
f"supported." f"supported."
) )
@@ -526,18 +520,18 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
return {} return {}
base_indent = lines[0].indent base_indent = lines[0].indent
if base_indent != 0: if base_indent != 0:
die( raise YamlSubsetError(
f"yaml-subset: top-level content must start in column 0 " f"yaml-subset: top-level content must start in column 0 "
f"(got column {base_indent} on line {lines[0].lineno})" f"(got column {base_indent} on line {lines[0].lineno})"
) )
value, consumed = _parse_block(lines, 0, 0) value, consumed = _parse_block(lines, 0, 0)
if consumed < len(lines): if consumed < len(lines):
die( raise YamlSubsetError(
f"yaml-subset: trailing content starting on line " f"yaml-subset: trailing content starting on line "
f"{lines[consumed].lineno}" f"{lines[consumed].lineno}"
) )
if not isinstance(value, dict): if not isinstance(value, dict):
die("yaml-subset: top-level value must be a mapping") raise YamlSubsetError("yaml-subset: top-level value must be a mapping")
return cast(dict[str, object], value) return cast(dict[str, object], value)
@@ -576,7 +570,7 @@ def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
fm_end_lineno = line_idx fm_end_lineno = line_idx
break break
if body_start < 0: if body_start < 0:
die("frontmatter: opening `---` has no matching closing `---`") raise YamlSubsetError("frontmatter: opening `---` has no matching closing `---`")
fm_text = text[line_starts[1]:line_starts[fm_end_lineno]] if fm_end_lineno > 1 else "" fm_text = text[line_starts[1]:line_starts[fm_end_lineno]] if fm_end_lineno > 1 else ""
fm = parse_yaml_subset(fm_text) fm = parse_yaml_subset(fm_text)
+4 -4
View File
@@ -13,13 +13,13 @@ Add Content-Length validation and a body-size cap to `git_http_backend.py` so ma
`bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler. `bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler.
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use. For comparison, `supervise_server.py` caps request bodies at 1 MiB. The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use.
## Goals / Success Criteria ## Goals / Success Criteria
- A missing or non-numeric Content-Length returns HTTP 400. - A missing or non-numeric Content-Length returns HTTP 400.
- A negative Content-Length returns HTTP 400. - A negative Content-Length returns HTTP 400.
- A body larger than the cap (1 MiB, matching `supervise_server.py`) returns HTTP 413. - A body larger than the cap (100 MiB) returns HTTP 413.
- Valid Git smart-HTTP pushes and fetches continue to work. - Valid Git smart-HTTP pushes and fetches continue to work.
- Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough. - Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough.
@@ -43,12 +43,12 @@ Out of scope:
## Design ## Design
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 1 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes. Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 100 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
## Testing Strategy ## Testing Strategy
- Unit tests using `unittest.mock` to drive the handler with crafted headers. - Unit tests using `unittest.mock` to drive the handler with crafted headers.
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, `Content-Length: 2097152` (over cap), and a normal small POST body. - Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, a declared length above `MAX_BODY_BYTES`, and a normal small POST body.
Run: Run:
+21
View File
@@ -199,6 +199,25 @@ Named inbound detectors: `naive_injection_detection`.
The manifest parser (`manifest_egress.py`) validates the `dlp` block and The manifest parser (`manifest_egress.py`) validates the `dlp` block and
rejects unknown detector names. rejects unknown detector names.
### Manifest schema — `git` block
HTTPS Git clone/fetch traffic is not implied by a host-level egress route.
Smart HTTP Git fetch uses `git-upload-pack`, which can transfer large repo
packfiles and bypass the git-gate mirror path. It is therefore blocked by
default and must be explicitly enabled per route:
```yaml
egress:
routes:
- host: github.com
git:
fetch: true
```
`git.fetch: true` permits read-only smart HTTP clone/fetch requests
(`git-upload-pack`) after the normal host and `matches` checks pass. HTTPS
Git push (`git-receive-pack`) remains blocked by the egress addon.
### `EgressRoute` changes ### `EgressRoute` changes
`EgressRoute` replaces `PathAllowlist` with `Matches` and gains two new `EgressRoute` replaces `PathAllowlist` with `Matches` and gains two new
@@ -232,6 +251,7 @@ class EgressRoute:
AuthScheme: str = "" AuthScheme: str = ""
TokenRef: str = "" TokenRef: str = ""
Role: tuple[str, ...] = () Role: tuple[str, ...] = ()
GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None # None = all enabled OutboundDetectors: tuple[str, ...] | None = None # None = all enabled
InboundDetectors: tuple[str, ...] | None = None # None = all enabled InboundDetectors: tuple[str, ...] | None = None # None = all enabled
``` ```
@@ -252,6 +272,7 @@ class Route:
matches: tuple[MatchEntry, ...] = () matches: tuple[MatchEntry, ...] = ()
auth_scheme: str = "" auth_scheme: str = ""
token_env: str = "" token_env: str = ""
git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None inbound_detectors: tuple[str, ...] | None = None
``` ```
@@ -1,6 +1,6 @@
# PRD prd-new: User-defined agent provider plugins # PRD 0053: User-defined agent provider plugins
- **Status:** Draft - **Status:** Active
- **Author:** claude - **Author:** claude
- **Created:** 2026-06-04 - **Created:** 2026-06-04
+318
View File
@@ -0,0 +1,318 @@
# PRD 0054: Named / Labelled Agents
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-03
- **Issue:** #171
## Summary
At agent launch time, present the operator with a curses modal to optionally
set a human-readable label and color for the agent before it launches. The
modal pre-fills the label with the current agent name pattern (e.g.
`implementer-a3f9`) and leaves color unset; Enter with no changes accepts
those defaults. Store both in the bottle's `metadata.json`. Display the label —
rendered in the chosen ANSI color — in `cli list active` output, replacing
the bare manifest key. Inject the label and color into the in-container
`claude.json` as `name` / `color` so Claude Code can surface them in its own
harness when upstream support lands.
## Problem
`cli list active` identifies each running instance by its manifest agent key
(e.g., `implementer`) plus a random slug suffix. When an operator runs three
`implementer` bottles simultaneously — one each for three different repos —
the output shows:
```
docker a3f9 implementer egress,pipelock
docker b81c implementer egress,pipelock
docker d220 implementer egress,pipelock
```
There is no way to tell which bottle is working on which task without attaching
to each one in turn. The slug is opaque; the manifest key is shared. Operators
working a multi-bottle session resort to keeping a mental map of slug→task,
which breaks the moment they switch windows.
## Goals / Success Criteria
1. After the operator selects an agent (picker or CLI argument) and backend,
a curses modal appears before the preflight. The modal pre-fills the label
with `<agent_name>-<slug_suffix>` (the same pattern currently shown in
`list active`). No color is pre-selected.
2. In the modal, any printable keystroke immediately replaces the pre-filled
label and starts building the new name. Backspace edits normally. Enter
at any point confirms — accepting the pre-fill if nothing was typed, or
the in-progress text otherwise.
3. After the label field is confirmed, the modal presents color selection:
a list of the 16 ANSI color names the operator can navigate with arrow
keys, or Enter / Esc with no selection to skip color entirely.
4. `label` and `color` are stored in `BottleMetadata` and written to the
bottle's `metadata.json`. Both fields default to `""` (empty / unset).
5. `ActiveAgent` carries `label` and `color`; `enumerate_active()` reads them
from `metadata.json`.
6. `cli list active` shows the label when non-empty (falling back to
`agent_name`). If a non-empty color is set and the terminal supports it,
the label is prefixed with the appropriate ANSI escape code and reset
afterward.
7. `BottleSpec` carries `label` and `color`; both backends' `prepare` steps
copy them into `BottleMetadata`.
8. `ClaudeAgentProvider.provision_plan()` writes `label``"name"` and
`color``"color"` into the generated `claude.json`. Fields are omitted
when empty.
9. `cmd_start` calls `name_color_modal` after backend selection and before
`_launch_bottle`; passes `label` / `color` into `BottleSpec`.
10. All existing unit tests stay green; no new tests are required for this
change (the label/color fields are thin plumbing with no branching logic
worth unit-testing beyond the already-tested metadata read/write path).
## Non-goals
- Showing the agent label inside the Claude Code TUI (status line, terminal
title, custom header). That requires upstream Claude Code / codex support.
Writing to `claude.json` is best-effort scaffolding for when that lands.
- Validating or constraining label content beyond the 64-byte printable cap.
- Editing the label or color of an already-running bottle.
## Design
### Data flow
```
operator input (modal)
BottleSpec.label, BottleSpec.color
├─► docker/prepare.py → BottleMetadata.label / .color → metadata.json
├─► smolmachines/prepare.py → BottleMetadata.label / .color → metadata.json
└─► contrib/claude/agent_provider.py → claude.json {"name": label, "color": color}
(omitted when empty)
cli list active
enumerate_active() → read_metadata(slug) → ActiveAgent.label / .color
cmd_list → label (with ANSI color) in the row string
```
### BottleSpec changes
```python
@dataclass(frozen=True)
class BottleSpec:
manifest: Manifest
agent_name: str
copy_cwd: bool
user_cwd: str
identity: str = ""
label: str = "" # operator-chosen display name; defaults to agent_name at render time
color: str = "" # one of the 16 ANSI color names, or "" for terminal default
```
`label` and `color` default to `""` so all existing callers remain valid with
no changes.
### BottleMetadata changes
Add two new fields with backward-compatible defaults:
```python
@dataclass
class BottleMetadata:
identity: str
agent_name: str
cwd: str
copy_cwd: bool
started_at: str
compose_project: str
backend: str
label: str = ""
color: str = ""
```
`metadata.json` written by older bot-bottle versions won't have these keys;
`read_metadata` already uses `dict.get` with defaults, so existing slugs load
cleanly with `label=""`, `color=""`.
### ActiveAgent changes
```python
@dataclass(frozen=True)
class ActiveAgent:
backend_name: str
slug: str
agent_name: str
started_at: str
services: tuple[str, ...]
label: str = ""
color: str = ""
```
`enumerate_active()` copies `label` and `color` out of `BottleMetadata` when
constructing each `ActiveAgent`. The smolmachines backend gets the same
additions for symmetry.
### `cli list active` rendering
The current row format is tab-separated:
`{backend}\t{slug}\t{agent_name}\t{services}`
With labels it becomes:
```python
display_name = a.label if a.label else a.agent_name
```
Color is rendered via ANSI escape codes. A small `_ansi_color(color_name)`
helper returns the appropriate escape prefix for the 16 named colors, or `""`
when the name is unrecognised or the terminal doesn't support color
(`NO_COLOR` env var or `not sys.stdout.isatty()`).
The 16 ANSI color name → escape mapping:
| Name | ANSI code |
|------|-----------|
| `black` | `\033[30m` |
| `red` | `\033[31m` |
| `green` | `\033[32m` |
| `yellow` | `\033[33m` |
| `blue` | `\033[34m` |
| `magenta` | `\033[35m` |
| `cyan` | `\033[36m` |
| `white` | `\033[37m` |
| `bright-black` | `\033[90m` |
| `bright-red` | `\033[91m` |
| `bright-green` | `\033[92m` |
| `bright-yellow` | `\033[93m` |
| `bright-blue` | `\033[94m` |
| `bright-magenta` | `\033[95m` |
| `bright-cyan` | `\033[96m` |
| `bright-white` | `\033[97m` |
Reset is `\033[0m`. Applied around the label substring only.
### The label+color modal
A single curses modal (`name_color_modal` in `bot_bottle/cli/tui.py`) handles
both label and color in two sequential steps within the same window.
```python
label, color = name_color_modal(default_label=f"{agent_name}-{slug_suffix}")
```
**Step 1 — label.** The window renders:
```
Name agent
──────────────────────────────────────
implementer-a3f9
──────────────────────────────────────
[any key] edit [Enter] confirm
```
The pre-filled text is shown in the input field. Any printable keystroke
immediately clears the pre-fill and starts a new name from that character
(first-keystroke-replaces semantics). Subsequent keystrokes append normally.
Backspace edits from the right. Enter confirms — accepting the pre-fill if
the field was never edited, or the typed text otherwise.
**Step 2 — color.** After confirming the label, the window transitions to:
```
Name agent
──────────────────────────────────────
implementer-a3f9 ← confirmed label
──────────────────────────────────────
Color (optional)
> (none)
red
green
blue
──────────────────────────────────────
[↑↓] move [Enter] select [Esc] skip
```
The list starts with `(none)` selected. Arrow keys move the cursor; Enter
confirms the highlighted choice; Esc or `q` skips color. Each color name in
the list is rendered in its own curses color so the operator can preview the
palette.
The function returns `(label, color)` — both strings, `color` is `""` when
`(none)` is selected or the step is skipped.
### Slug suffix for the default label
The default label is `<agent_name>-<slug_suffix>`, where `slug_suffix` is the
last four characters of the slug (the same short hash shown in `list active`).
In `cmd_start` the slug is minted inside `prepare`, after the modal appears.
The modal is therefore called with the manifest agent key as a fallback
(`default_label=agent_name`). Once `prepare` returns the plan (which contains
the slug), the `BottleSpec` is not reconstructed — the label entered by the
operator is already in the spec. The full `<agent_name>-<slug_suffix>` form is
only available for display in subsequent `list active` calls once the bottle
is running.
### Claude Code config injection
Per PRD 0050, the `claude.json` trust-marker file is written by
`ClaudeAgentProvider.provision_plan()` in
`bot_bottle/contrib/claude/agent_provider.py`. Add `label: str = ""` and
`color: str = ""` keyword parameters to `provision_plan()` on both the
`AgentProvider` ABC and `ClaudeAgentProvider`, and to the
`agent_provision_plan()` shim in `agent_provider.py`. Both `prepare.py`
modules pass `spec.label` / `spec.color`; `CodexAgentProvider` accepts the
params and ignores them.
In `ClaudeAgentProvider.provision_plan()`:
```python
payload = {
"hasCompletedOnboarding": True,
"theme": "dark",
"bypassPermissionsModeAccepted": True,
"projects": claude_projects,
}
if label:
payload["name"] = label
if color:
payload["color"] = color
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
```
## Implementation chunks
Two PRs, each independently mergeable.
### Chunk 1 — schema + storage
- Add `label: str = ""` and `color: str = ""` to `BottleSpec`,
`BottleMetadata`, and `ActiveAgent`.
- `docker/prepare.py` and `smolmachines/prepare.py`: copy `spec.label` /
`spec.color` into `BottleMetadata`; pass them to `agent_provision_plan()`.
- `docker/enumerate.py` and smolmachines equivalent: copy `metadata.label` /
`metadata.color` into `ActiveAgent`.
- Add `label: str = ""` and `color: str = ""` keyword params to
`AgentProvider.provision_plan()` (ABC), `ClaudeAgentProvider.provision_plan()`
(uses them in the `claude.json` write), and the `agent_provision_plan()` shim.
`CodexAgentProvider` accepts the params and ignores them.
- `cmd_list`: update `list active` row to use `label` when non-empty, with
ANSI color escape codes.
- No prompt changes; no UI changes. All existing behavior is identical.
### Chunk 2 — modal
- `bot_bottle/cli/tui.py`: add `name_color_modal(default_label)` implementing
the two-step curses window described above.
- `cmd_start`: call `name_color_modal(default_label=agent_name)` after backend
selection and before `_launch_bottle`; pass `label` / `color` into
`BottleSpec`.
## Open questions
None.
+148
View File
@@ -0,0 +1,148 @@
# PRD 0055: Egress traffic logging
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-06
- **PR:** #207
## Summary
Adds structured log levels to the egress proxy so operators can observe
traffic and security decisions without modifying any application code.
Three integer levels control verbosity: `0` (off), `1` (security events
only), and `2` (full request/response capture). All output is JSON lines
written to stderr.
## Problem
The egress proxy makes per-request allow/block decisions and DLP scans, but
until now those decisions are invisible unless something is actively blocked
and the caller inspects the 403 body. Debugging unexpected blocks, auditing
what an agent is sending upstream, and verifying DLP detector behaviour all
require adding ad-hoc instrumentation or tailing the sidecar container logs
with no structure to grep against.
## Goals / Success Criteria
1. **Level 0 (off, default):** no egress output to stderr beyond the boot
line. Existing behaviour for production deployments.
2. **Level 1 (blocks):** every block or DLP warn event is emitted to stderr
as a JSON line with the event type, human-readable reason (including the
secret type detected for DLP hits), and the request context (host, method,
path; plus upstream status code for response-phase events). No traffic
bodies are logged.
3. **Level 2 (full):** all level-1 events, plus a `egress_request` JSON line
for every forwarded request (method, path, headers, body after auth
injection) and an `egress_response` JSON line for every response that
passes DLP (status, headers, body).
4. The log level is a single integer field `log` at the top of the egress
config (routes.yaml in the sidecar; `egress.log` in the bottle manifest).
Values other than 0, 1, 2 are rejected at parse time on both sides.
5. The boot message includes the active log level label (`off`, `blocks`,
`full`).
## Non-goals
- Log rotation or file sinks — stderr output is captured by the container
runtime (Docker, smolmachines) and goes wherever the operator routes it.
- Per-route log levels — all routes share the global level.
- Redacting secrets from the level-2 body dump — at level 2 the operator
has explicitly requested full visibility; redaction belongs in the
log consumer, not the proxy.
## Design
### Wire format
`routes.yaml` gains an optional top-level `log` key:
```yaml
log: 1 # 0 = off (default), 1 = blocks, 2 = full
routes:
- host: "api.anthropic.com"
...
```
The field is omitted entirely when the level is 0 (default).
### Manifest format
```yaml
egress:
log: 1
routes:
- host: "api.anthropic.com"
...
```
`egress.log` accepts integers 0, 1, or 2. Booleans and strings are rejected.
### Log events
**Block / DLP block (level ≥ 1):**
```json
{
"event": "egress_block",
"reason": "egress DLP: GitHub token (classic) found in request",
"host": "api.github.com",
"method": "POST",
"path": "/gists"
}
```
Response-phase block also includes `"response_status"`.
**DLP warn (level ≥ 1):**
```json
{
"event": "egress_warn",
"reason": "egress DLP: possible prompt injection detected",
"host": "api.anthropic.com",
"method": "POST",
"path": "/v1/messages",
"response_status": 200
}
```
**Forwarded request (level 2):**
```json
{
"event": "egress_request",
"host": "api.anthropic.com",
"method": "POST",
"path": "/v1/messages",
"headers": { "authorization": "Bearer sk-ant-...", "content-type": "application/json" },
"body": "{\"model\": \"claude-opus-4-8\", ...}"
}
```
The request is logged after auth injection, so the outgoing `Authorization`
header is present. The agent's original `Authorization` header is stripped
before logging.
**Response (level 2):**
```json
{
"event": "egress_response",
"host": "api.anthropic.com",
"status": 200,
"headers": { "content-type": "application/json" },
"body": "{\"id\": \"msg_...\", ...}"
}
```
Responses are logged before DLP scanning, so the body is always the raw
upstream response.
### Implementation
- **`egress_addon_core.py`**: `Config.log: int = LOG_OFF` (`LOG_OFF=0`,
`LOG_BLOCKS=1`, `LOG_FULL=2`). `parse_config()` validates the integer and
rejects booleans.
- **`egress_addon.py`**: `_block()` emits JSON when `log >= LOG_BLOCKS`. The
`_req_ctx()` helper builds `{host, method, path}` for every call site.
`_log_request()` / `_log_response()` fire when `log >= LOG_FULL`.
- **`manifest_egress.py`**: `EgressConfig.Log: int = 0`, parsed from
`egress.log`, validated against `{0, 1, 2}`.
- **`egress.py`**: `egress_render_routes(routes, *, log: int = 0)` emits
`log: N` at the top of routes.yaml when N > 0. `EgressPlan.log: int = 0`.
+170
View File
@@ -0,0 +1,170 @@
# PRD 0056: Extended outbound DLP scan surfaces
- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-06
- **Issue:** #204
## Summary
The outbound DLP scan introduced in PRD 0052 covers only the request body and
the `Authorization` header. An agent can exfiltrate a provisioned secret or
credential token via several other request surfaces that are currently
unscanned: arbitrary HTTP headers, URL query parameters, URL path segments,
and DNS-level hostnames (DNS tunnelling). This PRD closes those gaps by
extending the scan corpus to cover all four surfaces.
## Problem
After PRD 0052, `egress_addon.py` builds the DLP scan text as:
```python
auth_header = flow.request.headers.get("authorization", "")
scan_text = body
if auth_header:
scan_text = auth_header + "\n" + body
```
This leaves the following attack surfaces unscanned:
| Surface | Example exfiltration vector |
|---|---|
| Other request headers | `X-Api-Key: <secret>`, `Cookie: token=<secret>` |
| Query parameters | `GET /search?api_key=<secret>` |
| URL path segments | `GET /proxy/<base64-secret>/endpoint` |
| DNS hostname | `<base64-secret>.attacker.com` (DNS tunnelling) |
Each surface allows a determined agent to bypass the current scan and leak
a provisioned `EGRESS_TOKEN_*` value or a known token pattern to an
upstream attacker.
## Goals / Success Criteria
1. All four surfaces (headers, query params, path, hostname) are included in
the outbound DLP scan text for every route that has outbound scanning
enabled.
2. A pure helper `build_outbound_scan_text(host, path, query, headers, body)`
in `egress_addon_core.py` assembles the scan corpus so the logic is fully
unit-testable without a mitmproxy dependency.
3. Unit tests demonstrate that `scan_outbound` blocks a request when a known
token pattern or provisioned secret appears in each surface independently.
4. No manifest schema changes — the `dlp` block's `outbound_detectors`
field continues to control which detectors run; all surfaces are scanned
by whichever detectors are active.
5. The auth-strip ordering invariant from PRD 0052 is preserved: the
outbound scan sees the original `Authorization` header before the addon
strips it.
## Non-goals
- Raw UDP/DNS queries — these bypass the HTTP proxy entirely and require a
network-level DNS sinkhole (tracked separately in issue #205).
- Structured query-param parsing — scanning the raw query string is
sufficient.
- Changes to the `dlp` block schema or detector names.
- Scanning outbound request bodies for prompt injection (inbound only,
per PRD 0052 design).
- LLM-based semantic detection or entropy-based secret scanning (deferred,
per PRD 0052 non-goals).
## Design
### `build_outbound_scan_text` in `egress_addon_core.py`
A new pure function assembles all request surfaces into a single newline-
delimited string suitable for passing to `scan_outbound`:
```python
def build_outbound_scan_text(
host: str,
path: str,
query: str,
headers: typing.Mapping[str, str],
body: str,
) -> str:
parts: list[str] = [host, path]
if query:
parts.append(query)
for name, value in headers.items():
parts.append(f"{name}: {value}")
if body:
parts.append(body)
return "\n".join(parts)
```
**Why hostname in the scan corpus?**
DNS tunnelling encodes data into subdomain labels
(`<base64-secret>.attacker.com`). The mitmproxy `request` hook sees the
`pretty_host` field before the TCP connection is fully established, so
scanning it catches this vector. Both the `token_patterns` and
`known_secrets` detectors handle encoded variants (raw, base64, URL-encoded,
hex), so the existing encoding-variant logic in `_encoded_variants` already
covers common DNS-tunnelling encodings.
### `egress_addon.py` update
The narrow scan-text construction is replaced with a call to
`build_outbound_scan_text`, which the addon has already split `path` and
`query` from `flow.request.path` at the top of `request()`:
```python
# Build full scan corpus: hostname + path + query + all headers + body
body = flow.request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
dict(flow.request.headers),
body,
)
dlp_result = scan_outbound(route, scan_text, os.environ)
```
The `Authorization` header is present in `flow.request.headers` at this
point (the strip happens below on line 115), so the auth-strip ordering
invariant is automatically preserved.
### `build_inbound_scan_text` in `egress_addon_core.py`
An analogous helper assembles the inbound response corpus (all response
headers + body) for `scan_inbound`. The `response()` hook now passes this
combined text instead of the body alone, closing the response-header
injection vector.
### WebSocket frame scanning
A new `websocket_message` hook in `EgressAddon` scans every frame after the
HTTP 101 upgrade. Outbound frames (`from_client=True`) are scanned for
credential patterns and known secrets; inbound frames are scanned for prompt
injection. On a block the entire WebSocket connection is killed via
`flow.kill()` (there is no HTTP response surface to write to after upgrade).
### Extended encoding variants in `_encoded_variants`
`_encoded_variants` is extended from 4 to 9 encoding forms:
| Added encoding | Rationale |
|---|---|
| Standard base64 without padding | Common in log lines where `=` is stripped |
| URL-safe base64 with padding | JWT / OAuth standard alphabet |
| URL-safe base64 without padding | Same, padding stripped |
| Hex uppercase | Complements existing hex-lowercase variant |
| Base32 | TOTP seeds; some DNS-exfil channels use base32 subdomains |
| gzip + base64 | Recognisable by `H4sI` prefix; naive compression before encode |
### OpenAI project key pattern
`TOKEN_PATTERNS` gains `sk-proj-[A-Za-z0-9_\-]{48,}` covering OpenAI's
newer project-scoped API key format.
## Implementation
Delivered across three commits on the same branch:
1. **Outbound scan surfaces**`build_outbound_scan_text`, `egress_addon.py`
`request()` rewrite, `TestBuildOutboundScanText`, `TestScanOutbound`.
2. **Remaining gaps** — extended `_encoded_variants`, `sk-proj-` pattern,
`build_inbound_scan_text`, response-header scanning, `websocket_message`
hook, and matching unit tests.
3. **PRD flip**`Status: Draft → Active` (committed with the first
implementation commit; updated here to reflect final scope).
+79
View File
@@ -0,0 +1,79 @@
# PRD 0057: Promote smolmachines to default backend; convert Docker to example-only
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-06
- **Issue:** #206
## Summary
Make smolmachines the default bot-bottle backend and demote Docker to an example-only configuration. This closes the DNS sinkhole gap that exists in the Docker backend: the mitmproxy egress addon intercepts HTTP(S) but cannot see raw UDP port-53 DNS queries, so an agent can exfiltrate data via DNS tunnelling without the egress guard seeing it. The smolmachines backend eliminates this gap at the VMM layer — DNS filtering is built in and the agent container cannot bypass it.
## Problem
The current default backend is Docker. The egress addon (PRDs 0052/0053) intercepts HTTPS and scans request/response surfaces, but it is an HTTP proxy: raw UDP/TCP port-53 DNS queries go to the OS resolver and never pass through it. An agent can encode secrets as base32 or hex subdomains in a DNS query (`<encoded>.attacker.com`) and exfiltrate them silently.
The smolmachines backend already solves this: its Transport Socket Interface (TSI) enforces a CIDR allowlist at the VMM layer, and DNS is handled via vsock port 6002 — the guest's `/etc/resolv.conf` points at `127.0.0.1`, and a guest-side DNS proxy tunnels queries over vsock to the host, which returns NXDOMAIN for anything not on the allowlist. The agent cannot bypass this by hardcoding IPs or by configuring an alternate resolver, because both mechanisms are enforced below the guest OS.
Docker has no equivalent. Adding dnsmasq to the Docker backend would close the gap at some cost (dnsmasq sidecar, iptables `NET_ADMIN`, per-launch config generation), but it is the wrong direction if smolmachines supersedes Docker anyway.
## Goals / Success Criteria
- `BOT_BOTTLE_BACKEND` defaults to `smolmachines` when not set.
- The existing Docker backend remains functional (not removed) but is no longer the default and is documented as legacy/example-only.
- Example bottles (`examples/bottles/`) reference smolmachines, not Docker.
- `AGENTS.md` documents the backend choice and the DNS gap closure.
- Existing Docker-backed integration tests continue to pass; they select Docker explicitly via `BOT_BOTTLE_BACKEND=docker` rather than relying on the default.
## Non-goals
- Removing the Docker backend or its tests.
- Implementing a dnsmasq layer for the Docker backend (closed by this change; not needed on the default path).
- Iptables / `NET_ADMIN` work for Docker (deferred).
- Subdomain-depth filtering for allowlisted zones (documented residual gap; tracked separately per the issue).
## Design
### Default backend change
`bot_bottle/backend/__init__.py`, line ~440:
```python
# Before
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
# After
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "smolmachines"
```
### DNS gap closure (how smolmachines handles it)
When the smolmachines backend launches an agent VM:
1. The VM's network device uses TSI (`--allow-host` / `--allow-cidr` flags), which enforces a CIDR allowlist at the VMM layer. The guest cannot dial IPs outside the allowlist even with raw sockets.
2. The guest's `/etc/resolv.conf` is set to `127.0.0.1`; a guest-side DNS proxy relays queries over vsock port 6002 to the host.
3. The host-side DNS filter returns NXDOMAIN for any hostname not in the allowlist derived from `egress.routes` in the bottle manifest.
This means DNS exfiltration via unknown subdomains is blocked by NXDOMAIN before the query leaves the host, and even if the agent hardcoded the IP of an attacker-controlled server, TSI would drop the packet at the VMM layer.
**Residual gap:** if the attacker controls a subdomain of an allowlisted zone (e.g., a legitimate zone like `api.anthropic.com` that the attacker can inject into via a separate compromise), DNS queries for that subdomain would be forwarded. This is accepted and documented.
### Example bottles
Update `examples/bottles/dev.md` and `examples/bottles/claude.md` to remove Docker-specific notes and reference smolmachines as the runtime.
### Integration test migration
Tests that exercise the Docker backend explicitly should set `BOT_BOTTLE_BACKEND=docker` rather than relying on the default. Tests that are backend-agnostic continue to use whatever `BOT_BOTTLE_BACKEND` is set to (defaulting to smolmachines in the test environment if available).
## Resolved questions
- **TSI + egress proxy loopback.** The implementation uses a per-bottle loopback alias rather than broad `127.0.0.1` passthrough. The smolmachines launch integration test now asserts that the guest receives proxy env vars on a `127.x` alias, can reach an allowlisted host through the proxy, cannot reach the same host directly with proxy vars unset, and cannot reach a non-allowlisted host through the proxy.
- **smolmachines availability check.** The smolmachines preflight error points operators at the smolvm installer and explicitly suggests `BOT_BOTTLE_BACKEND=docker` / `--backend=docker` for legacy Docker-backed runs.
## References
- `docs/research/smolmachines-as-vm-backend.md` — smolmachines evaluation
- `docs/research/network-egress-guard.md` — Approach 4 (DNS-based egress control)
- `docs/research/secret-exfil-tripwire-encodings.md` — DNS exfil discussion
- PRD 0052, PRD 0053 — egress DLP addon (HTTP-level; partial mitigation only)
+123
View File
@@ -0,0 +1,123 @@
# PRD 0058: Add built-in Pi agent provider
- **Status:** Active
- **Author:** codex
- **Created:** 2026-06-09
- **Issue:** #221
## Summary
Add `pi` as a built-in `agent_provider.template`. The provider runs the Pi
coding-agent CLI, provisions its agent config under `~/.pi/agent`, and writes a
provider settings file that targets an unauthenticated Ollama-compatible server.
The default settings assume an Ollama server at `http://ollama:11434/v1`, using
the `openai-completions` API with a dummy API key because Ollama ignores it.
Users can override the provider id, base URL, model list, API key, API-key env
reference, API type, and compatibility flags through a new
`agent_provider.settings` object.
## Problem
bot-bottle currently ships Claude and Codex as built-in agent providers. Pi is a
useful third harness, but using it today requires a custom provider plugin and a
custom image. That repeats boilerplate for prompt copying, skill copying,
provider config, and runtime registration.
Pi's local-model path is also easy to misconfigure: its custom-model docs require
`~/.pi/agent/models.json`, an API entry, at least one model id, and a dummy
`apiKey` for Ollama even though the server does not authenticate. bot-bottle
should generate that shape consistently.
## Goals / Success Criteria
- `agent_provider.template: pi` is accepted as a built-in provider.
- `bot_bottle/contrib/pi/` provides a Pi image and `PiAgentProvider`.
- Pi receives the bot-bottle prompt at `~/.bot-bottle-prompt.txt` and starts in
print-mode prompt delivery like Codex.
- Pi skills are copied into `~/.pi/agent/skills/<name>/`.
- Pi provider settings are configurable from the bottle manifest via
`agent_provider.settings`.
- The default Pi provider settings configure an unauthenticated Ollama-compatible
server.
- Unit tests cover manifest parsing, runtime selection, plan generation, prompt,
skills, and provider provisioning.
## Non-goals
- Managing or launching an Ollama server.
- Authenticating to Ollama or any remote Pi provider.
- Forwarding host Pi credentials.
- Implementing Pi extensions or MCP registration.
- Changing Claude or Codex provider behavior.
## Design
### Manifest
Extend `agent_provider` with an optional `settings` object. It is currently only
supported for built-in `pi`.
Supported keys:
- `base_url`: string, defaults to `http://ollama:11434/v1`
- `provider`: string, defaults to `ollama`
- `api`: string, defaults to `openai-completions`
- `api_key`: string, defaults to `ollama`
- `api_key_env`: string, optional host env var name for egress auth injection
- `models`: non-empty array of strings, defaults to `["qwen2.5-coder:7b"]`
- `context_window`: positive integer, defaults to `4096`; this is the Ollama
runtime context, and bot-bottle subtracts `max_tokens` before writing Pi's
`contextWindow` so output space is reserved
- `max_tokens`: positive integer, defaults to `1024`
- `max_tokens_field`: `max_tokens` or `max_completion_tokens`, defaults to
`max_tokens`
- `supports_developer_role`: boolean, defaults to `false`
- `supports_reasoning_effort`: boolean, defaults to `false`
The snake-case manifest keys are converted into Pi's JSON field names:
`baseUrl`, `apiKey`, `contextWindow`, `maxTokens`,
`supportsDeveloperRole`, and `supportsReasoningEffort`. `context_window`
describes the server's total context; Pi's `contextWindow` receives
`context_window - max_tokens` because Pi uses it as an input compaction target.
`api_key` and `api_key_env` are mutually exclusive. When targeting a hosted
provider through bot-bottle's egress sidecar, omit `api_key` and set
`api_key_env` to the host env var that holds the API key. The generated
`models.json` receives only an `egress-placeholder` API key, and the egress
route injects the real `Authorization` header from the sidecar env. For example,
OpenRouter can use provider id `openrouter` with
`api_key_env: OPENROUTER_API_KEY`, keeping the key out of the agent env and
`models.json`.
### Provider
`PiAgentProvider.provision_plan` writes `models.json` into the per-launch state
directory and returns an `AgentProvisionPlan` that copies it to
`~/.pi/agent/models.json`. The provider also declares an unauthenticated egress
route for the configured base URL host so the egress layer can allow the Ollama
endpoint.
The Pi runtime uses:
- `command="pi"`
- `prompt_mode="append_system_prompt"`
- `image="bot-bottle-pi:latest"`
- `bypass_args=()`
- `resume_args=()`
- `remote_control_args=()`
The Dockerfile installs `@earendil-works/pi-coding-agent` globally from npm and
keeps the same Debian/node base shape as the existing provider images.
### Supervise MCP
Pi does not have built-in MCP support in the current public docs, so
`provision_supervise_mcp` is a no-op. This keeps Pi bottles launchable with
`supervise: true` while preserving the explicit non-goal of implementing Pi
extensions.
## Merge rule(s)
This PR can merge when the focused unit tests pass and the PRD status is flipped
from Draft to Active in the final implementation commit.
+190
View File
@@ -0,0 +1,190 @@
# PRD 0059: macOS Container backend
- **Status:** Active
- **Author:** Codex
- **Created:** 2026-06-10
- **Issue:** #220
## Summary
Add a `macos-container` backend that integrates Apple's `container`
CLI as a host runtime on macOS. The shipped slices register the
backend, implement reusable host primitives (`build`, `exec`, `cp`,
image inspection, cleanup, active enumeration), make launch runnable
with the proven two-network sidecar topology, and add real-runtime
coverage without weakening bot-bottle's sidecar egress model.
## Problem
bot-bottle currently has two local execution paths:
- `docker`, which runs the whole bottle topology through Docker
Compose.
- `smolmachines`, which runs the agent in smolvm but still depends on
Docker for the sidecar bundle and image-building pipeline.
Issue #220 explored removing Docker as a host dependency. A follow-up
review comment verified that smolvm can publish guest ports back to
host loopback and that another smolvm guest can reach that service
through the existing per-bottle loopback alias plus `--allow-cidr`
path. That keeps the VM-contained sidecar direction viable and rejects
the host-process sidecar fallback.
Apple's `container` CLI is another macOS-native way to run OCI images
as lightweight Linux VMs. Its current command surface includes
Docker-like `build`, `run`, `exec`, `cp`, port publishing, image
inspection, and user-defined networks. That makes it a plausible local
backend, but it does not remove the need to preserve bot-bottle's
sidecar enforcement property: the agent must not have a direct egress
path around the egress sidecar.
## Goals / Success Criteria
- `--backend=macos-container` and
`BOT_BOTTLE_BACKEND=macos-container` are accepted by the existing
backend selector.
- Compatible macOS hosts default to `macos-container` when
`BOT_BOTTLE_BACKEND` and `--backend` are both unset.
- Backend availability is true only on macOS hosts with `container` on
`PATH`.
- The backend has tested wrappers for Apple Container image build,
image inspection, container `exec`, container `cp`, cleanup, and
active-agent enumeration.
- Full launch uses a host-only internal network for the agent and a
separate NAT egress network for the sidecar bundle.
- The agent container does not attach to the egress network. It reaches
allowed outbound hosts through HTTP(S)_PROXY pointing at the
sidecar's internal-network IP.
- `bottle.git` / git-gate bottles fail loudly on this backend until a
safe Apple Container key-delivery path exists.
- Real-runtime integration coverage is present and guarded by macOS and
Apple Container availability.
## Non-goals
- Do not remove or deprecate the Docker backend.
- Do not remove or deprecate the smolmachines backend.
- Do not run sidecar daemons as host processes.
- Do not launch a degraded backend where the agent can bypass the
egress sidecar through direct network access.
- Do not require Docker Desktop as part of the macOS Container backend.
## Design
### Backend name
The selectable backend name is `macos-container`. The Python package
uses `bot_bottle.backend.macos_container` because module names cannot
contain hyphens.
### Availability and preflight
`MacosContainerBottleBackend.is_available()` returns true only when:
- `platform.system() == "Darwin"`
- `container` is discoverable on `PATH`
`prepare()` calls `require_container()`, which produces a concrete
install pointer and rejects non-macOS hosts.
### Implemented primitives
The backend owns an Apple Container wrapper module instead of reusing
Docker wrappers. The wrapper maps bot-bottle's backend needs to
Apple's CLI:
| bot-bottle need | Apple Container command |
|---|---|
| Build provider image | `container build -t <ref> [-f Dockerfile] <context>` |
| Run agent commands | `container exec [--interactive --tty] <id> ...` |
| Copy files into guest | `container cp <host> <id>:<path>` |
| Inspect image identity | `container image inspect <ref>` |
| Cleanup stale containers | `container delete --force <id>` |
| Cleanup stale networks | `container network delete <name>` |
| Active enumeration | `container list --quiet` |
The bottle handle mirrors `DockerBottle`: it builds a host argv for
foreground agent execution, pipes shell snippets through stdin for
`Bottle.exec`, and exposes `cp_in` for provisioning.
### Launch topology
`launch()` uses Apple Container's two-network topology:
- create a host-only internal network for the bottle;
- create a normal NAT egress network for the sidecar bundle;
- start the sidecar bundle attached to the egress network first and the
internal network second;
- discover the sidecar's internal-network IPv4 address from
`container inspect`;
- start the agent attached only to the internal network, with
HTTP_PROXY / HTTPS_PROXY / lowercase proxy vars pointing at the
sidecar IP and egress port.
This keeps the agent off the outbound network while preserving the
proxy-env contract that existing agent tooling already honors. The
integration smoke also removes the proxy env in-guest and confirms
direct egress fails.
### Deferred git-gate support
Apple Container currently rejects single-file bind mounts, and
`container cp` into a stopped container is not available. Starting the
container earlier would allow `container cp` into a running container,
but it would also mean delivering SSH private key material into a live
sidecar before the git-gate daemon is ready to own it. Mounting broad
host SSH directories is not acceptable.
For this PRD, `bottle.git` / git-gate support is explicitly deferred on
the `macos-container` backend. Bottles with git-gate upstreams fail
loudly and should use `docker` or `smolmachines` until a narrower key
delivery design lands.
## Implementation chunks
1. Register `macos-container`, add availability/preflight, bottle
handle, utility wrappers, cleanup, active enumeration, unit tests,
and this PRD.
2. Spike Apple Container networking against real macOS 26 hosts:
repeated `--network`, internal network egress behavior, published
loopback reachability from another container, DNS behavior, and
labels/JSON output stability.
3. Implement launch once the enforcement shape is proven. Reuse the
existing sidecar bundle image and daemon subset env contract where
possible.
4. Add real-runtime integration tests guarded by `container` presence
and macOS version.
5. Consider moving smolmachines sidecar/image-building work to
VM-contained or Apple Container-backed execution only after the
`macos-container` launch path is trustworthy.
## Testing Strategy
- Unit tests cover backend registration through `known_backend_names`.
- Unit tests cover availability/preflight behavior without requiring
macOS.
- Unit tests cover `MacosContainerBottle` command construction and
stdin-based shell execution.
- Unit tests cover cleanup and active enumeration parsing.
- Unit tests cover launch argv/env construction, sidecar mount
staging, sidecar IP parsing, and git-gate rejection.
- Integration tests run on macOS hosts with Apple Container installed
and verify that egress cannot bypass the sidecar. They also preflight
Apple Container BuildKit DNS because image builds must resolve
package mirrors before a launch smoke can be meaningful. The backend
probes the running builder before image builds and leaves it alone
when its current resolver works. If the probe fails, or if the
operator explicitly sets `BOT_BOTTLE_MACOS_CONTAINER_DNS`, the backend
restarts the Apple Container builder with the configured DNS server.
Without an explicit override, that server is discovered from the
host's directly reachable IPv4 resolver before falling back to a
public resolver.
## References
- [Issue #220 review comment](https://gitea.dideric.is/didericis/bot-bottle/issues/220#issuecomment-1980):
smolvm `--port/-p` can expose a guest service to host loopback, and
another smolvm guest can reach it through the existing per-bottle
loopback alias path.
- Apple Container command reference: `container run`, `build`, `exec`,
port publishing, and network commands.
@@ -0,0 +1,360 @@
# Apple Container networking spike
Issue: https://gitea.dideric.is/didericis/bot-bottle/issues/230
## Summary
Apple Container 1.0.0 on macOS 26 can support the core two-network
sidecar shape, but not as a drop-in Docker Compose clone.
The viable shape is:
- agent container on one `--internal` host-only network;
- sidecar bundle container on both the NAT egress network and the
host-only agent network;
- sidecar network flags ordered with the NAT network first, because
Apple Container chooses the first network as the default route;
- explicit DNS on the sidecar, because the tested NAT gateway routed
packets but did not resolve DNS;
- agent talks to sidecar by the sidecar's host-only-network IP, not by
container name or host-published loopback alias.
This is enough to unblock a cautious `macos-container` launch spike if
the backend records inspect-derived IPs and avoids depending on Docker
Compose-style aliases. It is not enough to reuse the Docker backend's
service-name assumptions unchanged.
## Local Environment
Tested on 2026-06-10:
```console
$ sw_vers
ProductName: macOS
ProductVersion: 26.5.1
BuildVersion: 25F80
$ uname -m
arm64
$ container --version
container CLI version 1.0.0 (build: release, commit: ee848e3)
$ container system version --format json
[
{
"appName": "container",
"buildType": "release",
"commit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
"version": "1.0.0"
},
{
"appName": "container-apiserver",
"buildType": "release",
"commit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
"version": "container-apiserver version 1.0.0 (build: release, commit: ee848e3)"
}
]
$ container system status --format json
{
"apiServerAppName": "container-apiserver",
"apiServerBuild": "release",
"apiServerCommit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
"apiServerVersion": "container-apiserver version 1.0.0 (build: release, commit: ee848e3)",
"appRoot": "/Users/didericis/Library/Application Support/com.apple.container/",
"installRoot": "/usr/local/",
"status": "running"
}
```
Apple Container was installed from the official signed 1.0.0 GitHub
release package, `container-1.0.0-installer-signed.pkg`. The package was
signed by `Developer ID Installer: Apple Inc. - Containerization
(UPBK2H6LZM)` and notarized by Apple.
## Commands Run
Create the networks:
```bash
container network create bb-spike-230-agent \
--internal \
--label bot-bottle.spike=apple-container-networking
container network create bb-spike-230-egress \
--label bot-bottle.spike=apple-container-networking
```
`container network inspect bb-spike-230-agent bb-spike-230-egress`
showed:
```json
[
{
"configuration": {
"labels": {"bot-bottle.spike": "apple-container-networking"},
"mode": "hostOnly",
"name": "bb-spike-230-agent",
"plugin": "container-network-vmnet"
},
"id": "bb-spike-230-agent",
"status": {
"ipv4Gateway": "192.168.128.1",
"ipv4Subnet": "192.168.128.0/24"
}
},
{
"configuration": {
"labels": {"bot-bottle.spike": "apple-container-networking"},
"mode": "nat",
"name": "bb-spike-230-egress",
"plugin": "container-network-vmnet"
},
"id": "bb-spike-230-egress",
"status": {
"ipv4Gateway": "192.168.66.1",
"ipv4Subnet": "192.168.66.0/24"
}
}
]
```
Repeated `--network` flags are accepted. With the agent network first,
the sidecar got two interfaces but the default route pointed at the
host-only gateway, so egress failed:
```bash
container run --name bb-spike-230-sidecar \
--label bot-bottle.spike=apple-container-networking \
--network bb-spike-230-agent \
--network bb-spike-230-egress \
--detach --rm docker.io/python:alpine \
sh -c 'mkdir -p /srv && printf ok >/srv/index.html && cd /srv && python3 -m http.server 80 --bind 0.0.0.0'
container exec bb-spike-230-sidecar sh -c 'ip route && cat /etc/resolv.conf'
```
Observed:
```console
default via 192.168.128.1 dev eth0
192.168.66.0/24 dev eth1 scope link src 192.168.66.3
192.168.128.0/24 dev eth0 scope link src 192.168.128.3
nameserver 192.168.128.1
```
With the NAT network first and explicit DNS, the sidecar can egress:
```bash
container run --name bb-spike-230-sidecar \
--label bot-bottle.spike=apple-container-networking \
--network bb-spike-230-egress \
--network bb-spike-230-agent \
--dns 1.1.1.1 \
--detach docker.io/python:alpine \
sh -c 'mkdir -p /srv && printf ok >/srv/index.html && cd /srv && python3 -m http.server 80 --bind 0.0.0.0'
container exec bb-spike-230-sidecar sh -c 'ip route; cat /etc/resolv.conf; wget -T 8 -O- https://example.com'
```
Observed:
```console
default via 192.168.66.1 dev eth0
192.168.66.0/24 dev eth0 scope link src 192.168.66.5
192.168.128.0/24 dev eth1 scope link src 192.168.128.7
nameserver 1.1.1.1
Connecting to example.com (172.66.147.243:443)
... 100%
```
Start an agent only on the host-only network:
```bash
container run --name bb-spike-230-agent \
--label bot-bottle.spike=apple-container-networking \
--network bb-spike-230-agent \
--detach docker.io/alpine:latest sleep 600
```
Agent network probes:
```bash
container exec bb-spike-230-agent sh -c '
ip route
cat /etc/resolv.conf
wget -T 5 -O- http://192.168.128.7
wget -T 5 -O- http://bb-spike-230-sidecar || true
ping -c 2 1.1.1.1 || true
wget -T 5 -O- https://example.com || true
'
```
Observed:
```console
default via 192.168.128.1 dev eth0
192.168.128.0/24 dev eth0 scope link src 192.168.128.8
nameserver 192.168.128.1
Connecting to 192.168.128.7 (192.168.128.7:80)
ok
wget: bad address 'bb-spike-230-sidecar'
2 packets transmitted, 0 packets received, 100% packet loss
wget: bad address 'example.com'
```
Host-published loopback aliases work and are constrained to the bound
alias on the host:
```bash
container run --name bb-spike-230-sidecar-alias \
--label bot-bottle.spike=apple-container-networking \
--network bb-spike-230-egress \
--network bb-spike-230-agent \
--dns 1.1.1.1 \
--publish 127.0.0.31:18080:80 \
--detach docker.io/python:alpine \
sh -c 'mkdir -p /srv && printf ok >/srv/index.html && cd /srv && python3 -m http.server 80 --bind 0.0.0.0'
curl -fsS --max-time 5 http://127.0.0.31:18080
curl -fsS --max-time 5 http://127.0.0.1:18080
lsof -nP -iTCP:18080 -sTCP:LISTEN
```
Observed:
```console
$ curl -fsS --max-time 5 http://127.0.0.31:18080
ok
$ curl -fsS --max-time 5 http://127.0.0.1:18080
curl: (7) Failed to connect to 127.0.0.1 port 18080 after 0 ms: Couldn't connect to server
$ lsof -nP -iTCP:18080 -sTCP:LISTEN
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
container 17908 didericis 25u IPv4 ... 0t0 TCP 127.0.0.31:18080 (LISTEN)
```
The guest cannot reach that host loopback-published listener through
the host-only gateway or through its own loopback address:
```bash
container exec bb-spike-230-agent sh -c '
wget -T 5 -O- http://192.168.128.10
wget -T 5 -O- http://192.168.128.1:18080 || true
wget -T 5 -O- http://127.0.0.31:18080 || true
wget -T 5 -O- http://bb-spike-230-sidecar-alias || true
'
```
Observed:
```console
Connecting to 192.168.128.10 (192.168.128.10:80)
ok
Connecting to 192.168.128.1:18080 (192.168.128.1:18080)
wget: can't connect to remote host (192.168.128.1): Connection refused
Connecting to 127.0.0.31:18080 (127.0.0.31:18080)
wget: can't connect to remote host (127.0.0.31): Connection refused
wget: bad address 'bb-spike-230-sidecar-alias'
```
## Answers
### 1. Does `container network create --internal` prevent outbound internet access?
Yes in this run. `--internal` produced a `hostOnly` network. An
internal-only agent had a default route to the host-only gateway, but
could not ping `1.1.1.1` and could not resolve or fetch
`https://example.com`.
### 2. Can `container run` attach one container to multiple networks?
Yes. Repeated `--network` flags produced multiple interfaces and the
inspect JSON preserved both network attachments.
Important caveat: network order matters. The first network became
`eth0`, supplied the default route, and supplied `/etc/resolv.conf`.
For a sidecar that needs internet egress, put the NAT network first and
the internal agent network second.
### 3. Can the sidecar bundle sit on both an internal agent network and an egress-capable network?
Yes. The sidecar had a NAT interface and a host-only interface. With the
NAT network first and explicit DNS, it could fetch `https://example.com`
while the agent on only the host-only network could not.
### 4. Can Apple Container provide stable network aliases or service discovery equivalent to Docker Compose aliases?
Not by default in this run. The agent could not resolve
`bb-spike-230-sidecar` or `bb-spike-230-sidecar-alias`, even though
those were the container names and hostnames in inspect output. The
agent could reach the sidecar by the sidecar's host-only-network IP.
The backend should not assume Docker Compose-style aliases. It should
read the sidecar's host-only IP from `container inspect` and inject
that concrete endpoint into the agent environment/config, or run a
small internal DNS/hosts-file setup as an explicit backend feature.
### 5. Can a published sidecar port bound to a per-bottle loopback alias be reached from another Apple Container guest and constrained to that alias?
Host-side alias binding works and is constrained on the host:
`127.0.0.31:18080` reached the sidecar, while `127.0.0.1:18080` failed.
Guest-to-host-published-loopback did not work. From the agent,
`192.168.128.1:18080` and `127.0.0.31:18080` both failed. For
agent-to-sidecar traffic, use the sidecar's internal network IP rather
than a host-published loopback alias.
### 6. What structured output is available for robust enumeration and cleanup?
Confirmed structured output:
- `container list --all --format json`
- `container inspect <container...>` as JSON
- `container image inspect <image...>` as JSON
- `container network list --format json`
- `container network inspect <network...>` as JSON
- `container system status --format json`
- `container system version --format json`
Useful fields observed:
- containers: `id`, `configuration.labels`,
`configuration.networks`, `configuration.publishedPorts`,
`status.state`, `status.networks[].network`,
`status.networks[].ipv4Address`, `status.networks[].ipv4Gateway`;
- networks: `id`, `configuration.name`, `configuration.labels`,
`configuration.mode`, `status.ipv4Gateway`, `status.ipv4Subnet`;
- images: `id`, `configuration.name`, `configuration.descriptor`,
`variants[].platform`, `variants[].size`.
### 7. Are labels supported on containers and networks enough to replace prefix-only discovery?
Labels are present in container and network inspect/list JSON, so they
are sufficient as metadata if the backend lists resources and filters
client-side. I did not find or validate a server-side label filter for
`container list` or `container network list`.
## Recommendation
Proceed with a narrow `macos-container` launch prototype, but encode
the Apple Container-specific constraints directly:
- create one host-only agent network and one NAT egress network per
bottle;
- start the sidecar bundle with `--network <egress>` before
`--network <agent>`;
- set sidecar DNS explicitly, ideally from the bottle/host policy
rather than hardcoding a public resolver;
- start the agent only on the host-only network;
- discover the sidecar's host-only IP from `container inspect` and pass
concrete URLs to the agent;
- use host loopback publishing only for host-to-sidecar access, not
guest-to-sidecar access;
- enumerate and clean up by labels plus name prefixes until/unless the
CLI adds label filters.
Do not implement the backend as a direct clone of Docker Compose
service aliases. That assumption failed in this run.
@@ -0,0 +1,476 @@
# Apple Container transparent egress spike
Issue: https://gitea.dideric.is/didericis/bot-bottle/issues/230#issuecomment-1994
## Summary
Transparent egress is mechanically possible on Apple Container 1.0.0,
but it is not a free property of the platform and it is not a drop-in
replacement for `HTTP_PROXY` yet.
The spike proved two separate things:
- Plain routing/NAT works if the sidecar has `CAP_NET_ADMIN`, IP
forwarding, and masquerade rules, and if the agent default route is
changed to the sidecar's host-only-network IP.
- Transparent mitmproxy interception works if the sidecar redirects
agent-facing TCP 80/443 traffic to `mitmdump --mode transparent`.
Direct HTTP was logged by mitmproxy. Direct HTTPS reached mitmproxy;
it failed with normal certificate verification until the client
skipped verification, which is consistent with bot-bottle's existing
requirement that agents trust the sidecar CA.
- Running DNS on the sidecar and pointing the agent at the sidecar's
host-only IP also works. This is cleaner than relying on forwarded
UDP DNS to a public resolver and gives the backend a natural place to
enforce or observe DNS policy.
The hard blocker is agent routing. Apple Container 1.0.0 exposes no
documented `--network` gateway option. An ordinary agent container
cannot replace its default route:
```console
$ container exec bb-spike-230t-agent sh -c \
'ip route replace default via 192.168.128.2 dev eth0; ip route'
default via 192.168.128.1 dev eth0
192.168.128.0/24 dev eth0 scope link src 192.168.128.3
ip: RTNETLINK answers: Operation not permitted
```
The successful route-through-sidecar tests used `--cap-add
CAP_NET_ADMIN` on the agent so the route could be changed after start.
That is not an acceptable final design by itself: it expands the
agent's kernel-facing privilege and lets the agent mutate its own
network namespace. A production design needs either a backend-owned
init/shim that sets the route then drops privilege in a way the agent
cannot regain, a platform-supported gateway option, or a different
network attachment layer.
## Environment
Tested on 2026-06-10:
```console
$ sw_vers
ProductName: macOS
ProductVersion: 26.5.1
BuildVersion: 25F80
$ uname -m
arm64
$ container --version
container CLI version 1.0.0 (build: release, commit: ee848e3)
```
Apple Container system status:
```json
{
"apiServerAppName": "container-apiserver",
"apiServerBuild": "release",
"apiServerCommit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
"apiServerVersion": "container-apiserver version 1.0.0 (build: release, commit: ee848e3)",
"appRoot": "/Users/didericis/Library/Application Support/com.apple.container/",
"installRoot": "/usr/local/",
"status": "running"
}
```
## Baseline
Networks:
```bash
container network create bb-spike-230t-agent \
--internal \
--label bot-bottle.spike=transparent-egress
container network create bb-spike-230t-egress \
--label bot-bottle.spike=transparent-egress
```
Sidecar, dual-homed with NAT first:
```bash
container run --name bb-spike-230t-sidecar \
--label bot-bottle.spike=transparent-egress \
--network bb-spike-230t-egress \
--network bb-spike-230t-agent \
--dns 1.1.1.1 \
--detach docker.io/alpine:latest sleep 1800
```
Agent, host-only network:
```bash
container run --name bb-spike-230t-agent \
--label bot-bottle.spike=transparent-egress \
--network bb-spike-230t-agent \
--detach docker.io/alpine:latest sleep 1800
```
Observed sidecar addresses:
```console
eth0 192.168.66.2/24 # NAT egress network
eth1 192.168.128.2/24 # host-only agent network
default via 192.168.66.1 dev eth0
nameserver 1.1.1.1
```
Observed agent baseline:
```console
eth0 192.168.128.3/24
default via 192.168.128.1 dev eth0
nameserver 192.168.128.1
wget: bad address 'pypi.org'
```
That confirms the previous spike's baseline: sidecar can egress, agent
cannot egress directly.
## Plain NAT Test
Relaunch sidecar and agent with `CAP_NET_ADMIN`:
```bash
container run --name bb-spike-230t-sidecar \
--label bot-bottle.spike=transparent-egress \
--network bb-spike-230t-egress \
--network bb-spike-230t-agent \
--dns 1.1.1.1 \
--cap-add CAP_NET_ADMIN \
--detach docker.io/alpine:latest sleep 1800
container run --name bb-spike-230t-agent \
--label bot-bottle.spike=transparent-egress \
--network bb-spike-230t-agent \
--cap-add CAP_NET_ADMIN \
--detach docker.io/alpine:latest sleep 1800
```
Configure sidecar forwarding:
```bash
container exec bb-spike-230t-sidecar sh -c '
apk add --no-cache iptables iproute2
sysctl -w net.ipv4.ip_forward=1
iptables -t nat -A POSTROUTING -s 192.168.128.0/24 -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
'
```
Point the agent at the sidecar:
```bash
container exec bb-spike-230t-agent sh -c '
ip route replace default via 192.168.128.4 dev eth0
printf "nameserver 1.1.1.1\n" > /etc/resolv.conf
'
```
Normal direct PyPI fetch from the agent, with no proxy variables set:
```bash
container exec bb-spike-230t-agent sh -c '
for v in HTTP_PROXY HTTPS_PROXY http_proxy https_proxy ALL_PROXY all_proxy; do
if [ -n "$(printenv "$v")" ]; then echo "$v=SET"; fi
done
wget -T 10 -O- https://pypi.org/simple/pip/ | head -c 120
'
```
Observed:
```console
Connecting to pypi.org (151.101.0.223:443)
<!DOCTYPE html>
<html lang="en">
<head>
<meta name="pypi:repository-version" content="1.4">
```
Sidecar NAT counters increased:
```console
POSTROUTING MASQUERADE 3 packets / 168 bytes
FORWARD eth1 -> eth0 22 packets / 2806 bytes
FORWARD eth0 -> eth1 29 packets / 54781 bytes
```
Verdict: plain transparent routing through the sidecar works, but this
is only NAT. It does not apply bot-bottle's existing route allowlist,
authorization stripping/injection, or DLP logic.
## Transparent Mitmproxy Test
The current sidecar launcher uses explicit proxy mode:
```sh
MODE="--mode regular@9099"
exec mitmdump $CONFDIR_FLAG $MODE $LISTEN_HOST_FLAG $TRUST_FLAG -s /app/egress_addon.py
```
So transparent egress needs a launcher mode change plus iptables
redirects.
Run a test mitmproxy container:
```bash
container run --name bb-spike-230t-mitm \
--label bot-bottle.spike=transparent-egress \
--network bb-spike-230t-egress \
--network bb-spike-230t-agent \
--dns 1.1.1.1 \
--cap-add CAP_NET_ADMIN \
--detach mitmproxy/mitmproxy:11.1.3 \
sh -c 'apt-get update >/tmp/apt.log &&
apt-get install -y --no-install-recommends iptables iproute2 >>/tmp/apt.log &&
echo 1 > /proc/sys/net/ipv4/ip_forward &&
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-port 8080 &&
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j REDIRECT --to-port 8080 &&
mitmdump --mode transparent@8080 --set showhost=true --set ssl_insecure=true --set confdir=/tmp/mitm -v'
```
The container listened successfully:
```console
Transparent Proxy listening at *:8080.
```
It had an agent-facing address of `192.168.128.7`. Point the agent at
it and set DNS:
```bash
container exec bb-spike-230t-agent sh -c '
ip route replace default via 192.168.128.7 dev eth0
printf "nameserver 1.1.1.1\n" > /etc/resolv.conf
'
```
DNS also needs NAT/forwarding because only TCP 80/443 is redirected:
```bash
container exec bb-spike-230t-mitm sh -c '
iptables -t nat -A POSTROUTING -s 192.168.128.0/24 -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
'
```
An alternative, and likely better, DNS shape is to run a DNS forwarder on
the sidecar's host-only IP and point the agent at it. This was tested
with `dnsmasq`:
```bash
container exec bb-spike-230t-mitm sh -c '
apt-get install -y --no-install-recommends dnsmasq
cat >/tmp/dnsmasq.conf <<EOF
no-daemon
listen-address=192.168.128.7
bind-interfaces
server=1.1.1.1
log-queries
log-facility=-
EOF
(dnsmasq --conf-file=/tmp/dnsmasq.conf >/tmp/dnsmasq.log 2>&1 &)
sleep 1
ss -lunp | grep :53
'
```
Observed:
```console
UNCONN 0 0 192.168.128.7:53 0.0.0.0:* users:(("dnsmasq",pid=515,fd=4))
```
Point the agent to sidecar DNS:
```bash
container exec bb-spike-230t-agent sh -c '
printf "nameserver 192.168.128.7\n" > /etc/resolv.conf
nslookup pypi.org
'
```
Observed:
```console
Server: 192.168.128.7
Address: 192.168.128.7:53
Non-authoritative answer:
Name: pypi.org
Address: 151.101.128.223
Name: pypi.org
Address: 151.101.192.223
Name: pypi.org
Address: 151.101.64.223
Name: pypi.org
Address: 151.101.0.223
```
Direct HTTP from the agent worked and mitmproxy logged the request:
```console
$ container exec bb-spike-230t-agent sh -c \
'wget -T 10 -O- http://example.com | head -c 100'
Connecting to example.com (172.66.147.243:80)
<!doctype html><html lang="en"><head><title>Example Domain</title>
```
Mitmproxy log:
```console
192.168.128.5:39742: GET http://example.com/
Host: example.com
User-Agent: Wget
<< 200 OK 559b
```
After switching the agent to sidecar DNS, direct HTTP still hit
mitmproxy:
```console
192.168.128.5:50784: GET http://example.com/
Host: example.com
User-Agent: Wget
<< 200 OK 559b
```
Direct HTTPS from the agent reached mitmproxy but failed certificate
verification, as expected when the client does not trust the mitmproxy
CA:
```console
$ container exec bb-spike-230t-agent sh -c \
'wget -T 10 -O- https://pypi.org/simple/pip/ | head -c 100'
Connecting to pypi.org (151.101.128.223:443)
... certificate verify failed ...
```
Mitmproxy log:
```console
Client TLS handshake failed. The client does not trust the proxy's
certificate for pypi.org (tlsv1 alert unknown ca)
```
With verification disabled, the same direct URL succeeded and mitmproxy
logged the full HTTPS request:
```console
$ container exec bb-spike-230t-agent sh -c \
'wget --no-check-certificate -T 10 -O- https://pypi.org/simple/pip/ | head -c 100'
Connecting to pypi.org (151.101.128.223:443)
<!DOCTYPE html>
<html lang="en">
<head>
<meta name="pypi:repository-version" content="1.4">
```
Mitmproxy log:
```console
192.168.128.5:32802: GET https://pypi.org/simple/pip/
Host: pypi.org
User-Agent: Wget
<< 200 OK 103k
```
After switching the agent to sidecar DNS, direct HTTPS still hit
mitmproxy:
```console
192.168.128.5:50254: GET https://pypi.org/simple/pip/
Host: pypi.org
User-Agent: Wget
<< 200 OK 103k
```
Verdict: transparent mitmproxy mode works in this topology. The bot
agent would still need the egress CA installed, which bot-bottle already
does for explicit proxy mode.
## Answers
### Can the sidecar become the agent network's default gateway?
Not directly through Apple Container's documented CLI. The installed
`container run --help` documents `--network
<name>[,mac=XX:XX:XX:XX:XX:XX][,mtu=VALUE]`; it does not document a
gateway option.
The route can be changed after container start only if the agent has
`CAP_NET_ADMIN`. Without it, `ip route replace default via <sidecar>`
fails with `Operation not permitted`.
### Can Apple Container support sidecar forwarding/NAT/transparent proxying?
Yes. A dual-homed sidecar with `CAP_NET_ADMIN` can enable IP forwarding,
set iptables NAT/forwarding rules, and route agent traffic out through
the NAT network.
Transparent mitmproxy interception also works with `PREROUTING`
redirects to `mitmdump --mode transparent`.
### What capabilities/custom image are required?
At minimum:
- sidecar needs `CAP_NET_ADMIN`;
- sidecar image needs `iptables`/`iproute2` or equivalent nftables
tooling;
- sidecar should run a DNS listener on its host-only IP, or otherwise
provide a controlled resolver path for the agent;
- sidecar launcher needs a transparent mode variant;
- agent route must be changed to the sidecar's host-only IP;
- agent DNS should point to the sidecar DNS listener;
- agent must trust the sidecar CA for HTTPS interception.
The tested agent route mutation required agent `CAP_NET_ADMIN`, which
should not be accepted as the final design without a privilege-dropping
init/shim story.
### Can host-level `pf` or vmnet rules replace agent route mutation?
Not tested. The successful transparent paths did not use host `pf`;
they used container-local routing and iptables. Host-level `pf` remains
a possible escape hatch if Apple Container cannot set a custom gateway
and we reject agent `CAP_NET_ADMIN`.
### Can existing route policy and DLP semantics be preserved?
Likely, but not fully validated in this spike. Mitmproxy transparent
mode produced normal HTTP flows with correct `Host` values for both
HTTP and HTTPS. The existing `egress_addon.py` hooks should still see
`flow.request.pretty_host`, method, path, headers, and response bodies.
But the current sidecar entrypoint only starts `mitmdump` in regular
explicit-proxy mode. A real implementation must add a transparent mode
launcher and then run the existing egress addon test suite against
transparent flows.
## Recommendation
Do not switch `macos-container` to transparent egress yet, but keep it
as a plausible implementation path.
The next implementation spike should focus on removing the agent
`CAP_NET_ADMIN` requirement. Acceptable options:
- find or add an Apple Container-supported default-gateway setting;
- start the agent through a tiny root init that sets route/DNS, drops
capabilities, and then execs the agent as the normal user;
- include a sidecar DNS service and set the agent resolver to the
sidecar's host-only IP as part of that init/setup path;
- avoid routing mutation by using host/vmnet-level packet redirection;
- explicitly decide that route mutation is only a convenience layer and
keep explicit proxy env vars for v1.
Bluntly: transparent egress is feasible, but not production-ready until
the agent route can be controlled without leaving network-admin power in
the agent runtime.
+9 -2
View File
@@ -5,12 +5,19 @@ agent_provider:
egress: egress:
routes: routes:
- host: api.anthropic.com - host: api.anthropic.com
role: claude_code_oauth role: claude_code_oauth # wires Claude Code OAuth; do not change
auth: auth:
scheme: Bearer scheme: Bearer
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
# dlp is omitted → all detectors on by default (token_patterns,
# known_secrets outbound; naive_injection_detection inbound).
# To disable inbound scanning for this route:
# dlp:
# inbound_detectors: false
--- ---
Common Claude provider boundary. Drop this file into Common Claude provider boundary. Drop this file into
`~/.bot-bottle/bottles/claude.md`, then extend it from task-specific `~/.bot-bottle/bottles/claude.md`, then extend it from task-specific
bottles. bottles. The default smolmachines backend keeps DNS resolution under
the VM-layer egress policy; use `BOT_BOTTLE_BACKEND=docker` only for
legacy Docker-backed runs.
+2 -1
View File
@@ -10,4 +10,5 @@ The `dev` bottle — backs a generic development workflow.
Inherits the Claude provider boundary from `claude`. Drop this file Inherits the Claude provider boundary from `claude`. Drop this file
into `~/.bot-bottle/bottles/dev.md` and any agent referencing into `~/.bot-bottle/bottles/dev.md` and any agent referencing
`bottle: dev` will launch against this infrastructure. `bottle: dev` will launch against this infrastructure. By default,
bot-bottle runs this bottle on the smolmachines backend.
+1 -1
View File
@@ -35,5 +35,5 @@ chmod 600 "$fake_key_dir/fake-key"
# Build the image graph quietly so the recorded run shows only the # Build the image graph quietly so the recorded run shows only the
# bottle launch and the four `!` probes, not BuildKit progress. # bottle launch and the four `!` probes, not BuildKit progress.
docker build -q -f Dockerfile.claude -t bot-bottle-claude:latest . >/dev/null 2>&1 || true docker build -q -f bot_bottle/contrib/claude/Dockerfile -t bot-bottle-claude:latest . >/dev/null 2>&1 || true
docker build -q -f Dockerfile.git-gate -t bot-bottle-git-gate:latest . >/dev/null 2>&1 || true docker build -q -f Dockerfile.git-gate -t bot-bottle-git-gate:latest . >/dev/null 2>&1 || true
+2 -2
View File
@@ -46,12 +46,12 @@ def fixture_with_git_dict() -> dict[str, Any]:
"repos": { "repos": {
"bot-bottle": { "bot-bottle": {
"url": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git", "url": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"identity": "/dev/null", "key": {"provider": "static", "path": "/dev/null"},
"host_key": "ssh-ed25519 AAAA...", "host_key": "ssh-ed25519 AAAA...",
}, },
"foo": { "foo": {
"url": "ssh://git@github.com/didericis/foo.git", "url": "ssh://git@github.com/didericis/foo.git",
"identity": "/dev/null", "key": {"provider": "static", "path": "/dev/null"},
"host_key": "ssh-ed25519 BBBB...", "host_key": "ssh-ed25519 BBBB...",
}, },
}, },
-219
View File
@@ -1,219 +0,0 @@
"""Integration: drive `apply_capability_change` against a real
container that mimics the agent's name + filesystem layout (PRD 0016).
The real `cli.py start <agent>` flow is too heavy for an integration
test (it builds the agent image, brings up all the sidecars, attaches
an interactive agent session). Instead, this test stages the
minimum the orchestrator interacts with:
- A lightweight `alpine:latest sleep infinity` container named
`bot-bottle-<slug>` (matches the agent container name pattern)
on the per-bottle internal network.
- A marker file under `/home/node/.claude/` so we can assert the
transcript snapshot path actually transferred bytes.
Then `apply_capability_change` runs and we verify:
- Per-bottle Dockerfile written.
- Containers + networks removed.
- Transcript snapshot dir on the host has the marker file.
docker exec / cp / rm work across the docker socket boundary, so
this test runs in DinD too no act_runner skip needed.
"""
from __future__ import annotations
import os
import subprocess
import tempfile
import time
import unittest
from pathlib import Path
from bot_bottle import supervise
from bot_bottle.backend.docker import bottle_state
from bot_bottle.backend.docker.capability_apply import apply_capability_change
from bot_bottle.backend.docker.network import (
network_create_egress,
network_create_internal,
network_remove,
)
from bot_bottle.backend.docker.sidecar_bundle import (
sidecar_bundle_container_name,
)
from tests._docker import skip_unless_docker
ALPINE_IMAGE = "alpine:latest"
@skip_unless_docker()
class TestCapabilityApply(unittest.TestCase):
@classmethod
def setUpClass(cls):
r = subprocess.run(
["docker", "pull", ALPINE_IMAGE],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
if r.returncode != 0:
raise unittest.SkipTest(f"could not pull {ALPINE_IMAGE}")
def setUp(self):
self.slug = f"cb-test-cap-{os.getpid()}-{int(time.time())}"
self.agent_name = f"bot-bottle-{self.slug}"
self.sidecar_names: list[str] = []
self.internal_net = ""
self.egress_net = ""
# Fake home so tests don't touch ~/.bot-bottle/.
self._tmp = tempfile.TemporaryDirectory(prefix="cap-apply-int.")
self._original_root = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
def tearDown(self):
supervise.bot_bottle_root = self._original_root # type: ignore[assignment]
for name in [self.agent_name, *self.sidecar_names]:
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
for n in (self.internal_net, self.egress_net):
if n:
network_remove(n)
self._tmp.cleanup()
def _bring_up_fake_bottle(self) -> None:
self.internal_net = network_create_internal(self.slug)
self.egress_net = network_create_egress(self.slug)
# Agent container with the canonical name.
r = subprocess.run(
[
"docker", "run", "-d",
"--name", self.agent_name,
"--network", self.internal_net,
ALPINE_IMAGE,
"sh", "-c",
"mkdir -p /home/node/.claude && "
"echo 'transcript-marker' > /home/node/.claude/sessions.json && "
"sleep 3600",
],
capture_output=True, text=True, check=False,
)
self.assertEqual(0, r.returncode, r.stderr)
# Also start a fake sidecar bundle so teardown has something
# extra to clean up (mirrors a real bottle's container set).
sidecar = sidecar_bundle_container_name(self.slug)
subprocess.run(
[
"docker", "run", "-d",
"--name", sidecar,
"--network", self.internal_net,
ALPINE_IMAGE, "sleep", "3600",
],
capture_output=True, text=True, check=False,
)
self.sidecar_names.append(sidecar)
def _containers_named_like(self) -> list[str]:
"""All running/stopped containers whose names start with
the bottle's slug — both agent + sidecars."""
r = subprocess.run(
[
"docker", "ps", "-a",
"--filter", f"name={self.agent_name}",
"--format", "{{.Names}}",
],
capture_output=True, text=True, check=False,
)
return [line for line in (r.stdout or "").splitlines() if line]
def _networks_named_like(self) -> list[str]:
r = subprocess.run(
[
"docker", "network", "ls",
"--filter", f"name={self.slug}",
"--format", "{{.Name}}",
],
capture_output=True, text=True, check=False,
)
return [line for line in (r.stdout or "").splitlines() if line]
def test_apply_writes_dockerfile_and_tears_down(self):
self._bring_up_fake_bottle()
self.assertIn(self.agent_name, self._containers_named_like())
new_dockerfile = "FROM python:3.13\nRUN apk add ripgrep\n"
before, after = apply_capability_change(self.slug, new_dockerfile)
# Before is the repo Dockerfile (no prior per-bottle override);
# after is what we passed in.
self.assertIn("FROM ", before)
self.assertEqual(new_dockerfile, after)
# Per-bottle Dockerfile written on the host.
self.assertEqual(
new_dockerfile,
bottle_state.per_bottle_dockerfile(self.slug),
)
# Agent + sidecars gone.
self.assertEqual([], self._containers_named_like())
# Networks removed (matching the slug substring).
nets = self._networks_named_like()
self.assertEqual([], nets)
# Mark them as already cleaned so tearDown is idempotent.
self.internal_net = ""
self.egress_net = ""
self.sidecar_names = []
def test_transcript_snapshot_captured(self):
self._bring_up_fake_bottle()
apply_capability_change(self.slug, "FROM x\n")
snap = bottle_state.transcript_snapshot_dir(self.slug)
self.assertTrue(snap.is_dir(), f"transcript snapshot dir {snap} missing")
# docker cp <container>:/home/node/.claude <dst> produces
# <dst>/.claude/sessions.json (it preserves the source dir name
# inside the destination if the destination already exists).
# Walk the snapshot looking for the marker contents.
marker_found = False
for path in snap.rglob("sessions.json"):
if "transcript-marker" in path.read_text():
marker_found = True
break
self.assertTrue(marker_found, f"marker not found under {snap}")
# Cleaned up by apply already.
self.internal_net = ""
self.egress_net = ""
self.sidecar_names = []
def test_subsequent_apply_uses_per_bottle_dockerfile_for_before(self):
# First change: before is repo's Dockerfile.
self._bring_up_fake_bottle()
first_before, _ = apply_capability_change(self.slug, "FROM v1\n")
self.assertIn("FROM ", first_before)
# Second change: before is "FROM v1\n" (the per-bottle override
# from the first change), proving the state persists across
# rebuilds.
self._bring_up_fake_bottle()
second_before, second_after = apply_capability_change(self.slug, "FROM v2\n")
self.assertEqual("FROM v1\n", second_before)
self.assertEqual("FROM v2\n", second_after)
self.internal_net = ""
self.egress_net = ""
self.sidecar_names = []
def test_teardown_idempotent_when_nothing_running(self):
# No bottle ever brought up — teardown still doesn't raise.
apply_capability_change(self.slug, "FROM x\n")
self.assertEqual(
"FROM x\n",
bottle_state.per_bottle_dockerfile(self.slug),
)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,239 @@
"""Integration: macOS Container launch topology.
End-to-end against Apple's real `container` runtime. The smoke launches
a bottle with the experimental macOS Container backend and verifies the
properties that make the explicit-proxy launch acceptable:
- the agent can exec commands after provisioning;
- HTTP(S)_PROXY points at the sidecar's internal-network IP;
- allowlisted HTTPS reaches the egress sidecar;
- direct egress with proxy env removed fails from the internal-only
agent network;
- non-allowlisted proxy traffic is blocked.
Skipped under Gitea Actions and on hosts without Apple's `container`.
"""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
import tempfile
import unittest
from pathlib import Path
from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.backend.macos_container.util import (
dns_server as _container_dns_server,
is_available as _container_available,
)
from bot_bottle.manifest import Manifest
_AGENT_PROMPT = "You are a launch smoke-test agent. Be brief."
def _minimal_agent_dockerfile(path: Path) -> None:
path.write_text(
"\n".join((
"FROM node:22-slim",
"RUN apt-get update \\",
" && apt-get install -y --no-install-recommends \\",
" ca-certificates curl git \\",
" && rm -rf /var/lib/apt/lists/*",
"USER node",
"WORKDIR /home/node",
"CMD [\"sleep\", \"infinity\"]",
"",
)),
encoding="utf-8",
)
def _minimal_manifest(dockerfile: Path) -> Manifest:
return Manifest.from_json_obj({
"bottles": {
"dev": {
"agent_provider": {
"template": "pi",
"dockerfile": str(dockerfile),
"settings": {
"provider": "example",
"base_url": "https://example.com/v1",
"models": ["smoke"],
},
},
"egress": {
"routes": [
{"host": "example.com"},
],
},
},
},
"agents": {
"demo": {
"skills": [],
"prompt": _AGENT_PROMPT,
"bottle": "dev",
},
},
})
def _buildkit_dns_available() -> bool:
if platform.system() != "Darwin" or not _container_available():
return False
stage = Path(tempfile.mkdtemp(prefix="cb-container-buildkit-dns."))
image = "bot-bottle-buildkit-dns-check:latest"
try:
dockerfile = stage / "Dockerfile"
dockerfile.write_text(
"FROM debian:bookworm-slim\n"
"RUN getent hosts deb.debian.org\n",
encoding="utf-8",
)
result = subprocess.run(
[
"container", "build",
"--dns", _container_dns_server(),
"-t", image,
"-f", str(dockerfile),
str(stage),
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
return result.returncode == 0
finally:
subprocess.run(
["container", "image", "delete", image],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
shutil.rmtree(stage, ignore_errors=True)
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: cannot host Apple Container VMs",
)
@unittest.skipUnless(
platform.system() == "Darwin",
"Apple Container is macOS-only",
)
@unittest.skipUnless(
_container_available(),
"Apple Container not on PATH; install from "
"https://github.com/apple/container/releases",
)
@unittest.skipUnless(
_buildkit_dns_available(),
"Apple Container BuildKit cannot resolve deb.debian.org on this host",
)
class TestMacosContainerLaunch(unittest.TestCase):
"""Launch once and reuse the bottle across probes."""
@classmethod
def setUpClass(cls) -> None:
cls.stage = Path(tempfile.mkdtemp(prefix="cb-macos-container-launch."))
cls._launch = None
cls.bottle = None
dockerfile = cls.stage / "Dockerfile.agent-smoke"
_minimal_agent_dockerfile(dockerfile)
os.environ["BOT_BOTTLE_BACKEND"] = "macos-container"
try:
backend = get_bottle_backend()
spec = BottleSpec(
manifest=_minimal_manifest(dockerfile),
agent_name="demo",
copy_cwd=False,
user_cwd=str(cls.stage),
)
cls.plan = backend.prepare(spec, stage_dir=cls.stage)
cls._launch = backend.launch(cls.plan)
cls.bottle = cls._launch.__enter__()
except BaseException:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
raise
@classmethod
def tearDownClass(cls) -> None:
try:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
finally:
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
def test_smoke_exec_echo(self):
r = self.bottle.exec( # type: ignore[union-attr]
"echo hello-from-macos-container"
)
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("hello-from-macos-container", r.stdout)
def test_proxy_env_points_at_sidecar_internal_ip(self):
r = self.bottle.exec( # type: ignore[union-attr]
"printf '%s\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\" "
"\"$NO_PROXY\" \"$NODE_EXTRA_CA_CERTS\""
)
self.assertEqual(0, r.returncode, msg=r.stderr)
values = [line.strip() for line in r.stdout.splitlines()]
self.assertEqual(4, len(values), values)
self.assertEqual(values[0], values[1], values)
self.assertRegex(values[0], r"^http://[0-9.]+:9099$")
self.assertNotIn("127.0.0.1", values[0])
sidecar_host = values[0].removeprefix("http://").removesuffix(":9099")
self.assertIn(sidecar_host, values[2])
self.assertEqual(
"/usr/local/share/ca-certificates/bot-bottle-mitm-ca.crt",
values[3],
)
def test_allowlisted_https_reaches_egress_proxy(self):
r = self.bottle.exec( # type: ignore[union-attr]
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
)
self.assertEqual(0, r.returncode, msg=r.stderr + r.stdout)
self.assertIn("OK", r.stdout)
def test_direct_egress_bypass_without_proxy_fails(self):
r = self.bottle.exec( # type: ignore[union-attr]
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
"curl -s --show-error --max-time 5 https://example.com 2>&1 || true"
)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower()
or "could not resolve" in r.stdout.lower()
or "connection reset" in r.stdout.lower(),
f"expected direct egress to fail; got: {r.stdout!r}",
)
def test_non_allowlisted_host_fails_through_proxy(self):
r = self.bottle.exec( # type: ignore[union-attr]
"curl -s --show-error --max-time 10 https://iana.org 2>&1 || true"
)
self.assertTrue(
"403" in r.stdout
or "502" in r.stdout
or "blocked" in r.stdout.lower()
or "not allowed" in r.stdout.lower()
or "not in the bottle's egress.routes allowlist" in r.stdout.lower()
or "forbidden" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected non-allowlisted proxy request to fail; got: {r.stdout!r}",
)
if __name__ == "__main__":
unittest.main()
+5 -4
View File
@@ -11,8 +11,9 @@ asserts each one is blocked:
5. Secret exfil via README link pushed through git-gate 5. Secret exfil via README link pushed through git-gate
The suite is backend-agnostic it goes through `get_bottle_backend()` The suite is backend-agnostic it goes through `get_bottle_backend()`
so a future smolmachines backend can be tested by setting so smolmachines can be tested by setting `BOT_BOTTLE_BACKEND=smolmachines`.
`BOT_BOTTLE_BACKEND=smolmachines` without touching this file. When unset, this integration test pins Docker explicitly to preserve
the Docker-backed CI path.
PRD 0022 chunk 1 (this commit): fixture + setUpClass + PRD 0022 chunk 1 (this commit): fixture + setUpClass +
tearDownClass + preflight tool check. Attack tests land in tearDownClass + preflight tool check. Attack tests land in
@@ -29,7 +30,7 @@ import unittest
from pathlib import Path from pathlib import Path
from bot_bottle.backend import BottleSpec, get_bottle_backend from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.backend.docker.bottle_state import cleanup_state from bot_bottle.bottle_state import cleanup_state
from bot_bottle.manifest import Manifest from bot_bottle.manifest import Manifest
from tests._docker import skip_unless_docker from tests._docker import skip_unless_docker
@@ -146,7 +147,7 @@ class TestSandboxEscape(unittest.TestCase):
cls._stage_dir = Path(tempfile.mkdtemp(prefix="sandbox-escape-stage.")) cls._stage_dir = Path(tempfile.mkdtemp(prefix="sandbox-escape-stage."))
try: try:
backend = get_bottle_backend() backend = get_bottle_backend(backend_name)
plan = backend.prepare(spec, stage_dir=cls._stage_dir) plan = backend.prepare(spec, stage_dir=cls._stage_dir)
cls._identity = plan.slug cls._identity = plan.slug
@@ -56,7 +56,7 @@ class TestSidecarBundleCompose(unittest.TestCase):
stage_dir = Path(tempfile.mkdtemp(prefix="cb-bundle-smoke.")) stage_dir = Path(tempfile.mkdtemp(prefix="cb-bundle-smoke."))
try: try:
with patch.dict(os.environ, {"BOT_BOTTLE_SIDECAR_BUNDLE": "1"}): with patch.dict(os.environ, {"BOT_BOTTLE_SIDECAR_BUNDLE": "1"}):
backend = get_bottle_backend() backend = get_bottle_backend("docker")
spec = BottleSpec( spec = BottleSpec(
manifest=_manifest(), manifest=_manifest(),
agent_name="demo", agent_name="demo",
+63 -8
View File
@@ -2,16 +2,17 @@
round trip + the acceptance probes. round trip + the acceptance probes.
The smoke confirms the launch flow (per-bottle docker bridge The smoke confirms the launch flow (per-bottle docker bridge
sidecar bundle with pinned IP smolvm guest with TSI allowlist sidecar bundle with host-loopback published ports smolvm guest
exec) plumbs together end to end. The two probes confirm the with TSI allowlist exec) plumbs together end to end. The probes confirm the
security properties the design pivot was about: security properties the design pivot was about:
- **localhost-reach probe** guest tries to dial a service - **localhost-reach probe** guest tries to dial a service
bound on the host's `127.0.0.1`. TSI's `<bundle-ip>/32` bound on the host's `127.0.0.1`. TSI's per-bottle loopback
allowlist must refuse the connect. (PRD 0023's first draft alias allowlist must refuse the connect.
worried about `--outbound-localhost-only` opening the whole
`127.0.0.0/8`; with `--allow-cidr <bundle-ip>/32` instead, - **egress proxy probe** guest reaches the egress proxy through
the gap closes.) the injected `HTTPS_PROXY`/`HTTP_PROXY` URL on the per-bottle
loopback alias, while direct egress with proxy vars unset fails.
- **egress-port-bypass probe** guest tries to dial - **egress-port-bypass probe** guest tries to dial
`<bundle-ip>:9099` (egress's port). TSI permits the IP but `<bundle-ip>:9099` (egress's port). TSI permits the IP but
@@ -43,7 +44,15 @@ _AGENT_PROMPT = "You are demo. Be brief."
def _minimal_manifest() -> Manifest: def _minimal_manifest() -> Manifest:
return Manifest.from_json_obj({ return Manifest.from_json_obj({
"bottles": {"dev": {}}, "bottles": {
"dev": {
"egress": {
"routes": [
{"host": "example.com"},
],
},
},
},
"agents": { "agents": {
"demo": { "demo": {
"skills": [], "skills": [],
@@ -124,6 +133,52 @@ class TestSmolmachinesLaunch(unittest.TestCase):
f"expected a connect-refusal message; got: {r.stdout!r}", f"expected a connect-refusal message; got: {r.stdout!r}",
) )
def test_egress_proxy_reachable_through_tsi_loopback_alias(self):
r = self.bottle.exec(
"printf '%s\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\""
)
self.assertEqual(0, r.returncode, msg=r.stderr)
proxies = [line.strip() for line in r.stdout.splitlines()]
self.assertEqual(2, len(proxies), proxies)
self.assertEqual(proxies[0], proxies[1], proxies)
self.assertTrue(proxies[0].startswith("http://127."), proxies[0])
r = self.bottle.exec(
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
)
self.assertEqual(0, r.returncode, msg=r.stderr + r.stdout)
self.assertIn("OK", r.stdout)
def test_direct_egress_bypass_without_proxy_fails(self):
r = self.bottle.exec(
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
"curl -s --show-error --max-time 5 https://example.com 2>&1 || true"
)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower()
or "could not resolve" in r.stdout.lower()
or "connection reset" in r.stdout.lower(),
f"expected direct egress to fail; got: {r.stdout!r}",
)
def test_non_allowlisted_host_fails_through_proxy(self):
r = self.bottle.exec(
"curl -s --show-error --max-time 10 https://iana.org 2>&1 || true"
)
self.assertTrue(
"403" in r.stdout
or "502" in r.stdout
or "blocked" in r.stdout.lower()
or "not allowed" in r.stdout.lower()
or "not in the bottle's egress.routes allowlist" in r.stdout.lower()
or "forbidden" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected non-allowlisted proxy request to fail; got: {r.stdout!r}",
)
def test_prompt_file_lands_in_guest(self): def test_prompt_file_lands_in_guest(self):
# provision_prompt copies the host-side prompt.txt into the # provision_prompt copies the host-side prompt.txt into the
# guest at /home/node/.bot-bottle-prompt.txt. The content # guest at /home/node/.bot-bottle-prompt.txt. The content
+212 -21
View File
@@ -10,7 +10,8 @@ from pathlib import Path
from bot_bottle.agent_provider import ( from bot_bottle.agent_provider import (
CODEX_HOST_CREDENTIAL_HOSTS, CODEX_HOST_CREDENTIAL_HOSTS,
agent_provision_plan, build_agent_provision_plan,
prompt_args,
) )
from bot_bottle.egress import CODEX_HOST_CREDENTIAL_TOKEN_REF from bot_bottle.egress import CODEX_HOST_CREDENTIAL_TOKEN_REF
@@ -25,11 +26,12 @@ def _jwt(exp: int) -> str:
class TestAgentProviderRuntime(unittest.TestCase): class TestAgentProviderRuntime(unittest.TestCase):
def test_codex_plan_declares_home_state(self): def test_codex_plan_declares_home_state(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="/tmp/Dockerfile.codex", dockerfile="/tmp/Dockerfile.codex",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
) )
config = Path(tmp, "codex-config.toml").read_text() config = Path(tmp, "codex-config.toml").read_text()
self.assertEqual("codex", plan.template) self.assertEqual("codex", plan.template)
@@ -50,16 +52,38 @@ class TestAgentProviderRuntime(unittest.TestCase):
def test_codex_trusts_requested_project_path(self): def test_codex_trusts_requested_project_path(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
agent_provision_plan( build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
trusted_project_path="/home/node/workspace", trusted_project_path="/home/node/workspace",
) )
config = Path(tmp, "codex-config.toml").read_text() config = Path(tmp, "codex-config.toml").read_text()
self.assertIn('[projects."/home/node/workspace"]', config) self.assertIn('[projects."/home/node/workspace"]', config)
def test_codex_writes_tui_settings_without_mutating_prompt(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
prompt_file = Path(tmp) / "prompt.txt"
prompt_file.write_text("Existing instructions.\n")
plan = build_agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=prompt_file,
label="review-api",
color="bright-cyan",
)
prompt = prompt_file.read_text()
config = Path(tmp, "codex-config.toml").read_text()
self.assertTrue(plan.has_prompt)
self.assertEqual("Existing instructions.\n", prompt)
self.assertIn("[tui]", config)
self.assertIn('status_line = ["model-with-reasoning"]', config)
self.assertIn('terminal_title = ["spinner", "project"]', config)
def test_codex_forward_host_credentials_adds_auth_and_verify(self): def test_codex_forward_host_credentials_adds_auth_and_verify(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
home = Path(tmp) / "host-codex" home = Path(tmp) / "host-codex"
@@ -68,11 +92,12 @@ class TestAgentProviderRuntime(unittest.TestCase):
"auth_mode": "chatgpt", "auth_mode": "chatgpt",
"tokens": {"access_token": _jwt(2000000000)}, "tokens": {"access_token": _jwt(2000000000)},
})) }))
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
guest_env={"CODEX_HOME": "/run/codex-home"}, guest_env={"CODEX_HOME": "/run/codex-home"},
forward_host_credentials=True, forward_host_credentials=True,
host_env={"CODEX_HOME": str(home)}, host_env={"CODEX_HOME": str(home)},
@@ -88,11 +113,12 @@ class TestAgentProviderRuntime(unittest.TestCase):
def test_claude_with_auth_token_injects_provider_route_and_placeholder(self): def test_claude_with_auth_token_injects_provider_route_and_placeholder(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="claude", template="claude",
dockerfile="/tmp/Dockerfile.claude", dockerfile="/tmp/Dockerfile.claude",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
auth_token="BOT_BOTTLE_CLAUDE_OAUTH_TOKEN", auth_token="BOT_BOTTLE_CLAUDE_OAUTH_TOKEN",
) )
claude_config = json.loads(Path(tmp, "claude.json").read_text()) claude_config = json.loads(Path(tmp, "claude.json").read_text())
@@ -110,17 +136,38 @@ class TestAgentProviderRuntime(unittest.TestCase):
def test_claude_trusts_requested_project_path(self): def test_claude_trusts_requested_project_path(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
agent_provision_plan( build_agent_provision_plan(
guest_home="/home/node",
template="claude", template="claude",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
trusted_project_path="/home/node/workspace", trusted_project_path="/home/node/workspace",
) )
config = json.loads(Path(tmp, "claude.json").read_text()) config = json.loads(Path(tmp, "claude.json").read_text())
self.assertIn("/home/node", config["projects"]) self.assertIn("/home/node", config["projects"])
self.assertIn("/home/node/workspace", config["projects"]) self.assertIn("/home/node/workspace", config["projects"])
def test_claude_writes_statusline_and_theme_without_mutating_prompt(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
prompt_file = Path(tmp) / "prompt.txt"
prompt_file.write_text("Existing instructions.\n")
plan = build_agent_provision_plan(
template="claude",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=prompt_file,
label="research-ui",
color="green",
)
prompt = prompt_file.read_text()
settings = json.loads(Path(tmp, "claude-settings.json").read_text())
self.assertTrue(plan.has_prompt)
self.assertEqual("Existing instructions.\n", prompt)
self.assertEqual("~/.claude/statusline.sh", settings["statusLine"]["command"])
self.assertEqual("custom:bot-bottle-research-ui", settings["theme"])
def test_codex_forward_host_credentials_populates_egress_routes(self): def test_codex_forward_host_credentials_populates_egress_routes(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
home = Path(tmp) / "host-codex" home = Path(tmp) / "host-codex"
@@ -129,11 +176,12 @@ class TestAgentProviderRuntime(unittest.TestCase):
"auth_mode": "chatgpt", "auth_mode": "chatgpt",
"tokens": {"access_token": _jwt(2000000000)}, "tokens": {"access_token": _jwt(2000000000)},
})) }))
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
forward_host_credentials=True, forward_host_credentials=True,
host_env={"CODEX_HOME": str(home)}, host_env={"CODEX_HOME": str(home)},
) )
@@ -145,11 +193,12 @@ class TestAgentProviderRuntime(unittest.TestCase):
def test_codex_without_forward_host_credentials_has_passthrough_egress_routes(self): def test_codex_without_forward_host_credentials_has_passthrough_egress_routes(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
forward_host_credentials=False, forward_host_credentials=False,
) )
self.assertEqual( self.assertEqual(
@@ -162,11 +211,12 @@ class TestAgentProviderRuntime(unittest.TestCase):
def test_claude_without_auth_token_has_passthrough_egress_route(self): def test_claude_without_auth_token_has_passthrough_egress_route(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="claude", template="claude",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
) )
self.assertEqual(1, len(plan.egress_routes)) self.assertEqual(1, len(plan.egress_routes))
route = plan.egress_routes[0] route = plan.egress_routes[0]
@@ -185,11 +235,12 @@ class TestAgentProviderRuntime(unittest.TestCase):
"auth_mode": "chatgpt", "auth_mode": "chatgpt",
"tokens": {"access_token": access}, "tokens": {"access_token": access},
})) }))
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
forward_host_credentials=True, forward_host_credentials=True,
host_env={"CODEX_HOME": str(home)}, host_env={"CODEX_HOME": str(home)},
) )
@@ -200,15 +251,155 @@ class TestAgentProviderRuntime(unittest.TestCase):
def test_codex_without_forward_host_credentials_has_empty_provisioned_env(self): def test_codex_without_forward_host_credentials_has_empty_provisioned_env(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp: with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan( plan = build_agent_provision_plan(
guest_home="/home/node",
template="codex", template="codex",
dockerfile="", dockerfile="",
state_dir=Path(tmp), state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
forward_host_credentials=False, forward_host_credentials=False,
) )
self.assertEqual({}, plan.provisioned_env) self.assertEqual({}, plan.provisioned_env)
def test_pi_plan_writes_default_ollama_models(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = build_agent_provision_plan(
template="pi",
dockerfile="/tmp/Dockerfile.pi",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
)
models = json.loads(Path(tmp, "pi-models.json").read_text())
self.assertEqual("pi", plan.template)
self.assertEqual("pi", plan.command)
self.assertEqual("append_system_prompt", plan.prompt_mode)
self.assertEqual("/tmp/Dockerfile.pi", plan.dockerfile)
self.assertEqual("bot-bottle-pi:latest", plan.image)
self.assertEqual(
("/home/node/.pi/agent",),
tuple(d.guest_path for d in plan.dirs),
)
self.assertEqual(
("/home/node/.pi/agent/models.json",),
tuple(f.guest_path for f in plan.files),
)
self.assertEqual(("--models", "ollama/qwen2.5-coder:7b"), plan.startup_args)
provider = models["providers"]["ollama"]
self.assertEqual("http://ollama:11434/v1", provider["baseUrl"])
self.assertEqual("openai-completions", provider["api"])
self.assertEqual("ollama", provider["apiKey"])
self.assertEqual("max_tokens", provider["compat"]["maxTokensField"])
self.assertEqual(
[{
"id": "qwen2.5-coder:7b",
"name": "qwen2.5-coder:7b",
"contextWindow": 3072,
"maxTokens": 1024,
}],
provider["models"],
)
self.assertEqual("ollama", plan.egress_routes[0].host)
self.assertEqual("", plan.egress_routes[0].auth_scheme)
self.assertEqual("", plan.egress_routes[0].token_ref)
def test_pi_plan_uses_provider_settings(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
build_agent_provision_plan(
template="pi",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
provider_settings={
"base_url": "http://host.docker.internal:11434/v1",
"api": "openai-responses",
"api_key": "local",
"models": ["gpt-oss:20b", "qwen3:14b"],
"context_window": 65536,
"max_tokens_field": "max_completion_tokens",
"max_tokens": 12000,
"supports_developer_role": True,
"supports_reasoning_effort": True,
},
)
models = json.loads(Path(tmp, "pi-models.json").read_text())
provider = models["providers"]["ollama"]
self.assertEqual("http://host.docker.internal:11434/v1", provider["baseUrl"])
self.assertEqual("openai-responses", provider["api"])
self.assertEqual("local", provider["apiKey"])
self.assertEqual(
[
{
"id": "gpt-oss:20b",
"name": "gpt-oss:20b",
"contextWindow": 53536,
"maxTokens": 12000,
},
{
"id": "qwen3:14b",
"name": "qwen3:14b",
"contextWindow": 53536,
"maxTokens": 12000,
},
],
provider["models"],
)
self.assertTrue(provider["compat"]["supportsDeveloperRole"])
self.assertTrue(provider["compat"]["supportsReasoningEffort"])
self.assertEqual(
"max_completion_tokens",
provider["compat"]["maxTokensField"],
)
def test_pi_plan_can_target_openrouter_with_egress_injected_api_key(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = build_agent_provision_plan(
template="pi",
dockerfile="",
state_dir=Path(tmp),
instance_name="bot-bottle-test",
prompt_file=Path(tmp) / "prompt.txt",
provider_settings={
"provider": "openrouter",
"base_url": "https://openrouter.ai/api/v1",
"api": "openai-completions",
"api_key_env": "OPENROUTER_API_KEY",
"models": ["google/gemma-4-26b-a4b-it:free"],
"supports_reasoning_effort": True,
},
)
models = json.loads(Path(tmp, "pi-models.json").read_text())
provider = models["providers"]["openrouter"]
self.assertEqual("https://openrouter.ai/api/v1", provider["baseUrl"])
self.assertEqual("openai-completions", provider["api"])
self.assertEqual("egress-placeholder", provider["apiKey"])
self.assertEqual("max_tokens", provider["compat"]["maxTokensField"])
self.assertEqual(
[{
"id": "google/gemma-4-26b-a4b-it:free",
"name": "google/gemma-4-26b-a4b-it:free",
"contextWindow": 3072,
"maxTokens": 1024,
}],
provider["models"],
)
self.assertEqual(
("--models", "openrouter/google/gemma-4-26b-a4b-it:free"),
plan.startup_args,
)
self.assertEqual("openrouter.ai", plan.egress_routes[0].host)
self.assertEqual("Bearer", plan.egress_routes[0].auth_scheme)
self.assertEqual("OPENROUTER_API_KEY", plan.egress_routes[0].token_ref)
self.assertNotIn("OPENROUTER_API_KEY", plan.guest_env)
self.assertTrue(provider["compat"]["supportsReasoningEffort"])
def test_pi_prompt_mode_appends_system_prompt_interactively(self):
self.assertEqual(
["--append-system-prompt", "/home/node/.bot-bottle-prompt.txt"],
prompt_args("append_system_prompt", "/home/node/.bot-bottle-prompt.txt"),
)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+119
View File
@@ -0,0 +1,119 @@
"""Unit: shared backend prepare wiring.
These tests keep the base `BottleBackend.prepare` template honest:
backend-specific preflight/env hooks must be wired through, and launch
metadata must record the backend that actually prepared the plan.
"""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle import bottle_state
from bot_bottle import supervise
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.smolmachines import SmolmachinesBottleBackend
from bot_bottle.manifest import Manifest
def _manifest() -> Manifest:
return Manifest.from_json_obj({
"bottles": {
"dev": {
"env": {
"LITERAL_ENV": "literal-value",
"FORWARDED_ENV": "${HOST_SECRET_ENV}",
},
},
},
"agents": {
"demo": {
"bottle": "dev",
"skills": [],
"prompt": "hello",
},
},
})
def _spec(tmp: Path, *, identity: str) -> BottleSpec:
return BottleSpec(
manifest=_manifest(),
agent_name="demo",
copy_cwd=False,
user_cwd=str(tmp),
identity=identity,
)
class _FakeStateMixin:
def setUp(self) -> None:
self.tmp = tempfile.TemporaryDirectory(prefix="backend-prepare.")
self.root = Path(self.tmp.name) / ".bot-bottle"
self.original_root = supervise.bot_bottle_root
supervise.bot_bottle_root = lambda: self.root # type: ignore[assignment]
def tearDown(self) -> None:
supervise.bot_bottle_root = self.original_root # type: ignore[assignment]
self.tmp.cleanup()
class TestDockerPrepare(_FakeStateMixin, unittest.TestCase):
def test_records_backend_and_preserves_env_split(self) -> None:
backend = DockerBottleBackend()
spec = _spec(Path(self.tmp.name), identity="demo-docker")
with (
patch.dict("os.environ", {"HOST_SECRET_ENV": "secret-value"}),
patch(
"bot_bottle.backend.docker.resolve_plan.docker_mod.require_docker",
) as require_docker,
patch(
"bot_bottle.backend.docker.resolve_plan.docker_mod.runsc_available",
return_value=False,
),
):
plan = backend.prepare(spec, Path(self.tmp.name) / "stage")
require_docker.assert_called_once_with()
metadata = bottle_state.read_metadata("demo-docker")
self.assertIsNotNone(metadata)
assert metadata is not None
self.assertEqual("docker", metadata.backend)
self.assertEqual({"FORWARDED_ENV": "secret-value"}, plan.forwarded_env)
self.assertEqual("literal-value", plan.agent_provision.guest_env["LITERAL_ENV"])
self.assertNotIn("FORWARDED_ENV", plan.agent_provision.guest_env)
class TestSmolmachinesPrepare(_FakeStateMixin, unittest.TestCase):
def test_records_backend_and_builds_guest_env(self) -> None:
backend = SmolmachinesBottleBackend()
spec = _spec(Path(self.tmp.name), identity="demo-smol")
with (
patch.dict("os.environ", {"HOST_SECRET_ENV": "secret-value"}),
patch(
"bot_bottle.backend.smolmachines.resolve_plan.smolmachines_preflight",
) as preflight,
):
plan = backend.prepare(spec, Path(self.tmp.name) / "stage")
preflight.assert_called_once_with()
metadata = bottle_state.read_metadata("demo-smol")
self.assertIsNotNone(metadata)
assert metadata is not None
self.assertEqual("smolmachines", metadata.backend)
self.assertEqual("literal-value", plan.guest_env["LITERAL_ENV"])
self.assertEqual("secret-value", plan.guest_env["FORWARDED_ENV"])
self.assertEqual(
"/etc/ssl/certs/ca-certificates.crt",
plan.guest_env["SSL_CERT_FILE"],
)
if __name__ == "__main__":
unittest.main()
+35 -5
View File
@@ -32,10 +32,37 @@ class TestGetBottleBackend(unittest.TestCase):
b = get_bottle_backend() b = get_bottle_backend()
self.assertEqual("smolmachines", b.name) self.assertEqual("smolmachines", b.name)
def test_default_docker(self): def test_default_macos_container_when_available(self):
with patch.dict(os.environ, {}, clear=True): class _FakeBackend:
name = "macos-container"
def is_available(self) -> bool:
return True
with patch.dict(os.environ, {}, clear=True), \
patch.object(backend_mod, "_BACKENDS", {
"macos-container": _FakeBackend(),
"smolmachines": _FakeBackend(),
}):
b = get_bottle_backend() b = get_bottle_backend()
self.assertEqual("docker", b.name) self.assertEqual("macos-container", b.name)
def test_default_smolmachines_when_macos_container_unavailable(self):
class _FakeBackend:
def __init__(self, name: str, available: bool) -> None:
self.name = name
self._available = available
def is_available(self) -> bool:
return self._available
with patch.dict(os.environ, {}, clear=True), \
patch.object(backend_mod, "_BACKENDS", {
"macos-container": _FakeBackend("macos-container", False),
"smolmachines": _FakeBackend("smolmachines", False),
}):
b = get_bottle_backend()
self.assertEqual("smolmachines", b.name)
def test_unknown_dies(self): def test_unknown_dies(self):
with patch.object(backend_mod, "die", side_effect=SystemExit("die")): with patch.object(backend_mod, "die", side_effect=SystemExit("die")):
@@ -44,8 +71,11 @@ class TestGetBottleBackend(unittest.TestCase):
class TestKnownBackendNames(unittest.TestCase): class TestKnownBackendNames(unittest.TestCase):
def test_returns_both_backends_sorted(self): def test_returns_backends_sorted(self):
self.assertEqual(("docker", "smolmachines"), known_backend_names()) self.assertEqual(
("docker", "macos-container", "smolmachines"),
known_backend_names(),
)
class TestEnumerateActiveAgents(unittest.TestCase): class TestEnumerateActiveAgents(unittest.TestCase):
+83
View File
@@ -0,0 +1,83 @@
"""Unit tests for backend/terminal.py palette and shell-script helpers."""
from __future__ import annotations
import unittest
from bot_bottle.backend.terminal import exec_shell_script, palette_printf
class TestPalettePrintf(unittest.TestCase):
def test_known_color_returns_printf(self):
cmd = palette_printf("red")
self.assertTrue(cmd.startswith("printf '"))
self.assertIn("\\033]4;1;", cmd) # normal red
self.assertIn("\\033]4;9;", cmd) # bright red
self.assertIn("\\033]11;", cmd) # default background tint
def test_bright_variant_sets_both_slots(self):
cmd = palette_printf("bright-blue")
self.assertIn("\\033]4;12;", cmd) # bright-blue
self.assertIn("\\033]4;4;", cmd) # blue
def test_unknown_color_returns_empty(self):
self.assertEqual("", palette_printf(""))
self.assertEqual("", palette_printf("neon-pink"))
def test_all_named_colors_produce_output(self):
colors = [
"black", "red", "green", "yellow",
"blue", "magenta", "cyan", "white",
"bright-black", "bright-red", "bright-green", "bright-yellow",
"bright-blue", "bright-magenta", "bright-cyan", "bright-white",
]
for color in colors:
with self.subTest(color=color):
self.assertTrue(palette_printf(color))
class TestExecShellScript(unittest.TestCase):
_ARGV = ["smolvm", "machine", "exec", "--name", "x", "--", "claude"]
def test_no_decoration_returns_none(self):
self.assertIsNone(exec_shell_script(self._ARGV))
self.assertIsNone(exec_shell_script(self._ARGV, terminal_title="", terminal_color=""))
def test_title_only_uses_exec(self):
script = exec_shell_script(self._ARGV, terminal_title="my-agent")
assert script is not None
self.assertIn("printf", script)
self.assertIn("my-agent", script)
self.assertIn("exec ", script)
# No palette reset when there's no color
self.assertNotIn("\\033]104", script)
def test_color_only_sets_palette_and_resets(self):
script = exec_shell_script(self._ARGV, terminal_color="green")
assert script is not None
self.assertIn("\\033]4;", script) # indexed palette
self.assertIn("\\033]11;", script) # background tint
self.assertIn("\\033]104", script) # palette reset
self.assertIn("\\033]111", script) # background reset
# No exec-replace when palette is active (shell must survive for reset)
parts = script.split("; ")
agent_part = next(p for p in parts if "smolvm" in p)
self.assertFalse(agent_part.startswith("exec "))
def test_title_and_color_both_appear(self):
script = exec_shell_script(self._ARGV, terminal_title="bot", terminal_color="cyan")
assert script is not None
self.assertIn("bot", script)
self.assertIn("\\033]4;", script)
self.assertIn("\\033]11;", script)
self.assertIn("\\033]104", script)
self.assertIn("\\033]111", script)
def test_title_with_special_chars_is_quoted(self):
script = exec_shell_script(self._ARGV, terminal_title="my agent's label")
assert script is not None
self.assertNotIn("my agent's label", script) # must be shell-quoted
if __name__ == "__main__":
unittest.main()
+157
View File
@@ -0,0 +1,157 @@
"""Unit: runtime workspace provisioning.
Workspace copy is intentionally handled through
`BottleBackend.provision_workspace` against a running bottle. The
Docker derived-image workspace path stays disabled.
"""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest.mock import MagicMock, call, patch
from bot_bottle import bottle_state
from bot_bottle import supervise
from bot_bottle.backend import Bottle, BottleSpec, ExecResult
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.smolmachines import SmolmachinesBottleBackend
from bot_bottle.manifest import Manifest
def _manifest() -> Manifest:
return Manifest.from_json_obj({
"bottles": {"dev": {}},
"agents": {
"demo": {
"bottle": "dev",
"skills": [],
"prompt": "",
},
},
})
def _spec(tmp: Path, *, copy_cwd: bool = True, identity: str = "demo-work") -> BottleSpec:
return BottleSpec(
manifest=_manifest(),
agent_name="demo",
copy_cwd=copy_cwd,
user_cwd=str(tmp),
identity=identity,
)
def _bottle() -> MagicMock:
bottle = MagicMock(spec=Bottle)
bottle.name = "bot-bottle-demo-work"
bottle.exec.return_value = ExecResult(returncode=0, stdout="", stderr="")
return bottle
class _FakeStateMixin:
def setUp(self) -> None:
self.tmp_dir = tempfile.TemporaryDirectory(prefix="backend-workspace.")
self.tmp = Path(self.tmp_dir.name)
self.root = self.tmp / ".bot-bottle"
self.original_root = supervise.bot_bottle_root
supervise.bot_bottle_root = lambda: self.root # type: ignore[assignment]
def tearDown(self) -> None:
supervise.bot_bottle_root = self.original_root # type: ignore[assignment]
self.tmp_dir.cleanup()
class TestRuntimeWorkspaceProvisioning(_FakeStateMixin, unittest.TestCase):
def test_default_backend_method_copies_workspace_to_running_bottle(self) -> None:
(self.tmp / "src.txt").write_text("hello\n")
(self.tmp / ".git").mkdir()
backend = DockerBottleBackend()
with (
patch("bot_bottle.backend.docker.resolve_plan.docker_mod.require_docker"),
patch(
"bot_bottle.backend.docker.resolve_plan.docker_mod.runsc_available",
return_value=False,
),
):
plan = backend.prepare(_spec(self.tmp), self.tmp / "stage")
bottle = _bottle()
backend.provision_workspace(plan, bottle)
self.assertEqual(
[
call(
"rm -rf /home/node/workspace && mkdir -p /home/node",
user="root",
),
call(
"chown -R node:node /home/node/workspace && "
"chmod 755 /home/node/workspace",
user="root",
),
],
bottle.exec.call_args_list,
)
bottle.cp_in.assert_called_once_with(str(self.tmp), "/home/node/workspace")
def test_default_backend_method_noops_without_copy_cwd(self) -> None:
backend = DockerBottleBackend()
with (
patch("bot_bottle.backend.docker.resolve_plan.docker_mod.require_docker"),
patch(
"bot_bottle.backend.docker.resolve_plan.docker_mod.runsc_available",
return_value=False,
),
):
plan = backend.prepare(_spec(self.tmp, copy_cwd=False), self.tmp / "stage")
bottle = _bottle()
backend.provision_workspace(plan, bottle)
bottle.exec.assert_not_called()
bottle.cp_in.assert_not_called()
def test_smolmachines_uses_same_running_bottle_method(self) -> None:
backend = SmolmachinesBottleBackend()
with patch(
"bot_bottle.backend.smolmachines.resolve_plan.smolmachines_preflight",
):
plan = backend.prepare(
_spec(self.tmp, identity="demo-smol-work"),
self.tmp / "stage",
)
bottle = _bottle()
backend.provision_workspace(plan, bottle)
bottle.cp_in.assert_called_once_with(str(self.tmp), "/home/node/workspace")
metadata = bottle_state.read_metadata("demo-smol-work")
self.assertIsNotNone(metadata)
assert metadata is not None
self.assertEqual("smolmachines", metadata.backend)
class TestWorkspaceTrustPath(_FakeStateMixin, unittest.TestCase):
def test_prepare_trusts_workspace_path_when_copying_cwd(self) -> None:
backend = DockerBottleBackend()
with (
patch("bot_bottle.backend.docker.resolve_plan.docker_mod.require_docker"),
patch(
"bot_bottle.backend.docker.resolve_plan.docker_mod.runsc_available",
return_value=False,
),
):
plan = backend.prepare(_spec(self.tmp), self.tmp / "stage")
claude_config = self.root / "state" / "demo-work" / "agent" / "claude.json"
config = claude_config.read_text()
self.assertIn('"/home/node/workspace"', config)
self.assertEqual("/home/node/workspace", plan.workspace_plan.workdir)
if __name__ == "__main__":
unittest.main()
+3 -3
View File
@@ -7,8 +7,8 @@ import unittest
from pathlib import Path from pathlib import Path
from bot_bottle import supervise from bot_bottle import supervise
from bot_bottle.backend.docker import bottle_state from bot_bottle import bottle_state
from bot_bottle.backend.docker.bottle_state import ( from bot_bottle.bottle_state import (
BottleMetadata, BottleMetadata,
read_metadata, read_metadata,
write_metadata, write_metadata,
@@ -260,7 +260,7 @@ class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
def test_missing_backend_field_defaults_to_empty(self): def test_missing_backend_field_defaults_to_empty(self):
# Old state dirs written before PRD 0040 have no backend key. # Old state dirs written before PRD 0040 have no backend key.
import json import json
from bot_bottle.backend.docker import bottle_state as bs from bot_bottle import bottle_state as bs
path = bs.metadata_path("dev-b3") path = bs.metadata_path("dev-b3")
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps({ path.write_text(json.dumps({
-131
View File
@@ -1,131 +0,0 @@
"""Unit: capability_apply helpers (PRD 0016 Phase 2).
docker cp / exec / rm / network rm paths are covered by the
integration test in Phase 4. Here we cover:
- fetch_current_dockerfile fallback chain (per-bottle repo)
- apply_capability_change writes the per-bottle Dockerfile and
returns the correct (before, after).
- apply_capability_change rejects empty input.
"""
import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from bot_bottle.backend.docker import bottle_state, capability_apply
from bot_bottle.backend.docker.capability_apply import (
CapabilityApplyError,
apply_capability_change,
fetch_current_dockerfile,
)
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="cap-apply-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
def _teardown_fake_home(self):
self._restore()
self._tmp.cleanup()
class TestFetchCurrentDockerfile(_FakeHomeMixin, unittest.TestCase):
def setUp(self):
self._setup_fake_home()
def tearDown(self):
self._teardown_fake_home()
def test_returns_per_bottle_dockerfile_when_present(self):
bottle_state.write_per_bottle_dockerfile("dev", "FROM rebuilt\n")
self.assertEqual("FROM rebuilt\n", fetch_current_dockerfile("dev"))
def test_falls_back_to_repo_dockerfile_when_no_override(self):
# The repo's Dockerfile actually exists; the test just checks
# we get its content (non-empty) when no per-bottle override
# is set.
content = fetch_current_dockerfile("dev-no-override")
self.assertIn("FROM ", content)
class TestApplyCapabilityChange(_FakeHomeMixin, unittest.TestCase):
def setUp(self):
self._setup_fake_home()
# Stub out the docker-dependent helpers. The orchestrator's
# job is to sequence write + snapshot + push + teardown; we
# validate that sequence here, not the docker primitives.
self._calls: list[str] = []
self._orig_snapshot = capability_apply.snapshot_transcript
self._orig_push = capability_apply._push_working_tree
self._orig_teardown = capability_apply._teardown_bottle
def stub_snapshot(slug: object) -> None: # type: ignore
self._calls.append(f"snapshot:{slug}")
def stub_push(slug: object) -> None: # type: ignore
self._calls.append(f"push:{slug}")
def stub_teardown(slug: object) -> None: # type: ignore
self._calls.append(f"teardown:{slug}")
capability_apply.snapshot_transcript = stub_snapshot # type: ignore[assignment]
capability_apply._push_working_tree = stub_push # type: ignore[assignment]
capability_apply._teardown_bottle = stub_teardown # type: ignore[assignment]
def tearDown(self):
capability_apply.snapshot_transcript = self._orig_snapshot # type: ignore[assignment]
capability_apply._push_working_tree = self._orig_push # type: ignore[assignment]
capability_apply._teardown_bottle = self._orig_teardown # type: ignore[assignment]
self._teardown_fake_home()
def test_writes_per_bottle_dockerfile_and_returns_before_after(self):
bottle_state.write_per_bottle_dockerfile("dev", "FROM old\n")
before, after = apply_capability_change("dev", "FROM new\nRUN apk add ripgrep\n")
self.assertEqual("FROM old\n", before)
self.assertEqual("FROM new\nRUN apk add ripgrep\n", after)
self.assertEqual(
"FROM new\nRUN apk add ripgrep\n",
bottle_state.per_bottle_dockerfile("dev"),
)
def test_calls_snapshot_push_teardown_in_order(self):
apply_capability_change("dev", "FROM new\n")
# Snapshot + push must happen BEFORE write_per_bottle_dockerfile
# (so they capture pre-rebuild state) and BEFORE teardown (so
# the agent container still exists to docker exec / cp from).
# Teardown must be last.
self.assertEqual(
["snapshot:dev", "push:dev", "teardown:dev"],
self._calls,
)
def test_marks_preserved_before_teardown(self):
# cli.py's session-end cleanup reads the marker after the
# bottle is torn down. The marker must therefore be written
# before teardown — otherwise the cleanup would see no
# marker and rm the state dir we just populated.
apply_capability_change("dev", "FROM new\n")
self.assertTrue(bottle_state.is_preserved("dev"))
def test_first_change_falls_back_to_repo_dockerfile_for_before(self):
# No per-bottle override yet — before-diff comes from the
# repo's Dockerfile.
before, after = apply_capability_change("dev-fresh", "FROM new\n")
self.assertIn("FROM ", before)
self.assertEqual("FROM new\n", after)
def test_empty_dockerfile_rejected(self):
with self.assertRaises(CapabilityApplyError):
apply_capability_change("dev", " \n\t\n")
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -39,7 +39,7 @@ class TestStartBackendFlag(unittest.TestCase):
self.assertEqual("smolmachines", args.backend) self.assertEqual("smolmachines", args.backend)
self.assertEqual("researcher", args.name) self.assertEqual("researcher", args.name)
def test_flag_default_none_means_env_or_docker(self): def test_flag_default_none_means_env_or_default_backend(self):
args = self._build_parser().parse_args(["researcher"]) args = self._build_parser().parse_args(["researcher"])
self.assertIsNone(args.backend) self.assertIsNone(args.backend)
+19 -23
View File
@@ -1,7 +1,7 @@
"""Unit: cmd_start selector dispatch (PRD 0051). """Unit: cmd_start selector dispatch (PRD 0051).
Tests that cmd_start calls filter_select when name / backend are absent, Tests that cmd_start calls filter_select only when the agent name is
skips them when both are explicit, and returns 0 on cancel. absent, skips it when the agent is explicit, and returns 0 on cancel.
All actual launch work is stubbed so no container is created. All actual launch work is stubbed so no container is created.
""" """
@@ -45,7 +45,8 @@ class TestCmdStartSelector(unittest.TestCase):
self._tui_patch = patch.object(tui_mod, "filter_select") self._tui_patch = patch.object(tui_mod, "filter_select")
self._tui_mock = self._tui_patch.start() self._tui_mock = self._tui_patch.start()
# Ensure BOT_BOTTLE_BACKEND is absent so the backend picker fires. # Ensure BOT_BOTTLE_BACKEND is absent so omitted --backend
# flows through to the resolver default.
self._env_patch = patch.dict(os.environ, {}, clear=False) self._env_patch = patch.dict(os.environ, {}, clear=False)
self._env_patch.start() self._env_patch.start()
os.environ.pop("BOT_BOTTLE_BACKEND", None) os.environ.pop("BOT_BOTTLE_BACKEND", None)
@@ -89,22 +90,16 @@ class TestCmdStartSelector(unittest.TestCase):
self._launch_mock.assert_not_called() self._launch_mock.assert_not_called()
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Agent explicit, backend absent → backend picker fires # Agent explicit, backend absent → no picker
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def test_backend_absent_shows_backend_picker(self): def test_backend_absent_uses_default_without_picker(self):
self._tui_mock.return_value = "docker"
rc = start_mod.cmd_start(["researcher"]) rc = start_mod.cmd_start(["researcher"])
self.assertEqual(0, rc) self.assertEqual(0, rc)
self._tui_mock.assert_called_once() self._tui_mock.assert_not_called()
call_kwargs = self._tui_mock.call_args self._launch_mock.assert_called_once()
self.assertIn("backend", call_kwargs[1]["title"].lower()) _, kwargs = self._launch_mock.call_args
self.assertIsNone(kwargs["backend_name"])
def test_backend_picker_cancel_returns_0(self):
self._tui_mock.return_value = None
rc = start_mod.cmd_start(["researcher"])
self.assertEqual(0, rc)
self._launch_mock.assert_not_called()
def test_bot_bottle_backend_env_skips_backend_picker(self): def test_bot_bottle_backend_env_skips_backend_picker(self):
os.environ["BOT_BOTTLE_BACKEND"] = "docker" os.environ["BOT_BOTTLE_BACKEND"] = "docker"
@@ -116,18 +111,19 @@ class TestCmdStartSelector(unittest.TestCase):
self._tui_mock.assert_not_called() self._tui_mock.assert_not_called()
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Both absent → agent picker then backend picker # Both absent → only agent picker
# ------------------------------------------------------------------ # ------------------------------------------------------------------
def test_both_absent_shows_both_pickers_in_order(self): def test_both_absent_shows_only_agent_picker(self):
self._tui_mock.side_effect = ["researcher", "docker"] self._tui_mock.return_value = "researcher"
rc = start_mod.cmd_start([]) rc = start_mod.cmd_start([])
self.assertEqual(0, rc) self.assertEqual(0, rc)
self.assertEqual(2, self._tui_mock.call_count) self._tui_mock.assert_called_once()
first_title = self._tui_mock.call_args_list[0][1]["title"].lower() title = self._tui_mock.call_args[1]["title"].lower()
second_title = self._tui_mock.call_args_list[1][1]["title"].lower() self.assertIn("agent", title)
self.assertIn("agent", first_title) self._launch_mock.assert_called_once()
self.assertIn("backend", second_title) _, kwargs = self._launch_mock.call_args
self.assertIsNone(kwargs["backend_name"])
def test_both_absent_agent_cancel_skips_backend_picker(self): def test_both_absent_agent_cancel_skips_backend_picker(self):
self._tui_mock.side_effect = [None] self._tui_mock.side_effect = [None]
+29 -15
View File
@@ -9,7 +9,7 @@ import unittest
from pathlib import Path from pathlib import Path
from bot_bottle import supervise from bot_bottle import supervise
from bot_bottle.backend.docker import bottle_state from bot_bottle import bottle_state
from bot_bottle.cli import start as start_mod from bot_bottle.cli import start as start_mod
@@ -29,29 +29,20 @@ class _FakeHomeMixin:
class TestCaptureSessionState(_FakeHomeMixin, unittest.TestCase): class TestCaptureSessionState(_FakeHomeMixin, unittest.TestCase):
# snapshot_transcript is commented out (capability_apply is disabled);
# capture_claude_session_state now only handles the preserve marker.
def setUp(self): def setUp(self):
self._setup_fake_home() self._setup_fake_home()
# Stub the docker-dependent snapshot call so this stays a
# unit test. apply_capability_change's integration test
# covers the real docker cp path.
self._snap_calls: list[str] = []
self._orig_snap = start_mod.snapshot_transcript
start_mod.snapshot_transcript = lambda identity: ( # type: ignore
self._snap_calls.append(identity)
)
def tearDown(self): def tearDown(self):
start_mod.snapshot_transcript = self._orig_snap
self._teardown_fake_home() self._teardown_fake_home()
def test_clean_exit_snapshots_but_does_not_mark(self): def test_clean_exit_does_not_mark(self):
start_mod.capture_claude_session_state("dev-abc", exit_code=0) start_mod.capture_claude_session_state("dev-abc", exit_code=0)
self.assertEqual(["dev-abc"], self._snap_calls)
self.assertFalse(bottle_state.is_preserved("dev-abc")) self.assertFalse(bottle_state.is_preserved("dev-abc"))
def test_crash_snapshots_and_marks(self): def test_crash_marks_preserved(self):
start_mod.capture_claude_session_state("dev-abc", exit_code=137) start_mod.capture_claude_session_state("dev-abc", exit_code=137)
self.assertEqual(["dev-abc"], self._snap_calls)
self.assertTrue(bottle_state.is_preserved("dev-abc")) self.assertTrue(bottle_state.is_preserved("dev-abc"))
def test_ctrl_c_treated_as_crash(self): def test_ctrl_c_treated_as_crash(self):
@@ -64,7 +55,7 @@ class TestCaptureSessionState(_FakeHomeMixin, unittest.TestCase):
# Backends without an identity field shouldn't crash this # Backends without an identity field shouldn't crash this
# path (the _identity_from_plan helper falls back to ""). # path (the _identity_from_plan helper falls back to "").
start_mod.capture_claude_session_state("", exit_code=137) start_mod.capture_claude_session_state("", exit_code=137)
self.assertEqual([], self._snap_calls) self.assertFalse(bottle_state.is_preserved(""))
class TestSettleState(_FakeHomeMixin, unittest.TestCase): class TestSettleState(_FakeHomeMixin, unittest.TestCase):
@@ -89,5 +80,28 @@ class TestSettleState(_FakeHomeMixin, unittest.TestCase):
start_mod.settle_state("") # should not raise start_mod.settle_state("") # should not raise
class TestAttachAgent(unittest.TestCase):
def test_passes_provider_startup_args(self):
class Bottle:
argv: list[str] = []
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
self.argv = list(argv)
return 0
bottle = Bottle()
exit_code = start_mod.attach_agent(
bottle, # type: ignore[arg-type]
agent_provider_template="pi",
startup_args=("--models", "openrouter/google/gemma"),
)
self.assertEqual(0, exit_code)
self.assertEqual(
["--models", "openrouter/google/gemma"],
bottle.argv,
)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+8 -13
View File
@@ -33,7 +33,6 @@ from bot_bottle.egress import (
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
from bot_bottle.manifest import Manifest from bot_bottle.manifest import Manifest
from bot_bottle.supervise import SupervisePlan from bot_bottle.supervise import SupervisePlan
from bot_bottle.workspace import workspace_plan
SLUG = "demo-abc12" SLUG = "demo-abc12"
@@ -52,7 +51,7 @@ def _manifest(*, supervise: bool, with_git: bool, with_egress: bool) -> Manifest
bottle["git-gate"] = {"repos": { bottle["git-gate"] = {"repos": {
"upstream": { "upstream": {
"url": "ssh://git@example.com:22/x/y.git", "url": "ssh://git@example.com:22/x/y.git",
"identity": "/etc/hostname", # any existing file "key": {"provider": "static", "path": "/etc/hostname"},
}, },
}} }}
if with_egress: if with_egress:
@@ -149,19 +148,10 @@ def _plan(
spec = _spec(supervise=supervise, with_git=with_git, with_egress=with_egress) spec = _spec(supervise=supervise, with_git=with_git, with_egress=with_egress)
return DockerBottlePlan( return DockerBottlePlan(
guest_home="/home/node",
spec=spec, spec=spec,
stage_dir=STAGE, stage_dir=STAGE,
slug=SLUG, slug=SLUG,
container_name=f"bot-bottle-{SLUG}",
container_name_pinned=False,
image="bot-bottle-claude:latest",
derived_image="",
runtime_image="bot-bottle-claude:latest",
dockerfile_path="",
env_file=Path("/dev/null"), # exists, size 0 → renderer skips env_file
forwarded_env={"CLAUDE_CODE_OAUTH_TOKEN": "x"}, forwarded_env={"CLAUDE_CODE_OAUTH_TOKEN": "x"},
prompt_file=STAGE / "prompt",
git_gate_plan=_git_gate_plan(upstreams), git_gate_plan=_git_gate_plan(upstreams),
egress_plan=_egress_plan(routes), egress_plan=_egress_plan(routes),
supervise_plan=_supervise_plan() if supervise else None, supervise_plan=_supervise_plan() if supervise else None,
@@ -172,9 +162,11 @@ def _plan(
prompt_mode="append_file", prompt_mode="append_file",
image="bot-bottle-claude:latest", image="bot-bottle-claude:latest",
dockerfile="", dockerfile="",
guest_home="/home/node",
instance_name=f"bot-bottle-{SLUG}",
prompt_file=STAGE / "prompt",
guest_env={}, guest_env={},
), ),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
) )
@@ -210,7 +202,7 @@ class TestAgentAlwaysPresent(unittest.TestCase):
def test_agent_image_uses_runtime_image(self): def test_agent_image_uses_runtime_image(self):
plan = _plan() plan = _plan()
s = bottle_plan_to_compose(plan)["services"]["agent"] s = bottle_plan_to_compose(plan)["services"]["agent"]
self.assertEqual(plan.runtime_image, s["image"]) self.assertEqual(plan.image, s["image"])
def test_agent_only_on_internal_network(self): def test_agent_only_on_internal_network(self):
s = bottle_plan_to_compose(_plan())["services"]["agent"] s = bottle_plan_to_compose(_plan())["services"]["agent"]
@@ -252,6 +244,9 @@ class TestAgentAlwaysPresent(unittest.TestCase):
prompt_mode="read_prompt_file", prompt_mode="read_prompt_file",
image="bot-bottle-codex:latest", image="bot-bottle-codex:latest",
dockerfile="", dockerfile="",
guest_home="/home/node",
instance_name=f"bot-bottle-{SLUG}",
prompt_file=STAGE / "prompt",
guest_env={"CODEX_HOME": "/home/node/.codex"}, guest_env={"CODEX_HOME": "/home/node/.codex"},
) )
plan = type(plan)(**{**vars(plan), "agent_provision": provision}) # type: ignore plan = type(plan)(**{**vars(plan), "agent_provision": provision}) # type: ignore

Some files were not shown because too many files have changed in this diff Show More