Compare commits

..

113 Commits

Author SHA1 Message Date
didericis-claude cc0c952d0b fix(security): harden git_gate.py shell rendering with shlex.quote and name validation
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 44s
test / unit (push) Successful in 32s
test / integration (push) Successful in 41s
Use shlex.quote() on name and upstream_url in git_gate_render_entrypoint()
so special characters (single quotes, spaces, semicolons) cannot break or
inject into the generated sh script.

Add _GIT_NAME_RE validation in GitEntry.from_repos_entry() to restrict
repo names to [A-Za-z0-9._-]+, making the manifest the first line of
defence and shlex.quote() the belt-and-suspenders backstop.

Closes #155
2026-06-03 04:40:21 +00:00
didericis-claude 8c9d4fbc46 refactor: address PR review feedback — de-privatize helpers and rename modules
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 43s
test / unit (push) Successful in 34s
test / integration (push) Successful in 43s
- Rename _manifest_util.py → manifest_util.py (module isn't private)
- Rename _as_json_object → as_json_object, _parse_git_upstream → parse_git_upstream,
  _parse_git_gate_config → parse_git_gate_config,
  _validate_unique_git_names → validate_unique_git_names,
  _validate_egress_routes → validate_egress_routes (none are private at
  module boundary — underscore prefix was a carry-over from the old
  monolithic manifest.py where everything lived in one namespace)
- Move _is_ip_literal → util.is_ip_literal (generic, belongs in the
  top-level util module)
- Update all import sites across manifest_*.py, manifest_extends.py,
  manifest_schema.py; existing callers of manifest.py are unaffected

All 867 unit tests pass.
2026-06-03 00:33:02 -04:00
didericis-claude b9ab1263c2 refactor: split manifest.py into domain-specific modules
Closes #157. Distributes the 1,026-line manifest.py across four
focused modules:

- _manifest_util.py: ManifestError + _as_json_object (shared base)
- manifest_git.py: GitEntry, GitUser, git-gate config helpers
- manifest_egress.py: EgressRoute, EgressConfig, PipelockRoutePolicy
- manifest_agent.py: AgentProvider, Agent

manifest.py is now the residual orchestration layer: Bottle, Manifest,
and re-exports of all public names so existing callers are unaffected.
All 867 unit tests pass.
2026-06-03 00:33:02 -04:00
didericis-claude 9282bceaf8 fix: emit WARNING when Docker teardown ExitStack raises (issue #156)
test / unit (pull_request) Successful in 37s
test / integration (pull_request) Successful in 40s
test / unit (push) Successful in 32s
test / integration (push) Successful in 43s
Replace the bare `except BaseException: pass` in the `teardown` closure
with a `warn()` call that includes the container name and operation type
("compose-down"), so cleanup failures are visible in the log rather than
silently discarded.  Non-blocking: the exception is consumed and teardown
continues, preserving the original error-propagation contract.

Add test_docker_launch_teardown.py to lock the new behaviour: it injects
a RuntimeError via a mocked `compose_down` callback and asserts the
WARNING message contains the container name and operation label.
2026-06-03 04:13:53 +00:00
didericis-claude 3e50079bcc docs(prd): activate git-gate manifest redesign
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 1m10s
test / unit (push) Successful in 39s
test / integration (push) Successful in 54s
PRD 0047 is now shipped to main.
2026-06-02 23:59:34 -04:00
didericis-claude cf9aaf68e7 chore: update demo manifest and example agent to git-gate (PRD 0047)
bot-bottle.demo.json: git array → git-gate.repos with url/identity/host_key
examples/agents/implementer.md: git.user → git-gate.user
2026-06-02 23:59:34 -04:00
didericis-claude 4cf2cfc55d test: update test suite for git-gate manifest redesign (PRD 0047)
- fixtures.py: fixture_with_git_dict uses git-gate.repos + url/identity/host_key
- test_manifest_git: rewrite to use git-gate.repos; replace duplicate-name
  test (names = dict keys, always unique) with two-repos-different-hosts test
- test_manifest_git_user: _manifest → git-gate.user; update error message assertions
- test_manifest_agent_git_user: git → git-gate throughout; repos rejection test
- test_manifest_extends: git.remotes/git.user → git-gate.repos/git-gate.user
- test_provision_git: IP test updated — no host alias, single insteadOf
- test_compose: git.remotes → git-gate.repos + new field names
- test_docker_provision_git_user: git.user → git-gate.user
- test_git_gate: inline manifest dict updated to git-gate.repos
- test_smolmachines_provision: git_json → git_gate_json; remove _remote_host
2026-06-02 23:59:34 -04:00
didericis-claude 7c285fde7a feat(manifest): replace git key with git-gate (PRD 0047)
- BOTTLE_KEYS and AGENT_KEYS_OPTIONAL: "git" → "git-gate"
- GitEntry: remove from_dict/from_remote_dict; add from_repos_entry
  parsing url/identity/host_key with repo name as the dict key
- GitUser.from_dict: error messages updated to git-gate.user
- _parse_git_config → _parse_git_gate_config; repos/user subkeys
- Bottle.from_dict: reads git-gate key; "git" key raises a migration error
- Agent.from_dict: reads git-gate key; repos rejected at agent level
- manifest_extends: _child_declares_git_remotes → _child_declares_git_gate_repos
- manifest_loader: threads git-gate frontmatter key into agent_dict
2026-06-02 23:59:34 -04:00
didericis-claude 64ac204c05 docs(prd): consolidate git.user into git-gate per review
Move git.user under git-gate and remove git as a top-level key
entirely, so all git configuration lives under a single section.
2026-06-02 23:59:34 -04:00
didericis-claude 59fd132b9d docs(prd): add git-gate manifest redesign plan
PRD 0047 proposes replacing git.remotes with a top-level git-gate.repos
section and snake_case field names to make clear the config is
specifically for git-gate routing, not generic git or SSH config.

Closes #160
2026-06-02 23:59:34 -04:00
didericis f427d35e72 fix(git-http): log access-hook denial detail to stdout
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 39s
test / unit (push) Successful in 43s
test / integration (push) Successful in 59s
Previously when the access-hook returned non-zero, git-http would pipe
the hook's stderr into the 403 body sent back to the agent's git
client but never log it locally, so docker logs just showed
`"GET ... 403 -"` with no explanation. Operators had to shell into
the sidecar and re-run the hook by hand to find out why a clone was
being refused (e.g. upstream SSH unreachable, missing credentials).

Route the hook's stderr/stdout through the existing log_message
channel before sending the 403, one log line per output line so the
default request-log format stays readable. When the hook exits
non-zero with no output, log the exit code so the line is still
informative.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 23:29:39 -04:00
didericis-codex 1105d9a269 chore(skills): add quality evaluation skill
test / unit (push) Successful in 48s
test / integration (push) Successful in 56s
2026-06-02 18:42:48 +00:00
didericis-codex 46e596d0b1 docs(prd): renumber host override removal to 0046
test / unit (push) Successful in 46s
test / integration (push) Successful in 56s
2026-06-02 18:32:55 +00:00
didericis-codex a3a8a01b09 docs(prd): activate git remote host override removal
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 44s
test / unit (push) Successful in 36s
test / integration (push) Successful in 52s
2026-06-02 18:17:29 +00:00
didericis-codex 941f316462 feat(git-gate): remove git remote host override plumbing 2026-06-02 18:17:24 +00:00
didericis-codex be3defe5d8 docs(prd): add git remote host override removal plan 2026-06-02 18:16:24 +00:00
didericis 3885e2f5ad fix(workspace): include hidden cwd files in docker layer
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 42s
test / unit (push) Successful in 33s
test / integration (push) Successful in 40s
2026-06-02 13:12:40 -04:00
didericis-codex a08829573d docs(prd): activate workspace porting plan
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 40s
2026-06-02 17:01:53 +00:00
didericis-codex d5fcbe53ef feat(workspace): port cwd across backends 2026-06-02 17:01:19 +00:00
didericis-codex 6150497b47 feat(workspace): trust resolved project path 2026-06-02 16:57:52 +00:00
didericis-codex 5308d53288 feat(workspace): add shared workspace plan 2026-06-02 16:56:57 +00:00
didericis-codex d01f4b6613 docs(prd): add workspace porting plan 2026-06-02 16:54:50 +00:00
didericis 44273be9eb fix(dashboard): stop agents in dashboard from moving during selection
test / unit (push) Successful in 38s
test / integration (push) Successful in 58s
2026-06-02 12:47:00 -04:00
didericis 096c7b8196 fix(codex): update cli image version
test / unit (push) Successful in 37s
test / integration (push) Successful in 57s
2026-06-02 12:42:09 -04:00
didericis 0432a5d3ff fix(codex): keep dummy auth refresh timestamp valid
test / unit (push) Successful in 49s
test / integration (push) Successful in 1m0s
2026-06-02 12:40:14 -04:00
didericis-claude fcd1b34e49 docs: mark PRD 0044 Active
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 41s
test / unit (push) Successful in 34s
test / integration (push) Successful in 43s
2026-06-02 12:12:08 -04:00
didericis-claude a0762ac2d3 test: add cross-backend print parity tests (PRD 0044)
Shared fixtures build DockerBottlePlan and SmolmachinesBottlePlan from
identical git_gate_plan and egress_plan inputs and assert that both
backends render the same git gate lines (name → host:port) and egress
lines (host [auth:scheme] when authenticated, host alone otherwise).
2026-06-02 12:12:08 -04:00
didericis-claude 53219a55e1 refactor: hoist plan fields and print to BottlePlan base class (PRD 0044)
Move git_gate_plan, egress_plan, supervise_plan, and agent_provision
from DockerBottlePlan and SmolmachinesBottlePlan into BottlePlan.
Replace the abstract print method with a single concrete implementation
that renders git gate entries as "name → upstream_host:upstream_port"
and egress routes with conditional "[auth:scheme]" annotations.
2026-06-02 12:12:08 -04:00
didericis-claude 71ac555f25 docs(prd): add PRD 0044 — print parity across backends 2026-06-02 12:12:08 -04:00
didericis-claude f25fa589fe fix(git-http): extract peer variable to clarify access hook call convention
test / unit (push) Successful in 31s
test / integration (push) Successful in 43s
Both remote-addr and peer-addr args to the access hook are the same
TCP peer in this non-proxied stack. Extract a `peer` variable so the
intentional repetition is visible. Closes #148.
2026-06-02 16:08:15 +00:00
didericis-claude 4fdf354b4f docs: mark PRD 0043 Active
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 43s
test / unit (push) Successful in 34s
test / integration (push) Successful in 41s
2026-06-02 11:48:24 -04:00
didericis-claude 5a2011c48f fix: close child stdout pipes on restart and loop convergence (PRD 0043)
Closes #140. In restart_daemon, the old process's stdout pipe was never
explicitly closed after p.wait() returned, leaking the fd until the
supervisor object was GC'd. Similarly, when the watch loop converged
(all children dead), no pipe was closed. Both paths now call
p.stdout.close() immediately after the process is confirmed exited.
Tests enforce this with warnings.simplefilter("error", ResourceWarning)
in TestSupervisor.setUp.
2026-06-02 11:48:24 -04:00
didericis-claude 19ebcd52a1 docs: add PRD 0043 2026-06-02 11:48:24 -04:00
didericis-claude 2c061d9cd9 docs: mark PRD 0042 Active
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 55s
test / unit (push) Successful in 40s
test / integration (push) Successful in 46s
2026-06-02 11:30:54 -04:00
didericis-claude cceb300d58 test: add cross-backend parity tests (PRD 0042)
Closes #139. Adds tests/unit/test_backend_parity.py which verifies that
DockerBottle and SmolmachinesBottle expose identical observable contracts
for agent_argv shape, env injection, exec user-switching, ExecResult
fields, and close() idempotency. All assertions use mock subprocess
layers — no live Docker daemon or VM required.
2026-06-02 11:30:54 -04:00
didericis-claude b63927368a docs: add PRD 0042 2026-06-02 11:30:54 -04:00
didericis 4319b4ef3b refactor(git-http): rename variable to indicate configurability
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 54s
test / unit (push) Successful in 40s
test / integration (push) Successful in 57s
2026-06-02 11:24:54 -04:00
didericis-claude 71005d56e2 docs: mark PRD 0041 Active
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 53s
2026-06-02 11:23:19 -04:00
didericis-claude 96b0c3f1fa fix(git-http): validate Content-Length and cap body size (PRD 0041)
Before this change, int() on a non-numeric Content-Length raised an
unhandled ValueError, crashing the request handler. There was also no
upper bound on how much memory a POST body could consume.

After this change:
- Non-numeric or missing Content-Length returns HTTP 400.
- Negative Content-Length returns HTTP 400.
- Bodies declared larger than 1 MiB (_MAX_BODY_BYTES) return HTTP 413,
  matching the cap already in supervise_server.py.

Closes #138
2026-06-02 11:23:19 -04:00
didericis-claude 3087a9aa8b docs: add PRD 0041 2026-06-02 11:23:19 -04:00
didericis-claude e43f75dd1b refactor: rename machine_name to instance_name in _bottle_for_slug
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 41s
test / unit (push) Successful in 39s
test / integration (push) Successful in 1m0s
2026-06-02 11:16:17 -04:00
didericis-claude 4ad1ff3898 docs: mark PRD 0040 Active 2026-06-02 11:16:17 -04:00
didericis-claude a3d9ac9605 feat: persist backend in BottleMetadata; use it in resume and dashboard reattach (PRD 0040)
BottleMetadata gains a backend field (default ""). Docker prepare writes
"docker"; smolmachines prepare writes "smolmachines". read_metadata
deserialises it with "" as the backward-compatible default.

resume now passes metadata.backend to _launch_bottle so a preserved
smolmachines bottle is resumed on the right backend without requiring
BOT_BOTTLE_BACKEND to be set manually.

_bottle_for_slug now reads metadata.backend and constructs a
SmolmachinesBottle for smolmachines slugs instead of always defaulting
to DockerBottle. No-metadata slugs still fall back to Docker.

Closes #137
2026-06-02 11:16:17 -04:00
didericis-claude 70c9f7254c docs: add PRD 0040 2026-06-02 11:16:17 -04:00
didericis-claude b9108339e7 docs: mark PRD 0039 Active
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 43s
test / unit (push) Successful in 30s
test / integration (push) Successful in 41s
2026-06-02 11:15:27 -04:00
didericis-claude e5b5dd16f1 feat(dashboard): guard capability-block approval for smolmachines bottles (PRD 0039)
apply_capability_change is Docker-only teardown/apply code. Before this
change it was called regardless of backend, so approving a capability-block
proposal from a smolmachines agent would run Docker commands against a
slug that has no Docker container.

After this change approve() reads the bottle's metadata: if compose_project
is empty (the smolmachines indicator) it raises CapabilityApplyError with
a clear operator message before any teardown runs. Docker bottles (non-empty
compose_project) and unknown bottles (no metadata) fall through to the
existing Docker path unchanged.

Closes #136
2026-06-02 11:15:27 -04:00
didericis-claude cf76d1a245 docs: add PRD 0039 2026-06-02 11:15:27 -04:00
didericis-claude 717a9126e1 docs: mark PRD 0038 Active
test / unit (pull_request) Successful in 38s
test / integration (pull_request) Successful in 56s
test / unit (push) Successful in 31s
test / integration (push) Successful in 42s
2026-06-02 14:38:44 +00:00
didericis-claude 8830306101 feat(smolmachines): resolve manifest env through resolve_env() (PRD 0038)
Before this change smolmachines prepare.py spliced bottle.env directly
into guest_env, so ?prompt and ${HOST_VAR} entries reached the VM as
raw sentinels rather than being prompted or interpolated.

After this change prepare.py calls resolve_env(), matching the Docker
backend's contract. Forwarded (secret/interpolated) values still flow
through smolvm -e K=V argv — the known exposure gap documented in PRD
0038's open question.

Closes #135
2026-06-02 14:38:36 +00:00
didericis-claude 1c242b0ad9 docs: add PRD 0038
test / unit (pull_request) Successful in 52s
test / integration (pull_request) Successful in 1m2s
2026-06-02 10:28:04 -04:00
didericis-codex f95ef0c446 complete(prd): mark PRD 0037 active
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 44s
test / unit (push) Successful in 29s
test / integration (push) Successful in 47s
2026-06-02 08:15:20 +00:00
didericis-codex 6e954da9b7 fix(pipelock): validate yaml render config 2026-06-02 08:15:20 +00:00
didericis-codex 9185c145a1 docs(prd): add pipelock yaml contract
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 42s
2026-06-02 04:14:45 -04:00
didericis-codex a79ef61b62 complete(prd): mark PRD 0036 active
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 44s
test / unit (push) Successful in 31s
test / integration (push) Successful in 45s
2026-06-02 08:10:34 +00:00
didericis-codex 0a8bba58c7 fix(codex): harden auth redaction 2026-06-02 08:10:34 +00:00
didericis-codex 2247d730cd docs(prd): add codex auth redaction policy
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 42s
2026-06-02 04:09:18 -04:00
didericis-codex 3472e06efb complete(prd): mark PRD 0035 active
test / unit (pull_request) Successful in 45s
test / integration (pull_request) Successful in 1m4s
test / unit (push) Successful in 36s
test / integration (push) Successful in 46s
2026-06-02 08:06:53 +00:00
didericis-codex 82ce5d3034 fix(supervise): bound response waits 2026-06-02 08:06:45 +00:00
didericis-codex 7c260eeff9 docs(prd): add supervise wait bounds
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 54s
2026-06-02 07:58:39 +00:00
didericis-codex fe6059e4a6 complete(prd): mark PRD 0034 active
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 52s
test / unit (push) Successful in 34s
test / integration (push) Successful in 50s
2026-06-02 07:52:38 +00:00
didericis-codex 31708abfad fix(sidecar): queue restart signals 2026-06-02 07:52:19 +00:00
didericis-codex 1b34b1df85 docs(prd): add sidecar restart semantics
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 59s
2026-06-02 07:43:34 +00:00
didericis-codex 51831bf9c0 complete(prd): mark PRD 0033 active
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 57s
test / unit (push) Successful in 39s
test / integration (push) Successful in 56s
2026-06-02 07:32:29 +00:00
didericis-codex 8f28bd81a7 refactor(manifest): split schema boundaries 2026-06-02 07:32:06 +00:00
didericis-codex 662e3e1f95 docs(prd): point manifest boundaries to issue 125
test / unit (pull_request) Successful in 41s
test / integration (pull_request) Successful in 57s
2026-06-02 07:31:29 +00:00
didericis-codex 6315456a59 docs(prd): add manifest schema boundaries
test / unit (pull_request) Successful in 48s
test / integration (pull_request) Successful in 1m4s
2026-06-02 07:23:04 +00:00
didericis-claude a81f0ffa49 fix(smolmachines): raise SmolvmError instead of die() on wait_exec_ready timeout
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 58s
test / unit (push) Successful in 38s
test / integration (push) Successful in 55s
die() raises Die(SystemExit), which implies a process exit. A timeout in
wait_exec_ready is a bringup failure — raising SmolvmError lets the caller
decide whether it's fatal, consistent with how machine_start failures propagate.
2026-06-02 06:29:05 +00:00
didericis-claude c39bbe265b complete(prd): mark PRD 0032 active
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 58s
All three issues implemented and 805 tests passing.
2026-06-02 06:23:46 +00:00
didericis-claude 0d922371b0 refactor(smolmachines): decompose launch(), add wait_exec_ready, file-lock allocate() (PRD 0032)
Decompose the 207-line launch() into six named helpers: _allocate_resources,
_mint_certs, _start_bundle, _discover_urls, _launch_vm, _init_vm. Each has
explicit inputs/outputs and is independently testable.

Replace time.sleep(1.5) with smolvm.wait_exec_ready(), which polls
`machine exec true` with exponential backoff. Exits as soon as the exec
channel is ready; dies loudly with a timeout message instead of silently
leaving the VM in an unknown state.

File-lock loopback_alias.allocate() with fcntl.flock(LOCK_EX) so concurrent
bottle launches can't race on docker state and claim the same alias.
2026-06-02 06:23:39 +00:00
didericis-claude fe97b6014d docs(prd): PRD 0032 — smolmachines launch decomposition
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 44s
Split launch() into named per-step helpers, replace time.sleep(1.5) with
a readiness poll, and file-lock loopback alias allocation. Addresses the
three actionable items from the #117 hotspot review of smolmachines/launch.py.
2026-06-02 06:14:16 +00:00
didericis-claude 07c8593999 refactor(egress): EgressRoute inherits Route from egress_addon_core
test / unit (pull_request) Successful in 32s
test / integration (pull_request) Successful in 47s
test / unit (push) Successful in 31s
test / integration (push) Successful in 38s
EgressRoute now extends egress_addon_core.Route, which holds the four
wire-visible fields (host, path_allowlist, auth_scheme, token_env).
EgressRoute adds only the three host-side fields (token_ref, roles,
tls_passthrough) that are never serialised to the sidecar.

_route_to_yaml_fields is typed as Route -> dict, making the host→wire
boundary explicit: only fields declared on the base class cross into the
YAML the addon reads.
2026-06-02 05:58:59 +00:00
didericis-claude f15721b424 complete(prd): mark PRD 0031 active
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 46s
Provisioned-wins merge and _route_to_yaml_fields are implemented and all
tests pass.
2026-06-02 05:45:28 +00:00
didericis-claude 10d0872043 refactor(egress): provisioned-wins merge + _route_to_yaml_fields (PRD 0031)
Replace _merge_provider_route's five-case nested conditional with a flat
provisioned-wins merge: provider routes claim their hosts outright, manifest
routes for unclaimed hosts append unchanged. Token slot assignment moves to a
single _assign_token_slots pass over the merged list.

Add _route_to_yaml_fields as the single authoritative EgressRoute→YAML mapping,
eliminating the risk of EgressRoute and egress_addon_core.Route silently
drifting apart when new fields are added.

egress_manifest_routes is now a pure lifter with no slot assignment.
_merge_provider_route and _find_or_alloc_token_env are removed.

Tests updated: conflict-die case removed, upgrade-bare replaced with
provider-wins semantics, slot-assignment tests moved to TestSlotAssignment.
2026-06-02 05:45:20 +00:00
didericis-claude ae33d1abfb docs(prd): revise PRD 0031 — provisioned-wins merge + Route type consolidation
test / unit (pull_request) Successful in 42s
test / integration (pull_request) Successful in 1m0s
Expands scope to cover both remaining egress hotspot tasks from #117:
- Replaces the named-helper design with a flat provisioned-wins merge
  (provider routes own their hosts; manifest fills gaps; no upgrade or
  conflict-detection logic needed).
- Adds _route_to_yaml_fields as the single authoritative EgressRoute→Route
  mapping to prevent silent type drift between host and addon.
- Notes that the mitmproxy pure-function split is already clean (decide +
  is_git_push_request) and requires no structural change.
2026-06-02 05:26:15 +00:00
didericis-claude f596464f3f docs(prd): add PRD 0031 — split _merge_provider_route into named case helpers
test / unit (pull_request) Successful in 31s
test / integration (pull_request) Successful in 41s
2026-06-02 05:08:59 +00:00
didericis-claude e528d5c5af docs(prd): update PRD 0030 design to reflect provisioned_env approach
test / unit (pull_request) Successful in 33s
test / integration (pull_request) Successful in 45s
test / unit (push) Successful in 31s
test / integration (push) Successful in 43s
Revises the Design section to describe the implemented solution:
provisioned_env on AgentProvisionPlan rather than an intermediate
egress_resolve_token_values_with_provider function. Drops the old
sentinel/lazy-import design narrative.
2026-06-02 04:54:09 +00:00
didericis-claude 0e29bcc829 refactor(egress): use provisioned_env instead of sentinel for Codex token (PRD 0030)
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 45s
Add `provisioned_env: dict[str, str]` to `AgentProvisionPlan`. When
`forward_host_credentials=True`, `agent_provision_plan` reads the host
Codex access token at prepare time and stores it under
`CODEX_HOST_CREDENTIAL_TOKEN_REF`. Both backends merge `provisioned_env`
over `os.environ` before calling `egress_resolve_token_values`, so the
token slot resolves like any other manifest-declared token ref.

Removes `egress_resolve_token_values_with_provider` and the sentinel
`continue` skip from `egress_resolve_token_values`. The function is now
fully generic — it neither knows nor cares about provider identity.
2026-06-02 04:53:23 +00:00
didericis-claude 8c2b59ca94 complete(prd): mark PRD 0030 active
test / unit (push) Successful in 45s
test / integration (push) Successful in 1m0s
test / unit (pull_request) Successful in 39s
test / integration (pull_request) Successful in 58s
2026-06-02 04:22:52 +00:00
didericis-claude 75f0f9d907 refactor(egress): deduplicate token resolution across backends (PRD 0030)
Extract egress_resolve_token_values_with_provider into bot_bottle/egress.py.
Both docker and smolmachines launch paths now call the shared function
instead of duplicating the forward_host_credentials / CODEX_HOST_CREDENTIAL_TOKEN_REF
resolution block.

Also fixes the host_env: object annotation on smolmachines._resolve_token_env
to the correct dict[str, str].

Closes #118.
2026-06-02 04:22:43 +00:00
didericis-claude 6682357fbb docs(prd): add PRD 0030 — deduplicate egress token resolution
Extracts the forward_host_credentials / CODEX_HOST_CREDENTIAL_TOKEN_REF
resolution block, currently copy-pasted in both docker and smolmachines
launch files, into a single shared function in bot_bottle/egress.py.

Closes #118. Found via #117 hotspot review.
2026-06-02 04:17:39 +00:00
didericis 2dd8113f7c fix(smolmachines): retry CA install after exec SIGKILL
test / unit (push) Successful in 38s
test / integration (push) Successful in 54s
2026-06-01 23:28:33 -04:00
didericis-codex 36e3443d2e fix(codex): defer workspace trust handling
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 42s
test / unit (push) Successful in 30s
test / integration (push) Successful in 44s
2026-06-02 03:11:51 +00:00
didericis-codex d6ebd0d2eb fix(egress): skip token slots for unauth provider routes
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 43s
2026-06-02 03:06:10 +00:00
didericis-codex eb6bace84f complete(prd): mark PRD 0029 active
test / unit (pull_request) Successful in 35s
test / integration (pull_request) Successful in 42s
2026-06-02 03:00:47 +00:00
didericis-claude f8fc29ce87 refactor(manifest): remove empty EGRESS_ROLES and related plumbing
test / unit (pull_request) Successful in 36s
test / integration (pull_request) Successful in 53s
EGRESS_ROLES, EGRESS_SINGLETON_ROLES, and PROVIDER_EGRESS_ROLES were
all empty frozensets after the codex_auth and claude_code_oauth roles
were removed. Delete the constants and all validation code that iterated
over them (the singleton-role loop and provider-role check in
_validate_egress_routes, the EGRESS_ROLES membership test in
EgressRoute.from_dict). EgressRoute.from_dict now rejects any role
string unconditionally; _validate_egress_routes loses its
agent_provider_template parameter entirely.

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-claude 938a0e05d6 refactor(manifest): remove codex_auth egress role
Both provider-owned roles are now gone. Provider auth routes are
provisioner-owned (claude: auth_token, codex: forward_host_credentials);
the role field and validation plumbing stay for future use but EGRESS_ROLES
is empty. Any manifest declaring a role now fails at parse time.

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis f768d3a853 fix(agent): move default claude env vars to the right location 2026-06-01 22:24:17 -04:00
didericis-claude f32b7eb299 fix(agent): always emit passthrough egress route for api.anthropic.com
Mirrors the Codex pattern: Claude always gets a tls_passthrough route
for api.anthropic.com so user-set tokens aren't stripped by pipelock,
whether or not auth_token is declared. Auth injection (scheme + token_ref)
and the placeholder env only apply when auth_token is set.

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-claude de9bd7eb83 feat(manifest): add agent_provider.auth_token for Claude OAuth via egress
Operators can now declare:

  agent_provider:
    template: claude
    auth_token: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN

and the provisioner injects a provider-owned api.anthropic.com egress
route (Bearer, tls_passthrough) rather than requiring a manually
declared route with the former claude_code_oauth role.

Changes:
- Add auth_token field to AgentProvider; validate claude-only.
- Remove claude_code_oauth from EGRESS_ROLES / PROVIDER_EGRESS_ROLES.
  Manifests that declare the role now fail at parse time with "unknown
  role" — the provisioner owns the route.
- agent_provision_plan: replace manifest_egress_routes/has_provider_auth
  with auth_token; Claude branch injects the api.anthropic.com route,
  placeholder env, and nonessential-traffic flags when auth_token is set.
- Add hidden_env_names: frozenset[str] to AgentProvisionPlan; Claude
  branch populates it with CLAUDE_CODE_OAUTH_TOKEN.
- Remove auth_role from AgentProviderRuntime and placeholder_env_for().
- print_util.visible_agent_env_names: accept hidden_env_names from the
  plan instead of dispatching on agent_provider_template.
- Both backends: drop manifest_egress_routes call, pass auth_token.
- PRD 0029 rescoped to cover both Codex and Claude provider auth.

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-claude 952dcd7eec refactor(agent): move placeholder env injection into agent_provision_plan
The has_provider_auth check and egress-placeholder injection were
duplicated in both backends. Move them into agent_provision_plan so
the provisioner owns that decision entirely:

- Replace has_provider_auth: bool param with manifest_egress_routes,
  compute has_provider_auth internally from the route roles.
- Inject CLAUDE_CODE_OAUTH_TOKEN=egress-placeholder inside the plan
  when has_provider_auth, alongside the existing nonessential-traffic
  vars. Backends no longer touch the placeholder env.
- Remove placeholder_env from AgentProviderRuntime; expose
  placeholder_env_for() for print_util's hide-from-summary logic.

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-claude 59df0b0f0f fix(codex): emit passthrough egress routes when not forwarding host credentials
When forward_host_credentials is false, Codex bottles should still get
tls_passthrough routes for the OpenAI/ChatGPT hosts so that tokens a
user sets via `codex login` after launch aren't stripped by pipelock's
header DLP. Previously no routes were emitted, which would have blocked
those requests entirely once pipelock enforcement tightens.

Rename the test to reflect the new expected behavior.

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-claude c0219dddd5 fix(egress): break circular import with manifest via TYPE_CHECKING
manifest → agent_provider → egress → manifest created a cycle that
caused ImportError on any module import. With from __future__ import
annotations already present, Bottle is only needed at type-check time
(annotations are lazy strings under PEP 563).

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-claude 884cedc160 refactor: provision egress routes via AgentProvisionPlan
Remove provider-specific branching from egress.py and pipelock.py.
Previously, `egress_routes_for_bottle` and `pipelock_effective_tls_passthrough`
both contained `template == "codex"` checks — the same pattern the rest
of the PR moved out of the backends.

Root cause: `EgressRoute` had no `tls_passthrough` field, so pipelock
couldn't learn from the synthesised Codex routes that they needed
passthrough. Fix:

- Add `EgressRoute.tls_passthrough: bool`. `egress_manifest_routes` lifts
  the existing `pipelock.tls_passthrough` manifest flag here; provider
  routes set it directly.
- Add `AgentProvisionPlan.egress_routes`. `agent_provision_plan` populates
  it for Codex + `forward_host_credentials`, including `tls_passthrough=True`.
- Replace Codex-specific `egress_routes_for_bottle` logic with a generic
  `_merge_provider_route` helper. Backends call `egress_routes_for_bottle(bottle,
  plan.egress_routes)`; no provider type checks inside egress or pipelock.
- Rewrite `pipelock_effective_tls_passthrough` to read `route.tls_passthrough`
  from the merged route set instead of re-implementing the provider check.
- Both backends now call `agent_provision_plan` before `Egress.prepare` and
  `PipelockProxy.prepare`, threading `plan.egress_routes` to both. `has_provider_auth`
  is derived from `egress_manifest_routes` (manifest routes only — provider
  routes carry no auth roles, so the result is identical).

Assisted-by: Claude Code
2026-06-01 22:24:17 -04:00
didericis-codex 76a7921ae6 refactor(agent): move claude env defaults into plan 2026-06-01 22:24:17 -04:00
didericis-codex c8ab0c67a8 refactor(agent): surface provider env defaults 2026-06-01 22:24:17 -04:00
didericis-codex e808e81b87 refactor(agent): group provider provisioning into plan 2026-06-01 22:24:17 -04:00
didericis-codex 36ce7aed4f refactor(codex): derive trusted paths from guest home 2026-06-01 22:24:17 -04:00
didericis-codex a5d83bdcdc fix(codex): trust launch home directory 2026-06-01 22:24:17 -04:00
didericis-codex 8e6583fcb7 fix(codex): trust bottle workspace on launch 2026-06-01 22:24:17 -04:00
didericis-codex ac1aa197d4 fix(smolmachines): reset codex runtime db before auth check 2026-06-01 22:24:17 -04:00
didericis-codex 68e5097534 fix(codex): make host-credential bottles actually authenticate
Debugging a live codex smolmachines bottle surfaced three independent
failures past the sign-in screen; fix each so forward_host_credentials
works end to end:

- codex_auth: dummy access/id tokens now inherit the *real* host token's
  exp instead of now+1h. Codex (0.135) refreshes when its local token's
  JWT exp lapses; with a placeholder refresh_token that refresh fails and
  drops to the sign-in screen. Aligning exp tracks the real token's life.

- prepare: set CODEX_CA_CERTIFICATE to the agent CA bundle for codex
  bottles. Codex is rustls and ignores the system store / NODE_EXTRA_CA_
  CERTS; it reads CODEX_CA_CERTIFICATE (fallback SSL_CERT_FILE) for custom
  roots across HTTPS + wss, so it must be pointed at the egress MITM CA or
  injection can't work without tls_passthrough.

- pipelock: auto tls_passthrough the Codex API hosts when
  forward_host_credentials is on. Egress injects the bearer before
  pipelock, whose header DLP then flags the JWT ("request header contains
  secret") and the retry storm trips its 429. passthrough host-gates the
  CONNECT but skips decrypt+rescan of egress-owned auth. The auto-added
  routes aren't in bottle.egress.routes, so the hosts are added explicitly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:24:17 -04:00
didericis-codex f8a4e6f40b fix(codex): include account claims in dummy auth 2026-06-01 22:24:17 -04:00
didericis-codex a6332b9535 fix(codex): provision dummy user auth state 2026-06-01 22:24:17 -04:00
didericis-codex 62dd7b2aa5 fix(codex): forward host credentials to api route 2026-06-01 22:24:17 -04:00
didericis-codex 711cb9c194 feat(codex): inject host credentials via egress 2026-06-01 22:24:17 -04:00
didericis-codex 0b80ffb16a docs(prd): add Codex host credentials egress plan 2026-06-01 22:24:17 -04:00
didericis 2350cd11e0 build(images): install python3 in claude/codex bottles
test / unit (push) Successful in 37s
test / integration (push) Successful in 56s
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 22:23:38 -04:00
didericis-codex 6ea19a8d53 fix(git-gate): use smart http for smolmachines pushes
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 54s
test / unit (push) Successful in 37s
test / integration (push) Successful in 44s
2026-05-29 23:21:50 -04:00
didericis-codex 630e65e9a4 test(egress): cover blocked git push fail-fast
test / unit (pull_request) Successful in 28s
test / integration (pull_request) Successful in 42s
2026-05-29 22:13:17 -04:00
didericis-codex 7bffaa791c fix(git-gate): shorten daemon client timeout
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 42s
2026-05-29 22:02:17 -04:00
didericis-codex de2267d1b4 fix(git-gate): bound daemon client sessions
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 44s
2026-05-29 21:57:31 -04:00
didericis-codex dcaee53cec docs(codex): clarify codex auth marker
test / unit (pull_request) Successful in 28s
test / integration (pull_request) Successful in 40s
test / unit (push) Successful in 28s
test / integration (push) Successful in 42s
2026-05-29 02:45:00 -04:00
didericis-codex cea832b21d fix(codex): stop injecting api key placeholder
test / unit (pull_request) Successful in 27s
test / integration (pull_request) Successful in 41s
2026-05-29 02:39:37 -04:00
110 changed files with 9248 additions and 2327 deletions
+76
View File
@@ -0,0 +1,76 @@
---
name: quality-eval
description: Use when the user asks to objectively evaluate, score, rate, audit, or quality-gate code, codebases, files, pull requests, or snippets using a strict 5-dimension engineering rubric with scores and refactoring steps.
metadata:
short-description: Score code quality with a strict rubric
---
# Quality Eval
## Role
Act as a Staff Software Engineer and automated quality gate. Evaluate code objectively against the rubric below, surface hidden anti-patterns, and provide a mathematical grade with atomic refactoring steps.
## Evaluation Rules
- Evaluate only against the five rubric dimensions.
- Be candid. Do not inflate scores for politeness.
- Avoid generic advice. Every recommendation must name a specific code location, behavior, or pattern and include a concrete improvement direction.
- Inspect the code before scoring. For codebases, read enough representative files, tests, and architecture boundaries to justify the scope.
- When exact line numbers are available, cite them.
- Do not reveal private chain-of-thought. In the required `Chain of Thought Analysis` section, provide a concise, step-by-step audit rationale with observable findings and score justifications.
## Rubric
Score each dimension from 1 to 5 using these anchors:
| Dimension | Score 1 (Fail) | Score 3 (Pass) | Score 5 (Exemplary) |
| :--- | :--- | :--- | :--- |
| **Architecture** | Spaghettified; tight coupling; violated separation of concerns. | Modular but relies on leaky abstractions or mixed domains. | Strict domain isolation; follows SOLID; clear dependency inversion. |
| **Readability** | Cryptic naming; deep nesting (>3 levels); widespread DRY violations. | Idiomatic but features over-complex functions or sparse documentation. | Self-documenting; expressive naming; high cohesion; flat structure. |
| **Resilience** | Swallows errors blindly; lacks contextual logging; fragile to bad input. | Basic try/catch blocks present but lacks granular, typed error handling. | Explicit error boundaries; contextual logging; structured failure modes. |
| **Testability** | Hardcoded dependencies make mocking or isolated testing impossible. | Pure functions are testable, but side-effect heavy logic lacks test hooks. | Decoupled IO; deterministic execution; structured for unit and integration tests. |
| **SecOps** | Hardcoded secrets; O(n^2) bottlenecks; zero input sanitization. | Safe from obvious flaws but lacks deep defensive optimization. | Validated inputs; optimized algorithmic complexity; zero security debt. |
## Scoring Method
1. Determine the evaluated scope and primary language.
2. Identify concrete evidence for each dimension.
3. Assign integer dimension scores from 1 to 5.
4. Compute `composite_score` as the arithmetic mean of the five dimension scores, rounded to one decimal place.
5. Include code snippets only when they make a refactoring step more actionable.
## Required Output
Structure every response into exactly these three Markdown sections:
### 1. Chain of Thought Analysis
Provide a concise step-by-step audit rationale. Name specific files, functions, patterns, anti-patterns, and rubric anchors. Keep it evidence-based and do not include hidden private reasoning.
### 2. Normalized Score Report
```json
{
"evaluation_metadata": {
"target_scope": "string",
"primary_language": "string"
},
"metrics": {
"architecture_and_modularity": 0,
"readability_and_maintainability": 0,
"error_handling_and_resilience": 0,
"testability_and_mocking": 0,
"security_and_performance": 0
},
"composite_score": 0.0
}
```
### 3. Atomic Refactoring Playbook
* **High Priority (To lift Score 1/2 to 3):**
- [ ] Actionable, specific refactoring step with file/line/context reference.
* **Medium Priority (To lift Score 3 to 4/5):**
- [ ] Optimization or architectural pattern implementation step.
@@ -0,0 +1,3 @@
display_name: Quality Eval
short_description: Scores code quality with a strict five-dimension rubric and refactoring playbook.
default_prompt: Evaluate this code objectively using the quality-eval rubric and return the three-section score report.
+1 -1
View File
@@ -23,7 +23,7 @@ FROM node:22-slim
# tool (curl itself, plus anything that shells out to it) works
# against pipelock's bumped TLS without the agent needing local DNS.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
# Install claude-code globally. Pinned to the version verified in the v1
+2 -2
View File
@@ -6,10 +6,10 @@
FROM node:22-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
&& rm -rf /var/lib/apt/lists/*
RUN npm install -g --no-fund --no-audit @openai/codex@0.134.0 \
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
&& npm cache clean --force
USER node
+3 -1
View File
@@ -31,6 +31,7 @@
# 9099 egress (mitmproxy, pipelock's upstream — not externally
# addressed by the agent)
# 9418 git-gate (git-daemon)
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
# 9100 supervise (MCP HTTP)
# Stage 1: pipelock binary. The upstream pipelock image is a
@@ -81,6 +82,7 @@ COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
RUN chmod +x /app/egress-entrypoint.sh
@@ -97,7 +99,7 @@ RUN mkdir -p \
# Documentation only — the compose renderer publishes whichever
# subset the bottle uses.
EXPOSE 8888 9099 9418 9100
EXPOSE 8888 9099 9418 9420 9100
# WORKDIR matches Dockerfile.supervise's prior layout so the
# in-app same-dir import in supervise_server.py stays deterministic.
+36 -13
View File
@@ -157,14 +157,8 @@ and MCP endpoints resolve without an agent-side change.
upstream has *now* (fail-closed if unreachable). The agent's
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
so push, fetch, clone, and pull all route through. The agent
never sees the upstream credential. If the upstream's hostname
isn't resolvable from the gate container (e.g. a Tailscale-only
host whose public DNS points elsewhere), pin its IP via
`ExtraHosts: { "<hostname>": "<ip>" }` on the `bottle.git` entry —
the gate's `/etc/hosts` gets the override while the agent's
`insteadOf` rewrite still keys off the original hostname. Brought
up only when `bottle.git` has entries. Design in
`docs/prds/0008-git-gate.md`.
never sees the upstream credential. Brought up only when
`bottle.git` has entries. Design in `docs/prds/0008-git-gate.md`.
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
base, stdlib-only) that holds API tokens declared in
`bottle.cred_proxy.routes`. Each route names a `path`,
@@ -349,11 +343,40 @@ The `gitea-dev` bottle. Backs my work on personal projects: provider
auth through egress and gitea.dideric.is over SSH.
````
For a Codex-backed base bottle, set `agent_provider.template: codex`
and use the `codex_auth` egress role for the OpenAI API route. The
built-in Codex template uses `Dockerfile.codex`; set
`agent_provider.dockerfile` to build the agent from a custom
Dockerfile while keeping the bot-bottle sidecars in place.
For a Codex-backed base bottle, set `agent_provider.template: codex`.
The Codex template expects ChatGPT/device login state instead of an
`OPENAI_API_KEY` env var; no API-key placeholder is forwarded into the
agent. To let bot-bottle read the host's current Codex ChatGPT access
token and inject it from egress only for Codex's API calls, opt in
explicitly:
```yaml
agent_provider:
template: codex
forward_host_credentials: true
egress:
routes:
- host: auth.openai.com
path_allowlist:
- /api/accounts/deviceauth/
```
Run `codex login --device-auth` on the host before launch. The
launcher reads `tokens.access_token` from the host's
`~/.codex/auth.json`, verifies it is fresh user/device auth, and passes
it to the sidecar's `EGRESS_TOKEN_N` env slot. The agent container gets
a dummy `~/.codex/auth.json` that preserves the host auth-mode shape
but replaces credential values with placeholders. It keeps the selected
ChatGPT account id so Codex sends requests for the same account while
egress owns the real bearer token. The agent never receives real access
tokens, refresh tokens, or `OPENAI_API_KEY`. The effective egress table
automatically adds or upgrades `api.openai.com` and `chatgpt.com` to
authenticated routes when `forward_host_credentials` is true.
The built-in Codex template uses `Dockerfile.codex`; set
`agent_provider.dockerfile` to build the agent from a custom Dockerfile
while keeping the bot-bottle sidecars in place.
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
+8 -7
View File
@@ -4,14 +4,15 @@
"env": {
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
},
"git": [
{
"Name": "foo",
"Upstream": "ssh://git@upstream.invalid/path.git",
"IdentityFile": "~/.cache/bot-bottle-demo/fake-key",
"KnownHostKey": "ssh-ed25519 AAAAEXAMPLE"
"git-gate": {
"repos": {
"foo": {
"url": "ssh://git@upstream.invalid/path.git",
"identity": "~/.cache/bot-bottle-demo/fake-key",
"host_key": "ssh-ed25519 AAAAEXAMPLE"
}
}
]
}
}
},
+187 -7
View File
@@ -7,14 +7,24 @@ command, default image, and prompt/auth behavior.
from __future__ import annotations
from dataclasses import dataclass
import json
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import Literal
from .codex_auth import codex_host_access_token, write_codex_dummy_auth_file
from .egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
PROVIDER_CLAUDE = "claude"
PROVIDER_CODEX = "codex"
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
# Hosts that egress injects the host ChatGPT bearer on when Codex
# forward_host_credentials is enabled. Pipelock must pass these through
# (no TLS MITM) or its header DLP blocks the injected JWT.
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
PromptMode = Literal["append_file", "read_prompt_file"]
@@ -24,14 +34,68 @@ class AgentProviderRuntime:
command: str
image: str
dockerfile: str
auth_role: str
placeholder_env: str
prompt_mode: PromptMode
bypass_args: tuple[str, ...]
resume_args: tuple[str, ...]
remote_control_args: tuple[str, ...]
@dataclass(frozen=True)
class AgentProvisionDir:
guest_path: str
mode: str = "700"
owner: str = "node:node"
@dataclass(frozen=True)
class AgentProvisionFile:
host_path: Path
guest_path: str
mode: str = "600"
owner: str = "node:node"
@dataclass(frozen=True)
class AgentProvisionCommand:
argv: tuple[str, ...]
error: str = ""
@dataclass(frozen=True)
class AgentProvisionPlan:
"""Provider-owned guest setup.
Backends interpret this plan with their own copy/exec primitives.
Provider-specific content stays here so future provider plugins can
return the same shape without adding backend-plan fields.
`egress_routes` are provider-declared EgressRoutes that backends
pass to `Egress.prepare` and `PipelockProxy.prepare`. This keeps
provider logic out of the egress and pipelock modules — they merge
provider routes generically without knowing the provider type.
`hidden_env_names` is the set of env var names the provider injected
as non-secret placeholders. `print_util.visible_agent_env_names` uses
this to suppress them from the preflight summary so operators don't
mistake them for real credentials.
"""
template: str
command: str
prompt_mode: PromptMode
image: str
dockerfile: str
guest_env: dict[str, str]
env_vars: dict[str, str] = field(default_factory=dict)
dirs: tuple[AgentProvisionDir, ...] = ()
files: tuple[AgentProvisionFile, ...] = ()
pre_copy: tuple[AgentProvisionCommand, ...] = ()
verify: tuple[AgentProvisionCommand, ...] = ()
egress_routes: tuple[EgressRoute, ...] = ()
hidden_env_names: frozenset[str] = field(default_factory=frozenset)
provisioned_env: dict[str, str] = field(default_factory=dict)
_REPO_ROOT = Path(__file__).resolve().parent.parent
@@ -41,8 +105,6 @@ _RUNTIMES = {
command="claude",
image="bot-bottle-claude:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
auth_role="claude_code_oauth",
placeholder_env="CLAUDE_CODE_OAUTH_TOKEN",
prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",),
@@ -53,8 +115,6 @@ _RUNTIMES = {
command="codex",
image="bot-bottle-codex:latest",
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
auth_role="codex_auth",
placeholder_env="OPENAI_API_KEY",
prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"),
@@ -67,6 +127,126 @@ def runtime_for(template: str) -> AgentProviderRuntime:
return _RUNTIMES[template]
def agent_provision_plan(
*,
template: str,
dockerfile: str,
state_dir: Path,
guest_home: str = "/home/node",
guest_env: dict[str, str] | None = None,
auth_token: str = "",
forward_host_credentials: bool = False,
host_env: dict[str, str] | None = None,
trusted_project_path: str = "",
) -> AgentProvisionPlan:
runtime = runtime_for(template)
resolved_guest_env = dict(guest_env or {})
trusted_path = trusted_project_path or guest_home
env_vars: dict[str, str] = {}
provisioned_env: dict[str, str] = {}
dirs: list[AgentProvisionDir] = []
files: list[AgentProvisionFile] = []
pre_copy: list[AgentProvisionCommand] = []
verify: list[AgentProvisionCommand] = []
egress_routes: list[EgressRoute] = []
hidden_env_names: frozenset[str] = frozenset()
if template == PROVIDER_CODEX:
env_vars["CODEX_CA_CERTIFICATE"] = "/etc/ssl/certs/ca-certificates.crt"
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
if forward_host_credentials:
env_vars["CODEX_HOME"] = auth_dir
dirs.append(AgentProvisionDir(auth_dir))
config_path = f"{auth_dir}/config.toml"
config_file = state_dir / "codex-config.toml"
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
config_file.write_text(
f'[projects."{toml_path}"]\n'
'trust_level = "trusted"\n'
)
config_file.chmod(0o600)
files.append(AgentProvisionFile(config_file, config_path))
for host in CODEX_HOST_CREDENTIAL_HOSTS:
egress_routes.append(EgressRoute(
host=host,
auth_scheme="Bearer" if forward_host_credentials else "",
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
tls_passthrough=True,
))
if forward_host_credentials:
_host_env = host_env or dict(os.environ)
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = codex_host_access_token(
_host_env,
)
auth_file = state_dir / "codex-auth.json"
write_codex_dummy_auth_file(auth_file, _host_env)
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
pre_copy.append(AgentProvisionCommand((
"find", auth_dir,
"-maxdepth", "1",
"-type", "f",
"(",
"-name", "*.sqlite",
"-o", "-name", "*.sqlite-*",
"-o", "-name", "*.codex-repair-*.bak",
")",
"-delete",
), "codex host credentials: could not reset runtime db files"))
verify.append(AgentProvisionCommand((
"runuser", "-u", "node", "--",
"env",
f"HOME={guest_home}",
f"CODEX_HOME={auth_dir}",
"codex", "login", "status",
), (
"codex host credentials: dummy auth was copied into the "
"guest, but Codex did not accept it"
)))
if template == PROVIDER_CLAUDE:
env_vars["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"] = "1"
env_vars["DISABLE_ERROR_REPORTING"] = "1"
claude_config = state_dir / "claude.json"
claude_projects = {
guest_home: {"hasTrustDialogAccepted": True},
}
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
claude_config.write_text(json.dumps({
"hasCompletedOnboarding": True,
"theme": "dark",
"bypassPermissionsModeAccepted": True,
"projects": claude_projects,
}, indent=2) + "\n")
claude_config.chmod(0o600)
files.append(AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"))
egress_routes.append(EgressRoute(
host="api.anthropic.com",
auth_scheme="Bearer" if auth_token else "",
token_ref=auth_token,
tls_passthrough=True,
))
if auth_token:
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
return AgentProvisionPlan(
template=template,
command=runtime.command,
prompt_mode=runtime.prompt_mode,
image=runtime.image,
dockerfile=dockerfile,
env_vars=env_vars,
guest_env=resolved_guest_env,
dirs=tuple(dirs),
files=tuple(files),
pre_copy=tuple(pre_copy),
verify=tuple(verify),
egress_routes=tuple(egress_routes),
hidden_env_names=hidden_env_names,
provisioned_env=provisioned_env,
)
def prompt_args(
prompt_mode: PromptMode,
prompt_path: str | None,
+75 -8
View File
@@ -32,15 +32,22 @@ manifest does not carry a backend field; the host picks.
from __future__ import annotations
import os
import sys
from abc import ABC, abstractmethod
from contextlib import AbstractContextManager
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Generic, Sequence, TypeVar
from ..log import die
from ..agent_provider import AgentProvisionPlan
from ..egress import EgressPlan
from ..git_gate import GitGatePlan
from ..log import die, info
from ..manifest import GitEntry, Manifest
from ..supervise import SupervisePlan
from ..util import expand_tilde
from ..workspace import WorkspacePlan
from .print_util import print_multi, visible_agent_env_names
from .util import host_skill_dir
@@ -65,15 +72,57 @@ class BottleSpec:
@dataclass(frozen=True)
class BottlePlan(ABC):
"""Base output of a backend's prepare step. Concrete subclasses
(e.g. DockerBottlePlan) add backend-specific resolved fields and
implement `print`."""
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
spec: BottleSpec
stage_dir: Path
git_gate_plan: GitGatePlan
egress_plan: EgressPlan
supervise_plan: SupervisePlan | None
agent_provision: AgentProvisionPlan
workspace_plan: WorkspacePlan
@abstractmethod
def print(self, *, remote_control: bool) -> None:
"""Render the y/N preflight summary to stderr."""
del remote_control
spec = self.spec
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
env_names = visible_agent_env_names(
sorted(
set(bottle.env.keys())
| set(self.agent_provision.guest_env.keys())
),
hidden_env_names=self.agent_provision.hidden_env_names,
)
print(file=sys.stderr)
info(f"agent : {spec.agent_name}")
info(f"provider : {self.agent_provision.template}")
print_multi("env ", env_names)
print_multi("skills ", list(agent.skills))
info(f"bottle : {agent.bottle}")
identity = manifest.git_identity_summary(spec.agent_name)
if identity:
info(f" git identity : {identity}")
git_lines = [
f"{u.name}{u.upstream_host}:{u.upstream_port}"
for u in self.git_gate_plan.upstreams
]
if git_lines:
print_multi(" git gate ", git_lines)
if self.egress_plan.routes:
egress_lines = []
for r in self.egress_plan.routes:
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
egress_lines.append(f"{r.host}{auth}")
print_multi(" egress ", egress_lines)
print(file=sys.stderr)
@dataclass(frozen=True)
@@ -273,7 +322,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
decide whether to add provider-specific prompt args to the agent's
argv.
Default orchestration: ca → prompt → skills → git →
Default orchestration: ca → prompt → skills → workspace → git →
supervise. CA install runs first so the agent's trust store
is rebuilt before anything inside the agent makes a TLS call.
Subclasses typically don't override this; they implement the
@@ -286,7 +335,9 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
intercepted without per-tool reconfiguration."""
self.provision_ca(plan, target)
prompt_path = self.provision_prompt(plan, target)
self.provision_provider_auth(plan, target)
self.provision_skills(plan, target)
self.provision_workspace(plan, target)
self.provision_git(plan, target)
self.provision_supervise(plan, target)
return prompt_path
@@ -300,6 +351,11 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
backend overrides to docker-cp the cert in and run
`update-ca-certificates`."""
def provision_provider_auth(self, plan: PlanT, target: str) -> None:
"""Install non-secret provider auth marker files into the agent
home when a provider needs them to select the right auth mode.
The default is no-op."""
@abstractmethod
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
"""Copy the prompt file into the running bottle. Returns the
@@ -312,6 +368,11 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
"""Copy the agent's named skills from the host into the
running bottle. No-op when the agent has no skills."""
def provision_workspace(self, plan: PlanT, target: str) -> None:
"""Copy the operator workspace into the running bottle when
the backend cannot bake it into the agent image. Default is
no-op for backends like Docker that handle this before launch."""
@abstractmethod
def provision_git(self, plan: PlanT, target: str) -> None:
"""Copy the host's cwd `.git` directory into the running
@@ -413,14 +474,20 @@ def enumerate_active_agents() -> list[ActiveAgent]:
"""All currently-running agents, across every available
backend. Used by CLI `list active` and the dashboard's agents
pane so neither has to know which backends exist. Skips
backends whose `is_available()` reports False. Ordered by
backend name, then by whatever each backend's
`enumerate_active` returns."""
backends whose `is_available()` reports False.
Sorted by `(started_at, slug)` so the list is stable across
dashboard refresh ticks — agents don't shift position while
the operator navigates with arrow keys. ISO 8601 timestamps
sort lexicographically in chronological order; `slug` is the
deterministic tiebreaker. Agents with missing metadata
(`started_at == ""`) sort first."""
out: list[ActiveAgent] = []
for name in known_backend_names():
if not has_backend(name):
continue
out.extend(_BACKENDS[name].enumerate_active())
out.sort(key=lambda a: (a.started_at, a.slug))
return out
+4
View File
@@ -29,6 +29,7 @@ from .bottle_plan import DockerBottlePlan
from .provision import ca as _ca
from .provision import git as _git
from .provision import prompt as _prompt
from .provision import provider_auth as _provider_auth
from .provision import skills as _skills
from .provision import supervise as _supervise_prov
@@ -62,6 +63,9 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
return _prompt.provision_prompt(plan, target)
def provision_provider_auth(self, plan: DockerBottlePlan, target: str) -> None:
_provider_auth.provision_provider_auth(plan, target)
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
_skills.provision_skills(plan, target)
+13 -62
View File
@@ -2,30 +2,25 @@
Carries the Docker-specific resolved fields produced by
DockerBottleBackend.prepare. The launch step consumes it without
further resolution; show_plan-style rendering is the `print` method.
further resolution; preflight rendering is inherited from BottlePlan.
"""
from __future__ import annotations
import sys
from dataclasses import dataclass, field
from pathlib import Path
from ...agent_provider import PromptMode
from ...egress import EgressPlan
from ...git_gate import GitGatePlan
from ...log import info
from ...pipelock import PipelockProxyPlan
from ...supervise import SupervisePlan
from .. import BottlePlan
from ..print_util import print_multi, visible_agent_env_names
@dataclass(frozen=True)
class DockerBottlePlan(BottlePlan):
"""Docker-specific resolved fields produced by
DockerBottleBackend.prepare. Inherits `spec` and `stage_dir` from
BottlePlan."""
DockerBottleBackend.prepare. Inherits `spec`, `stage_dir`,
`git_gate_plan`, `egress_plan`, `supervise_plan`, and
`agent_provision` from BottlePlan."""
slug: str
container_name: str
@@ -46,60 +41,16 @@ class DockerBottlePlan(BottlePlan):
forwarded_env: dict[str, str] = field(repr=False)
prompt_file: Path
proxy_plan: PipelockProxyPlan
git_gate_plan: GitGatePlan
egress_plan: EgressPlan
# None when bottle.supervise is False. PRD 0013 supervise sidecar
# is opt-in via the manifest's bottle.supervise field.
supervise_plan: SupervisePlan | None
use_runsc: bool
agent_command: str = "claude"
agent_prompt_mode: PromptMode = "append_file"
agent_provider_template: str = "claude"
def print(self, *, remote_control: bool) -> None:
"""Render the y/N preflight summary to stderr — compact form
intended to fit on screen without scrolling. The full
structured shape (image, container, runtime, etc.) lives on
this dataclass for tooling that wants to introspect it."""
del remote_control # not surfaced in the compact summary
spec = self.spec
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
# The agent sees the union of literal env names (rendered into
# --env-file) and forwarded env names (`-e NAME` with the
# value arriving via subprocess env). The forwarded set holds
# the OAuth token (CLAUDE_CODE_OAUTH_TOKEN) and any host-env
# interpolations from the manifest; egress holds
# upstream tokens in its own environ, so no token forwarding
# from the agent to the proxy is needed.
env_names = visible_agent_env_names(
sorted(set(bottle.env.keys()) | set(self.forwarded_env.keys())),
agent_provider_template=self.agent_provider_template,
)
@property
def agent_command(self) -> str:
return self.agent_provision.command
print(file=sys.stderr)
info(f"agent : {spec.agent_name}")
info(f"provider : {self.agent_provider_template}")
print_multi("env ", env_names)
print_multi("skills ", list(agent.skills))
info(f"bottle : {agent.bottle}")
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
identity = manifest.git_identity_summary(spec.agent_name)
if identity:
info(f" git identity : {identity}")
git_lines = [
f"{u.upstream_host}:{u.upstream_port}"
for u in self.git_gate_plan.upstreams
]
if git_lines:
print_multi(" git gate ", git_lines)
if self.egress_plan.routes:
egress_lines = []
for r in self.egress_plan.routes:
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
egress_lines.append(f"{r.host}{auth}")
print_multi(" egress ", egress_lines)
print(file=sys.stderr)
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@@ -105,6 +105,10 @@ class BottleMetadata:
# written before chunk 3 (resume / inspect should fall back to
# deriving from identity in that case).
compose_project: str = ""
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
# for state dirs written before PRD 0040; callers default to "docker"
# for backward compatibility.
backend: str = ""
def metadata_path(identity: str) -> Path:
@@ -138,6 +142,7 @@ def read_metadata(identity: str) -> BottleMetadata | None:
copy_cwd=bool(raw.get("copy_cwd", False)),
started_at=str(raw.get("started_at", "")),
compose_project=str(raw.get("compose_project", "")),
backend=str(raw.get("backend", "")),
)
+3 -6
View File
@@ -49,7 +49,7 @@ from ...egress import (
EGRESS_HOSTNAME,
EGRESS_ROUTES_IN_CONTAINER,
)
from ...git_gate import GIT_GATE_HOSTNAME, git_gate_aggregate_extra_hosts
from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn
from ...pipelock import PIPELOCK_HOSTNAME
from ...supervise import (
@@ -198,7 +198,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
env.append(token_env)
# --- git-gate ----------------------------------------------------
extra_hosts: list[str] = []
gp = plan.git_gate_plan
if gp.upstreams:
volumes += [
@@ -217,8 +216,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
u.known_hosts_file,
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
))
extra_map = git_gate_aggregate_extra_hosts(gp.upstreams)
extra_hosts = [f"{host}:{ip}" for host, ip in sorted(extra_map.items())]
# --- supervise ---------------------------------------------------
sp = plan.supervise_plan
@@ -261,8 +258,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"environment": env,
"volumes": volumes,
}
if extra_hosts:
service["extra_hosts"] = extra_hosts
return service
@@ -286,6 +281,8 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
# Forwarded vars (OAuth token, manifest host-interpolations):
# bare name → inherits from compose-up process env, value
# never lands on argv or in the compose file.
+11 -11
View File
@@ -43,7 +43,7 @@ from pathlib import Path
from typing import Callable, Generator
from ...egress import egress_resolve_token_values
from ...log import info
from ...log import info, warn
from . import network as network_mod
from . import util as docker_mod
from .bottle import DockerBottle
@@ -87,10 +87,11 @@ def launch(
def teardown() -> None:
try:
stack.close()
except BaseException:
# Teardown must not raise; swallow so the caller's
# __exit__ path can still propagate the original error.
pass
except BaseException as exc:
warn(
f"teardown failed for container {plan.container_name}"
f" (compose-down): {exc!r}"
)
try:
# Step 1: agent image build. Sidecar images get built lazily by
@@ -101,7 +102,7 @@ def launch(
)
if plan.derived_image:
docker_mod.build_image_with_cwd(
plan.derived_image, plan.image, plan.spec.user_cwd
plan.derived_image, plan.image, plan.workspace_plan
)
# Networks: compose-managed. The names are derived
@@ -176,11 +177,10 @@ def launch(
# Step 7: compose up. Token values + the OAuth placeholder
# flow through subprocess env; the compose file holds only
# bare names for the secret-carrying entries.
token_values: dict[str, str] = {}
if plan.egress_plan.routes:
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, dict(os.environ),
)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
compose_env: dict[str, str] = {
**os.environ,
**plan.forwarded_env,
+41 -36
View File
@@ -12,15 +12,17 @@ from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import runtime_for
from ...agent_provider import agent_provision_plan, runtime_for
from ...egress import Egress
from ...env import ResolvedEnv, resolve_env
from ...git_gate import GitGate
from ...log import die
from ...pipelock import PipelockProxy
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .. import BottleSpec
from . import util as docker_mod
from .bottle_plan import DockerBottlePlan
@@ -61,6 +63,8 @@ def resolve_plan(
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
# mints a random-suffixed identity (so parallel runs of the same
@@ -78,6 +82,7 @@ def resolve_plan(
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
compose_project=f"bot-bottle-{slug}",
backend="docker",
))
# Clear any leftover preserve marker from a prior capability-block
# so this fresh launch can be cleaned up at session-end unless
@@ -158,17 +163,45 @@ def resolve_plan(
prompt_file.write_text("")
prompt_file.chmod(0o600)
pipelock_dir = pipelock_state_dir(slug)
pipelock_dir.mkdir(parents=True, exist_ok=True)
proxy_plan = proxy.prepare(bottle, slug, pipelock_dir)
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
resolved = resolve_env(manifest, spec.agent_name)
# Everything that should reach the bottle by-name (so its value
# never lands on argv or in env_file) goes into one dict. Nothing
# mutates the host os.environ.
forwarded_env: dict[str, str] = dict(resolved.forwarded)
_write_env_file(resolved, env_file)
prompt_file.write_text(agent.prompt)
use_runsc = docker_mod.runsc_available()
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
)
guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=guest_env)
pipelock_dir = pipelock_state_dir(slug)
pipelock_dir.mkdir(parents=True, exist_ok=True)
proxy_plan = proxy.prepare(
bottle, slug, pipelock_dir, agent_provision.egress_routes,
)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = egress.prepare(bottle, slug, egress_dir)
egress_plan = egress.prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
@@ -196,33 +229,6 @@ def resolve_plan(
slug, supervise_dir,
dockerfile_content=dockerfile_content,
)
resolved = resolve_env(manifest, spec.agent_name)
# Everything that should reach the bottle by-name (so its value
# never lands on argv or in env_file) goes into one dict. Nothing
# mutates the host os.environ.
forwarded_env: dict[str, str] = dict(resolved.forwarded)
# When the bottle declares an egress route with the
# `claude_code_oauth` role marker, claude-code's outbound
# Authorization gets stripped + re-injected by egress. The
# agent's environ still needs *something* claude-code recognises
# as a credential or it refuses to start; ship a non-secret
# placeholder. The placeholder isn't any real token value, so
# leaking it would tell an attacker only that egress is in
# front. Manifest validation enforces singleton on this role.
has_provider_auth = any(
provider_runtime.auth_role in r.roles for r in egress_plan.routes
)
if has_provider_auth:
forwarded_env[provider_runtime.placeholder_env] = "egress-placeholder"
if provider.template == "claude" and has_provider_auth:
# Belt-and-braces: turn off telemetry endpoints (statsig,
# error reporting) that egress can't gate by auth.
forwarded_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
forwarded_env.setdefault("DISABLE_ERROR_REPORTING", "1")
_write_env_file(resolved, env_file)
prompt_file.write_text(agent.prompt)
use_runsc = docker_mod.runsc_available()
return DockerBottlePlan(
spec=spec,
@@ -242,9 +248,8 @@ def resolve_plan(
egress_plan=egress_plan,
supervise_plan=supervise_plan,
use_runsc=use_runsc,
agent_command=provider_runtime.command,
agent_prompt_mode=provider_runtime.prompt_mode,
agent_provider_template=provider.template,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
+8 -6
View File
@@ -3,7 +3,7 @@
Three concerns, all about git in the agent:
1. If --cwd was passed AND the host cwd has a .git, copy that .git
into /home/node/workspace/.git so the agent operates on the
into the planned guest workspace so the agent operates on the
user's repo.
2. If the bottle declares `git` entries (PRD 0008), write a
~/.gitconfig with insteadOf rules so every git operation
@@ -20,7 +20,6 @@ from __future__ import annotations
import os
import subprocess
from pathlib import Path
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
from ....log import info
@@ -40,19 +39,22 @@ def _provision_cwd_git(plan: DockerBottlePlan, target: str) -> None:
"""If --cwd was set and the host cwd has a .git directory, copy
it into /home/node/workspace/.git and fix ownership. No-op
otherwise."""
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
return
container = target
info(f"copying {plan.spec.user_cwd}/.git -> {container}:/home/node/workspace/.git")
guest_workspace_git = f"{workspace.guest_path}/.git"
host_git = str(workspace.host_path / ".git")
info(f"copying {host_git} -> {container}:{guest_workspace_git}")
subprocess.run(
["docker", "cp", f"{plan.spec.user_cwd}/.git", f"{container}:/home/node/workspace/.git"],
["docker", "cp", host_git, f"{container}:{guest_workspace_git}"],
stdout=subprocess.DEVNULL,
check=True,
)
subprocess.run(
[
"docker", "exec", "-u", "0", container,
"chown", "-R", "node:node", "/home/node/workspace/.git",
"chown", "-R", workspace.owner, guest_workspace_git,
],
stdout=subprocess.DEVNULL,
check=True,
@@ -0,0 +1,36 @@
"""Provision non-secret provider auth markers into a Docker bottle."""
from __future__ import annotations
import subprocess
from ..bottle_plan import DockerBottlePlan
def provision_provider_auth(plan: DockerBottlePlan, target: str) -> None:
"""Apply provider-owned guest setup through Docker primitives."""
provision = plan.agent_provision
for d in provision.dirs:
_exec(target, ["mkdir", "-p", d.guest_path])
_exec(target, ["chown", d.owner, d.guest_path])
_exec(target, ["chmod", d.mode, d.guest_path])
for command in provision.pre_copy:
_exec(target, list(command.argv))
for f in provision.files:
subprocess.run(
["docker", "cp", str(f.host_path), f"{target}:{f.guest_path}"],
stdout=subprocess.DEVNULL,
check=True,
)
_exec(target, ["chown", f.owner, f.guest_path])
_exec(target, ["chmod", f.mode, f.guest_path])
for command in provision.verify:
_exec(target, list(command.argv))
def _exec(target: str, argv: list[str]) -> None:
subprocess.run(
["docker", "exec", "-u", "0", target, *argv],
stdout=subprocess.DEVNULL,
check=True,
)
+31 -25
View File
@@ -7,9 +7,11 @@ from __future__ import annotations
import re
import shutil
import subprocess
import tempfile
from typing import Iterable, Iterator
from ...log import die, info
from ...workspace import WorkspacePlan
# Cap on the suffix the container-name conflict logic will try before
@@ -116,35 +118,39 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
subprocess.run(args, check=True)
_TRUST_DIALOG_NODE_SCRIPT = (
'const fs=require("fs"),p=process.env.HOME+"/.claude.json",'
'c=JSON.parse(fs.readFileSync(p,"utf8"));'
'c.projects=c.projects||{};'
'c.projects[process.env.HOME+"/workspace"]={hasTrustDialogAccepted:true};'
'fs.writeFileSync(p,JSON.stringify(c,null,2));'
)
def build_image_with_cwd(derived: str, base: str, cwd: str) -> None:
"""Build a thin derived image that copies <cwd> into
/home/node/workspace and adds a trust-dialog entry for it."""
def build_image_with_cwd(
derived: str,
base: str,
workspace: WorkspacePlan,
) -> None:
"""Build a thin derived image that copies the workspace into
the plan's guest path and sets the plan's workdir."""
import os
cwd = str(workspace.host_path)
if not os.path.isdir(cwd):
die(f"cwd not found at {cwd}")
info(f"building image {derived} from {base} with {cwd} -> /home/node/workspace")
dockerfile = (
f"FROM {base}\n"
f"COPY --chown=node:node . /home/node/workspace\n"
f"RUN node -e '{_TRUST_DIALOG_NODE_SCRIPT}'\n"
f"WORKDIR /home/node/workspace\n"
)
subprocess.run(
["docker", "build", "-t", derived, "-f", "-", cwd],
input=dockerfile,
text=True,
check=True,
)
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
context_dir = os.path.join(tmp, "context")
staged_workspace = os.path.join(context_dir, "workspace")
shutil.copytree(
cwd,
staged_workspace,
symlinks=True,
ignore=shutil.ignore_patterns(".git"),
)
dockerfile = (
f"FROM {base}\n"
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
f"WORKDIR {workspace.workdir}\n"
)
subprocess.run(
["docker", "build", "-t", derived, "-f", "-", context_dir],
input=dockerfile,
text=True,
check=True,
)
def image_id(ref: str) -> str:
+6 -10
View File
@@ -9,7 +9,6 @@ from __future__ import annotations
from typing import Sequence
from ..agent_provider import runtime_for
from ..log import info
@@ -30,16 +29,13 @@ def print_multi(label: str, values: Sequence[str]) -> None:
def visible_agent_env_names(
env_names: Sequence[str], *, agent_provider_template: str,
env_names: Sequence[str], *, hidden_env_names: frozenset[str],
) -> list[str]:
"""Env names worth showing in launch summaries.
Provider auth placeholders (`OPENAI_API_KEY`,
`CLAUDE_CODE_OAUTH_TOKEN`) are implementation details: they are
non-secret dummy values that satisfy the provider CLI while egress
injects the real upstream Authorization header. Showing them in
preflight makes the operator think a real key is entering the
agent, so hide only that provider-owned placeholder.
Provider-injected placeholder env vars are implementation details:
they are non-secret dummy values that satisfy provider CLIs while
egress injects the real Authorization header. The plan's
`hidden_env_names` carries exactly which names to suppress.
"""
hidden = {runtime_for(agent_provider_template).placeholder_env}
return sorted({name for name in env_names if name not in hidden})
return sorted({name for name in env_names if name and name not in hidden_env_names})
@@ -19,8 +19,10 @@ from .bottle_plan import SmolmachinesBottlePlan
from .provision import ca as _ca
from .provision import git as _git
from .provision import prompt as _prompt
from .provision import provider_auth as _provider_auth
from .provision import skills as _skills
from .provision import supervise as _supervise
from .provision import workspace as _workspace
class SmolmachinesBottleBackend(
@@ -61,11 +63,21 @@ class SmolmachinesBottleBackend(
) -> str | None:
return _prompt.provision_prompt(plan, target)
def provision_provider_auth(
self, plan: SmolmachinesBottlePlan, target: str
) -> None:
_provider_auth.provision_provider_auth(plan, target)
def provision_skills(
self, plan: SmolmachinesBottlePlan, target: str
) -> None:
_skills.provision_skills(plan, target)
def provision_workspace(
self, plan: SmolmachinesBottlePlan, target: str
) -> None:
_workspace.provision_workspace(plan, target)
def provision_git(
self, plan: SmolmachinesBottlePlan, target: str
) -> None:
+15 -24
View File
@@ -45,19 +45,11 @@ _HOME_FOR = {
}
def _env_flags_for(user: str) -> list[str]:
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
home = _HOME_FOR.get(user, f"/home/{user}")
return ["-e", f"HOME={home}", "-e", f"USER={user}"]
def _guest_env_flags(env: Mapping[str, str]) -> list[str]:
"""Render `{K: V}` into a flat `-e K=V` argv slice for
`smolvm machine exec`. `smolvm machine create -e` set env
on PID 1 but it doesn't propagate to fresh exec process
trees, so we have to re-pass them every call."""
out: list[str] = []
out = [f"HOME={home}", f"USER={user}"]
for k, v in env.items():
out += ["-e", f"{k}={v}"]
out.append(f"{k}={v}")
return out
@@ -98,9 +90,8 @@ class SmolmachinesBottle(Bottle):
flags = ["smolvm", "machine", "exec", "--name", self.name]
if tty:
flags += ["-i", "-t"]
flags += _env_flags_for("node")
flags += _guest_env_flags(self._guest_env)
agent_tail = [self.agent_command]
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
self.agent_command]
provider_prompt_args = prompt_args(
self._agent_prompt_mode, self._prompt_path, argv=argv,
)
@@ -148,16 +139,16 @@ class SmolmachinesBottle(Bottle):
on both backends. Pass `user="root"` for tests that need
root.
`runuser -u <user> -- /bin/sh -c <script>` switches UID
without invoking a login shell; HOME / USER are set via
`smolvm -e` (see `_env_flags_for`)."""
argv = (
_env_flags_for(user)
+ _guest_env_flags(self._guest_env)
+ ["--", "runuser", "-u", user, "--", "/bin/sh", "-c", script]
)
# _smolvm.machine_exec expects argv (the bit after `--`);
# the -e flags go before, so call smolvm directly.
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
without invoking a login shell, then sets HOME / USER and the
bottle env in the child process."""
argv = [
"--", "runuser", "-u", user, "--",
"env", *_env_assignments_for(user, self._guest_env),
"/bin/sh", "-c", script,
]
# Call smolvm directly because this path needs the host-side
# subprocess capture shape used by the Docker backend.
r = subprocess.run(
["smolvm", "machine", "exec", "--name", self.name] + argv,
capture_output=True, text=True, check=False,
+16 -50
View File
@@ -8,25 +8,20 @@ in chunk 4."""
from __future__ import annotations
import sys
from dataclasses import dataclass
from pathlib import Path
from ...agent_provider import PromptMode
from ...egress import EgressPlan
from ...git_gate import GitGatePlan
from ...log import info
from ...pipelock import PipelockProxyPlan
from ...supervise import SupervisePlan
from .. import BottlePlan
from ..print_util import print_multi, visible_agent_env_names
@dataclass(frozen=True)
class SmolmachinesBottlePlan(BottlePlan):
"""Resolved fields the launch step needs to bring up the bottle.
Inherits `spec` and `stage_dir` from BottlePlan."""
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
`supervise_plan`, and `agent_provision` from BottlePlan."""
slug: str
# Per-bottle docker subnet for the sidecar bundle container.
@@ -68,7 +63,7 @@ class SmolmachinesBottlePlan(BottlePlan):
# empty when the agent has no prompt — claude-code reads it
# via --append-system-prompt-file only when non-empty.
prompt_file: Path
# Inner Plans for the four bundle daemons. The same shape the
# Inner Plans for the sidecar bundle daemons. The same shape the
# docker backend uses — same `.prepare()` calls produced
# them — but our launch step doesn't populate the
# docker-specific network fields (internal_network,
@@ -77,11 +72,6 @@ class SmolmachinesBottlePlan(BottlePlan):
# per-bottle bridge with a pinned IP. The unused fields stay
# at their dataclass defaults.
proxy_plan: PipelockProxyPlan
git_gate_plan: GitGatePlan
egress_plan: EgressPlan
# None when bottle.supervise is False, matching the docker
# backend's convention.
supervise_plan: SupervisePlan | None
# Agent-side endpoints. On Docker Desktop the docker bridge
# IPs aren't reachable from the smolvm guest (TSI uses macOS
# networking; docker container IPs live in the daemon's VM),
@@ -93,43 +83,19 @@ class SmolmachinesBottlePlan(BottlePlan):
agent_proxy_url: str = ""
agent_git_gate_host: str = ""
agent_supervise_url: str = ""
agent_command: str = "claude"
agent_prompt_mode: PromptMode = "append_file"
agent_provider_template: str = "claude"
agent_dockerfile_path: str = ""
def print(self, *, remote_control: bool) -> None:
"""Compact y/N preflight. Same shape as the Docker
backend's so operators see one format across backends."""
del remote_control # not surfaced in the compact summary
spec = self.spec
manifest = spec.manifest
agent = manifest.agents[spec.agent_name]
bottle = manifest.bottle_for(spec.agent_name)
@property
def agent_command(self) -> str:
return self.agent_provision.command
env_names = visible_agent_env_names(
sorted(bottle.env.keys()),
agent_provider_template=self.agent_provider_template,
)
upstreams = [
f"{g.Name}{g.Upstream}" for g in bottle.git
]
# Use the resolved egress_plan (lowercase `host` on the
# plan-level EgressRoute) rather than `bottle.egress.routes`,
# which is the manifest's capitalized-attr form.
routes = [r.host for r in self.egress_plan.routes]
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
print(file=sys.stderr)
info(f"agent : {spec.agent_name}")
info(f"provider : {self.agent_provider_template}")
print_multi("env ", env_names)
print_multi("skills ", list(agent.skills))
info(f"bottle : {agent.bottle}")
identity = manifest.git_identity_summary(spec.agent_name)
if identity:
info(f" git identity : {identity}")
if upstreams:
print_multi(" git gate ", upstreams)
if routes:
print_multi(" egress ", routes)
print(file=sys.stderr)
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@property
def agent_dockerfile_path(self) -> str:
return self.agent_provision.dockerfile
+199 -203
View File
@@ -21,12 +21,14 @@ from __future__ import annotations
import dataclasses
import os
import time
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_resolve_token_values,
)
from ...pipelock import (
PIPELOCK_CA_CERT_IN_CONTAINER,
PIPELOCK_CA_KEY_IN_CONTAINER,
@@ -45,7 +47,6 @@ from ..docker.git_gate import (
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
GIT_GATE_PORT as _GIT_GATE_PORT,
)
from ..docker.pipelock import (
BUNDLE_LOCAL_PIPELOCK_URL,
@@ -77,6 +78,7 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
# them up post-start. Pipelock's port is an env-overridable string
# in docker.pipelock; coerce to int here.
_PIPELOCK_PORT = int(_PIPELOCK_PORT_STR)
_GIT_HTTP_PORT = 9420
_SUPERVISE_PORT = SUPERVISE_PORT
@@ -91,199 +93,23 @@ def launch(
via the ExitStack."""
stack = ExitStack()
try:
# 1. Reserve a loopback alias for this bottle. macOS only
# routes 127.0.0.1 by default; the per-bottle alias is
# what bundles the docker port-publishes and TSI allowlist
# against, so this bottle can't reach other bottles' (or
# other host services') ports on the loopback. Lazy
# sudo-driven on first use per boot. No-op on Linux.
_loopback.ensure_pool()
loopback_ip = _loopback.allocate(plan.slug)
loopback_ip, network = _allocate_resources(plan, stack)
plan = _mint_certs(plan)
plan = _start_bundle(plan, network, loopback_ip, stack)
plan = _discover_urls(plan, loopback_ip)
# 2. Per-bottle docker bridge.
network = _bundle.bundle_network_name(plan.slug)
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
stack.callback(_bundle.remove_bundle_network, network)
# 2. Mint per-bottle CAs and update the inner Plans with
# their launch-time paths. pipelock always runs in the
# bundle; egress's CA is only minted when the bottle
# declares routes (otherwise egress runs idle without
# MITM and the CA files would be unused).
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
proxy_plan = dataclasses.replace(
plan.proxy_plan,
ca_cert_host_path=ca_cert_host,
ca_key_host_path=ca_key_host,
)
egress_plan = plan.egress_plan
if egress_plan.routes:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
plan.egress_plan.routes_path.parent,
)
egress_plan = dataclasses.replace(
egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
pipelock_ca_host_path=ca_cert_host,
# On smolmachines, egress's upstream is pipelock
# on the bundle's localhost — they're in the same
# container's network namespace.
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
)
plan = dataclasses.replace(
plan, proxy_plan=proxy_plan, egress_plan=egress_plan,
)
# 3. Build the BundleLaunchSpec from the (now-resolved)
# inner Plans: daemon subset, env, bind-mounts, and the
# loopback alias to bind published ports against. The
# spec's ports_to_publish list expands depending on which
# daemons the agent needs to reach from the smolvm guest.
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
token_env = _resolve_token_env(plan, os.environ)
_bundle.ensure_bundle_image(bundle_spec.image)
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
stack.callback(_bundle.stop_bundle, plan.slug)
# 4. Discover the host-side ports docker assigned for the
# bundle's published container ports, and bind the
# agent's URLs to `<loopback_ip>:<host port>`. Docker
# container IPs (192.168.x.x in the daemon's bridge)
# aren't reachable from the smolvm guest on macOS — TSI
# uses macOS networking, and macOS sees the daemon's
# bridge via the published-port loopback forward only.
#
# Proxy hop order matches the docker backend: when the
# bottle declares egress routes, the agent's first hop is
# egress (for token injection), then pipelock. Without
# routes, the agent dials pipelock directly. Whichever
# one is "agent-facing" is the daemon whose port we
# publish on host loopback; the other stays bundle-
# internal as the upstream proxy.
if plan.egress_plan.routes:
agent_facing_port = _EGRESS_PORT
else:
agent_facing_port = _PIPELOCK_PORT
agent_facing_host_port = _bundle.bundle_host_port(
plan.slug, agent_facing_port, host_ip=loopback_ip,
)
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
agent_git_gate_host = ""
if plan.git_gate_plan.upstreams:
git_gate_host_port = _bundle.bundle_host_port(
plan.slug, _GIT_GATE_PORT, host_ip=loopback_ip,
)
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
agent_supervise_url = ""
if plan.supervise_plan is not None:
supervise_host_port = _bundle.bundle_host_port(
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
)
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
# Stamp the URLs onto the plan + guest_env. provision_git
# and provision_supervise read the plan fields; the agent
# reads guest_env on every exec_agent.
#
# NO_PROXY has to include the per-bottle loopback alias —
# otherwise claude's HTTPS_PROXY catches direct calls to
# the supervise URL (`http://<alias>:<port>/`) and proxies
# them through egress, which has no route for the alias
# and rejects with "Failed to connect". The git-gate URL
# uses git://, not affected by HTTP_PROXY, so the alias
# only has to be in NO_PROXY for the MCP / supervise
# path. Append rather than overwrite so prepare.py's
# `localhost,127.0.0.1` baseline stays in place.
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
guest_env = {
**plan.guest_env,
"HTTPS_PROXY": agent_proxy_url,
"HTTP_PROXY": agent_proxy_url,
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
}
if agent_git_gate_host:
guest_env["GIT_GATE_URL"] = f"git://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
plan = dataclasses.replace(
plan,
guest_env=guest_env,
agent_proxy_url=agent_proxy_url,
agent_git_gate_host=agent_git_gate_host,
agent_supervise_url=agent_supervise_url,
)
# 5. Build the agent image and pack it into a
# `.smolmachine` artifact (or hit the per-Dockerfile-digest
# cache). Runs here, not in prepare, so the docker-build
# output doesn't garble the dashboard's preflight modal:
# both the curses-endwin path and the tmux pane-routing
# path redirect stderr around `launch` already.
# Build the agent image and pack it into a `.smolmachine`
# artifact (or hit the per-Dockerfile-digest cache). Runs
# here, not in prepare, so the docker-build output doesn't
# garble the dashboard's preflight modal.
agent_from_path = _ensure_smolmachine(
plan.agent_image_ref,
dockerfile=plan.agent_dockerfile_path,
)
# smolvm VM. --from carries the pre-packed .smolmachine
# artifact; --allow-cidr + -e carry the per-bottle TSI
# allowlist + env. The allowlist is the per-bottle
# loopback alias — narrowing it to one /32 keeps the
# agent from reaching other host loopback services or
# other bottles' published ports. Smolfile isn't usable
# here — smolvm 0.8.0 makes `--from` and `--smolfile`
# mutually exclusive.
_smolvm.machine_create(
plan.machine_name,
from_path=agent_from_path,
allow_cidrs=[f"{loopback_ip}/32"],
env=plan.guest_env,
)
stack.callback(_smolvm.machine_delete, plan.machine_name)
# Workaround smolvm 0.8.0: `--allow-cidr` is silently
# dropped when combined with `--from`. Patch the persisted
# state DB to set the allowlist before start so the booted
# VM's TSI actually enforces. See loopback_alias's module
# docstring for the investigation that led here.
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
_smolvm.machine_start(plan.machine_name)
stack.callback(_smolvm.machine_stop, plan.machine_name)
_launch_vm(plan, agent_from_path, loopback_ip, stack)
_init_vm(plan)
# 6. Repair filesystem ownership + perms that smolvm's
# pack process remapped to the host invoker's uid (501
# on macOS) rather than preserving the image's expected
# ownership.
#
# - /home/node → node:node so the node user can write
# its own dotfiles (claude appendFileSync on
# ~/.claude.json otherwise bails with ENOENT/EPERM
# and the TUI hangs without surfacing the error).
# - /tmp + /var/tmp → root:root mode 1777 so non-root
# processes can create their per-uid scratch dirs
# (claude-code creates /tmp/claude-<uid>/ as soon as
# it spawns a Bash tool call).
#
# All folded into one sh -c so we only pay one
# machine_exec round trip — back-to-back exec calls
# right after machine_start hit a SIGKILL race in
# libkrun's exec channel (see provision_ca for the
# other half of this same workaround).
_smolvm.machine_exec(plan.machine_name, [
"sh", "-c",
"chown -R node:node /home/node && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp",
])
# Wait briefly for the VM to settle. Back-to-back smolvm
# machine_exec calls immediately after machine_start
# occasionally SIGKILL the in-VM child at ~100ms (looks
# like a VM warm-up race in libkrun's exec channel).
# 1.5s is empirically enough to dodge it; provisioning
# already takes seconds so the wait is amortized.
time.sleep(1.5)
# 7. Provision (CA / prompt / skills / git / supervise).
prompt_path = provision(plan, plan.machine_name)
yield SmolmachinesBottle(
@@ -297,6 +123,180 @@ def launch(
stack.close()
def _allocate_resources(
plan: SmolmachinesBottlePlan,
stack: ExitStack,
) -> tuple[str, str]:
"""Reserve a loopback alias and create the per-bottle docker bridge.
macOS only routes 127.0.0.1 by default; the per-bottle alias
scopes TSI's allowlist to this bottle's published ports so the
agent can't reach other bottles' or host services' ports on
loopback. No-op on Linux."""
_loopback.ensure_pool()
loopback_ip = _loopback.allocate(plan.slug)
network = _bundle.bundle_network_name(plan.slug)
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
stack.callback(_bundle.remove_bundle_network, network)
return loopback_ip, network
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
"""Mint per-bottle CAs and return the plan with CA paths filled.
Pipelock always runs in the bundle. Egress's CA is only minted
when the bottle declares routes — otherwise egress runs idle
without MITM and the CA files would be unused."""
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
proxy_plan = dataclasses.replace(
plan.proxy_plan,
ca_cert_host_path=ca_cert_host,
ca_key_host_path=ca_key_host,
)
egress_plan = plan.egress_plan
if egress_plan.routes:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
plan.egress_plan.routes_path.parent,
)
egress_plan = dataclasses.replace(
egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
pipelock_ca_host_path=ca_cert_host,
# On smolmachines, egress's upstream is pipelock on the
# bundle's localhost — they're in the same container's
# network namespace.
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
)
return dataclasses.replace(plan, proxy_plan=proxy_plan, egress_plan=egress_plan)
def _start_bundle(
plan: SmolmachinesBottlePlan,
network: str,
loopback_ip: str,
stack: ExitStack,
) -> SmolmachinesBottlePlan:
"""Build the BundleLaunchSpec, resolve token env, start the
sidecar bundle container, and register teardown."""
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
token_env = _resolve_token_env(plan, dict(os.environ))
_bundle.ensure_bundle_image(bundle_spec.image)
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
stack.callback(_bundle.stop_bundle, plan.slug)
return plan
def _discover_urls(
plan: SmolmachinesBottlePlan,
loopback_ip: str,
) -> SmolmachinesBottlePlan:
"""Discover host-side ports for published container ports and
return the plan with URLs + guest_env stamped in.
Docker container IPs (192.168.x.x in the daemon's bridge)
aren't reachable from the smolvm guest on macOS — TSI uses
macOS networking, and macOS sees the daemon's bridge via the
published-port loopback forward only.
Proxy hop order: when the bottle declares egress routes, the
agent's first hop is egress (for token injection), then
pipelock. Without routes, the agent dials pipelock directly.
NO_PROXY includes the per-bottle loopback alias so the
supervise + git-gate URLs bypass HTTPS_PROXY."""
if plan.egress_plan.routes:
agent_facing_port = _EGRESS_PORT
else:
agent_facing_port = _PIPELOCK_PORT
agent_facing_host_port = _bundle.bundle_host_port(
plan.slug, agent_facing_port, host_ip=loopback_ip,
)
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
agent_git_gate_host = ""
if plan.git_gate_plan.upstreams:
git_gate_host_port = _bundle.bundle_host_port(
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
)
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
agent_supervise_url = ""
if plan.supervise_plan is not None:
supervise_host_port = _bundle.bundle_host_port(
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
)
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
guest_env = {
**plan.guest_env,
"HTTPS_PROXY": agent_proxy_url,
"HTTP_PROXY": agent_proxy_url,
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
}
if agent_git_gate_host:
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
return dataclasses.replace(
plan,
guest_env=guest_env,
agent_proxy_url=agent_proxy_url,
agent_git_gate_host=agent_git_gate_host,
agent_supervise_url=agent_supervise_url,
)
def _launch_vm(
plan: SmolmachinesBottlePlan,
agent_from_path: Path,
loopback_ip: str,
stack: ExitStack,
) -> None:
"""Create, patch, and start the smolvm VM; register teardown.
--allow-cidr is the per-bottle loopback alias so the guest can
only reach this bottle's bundle ports. force_allowlist patches
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
and --smolfile mutually exclusive."""
_smolvm.machine_create(
plan.machine_name,
from_path=agent_from_path,
allow_cidrs=[f"{loopback_ip}/32"],
env=plan.guest_env,
)
stack.callback(_smolvm.machine_delete, plan.machine_name)
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
# when combined with `--from`. Patch the persisted state DB
# before start so the booted VM's TSI actually enforces.
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
_smolvm.machine_start(plan.machine_name)
stack.callback(_smolvm.machine_stop, plan.machine_name)
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
"""Repair filesystem ownership and wait for exec channel readiness.
Ownership repair: smolvm's pack process remaps files to the host
invoker's uid (501 on macOS). /home/node must be node:node so
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
mode 1777 so non-root processes can create per-uid scratch dirs.
All folded into one sh -c to avoid back-to-back exec calls
immediately after machine_start (libkrun exec-channel race).
wait_exec_ready polls until the exec channel is ready for the
subsequent provision calls, replacing the empirical sleep."""
_smolvm.machine_exec(plan.machine_name, [
"sh", "-c",
"chown -R node:node /home/node && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp",
])
_smolvm.wait_exec_ready(plan.machine_name)
def _bundle_launch_spec(
plan: SmolmachinesBottlePlan, network: str, loopback_ip: str,
) -> _bundle.BundleLaunchSpec:
@@ -305,10 +305,10 @@ def _bundle_launch_spec(
Daemons in the CSV:
- egress + pipelock are always present (pipelock is the
agent's first hop; egress is its upstream).
- git-gate is conditional on plan.git_gate_plan.upstreams.
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
- supervise is conditional on plan.supervise_plan.
Env + volumes are the union of the four daemons' needs, with
Env + volumes are the union of the sidecar daemons' needs, with
daemon-private values only (HTTPS_PROXY is scoped to the
egress process by egress_entrypoint.sh — see PRD 0024's bundle
bind-address PR)."""
@@ -320,10 +320,9 @@ def _bundle_launch_spec(
# is "agent-facing" gets its port published on the host
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
# other stays bundle-internal. The bundle is NOT reachable by
# bridge IP from the smolvm guest, so the
# PRD-0023-chunk-3 EGRESS_LISTEN_HOST=127.0.0.1 mitigation
# isn't needed: the agent can only dial whatever daemon's
# host port we publish, period.
# bridge IP from the smolvm guest on macOS — TSI uses macOS
# networking, and macOS sees the daemon's bridge via the
# published-port loopback forward only.
# --- pipelock ---------------------------------------------
pp = plan.proxy_plan
@@ -350,10 +349,9 @@ def _bundle_launch_spec(
env.append(token_env)
# --- git-gate ---------------------------------------------
extra_hosts: list[str] = []
gp = plan.git_gate_plan
if gp.upstreams:
daemons.append("git-gate")
daemons += ["git-gate", "git-http"]
volumes += [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
@@ -395,7 +393,7 @@ def _bundle_launch_spec(
else:
ports_to_publish = [_PIPELOCK_PORT]
if gp.upstreams:
ports_to_publish.append(_GIT_GATE_PORT)
ports_to_publish.append(_GIT_HTTP_PORT)
if sp is not None:
ports_to_publish.append(_SUPERVISE_PORT)
@@ -414,15 +412,13 @@ def _bundle_launch_spec(
def _resolve_token_env(
plan: SmolmachinesBottlePlan, host_env: object
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
) -> dict[str, str]:
"""Resolve the egress token env-var values from the host's
environ so they reach the bundle's process env via docker's
`-e NAME` inheritance. Empty when no routes declare auth."""
ep = plan.egress_plan
if not ep.routes:
return {}
return egress_resolve_token_values(ep.token_env_map, dict(host_env))
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
@@ -45,6 +45,7 @@ alias gets handed to a new bottle."""
from __future__ import annotations
import fcntl
import json
import os
import platform
@@ -83,6 +84,14 @@ _POOL_START = 16
_POOL_END = 31 # inclusive
# File lock that serialises concurrent allocate() calls so two
# simultaneous launches can't read the same docker state and claim
# the same alias. Narrowed to the allocate() call itself; docker run
# runs after the lock is released. Once the container is running it
# appears in docker state and future allocate() calls will see it.
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
def _pool_addresses() -> list[str]:
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
@@ -179,9 +188,20 @@ def allocate(slug: str) -> str:
On non-macOS the whole `127.0.0.0/8` is loopback by default;
`127.0.0.1` is fine to share and we skip the alias dance.
This still returns a deterministic address so launch.py's
callers don't have to branch on platform."""
callers don't have to branch on platform.
An exclusive file lock serialises concurrent calls so two
simultaneous launches don't read the same docker state and
claim the same alias."""
if not _is_macos():
return "127.0.0.1"
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(_ALLOC_LOCK_PATH, "w") as lf:
fcntl.flock(lf, fcntl.LOCK_EX)
return _allocate_locked()
def _allocate_locked() -> str:
in_use = _aliases_in_use()
for ip in _pool_addresses():
if ip not in in_use:
+60 -56
View File
@@ -12,9 +12,10 @@ from __future__ import annotations
import os
from datetime import datetime, timezone
from dataclasses import replace
from pathlib import Path
from ...agent_provider import runtime_for
from ...agent_provider import agent_provision_plan, runtime_for
from ...backend import BottleSpec
from ...backend.docker.bottle_state import (
BottleMetadata,
@@ -27,9 +28,11 @@ from ...backend.docker.bottle_state import (
write_metadata,
)
from ...egress import Egress
from ...env import resolve_env
from ...git_gate import GitGate
from ...pipelock import PipelockProxy
from ...supervise import Supervise
from ...workspace import workspace_plan as resolve_workspace_plan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
@@ -58,6 +61,8 @@ def resolve_plan(
bottle = manifest.bottle_for(spec.agent_name)
provider = bottle.agent_provider
provider_runtime = runtime_for(provider.template)
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", "/home/node")
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
slug = spec.identity or bottle_identity(spec.agent_name)
@@ -69,72 +74,34 @@ def resolve_plan(
cwd=spec.user_cwd if spec.copy_cwd else "",
copy_cwd=spec.copy_cwd,
started_at=datetime.now(timezone.utc).isoformat(),
# No compose project for smolmachines bottles; chunk 4
# will give dashboard discovery a backend-specific path.
compose_project="",
backend="smolmachines",
))
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
# Agent's env: the prepare-time view doesn't yet know the
# host loopback ports the bundle's daemons get published on
# (those come from docker AFTER `docker run` returns), so
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are
# populated in launch.py and stamped onto guest_env there.
# What we set here is the part that doesn't depend on
# bundle bringup — bottle.env literals, the empty-NO_PROXY
# safe default, and the TLS trust env trio
# (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / REQUESTS_CA_BUNDLE)
# pointing at Debian's update-ca-certificates output bundle.
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
resolved = resolve_env(manifest, spec.agent_name)
guest_env: dict[str, str] = {
**bottle.env,
**resolved.literals,
**resolved.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
# Inner Plans for the four bundle daemons. The ABCs are
# platform-neutral — `.prepare()` writes config files + returns
# a Plan dataclass with no backend-specific assumptions. State
# dirs are still keyed by slug under the docker backend's
# bottle_state layout (shared on-host convention; not a docker
# dependency).
pipelock_dir = pipelock_state_dir(slug)
pipelock_dir.mkdir(parents=True, exist_ok=True)
proxy_plan = PipelockProxy().prepare(bottle, slug, pipelock_dir)
git_gate_dir = git_gate_state_dir(slug)
git_gate_dir.mkdir(parents=True, exist_ok=True)
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = Egress().prepare(bottle, slug, egress_dir)
# Claude-code refuses to start without *something* it
# recognises as a credential. When the bottle has an egress
# route carrying the `claude_code_oauth` role marker, egress
# strips + re-injects the real Authorization header on the
# outbound leg using a token held in egress's own environ — so
# the agent gets a non-secret placeholder here (matches the
# docker backend's forwarded_env logic in
# bot_bottle/backend/docker/prepare.py).
has_provider_auth = any(
provider_runtime.auth_role in r.roles for r in egress_plan.routes
)
if has_provider_auth:
guest_env[provider_runtime.placeholder_env] = "egress-placeholder"
if provider.template == "claude" and has_provider_auth:
guest_env.setdefault("CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC", "1")
guest_env.setdefault("DISABLE_ERROR_REPORTING", "1")
supervise_plan = None
if bottle.supervise:
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = Supervise().prepare(slug, supervise_dir)
# Prompt file is always written (mode 0o600) so the in-VM
# path always exists. Content is the agent's `prompt`
# field (markdown body) — empty for agents with no prompt.
@@ -162,6 +129,45 @@ def resolve_plan(
else:
image_default = provider_runtime.image
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
agent_provision = agent_provision_plan(
template=provider.template,
dockerfile=agent_dockerfile_path,
state_dir=agent_dir,
guest_home=guest_home,
guest_env=guest_env,
forward_host_credentials=provider.forward_host_credentials,
auth_token=provider.auth_token,
host_env=dict(os.environ),
trusted_project_path=workspace_plan.workdir,
)
merged_guest_env = dict(agent_provision.guest_env)
for key, val in agent_provision.env_vars.items():
merged_guest_env.setdefault(key, val)
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
# Inner Plans for the four bundle daemons. The ABCs are
# platform-neutral — `.prepare()` writes config files + returns
# a Plan dataclass with no backend-specific assumptions. State
# dirs are still keyed by slug under the docker backend's
# bottle_state layout (shared on-host convention; not a docker
# dependency).
pipelock_dir = pipelock_state_dir(slug)
pipelock_dir.mkdir(parents=True, exist_ok=True)
proxy_plan = PipelockProxy().prepare(
bottle, slug, pipelock_dir, agent_provision.egress_routes,
)
egress_dir = egress_state_dir(slug)
egress_dir.mkdir(parents=True, exist_ok=True)
egress_plan = Egress().prepare(
bottle, slug, egress_dir, agent_provision.egress_routes,
)
supervise_plan = None
if bottle.supervise:
supervise_dir = supervise_state_dir(slug)
supervise_dir.mkdir(parents=True, exist_ok=True)
supervise_plan = Supervise().prepare(slug, supervise_dir)
return SmolmachinesBottlePlan(
spec=spec,
@@ -172,16 +178,14 @@ def resolve_plan(
bundle_ip=bundle_ip,
machine_name=machine_name,
agent_image_ref=agent_image_ref,
guest_env=guest_env,
guest_env=agent_provision.guest_env,
prompt_file=prompt_file,
proxy_plan=proxy_plan,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_command=provider_runtime.command,
agent_prompt_mode=provider_runtime.prompt_mode,
agent_provider_template=provider.template,
agent_dockerfile_path=agent_dockerfile_path,
agent_provision=agent_provision,
workspace_plan=workspace_plan,
)
+32 -11
View File
@@ -15,6 +15,8 @@ flag exists; the VM init is root), so we don't need the explicit
from __future__ import annotations
import time
from ....log import die
from ...util import (
AGENT_CA_BUNDLE,
@@ -26,6 +28,9 @@ from .. import smolvm as _smolvm
from ..bottle_plan import SmolmachinesBottlePlan
_SIGKILL_EXIT = 128 + 9
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
"""Copy the agent-facing CA cert into the guest, rebuild the
trust bundle, emit a one-line fingerprint log. Called from
@@ -40,17 +45,16 @@ def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
# `requests` / libraries that don't load the system bundle.
#
# chown + chmod + update-ca-certificates run in one
# `sh -c` so we only pay one machine_exec round trip; the
# `&&` chaining surfaces the first failure as the return
# code.
r = _smolvm.machine_exec(target, [
"sh", "-c",
f"chown root:root {AGENT_CA_PATH} && "
f"chmod 644 {AGENT_CA_PATH} && "
f"update-ca-certificates",
])
if r.returncode != 0 or "1 added" not in (r.stdout or ""):
r = _install_ca(target)
if r.returncode == _SIGKILL_EXIT:
# smolvm/libkrun can SIGKILL an otherwise-normal exec
# during early-VM provisioning. `update-ca-certificates`
# is idempotent, so retry the same install once after a
# short settle delay before treating it as fatal.
time.sleep(1.0)
r = _install_ca(target)
if r.returncode != 0:
# update-ca-certificates not adding our cert is fatal —
# claude-code's TLS handshake against the egress-MITM'd
# api.anthropic.com would fail downstream. Bail early
@@ -66,6 +70,23 @@ def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
log_ca_fingerprint(cert_host_path, label)
def _install_ca(target: str) -> _smolvm.SmolvmRunResult:
# chown + chmod + update-ca-certificates + bundle
# verification run in one `sh -c` so we only pay one
# machine_exec round trip; the `&&` chaining surfaces the
# first failure as the return code. The verify check is more
# stable than requiring "1 added" in stdout: a retry after a
# partially-completed first run may legitimately report "0
# added" while the cert is already installed.
return _smolvm.machine_exec(target, [
"sh", "-c",
f"chown root:root {AGENT_CA_PATH} && "
f"chmod 644 {AGENT_CA_PATH} && "
f"update-ca-certificates && "
f"openssl verify -CAfile {AGENT_CA_BUNDLE} {AGENT_CA_PATH}",
])
# Re-exported for the launch/provision_ca caller + tests. The path
# constants live in the shared `backend.util` (Debian's
# `update-ca-certificates` layout is the same in both backends).
@@ -4,7 +4,7 @@
Three concerns, all about git in the agent:
1. If --cwd was passed AND the host cwd has a .git, copy that
.git into /home/node/workspace/.git so the agent operates on
.git into the planned guest workspace so the agent operates on
the user's repo.
2. If the bottle declares `git` entries (PRD 0008), write a
~/.gitconfig with insteadOf rules so every git operation
@@ -18,7 +18,7 @@ Three concerns, all about git in the agent:
Differs from `backend.docker.provision.git` in one address detail:
the TSI-allowlisted guest can only reach the bundle's pinned IP
(no DNS resolver in the /32 allowlist), so the insteadOf URLs
are `git://<bundle_ip>:<port>/<name>.git` rather than the
are `http://<bundle_ip>:<port>/<name>.git` rather than the
docker backend's `git://git-gate/<name>.git`. The render itself
is the shared `git_gate_render_gitconfig` on the platform-neutral
git_gate module."""
@@ -58,20 +58,22 @@ def _provision_cwd_git(plan: SmolmachinesBottlePlan, target: str) -> None:
"""If --cwd was set and the host cwd has a .git directory, copy
it into <guest_home>/workspace/.git and fix ownership. No-op
otherwise."""
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
return
guest_workspace_git = f"{_guest_home()}/workspace/.git"
info(f"copying {plan.spec.user_cwd}/.git -> {target}:{guest_workspace_git}")
guest_workspace_git = f"{workspace.guest_path}/.git"
host_git = str(workspace.host_path / ".git")
info(f"copying {host_git} -> {target}:{guest_workspace_git}")
# mkdir -p the workspace dir so `machine cp` lands the .git
# directly there even on first-time bottles.
_smolvm.machine_exec(target, ["mkdir", "-p", f"{_guest_home()}/workspace"])
_smolvm.machine_exec(target, ["mkdir", "-p", workspace.guest_path])
_smolvm.machine_cp(
f"{plan.spec.user_cwd}/.git", f"{target}:{guest_workspace_git}",
host_git, f"{target}:{guest_workspace_git}",
)
# `machine cp` lands files as root; the agent runs as node so
# the workspace tree must be chowned over.
_smolvm.machine_exec(
target, ["chown", "-R", "node:node", guest_workspace_git],
target, ["chown", "-R", workspace.owner, guest_workspace_git],
)
@@ -82,12 +84,14 @@ def _provision_git_gate_config(plan: SmolmachinesBottlePlan, target: str) -> Non
if not bottle.git:
return
# `127.0.0.1:<host port>` form: the bundle's git-gate port
# is published on host loopback at launch time so the
# smolvm guest (which can only reach macOS networking via
# `<loopback alias>:<host port>` form: the bundle's git-gate
# HTTP port is published on host loopback at launch time so
# the smolvm guest (which can only reach macOS networking via
# TSI, not the docker bridge IP) can dial it. launch.py
# populates `plan.agent_git_gate_host` after bundle bringup.
content = git_gate_render_gitconfig(bottle.git, plan.agent_git_gate_host)
content = git_gate_render_gitconfig(
bottle.git, plan.agent_git_gate_host, scheme="http",
)
guest_gitconfig = f"{_guest_home()}/.gitconfig"
# Stage the file under the plan's stage_dir so `machine cp`
@@ -0,0 +1,33 @@
"""Provision non-secret provider auth markers into a smolmachines bottle."""
from __future__ import annotations
from ....log import die
from .. import smolvm as _smolvm
from ..bottle_plan import SmolmachinesBottlePlan
def provision_provider_auth(plan: SmolmachinesBottlePlan, target: str) -> None:
"""Apply provider-owned guest setup through smolvm primitives."""
provision = plan.agent_provision
for d in provision.dirs:
_exec(target, ["mkdir", "-p", d.guest_path], f"could not create {d.guest_path}")
_exec(target, ["chown", d.owner, d.guest_path], f"could not chown {d.guest_path}")
_exec(target, ["chmod", d.mode, d.guest_path], f"could not chmod {d.guest_path}")
for command in provision.pre_copy:
_exec(target, list(command.argv), command.error)
for f in provision.files:
_smolvm.machine_cp(str(f.host_path), f"{target}:{f.guest_path}")
_exec(target, ["chown", f.owner, f.guest_path], f"could not chown {f.guest_path}")
_exec(target, ["chmod", f.mode, f.guest_path], f"could not chmod {f.guest_path}")
for command in provision.verify:
_exec(target, list(command.argv), command.error)
def _exec(target: str, argv: list[str], error: str) -> None:
result = _smolvm.machine_exec(target, argv)
if result.returncode != 0:
detail = (result.stderr or result.stdout).strip()
if detail:
detail = f": {detail}"
die(f"agent provider provisioning: {error}{detail}")
@@ -0,0 +1,36 @@
"""Copy the operator workspace into a smolmachines guest."""
from __future__ import annotations
import shlex
from ....log import info
from .. import smolvm as _smolvm
from ..bottle_plan import SmolmachinesBottlePlan
def provision_workspace(plan: SmolmachinesBottlePlan, target: str) -> None:
"""Copy host cwd contents to the planned guest workspace."""
workspace = plan.workspace_plan
if not (workspace.enabled and workspace.copy_contents):
return
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
guest_path_q = shlex.quote(workspace.guest_path)
guest_parent_q = shlex.quote(guest_parent)
owner_q = shlex.quote(workspace.owner)
mode_q = shlex.quote(workspace.mode)
info(f"copying {workspace.host_path} -> {target}:{workspace.guest_path}")
_smolvm.machine_exec(
target,
["sh", "-c", f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}"],
)
_smolvm.machine_cp(str(workspace.host_path), f"{target}:{workspace.guest_path}")
_smolvm.machine_exec(
target,
[
"sh", "-c",
f"chown -R {owner_q} {guest_path_q} && "
f"chmod {mode_q} {guest_path_q}",
],
)
+30
View File
@@ -27,11 +27,13 @@ from __future__ import annotations
import shutil
import subprocess
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Mapping, Sequence
_SMOLVM = "smolvm"
@@ -197,6 +199,34 @@ def machine_exec(
)
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
channel needs a brief warm-up before back-to-back exec calls are
safe. Polling exits as soon as the channel is ready and fails
loudly if the VM never responds."""
deadline = time.monotonic() + timeout
delay = 0.1
while time.monotonic() < deadline:
r = machine_exec(name, ["true"])
if r.returncode == 0:
return
remaining = deadline - time.monotonic()
if remaining <= 0:
break
time.sleep(min(delay, remaining))
delay = min(delay * 2, 0.5)
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
raise SmolvmError(
argv,
subprocess.CompletedProcess(
args=argv, returncode=-1, stdout="",
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
),
)
def machine_cp(src: str, dst: str) -> None:
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
reference a path inside the VM, bare path for the host. Both
+24 -14
View File
@@ -175,6 +175,13 @@ def approve(
qp.proposal.bottle_slug, file_to_apply,
)
elif qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
_meta = read_metadata(qp.proposal.bottle_slug)
if _meta is not None and not _meta.compose_project:
raise CapabilityApplyError(
"capability-block remediation is not supported for smolmachines "
"bottles. Reject this proposal or handle the capability change "
"manually, then restart the bottle."
)
diff_before, diff_after = apply_capability_change(
qp.proposal.bottle_slug, file_to_apply,
)
@@ -640,23 +647,19 @@ def _bottle_for_slug(
) -> tuple["object", str]:
"""Return `(bottle_handle, prompt_path_hint)` for a re-attach.
If the slug is in `bottles` (dashboard-owned), return the stored
handle directly. Otherwise synthesize a `DockerBottle` from the
container name `bot-bottle-<slug>`. For synthesized bottles
the prompt-file path comes from the manifest's agent if we can
resolve it via metadata.json + the loaded manifest; otherwise
the re-attach runs without `--append-system-prompt-file`.
handle directly. Otherwise synthesize a bottle from the persisted
metadata. The backend field in metadata (PRD 0040) selects Docker
or smolmachines; unknown or missing metadata defaults to Docker.
Returns the empty string for prompt_path_hint when we omit the
flag — the caller passes None to DockerBottle in that case."""
from ..backend.docker.bottle import DockerBottle
from ..backend.docker.bottle_state import read_metadata
from ..backend.smolmachines.bottle import SmolmachinesBottle
if slug in bottles:
_cm, bottle, _identity = bottles[slug]
return bottle, ""
# The container hosting the agent's agent process is named
# `bot-bottle-<slug>` — set by the compose renderer
# (no service suffix on the agent service, by design).
container_name = f"bot-bottle-{slug}"
instance_name = f"bot-bottle-{slug}"
prompt_path: str | None = None
metadata = read_metadata(slug)
if metadata is not None and manifest is not None:
@@ -666,11 +669,18 @@ def _bottle_for_slug(
"BOT_BOTTLE_CONTAINER_HOME", "/home/node",
)
prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
synth = DockerBottle(
container=container_name,
teardown=lambda: None,
prompt_path_in_container=prompt_path,
)
backend = metadata.backend if metadata is not None else ""
if backend == "smolmachines":
synth: object = SmolmachinesBottle(
instance_name,
prompt_path=prompt_path,
)
else:
synth = DockerBottle(
container=instance_name,
teardown=lambda: None,
prompt_path_in_container=prompt_path,
)
return synth, (prompt_path or "")
+2
View File
@@ -52,8 +52,10 @@ def cmd_resume(argv: list[str]) -> int:
user_cwd=metadata.cwd or USER_CWD,
identity=metadata.identity,
)
backend_name = metadata.backend or None
return _launch_bottle(
spec,
dry_run=args.dry_run,
remote_control=args.remote_control,
backend_name=backend_name,
)
+325
View File
@@ -0,0 +1,325 @@
"""Host Codex auth helpers.
Reads the host's Codex ChatGPT/device-login auth state and returns only
the short-lived access token needed by egress. This module deliberately
does not expose refresh tokens or raw auth payloads.
"""
from __future__ import annotations
import base64
import json
import os
from copy import deepcopy
from datetime import datetime, timezone
from pathlib import Path
from .log import die
from .util import expand_tilde
def codex_auth_path(host_env: dict[str, str] | None = None) -> Path:
env = os.environ if host_env is None else host_env
home = env.get("CODEX_HOME")
if home:
return Path(expand_tilde(home)) / "auth.json"
return Path.home() / ".codex" / "auth.json"
def codex_host_access_token(
host_env: dict[str, str] | None = None,
*,
now: datetime | None = None,
) -> str:
path = codex_auth_path(host_env)
if not path.is_file():
die(
f"codex host credentials: auth file missing at {path}. "
"Run `codex login --device-auth` on the host or disable "
"agent_provider.forward_host_credentials."
)
raw = _read_auth_object(path)
auth_mode = raw.get("auth_mode")
if not isinstance(auth_mode, str) or auth_mode == "api_key":
die(
"codex host credentials: host Codex auth is not user/device "
"auth. Run `codex login --device-auth` on the host."
)
tokens = raw.get("tokens")
if not isinstance(tokens, dict):
die(f"codex host credentials: {path} is missing tokens")
access = tokens.get("access_token")
if not isinstance(access, str) or not access:
die(
f"codex host credentials: {path} is missing tokens.access_token. "
"Run `codex login --device-auth` on the host."
)
exp = _jwt_exp(access)
if exp is None:
die("codex host credentials: tokens.access_token is not a JWT with exp")
check_now = now or datetime.now(timezone.utc)
if exp <= check_now:
die(
"codex host credentials: host Codex access token is expired. "
"Run `codex login --device-auth` on the host and restart the bottle."
)
return access
def codex_dummy_auth_json(
host_env: dict[str, str] | None = None,
*,
now: datetime | None = None,
) -> str:
"""Return a non-secret `auth.json` that keeps Codex in the host's
auth branch while egress owns the real bearer token.
The dummy access/id tokens carry the *host* token's real `exp` so
Codex's proactive refresh lifecycle (it refreshes when its local
access token is at/past expiry) tracks the real token instead of
firing after an artificial TTL. Codex cannot refresh inside the
bottle the refresh token is a placeholder and the OpenAI token
endpoint is off-route so a shorter dummy exp would drop Codex to
the sign-in screen the moment it lapsed, even while egress still
holds a valid bearer."""
path = codex_auth_path(host_env)
access = codex_host_access_token(host_env, now=now)
raw = _read_auth_object(path)
host_exp = _jwt_exp(access)
exp_ts = int(host_exp.timestamp()) if host_exp is not None else None
dummy = _redact_codex_auth(deepcopy(raw), now=now, exp_ts=exp_ts)
return json.dumps(dummy, indent=2, sort_keys=True) + "\n"
def write_codex_dummy_auth_file(
path: Path,
host_env: dict[str, str] | None = None,
*,
now: datetime | None = None,
) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(codex_dummy_auth_json(host_env, now=now))
path.chmod(0o600)
def _read_auth_object(path: Path) -> dict:
try:
raw = json.loads(path.read_text())
except (OSError, json.JSONDecodeError) as e:
die(f"codex host credentials: could not read valid JSON at {path}: {e}")
if not isinstance(raw, dict):
die(f"codex host credentials: {path} must contain a JSON object")
return raw
def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
if exp_ts is not None:
return exp_ts
check_now = now or datetime.now(timezone.utc)
return int(check_now.timestamp()) + 3600
def _dummy_timestamp(now: datetime | None = None) -> str:
check_now = now or datetime.now(timezone.utc)
if check_now.tzinfo is None:
check_now = check_now.replace(tzinfo=timezone.utc)
check_now = check_now.astimezone(timezone.utc)
return check_now.isoformat(timespec="milliseconds").replace("+00:00", "Z")
def _dummy_jwt(now: datetime | None = None, *, exp_ts: int | None = None) -> str:
return _encode_dummy_jwt({
"exp": _dummy_exp(now, exp_ts),
"sub": "bot-bottle-placeholder",
})
def _dummy_jwt_from_host(
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
) -> str:
if not isinstance(value, str):
return _dummy_jwt(now, exp_ts=exp_ts)
parts = value.split(".")
if len(parts) < 2:
return _dummy_jwt(now, exp_ts=exp_ts)
try:
payload = json.loads(_b64url_decode(parts[1]))
except (ValueError, json.JSONDecodeError):
return _dummy_jwt(now, exp_ts=exp_ts)
if not isinstance(payload, dict):
return _dummy_jwt(now, exp_ts=exp_ts)
return _encode_dummy_jwt(_redact_jwt_payload(payload, now=now, exp_ts=exp_ts))
def _encode_dummy_jwt(payload: dict) -> str:
def enc(obj: dict) -> str:
raw = json.dumps(obj, separators=(",", ":")).encode()
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
return f"{enc({'alg': 'none', 'typ': 'JWT'})}.{enc(payload)}.placeholder"
def _redact_jwt_payload(
payload: dict,
*,
now: datetime | None = None,
exp_ts: int | None = None,
) -> dict:
out = _redact_claims(payload)
if not isinstance(out, dict):
out = {}
out["exp"] = _dummy_exp(now, exp_ts)
out.setdefault("sub", "bot-bottle-placeholder")
return out
def _redact_claims(value: object) -> object:
if isinstance(value, dict):
out: dict[str, object] = {}
for key, inner in value.items():
lower = key.lower()
if key == "https://api.openai.com/profile":
out[key] = _redact_profile_claim(inner)
elif key == "https://api.openai.com/auth":
out[key] = _redact_auth_claim(inner)
elif lower == "email":
out[key] = "bot-bottle@example.invalid"
elif lower == "email_verified":
out[key] = True
elif lower in {"exp", "iat", "nbf", "auth_time", "pwd_auth_time"}:
out[key] = inner if isinstance(inner, (int, float)) else 0
elif lower in {"aud", "scp", "amr"}:
out[key] = inner if isinstance(inner, list) else []
elif isinstance(inner, bool):
out[key] = inner
elif isinstance(inner, dict):
out[key] = {}
elif isinstance(inner, list):
out[key] = []
else:
out[key] = "bot-bottle-placeholder"
return out
if isinstance(value, list):
return []
return "bot-bottle-placeholder"
def _redact_profile_claim(value: object) -> dict:
profile = value if isinstance(value, dict) else {}
return {
"email": "bot-bottle@example.invalid",
"email_verified": bool(profile.get("email_verified", True)),
}
def _redact_auth_claim(value: object) -> dict:
auth = value if isinstance(value, dict) else {}
out: dict[str, object] = {}
for key, inner in auth.items():
lower = key.lower()
if lower == "chatgpt_plan_type" and isinstance(inner, str) and inner:
out[key] = inner
elif lower == "chatgpt_account_id" and isinstance(inner, str) and inner:
# Current Codex uses the selected account id when building
# ChatGPT requests. Keep that non-secret identifier aligned
# with the host while egress owns the real bearer token.
out[key] = inner
elif lower == "localhost" and isinstance(inner, bool):
out[key] = inner
elif isinstance(inner, bool):
out[key] = inner
elif isinstance(inner, list):
out[key] = []
elif isinstance(inner, dict):
out[key] = {}
else:
out[key] = "bot-bottle-placeholder"
out.setdefault("chatgpt_plan_type", "unknown")
out.setdefault("user_id", "bot-bottle-placeholder")
out.setdefault("chatgpt_user_id", "bot-bottle-placeholder")
out.setdefault("chatgpt_account_id", "bot-bottle-placeholder")
return out
def _redact_codex_auth(
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
) -> object:
auth = value if isinstance(value, dict) else {}
out: dict[str, object] = {}
for key, inner in auth.items():
lower = key.lower()
if lower == "auth_mode" and isinstance(inner, str) and inner:
out[key] = inner
elif lower == "openai_api_key":
out[key] = None
elif lower == "last_refresh":
# Codex parses this as a timestamp on startup. Keep the
# schema valid without copying host-side session metadata.
out[key] = _dummy_timestamp(now)
elif lower == "tokens":
out[key] = _redact_token_block(inner, now=now, exp_ts=exp_ts)
else:
out[key] = _redact_unknown_auth_value(inner)
return out
def _redact_token_block(
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
) -> dict[str, object]:
tokens = value if isinstance(value, dict) else {}
out: dict[str, object] = {}
for key, inner in tokens.items():
lower = key.lower()
if lower in {"access_token", "id_token"}:
out[key] = _dummy_jwt_from_host(inner, now=now, exp_ts=exp_ts)
elif lower == "account_id" and isinstance(inner, str) and inner:
# Current Codex uses this non-secret selected account id
# while egress owns the real bearer token.
out[key] = inner
else:
out[key] = _redact_unknown_auth_value(inner)
return out
def _redact_unknown_auth_value(value: object) -> object:
if isinstance(value, bool):
return value
if isinstance(value, dict):
return {}
if isinstance(value, list):
return []
if value is None:
return None
return "bot-bottle-placeholder"
def _jwt_exp(token: str) -> datetime | None:
parts = token.split(".")
if len(parts) < 2:
return None
try:
payload = json.loads(_b64url_decode(parts[1]))
except (ValueError, json.JSONDecodeError):
return None
if not isinstance(payload, dict):
return None
exp = payload.get("exp")
if not isinstance(exp, (int, float)):
return None
return datetime.fromtimestamp(exp, timezone.utc)
def _b64url_decode(value: str) -> str:
padded = value + ("=" * (-len(value) % 4))
return base64.urlsafe_b64decode(padded.encode("ascii")).decode("utf-8")
__all__ = [
"codex_auth_path",
"codex_dummy_auth_json",
"codex_host_access_token",
"write_codex_dummy_auth_file",
]
+110 -86
View File
@@ -24,12 +24,19 @@ flow (PRD 0014) at egress and renames the MCP tool.
from __future__ import annotations
import dataclasses
from abc import ABC, abstractmethod
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING
from .egress_addon_core import Route
from .log import die
from .manifest import Bottle
if TYPE_CHECKING:
from .manifest import Bottle
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
# DNS name agents will dial for the per-bottle egress sidecar.
@@ -48,32 +55,30 @@ EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
@dataclass(frozen=True)
class EgressRoute:
"""One resolved route on the egress sidecar.
class EgressRoute(Route):
"""Host-side extension of the addon's `Route`.
`host` matches the request's hostname (case-insensitive). The
optional `path_allowlist` constrains the URL path; empty tuple
means no path-level filtering. The `auth_scheme` / `token_env` /
`token_ref` triple is the credential-injection config; empty
strings mean "no auth injection" (the manifest's nested `auth`
block was omitted).
Inherits `host`, `path_allowlist`, `auth_scheme`, and `token_env`
from `egress_addon_core.Route` those are the fields that cross the
YAML wire into the sidecar. The three fields below are host-only and
are never serialised to the addon.
`token_env` is the env-var slot inside the egress container
(e.g. `EGRESS_TOKEN_0`); `token_ref` is the host env var
the CLI reads at launch and forwards into the container's environ
under `token_env`. Routes that share a `token_ref` coalesce to
one `token_env` slot.
`token_ref` is the host env var the CLI reads at launch and forwards
into the container's environ under `token_env`. Routes that share a
`token_ref` coalesce to one `token_env` slot.
`roles` carries the manifest route's optional role markers (see
`manifest.EGRESS_ROLES`). The launch step reads these for
side effects like the claude-code OAuth placeholder env."""
`roles` carries the manifest route's role tuple (reserved for
future use; always empty today).
`tls_passthrough` signals that pipelock must not TLS-MITM this
host either because the manifest declared `pipelock.tls_passthrough:
true` (lifted in `egress_manifest_routes`) or because a provider
route set it (e.g. egress injects its own Bearer on that host
after the agent boundary and pipelock's header DLP would block it)."""
host: str
path_allowlist: tuple[str, ...] = ()
auth_scheme: str = ""
token_env: str = ""
token_ref: str = ""
roles: tuple[str, ...] = ()
tls_passthrough: bool = False
@dataclass(frozen=True)
@@ -130,55 +135,60 @@ class EgressPlan:
def egress_manifest_routes(
bottle: Bottle,
) -> tuple[EgressRoute, ...]:
"""Lift each `bottle.egress.routes[]` manifest entry into a
resolved EgressRoute. Order is preserved so route lookup at
the proxy is stable.
Token-env slots are assigned per distinct `token_ref`: the first
authenticated route with `token_ref` "GH_PAT" gets
`EGRESS_TOKEN_0`; a second route with the same `token_ref`
shares slot 0. Unauthenticated routes (`auth` omitted) contribute
no slot.
This is the effective set the addon enforces. Provider runtime
routes are intentionally not injected implicitly; every allowed
host must come from the home-owned bottle manifest."""
"""Lift each `bottle.egress.routes[]` manifest entry into an EgressRoute.
Order is preserved. Token slots are not assigned here slot assignment
is a final step in `egress_routes_for_bottle` after provider and manifest
routes are merged."""
out: list[EgressRoute] = []
slot_for_token: dict[str, str] = {}
for r in bottle.egress.routes:
if r.AuthScheme and r.TokenRef:
token_env = slot_for_token.get(r.TokenRef)
if token_env is None:
token_env = f"EGRESS_TOKEN_{len(slot_for_token)}"
slot_for_token[r.TokenRef] = token_env
out.append(EgressRoute(
host=r.Host,
path_allowlist=r.PathAllowlist,
auth_scheme=r.AuthScheme,
token_env=token_env,
token_ref=r.TokenRef,
roles=r.Role,
))
else:
out.append(EgressRoute(
host=r.Host,
path_allowlist=r.PathAllowlist,
roles=r.Role,
))
out.append(EgressRoute(
host=r.Host,
path_allowlist=r.PathAllowlist,
auth_scheme=r.AuthScheme,
token_ref=r.TokenRef,
roles=r.Role,
tls_passthrough=r.Pipelock.TlsPassthrough,
))
return tuple(out)
def egress_routes_for_bottle(
bottle: Bottle,
provider_routes: tuple[EgressRoute, ...] = (),
) -> tuple[EgressRoute, ...]:
"""Effective egress routes. This is what gets rendered into
routes.yaml + what the addon enforces.
"""Effective egress routes for the agent.
Operators that want to allow a host declare it directly in
`bottle.egress.routes` as an authenticated route or bare-pass entry
(`- host: <name>`). The legacy `bottle.egress.allowlist`
folding is gone egress is the single allowlist surface."""
return egress_manifest_routes(bottle)
Provider routes own their hosts outright; manifest routes for hosts
not claimed by any provider are appended. Token slots are assigned
in a final pass over the merged list in order, so provisioned routes
get the lower slot numbers."""
manifest = egress_manifest_routes(bottle)
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
merged = list(provider_routes) + [
r for r in manifest if r.host.lower() not in provisioned_hosts
]
return _assign_token_slots(merged)
def _assign_token_slots(
routes: list[EgressRoute],
) -> tuple[EgressRoute, ...]:
"""Assign EGRESS_TOKEN_N slots to authenticated routes in order.
Routes sharing a token_ref share a slot. Unauthenticated routes
(no auth_scheme / token_ref) keep token_env empty."""
slot_for_ref: dict[str, str] = {}
out: list[EgressRoute] = []
for r in routes:
if r.auth_scheme and r.token_ref:
slot = slot_for_ref.get(r.token_ref)
if slot is None:
slot = f"EGRESS_TOKEN_{len(slot_for_ref)}"
slot_for_ref[r.token_ref] = slot
out.append(dataclasses.replace(r, token_env=slot))
else:
out.append(r)
return tuple(out)
def egress_token_env_map(
@@ -193,7 +203,7 @@ def egress_token_env_map(
silently picking one."""
out: dict[str, str] = {}
for r in routes:
if not r.token_env:
if not (r.auth_scheme and r.token_ref and r.token_env):
continue
existing = out.get(r.token_env)
if existing is not None and existing != r.token_ref:
@@ -206,35 +216,43 @@ def egress_token_env_map(
return out
def _route_to_yaml_fields(r: Route) -> dict:
"""Return the addon-visible fields for one route.
Single authoritative mapping between EgressRoute (host-side) and
egress_addon_core.Route (sidecar-side). When a field is added to
the addon's Route that must appear in the YAML, add it here and
in egress_addon_core._parse_one together."""
fields: dict = {"host": r.host}
if r.auth_scheme and r.token_env:
fields["auth_scheme"] = r.auth_scheme
fields["token_env"] = r.token_env
if r.path_allowlist:
fields["path_allowlist"] = list(r.path_allowlist)
return fields
def egress_render_routes(
routes: tuple[EgressRoute, ...],
) -> str:
"""Serialize the route table for the addon to read.
YAML content no token values, no host env-var names. The only
thing the addon needs at runtime is the host path_allowlist
+ auth_scheme + in-container env-var mapping. The actual token
values arrive via the container's environ.
Authenticated routes carry `auth_scheme` + `token_env`;
unauthenticated routes omit both keys (the addon's parser
enforces both-or-neither). Hand-rolled YAML in the style of
`pipelock_render_yaml` so the addon's parser
(`yaml_subset.parse_yaml_subset`) round-trips it cleanly."""
YAML content no token values, no host env-var names. Fields are
determined by `_route_to_yaml_fields`, which is the single point of
truth for the EgressRoute egress_addon_core.Route mapping."""
lines: list[str] = ["routes:"]
if not routes:
# `routes:` with an empty list on the same line — the parser
# needs SOMETHING here. Empty inline list is the cleanest.
lines[0] = "routes: []"
return "\n".join(lines) + "\n"
for r in routes:
lines.append(f' - host: "{r.host}"')
if r.auth_scheme and r.token_env:
lines.append(f' auth_scheme: "{r.auth_scheme}"')
lines.append(f' token_env: "{r.token_env}"')
if r.path_allowlist:
f = _route_to_yaml_fields(r)
lines.append(f' - host: "{f["host"]}"')
if "auth_scheme" in f:
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
lines.append(f' token_env: "{f["token_env"]}"')
if "path_allowlist" in f:
lines.append(" path_allowlist:")
for p in r.path_allowlist:
for p in f["path_allowlist"]:
lines.append(f' - "{p}"')
return "\n".join(lines) + "\n"
@@ -274,18 +292,23 @@ class Egress(ABC):
sidecar's start/stop lifecycle is backend-specific and lives on
concrete subclasses."""
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> EgressPlan:
"""Lift `bottle.egress.routes` into resolved routes,
render the routes file (mode 600) under `stage_dir`, and
def prepare(
self,
bottle: Bottle,
slug: str,
stage_dir: Path,
provider_routes: tuple[EgressRoute, ...] = (),
) -> EgressPlan:
"""Lift `bottle.egress.routes` + `provider_routes` into resolved
routes, render the routes file (mode 600) under `stage_dir`, and
return the plan. Pure host-side, no docker subprocess. The
token-env map records the mapping the launch step uses to
forward values from the host's environ into the sidecar's
environ.
forward values from the host's environ into the sidecar's environ.
Returned plan is incomplete: the launch step must fill
`internal_network` / `egress_network` / `pipelock_proxy_url`
via `dataclasses.replace` before passing it to `.start`."""
routes = egress_routes_for_bottle(bottle)
routes = egress_routes_for_bottle(bottle, provider_routes)
routes_path = stage_dir / "egress_routes.yaml"
routes_path.write_text(egress_render_routes(routes))
routes_path.chmod(0o600)
@@ -297,6 +320,7 @@ class Egress(ABC):
)
__all__ = [
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
"EGRESS_HOSTNAME",
"EGRESS_ROUTES_IN_CONTAINER",
"Egress",
+14 -47
View File
@@ -29,22 +29,21 @@ backend-specific and lives on concrete subclasses (see
from __future__ import annotations
import shlex
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from dataclasses import dataclass
from pathlib import Path
from typing import Mapping
from .log import die
from .manifest import Bottle, GitEntry
# Short network alias for git-gate inside the sidecar bundle. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate"
def _empty_str_map() -> dict[str, str]:
return {}
# Bound half-open git client sessions. If an agent/tool runner is
# interrupted during push, git daemon should reap the receive-pack
# child instead of keeping the gate wedged indefinitely.
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
@dataclass(frozen=True)
@@ -60,10 +59,7 @@ class GitGateUpstream:
KnownHostKey string from the manifest; the gate's start step
materialises it into a known_hosts file if non-empty.
`extra_hosts` is a `{hostname: ip}` map the backend injects into
the gate container's `/etc/hosts` via `--add-host` so the gate
can resolve upstream hostnames that aren't reachable via the
container's default DNS (e.g. Tailscale-only hosts)."""
the gate credential paths inside the running sidecar."""
name: str
upstream_url: str
@@ -72,7 +68,6 @@ class GitGateUpstream:
identity_file: str
known_host_key: str
known_hosts_file: Path = Path()
extra_hosts: Mapping[str, str] = field(default_factory=_empty_str_map)
@dataclass(frozen=True)
@@ -109,46 +104,19 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
upstream_port=e.UpstreamPort,
identity_file=e.IdentityFile,
known_host_key=e.KnownHostKey,
extra_hosts=dict(e.ExtraHosts),
)
for e in bottle.git
)
def git_gate_aggregate_extra_hosts(
upstreams: tuple[GitGateUpstream, ...],
) -> dict[str, str]:
"""Merge every upstream's `extra_hosts` into a single
`{hostname: ip}` map for `--add-host` on the gate container. Two
entries naming the same hostname with different IPs is a manifest
bug the gate has one /etc/hosts so die loudly with the
conflicting names rather than silently picking one."""
merged: dict[str, str] = {}
source: dict[str, str] = {}
for u in upstreams:
for host, ip in u.extra_hosts.items():
existing = merged.get(host)
if existing is None:
merged[host] = ip
source[host] = u.name
elif existing != ip:
die(
f"git-gate ExtraHosts conflict: '{host}' maps to "
f"'{existing}' in upstream '{source[host]}' and to "
f"'{ip}' in upstream '{u.name}'. The gate has one "
f"/etc/hosts; pick one IP."
)
return merged
def git_gate_render_gitconfig(
entries: tuple[GitEntry, ...], gate_host: str
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
"""Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
exposed for tests + reuse across backends.
`gate_host` is the part of the URL between `git://` and the
`gate_host` is the part of the URL between `<scheme>://` and the
repo path backends differ here:
- docker: `git-gate` (the short network alias)
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
@@ -165,7 +133,7 @@ def git_gate_render_gitconfig(
"# fetch-from-upstream-before-every-upload-pack via access-hook).\n",
]
for entry in entries:
out.append(f'[url "git://{gate_host}/{entry.Name}.git"]\n')
out.append(f'[url "{scheme}://{gate_host}/{entry.Name}.git"]\n')
out.append(f"\tinsteadOf = {entry.Upstream}\n")
if entry.RemoteKey and entry.RemoteKey != entry.UpstreamHost:
port = (
@@ -233,20 +201,20 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
" git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}",
"",
"mkdir -p /git",
]
for u in upstreams:
# Single-quote args so URL/path content (containing : and /)
# passes through ash unmangled. Names came through the manifest
# validator so they don't contain a single quote.
lines.append(f"init_repo '{u.name}' '{u.upstream_url}'")
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
lines.extend([
"",
"exec git daemon \\",
" --reuseaddr \\",
f" --timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
f" --init-timeout={GIT_GATE_DAEMON_TIMEOUT_SECS} \\",
" --base-path=/git \\",
" --export-all \\",
" --enable=receive-pack \\",
@@ -436,7 +404,6 @@ class GitGate(ABC):
identity_file=u.identity_file,
known_host_key=u.known_host_key,
known_hosts_file=known_hosts_file,
extra_hosts=dict(u.extra_hosts),
)
)
return GitGatePlan(
+175
View File
@@ -0,0 +1,175 @@
"""Tiny smart-HTTP wrapper for git-gate repos.
Used by the smolmachines backend where `git://` push traffic over the
host-published Docker port can hang before receive-pack reaches hooks.
The wrapper serves the same `/git/*.git` bare repos through
`git http-backend`, so pre-receive and upstream forwarding remain the
git-gate enforcement point.
"""
from __future__ import annotations
import os
import subprocess
import sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlsplit
DEFAULT_PORT = 9420
# Body-size cap matching supervise_server.py's 1 MiB limit.
MAX_BODY_BYTES = 1 * 1024 * 1024
class GitHttpHandler(BaseHTTPRequestHandler):
server_version = "bot-bottle-git-http/1"
def do_GET(self) -> None:
self._run_backend()
def do_POST(self) -> None:
self._run_backend()
def _run_backend(self) -> None:
parsed = urlsplit(self.path)
if self._is_upload_pack(parsed.path, parsed.query):
repo_dir = self._repo_dir(parsed.path)
if repo_dir is None:
self.send_error(404)
return
hook_path = os.environ.get(
"GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook",
)
peer = self.client_address[0]
hook = subprocess.run(
[hook_path, "upload-pack", str(repo_dir), peer, peer],
capture_output=True,
check=False,
)
if hook.returncode != 0:
detail = (hook.stderr or hook.stdout).decode(
"utf-8", errors="replace",
).rstrip()
if detail:
for line in detail.splitlines():
self.log_message("access-hook denied %s: %s",
parsed.path, line)
else:
self.log_message(
"access-hook denied %s: exit=%d (no output)",
parsed.path, hook.returncode,
)
self.send_response(403)
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.end_headers()
self.wfile.write(hook.stderr or hook.stdout)
return
env = os.environ.copy()
env.update({
"GIT_PROJECT_ROOT": os.environ.get("GIT_PROJECT_ROOT", "/git"),
"GIT_HTTP_EXPORT_ALL": "1",
"REQUEST_METHOD": self.command,
"PATH_INFO": parsed.path,
"QUERY_STRING": parsed.query,
"CONTENT_TYPE": self.headers.get("content-type", ""),
"CONTENT_LENGTH": self.headers.get("content-length", "0"),
"REMOTE_ADDR": self.client_address[0],
"REMOTE_PORT": str(self.client_address[1]),
"REMOTE_USER": "",
"SERVER_NAME": self.server.server_name,
"SERVER_PORT": str(self.server.server_port),
"SERVER_PROTOCOL": self.request_version,
})
for header, variable in (
("accept", "HTTP_ACCEPT"),
("content-encoding", "HTTP_CONTENT_ENCODING"),
("git-protocol", "HTTP_GIT_PROTOCOL"),
("user-agent", "HTTP_USER_AGENT"),
):
value = self.headers.get(header)
if value:
env[variable] = value
raw_length = self.headers.get("content-length", "0") or "0"
try:
length = int(raw_length)
except ValueError:
self.send_error(400, "Bad Content-Length")
return
if length < 0:
self.send_error(400, "Negative Content-Length")
return
if length > MAX_BODY_BYTES:
self.send_error(413, "Request body too large")
return
body = self.rfile.read(length) if length else b""
proc = subprocess.run(
["git", "http-backend"],
input=body,
env=env,
capture_output=True,
check=False,
)
self._write_cgi_response(proc.stdout)
def _repo_dir(self, path: str) -> Path | None:
root = Path(os.environ.get("GIT_PROJECT_ROOT", "/git")).resolve()
relative = path.lstrip("/").split(".git", 1)[0] + ".git"
candidate = (root / relative).resolve()
if root not in (candidate, *candidate.parents):
return None
if not candidate.is_dir():
return None
return candidate
@staticmethod
def _is_upload_pack(path: str, query: str) -> bool:
if path.endswith("/git-upload-pack"):
return True
if path.endswith("/info/refs"):
return any(
pair == "service=git-upload-pack"
for pair in query.split("&")
)
return False
def _write_cgi_response(self, raw: bytes) -> None:
head, sep, body = raw.partition(b"\r\n\r\n")
line_sep = b"\r\n"
if not sep:
head, sep, body = raw.partition(b"\n\n")
line_sep = b"\n"
status = 200
headers: list[tuple[str, str]] = []
for line in head.split(line_sep):
if not line:
continue
key, _, value = line.decode("latin1").partition(":")
value = value.strip()
if key.lower() == "status":
status = int(value.split()[0])
else:
headers.append((key, value))
self.send_response(status)
for key, value in headers:
self.send_header(key, value)
self.end_headers()
self.wfile.write(body)
def log_message(self, fmt: str, *args: object) -> None:
sys.stdout.write(fmt % args + "\n")
sys.stdout.flush()
def main() -> int:
port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT)))
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
sys.stdout.write(f"git-http listening on 0.0.0.0:{port}\n")
sys.stdout.flush()
server.serve_forever()
return 0
if __name__ == "__main__":
raise SystemExit(main())
+71 -1054
View File
File diff suppressed because it is too large Load Diff
+166
View File
@@ -0,0 +1,166 @@
"""Agent configuration manifest dataclasses."""
from __future__ import annotations
from dataclasses import dataclass
from typing import cast
from .agent_provider import PROVIDER_TEMPLATES
from .manifest_util import ManifestError, as_json_object
from .manifest_git import GitUser
from .manifest_schema import AGENT_MODEL_KEYS
@dataclass(frozen=True)
class AgentProvider:
"""Provider/template for the agent process inside a bottle.
`template` selects a built-in launch/runtime contract. `dockerfile`
optionally points at a custom agent-image Dockerfile while leaving
bot-bottle's sidecar infrastructure intact.
`auth_token` names the host env var that holds the provider's OAuth
token (Claude only). The provisioner injects a provider-owned egress
route for api.anthropic.com that re-injects this token as the Bearer
header, and sets a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent
so the Claude Code CLI starts.
`forward_host_credentials` forwards the host Codex auth token into
the egress sidecar (Codex only).
"""
template: str = "claude"
dockerfile: str = ""
auth_token: str = ""
forward_host_credentials: bool = False
@classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider":
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
for k in d:
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
f"allowed: template, dockerfile, auth_token, forward_host_credentials"
)
template = d.get("template", "claude")
if not isinstance(template, str) or not template:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.template must be a "
f"non-empty string"
)
if template not in PROVIDER_TEMPLATES:
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.template {template!r} "
f"is not one of {', '.join(sorted(PROVIDER_TEMPLATES))}"
)
dockerfile = d.get("dockerfile", "")
if not isinstance(dockerfile, str):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.dockerfile must be a "
f"string (was {type(dockerfile).__name__})"
)
auth_token = d.get("auth_token", "")
if not isinstance(auth_token, str):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.auth_token must be a "
f"string (was {type(auth_token).__name__})"
)
if auth_token and template != "claude":
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.auth_token is only "
f"supported for template 'claude'"
)
forward_host_credentials = d.get("forward_host_credentials", False)
if not isinstance(forward_host_credentials, bool):
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
f"must be a boolean (was {type(forward_host_credentials).__name__})"
)
if forward_host_credentials and template != "codex":
raise ManifestError(
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
"is currently only supported for template 'codex'"
)
return cls(
template=template,
dockerfile=dockerfile,
auth_token=auth_token,
forward_host_credentials=forward_host_credentials,
)
@dataclass(frozen=True)
class Agent:
bottle: str
skills: tuple[str, ...] = ()
prompt: str = ""
# Per-agent git identity (issue #94). Overlays the referenced
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
# `user` is allowed at the agent level; `repos` stays bottle-only
# because it carries credentials and host trust.
git_user: GitUser = GitUser()
@classmethod
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent":
d = as_json_object(raw, f"agent '{name}'")
unknown = set(d.keys()) - AGENT_MODEL_KEYS
if unknown:
allowed = ", ".join(sorted(AGENT_MODEL_KEYS))
raise ManifestError(
f"agent '{name}' has unknown key(s) {sorted(unknown)}; "
f"allowed keys are {allowed}."
)
bottle = d.get("bottle")
if not isinstance(bottle, str) or not bottle:
raise ManifestError(f"agent '{name}' must declare a 'bottle' field naming a defined bottle")
if bottle not in bottle_names:
available = ", ".join(sorted(bottle_names)) or "(none defined)"
raise ManifestError(
f"agent '{name}' references bottle '{bottle}', which is not defined. "
f"Available: {available}"
)
skills: tuple[str, ...] = ()
skills_raw = d.get("skills")
if skills_raw is not None:
if not isinstance(skills_raw, list):
raise ManifestError(f"agent '{name}' skills must be an array (was {type(skills_raw).__name__})")
collected: list[str] = []
skills_list = cast(list[object], skills_raw)
for i, skill in enumerate(skills_list):
if not isinstance(skill, str):
raise ManifestError(
f"agent '{name}' skills[{i}] must be a string "
f"(was {type(skill).__name__})"
)
collected.append(skill)
skills = tuple(collected)
prompt_raw = d.get("prompt")
if prompt_raw is None:
prompt = ""
elif isinstance(prompt_raw, str):
prompt = prompt_raw
else:
raise ManifestError(f"agent '{name}' prompt must be a string (was {type(prompt_raw).__name__})")
# git-gate: agents may declare only `git-gate.user` (name/email).
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
git_user = GitUser()
git_raw = d.get("git-gate")
if git_raw is not None:
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
for k in gd.keys():
if k != "user":
raise ManifestError(
f"agent '{name}' git-gate.{k} is not allowed at the "
f"agent level; only git-gate.user (name/email) may be "
f"set on an agent. git-gate.repos is bottle-only "
f"(it carries credentials and host trust)."
)
if "user" in gd:
git_user = GitUser.from_dict(name, gd["user"])
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
+286
View File
@@ -0,0 +1,286 @@
"""Egress routing manifest dataclasses and helpers."""
from __future__ import annotations
import ipaddress
from dataclasses import dataclass, field
from typing import cast
from .manifest_util import ManifestError, as_json_object
# Auth schemes for the egress route's optional `auth` block.
# Same values cred-proxy accepts today; `token` sidesteps the Gitea
# token-not-Bearer quirk (go-gitea/gitea#16734).
EGRESS_AUTH_SCHEMES = ("Bearer", "token")
def validate_egress_routes(
bottle_name: str,
routes: tuple[EgressRoute, ...],
) -> None:
"""Cross-validation for `bottle.egress.routes`: hosts must be unique.
The proxy matches by exact-host (v1); duplicate hosts leave the
route choice ambiguous so we reject them up front.
No cross-validation against `bottle.git-gate.repos` is performed.
git-gate (SSH push/fetch) and egress (HTTPS) broker different
protocols; declaring both for the same host is a legitimate dev
setup."""
seen_hosts: dict[str, None] = {}
for r in routes:
key = r.Host.lower()
if key in seen_hosts:
raise ManifestError(
f"bottle '{bottle_name}' egress.routes has duplicate host "
f"{r.Host!r}; each host must be unique on the proxy."
)
seen_hosts[key] = None
@dataclass(frozen=True)
class PipelockRoutePolicy:
"""Per-route pipelock policy overrides.
`TlsPassthrough` adds the route host to pipelock's
`tls_interception.passthrough_domains`, so pipelock still enforces
the hostname allowlist but does not MITM/decrypt request bodies or
headers for that host.
`SsrfIpAllowlist` adds explicit IPs/CIDRs to pipelock's SSRF
allowlist for private/internal destinations behind this route.
"""
TlsPassthrough: bool = False
SsrfIpAllowlist: tuple[str, ...] = ()
@classmethod
def from_dict(
cls, bottle_name: str, idx: int, raw: object,
) -> "PipelockRoutePolicy":
label = f"bottle '{bottle_name}' egress.routes[{idx}] pipelock"
d = as_json_object(raw, label)
for k in d:
if k not in ("tls_passthrough", "ssrf_ip_allowlist"):
raise ManifestError(
f"{label} has unknown key {k!r}; "
f"only 'tls_passthrough' and 'ssrf_ip_allowlist' "
f"are accepted"
)
tls_passthrough_raw = d.get("tls_passthrough", False)
if not isinstance(tls_passthrough_raw, bool):
raise ManifestError(
f"{label}.tls_passthrough must be a boolean "
f"(was {type(tls_passthrough_raw).__name__})"
)
ssrf_raw = d.get("ssrf_ip_allowlist", [])
if not isinstance(ssrf_raw, list):
raise ManifestError(
f"{label}.ssrf_ip_allowlist must be an array "
f"(was {type(ssrf_raw).__name__})"
)
ssrf_ip_allowlist: list[str] = []
for j, item in enumerate(ssrf_raw):
if not isinstance(item, str) or not item:
raise ManifestError(
f"{label}.ssrf_ip_allowlist[{j}] must be a non-empty "
f"string (was {type(item).__name__})"
)
try:
ipaddress.ip_network(item, strict=False)
except ValueError as e:
raise ManifestError(
f"{label}.ssrf_ip_allowlist[{j}] must be an IP address "
f"or CIDR (was {item!r}): {e}"
)
ssrf_ip_allowlist.append(item)
return cls(
TlsPassthrough=tls_passthrough_raw,
SsrfIpAllowlist=tuple(ssrf_ip_allowlist),
)
@dataclass(frozen=True)
class EgressRoute:
"""One route on the per-bottle egress sidecar (PRD 0017).
`Host` matches the request's hostname (case-insensitive). The
optional `PathAllowlist` constrains the URL path to a set of
prefixes; empty tuple means no path-level filtering. The optional
`AuthScheme` / `TokenRef` pair drives credential injection:
when set, the proxy strips any inbound Authorization and injects
`<AuthScheme> <value-of-host-env-named-by-TokenRef>`. When the
manifest's `auth` block is omitted both fields are empty strings —
no Authorization is written, no token forwarded.
`Role` is reserved for future use; all role strings are currently
rejected by the validator.
Validation rules (enforced in `from_dict`):
- `host` required, non-empty.
- `path_allowlist` optional, list of absolute path prefixes.
- `auth` optional. If present, MUST carry both `scheme` and
`token_ref` as non-empty strings; an empty `auth: {}` is an
error rather than a synonym for "no auth" (omit `auth` for
that case).
- `role` optional, reserved any non-empty value is rejected.
"""
Host: str
PathAllowlist: tuple[str, ...] = ()
AuthScheme: str = ""
TokenRef: str = ""
Role: tuple[str, ...] = ()
Pipelock: PipelockRoutePolicy = field(default_factory=PipelockRoutePolicy)
@classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
label = f"bottle '{bottle_name}' egress.routes[{idx}]"
d = as_json_object(raw, label)
host = d.get("host")
if not isinstance(host, str) or not host:
raise ManifestError(f"{label} missing required string field 'host'")
path_allow_raw = d.get("path_allowlist")
prefixes: tuple[str, ...] = ()
if path_allow_raw is not None:
if not isinstance(path_allow_raw, list):
raise ManifestError(
f"{label} path_allowlist must be an array "
f"(was {type(path_allow_raw).__name__})"
)
path_list = cast(list[object], path_allow_raw)
collected: list[str] = []
for j, p in enumerate(path_list):
if not isinstance(p, str):
raise ManifestError(
f"{label} path_allowlist[{j}] must be a string "
f"(was {type(p).__name__})"
)
if not p.startswith("/"):
raise ManifestError(
f"{label} path_allowlist[{j}] {p!r} must be an "
f"absolute path prefix starting with '/'"
)
collected.append(p)
prefixes = tuple(collected)
auth_scheme = ""
token_ref = ""
if "auth" in d:
auth_raw = d.get("auth")
auth_d = as_json_object(auth_raw, f"{label} auth")
if not auth_d:
raise ManifestError(
f"{label} auth is empty ({{}}); omit the 'auth' key "
f"entirely if this route is unauthenticated. Otherwise "
f"both 'scheme' and 'token_ref' are required."
)
auth_scheme_raw = auth_d.get("scheme")
if not isinstance(auth_scheme_raw, str) or not auth_scheme_raw:
raise ManifestError(
f"{label} auth.scheme is required when 'auth' is set "
f"(non-empty string)"
)
if auth_scheme_raw not in EGRESS_AUTH_SCHEMES:
raise ManifestError(
f"{label} auth.scheme {auth_scheme_raw!r} is not one of "
f"{', '.join(EGRESS_AUTH_SCHEMES)}"
)
token_ref_raw = auth_d.get("token_ref")
if not isinstance(token_ref_raw, str) or not token_ref_raw:
raise ManifestError(
f"{label} auth.token_ref is required when 'auth' is set "
f"(name of the host env var holding the token value)"
)
for k in auth_d:
if k not in ("scheme", "token_ref"):
raise ManifestError(
f"{label} auth has unknown key {k!r}; "
f"only 'scheme' and 'token_ref' are accepted"
)
auth_scheme = auth_scheme_raw
token_ref = token_ref_raw
role_raw = d.get("role")
roles: tuple[str, ...] = ()
if role_raw is None:
roles = ()
elif isinstance(role_raw, str):
roles = (role_raw,)
elif isinstance(role_raw, list):
role_list = cast(list[object], role_raw)
collected_roles: list[str] = []
for r in role_list:
if not isinstance(r, str):
raise ManifestError(f"{label} role items must be strings (got {type(r).__name__})")
collected_roles.append(r)
roles = tuple(collected_roles)
else:
raise ManifestError(
f"{label} role must be a string or a list of strings "
f"(was {type(role_raw).__name__})"
)
if roles:
raise ManifestError(
f"{label} role {roles[0]!r} is not accepted; "
f"the 'role' field is reserved for future use"
)
pipelock = (
PipelockRoutePolicy.from_dict(bottle_name, idx, d["pipelock"])
if "pipelock" in d
else PipelockRoutePolicy()
)
for k in d:
if k not in ("host", "path_allowlist", "auth", "role", "pipelock"):
raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are "
f"'host', 'path_allowlist', 'auth', 'role', 'pipelock'"
)
return cls(
Host=host,
PathAllowlist=prefixes,
AuthScheme=auth_scheme,
TokenRef=token_ref,
Role=roles,
Pipelock=pipelock,
)
@dataclass(frozen=True)
class EgressConfig:
"""Per-bottle egress configuration. Today this is just the
route table; the nesting under `egress:` leaves room for
per-bottle proxy settings (port override, log level, etc.) in
follow-ups."""
routes: tuple[EgressRoute, ...] = ()
@classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig":
d = as_json_object(raw, f"bottle '{bottle_name}' egress")
routes_raw = d.get("routes")
routes: tuple[EgressRoute, ...] = ()
if routes_raw is not None:
if not isinstance(routes_raw, list):
raise ManifestError(
f"bottle '{bottle_name}' egress.routes must be an array "
f"(was {type(routes_raw).__name__})"
)
routes_list = cast(list[object], routes_raw)
routes = tuple(
EgressRoute.from_dict(bottle_name, i, entry)
for i, entry in enumerate(routes_list)
)
validate_egress_routes(bottle_name, routes)
for k in d:
if k != "routes":
raise ManifestError(
f"bottle '{bottle_name}' egress has unknown key {k!r}; "
f"only 'routes' is accepted"
)
return cls(routes=routes)
+142
View File
@@ -0,0 +1,142 @@
"""Internal bottle `extends:` resolution for manifests."""
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .manifest import Bottle, GitEntry
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
"""Apply `extends:` chains and return resolved Bottle objects."""
cache: dict[str, Bottle] = {}
for name in raws:
if name not in cache:
_resolve_one_bottle(name, raws, cache, ())
return cache
def _resolve_one_bottle(
name: str,
raws: dict[str, dict[str, object]],
cache: dict[str, Bottle],
seen: tuple[str, ...],
) -> Bottle:
from .manifest import Bottle, ManifestError
if name in cache:
return cache[name]
if name in seen:
chain = " -> ".join(seen + (name,))
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
raw = raws[name]
parent_name_raw = raw.get("extends")
# Strip `extends:` before passing to Bottle.from_dict so it
# is not accidentally treated as a real Bottle field by future
# schema additions. It is only meaningful here.
child_raw = {k: v for k, v in raw.items() if k != "extends"}
if parent_name_raw is None:
bottle = Bottle.from_dict(name, child_raw)
cache[name] = bottle
return bottle
if not isinstance(parent_name_raw, str):
raise ManifestError(
f"bottle '{name}' extends must be a string "
f"(was {type(parent_name_raw).__name__})"
)
parent_name: str = parent_name_raw
if parent_name == name:
raise ManifestError(
f"bottle '{name}' extends itself; remove the "
f"self-reference"
)
if parent_name not in raws:
avail = ", ".join(sorted(raws.keys())) or "(none)"
raise ManifestError(
f"bottle '{name}' extends '{parent_name}' which is not "
f"defined. Available bottles: {avail}"
)
parent = _resolve_one_bottle(parent_name, raws, cache, seen + (name,))
bottle = _merge_bottles(parent, child_raw, name)
cache[name] = bottle
return bottle
def _merge_bottles(
parent: Bottle,
child_raw: dict[str, object],
name: str,
) -> Bottle:
"""Apply PRD 0025 merge rules."""
from .manifest import Bottle, GitUser
from .manifest_egress import validate_egress_routes
# Parse the child's declared fields into a Bottle (with the
# usual defaults for anything missing). Validation runs the same
# way it would for a leaf bottle: typos / wrong types die here.
child = Bottle.from_dict(name, child_raw)
# env: dict merge, child wins on collision.
merged_env = {**parent.env, **child.env}
# git-gate.user: per-field overlay. Each non-empty field on child
# wins; empties fall through to parent. The default GitUser()
# is two empty strings, so a child that omits git-gate.user
# inherits the parent's user verbatim.
merged_git_user = GitUser(
name=child.git_user.name or parent.git_user.name,
email=child.git_user.email or parent.git_user.email,
)
# git-gate.repos: missing means inherit; an explicit empty object
# clears; otherwise parent and child merge by UpstreamHost with
# child entries replacing duplicate hosts.
if _child_declares_git_gate_repos(child_raw):
merged_git = _merge_git_remotes(parent.git, child.git) if child.git else ()
else:
merged_git = parent.git
# Presence-driven full-replace for the remaining list-valued +
# scalar fields.
merged_egress = child.egress if "egress" in child_raw else parent.egress
merged_agent_provider = (
child.agent_provider
if "agent_provider" in child_raw
else parent.agent_provider
)
merged_supervise = (
child.supervise if "supervise" in child_raw else parent.supervise
)
validate_egress_routes(name, merged_egress.routes)
return Bottle(
env=merged_env,
agent_provider=merged_agent_provider,
git=merged_git,
git_user=merged_git_user,
egress=merged_egress,
supervise=merged_supervise,
)
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
from .manifest_util import as_json_object
git_raw = child_raw.get("git-gate")
if git_raw is None:
return False
git_obj = as_json_object(git_raw, "child git-gate")
return "repos" in git_obj
def _merge_git_remotes(
parent: tuple[GitEntry, ...],
child: tuple[GitEntry, ...],
) -> tuple[GitEntry, ...]:
by_host = {entry.UpstreamHost: entry for entry in parent}
for entry in child:
by_host[entry.UpstreamHost] = entry
return tuple(by_host.values())
+222
View File
@@ -0,0 +1,222 @@
"""Git-related manifest dataclasses and helpers."""
from __future__ import annotations
import re
from dataclasses import dataclass
from .manifest_util import ManifestError, as_json_object
# Shell-safe characters for git-gate repo names. Names are embedded in
# the generated entrypoint shell script (shlex.quote is the primary
# defence; this regex is belt-and-suspenders and documents intent).
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
def _opt_str(value: object, label: str) -> str:
if value is None:
return ""
if not isinstance(value, str):
raise ManifestError(f"{label} must be a string (was {type(value).__name__})")
return value
def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
"""Parse `ssh://user@host[:port]/path` into (user, host, port, path).
Dies if `url` doesn't match the ssh:// shape v1 supports. Default
port is 22 (matches OpenSSH)."""
if not url.startswith("ssh://"):
raise ManifestError(f"{label} must be an ssh:// URL (was {url!r})")
rest = url[len("ssh://"):]
if "@" not in rest:
raise ManifestError(f"{label} must include a user (e.g. ssh://git@host/path.git); was {url!r}")
user, _, hostpart = rest.partition("@")
if not user:
raise ManifestError(f"{label} user is empty in {url!r}")
if "/" not in hostpart:
raise ManifestError(f"{label} must include a path (e.g. ssh://git@host/path.git); was {url!r}")
hostport, _, path = hostpart.partition("/")
if not path:
raise ManifestError(f"{label} path is empty in {url!r}")
if ":" in hostport:
host, _, port = hostport.partition(":")
if not port.isdigit():
raise ManifestError(f"{label} port must be numeric in {url!r}")
else:
host = hostport
port = "22"
if not host:
raise ManifestError(f"{label} host is empty in {url!r}")
return (user, host, port, path)
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None:
seen: dict[str, None] = {}
for g in git:
if g.Name in seen:
raise ManifestError(
f"bottle '{bottle_name}' git-gate.repos has duplicate name '{g.Name}'; "
f"each entry maps to a distinct bare repo on the gate."
)
seen[g.Name] = None
@dataclass(frozen=True)
class GitEntry:
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
talk to. `Upstream` is the real remote URL the agent would push to
if there were no gate; the gate hosts a bare repo at /git/<Name>.git
and `IdentityFile` is the SSH key the gate uses to push that repo
upstream after gitleaks passes. The agent itself never holds the
upstream credential.
The Upstream URL is parsed once at construction and the pieces are
stashed in the `Upstream*` fields so the git-gate render step
doesn't have to re-parse.
Manifest source: `git-gate.repos.<Name>` (PRD 0047). The YAML keys
are `url`, `identity`, and `host_key`; the internal field names are
stable across that rename."""
Name: str
Upstream: str
IdentityFile: str
KnownHostKey: str = ""
RemoteKey: str = ""
UpstreamUser: str = ""
UpstreamHost: str = ""
UpstreamPort: str = ""
UpstreamPath: str = ""
@classmethod
def from_repos_entry(
cls, bottle_name: str, repo_name: str, raw: object
) -> "GitEntry":
"""Parse one entry from `git-gate.repos.<repo_name>`.
YAML keys: `url` (required), `identity` (required),
`host_key` (optional). The repo_name becomes `Name`."""
if not repo_name:
raise ManifestError(
f"bottle '{bottle_name}' git-gate.repos has an empty key"
)
if not _GIT_NAME_RE.match(repo_name):
raise ManifestError(
f"bottle '{bottle_name}' git-gate.repos name {repo_name!r} is invalid; "
f"allowed characters: A-Z a-z 0-9 . _ -"
)
label = f"git-gate.repos[{repo_name!r}]"
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
for k in d:
if k not in {"url", "identity", "host_key"}:
raise ManifestError(
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
f"allowed: url, identity, host_key"
)
upstream = d.get("url")
if not isinstance(upstream, str) or not upstream:
raise ManifestError(
f"bottle '{bottle_name}' {label} missing required string field 'url'"
)
ident = d.get("identity")
if not isinstance(ident, str) or not ident:
raise ManifestError(
f"bottle '{bottle_name}' {label} missing required string field 'identity'"
)
khk = _opt_str(
d.get("host_key"),
f"bottle '{bottle_name}' {label} host_key",
)
user, host, port, path = parse_git_upstream(
upstream, f"bottle '{bottle_name}' {label} url"
)
return cls(
Name=repo_name,
Upstream=upstream,
IdentityFile=ident,
KnownHostKey=khk,
RemoteKey=host,
UpstreamUser=user,
UpstreamHost=host,
UpstreamPort=port,
UpstreamPath=path,
)
@dataclass(frozen=True)
class GitUser:
"""Per-bottle `git config --global user.name` / `user.email`
pair (issue #86). The agent's commits inside the bottle are
attributed to this identity rather than the agent image's
image-baked default (no user, or whatever the image dropped
in). Either or both fields can be set independently.
`from_dict` is forgiving on shape (a single missing field is
fine we just skip that config line at provisioning) but
strict on types (string-or-die)."""
name: str = ""
email: str = ""
@classmethod
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
for k in d.keys():
if k not in {"name", "email"}:
raise ManifestError(
f"bottle '{bottle_name}' git-gate.user has unknown key {k!r}; "
f"allowed: name, email"
)
name = d.get("name", "")
email = d.get("email", "")
if not isinstance(name, str):
raise ManifestError(
f"bottle '{bottle_name}' git-gate.user.name must be a string "
f"(was {type(name).__name__})"
)
if not isinstance(email, str):
raise ManifestError(
f"bottle '{bottle_name}' git-gate.user.email must be a string "
f"(was {type(email).__name__})"
)
if not name and not email:
raise ManifestError(
f"bottle '{bottle_name}' git-gate.user is set but neither "
f"name nor email is non-empty; remove the block or "
f"fill at least one field."
)
return cls(name=name, email=email)
def is_empty(self) -> bool:
return not self.name and not self.email
def parse_git_gate_config(
bottle_name: str,
raw: object,
) -> tuple[tuple[GitEntry, ...], GitUser]:
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
for k in d.keys():
if k not in {"user", "repos"}:
raise ManifestError(
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
f"allowed: user, repos"
)
git_user = (
GitUser.from_dict(bottle_name, d["user"])
if "user" in d
else GitUser()
)
git: tuple[GitEntry, ...] = ()
repos_raw = d.get("repos")
if repos_raw is not None:
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
git = tuple(
GitEntry.from_repos_entry(bottle_name, name, entry)
for name, entry in repos.items()
)
validate_unique_git_names(bottle_name, git)
return git, git_user
+105
View File
@@ -0,0 +1,105 @@
"""Internal per-file Markdown manifest loader."""
from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
from .log import warn
from .manifest_schema import (
entity_name_from_path,
validate_agent_frontmatter_keys,
validate_bottle_frontmatter_keys,
)
from .yaml_subset import YamlSubsetError, parse_frontmatter
if TYPE_CHECKING:
from .manifest import Agent, Bottle
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
not. The manifest format changed in PRD 0011 and we do not want
to silently leave the JSON content unused."""
from .manifest import ManifestError
legacy = dir_path / "bot-bottle.json"
if legacy.is_file() and not md_dir.exists():
raise ManifestError(
f"found {legacy} but {md_dir} does not exist. The manifest "
f"format changed in PRD 0011 — rewrite the JSON content "
f"as per-file Markdown under {md_dir}/bottles/ and "
f"{md_dir}/agents/. See README.md for the schema. "
f"({label})"
)
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
`{name: Bottle}`. Missing dir returns an empty dict."""
from .manifest import ManifestError
from .manifest_extends import resolve_bottles
raws: dict[str, dict[str, object]] = {}
if not bottles_dir.is_dir():
return {}
for path in sorted(bottles_dir.glob("*.md")):
name = entity_name_from_path(path)
if name is None:
warn(
f"skipping {path}: filename must match "
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
)
continue
try:
fm, _body = parse_frontmatter(path.read_text())
except OSError as e:
raise ManifestError(f"could not read {path}: {e}")
except YamlSubsetError as e:
raise ManifestError(f"{path}: {e}")
validate_bottle_frontmatter_keys(path, fm.keys())
raws[name] = fm
return resolve_bottles(raws)
def load_agents_from_dir(
agents_dir: Path,
bottle_names: set[str],
*,
source: str,
) -> dict[str, Agent]:
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
`{name: Agent}`. The Markdown body becomes the agent's prompt.
Missing dir returns an empty dict."""
from .manifest import Agent, ManifestError
out: dict[str, Agent] = {}
if not agents_dir.is_dir():
return out
for path in sorted(agents_dir.glob("*.md")):
name = entity_name_from_path(path)
if name is None:
warn(
f"skipping {path}: filename must match "
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
)
continue
try:
fm, body = parse_frontmatter(path.read_text())
except OSError as e:
raise ManifestError(f"could not read {path}: {e}")
except YamlSubsetError as e:
raise ManifestError(f"{path}: {e}")
validate_agent_frontmatter_keys(path, fm.keys())
# Build the dict Agent.from_dict expects. The body becomes
# prompt; Claude Code passthrough fields stay in fm and get
# ignored by Agent.from_dict (reads bottle/skills/git-gate/prompt).
agent_dict: dict[str, object] = {
"bottle": fm.get("bottle"),
"skills": fm.get("skills", []),
"prompt": body.strip(),
}
if "git-gate" in fm:
agent_dict["git-gate"] = fm["git-gate"]
out[name] = Agent.from_dict(name, agent_dict, bottle_names)
return out
+70
View File
@@ -0,0 +1,70 @@
"""Internal manifest schema policy helpers."""
from __future__ import annotations
import re
from pathlib import Path
# Filename-as-key uses kebab-case ASCII. The first character is a
# letter so we don't conflict with hidden files / Markdown special
# names (`.md`, `_template.md`, etc.). Filenames that fail this
# pattern are skipped with a warning rather than crashing the load.
_FILENAME_RX = re.compile(r"^[a-z][a-z0-9-]*$")
# Frontmatter keys we accept on each entity. Anything not in these
# sets dies with a "did you mean" pointer: typos should not silently
# ghost into an empty config.
BOTTLE_KEYS = frozenset(
{"env", "extends", "agent_provider", "git-gate", "egress", "supervise"}
)
AGENT_KEYS_REQUIRED = frozenset({"bottle"})
AGENT_KEYS_OPTIONAL = frozenset({"skills", "git-gate"})
# Claude Code subagent fields bot-bottle ignores at launch but does
# not reject. This lets the same file double as
# `~/.claude/agents/*.md` without modification.
CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS = frozenset({
"name", "description", "model", "color", "memory",
})
AGENT_KEYS = (
AGENT_KEYS_REQUIRED | AGENT_KEYS_OPTIONAL | CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS
)
AGENT_MODEL_KEYS = AGENT_KEYS | frozenset({"prompt"})
def entity_name_from_path(path: Path) -> str | None:
"""Return the entity name implied by the filename, or None if the
filename does not fit the [a-z][a-z0-9-]* convention."""
if path.suffix != ".md":
return None
stem = path.stem
if not _FILENAME_RX.match(stem):
return None
return stem
def validate_bottle_frontmatter_keys(path: Path, keys: object) -> None:
_validate_frontmatter_keys("bottle", path, keys, BOTTLE_KEYS)
def validate_agent_frontmatter_keys(path: Path, keys: object) -> None:
_validate_frontmatter_keys("agent", path, keys, AGENT_KEYS)
def _validate_frontmatter_keys(
kind: str,
path: Path,
keys: object,
allowed_keys: frozenset[str],
) -> None:
from .manifest_util import ManifestError
key_set = set(keys)
unknown = key_set - allowed_keys
if unknown:
allowed = ", ".join(sorted(allowed_keys))
raise ManifestError(
f"{kind} file {path}: unknown frontmatter key(s) "
f"{sorted(unknown)}; allowed keys are {allowed}."
)
+24
View File
@@ -0,0 +1,24 @@
"""Shared manifest primitives used by all manifest sub-modules."""
from __future__ import annotations
from typing import cast
class ManifestError(Exception):
"""A manifest file (or the manifest tree) is invalid."""
def as_json_object(value: object, label: str) -> dict[str, object]:
"""Assert that `value` is a JSON object (str-keyed dict) and return
a view typed as `dict[str, object]` so downstream `.get(...)` calls
have a typed surface."""
if not isinstance(value, dict):
raise ManifestError(f"{label} must be a JSON object (was {type(value).__name__})")
items = cast(dict[object, object], value)
out: dict[str, object] = {}
for k, v in items.items():
if not isinstance(k, str):
raise ManifestError(f"{label} keys must be strings (found {type(k).__name__})")
out[k] = v
return out
+228 -30
View File
@@ -19,9 +19,8 @@ from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import cast
from .egress import EGRESS_HOSTNAME, egress_routes_for_bottle
from .egress import EGRESS_HOSTNAME, EgressRoute, egress_routes_for_bottle
from .supervise import SUPERVISE_HOSTNAME
from .manifest import Bottle
@@ -50,14 +49,17 @@ PIPELOCK_HOSTNAME = "pipelock"
# --- Allowlist resolution --------------------------------------------------
def pipelock_effective_allowlist(bottle: Bottle) -> list[str]:
def pipelock_effective_allowlist(
bottle: Bottle,
provider_routes: tuple[EgressRoute, ...] = (),
) -> list[str]:
"""Hostnames pipelock allows. Sorted for stability.
Always mirrors `egress_routes_for_bottle(bottle)` egress is the
single allowlist surface, and pipelock's allowlist is the downstream
copy for defense-in-depth + DLP body scanning. For bottles without
any `egress.routes[]` declared, this is empty except for supervise
sidecar traffic when `supervise: true`.
Always mirrors `egress_routes_for_bottle(bottle, provider_routes)`
egress is the single allowlist surface, and pipelock's allowlist is
the downstream copy for defense-in-depth + DLP body scanning. For
bottles without any `egress.routes[]` declared, this is empty except
for supervise sidecar traffic when `supervise: true`.
The supervise sidecar's hostname is auto-added when supervise
is enabled (sibling-sidecar traffic that flows through pipelock
@@ -65,7 +67,7 @@ def pipelock_effective_allowlist(bottle: Bottle) -> list[str]:
`bottle.git` do NOT contribute here git traffic flows
through git-gate (PRD 0008), not pipelock."""
seen: dict[str, None] = {}
for r in egress_routes_for_bottle(bottle):
for r in egress_routes_for_bottle(bottle, provider_routes):
if r.host:
seen.setdefault(r.host, None)
if bottle.supervise:
@@ -98,19 +100,23 @@ def pipelock_seed_phrase_detection_enabled(bottle: Bottle) -> bool:
return False
def pipelock_effective_tls_passthrough(bottle: Bottle) -> list[str]:
def pipelock_effective_tls_passthrough(
bottle: Bottle,
provider_routes: tuple[EgressRoute, ...] = (),
) -> list[str]:
"""Hostnames pipelock should pass through (no TLS MITM).
A route opts in with `pipelock.tls_passthrough: true`. This is
useful for provider API routes where egress injects the
Authorization header after the agent boundary; pipelock still
enforces the host allowlist but does not decrypt and scan that
provider request.
A manifest route opts in with `pipelock.tls_passthrough: true`
(lifted into `EgressRoute.tls_passthrough` in `egress_manifest_routes`).
Provider routes that set `tls_passthrough=True` (e.g. Codex credential
routes where egress injects the host bearer after the agent boundary)
are also included. Both arrive via `egress_routes_for_bottle` no
provider-specific branching needed here.
"""
seen: dict[str, None] = {host: None for host in DEFAULT_TLS_PASSTHROUGH}
for route in bottle.egress.routes:
if route.Pipelock.TlsPassthrough:
seen.setdefault(route.Host, None)
for route in egress_routes_for_bottle(bottle, provider_routes):
if route.tls_passthrough:
seen.setdefault(route.host, None)
return sorted(seen.keys())
@@ -142,6 +148,7 @@ def pipelock_build_config(
ca_cert_path: str = "",
ca_key_path: str = "",
ssrf_ip_allowlist: tuple[str, ...] = (),
provider_routes: tuple[EgressRoute, ...] = (),
) -> dict[str, object]:
"""Build the structured pipelock config dict the sidecar will load.
@@ -171,7 +178,7 @@ def pipelock_build_config(
"version": 1,
"mode": "strict",
"enforce": True,
"api_allowlist": pipelock_effective_allowlist(bottle),
"api_allowlist": pipelock_effective_allowlist(bottle, provider_routes),
"forward_proxy": {"enabled": True},
}
if not pipelock_seed_phrase_detection_enabled(bottle):
@@ -205,7 +212,7 @@ def pipelock_build_config(
"enabled": True,
"ca_cert": ca_cert_path,
"ca_key": ca_key_path,
"passthrough_domains": pipelock_effective_tls_passthrough(bottle),
"passthrough_domains": pipelock_effective_tls_passthrough(bottle, provider_routes),
}
effective_ssrf_ip_allowlist = pipelock_effective_ssrf_ip_allowlist(
bottle, ssrf_ip_allowlist,
@@ -215,6 +222,180 @@ def pipelock_build_config(
return cfg
_PIPELOCK_TOP_LEVEL_KEYS = {
"version",
"mode",
"enforce",
"api_allowlist",
"seed_phrase_detection",
"forward_proxy",
"dlp",
"request_body_scanning",
"tls_interception",
"ssrf",
}
def _pipelock_render_error(section: str, key: str, expected: str) -> ValueError:
return ValueError(
f"pipelock_render_yaml: {section}.{key} must be {expected}"
)
def _reject_unknown_keys(
section: str,
obj: dict[str, object],
allowed: set[str],
) -> None:
for key in sorted(set(obj) - allowed):
raise ValueError(f"pipelock_render_yaml: {section}.{key} is unsupported")
def _required_dict(
obj: dict[str, object],
section: str,
key: str,
) -> dict[str, object]:
value = obj.get(key)
if not isinstance(value, dict):
raise _pipelock_render_error(section, key, "a mapping")
return value
def _required_bool(obj: dict[str, object], section: str, key: str) -> bool:
value = obj.get(key)
if not isinstance(value, bool):
raise _pipelock_render_error(section, key, "a boolean")
return value
def _required_int(obj: dict[str, object], section: str, key: str) -> int:
value = obj.get(key)
if isinstance(value, bool) or not isinstance(value, int):
raise _pipelock_render_error(section, key, "an integer")
return value
def _required_str(obj: dict[str, object], section: str, key: str) -> str:
value = obj.get(key)
if not isinstance(value, str):
raise _pipelock_render_error(section, key, "a string")
return value
def _required_str_list(
obj: dict[str, object],
section: str,
key: str,
) -> list[str]:
value = obj.get(key)
if not isinstance(value, list) or not all(isinstance(v, str) for v in value):
raise _pipelock_render_error(section, key, "a list of strings")
return value
def _optional_str_list(
obj: dict[str, object],
section: str,
key: str,
) -> list[str]:
if key not in obj:
return []
return _required_str_list(obj, section, key)
def _optional_bool(
obj: dict[str, object],
section: str,
key: str,
) -> bool | None:
if key not in obj:
return None
return _required_bool(obj, section, key)
def _optional_str(
obj: dict[str, object],
section: str,
key: str,
) -> str | None:
if key not in obj:
return None
return _required_str(obj, section, key)
def _validate_pipelock_render_config(cfg: dict[str, object]) -> dict[str, object]:
_reject_unknown_keys("config", cfg, _PIPELOCK_TOP_LEVEL_KEYS)
normalized: dict[str, object] = {
"version": _required_int(cfg, "config", "version"),
"mode": _required_str(cfg, "config", "mode"),
"enforce": _required_bool(cfg, "config", "enforce"),
"api_allowlist": _required_str_list(cfg, "config", "api_allowlist"),
}
if "seed_phrase_detection" in cfg:
spd = _required_dict(cfg, "config", "seed_phrase_detection")
_reject_unknown_keys("seed_phrase_detection", spd, {"enabled"})
normalized["seed_phrase_detection"] = {
"enabled": _required_bool(spd, "seed_phrase_detection", "enabled"),
}
fp = _required_dict(cfg, "config", "forward_proxy")
_reject_unknown_keys("forward_proxy", fp, {"enabled"})
normalized["forward_proxy"] = {
"enabled": _required_bool(fp, "forward_proxy", "enabled"),
}
dlp = _required_dict(cfg, "config", "dlp")
_reject_unknown_keys("dlp", dlp, {"include_defaults", "scan_env"})
normalized["dlp"] = {
"include_defaults": _required_bool(dlp, "dlp", "include_defaults"),
"scan_env": _required_bool(dlp, "dlp", "scan_env"),
}
rbs = _required_dict(cfg, "config", "request_body_scanning")
_reject_unknown_keys(
"request_body_scanning",
rbs,
{"action", "scan_headers", "header_mode"},
)
normalized_rbs: dict[str, object] = {
"action": _required_str(rbs, "request_body_scanning", "action"),
}
scan_headers = _optional_bool(rbs, "request_body_scanning", "scan_headers")
if scan_headers is not None:
normalized_rbs["scan_headers"] = scan_headers
header_mode = _optional_str(rbs, "request_body_scanning", "header_mode")
if header_mode is not None:
normalized_rbs["header_mode"] = header_mode
normalized["request_body_scanning"] = normalized_rbs
if "tls_interception" in cfg:
tls = _required_dict(cfg, "config", "tls_interception")
_reject_unknown_keys(
"tls_interception",
tls,
{"enabled", "ca_cert", "ca_key", "passthrough_domains"},
)
normalized["tls_interception"] = {
"enabled": _required_bool(tls, "tls_interception", "enabled"),
"ca_cert": _required_str(tls, "tls_interception", "ca_cert"),
"ca_key": _required_str(tls, "tls_interception", "ca_key"),
"passthrough_domains": _optional_str_list(
tls, "tls_interception", "passthrough_domains",
),
}
if "ssrf" in cfg:
ssrf = _required_dict(cfg, "config", "ssrf")
_reject_unknown_keys("ssrf", ssrf, {"ip_allowlist"})
normalized["ssrf"] = {
"ip_allowlist": _required_str_list(ssrf, "ssrf", "ip_allowlist"),
}
return normalized
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
"""Render a pipelock config dict (as produced by
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
@@ -222,31 +403,38 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
def _bool(b: object) -> str:
return "true" if b else "false"
cfg = _validate_pipelock_render_config(cfg)
lines: list[str] = []
lines.append(f"version: {cfg['version']}")
lines.append(f"mode: {cfg['mode']}")
lines.append(f"enforce: {_bool(cfg['enforce'])}")
lines.append("")
lines.append("api_allowlist:")
for h in cast(list[str], cfg["api_allowlist"]):
api_allowlist = cfg["api_allowlist"]
assert isinstance(api_allowlist, list)
for h in api_allowlist:
lines.append(f' - "{h}"')
lines.append("")
if "seed_phrase_detection" in cfg:
lines.append("seed_phrase_detection:")
spd = cast(dict[str, object], cfg["seed_phrase_detection"])
spd = cfg["seed_phrase_detection"]
assert isinstance(spd, dict)
lines.append(f" enabled: {_bool(spd['enabled'])}")
lines.append("")
lines.append("forward_proxy:")
fp = cast(dict[str, object], cfg["forward_proxy"])
fp = cfg["forward_proxy"]
assert isinstance(fp, dict)
lines.append(f" enabled: {_bool(fp['enabled'])}")
lines.append("")
lines.append("dlp:")
dlp = cast(dict[str, object], cfg["dlp"])
dlp = cfg["dlp"]
assert isinstance(dlp, dict)
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
lines.append("")
lines.append("request_body_scanning:")
rbs = cast(dict[str, object], cfg["request_body_scanning"])
rbs = cfg["request_body_scanning"]
assert isinstance(rbs, dict)
lines.append(f' action: "{rbs["action"]}"')
if "scan_headers" in rbs:
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
@@ -255,11 +443,13 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
if "tls_interception" in cfg:
lines.append("")
lines.append("tls_interception:")
tls = cast(dict[str, object], cfg["tls_interception"])
tls = cfg["tls_interception"]
assert isinstance(tls, dict)
lines.append(f" enabled: {_bool(tls['enabled'])}")
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
lines.append(f' ca_key: "{tls["ca_key"]}"')
passthrough = cast(list[str], tls.get("passthrough_domains", []))
passthrough = tls["passthrough_domains"]
assert isinstance(passthrough, list)
if passthrough:
lines.append(" passthrough_domains:")
for d in passthrough:
@@ -267,9 +457,12 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
if "ssrf" in cfg:
lines.append("")
lines.append("ssrf:")
ssrf = cast(dict[str, object], cfg["ssrf"])
ssrf = cfg["ssrf"]
assert isinstance(ssrf, dict)
lines.append(" ip_allowlist:")
for ip in cast(list[str], ssrf["ip_allowlist"]):
ip_allowlist = ssrf["ip_allowlist"]
assert isinstance(ip_allowlist, list)
for ip in ip_allowlist:
lines.append(f' - "{ip}"')
return "\n".join(lines) + "\n"
@@ -319,7 +512,11 @@ class PipelockProxy:
(`PIPELOCK_CA_CERT_IN_CONTAINER` / `PIPELOCK_CA_KEY_IN_CONTAINER`)."""
def prepare(
self, bottle: Bottle, slug: str, stage_dir: Path
self,
bottle: Bottle,
slug: str,
stage_dir: Path,
provider_routes: tuple[EgressRoute, ...] = (),
) -> PipelockProxyPlan:
"""Write the pipelock yaml config (mode 600) under `stage_dir`
and return the plan for launch. Pure host-side, no docker
@@ -342,6 +539,7 @@ class PipelockProxy:
bottle,
ca_cert_path=PIPELOCK_CA_CERT_IN_CONTAINER,
ca_key_path=PIPELOCK_CA_KEY_IN_CONTAINER,
provider_routes=provider_routes,
)
yaml_path.write_text(pipelock_render_yaml(cfg))
yaml_path.chmod(0o600)
+59 -8
View File
@@ -20,7 +20,7 @@ sick daemon."
Daemon subset is env-driven. The compose renderer narrows it via
`BOT_BOTTLE_SIDECAR_DAEMONS=egress,pipelock` for bottles that
don't use git-gate or supervise. Default: all four.
don't use git-gate or supervise. Default: all daemons.
Stdlib-only by design adding supervisord/s6/runit for four
daemons is heavier than this script.
@@ -98,6 +98,7 @@ _DAEMONS: tuple[_DaemonSpec, ...] = (
"--listen", "0.0.0.0:8888"),
),
_DaemonSpec("git-gate", ("/bin/sh", "/git-gate-entrypoint.sh")),
_DaemonSpec("git-http", ("python3", "/app/git_http_backend.py")),
_DaemonSpec("supervise", ("python3", "/app/supervise_server.py")),
)
@@ -162,6 +163,10 @@ class _Supervisor:
# Names of children that have been logged as having exited
# so we only log each death once across watch-loop ticks.
self._logged_dead: set[str] = set()
# Signal handlers add daemon names here and return quickly.
# The main watch loop drains the set, so repeated restart
# requests for one daemon coalesce into one restart.
self._restart_requested: set[str] = set()
def start_all(self) -> None:
for spec in self.specs:
@@ -172,6 +177,7 @@ class _Supervisor:
if self.shutdown_at is not None:
return
self.shutdown_at = time.monotonic()
self._restart_requested.clear()
_log(f"shutting down ({reason}); forwarding SIGTERM")
for _, p in self.procs:
if p.poll() is None:
@@ -180,6 +186,24 @@ class _Supervisor:
except ProcessLookupError:
pass
def request_restart(self, daemon_name: str) -> bool:
"""Queue a daemon restart for the main loop to process.
Signal handlers use this non-blocking path instead of doing
subprocess lifecycle work directly. Requests coalesce by
daemon name: one pending restart is enough to make the daemon
reread the latest config from disk.
Returns True iff a daemon by that name is known to the
supervisor and shutdown has not started."""
if self.shutdown_at is not None:
_log(f"restart {daemon_name} skipped; supervisor is shutting down")
return False
if not any(spec.name == daemon_name for spec, _ in self.procs):
return False
self._restart_requested.add(daemon_name)
return True
def tick(self) -> bool:
"""One iteration of the watch loop. Returns True when every
child has exited and the supervisor can return.
@@ -187,6 +211,8 @@ class _Supervisor:
A child dying unexpectedly is logged but does NOT initiate
shutdown see the module docstring's failure-policy
section. Shutdown is signal-driven only."""
self._drain_restart_requests()
for spec, p in self.procs:
rc = p.poll()
if rc is None or spec.name in self._logged_dead:
@@ -219,14 +245,37 @@ class _Supervisor:
except ProcessLookupError:
pass
return all(p.poll() is not None for _, p in self.procs)
done = all(p.poll() is not None for _, p in self.procs)
if done:
for _, p in self.procs:
if p.stdout is not None:
p.stdout.close()
return done
def exit_code(self) -> int:
"""Worst child returncode wins. On graceful shutdown every
child is signal-killed (negative returncode) and max()
returns 0; if some child crashed nonzero before the signal
the operator gets that code on container exit."""
return max((p.returncode for _, p in self.procs), default=0)
"""Positive child failures win; otherwise report success.
Python represents signal-terminated children as negative
return codes. A signal-only graceful shutdown should not leak
that platform-specific detail into the container exit status,
but a positive crash before shutdown should remain visible."""
positives = [
p.returncode for _, p in self.procs
if p.returncode is not None and p.returncode > 0
]
return max(positives, default=0)
def _drain_restart_requests(self) -> None:
if self.shutdown_at is not None:
self._restart_requested.clear()
return
requested = tuple(sorted(self._restart_requested))
self._restart_requested.clear()
for daemon_name in requested:
if self.shutdown_at is not None:
self._restart_requested.clear()
return
self.restart_daemon(daemon_name)
def forward_signal(self, sig: int, daemon_name: str) -> bool:
"""Forward a signal to one named child. Used by the SIGHUP
@@ -291,6 +340,8 @@ class _Supervisor:
except ProcessLookupError:
pass
p.wait()
if p.stdout is not None:
p.stdout.close()
self._logged_dead.discard(daemon_name)
new_proc = _spawn(spec)
self.procs[idx] = (spec, new_proc)
@@ -322,7 +373,7 @@ def main(argv: Sequence[str] | None = None) -> int:
# supervisor restarts the pipelock daemon in place (other
# daemons keep running — specifically supervise, whose MCP
# socket would drop on a whole-container `docker restart`).
signal.signal(signal.SIGUSR1, lambda *_: sup.restart_daemon("pipelock"))
signal.signal(signal.SIGUSR1, lambda *_: sup.request_restart("pipelock"))
while not sup.tick():
time.sleep(_POLL_INTERVAL)
+66 -4
View File
@@ -35,6 +35,7 @@ import json
import os
import socketserver
import sys
import time
import typing
import urllib.error
import urllib.parse
@@ -63,6 +64,10 @@ ERR_METHOD_NOT_FOUND = -32601
ERR_INVALID_PARAMS = -32602
ERR_INTERNAL = -32603
DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
EGRESS_LIST_TIMEOUT_SECONDS = 5.0
@dataclass(frozen=True)
class JsonRpcRequest:
@@ -412,6 +417,7 @@ def _validate_and_bundle_egress_route(
class ServerConfig:
bottle_slug: str
queue_dir: Path
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS
def handle_initialize(_params: dict[str, object]) -> dict[str, object]:
@@ -442,7 +448,7 @@ def handle_list_egress_routes(
})
opener = urllib.request.build_opener(proxy_handler)
try:
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=5) as resp:
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=EGRESS_LIST_TIMEOUT_SECONDS) as resp:
body = resp.read().decode("utf-8")
except (urllib.error.URLError, OSError) as e:
return {
@@ -520,7 +526,20 @@ def handle_tools_call(
f"for bottle {config.bottle_slug}; waiting for operator...\n"
)
sys.stderr.flush()
response = _sv.wait_for_response(config.queue_dir, proposal.id)
deadline = time.monotonic() + config.response_timeout_seconds
try:
response = _sv.wait_for_response(
config.queue_dir,
proposal.id,
poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS,
deadline=deadline,
)
except TimeoutError:
text = format_pending_response_text(config.response_timeout_seconds)
return {
"content": [{"type": "text", "text": text}],
"isError": False,
}
_sv.archive_proposal(config.queue_dir, proposal.id)
text = format_response_text(response)
@@ -542,6 +561,16 @@ def format_response_text(response: "_sv.Response") -> str:
return "\n".join(lines)
def format_pending_response_text(timeout_seconds: float) -> str:
return "\n".join([
"status: pending",
(
"notes: operator response timed out after "
f"{timeout_seconds:g}s; proposal remains queued"
),
])
# --- HTTP transport --------------------------------------------------------
@@ -654,10 +683,15 @@ def serve(
queue_dir: Path,
port: int = _sv.SUPERVISE_PORT,
bind: str = "0.0.0.0",
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
) -> typing.NoReturn:
queue_dir.mkdir(parents=True, exist_ok=True)
server = MCPServer((bind, port), MCPHandler)
server.config = ServerConfig(bottle_slug=bottle_slug, queue_dir=queue_dir)
server.config = ServerConfig(
bottle_slug=bottle_slug,
queue_dir=queue_dir,
response_timeout_seconds=response_timeout_seconds,
)
sys.stderr.write(
f"supervise listening on {bind}:{port}; "
f"slug={bottle_slug!r}; queue={queue_dir}; "
@@ -682,9 +716,37 @@ def main(argv: list[str]) -> int:
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
serve(bottle_slug=bottle_slug, queue_dir=queue_dir, port=port, bind=bind)
try:
response_timeout_seconds = _response_timeout_from_env(os.environ)
except ValueError as e:
sys.stderr.write(f"supervise: {e}\n")
return 2
serve(
bottle_slug=bottle_slug,
queue_dir=queue_dir,
port=port,
bind=bind,
response_timeout_seconds=response_timeout_seconds,
)
return 0 # serve() does not return
def _response_timeout_from_env(env: typing.Mapping[str, str]) -> float:
raw = env.get("SUPERVISE_RESPONSE_TIMEOUT_SECONDS", "").strip()
if not raw:
return DEFAULT_RESPONSE_TIMEOUT_SECONDS
try:
value = float(raw)
except ValueError as e:
raise ValueError(
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
) from e
if value <= 0:
raise ValueError(
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
)
return value
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+9
View File
@@ -5,9 +5,18 @@ level deeper, under their backend package."""
from __future__ import annotations
import ipaddress
import os
def is_ip_literal(value: str) -> bool:
try:
ipaddress.ip_address(value)
except ValueError:
return False
return True
def expand_tilde(path: str) -> str:
"""Expand a leading '~' to $HOME. Leaves paths without a leading
tilde unchanged. Falls back to the empty string if $HOME is unset
+52
View File
@@ -0,0 +1,52 @@
"""Backend-neutral plan for porting the operator workspace."""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol
WORKSPACE_DIRNAME = "workspace"
DEFAULT_WORKSPACE_OWNER = "node:node"
DEFAULT_WORKSPACE_MODE = "755"
class WorkspaceSpec(Protocol):
copy_cwd: bool
user_cwd: str
@dataclass(frozen=True)
class WorkspacePlan:
"""Resolved workspace contract shared by all bottle backends."""
enabled: bool
host_path: Path
guest_home: str
guest_path: str
workdir: str
owner: str = DEFAULT_WORKSPACE_OWNER
mode: str = DEFAULT_WORKSPACE_MODE
copy_contents: bool = True
copy_git: bool = True
has_host_git_dir: bool = False
def workspace_plan(spec: WorkspaceSpec, *, guest_home: str) -> WorkspacePlan:
"""Resolve the in-bottle workspace path from CLI intent."""
host_path = Path(spec.user_cwd).expanduser()
if spec.copy_cwd:
guest_path = f"{guest_home.rstrip('/')}/{WORKSPACE_DIRNAME}"
workdir = guest_path
else:
guest_path = guest_home
workdir = guest_home
return WorkspacePlan(
enabled=spec.copy_cwd,
host_path=host_path,
guest_home=guest_home,
guest_path=guest_path,
workdir=workdir,
has_host_git_dir=(host_path / ".git").is_dir(),
)
+1 -6
View File
@@ -83,12 +83,7 @@ for a declared upstream:
- **Manifest field.** `bottle.git` — a list of git remotes the
bottle is allowed to talk to, each with the credential the gate
uses to push upstream. The agent gets no parallel `bottle.ssh`
entry for those upstreams. Each entry may also carry an
`ExtraHosts: { hostname: ip }` map, surfaced to the gate as
`--add-host` so the gate can resolve upstreams whose public DNS
doesn't point at the reachable IP (e.g. Tailscale-only hosts).
The agent-side `insteadOf` rewrite keys off the original hostname,
so the manifest's `Upstream` URL stays human-readable.
entry for those upstreams.
- **Agent-side URL rewrite.** Provisioner emits `~/.gitconfig`
with `[url "<gate-url>"] insteadOf = <real-url>` so every git
operation against the declared upstream (push, fetch, clone,
+1 -2
View File
@@ -88,8 +88,7 @@ the unused path.
- **Pipelock interaction.** Drop the SSH-derived branch from
pipelock's `ssrf.ip_allowlist` build. With no `bottle.ssh`
there is no per-upstream IP carve-out to render; git-gate
has its own egress network and pulls in upstream resolution
via `ExtraHosts` plus DNS.
has its own egress network.
- **Tests.** Delete the ssh-gate unit + integration suites,
the ssh fixtures in `tests/fixtures.py`, and the
shadow-route assertions in `test_manifest_git.py`. Adjust
-2
View File
@@ -274,8 +274,6 @@ git:
Name: bot-bottle
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
IdentityFile: ~/.ssh/gitea-delos-2.pem
ExtraHosts:
gitea.dideric.is: 100.78.141.42
KnownHostKey: ssh-rsa AAAAB3...
egress:
allowlist:
+1 -2
View File
@@ -161,8 +161,7 @@ expectation. (Same model as shell `export` precedence.)
`git.remotes` is also keyed, so it follows dict-style inheritance:
children can override one host without restating every remote. The
remote entry is replaced as a whole on host collision because
`Upstream`, `IdentityFile`, `KnownHostKey`, and `ExtraHosts` are
tightly coupled.
`Upstream`, `IdentityFile`, and `KnownHostKey` are tightly coupled.
The `git.user` dataclass-overlay (each non-empty field wins
individually) is so a parent can declare `git.user.name` and a
+4 -1
View File
@@ -86,7 +86,10 @@ agent_provider:
## Open questions
- The initial Codex auth role is `codex_auth`; it provides a non-secret `OPENAI_API_KEY` placeholder to the agent while egress holds the real token.
- `codex_auth` is retained as a placeholder marker for follow-up Codex
credential-injection work. The Codex template should not inject an
`OPENAI_API_KEY` placeholder; Codex bottles use device/ChatGPT login
state instead.
- Existing state-folder transcript capture is Claude-specific and should remain gated to Claude until the follow-up state/transcript refactor.
## References
@@ -0,0 +1,198 @@
# PRD 0029: Provider auth credentials through egress
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-05-29
- **Issue:** #109
## Summary
Allow provider bottles to inject host credentials into the egress
sidecar without exposing them to the agent. Codex uses
`agent_provider.forward_host_credentials` for ChatGPT/device-login
access tokens. Claude uses `agent_provider.auth_token` to name the host
env var holding its OAuth token, which egress injects on
`api.anthropic.com` requests.
## Problem
Codex bottles can reach OpenAI hosts after they are added to egress, but
requests to Codex's ChatGPT-backed API endpoints still fail with HTTP
403 when the egress route is unauthenticated. The egress proxy strips
agent-originated `Authorization` headers and only re-injects auth for
routes that declare an egress-owned token. Bare `api.openai.com` or
`chatgpt.com` routes therefore forward Codex requests without the
ChatGPT bearer token.
Copying the host `~/.codex/auth.json` into the agent would solve auth
mode detection but would also put access and refresh material inside the
agent sandbox. That cuts against bot-bottle's credential minimization
model: provider credentials should live in the sidecar boundary when
possible, not in the agent.
## Goals / Success Criteria
- A Codex bottle with host ChatGPT auth can call Codex's
`api.openai.com` and `chatgpt.com` endpoints through egress.
- Host credential forwarding happens only when the bottle declares
`agent_provider.forward_host_credentials: true`.
- The agent container does not receive `OPENAI_API_KEY`,
`CODEX_ACCESS_TOKEN`, or real `tokens.access_token` /
`tokens.refresh_token` values.
- The agent container receives only a dummy Codex `auth.json` that
preserves the host auth-mode shape, keeps the selected ChatGPT
account id, and replaces credential values with placeholders.
- Egress route files remain non-secret: they contain only host/path/auth
slot metadata, never token values.
- Missing, API-key, malformed, or expired host Codex auth fails
launch with a clear operator-facing message.
- Existing Claude OAuth placeholder behavior remains unchanged.
## Non-goals
- Refreshing Codex tokens in the sidecar. The first cut reads the host's
current access token at launch; operators can restart after host Codex
refreshes auth.
- Copying host `~/.codex/auth.json` credentials into the agent.
- Allowing arbitrary host credential forwarding beyond the two providers
covered here (Codex ChatGPT/device-login and Claude OAuth).
- Hot-applying new authenticated Codex routes to an existing running
sidecar. The current hot-apply path cannot safely populate new token
env slots in an already-running container.
## Scope
### In scope
- Add `agent_provider.forward_host_credentials` to the bottle manifest
schema, defaulting to `false`.
- Support the flag for `agent_provider.template: codex`.
- Add `agent_provider.auth_token` to the bottle manifest schema.
- Support the field for `agent_provider.template: claude`: the named
host env var is forwarded only into the egress sidecar as the Bearer
token for `api.anthropic.com`, and a placeholder
`CLAUDE_CODE_OAUTH_TOKEN` is set in the agent so the Claude Code CLI
starts without a real credential.
- Remove the `claude_code_oauth` egress route role, which previously
required operators to declare the OAuth route manually. The provisioner
now injects it from `auth_token`.
- Read host Codex auth from `$CODEX_HOME/auth.json` when `CODEX_HOME` is
set, otherwise from `~/.codex/auth.json`.
- Extract only `tokens.access_token` for egress injection.
- Generate a dummy agent-side `auth.json` from the host auth file's
mode and key shape, without copying real token values.
- Validate that host auth is not API-key mode and the access token is
present, JWT-shaped, and not expired.
- Add or upgrade `api.openai.com` and `chatgpt.com` egress routes to
inject that access token via a shared `EGRESS_TOKEN_N` sidecar env
slot.
- Pass the extracted token only into the sidecar compose/run
environment, alongside other egress token values.
### Out of scope
- Sidecar-owned refresh using `tokens.refresh_token`.
- Sharing full Codex auth state with the agent.
- Supporting host credential forwarding for non-Codex providers.
## Design
### Manifest
Extend `agent_provider`:
```yaml
agent_provider:
template: codex
forward_host_credentials: true
```
The field defaults to `false`. If set on a non-Codex provider, manifest
validation should reject it until that provider has a concrete,
credential-minimizing implementation.
### Host auth extraction
At prepare/launch time, when the flag is enabled for Codex:
1. Resolve the host Codex home directory from `$CODEX_HOME`, falling
back to `~/.codex`.
2. Parse `auth.json`.
3. Require user/device auth mode rather than API-key auth.
4. Require a non-empty `tokens.access_token`.
5. Parse the JWT payload enough to require an `exp` claim in the future.
6. Return only the access token value to the launch path.
Errors should name the missing or invalid condition and point the
operator at `codex login --device-auth`, without printing token values.
### Egress route
When forwarding host Codex credentials, the effective egress route table
should contain authenticated `api.openai.com` and `chatgpt.com` routes.
If the bottle already declares either host as a bare-pass route, upgrade
it in the effective route table rather than requiring a duplicate
manifest entry. If the bottle already declares an authenticated route for
either host, fail rather than guessing whether to override
operator-provided auth, unless that route already uses the synthetic
Codex host credential token reference.
The rendered route should look like any other egress-owned auth route:
```yaml
routes:
- host: "api.openai.com"
auth_scheme: "Bearer"
token_env: "EGRESS_TOKEN_N"
- host: "chatgpt.com"
auth_scheme: "Bearer"
token_env: "EGRESS_TOKEN_N"
```
The access token value is supplied through the sidecar process
environment for that `EGRESS_TOKEN_N` slot. It must not be written to
`routes.yaml`, compose files, env files, logs, or user-facing output.
### Data flow
```mermaid
flowchart LR
H["Host ~/.codex/auth.json"] --> L["bot-bottle launch"]
L -->|access token only| S["egress sidecar env"]
L -->|dummy auth.json only| A
A["Codex agent"] -->|HTTPS via proxy, auth stripped| E["egress"]
E -->|Bearer injected from env| C["api.openai.com / chatgpt.com"]
```
## Implementation chunks
1. **PRD first.** Land this document as the first commit on the feature
branch.
2. **Manifest schema.** Add `forward_host_credentials`, validation, and
unit tests.
3. **Host Codex auth reader.** Add a small stdlib-only helper for
parsing and validating host Codex auth without printing values.
4. **Effective egress route.** Add/upgrade the Codex API routes when the
flag is enabled, and add tests for bare route upgrade,
missing-route insertion, and authenticated-route conflict.
5. **Agent auth marker.** Provision a dummy Codex `auth.json` into the
agent home so Codex selects the host's user/device auth branch while
real credentials stay in egress.
6. **Launch wiring.** Pass the host access token into the egress sidecar
env for Docker and smolmachines without exposing it to the agent.
7. **Docs and tests.** Update README examples and run the unit suite.
## Open questions
- Should a later version support sidecar refresh using the host refresh
token, or should restart-on-expiry remain the policy?
- Should telemetry hosts such as `ab.chatgpt.com` stay blocked by
default even when Codex ChatGPT auth is enabled?
## References
- Gitea issue #109: Codex ChatGPT auth should inject host access token
via egress.
- PRD 0017: Egress-proxy — universal MITM with path filtering + auth
injection.
- PRD 0026: Agent provider templates.
@@ -0,0 +1,121 @@
# PRD 0030: Deduplicate egress token resolution across backends
- **Status:** Active
- **Author:** didericis-claude
- **Created:** 2026-06-02
- **Issue:** #118
## Summary
Eliminate the duplicated egress token resolution block — which resolved
manifest-declared tokens and the Codex host credential — by moving
provider-specific token reading into `AgentProvisionPlan.provisioned_env`
at prepare time, and having both backends merge that map into `os.environ`
before calling the now-generic `egress_resolve_token_values`.
## Problem
The same logic block appeared in two places:
- `bot_bottle/backend/docker/launch.py` (~line 183): inline inside
`launch()`, building `token_values` before `compose_up`.
- `bot_bottle/backend/smolmachines/launch.py` (~line 422): the private
`_resolve_token_env` helper, called before `_bundle.start_bundle`.
Both blocks:
1. Short-circuit to `{}` when there are no egress routes.
2. Call `egress_resolve_token_values(token_env_map, host_env)` to
resolve ordinary manifest-declared token refs.
3. Check `agent_provider.forward_host_credentials` and, when true,
call `codex_host_access_token` and slot the result into any
`token_env` whose `token_ref` is `CODEX_HOST_CREDENTIAL_TOKEN_REF`.
The duplication means any change to step 3 must be applied twice.
PRD 0029, which introduced `forward_host_credentials`, already had to
wire both backends; the next change would too. This is a near-certain
future sync bug.
`egress_resolve_token_values` also carried a sentinel `continue` skip
for `CODEX_HOST_CREDENTIAL_TOKEN_REF`, which tied an otherwise generic
egress helper to a Codex-specific contract.
## Goals / Success Criteria
- The `forward_host_credentials` resolution logic exists in exactly one
place in the codebase.
- Both `docker/launch.py` and `smolmachines/launch.py` call
`egress_resolve_token_values` with no provider-specific arguments.
- `egress_resolve_token_values` is fully generic — it neither knows nor
cares about provider identity or the `CODEX_HOST_CREDENTIAL_TOKEN_REF`
sentinel.
- No behaviour change for either backend.
## Non-goals
- Changes to token resolution semantics. This is a pure refactor.
- Adding support for any new credential type or provider.
- Consolidating any other backend differences beyond this one block.
## Design
### `AgentProvisionPlan.provisioned_env`
Add `provisioned_env: dict[str, str]` (default empty) to
`AgentProvisionPlan`. This map holds host-side secrets that the
provisioning stage resolved and that egress needs injected into the
sidecar environ.
When `forward_host_credentials=True` for Codex, `agent_provision_plan`
calls `codex_host_access_token(host_env)` and stores the result under
`CODEX_HOST_CREDENTIAL_TOKEN_REF`. This is already the prepare-time
stage where `write_codex_dummy_auth_file` runs, so the access-token
read is colocated with all other Codex-specific provisioning.
### Backend call sites
Both backends merge `provisioned_env` over `os.environ` before calling
`egress_resolve_token_values`:
```python
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
```
`provisioned_env` values take precedence over same-named host env vars,
so the Codex token slot resolves from the map written at prepare time
rather than from a raw `os.environ` lookup.
### `egress_resolve_token_values`
Remove the `CODEX_HOST_CREDENTIAL_TOKEN_REF` sentinel `continue` skip.
The function now resolves every slot in `token_env_map` from `host_env`
without special-casing any key name. The `CODEX_HOST_CREDENTIAL_TOKEN_REF`
key is present in `effective_env` (injected by `provisioned_env`) exactly
when it is needed.
## Implementation
- `bot_bottle/agent_provider.py`: add `provisioned_env` field; populate
it for Codex when `forward_host_credentials=True`.
- `bot_bottle/egress.py`: remove sentinel skip; remove
`egress_resolve_token_values_with_provider` (the intermediate function
that was introduced and then superseded by this design); drop the
`codex_auth` import.
- `bot_bottle/backend/docker/launch.py`: replace the provider-aware
resolution block with the `effective_env` merge.
- `bot_bottle/backend/smolmachines/launch.py`: same replacement in
`_resolve_token_env`.
- `tests/unit/test_agent_provider.py`: add tests verifying
`provisioned_env` is populated (or empty) for Codex with and without
`forward_host_credentials`.
- `tests/unit/test_egress.py`: remove `TestResolveTokenValuesWithProvider`;
replace the sentinel-skip test with a test verifying the Codex token
ref resolves normally when present in `host_env`.
## References
- Issue #118: Deduplicate egress token resolution across backends.
- Issue #117: Complexity hotspots in launch, egress, and auth paths
(source of the finding).
- PRD 0029: Provider auth credentials through egress (introduced the
duplicated block).
@@ -0,0 +1,241 @@
# PRD 0031: Simplify egress route merge and consolidate Route types
- **Status:** Active
- **Author:** didericis-claude
- **Created:** 2026-06-02
- **Issue:** #120
## Summary
Replace `_merge_provider_route`'s five-case nested conditional with a
flat provisioned-wins merge, and make the mapping between the host-side
`EgressRoute` and the addon's `Route` explicit in one place. Covers the
two remaining open tasks from the #117 hotspot review.
## Problem
### 1. `_merge_provider_route` branching
`_merge_provider_route` in `bot_bottle/egress.py` handles five distinct
cases in a single function with interleaved conditions:
1. **append-new** — host not in manifest; append a fresh route.
2. **upgrade-bare** — host found, no existing auth; adopt provider auth.
3. **no-op** — host found, same auth; return unchanged.
4. **tls-passthrough upgrade** — same as no-op but provider sets
`tls_passthrough=True`; flip the flag on the existing route.
5. **conflict-die** — host found, different auth; hard error.
Cases 3 and 4 share a block with no-op as the invisible fall-through.
`_find_or_alloc_token_env` is duplicated between cases 2 and 1. In-place
replacements spell out every `EgressRoute` field explicitly, so a new
field added to the dataclass silently drops its value in any replacement
site that wasn't updated.
The root cause of the complexity is that the current merge tries to be
cooperative: it lets manifest routes coexist with provider routes and
attempts to upgrade bare manifest entries. This makes sense if the
manifest is authoritative, but the actual intended hierarchy is the
opposite — provider routes claim their hosts outright and the manifest
fills in what's left.
### 2. Three-way Route type fragmentation
`EgressRoute` (in `egress.py`) and `egress_addon_core.Route` are
separate dataclasses with overlapping but not identical field sets:
| Field | `EgressRoute` | addon `Route` |
|---|---|---|
| `host` | ✓ | ✓ |
| `path_allowlist` | ✓ | ✓ |
| `auth_scheme` | ✓ | ✓ |
| `token_env` | ✓ | ✓ |
| `token_ref` | ✓ (host-side) | — |
| `roles` | ✓ (host-side) | — |
| `tls_passthrough` | ✓ (pipelock concern) | — |
`egress_render_routes` serialises `EgressRoute` fields to YAML; the
addon's `load_routes` deserialises that YAML into `Route` objects. If a
field is added to `EgressRoute` that should appear in the YAML, both
`egress_render_routes` and `_parse_one` must be updated consistently.
The render function spells the field list out inline with no reference
to the addon's parser, so divergence is silent until runtime.
`egress_addon_core.Route` cannot be replaced by `EgressRoute` — the
addon file is copied flat into the sidecar container image (`/app/`) and
has no access to the `bot_bottle` package. The types must stay separate;
the risk is that they drift.
## Goals / Success Criteria
- `egress_routes_for_bottle` implements a flat provisioned-wins merge:
provider routes claim their hosts; manifest routes for unclaimed hosts
append. No upgrade logic, no conflict detection.
- Token slot assignment is a single pass over the merged list, not
interleaved with the merge.
- `egress_render_routes` uses a single `_route_to_yaml_fields` helper
that explicitly lists the addon-visible fields, creating one place
where the `EgressRoute``Route` mapping is spelled out.
- All existing `TestProviderRouteMerge` and `TestRenderRoutes` tests
pass (adjusting assertions for any semantics changes described below).
- No behaviour change for existing manifests that don't trigger the
conflict-die or upgrade-bare paths.
## Non-goals
- Merging `EgressRoute` and `egress_addon_core.Route` into one class
(impossible: addon runs in a stdlib-only container environment).
- Changing what the addon does with a route once it has one.
- Changing `decide()` or `is_git_push_request()` in `egress_addon_core`
— those are already pure functions with good separation.
## Design
### Merge: provisioned wins
The new hierarchy: **provisioned routes own their hosts; manifest routes
fill the gaps.**
```
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
effective = list(provider_routes)
effective += [r for r in manifest_routes if r.host.lower() not in provisioned_hosts]
```
Token slot assignment runs as a final pass over `effective` in order:
```python
def _assign_token_slots(
routes: list[EgressRoute],
) -> tuple[EgressRoute, ...]:
slot_for_ref: dict[str, str] = {}
out: list[EgressRoute] = []
for r in routes:
if r.auth_scheme and r.token_ref and not r.token_env:
token_env = slot_for_ref.get(r.token_ref)
if token_env is None:
token_env = f"EGRESS_TOKEN_{len(slot_for_ref)}"
slot_for_ref[r.token_ref] = token_env
r = dataclasses.replace(r, token_env=token_env)
out.append(r)
return tuple(out)
```
This replaces `_merge_provider_route`, `_find_or_alloc_token_env`, and
the slot-assignment loop inside `egress_manifest_routes`.
#### Semantics change
Under the old design, a manifest route for a provisioned host with a
*different* `auth_scheme` or `token_ref` raised a hard error. Under
provisioned-wins, the manifest entry is silently dropped. Operators who
relied on the conflict error to catch misconfigurations should audit
their manifests, but in practice this path was only reachable when a
manifest declared auth for `api.openai.com` or `chatgpt.com` with a
token ref other than `CODEX_HOST_CREDENTIAL_TOKEN_REF` while also
enabling `forward_host_credentials` — an unlikely combination.
Similarly, the "upgrade-bare" path (provider adopts a bare manifest
route's `path_allowlist`) is dropped: a provisioned host takes the
provider route's fields wholesale, and the manifest's `path_allowlist`
for that host is ignored.
### Route type mapping: `_route_to_yaml_fields`
Add a pure function in `egress.py`:
```python
def _route_to_yaml_fields(r: EgressRoute) -> dict:
"""Return the addon-visible fields for one route.
This is the single authoritative mapping between `EgressRoute`
(host-side) and `egress_addon_core.Route` (sidecar-side). If a
field is added to `Route` that must appear in the YAML, add it
here and in `egress_addon_core._parse_one` together."""
fields: dict = {"host": r.host}
if r.auth_scheme and r.token_env:
fields["auth_scheme"] = r.auth_scheme
fields["token_env"] = r.token_env
if r.path_allowlist:
fields["path_allowlist"] = list(r.path_allowlist)
return fields
```
`egress_render_routes` delegates to it:
```python
for r in routes:
f = _route_to_yaml_fields(r)
lines.append(f' - host: "{f["host"]}"')
if "auth_scheme" in f:
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
lines.append(f' token_env: "{f["token_env"]}"')
if "path_allowlist" in f:
lines.append(" path_allowlist:")
for p in f["path_allowlist"]:
lines.append(f' - "{p}"')
```
The docstring on `_route_to_yaml_fields` is the explicit callout to
update both it and `_parse_one` together when the schema changes.
### `egress_manifest_routes` / `egress_routes_for_bottle`
`egress_manifest_routes` becomes a pure lifter with no slot assignment:
it reads each manifest route entry and returns an `EgressRoute` with
`token_env=""` (the slot to be filled later). The function's docstring
currently promises slot assignment; that promise moves to
`egress_routes_for_bottle`.
`egress_routes_for_bottle` becomes:
```python
def egress_routes_for_bottle(
bottle: Bottle,
provider_routes: tuple[EgressRoute, ...] = (),
) -> tuple[EgressRoute, ...]:
manifest = egress_manifest_routes(bottle)
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
merged = list(provider_routes) + [
r for r in manifest if r.host.lower() not in provisioned_hosts
]
return _assign_token_slots(merged)
```
### Mitmproxy request logic
`egress_addon_core.decide()` and `is_git_push_request()` are already
pure functions; `egress_addon.EgressAddon.request()` is the minimal
mitmproxy glue (read host/path/headers from flow → call pure functions
→ apply result to flow). This split is already clean and requires no
structural change in this PRD.
## Test impact
- **`TestProviderRouteMerge`**: the `test_provider_route_upgrades_bare_manifest_route`
test asserts that a provider route preserves a bare manifest route's
`path_allowlist`. Under provisioned-wins that `path_allowlist` is
dropped. Update the test to reflect the new semantics.
- **`test_provider_route_conflicts_with_different_authed_manifest_route`**:
the conflict-die case no longer exists. Remove this test.
- All other merge and render tests should pass without modification.
## Implementation chunks
1. **PRD (this commit).** Sets the design.
2. **Merge refactor.** Replace `_merge_provider_route` and
`_find_or_alloc_token_env` with `_assign_token_slots` and the
flat provisioned-wins logic in `egress_routes_for_bottle`. Strip
slot assignment from `egress_manifest_routes`.
3. **Render consolidation.** Add `_route_to_yaml_fields`; update
`egress_render_routes` to use it.
4. **Test updates.** Adjust `TestProviderRouteMerge` for the semantics
changes above; confirm all render tests pass.
## References
- Issue #120: Refactor `_merge_provider_route` (expanded to include
Route type fragmentation).
- Issue #117: Complexity hotspots — source of both findings.
- PRD 0030: Deduplicate egress token resolution (prior egress cleanup).
@@ -0,0 +1,221 @@
# PRD 0032: Decompose smolmachines launch and harden bringup sequencing
- **Status:** Active
- **Author:** didericis-claude
- **Created:** 2026-06-02
- **Issue:** #122
## Summary
Split `launch()` into named per-step helpers, replace the empirical
`time.sleep(1.5)` with a readiness poll, and file-lock loopback alias
allocation. Addresses the three actionable issues from the #117 hotspot
review of `smolmachines/launch.py`.
## Problem
### 1. `launch()` step ordering
`launch()` in `smolmachines/launch.py` is 207 lines. Seven sequenced
steps are marked by numbered inline comments (`# 1. Reserve a loopback
alias`, `# 2. Mint per-bottle CAs`, ...) — the sequencing is
load-bearing (CA paths must be filled before the bundle spec is built;
the bundle must be running before port discovery; the VM must be created
before the allowlist is patched), but the dependencies are enforced only
by linear ordering within one function. Adding a new daemon, changing
the port-forward strategy, or debugging a bringup failure requires
reading the whole function to understand what state each step produces.
Each step is also not individually testable without mocking the entire
surrounding context.
### 2. `time.sleep(1.5)` for libkrun exec-channel race
After `machine_start`, back-to-back `machine_exec` calls occasionally
hit a SIGKILL in libkrun's exec channel at ~100ms. The sleep is
documented as "1.5s is empirically enough; provisioning already takes
seconds so the wait is amortized." The failure mode if the sleep is
insufficient: the filesystem-repair exec (`chown -R node:node /home/node`)
is SIGKILLed silently, and the agent later bails with `ENOENT`/`EPERM`
when Claude Code tries to write to `~/.claude.json`. A poll-until-ready
loop is more robust than a fixed duration: it exits as soon as the exec
channel is up, fails loudly with a timeout if the VM never becomes
responsive, and is self-documenting about what it is waiting for.
### 3. Loopback alias allocation is not concurrent-safe
`loopback_alias.allocate()` reads docker container state to determine
which aliases are already in use, then returns the lowest free alias.
There is no lock between that read and the bundle's `docker run` (which
creates the container that will appear in future `docker ps` output). Two
simultaneous bottle launches can both see the same alias as free and
claim it, causing both bundles to bind on the same loopback IP. On macOS,
where users occasionally start multiple agents in quick succession, this
is a realistic failure mode.
## Non-goals
- Removing `force_allowlist` / the `--allow-cidr` DB patch. That is a
workaround for a smolvm 0.8.0 bug; removal is a one-liner when smolvm
honors the CLI flag upstream.
- Changing the ephemeral registry / crane detour in `local_registry.py`.
Required by Docker Desktop's network topology.
- Changing `_ensure_smolmachine`'s cache design. Cache invalidation by
docker image ID works; issue #111 tracks a separate stale-sidecar
concern.
## Design
### 1. Decompose `launch()` into named helpers
Extract six focused helpers. `launch()` becomes a coordinator that calls
them in order, passing the `ExitStack` for teardown registration:
```
_allocate_resources(plan, stack) → (loopback_ip, network)
```
Reserve the loopback alias, create the docker bridge network, register
teardown callbacks for both.
```
_mint_certs(plan) → plan
```
Pipelock TLS init (always). Egress TLS init when `plan.egress_plan.routes`
is non-empty. Returns the plan with CA paths filled via
`dataclasses.replace`.
```
_start_bundle(plan, network, loopback_ip, stack) → plan
```
Build the `BundleLaunchSpec`, resolve token env, start the bundle
container, register teardown. Returns the plan with `bundle_spec` updated
(or unchanged if no plan field carries it — callers consume `bundle_spec`
directly from this call's return value if needed).
```
_discover_urls(plan, loopback_ip) → plan
```
Look up host-side ports for the published container ports; assemble
`agent_proxy_url`, `agent_git_gate_host`, `agent_supervise_url`; stamp
them onto the plan and into `guest_env`.
```
_launch_vm(plan, agent_from_path, stack) → None
```
`machine_create` + `force_allowlist` + `machine_start`. Register
`machine_stop` and `machine_delete` teardown callbacks on the stack.
```
_init_vm(plan) → None
```
Filesystem-repair exec (`chown`/`chmod`) followed by
`_wait_exec_ready()`.
`launch()` reduces to:
```python
loopback_ip, network = _allocate_resources(plan, stack)
plan = _mint_certs(plan)
plan = _start_bundle(plan, network, loopback_ip, stack)
plan = _discover_urls(plan, loopback_ip)
agent_from_path = _ensure_smolmachine(plan.agent_image_ref,
dockerfile=plan.agent_dockerfile_path)
_launch_vm(plan, agent_from_path, stack)
_init_vm(plan)
prompt_path = provision(plan, plan.machine_name)
yield SmolmachinesBottle(...)
```
Each helper's inputs and outputs are explicit; each is independently
testable with a minimal set of mocks.
### 2. Replace `time.sleep(1.5)` with `_wait_exec_ready`
Add to `smolvm.py`:
```python
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
"""Poll until `machine exec true` exits 0 or `timeout` elapses.
Replaces a fixed sleep after machine_start for the libkrun
exec-channel warm-up race."""
deadline = time.monotonic() + timeout
delay = 0.1
while time.monotonic() < deadline:
r = machine_exec(name, ["true"])
if r.returncode == 0:
return
remaining = deadline - time.monotonic()
if remaining <= 0:
break
time.sleep(min(delay, remaining))
delay = min(delay * 2, 0.5)
die(
f"smolvm machine {name!r}: exec channel not ready after "
f"{timeout:.0f}s — VM may have failed to boot."
)
```
`_init_vm` calls `wait_exec_ready` after the chown/chmod exec instead of
`time.sleep(1.5)`. The `time` import in `launch.py` is removed.
### 3. File-lock loopback alias allocation
Add to `loopback_alias.py`:
```python
import fcntl
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
def allocate(slug: str) -> str:
if not _is_macos():
return "127.0.0.1"
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(_ALLOC_LOCK_PATH, "w") as lf:
fcntl.flock(lf, fcntl.LOCK_EX)
return _allocate_locked(slug)
def _allocate_locked(slug: str) -> str:
in_use = _aliases_in_use()
for ip in _pool_addresses():
if ip not in in_use:
return ip
die(...)
return ""
```
The lock is held only for the duration of `_aliases_in_use()` + the
`allocate` return. The bundle's `docker run` runs after the lock is
released. This is sufficient: once `docker run` returns, the container
is visible in docker state and future `allocate()` calls will see it.
The remaining window (lock released → container appears in docker state)
is narrowed from "the entire bringup sequence" to "a single subprocess
call," making a collision between two concurrent launches effectively
impossible in practice.
The lock is a no-op on Linux (the `_is_macos()` early-return fires
before the lock path is opened).
## Test impact
- Unit tests for each extracted helper can mock one subprocess boundary
at a time (smolvm, docker, pipelock TLS init) without wiring the full
`launch()` ExitStack.
- `wait_exec_ready` needs a test with `machine_exec` stubbed to return
non-zero N times before 0 — verifies the backoff loop and the timeout
die path.
- `allocate` tests are unchanged in shape; the lock is acquired and
released within the call so tests don't need to be aware of it.
## Implementation chunks
1. **PRD (this commit).** Sets the design.
2. **Decompose `launch()`.**
3. **Replace sleep with `wait_exec_ready`.**
4. **File-lock `allocate()`.**
5. **Tests.** Unit tests for each helper; `wait_exec_ready` backoff + timeout.
## References
- Issue #122: Decompose smolmachines launch and harden bringup sequencing.
- Issue #117: Complexity hotspots — source of the smolmachines/launch.py finding.
- Issue #111: Smolmachine sidecar doesn't reliably get refreshed (separate, not addressed here).
@@ -0,0 +1,169 @@
# PRD 0033: Manifest Schema Boundaries
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #125
## Summary
Split the manifest loader's schema validation, filesystem loading, `extends:`
resolution, and compatibility passthrough policy into named internal boundaries
without changing the public manifest format. The goal is to make
`bot_bottle/manifest.py` cheaper to extend and review while preserving the
strict validation behavior that keeps manifest mistakes visible.
## Problem
`bot_bottle/manifest.py` has become a broad schema surface. It owns dataclass
models, per-field validators, per-section unknown-key policy, Markdown
frontmatter loading, two-pass bottle inheritance, merge semantics, and
effective agent-to-bottle overlays in one file. The logic is deterministic and
well covered, but the number of concerns makes schema changes expensive:
reviewers have to re-derive loader behavior, parse-time validation, and
post-parse composition rules together.
One specific coupling is especially easy to miss: agent Markdown files are
allowed to double as Claude Code subagent files, so the manifest parser accepts
and ignores Claude Code frontmatter fields such as `name`, `description`,
`model`, `color`, and `memory`. That compatibility rule is encoded as a
passthrough allowlist alongside bot-bottle's own agent schema. If Claude Code
adds a frontmatter field and users start sharing files between
`~/.claude/agents/` and `.bot-bottle/agents/`, bot-bottle raises
`ManifestError` until the local passthrough policy is updated.
The current shape is workable, but it creates unnecessary risk for future
manifest features. A new field can accidentally mix parsing, inheritance, and
compatibility concerns in the same edit, or update one entry path
(`from_json_obj`) without matching the Markdown path (`from_md_dirs`).
## Goals / Success Criteria
- Preserve the existing public manifest schema and runtime behavior.
- Keep `Manifest`, `Bottle`, `Agent`, `GitEntry`, `GitUser`, `AgentProvider`,
`EgressRoute`, `EgressConfig`, and `PipelockRoutePolicy` import-compatible
from `bot_bottle.manifest`.
- Move Markdown file discovery and frontmatter loading behind a small internal
loader boundary with tests that show `$HOME` bottles, `$HOME` agents, `$CWD`
agent overrides, and ignored `$CWD` bottles still behave as before.
- Move bottle `extends:` resolution and merge rules behind a named internal
resolver boundary with tests for inheritance, replacement, cycle detection,
missing parents, and per-field `git.user` overlays.
- Centralize top-level allowed-key policy for bottle and agent schemas so
unknown-key errors remain strict and the allowed set is visible in one place
per schema.
- Make Claude Code passthrough fields a named compatibility policy with focused
tests that distinguish accepted passthrough keys from bot-bottle schema keys
and true typos.
- Keep both entry points, `Manifest.from_json_obj` and
`Manifest.from_md_dirs`, covered by tests for shared validation and shared
inheritance behavior.
## Non-goals
- No manifest format changes.
- No migration away from Markdown frontmatter or the stdlib-only YAML subset
parser.
- No dependency on Pydantic, PyYAML, JSON Schema, or another schema framework.
- No relaxation of strict unknown-key validation for bot-bottle fields.
- No provider-specific workspace, auth, launch, or egress changes.
- No user-facing CLI behavior changes.
## Scope
In scope:
- Internal module organization for manifest loading and composition.
- Validator helpers or schema-policy helpers that reduce duplicated
unknown-key and type-checking logic.
- Focused regression tests around the two existing load paths.
- Documentation comments that clarify compatibility policy where it is encoded.
Out of scope:
- Renaming public dataclass fields or changing their capitalization.
- Reworking callers outside the manifest boundary except for import updates
that are mechanically required by an internal split.
- Adding new manifest fields.
- Changing how `bot-bottle.json` legacy-file errors are reported.
## Design
Keep `bot_bottle.manifest` as the public facade. Existing imports should
continue to work from that module, even if implementation moves into internal
modules such as:
- `bot_bottle/manifest_model.py` for dataclasses and field-level parsing.
- `bot_bottle/manifest_loader.py` for filesystem layout, Markdown
frontmatter loading, stale legacy-file checks, and `$CWD` override rules.
- `bot_bottle/manifest_extends.py` for raw-bottle inheritance, cycle checks,
and merge semantics.
- `bot_bottle/manifest_schema.py` for allowed-key sets, passthrough policy,
and small validation helpers.
The exact filenames are not required. The required boundary is conceptual:
raw input loading, schema validation, bottle inheritance, and effective
agent-to-bottle overlays should be separable when reading and testing the code.
`Manifest.from_json_obj` should continue to accept a raw JSON-like dict and
feed the same raw bottle resolver used by Markdown loading. `Manifest.from_md_dirs`
should perform only filesystem discovery and Markdown parsing before passing
the same raw sections into the same validator/composer path. That shared path
prevents a future schema field from working in one entry point but not the
other.
Claude Code passthrough fields should be represented as an explicit
compatibility allowlist, named as such, and documented near the agent schema
policy. The parser should still ignore those fields after validation. Tests
should cover every passthrough field currently accepted and at least one
unknown field that remains an error.
The `extends:` resolver should remain raw-dict based until after inheritance is
resolved. Merge rules stay unchanged:
- scalar fields use child value when present.
- `env` merges by key with child values winning.
- `git.remotes` merges by upstream host, with child entries replacing duplicate
hosts and explicit empty maps clearing inherited remotes.
- `git.user` overlays per field.
- `egress` remains full-replace when declared by the child.
- cycles, missing parents, and self-reference remain `ManifestError`s.
## Implementation Chunks
1. Add focused characterization tests for agent allowed keys, Claude Code
passthrough fields, and parity between `from_json_obj` and Markdown loading.
2. Extract allowed-key and compatibility policy helpers while keeping
`bot_bottle.manifest` as the import surface.
3. Extract raw Markdown loading into a loader boundary and rerun existing
PRD 0011 tests unchanged.
4. Extract bottle inheritance and merge rules into a resolver boundary and
rerun existing PRD 0025 tests unchanged.
5. Trim `bot_bottle.manifest` to the public facade and model composition,
leaving compatibility imports for existing callers.
Each chunk should be mergeable on its own and should keep the test suite green.
## Testing Strategy
Run the existing manifest-focused unit tests after each chunk:
- `tests/unit/test_manifest_md_load.py`
- `tests/unit/test_manifest_extends.py`
- `tests/unit/test_manifest_git.py`
- `tests/unit/test_manifest_git_user.py`
- `tests/unit/test_manifest_agent_git_user.py`
- `tests/unit/test_manifest_egress.py`
- `tests/unit/test_manifest_runtime.py`
Add new tests only where they lock down boundary behavior not already covered,
especially compatibility passthrough and entry-point parity.
## Open Questions
- Should the Claude Code passthrough allowlist intentionally track a documented
upstream schema, or should bot-bottle keep a narrow local allowlist and update
it only when users need a new shared-file field?
- Should the public facade continue exposing every helper that tests currently
import from `bot_bottle.manifest`, or should tests move to public behavior
only during this cleanup?
@@ -0,0 +1,151 @@
# PRD 0034: Sidecar Restart and Shutdown Semantics
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #126
## Summary
Make the sidecar bundle supervisor's signal, restart, and exit-code behavior
explicit and easier to reason about. In particular, move pipelock restart work
out of direct SIGUSR1 handler execution while preserving the caller-visible
`docker kill --signal USR1 <bundle>` contract used by pipelock apply flows.
## Problem
`bot_bottle/sidecar_init.py` is PID 1 for the bundled sidecar container. It
starts egress, pipelock, git-gate/git-http, and supervise; forwards shutdown
signals; forwards SIGHUP to egress; and restarts pipelock on SIGUSR1 after
allowlist changes.
The current SIGUSR1 handler calls `sup.restart_daemon("pipelock")` directly.
That restart path can terminate a child, wait up to a grace timeout, SIGKILL a
stubborn child, spawn a replacement with `subprocess.Popen`, and start a new
pump thread. In CPython signal handlers run between bytecodes in the main
thread, so this is not the same as POSIX async-signal-unsafe C code, but it
still means signal handling can block the supervisor loop for the restart grace
window and makes stacked signals harder to reason about.
The exit-code contract is also easy to misread. `_Supervisor.exit_code()`
returns the maximum child return code. That preserves a positive crash code
when a child failed before graceful shutdown, but the docstring currently
frames graceful shutdown as returning zero because signal-killed children have
negative return codes. The implementation is reasonable; the contract needs to
be deliberate and tested around crash-then-shutdown behavior.
## Goals / Success Criteria
- Preserve the external signal contract:
- SIGTERM/SIGINT requests bundle shutdown.
- SIGHUP still forwards to the live egress child.
- SIGUSR1 still requests an in-place pipelock restart.
- Keep signal handlers small: handlers should record intent and return, not
perform blocking subprocess lifecycle work directly.
- Process queued restart requests from the supervisor's main loop so restart
behavior is serialized with `tick()` and shutdown state.
- Avoid respawning children after shutdown has started.
- Coalesce or serialize repeated pipelock restart requests in a documented way
so stacked SIGUSR1 delivery cannot overlap restarts.
- Clarify and test aggregate exit-code semantics:
- clean unattended exits return zero when every child exits zero.
- signal-only shutdown does not invent a positive failure code.
- a positive child crash before shutdown remains visible on supervisor exit.
- Keep the implementation stdlib-only.
## Non-goals
- No new process supervisor dependency such as supervisord, s6, or runit.
- No automatic restart policy for arbitrary unexpected child death.
- No changes to the bundle's daemon set, daemon argv, env filtering, or Docker
compose contract.
- No changes to egress route reload semantics beyond preserving SIGHUP
forwarding.
- No user-facing CLI changes.
## Scope
In scope:
- Internal signal handling and supervisor event-loop structure in
`bot_bottle/sidecar_init.py`.
- Tests in `tests/unit/test_sidecar_init.py` for queued restart behavior,
shutdown/restart ordering, repeated restart requests, and exit-code
semantics.
- Docstring/comment updates that describe the concrete signal and exit-code
contracts.
Out of scope:
- Changing pipelock itself to reload config in process.
- Restarting egress, git-gate, git-http, or supervise on demand.
- Reporting restart events to the supervise MCP plane.
- Changing the interim policy that unexpected child death leaves surviving
daemons running.
## Design
Keep `_Supervisor` as the owner of child process state, but add an explicit
pending-action boundary between signal delivery and subprocess lifecycle work.
The exact API can be small, for example:
- `request_shutdown(reason)` keeps its existing idempotent behavior.
- `request_restart(daemon_name)` records a pending restart request unless
shutdown is already in progress.
- `tick()` drains pending restart work before or after child-death logging in a
documented order.
The SIGUSR1 handler should call only the non-blocking request method. The main
loop should continue to call `tick()` and sleep on `_POLL_INTERVAL`; `tick()`
then performs the actual `restart_daemon("pipelock")` work while normal Python
control flow is in the supervisor loop.
Repeated restart requests should not overlap. Restart requests coalesce by
daemon name: if three SIGUSR1 signals arrive before the next loop turn, one
pipelock restart is enough because each restart rereads the latest
`pipelock.yaml` from disk. This treats SIGUSR1 as "make pipelock reflect the
current config" rather than "run exactly one restart per signal."
Shutdown wins over restart. If SIGTERM/SIGINT is received while a restart is
pending, the supervisor should drop the pending restart and terminate live
children. If shutdown starts while `restart_daemon` is already executing in the
main loop, the existing restart operation may finish, but no additional queued
restart should start after shutdown state is set. A simpler implementation may
check shutdown only before each queued restart, because signal handlers execute
between bytecodes and cannot interrupt a single blocking `wait()` until control
returns to Python.
Exit-code behavior should be documented as "positive failures win, otherwise
return zero." Positive process failures remain visible, while a clean shutdown
of only zero-exit or signal-terminated children returns zero instead of leaking
platform-specific negative signal return codes to the container exit status.
## Implementation Chunks
1. Add characterization tests for SIGUSR1 queuing, repeated restart coalescing,
shutdown dropping pending restarts, and crash-then-shutdown exit codes.
2. Add a pending restart request structure to `_Supervisor` and a
non-blocking request method.
3. Change the SIGUSR1 handler in `main()` to enqueue the pipelock restart
instead of calling `restart_daemon` directly.
4. Drain pending restarts from `tick()` with shutdown checks and documented
ordering.
5. Update docstrings and comments around signal handling and `exit_code()`.
## Testing Strategy
Run the existing sidecar unit tests:
- `python3 -m unittest tests.unit.test_sidecar_init`
Add focused unit tests that avoid process-wide signal handler races where
possible by driving `_Supervisor` directly. End-to-end signal tests can remain
limited to `main()` behavior that cannot be exercised otherwise.
Also run the full unit suite before merge:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
+99
View File
@@ -0,0 +1,99 @@
# PRD 0035: Supervise Wait Bounds
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #128
## Summary
Bound the supervise sidecar's request-thread waits so an agent tool call cannot
hold an HTTP worker forever while waiting for operator action. Preserve the MCP
tool surface, but make timeout behavior explicit, observable, and tested.
## Problem
`bot_bottle/supervise_server.py` handles MCP over a threaded stdlib HTTP
server. Tool calls validate a proposal, write it to the supervise queue, and
then wait for the operator response file. Today that wait can last forever.
Each outstanding tool call consumes one server thread until the operator acts.
The route-listing helper also performs a live HTTP request to egress inside the
request thread. It has a short timeout today, but the behavior is not described
as part of a broader request-thread budget.
This is operationally risky in multi-agent or repeated-call scenarios: a stuck
or ignored proposal can accumulate blocked threads, and callers do not get a
clear "still pending" answer they can reason about.
## Goals / Success Criteria
- Add a bounded wait for operator responses to supervise tool calls.
- Return a clear JSON-RPC tool result when the proposal remains pending after
the timeout; do not treat pending operator action as an internal server error.
- Keep the queued proposal on disk after timeout so the operator can still act.
- Make the wait duration configurable by environment with a conservative
default.
- Preserve current success and rejection result shapes for completed operator
responses.
- Keep `list-egress-routes` bounded and document its timeout behavior.
- Add focused tests for approved responses, timed-out pending responses, and
route-list timeout/error handling.
## Non-goals
- No asynchronous framework or new runtime dependency.
- No replacement of the stdlib threaded HTTP server.
- No change to the host-side supervise queue format.
- No cancellation protocol between the agent and operator UI.
- No dashboard or TUI changes.
## Scope
In scope:
- `bot_bottle/supervise_server.py` request handling.
- Any small helper in `bot_bottle/supervise.py` needed to support a bounded
wait cleanly.
- Unit tests around tool-call response waiting and route-list behavior.
Out of scope:
- Reworking proposal persistence.
- Changing egress apply or pipelock apply flows.
- Adding background workers to complete HTTP requests after the client returns.
## Design
Introduce a supervise response wait budget,
`SUPERVISE_RESPONSE_TIMEOUT_SECONDS`, with a 30 second default. The existing
poll loop should stop after that budget and return a normal tool result such as
`{"status": "pending", "notes": "operator response timed out; proposal remains queued"}`.
The exact field names should fit the existing response schema so agents can
handle success, rejection, and pending with one result parser.
The proposal file must remain in the queue when the HTTP call times out. The
operator can still approve or reject it later, but that later response will not
resume the original HTTP request.
Route-listing should continue to use a short HTTP timeout to egress. Errors
should be returned as tool results or JSON-RPC errors consistently with the
existing server behavior; the implementation should avoid an unbounded socket
wait in the request thread.
## Testing Strategy
- Unit-test a tool call whose response appears before the timeout.
- Unit-test a tool call whose response never appears and assert the request
returns a pending result while the proposal remains queued.
- Unit-test invalid timeout env values fall back or fail clearly.
- Unit-test `list-egress-routes` timeout/error behavior with a fake URL opener.
Run:
- `python3 -m unittest tests.unit.test_supervise_server`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,111 @@
# PRD 0036: Codex Auth Redaction Policy
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #129
## Summary
Make Codex host-auth redaction explicit and fixture-driven so dummy
`auth.json` generation cannot accidentally preserve future sensitive fields.
Keep forwarding only the short-lived host access token through egress, while the
guest receives a non-secret auth file whose schema remains useful to Codex.
## Problem
`bot_bottle/codex_auth.py` reads the host Codex auth file, extracts the access
token for egress, and writes a dummy guest `auth.json`. The code redacts JWT
claims and auth JSON fields with a mix of schema-specific handling and generic
placeholder behavior.
That is safer than copying raw auth, but it is still coverage-sensitive. If
Codex adds a new field that carries a token, session identifier, refresh secret,
or account metadata and the field name does not match current heuristics, the
dummy auth file could preserve more information than intended. Because this is
credential-adjacent code, the desired behavior should be allowlist-oriented and
backed by explicit fixtures.
## Goals / Success Criteria
- Define a durable redaction policy for Codex `auth.json`:
- host access token is read for egress only.
- guest dummy auth contains no bearer, refresh, session, or secret values.
- selected non-secret fields may be preserved only when needed by Codex.
- Prefer explicit per-field preservation over broad heuristic pass-through.
- Add representative fixture tests for current Codex auth shapes.
- Add regression tests for unknown nested fields, sensitive-looking field names,
lists, dictionaries, and JWT custom claims.
- Preserve dummy token expiration alignment with the host access token.
- Keep existing errors for missing, invalid, non-device, or expired auth.
## Non-goals
- No change to the egress credential-forwarding contract.
- No attempt to refresh Codex tokens inside the bottle.
- No copying of refresh tokens or raw host auth into the guest.
- No dependency on a Codex SDK or external schema package.
- No user-facing CLI changes.
## Scope
In scope:
- `bot_bottle/codex_auth.py` redaction helpers.
- Unit tests in `tests/unit/test_codex_auth.py`.
- Small documentation comments that distinguish preserved non-secret fields from
redacted credential material.
Out of scope:
- Provider provisioning outside Codex auth file generation.
- Egress route construction for Codex.
- Runtime calls to Codex/OpenAI services.
## Design
Treat the dummy guest `auth.json` as a deliberately synthesized compatibility
file, not as a redacted copy of the host file. The implementation may continue
to start from the host object for convenience, but preserved fields should be
controlled by explicit allowlists at known schema locations.
At the top level, preserve only `auth_mode`, replace `OPENAI_API_KEY` /
`openai_api_key` with `null`, and synthesize the `tokens` block. Unknown scalar
top-level fields become placeholders, unknown lists become empty lists, and
unknown dictionaries become empty objects.
In token blocks, replace `access_token` and `id_token` with dummy JWTs, preserve
the selected non-secret `account_id`, and redact every other token-block field
with the same placeholder / empty container policy. Refresh, session, and future
token values are never copied to the guest.
In JWT payloads, preserve only claims that are known to be non-secret and
required for Codex behavior. Unknown scalar claims become placeholders, unknown
lists become empty lists, and unknown objects become empty objects.
For the OpenAI auth claim, preserve only currently necessary non-secret values
such as plan type, selected account id, and boolean localhost state. Everything
else is placeholder, empty object, or empty list according to the policy.
Tests should use fixture auth objects that include both current expected fields
and intentionally hostile future-looking fields such as `session_context`,
`bearer`, `refreshSecret`, nested `token_value`, and opaque arrays. The dummy
output must not contain the original secret strings.
## Testing Strategy
- Existing `tests/unit/test_codex_auth.py` should continue to pass.
- Add tests that assert original access/refresh/session strings do not appear in
`codex_dummy_auth_json`.
- Add tests for nested JWT and auth-claim redaction behavior.
- Add tests that the dummy access/id token `exp` still matches the host access
token expiry.
Run:
- `python3 -m unittest tests.unit.test_codex_auth`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,113 @@
# PRD 0037: Pipelock YAML Render Contract
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #130
## Summary
Lock down the contract between `pipelock_build_config` and
`pipelock_render_yaml` so hand-rendered pipelock YAML stays aligned with the
structured config bot-bottle builds. Keep the stdlib-only renderer, but add
shape validation and semantic tests for every supported section.
## Problem
`bot_bottle/pipelock.py` builds a structured dict and then renders a fixed YAML
shape by hand. This avoids a runtime YAML dependency, but it also means the
renderer directly indexes expected keys. If `pipelock_build_config` adds,
renames, or conditionalizes a section, rendering can fail at runtime or emit
YAML that no longer matches the config semantics.
Existing tests assert important rendered fragments, but they do not fully lock
the build/render contract or optional-section combinations. A mismatch here can
weaken DLP enforcement or break bottle launch after a future pipelock policy
change.
## Goals / Success Criteria
- Keep the renderer stdlib-only.
- Define the supported pipelock config shape in one place.
- Fail clearly when `pipelock_render_yaml` receives an unsupported or malformed
config shape.
- Add tests covering all supported sections:
- base allowlist and forward proxy.
- seed phrase detection toggle.
- DLP and request-body/header scanning.
- TLS interception and passthrough domains.
- SSRF IP allowlist.
- Add semantic tests that compare structured config values to rendered YAML
output without relying only on brittle substring assertions.
- Preserve current rendered YAML for existing configs unless a clearer failure
path requires an error message change.
## Non-goals
- No PyYAML or other runtime dependency.
- No change to pipelock policy defaults.
- No change to egress-to-pipelock topology.
- No change to pipelock image version or config schema beyond validation of the
shape bot-bottle already emits.
## Scope
In scope:
- `bot_bottle/pipelock.py` render helpers and validation.
- Unit tests in `tests/unit/test_pipelock_yaml.py` and related focused
pipelock tests.
- Small helper functions for typed access to config sections, if useful.
Out of scope:
- Launch/backend changes.
- Integration tests that start a real pipelock container.
- Changing the manifest schema for route-level pipelock policy.
## Design
Treat `pipelock_render_yaml` as a serializer for the narrow config shape
produced by `pipelock_build_config`, not as a generic YAML renderer. Before
rendering a section, validate that required keys exist with the expected
primitive/list/dict types. Missing or unsupported shapes should raise a clear
`ValueError` naming the section and key.
The supported top-level shape is `version`, `mode`, `enforce`,
`api_allowlist`, `seed_phrase_detection`, `forward_proxy`, `dlp`,
`request_body_scanning`, `tls_interception`, and `ssrf`. Required sections are
validated before rendering; optional sections keep the current omission
behavior. `request_body_scanning.scan_headers`,
`request_body_scanning.header_mode`, and
`tls_interception.passthrough_domains` remain optional for compatibility with
parsed running configs that only contain the older rendered subset.
Tests should cover both normal output and failure cases. Because the project is
stdlib-only, semantic tests can use a small purpose-built parser for the exact
rendered shape or compare rendered lines to values from the structured config
through helper assertions. The goal is to detect drift between config dict and
YAML without adding a general YAML dependency.
Optional sections should be exercised in combinations:
- no TLS and no SSRF.
- TLS enabled with empty and non-empty passthrough domains.
- SSRF enabled with one or more IP/CIDR entries.
- all optional sections enabled together.
## Testing Strategy
- Extend `tests/unit/test_pipelock_yaml.py` with semantic assertions tying each
rendered section back to the config dict.
- Add malformed-config tests for missing required keys and wrong section types.
- Keep existing render fragment tests where they protect exact pipelock syntax.
Run:
- `python3 -m unittest tests.unit.test_pipelock_yaml`
- `python3 -m unittest tests.unit.test_pipelock_allowlist`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
+102
View File
@@ -0,0 +1,102 @@
# PRD 0038: smolmachines Env Contract and Secret-Safe Injection
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #135
## Summary
Make smolmachines env handling match Docker's contract: resolve manifest env
entries through `resolve_env()`, keep secret and interpolated values out of
host argv, and document or enforce an explicit env contract for the backend.
## Problem
`bot_bottle/backend/smolmachines/prepare.py` builds the guest env from
`bottle.env` directly, bypassing `resolve_env()`. Entries like `?prompt` and
`${HOST_VAR}` can reach the guest literally rather than being prompted or
resolved. In contrast, Docker resolves env through `resolve_env()` before
writing a mode-600 env file.
`smolmachines/smolvm.py` renders env as `-e KEY=VALUE` on `smolvm machine
create` argv, and `SmolmachinesBottle.agent_argv` / `exec` prepend
`env KEY=VALUE …` onto the `smolvm machine exec` argv. Any literal or resolved
secret value is therefore visible in the host process table.
The two backends have no shared env contract document. Divergence will silently
widen as new manifest env features are added.
## Goals / Success Criteria
- Manifest env entries are resolved through `resolve_env()` before being
injected into the smolmachines guest, matching Docker behaviour.
- No manifest env value (literal or resolved) appears on host argv during
machine creation or exec.
- Define and document an explicit smolmachines env contract covering literals,
`?prompt` secrets, and `${HOST_VAR}` interpolations.
- Unit tests cover: literal passthrough, prompted-secret resolution,
host-var interpolation, and the no-argv-leak invariant.
## Non-goals
- No changes to the Docker env path.
- No changes to manifest schema or `resolve_env()` itself.
- No changes to smolmachines networking or mount handling.
- No new runtime dependencies.
## Scope
In scope:
- `bot_bottle/backend/smolmachines/prepare.py` env resolution.
- `bot_bottle/backend/smolmachines/smolvm.py` machine-create argv.
- `bot_bottle/backend/smolmachines/bottle.py` `agent_argv` / `exec` env
injection.
- `bot_bottle/env.py` if helper changes are needed to support the smolmachines
path.
- Unit tests in `tests/unit/` covering the above.
Out of scope:
- Integration tests that start a live smolmachines VM.
- Docker backend changes.
- Dashboard or CLI changes.
## Design
Run smolmachines env through `resolve_env()` at prepare time, exactly as Docker
does. After resolution, inject env into the guest through a mechanism that does
not expose values on host argv — for example by writing a mode-600 env file
into the machine's state directory and loading it at exec time, or by passing
env through `smolvm`'s stdin if the tool supports it.
If `smolvm` provides no stdin or env-file injection path, document this as a
known limitation and at minimum move env values behind a per-invocation
tmpfile rather than inline argv.
The env contract for smolmachines should mirror Docker's:
- Literals: passed as-is after resolution.
- `?prompt` entries: prompted at prepare time; resolved value injected, never
on argv.
- `${HOST_VAR}` entries: interpolated from the operator's env at prepare time;
resolved value injected, never on argv.
## Testing Strategy
- Unit tests for `prepare.py` asserting `resolve_env()` is called and that
resolution results are used rather than raw `bottle.env` values.
- Unit tests for `smolvm.py` machine-create argv asserting no env value appears
inline.
- Unit tests for `bottle.py` exec path asserting the same argv invariant.
Run:
- `python3 -m unittest tests.unit.test_smolmachines_prepare`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
- Does `smolvm machine create` support an env-file flag or stdin injection that
avoids `-e KEY=VALUE` argv?
@@ -0,0 +1,87 @@
# PRD 0039: smolmachines Capability-Block Remediation
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #136
## Summary
Make capability-block remediation backend-aware. Today the dashboard approval
path calls Docker-only teardown and apply code regardless of which backend
created the bottle. Either implement smolmachines remediation or add a clean
disable/unsupported path so operators never get a partial Docker teardown
against a smolmachines slug.
## Problem
`bot_bottle/cli/dashboard.py` dispatches every capability-block approval to
`bot_bottle/backend/docker/capability_apply.py`. That code snapshots with
`docker cp`, pushes via `docker exec`, rewrites a Dockerfile override, and
removes Docker containers and networks. It does not stop or delete a smolvm
machine.
smolmachines bottles still receive the capability-block supervise tool through
`backend/smolmachines/provision/supervise.py`, so agents can queue a
remediation the host cannot correctly apply. A partial Docker teardown against
a smolmachines slug corrupts neither backend cleanly.
## Goals / Success Criteria
- Capability-block approval is routed to backend-specific code.
- For the smolmachines backend, either:
a. A real remediation implementation that stops the VM, applies the
capability change, and restarts correctly; or
b. A clean unsupported response that tells the operator the action cannot
be taken and leaves the bottle in a consistent state.
- If option (b): smolmachines agents do not receive the capability-block tool,
so the operator is never prompted for an action that will fail.
- Unit tests cover the dispatch logic and the smolmachines path.
## Non-goals
- No changes to the Docker capability-apply path.
- No changes to other supervise tools (cred-block, pipelock-block).
- No changes to manifest or egress configuration.
## Scope
In scope:
- `bot_bottle/cli/dashboard.py` approval dispatch.
- `bot_bottle/backend/smolmachines/provision/supervise.py` tool registration.
- New or updated backend-specific capability apply/disable module for
smolmachines.
- Unit tests for dispatch routing and smolmachines path.
Out of scope:
- Changes to `backend/docker/capability_apply.py` internals.
- Integration tests that exercise a live smolmachines VM remediation.
## Design
Introduce a backend-aware dispatch at the approval call site. Each backend
exposes a capability remediation entry point; the dashboard calls the one that
matches the bottle's backend. If the backend does not support remediation,
the entry point returns a structured error that the dashboard surfaces as an
operator message without attempting any teardown.
If option (b) is chosen initially, suppress capability-block registration in
`smolmachines/provision/supervise.py` so agents never see the tool.
## Testing Strategy
- Unit test that approval dispatch selects the smolmachines path for a
smolmachines bottle and the Docker path for a Docker bottle.
- Unit test for the smolmachines path (unsupported response or real apply).
- Regression test that Docker approval still calls `capability_apply.py`.
Run:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
- Is a real smolmachines capability-apply implementation in scope for this PRD,
or should it be deferred to a follow-on after PRD 0040 lands?
@@ -0,0 +1,87 @@
# PRD 0040: Backend-Aware Resume and Dashboard Reattach
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #137
## Summary
Persist the backend name in `BottleMetadata` and thread it through `resume` and
dashboard reattach so both flows construct the correct backend bottle without
relying on env overrides or defaulting to Docker.
## Problem
`BottleMetadata` records identity, agent, cwd, started_at, and compose project,
but not the backend name. Without it:
- `cli/resume.py` cannot select the right backend from a preserved state dir
alone; operators must remember to set `BOT_BOTTLE_BACKEND=smolmachines`
separately.
- `cli/dashboard.py` `_bottle_for_slug` constructs a `DockerBottle` for any
externally discovered slug, so reattaching to a live smolmachines agent
from the dashboard sends Docker commands to a smolvm machine.
## Goals / Success Criteria
- `BottleMetadata` includes the backend name, written at bottle creation time
for both Docker and smolmachines.
- `cli resume` reads the persisted backend name and constructs the correct
bottle type without requiring an env override.
- Dashboard reattach (`_bottle_for_slug`) reads the persisted backend name and
constructs the correct bottle type.
- Existing Docker bottles without a persisted backend name fall back to Docker
(backward-compatible default).
- Unit tests cover write, read, backward-compatible fallback, and both
resume/reattach code paths.
## Non-goals
- No changes to manifest or egress configuration.
- No new CLI flags (backend selection at resume time should be automatic).
- No smolmachines capability-apply implementation (see PRD 0039).
## Scope
In scope:
- `bot_bottle/backend/docker/bottle_state.py` `BottleMetadata` schema and
write path.
- `bot_bottle/backend/docker/bottle.py` and
`bot_bottle/backend/smolmachines/bottle.py` metadata write at creation.
- `bot_bottle/cli/resume.py` backend selection from metadata.
- `bot_bottle/cli/dashboard.py` `_bottle_for_slug` backend selection.
- Unit tests covering the above.
Out of scope:
- Migration tooling for existing state dirs.
- Integration tests that exercise full resume across process restarts.
## Design
Add a `backend` field to `BottleMetadata` with a default of `"docker"` for
backward compatibility. Both `DockerBottle` and `SmolmachinesBottle` write
their backend name into metadata at creation time.
`resume` reads the metadata before constructing the bottle object and selects
the appropriate backend class. `_bottle_for_slug` does the same. A helper
function in the metadata module can encapsulate the backend-name-to-class
mapping so the logic is not duplicated.
## Testing Strategy
- Unit tests for `BottleMetadata` serialisation with and without the backend
field.
- Unit tests for the backward-compatible default.
- Unit tests for `resume` selecting smolmachines vs Docker from metadata.
- Unit tests for `_bottle_for_slug` selecting smolmachines vs Docker.
Run:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
+60
View File
@@ -0,0 +1,60 @@
# PRD 0041: Git HTTP Request Bounds
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #138
## Summary
Add Content-Length validation and a body-size cap to `git_http_backend.py` so malformed or oversized smart-HTTP requests fail cleanly rather than crashing the handler or exhausting memory.
## Problem
`bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler.
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use. For comparison, `supervise_server.py` caps request bodies at 1 MiB.
## Goals / Success Criteria
- A missing or non-numeric Content-Length returns HTTP 400.
- A negative Content-Length returns HTTP 400.
- A body larger than the cap (1 MiB, matching `supervise_server.py`) returns HTTP 413.
- Valid Git smart-HTTP pushes and fetches continue to work.
- Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough.
## Non-goals
- No changes to git-gate authentication or route logic.
- No changes to `supervise_server.py`.
- No streaming / chunked-transfer-encoding support.
- No TLS changes.
## Scope
In scope:
- `bot_bottle/git_http_backend.py` request parsing and body reading.
- Unit tests in `tests/unit/test_git_http_backend.py`.
Out of scope:
- Integration tests that drive a real Git client through the handler.
## Design
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 1 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
## Testing Strategy
- Unit tests using `unittest.mock` to drive the handler with crafted headers.
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, `Content-Length: 2097152` (over cap), and a normal small POST body.
Run:
- `python3 -m unittest tests.unit.test_git_http_backend`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,85 @@
# PRD 0042: smolmachines Cross-Backend Parity Tests
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #139
## Summary
Add tests that prove secrets, forwarded env, resume, and remediation behave
equivalently across Docker and smolmachines backends. The fixes in PRDs
00380040 are unverifiable without this coverage.
## Problem
The existing unit suite is broad but backend-specific. There are no tests that
run the same scenario against both Docker and smolmachines and assert the
outcomes match. A regression in one backend goes undetected until a live run,
and PRDs 00380040 can each pass their own unit tests while the backends still
diverge at the integration boundary.
## Goals / Success Criteria
- A parity test suite that covers at least:
- Secret env injection: `?prompt` and `${HOST_VAR}` entries produce the same
guest env on both backends.
- Forwarded env: literal manifest env values reach the guest on both backends.
- Resume: a preserved bottle state dir round-trips correctly on both backends
(relies on PRD 0040 metadata).
- Remediation: capability-block approval routes to the correct backend handler
(relies on PRD 0039 dispatch).
- Each scenario is parameterised so a failure names the backend that regressed.
- Tests run without a live VM or Docker daemon (mock or stub backends).
## Non-goals
- No end-to-end agent execution tests.
- No performance or load tests.
- No changes to production code (test-only PRD).
## Scope
In scope:
- New test file(s) under `tests/unit/` for parity scenarios.
- Stub or mock implementations of smolmachines and Docker backends as needed.
Out of scope:
- Changes to `bot_bottle/` production code.
- CI infrastructure changes beyond adding the new test file to the discover
invocation.
## Dependencies
- PRD 0038 should land before the env parity tests are finalised.
- PRDs 0039 and 0040 should land before the remediation and resume scenarios
are finalised; stubs can be written speculatively beforehand.
## Design
Parameterise each scenario over a list of backend factory functions. Each
factory returns a bottle instance wired to a stub subprocess layer. The test
body is backend-agnostic: it calls the same public API, captures the same
observable output, and asserts equality.
For env scenarios, capture the argv or env-file content passed to the guest
and compare against resolved manifest values. For resume, write metadata with
one backend class and read it back to verify correct selection. For remediation,
assert dispatch selects the per-backend handler.
## Testing Strategy
Run as part of the standard unit discover:
- `python3 -m unittest discover -s tests/unit`
Or directly:
- `python3 -m unittest tests.unit.test_backend_parity`
## Open Questions
- Should parity tests live under `tests/unit/` (mock-based) or
`tests/integration/` (live infra)? Mock-based is preferred to keep CI simple.
+74
View File
@@ -0,0 +1,74 @@
# PRD 0043: Sidecar Pipe Lifecycle Cleanup
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #140
## Summary
Close the unclosed child stdout pipe file descriptors that `sidecar_init.py`
leaks during restart and shutdown paths, eliminating `ResourceWarning` noise
and tightening the process lifecycle.
## Problem
Unit tests for `sidecar_init.py` pass, but restart and shutdown cases emit
`ResourceWarning: unclosed file <_io.BufferedReader …>` for child stdout pipes,
originating around lines 141 and 273. The warnings indicate the restart path
leaks pipe file descriptors: a pipe opened for a stopped or replaced child is
not explicitly closed before the next child is spawned or before the supervisor
exits.
## Goals / Success Criteria
- `python3 -m unittest tests.unit.test_sidecar_init` produces no
`ResourceWarning` output.
- Pipe file descriptors for stopped or replaced child processes are explicitly
closed in the restart path.
- Pipe file descriptors for all children are explicitly closed in the shutdown
path.
- No change to the external signal or exit-code contract from PRD 0034.
## Non-goals
- No changes to restart or shutdown policy (coalescing, ordering, timeout).
- No changes to egress, pipelock, git-gate, or supervise daemon argv.
- No new runtime dependencies.
## Scope
In scope:
- `bot_bottle/sidecar_init.py` pipe open/close lifecycle in `_Supervisor`.
- Unit tests in `tests/unit/test_sidecar_init.py` asserting no leaked pipes.
Out of scope:
- Changing how pumping threads read from pipes.
- Integration tests that start a live sidecar container.
## Design
Audit every code path in `_Supervisor` where a child process is stopped,
replaced, or reaches end-of-life, and ensure the corresponding stdout pipe is
explicitly closed before spawning a replacement or exiting the supervisor loop.
Where a pumping thread holds a reference to the pipe, coordinate closure so the
thread sees EOF and exits cleanly rather than blocking indefinitely.
## Testing Strategy
- Enable `ResourceWarning` as an error in test setUp:
`warnings.simplefilter("error", ResourceWarning)`.
- Run existing restart and shutdown test cases under this stricter setting.
- Add tests for restart-then-shutdown if not already covered.
Run:
- `python3 -m unittest tests.unit.test_sidecar_init`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,119 @@
# PRD 0044: Print Parity Across Backends
- **Status:** Active
- **Author:** didericis-claude
- **Created:** 2026-06-02
- **Issue:** #96
## Summary
Hoist `git_gate_plan`, `egress_plan`, `agent_provision`, and `supervise_plan`
from the concrete `BottlePlan` subclasses up to `BottlePlan`, and implement
`print` concretely there. This eliminates the two per-backend output divergences
and ensures any future backend gets correct preflight rendering for free.
## Problem
`BottlePlan.print` is `@abstractmethod`, so each backend provides its own
implementation. The two current implementations have drifted:
| Field | Docker | smolmachines |
|---|---|---|
| git gate lines | `upstream_host:upstream_port` from resolved `git_gate_plan.upstreams` | `Name → Upstream` from manifest `bottle.git` |
| egress lines | `host [auth:scheme]` | `host` only (auth dropped) |
The smolmachines docstring says "same shape as the Docker backend's so operators
see one format across backends" — that intent is real but nothing enforces it.
The env_names divergence previously noted in this issue was resolved by PRD 0038
(smolmachines env contract): `resolved.forwarded` is now merged into
`agent_provision.guest_env` at prepare time on both backends, so displayed env
names are equivalent.
## Goals / Success Criteria
- `BottlePlan` carries `git_gate_plan`, `egress_plan`, `agent_provision`, and
`supervise_plan` as concrete fields; subclasses no longer declare them
independently.
- `BottlePlan.print` is a concrete method; subclasses have no `print`
implementation of their own.
- Both backends render git gate lines as `name → upstream_host:upstream_port`
(using `git_gate_plan.upstreams`), not the manifest-level URL.
- Both backends render egress lines as `host [auth:scheme]` (dropping the
annotation only when `auth_scheme` is empty).
- Unit tests assert the unified output for both backends from a single shared
test helper.
## Non-goals
- No changes to the Docker or smolmachines launch, prepare, or cleanup paths.
- No changes to how env values are resolved or injected (that is PRD 0038).
- No changes to the manifest schema or `GitEntry`.
- No new CLI flags or dashboard changes.
## Scope
In scope:
- `bot_bottle/backend/__init__.py` — add `git_gate_plan`, `egress_plan`,
`agent_provision`, and `supervise_plan` fields to `BottlePlan`; replace
`@abstractmethod print` with a concrete implementation.
- `bot_bottle/backend/docker/bottle_plan.py` — remove the four hoisted fields
and the `print` method.
- `bot_bottle/backend/smolmachines/bottle_plan.py` — remove the four hoisted
fields and the `print` method.
- `tests/unit/` — add or update tests asserting unified preflight output; a
shared helper can build a minimal plan fixture for each backend and assert
the same lines appear.
Out of scope:
- Changes to `bot_bottle/backend/print_util.py` beyond what the new `print`
implementation requires.
- Changes to `BottleCleanupPlan.print` or any other print method.
- Integration tests that launch a real bottle.
## Design
Move the four fields that both concrete subclasses already declare —
`git_gate_plan: GitGatePlan`, `egress_plan: EgressPlan`,
`agent_provision: AgentProvisionPlan`, `supervise_plan: SupervisePlan | None`
— up to `BottlePlan`. Both backends' `prepare` paths already produce these with
the same types, so no prepare-time changes are needed.
Replace the `@abstractmethod` `print` with a concrete implementation on
`BottlePlan` that:
1. Builds `env_names` from `bottle.env.keys() | agent_provision.guest_env.keys()`
filtered through `agent_provision.hidden_env_names`.
2. Builds git gate lines from `git_gate_plan.upstreams` as
`f"{u.name} → {u.upstream_host}:{u.upstream_port}"`.
3. Builds egress lines from `egress_plan.routes` as
`f"{r.host} [auth:{r.auth_scheme}]"` when `r.auth_scheme` is non-empty,
else `r.host`.
4. Renders the standard two-column preflight block (leading blank line, agent,
provider, env, skills, bottle, git identity, git gate, egress, trailing blank
line).
Docker's `forwarded_env` keys are already merged into `agent_provision.guest_env`
via the `agent_provision_plan` builder, so no special handling is needed for
env_names.
## Testing Strategy
- Add a shared fixture builder (e.g. `make_plan(backend)`) in a new or existing
unit test module that constructs a minimal `DockerBottlePlan` and
`SmolmachinesBottlePlan` from the same spec and plan fields.
- Assert that `plan.print(remote_control=False)` produces identical git gate and
egress lines for both backends given the same `git_gate_plan` and
`egress_plan`.
- Test the `auth_scheme` annotation: present when non-empty, absent otherwise.
- Test git gate rendering: `name → host:port` format.
Run:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
+167
View File
@@ -0,0 +1,167 @@
# PRD 0045: Workspace Porting Plan
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #116
## Summary
Add a backend-neutral `WorkspacePlan` that describes how the operator's current
workspace is represented inside a bottle. Docker and smolmachines should both
use this plan for workspace path, working directory, content copy, `.git` copy,
ownership, and provider trust configuration instead of rediscovering
`/home/node/workspace` in separate launch and provisioning code paths.
## Problem
The current `--cwd` behavior is spread across backend-specific code:
- Docker builds a derived image that copies the host cwd to
`/home/node/workspace`, sets that as `WORKDIR`, and patches Claude trust in
the generated Dockerfile.
- Docker git provisioning separately copies `.git` into
`/home/node/workspace/.git`.
- smolmachines git provisioning reconstructs `<guest_home>/workspace/.git`, but
does not copy the full working tree.
- Codex provider setup trusts `guest_home`, not the copied workspace path.
These details create backend drift and make provider-specific workspace fixes
easy to hard-code in the wrong layer.
## Goals / Success Criteria
- `BottleSpec` remains the CLI intent shape (`copy_cwd`, `user_cwd`), while a
resolved `WorkspacePlan` carries the backend-neutral guest workspace contract.
- `BottlePlan` exposes `workspace_plan` so shared and backend-specific
provisioning paths consume one resolved object.
- The default in-bottle workspace path remains `/home/node/workspace` when
`--cwd` is enabled.
- Docker uses `WorkspacePlan` when building the derived cwd image and when
provisioning cwd `.git` state.
- smolmachines copies the host cwd contents into the same logical workspace
path and uses `WorkspacePlan` when provisioning cwd `.git` state.
- Provider trust configuration is written for the workspace path when `--cwd`
is enabled, and for the guest home when `--cwd` is disabled.
- Unit tests cover plan resolution, provider trust path selection, Docker
derived image rendering, and both backends' `.git` copy targets.
## Non-goals
- No new user-facing flags for custom workspace paths.
- No manifest schema changes.
- No redesign of git-gate or `bottle.git` entries.
- No switch from Docker image-copy to bind-mount.
- No unrelated provider auth changes.
## Scope
In scope:
- Add a small workspace planning module.
- Add `workspace_plan` to `BottlePlan` and populate it in Docker and
smolmachines prepare paths.
- Thread the trusted project path into provider provisioning.
- Replace hard-coded `/home/node/workspace` cwd copy and `.git` copy sites with
`WorkspacePlan` values.
- Copy full host cwd contents for smolmachines `--cwd` parity.
- Update focused unit tests.
Out of scope:
- Integration tests that launch real Docker containers or smolmachines VMs.
- Path customization in the bottle manifest or CLI.
- Runtime synchronization after bottle launch; this remains a launch-time copy.
## Design
Add `bot_bottle/workspace.py`:
```python
@dataclass(frozen=True)
class WorkspacePlan:
enabled: bool
host_path: Path
guest_home: str
guest_path: str
workdir: str
owner: str = "node:node"
mode: str = "755"
copy_contents: bool = True
copy_git: bool = True
has_host_git_dir: bool = False
```
`workspace_plan(spec, guest_home)` resolves:
- `enabled` from `spec.copy_cwd`.
- `host_path` from `spec.user_cwd`.
- `guest_path` as `<guest_home>/workspace` when enabled, else `guest_home`.
- `workdir` as `guest_path` when enabled, else `guest_home`.
- `has_host_git_dir` from `<host_path>/.git`.
Backends resolve this in `prepare` using their existing guest-home knobs:
- Docker: `BOT_BOTTLE_CONTAINER_HOME`, default `/home/node`.
- smolmachines: `BOT_BOTTLE_GUEST_HOME`, default `/home/node`.
`BottlePlan` carries the result so launch, git provisioning, and provider
provisioning stop consulting `spec.copy_cwd` and hard-coded paths directly.
### Docker
Keep the current derived-image transport. Change
`build_image_with_cwd(derived, base, cwd)` to accept a `WorkspacePlan` or
explicit guest path/workdir fields, then render:
- `COPY --chown=node:node . <workspace_plan.guest_path>`
- `WORKDIR <workspace_plan.workdir>`
Claude trust should move out of the generated cwd Dockerfile and into provider
provisioning so Docker and smolmachines share the same provider trust behavior.
### smolmachines
Copy host cwd contents into `workspace_plan.guest_path` during provisioning or
VM initialization, then chown the resulting workspace to `node:node`. Continue
to copy `.git` through the existing smolvm transport, but target
`<workspace_plan.guest_path>/.git`.
This intentionally closes the current parity gap where smolmachines receives
repo metadata without the working tree.
### Provider Trust
Extend provider planning with a `trusted_project_path` argument. Callers pass
`workspace_plan.workdir`.
Codex writes:
```toml
[projects."<trusted_project_path>"]
trust_level = "trusted"
```
Claude writes or updates `.claude.json` so `projects` includes
`trusted_project_path` with `hasTrustDialogAccepted: true`. This provisioning
belongs in `AgentProvisionPlan` so both backends apply it through their existing
provider file-copy primitives.
## Testing Strategy
- Unit-test `workspace_plan()` for enabled and disabled cwd, guest-home
overrides, and `.git` detection.
- Unit-test Docker cwd image rendering to prove it uses the plan's guest path
and workdir.
- Unit-test provider planning for Codex and Claude trusted project paths.
- Unit-test Docker and smolmachines git provisioning targets using mocked copy
and exec primitives.
- Unit-test smolmachines workspace content copy target and ownership command.
Run:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,64 @@
# PRD 0046: Remove Git Remote Host Overrides
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #152
## Summary
Remove git remote host override plumbing from bottle manifests and git-gate
startup. Git remote declarations should describe upstream repositories and the
git-gate credential material needed to mirror them; they should not also
configure hosts-file behavior for sidecars.
## Problem
The git remote model currently has a hosts override path that can make a git
upstream resolve differently inside the git-gate sidecar. That is surprising
because the same hostname may also be used for HTTP/API traffic that should keep
using the normal egress DNS and policy path.
Keeping host resolution in the git remote model makes repository routing,
sidecar hosts files, and egress behavior feel coupled even when the operator
only meant to configure git-gate.
## Goals / Success Criteria
- Git remote manifest parsing no longer stores host override data.
- Git-gate upstream plans no longer carry host override data.
- Docker compose rendering no longer emits sidecar `extra_hosts` entries from
git remote declarations.
- Smolmachines bundle launch planning has no unused host override path for
git-gate.
- Focused unit tests cover the absence of sidecar `extra_hosts` for git
upstreams.
- Current user-facing documentation no longer advertises git remote host
overrides.
## Non-goals
- No replacement hosts-file override feature.
- No SSH client config provisioning.
- No change to git-gate's SSH credential or known-host handling.
- No change to egress DNS, HTTP auth, or pipelock routing semantics.
## Design
Remove the host override field from the internal `GitEntry` and
`GitGateUpstream` models. Remove the git-gate aggregation helper and the Docker
compose code that converted those values into sidecar `extra_hosts`.
The manifest parser does not need a migration-specific error path. After this
change, the old hosts override key has no internal model field and no runtime
effect.
## Testing Strategy
Run:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,170 @@
# PRD 0047: Git-gate Manifest Redesign
- **Status:** Active
- **Author:** didericis
- **Created:** 2026-06-03
- **Issue:** #160
## Summary
Replace the `git` top-level key in bottle and agent manifests with `git-gate`,
consolidating git-identity configuration (`user`) and git-gate sidecar
configuration (`repos`) under a single section. Within `repos`, field names
move to lowercase snake_case and the local repo name is promoted to the YAML
key. The change removes the ambiguity in the current `git` block: its fields
are not generic git or SSH config — they are specifically the credential,
host-trust, and identity material that is managed in relation to git-gate.
## Problem
The current bottle manifest uses a `git` top-level key that mixes two concerns:
- `git.user``git config --global user.name / user.email` identity, which
the provisioner injects into the agent's shell.
- `git.remotes` — upstream URL, identity file, and host key material that the
git-gate sidecar consumes; the agent never sees these values.
That grouping suggests the `remotes` entries behave like an SSH config or a
generic `.gitconfig` remote declaration. They do not. The gate reads the
credential material to push upstream after gitleaks passes; the agent's
`.gitconfig` receives only the `insteadOf` rewrite that redirects traffic
through the gate. Nothing in the current key name or field names signals this.
Splitting `git.user` into a separate section from `git.remotes` also doesn't
help: both concepts exist because of git-gate, and keeping them under a single
`git-gate` key makes their relationship and purpose explicit.
The field names inside each remote entry also use PascalCase (`Name`,
`Upstream`, `IdentityFile`, `KnownHostKey`), inconsistent with every other
manifest section, which uses snake_case.
The current `git.remotes` dict is keyed by upstream host, which works for
simple remotes but forces a separate `Name` field to give the gate's bare repo
a local label. The host key and `Name` field are often redundant or confusing
(e.g., IP-literal upstreams where the key carries no semantic meaning).
## Goals / Success Criteria
- `git-gate` is accepted as a top-level bottle and agent key; `git` is removed
from both allowed-key sets.
- `git-gate.repos` is a named map where each key is the local repo name
exposed by the gate (bottle-only; rejected at the agent level).
- Each entry in `git-gate.repos` accepts exactly: `url` (required), `identity`
(required), `host_key` (optional).
- `git-gate.user` replaces `git.user` on both bottles and agents, with the
same `name` / `email` fields and overlay semantics.
- The manifest parser rejects `git.remotes` and `git.user` with errors that
point to the new keys.
- `GitEntry` internal fields are updated to match the new names; all callers
(provisioner, git-gate render, plan, tests) compile and pass.
- Existing unit tests in `tests/unit/test_manifest_git.py` and
`tests/unit/test_manifest_git_user.py` are rewritten to use the new YAML
shape; all other manifest unit tests remain green.
- The demo manifest (`bot-bottle.demo.json`) and any examples using the old
shape are updated.
## Non-goals
- No change to `git.user` / `git-gate.user` semantics or field names (`name`,
`email`).
- No change to git-gate runtime behavior (mirroring, gitleaks, access-hook
refresh).
- No change to the `insteadOf` rewrite the provisioner emits.
- No migration shim: the old `git.*` shape is rejected immediately with clear
error messages pointing to the new keys.
- No change to how agent-level user config overlays the bottle-level value.
## Design
### New manifest shape
**Before** (bottle frontmatter):
```yaml
git:
user:
name: implementer-bot
email: eric+implementer@dideric.is
remotes:
gitea.dideric.is:
Name: bot-bottle
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
IdentityFile: ~/.ssh/gitea-delos-2.pem
KnownHostKey: "ssh-rsa AAAA..."
```
**After**:
```yaml
git-gate:
user:
name: implementer-bot
email: eric+implementer@dideric.is
repos:
bot-bottle:
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
identity: ~/.ssh/gitea-delos-2.pem
host_key: "ssh-rsa AAAA..."
```
`git-gate` is the single optional top-level key for all git configuration.
Bottles that previously used only `git.user` now use only `git-gate.user`;
those that used only `git.remotes` now use only `git-gate.repos`.
### Key-name-as-repo-name
The YAML key in `git-gate.repos` becomes the local repo name (previously
`Name`). The upstream host is no longer the primary key; the provisioner and
gate derive it from the `url` field during parse. IP-literal upstreams work
without an artificial host-as-key constraint.
### Field renames
| Old field | New field |
|-----------|-----------|
| `Name` (from dict key) | YAML key in `repos` |
| `Upstream` | `url` |
| `IdentityFile` | `identity` |
| `KnownHostKey` | `host_key` |
### Parser changes
- `manifest_schema.py`: replace `"git"` with `"git-gate"` in `BOTTLE_KEYS`
and `AGENT_KEYS_OPTIONAL`.
- `manifest.py`: replace `_parse_git_config` with `_parse_git_gate_config`
that validates both `user` and `repos` subkeys. Update `Bottle.from_dict`
and `Agent.from_dict` to call it for the `"git-gate"` key.
- `Agent.from_dict` continues to reject `repos` at the agent level with a
clear error.
- Remove `from_remote_dict` and update `GitEntry._from_object` to accept the
new field names. Internal dataclass field names (`UpstreamUser`, etc.) are
unchanged — they are internal plumbing, not user-facing.
- Any existing `"git"` key raises a targeted error:
```
bottle 'dev' uses 'git' which has been replaced by 'git-gate' (PRD 0047).
Move git.user → git-gate.user and git.remotes → git-gate.repos.
```
## Testing Strategy
Run:
```
python3 -m unittest discover -s tests/unit
```
Test files to update:
- `tests/unit/test_manifest_git.py` — rewrite fixtures and assertions to use
`git-gate.repos` / lowercase fields. Cover: minimal entry, optional
`host_key`, missing `url`, missing `identity`, unknown key, IP-literal
upstreams, duplicate name rejection, old `git.remotes` and bare `git` key
both rejected.
- `tests/unit/test_manifest_git_user.py` and
`tests/unit/test_manifest_agent_git_user.py` — update fixtures to use
`git-gate.user` at both bottle and agent level.
## Open Questions
None.
+1 -1
View File
@@ -5,7 +5,7 @@ model: opus
bottle: dev
skills:
- init-prd
git:
git-gate:
user:
name: implementer-bot
email: eric+implementer@dideric.is
+11 -13
View File
@@ -38,23 +38,21 @@ def fixture_with_egress_dict() -> dict[str, Any]:
def fixture_with_git_dict() -> dict[str, Any]:
"""Bottle declares a git-gate upstream. JSON shape."""
"""Bottle declares git-gate upstreams. JSON shape."""
return {
"bottles": {
"dev": {
"git": {
"remotes": {
"gitea.dideric.is": {
"Name": "bot-bottle",
"Upstream": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"IdentityFile": "/dev/null",
"KnownHostKey": "ssh-ed25519 AAAA...",
"git-gate": {
"repos": {
"bot-bottle": {
"url": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"identity": "/dev/null",
"host_key": "ssh-ed25519 AAAA...",
},
"github.com": {
"Name": "foo",
"Upstream": "ssh://git@github.com/didericis/foo.git",
"IdentityFile": "/dev/null",
"KnownHostKey": "ssh-ed25519 BBBB...",
"foo": {
"url": "ssh://git@github.com/didericis/foo.git",
"identity": "/dev/null",
"host_key": "ssh-ed25519 BBBB...",
},
},
}
+209
View File
@@ -0,0 +1,209 @@
"""Unit: provider runtime defaults."""
from __future__ import annotations
import base64
import json
import tempfile
import unittest
from pathlib import Path
from bot_bottle.agent_provider import (
CODEX_HOST_CREDENTIAL_HOSTS,
agent_provision_plan,
runtime_for,
)
from bot_bottle.egress import CODEX_HOST_CREDENTIAL_TOKEN_REF
def _jwt(exp: int) -> str:
def enc(obj: dict) -> str:
raw = json.dumps(obj, separators=(",", ":")).encode()
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
return f"{enc({'alg': 'none'})}.{enc({'exp': exp})}.sig"
class TestAgentProviderRuntime(unittest.TestCase):
def test_codex_plan_declares_home_state(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan(
template="codex",
dockerfile="/tmp/Dockerfile.codex",
state_dir=Path(tmp),
)
config = Path(tmp, "codex-config.toml").read_text()
self.assertEqual("codex", plan.template)
self.assertEqual("codex", plan.command)
self.assertEqual("read_prompt_file", plan.prompt_mode)
self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile)
self.assertEqual(
"/etc/ssl/certs/ca-certificates.crt",
plan.env_vars["CODEX_CA_CERTIFICATE"],
)
self.assertEqual({}, plan.guest_env)
self.assertEqual(("/home/node/.codex",), tuple(d.guest_path for d in plan.dirs))
self.assertEqual(
("/home/node/.codex/config.toml",),
tuple(f.guest_path for f in plan.files),
)
self.assertIn('[projects."/home/node"]', config)
def test_codex_trusts_requested_project_path(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
trusted_project_path="/home/node/workspace",
)
config = Path(tmp, "codex-config.toml").read_text()
self.assertIn('[projects."/home/node/workspace"]', config)
def test_codex_forward_host_credentials_adds_auth_and_verify(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
home = Path(tmp) / "host-codex"
home.mkdir()
(home / "auth.json").write_text(json.dumps({
"auth_mode": "chatgpt",
"tokens": {"access_token": _jwt(2000000000)},
}))
plan = agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
guest_env={"CODEX_HOME": "/run/codex-home"},
forward_host_credentials=True,
host_env={"CODEX_HOME": str(home)},
)
self.assertIn(
"/run/codex-home/auth.json",
{f.guest_path for f in plan.files},
)
self.assertEqual("/run/codex-home", plan.env_vars["CODEX_HOME"])
self.assertEqual(1, len(plan.pre_copy))
self.assertEqual(1, len(plan.verify))
self.assertIn("CODEX_HOME=/run/codex-home", plan.verify[0].argv)
def test_claude_with_auth_token_injects_provider_route_and_placeholder(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan(
template="claude",
dockerfile="/tmp/Dockerfile.claude",
state_dir=Path(tmp),
auth_token="BOT_BOTTLE_CLAUDE_OAUTH_TOKEN",
)
claude_config = json.loads(Path(tmp, "claude.json").read_text())
self.assertEqual(1, len(plan.egress_routes))
route = plan.egress_routes[0]
self.assertEqual("api.anthropic.com", route.host)
self.assertEqual("Bearer", route.auth_scheme)
self.assertEqual("BOT_BOTTLE_CLAUDE_OAUTH_TOKEN", route.token_ref)
self.assertTrue(route.tls_passthrough)
self.assertEqual("egress-placeholder", plan.env_vars["CLAUDE_CODE_OAUTH_TOKEN"])
self.assertEqual("1", plan.env_vars["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"])
self.assertEqual("1", plan.env_vars["DISABLE_ERROR_REPORTING"])
self.assertEqual(frozenset({"CLAUDE_CODE_OAUTH_TOKEN"}), plan.hidden_env_names)
self.assertIn("/home/node", claude_config["projects"])
self.assertIn("/home/node/.claude.json", {f.guest_path for f in plan.files})
def test_claude_trusts_requested_project_path(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
agent_provision_plan(
template="claude",
dockerfile="",
state_dir=Path(tmp),
trusted_project_path="/home/node/workspace",
)
config = json.loads(Path(tmp, "claude.json").read_text())
self.assertIn("/home/node", config["projects"])
self.assertIn("/home/node/workspace", config["projects"])
def test_codex_forward_host_credentials_populates_egress_routes(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
home = Path(tmp) / "host-codex"
home.mkdir()
(home / "auth.json").write_text(json.dumps({
"auth_mode": "chatgpt",
"tokens": {"access_token": _jwt(2000000000)},
}))
plan = agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
forward_host_credentials=True,
host_env={"CODEX_HOME": str(home)},
)
hosts = [r.host for r in plan.egress_routes]
self.assertEqual(sorted(CODEX_HOST_CREDENTIAL_HOSTS), sorted(hosts))
for r in plan.egress_routes:
self.assertEqual("Bearer", r.auth_scheme)
self.assertEqual(CODEX_HOST_CREDENTIAL_TOKEN_REF, r.token_ref)
self.assertTrue(r.tls_passthrough)
def test_codex_without_forward_host_credentials_has_passthrough_egress_routes(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
forward_host_credentials=False,
)
self.assertEqual(
{r.host for r in plan.egress_routes},
set(CODEX_HOST_CREDENTIAL_HOSTS),
)
for r in plan.egress_routes:
self.assertEqual("", r.auth_scheme)
self.assertEqual("", r.token_ref)
self.assertTrue(r.tls_passthrough)
def test_claude_without_auth_token_has_passthrough_egress_route(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan(
template="claude",
dockerfile="",
state_dir=Path(tmp),
)
self.assertEqual(1, len(plan.egress_routes))
route = plan.egress_routes[0]
self.assertEqual("api.anthropic.com", route.host)
self.assertEqual("", route.auth_scheme)
self.assertEqual("", route.token_ref)
self.assertTrue(route.tls_passthrough)
self.assertNotIn("CLAUDE_CODE_OAUTH_TOKEN", plan.env_vars)
self.assertEqual(frozenset(), plan.hidden_env_names)
def test_codex_forward_host_credentials_populates_provisioned_env(self):
access = _jwt(2000000000)
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
home = Path(tmp) / "host-codex"
home.mkdir()
(home / "auth.json").write_text(json.dumps({
"auth_mode": "chatgpt",
"tokens": {"access_token": access},
}))
plan = agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
forward_host_credentials=True,
host_env={"CODEX_HOME": str(home)},
)
self.assertEqual(
{CODEX_HOST_CREDENTIAL_TOKEN_REF: access},
plan.provisioned_env,
)
def test_codex_without_forward_host_credentials_has_empty_provisioned_env(self):
with tempfile.TemporaryDirectory(prefix="bb-provider.") as tmp:
plan = agent_provision_plan(
template="codex",
dockerfile="",
state_dir=Path(tmp),
forward_host_credentials=False,
)
self.assertEqual({}, plan.provisioned_env)
if __name__ == "__main__":
unittest.main()
+240
View File
@@ -0,0 +1,240 @@
"""Cross-backend parity tests (PRD 0042).
Verifies that Docker and smolmachines bottles expose the same
observable contracts for env injection, agent argv, and exec. Tests
use mock subprocess layers so no live VM or Docker daemon is needed.
The scenarios here document what must hold across both backends. As
PRDs 00380040 land these tests provide regression coverage for the
contracts they establish.
"""
from __future__ import annotations
import subprocess
import unittest
from typing import Callable
from unittest.mock import MagicMock, call, patch
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _docker_bottle(guest_env: dict[str, str]) -> "object":
from bot_bottle.backend.docker.bottle import DockerBottle
return DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
agent_command="claude",
)
def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
return SmolmachinesBottle(
"bot-bottle-test",
guest_env=guest_env,
agent_command="claude",
)
# One entry per backend: (label, factory).
_BACKENDS: list[tuple[str, Callable[[dict[str, str]], object]]] = [
("docker", _docker_bottle),
("smolmachines", _smolmachines_bottle),
]
# ---------------------------------------------------------------------------
# agent_argv contracts
# ---------------------------------------------------------------------------
class TestAgentArgvParity(unittest.TestCase):
"""Both backends surface a non-empty agent_argv that includes the
agent command and can be used as a subprocess command list."""
def test_agent_argv_is_list_of_strings(self):
for label, factory in _BACKENDS:
with self.subTest(backend=label):
bottle = factory({"MY_VAR": "val"})
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
self.assertIsInstance(argv, list, f"{label}: argv is not a list")
for item in argv:
self.assertIsInstance(
item, str,
f"{label}: argv item {item!r} is not a str",
)
def test_agent_command_present_in_argv(self):
for label, factory in _BACKENDS:
with self.subTest(backend=label):
bottle = factory({})
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
joined = " ".join(argv)
self.assertIn(
"claude", joined,
f"{label}: 'claude' not found in agent_argv",
)
def test_extra_flags_propagate(self):
extra = ["--no-update-check", "--output-format", "stream-json"]
for label, factory in _BACKENDS:
with self.subTest(backend=label):
bottle = factory({})
argv = bottle.agent_argv(extra, tty=False) # type: ignore[union-attr]
for flag in extra:
self.assertIn(
flag, argv,
f"{label}: flag {flag!r} not in agent_argv",
)
class TestSmolmachinesEnvInArgv(unittest.TestCase):
"""smolmachines bottle includes guest_env values in exec argv."""
def test_guest_env_in_exec_argv(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle(
"bot-bottle-test",
guest_env={"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
)
argv = bottle.agent_argv([], tty=False)
joined = " ".join(argv)
self.assertIn("TOKEN=abc123", joined)
self.assertIn("PROXY=http://proxy:8888", joined)
# ---------------------------------------------------------------------------
# exec() user-switching contract
# ---------------------------------------------------------------------------
class TestExecUserSwitching(unittest.TestCase):
"""Both backends exec as 'node' by default and accept user='root'."""
def test_docker_exec_uses_node_user_by_default(self):
from bot_bottle.backend.docker.bottle import DockerBottle
bottle = DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
)
with patch("bot_bottle.backend.docker.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("echo hi")
call_args = run.call_args[0][0]
self.assertIn("node", call_args,
"docker exec should use 'node' user by default")
def test_smolmachines_exec_uses_node_user_by_default(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("echo hi")
call_args = run.call_args[0][0]
self.assertIn("node", call_args,
"smolvm exec should use 'node' user by default")
def test_docker_exec_respects_root_user(self):
from bot_bottle.backend.docker.bottle import DockerBottle
bottle = DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
)
with patch("bot_bottle.backend.docker.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("id", user="root")
call_args = run.call_args[0][0]
self.assertIn("root", call_args)
def test_smolmachines_exec_respects_root_user(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("id", user="root")
call_args = run.call_args[0][0]
self.assertIn("root", call_args)
# ---------------------------------------------------------------------------
# ExecResult shape parity
# ---------------------------------------------------------------------------
class TestExecResultParity(unittest.TestCase):
"""Both backends return ExecResult with returncode, stdout, stderr."""
def _stub_run(self, argv, **kwargs):
return subprocess.CompletedProcess(
argv, 0, stdout="out\n", stderr="err\n",
)
def test_docker_exec_result_shape(self):
from bot_bottle.backend.docker.bottle import DockerBottle
from bot_bottle.backend import ExecResult
bottle = DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
)
with patch("bot_bottle.backend.docker.bottle.subprocess.run",
side_effect=self._stub_run):
result = bottle.exec("echo hi")
self.assertIsInstance(result, ExecResult)
self.assertEqual(0, result.returncode)
self.assertIsInstance(result.stdout, str)
self.assertIsInstance(result.stderr, str)
def test_smolmachines_exec_result_shape(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
from bot_bottle.backend import ExecResult
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run",
side_effect=self._stub_run):
result = bottle.exec("echo hi")
self.assertIsInstance(result, ExecResult)
self.assertEqual(0, result.returncode)
self.assertIsInstance(result.stdout, str)
self.assertIsInstance(result.stderr, str)
# ---------------------------------------------------------------------------
# close() is a no-op / idempotent (ABC contract)
# ---------------------------------------------------------------------------
class TestCloseParity(unittest.TestCase):
def test_docker_close_is_idempotent(self):
from bot_bottle.backend.docker.bottle import DockerBottle
teardown_count = [0]
def count_teardown():
teardown_count[0] += 1
bottle = DockerBottle(
container="bot-bottle-test",
teardown=count_teardown,
prompt_path_in_container=None,
)
bottle.close()
bottle.close()
# DockerBottle.close calls teardown — once per call is fine;
# what matters is it doesn't raise.
def test_smolmachines_close_is_noop(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
bottle.close()
bottle.close()
if __name__ == "__main__":
unittest.main()
+40
View File
@@ -81,6 +81,46 @@ class TestEnumerateActiveAgents(unittest.TestCase):
):
self.assertEqual([a, b], enumerate_active_agents())
def test_sorts_by_started_at_then_slug_across_backends(self):
newer = ActiveAgent(
backend_name="docker", slug="docker-new", agent_name="impl",
started_at="2026-06-02T12:00:00Z", services=(),
)
tie_b = ActiveAgent(
backend_name="docker", slug="b-slug", agent_name="review",
started_at="2026-06-02T11:00:00Z", services=(),
)
missing_metadata = ActiveAgent(
backend_name="smolmachines", slug="missing-metadata",
agent_name="?", started_at="", services=(),
)
tie_a = ActiveAgent(
backend_name="smolmachines", slug="a-slug", agent_name="research",
started_at="2026-06-02T11:00:00Z", services=(),
)
class _FakeBackend:
def __init__(self, items):
self._items = items
def is_available(self):
return True
def enumerate_active(self):
return self._items
with patch.object(
backend_mod, "_BACKENDS",
{
"docker": _FakeBackend([newer, tie_b]),
"smolmachines": _FakeBackend([missing_metadata, tie_a]),
},
):
self.assertEqual(
[missing_metadata, tie_a, tie_b, newer],
enumerate_active_agents(),
)
def test_empty_when_no_backends_have_active(self):
class _FakeBackend:
def is_available(self):
+107
View File
@@ -216,5 +216,112 @@ class TestBottleMetadata(_FakeHomeMixin, unittest.TestCase):
self.assertEqual("t2", loaded.started_at)
class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
"""PRD 0040: backend field is persisted and read back."""
def setUp(self):
self._setup_fake_home()
def tearDown(self):
self._teardown_fake_home()
def test_backend_field_roundtrips_docker(self):
meta = BottleMetadata(
identity="dev-b1",
agent_name="dev",
cwd="",
copy_cwd=False,
started_at="2026-06-02T00:00:00+00:00",
compose_project="bot-bottle-dev-b1",
backend="docker",
)
write_metadata(meta)
loaded = read_metadata("dev-b1")
self.assertIsNotNone(loaded)
assert loaded is not None
self.assertEqual("docker", loaded.backend)
def test_backend_field_roundtrips_smolmachines(self):
meta = BottleMetadata(
identity="dev-b2",
agent_name="dev",
cwd="",
copy_cwd=False,
started_at="2026-06-02T00:00:00+00:00",
compose_project="",
backend="smolmachines",
)
write_metadata(meta)
loaded = read_metadata("dev-b2")
self.assertIsNotNone(loaded)
assert loaded is not None
self.assertEqual("smolmachines", loaded.backend)
def test_missing_backend_field_defaults_to_empty(self):
# Old state dirs written before PRD 0040 have no backend key.
import json
from bot_bottle.backend.docker import bottle_state as bs
path = bs.metadata_path("dev-b3")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps({
"identity": "dev-b3",
"agent_name": "dev",
"cwd": "",
"copy_cwd": False,
"started_at": "2026-06-02T00:00:00+00:00",
"compose_project": "bot-bottle-dev-b3",
}))
loaded = read_metadata("dev-b3")
self.assertIsNotNone(loaded)
assert loaded is not None
self.assertEqual("", loaded.backend)
class TestBottleForSlugBackend(_FakeHomeMixin, unittest.TestCase):
"""PRD 0040: _bottle_for_slug constructs the right bottle type."""
def setUp(self):
self._setup_fake_home()
def tearDown(self):
self._teardown_fake_home()
def test_docker_metadata_returns_docker_bottle(self):
from bot_bottle.backend.docker.bottle import DockerBottle
from bot_bottle.cli.dashboard import _bottle_for_slug
write_metadata(BottleMetadata(
identity="dev-d1",
agent_name="dev",
cwd="",
copy_cwd=False,
started_at="2026-06-02T00:00:00+00:00",
compose_project="bot-bottle-dev-d1",
backend="docker",
))
bottle, _ = _bottle_for_slug("dev-d1", {}, None)
self.assertIsInstance(bottle, DockerBottle)
def test_smolmachines_metadata_returns_smolmachines_bottle(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
from bot_bottle.cli.dashboard import _bottle_for_slug
write_metadata(BottleMetadata(
identity="dev-s1",
agent_name="dev",
cwd="",
copy_cwd=False,
started_at="2026-06-02T00:00:00+00:00",
compose_project="",
backend="smolmachines",
))
bottle, _ = _bottle_for_slug("dev-s1", {}, None)
self.assertIsInstance(bottle, SmolmachinesBottle)
def test_no_metadata_defaults_to_docker_bottle(self):
from bot_bottle.backend.docker.bottle import DockerBottle
from bot_bottle.cli.dashboard import _bottle_for_slug
bottle, _ = _bottle_for_slug("unknown-slug", {}, None)
self.assertIsInstance(bottle, DockerBottle)
if __name__ == "__main__":
unittest.main()
+297
View File
@@ -0,0 +1,297 @@
"""Unit: host Codex auth extraction."""
from __future__ import annotations
import base64
import json
import tempfile
import unittest
from datetime import datetime, timezone
from pathlib import Path
from bot_bottle.codex_auth import (
codex_auth_path,
codex_dummy_auth_json,
codex_host_access_token,
)
from bot_bottle.log import Die
def _jwt(exp: int) -> str:
return _jwt_with_payload({"exp": exp})
def _jwt_with_payload(payload: dict) -> str:
def enc(obj: dict) -> str:
raw = json.dumps(obj, separators=(",", ":")).encode()
return base64.urlsafe_b64encode(raw).decode().rstrip("=")
return f"{enc({'alg': 'none'})}.{enc(payload)}.sig"
def _jwt_payload(token: str) -> dict:
payload = token.split(".")[1]
payload += "=" * (-len(payload) % 4)
return json.loads(base64.urlsafe_b64decode(payload.encode()).decode())
class TestCodexHostAccessToken(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory(prefix="cb-codex-auth.")
self.home = Path(self.tmp.name)
self.auth_path = self.home / "auth.json"
def tearDown(self):
self.tmp.cleanup()
def _write(self, payload: dict) -> None:
self.auth_path.write_text(json.dumps(payload))
def test_auth_path_uses_codex_home(self):
self.assertEqual(
self.auth_path,
codex_auth_path({"CODEX_HOME": str(self.home)}),
)
def test_returns_fresh_chatgpt_access_token(self):
token = _jwt(2000000000)
self._write({
"auth_mode": "chatgpt",
"tokens": {"access_token": token, "refresh_token": "hidden"},
})
out = codex_host_access_token(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
self.assertEqual(token, out)
def test_missing_auth_file_dies(self):
with self.assertRaises(Die):
codex_host_access_token({"CODEX_HOME": str(self.home)})
def test_non_chatgpt_auth_dies(self):
self._write({"auth_mode": "api_key", "tokens": {"access_token": _jwt(2)}})
with self.assertRaises(Die):
codex_host_access_token({"CODEX_HOME": str(self.home)})
def test_user_auth_mode_is_allowed(self):
token = _jwt(2000000000)
self._write({"auth_mode": "user", "tokens": {"access_token": token}})
out = codex_host_access_token(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
self.assertEqual(token, out)
def test_expired_token_dies(self):
self._write({
"auth_mode": "chatgpt",
"tokens": {"access_token": _jwt(1000)},
})
with self.assertRaises(Die):
codex_host_access_token(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
def test_non_jwt_token_dies(self):
self._write({
"auth_mode": "chatgpt",
"tokens": {"access_token": "not-a-jwt"},
})
with self.assertRaises(Die):
codex_host_access_token({"CODEX_HOME": str(self.home)})
def test_dummy_auth_preserves_mode_and_redacts_tokens(self):
access = _jwt(2000000000)
refresh = "host-refresh-token"
self._write({
"auth_mode": "chatgpt",
"OPENAI_API_KEY": None,
"tokens": {
"access_token": access,
"id_token": _jwt(2000000000),
"refresh_token": refresh,
"account_id": "acct-host",
},
"last_refresh": "2026-05-29T00:00:00.000Z",
})
dummy = json.loads(codex_dummy_auth_json(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
))
self.assertEqual("chatgpt", dummy["auth_mode"])
self.assertIsNone(dummy["OPENAI_API_KEY"])
self.assertNotEqual(access, dummy["tokens"]["access_token"])
self.assertNotEqual(refresh, dummy["tokens"]["refresh_token"])
self.assertEqual("bot-bottle-placeholder", dummy["tokens"]["refresh_token"])
self.assertEqual("acct-host", dummy["tokens"]["account_id"])
self.assertIsNotNone(
codex_host_access_token(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
)
def test_dummy_auth_tokens_inherit_host_token_exp(self):
# Codex refreshes when its local access token is at/past exp;
# the dummy must carry the host token's real exp so Codex does
# not drop to the sign-in screen after an artificial TTL while
# egress still holds a valid bearer.
host_exp = 2000000000
self._write({
"auth_mode": "chatgpt",
"tokens": {
"access_token": _jwt(host_exp),
"id_token": _jwt(host_exp),
"refresh_token": "hidden",
},
})
dummy = json.loads(codex_dummy_auth_json(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
))
self.assertEqual(
host_exp, _jwt_payload(dummy["tokens"]["access_token"])["exp"],
)
self.assertEqual(
host_exp, _jwt_payload(dummy["tokens"]["id_token"])["exp"],
)
def test_dummy_auth_replaces_last_refresh_with_valid_timestamp(self):
self._write({
"auth_mode": "chatgpt",
"last_refresh": "host-refresh-metadata",
"tokens": {
"access_token": _jwt(2000000000),
"refresh_token": "hidden",
},
})
dummy = json.loads(codex_dummy_auth_json(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, 2, 3, 4, 5000, tzinfo=timezone.utc),
))
self.assertEqual("2026-01-01T02:03:04.005Z", dummy["last_refresh"])
self.assertNotEqual("host-refresh-metadata", dummy["last_refresh"])
def test_dummy_auth_keeps_required_account_claim_shape(self):
self._write({
"auth_mode": "chatgpt",
"tokens": {
"access_token": _jwt_with_payload({
"exp": 2000000000,
"https://api.openai.com/auth": {
"chatgpt_plan_type": "plus",
"chatgpt_account_id": "acct-real",
"chatgpt_user_id": "user-real",
"user_id": "auth-user-real",
"localhost": True,
},
"https://api.openai.com/profile": {
"email": "real@example.invalid",
"email_verified": True,
},
}),
"id_token": _jwt_with_payload({
"exp": 2000000000,
"email": "real@example.invalid",
"email_verified": True,
"https://api.openai.com/auth": {
"chatgpt_plan_type": "plus",
"chatgpt_account_id": "acct-real",
},
}),
"refresh_token": "hidden",
},
})
dummy = json.loads(codex_dummy_auth_json(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
))
access_payload = _jwt_payload(dummy["tokens"]["access_token"])
auth = access_payload["https://api.openai.com/auth"]
profile = access_payload["https://api.openai.com/profile"]
self.assertEqual("plus", auth["chatgpt_plan_type"])
self.assertEqual("acct-real", auth["chatgpt_account_id"])
self.assertEqual("bot-bottle-placeholder", auth["chatgpt_user_id"])
self.assertEqual("bot-bottle@example.invalid", profile["email"])
self.assertTrue(profile["email_verified"])
def test_dummy_auth_redacts_unknown_future_auth_fields(self):
secrets = [
"top-session-secret",
"top-nested-secret",
"refresh-secret",
"session-token-secret",
"jwt-custom-secret",
"jwt-nested-secret",
"jwt-list-secret",
"id-token-secret",
"auth-claim-secret",
"auth-claim-nested-secret",
"top-list-secret",
"token-nested-secret",
"token-list-secret",
"last-refresh-secret",
]
self._write({
"auth_mode": "chatgpt",
"session_context": "top-session-secret",
"last_refresh": "last-refresh-secret",
"future_nested": {"value": "top-nested-secret"},
"future_list": ["top-list-secret"],
"tokens": {
"access_token": _jwt_with_payload({
"exp": 2000000000,
"custom_session": "jwt-custom-secret",
"future_nested": {"value": "jwt-nested-secret"},
"future_list": ["jwt-list-secret"],
"https://api.openai.com/auth": {
"chatgpt_plan_type": "plus",
"chatgpt_account_id": "acct-real",
"session_context": "auth-claim-secret",
"nested": {"value": "auth-claim-nested-secret"},
},
}),
"id_token": _jwt_with_payload({
"exp": 2000000000,
"opaque": "id-token-secret",
}),
"refresh_token": "refresh-secret",
"session_token": "session-token-secret",
"future_object": {"value": "token-nested-secret"},
"future_list": ["token-list-secret"],
"account_id": "acct-host",
},
})
dummy_json = codex_dummy_auth_json(
{"CODEX_HOME": str(self.home)},
now=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
for secret in secrets:
self.assertNotIn(secret, dummy_json)
dummy = json.loads(dummy_json)
self.assertEqual("bot-bottle-placeholder", dummy["session_context"])
self.assertEqual("2026-01-01T00:00:00.000Z", dummy["last_refresh"])
self.assertEqual({}, dummy["future_nested"])
self.assertEqual([], dummy["future_list"])
self.assertEqual("bot-bottle-placeholder", dummy["tokens"]["refresh_token"])
self.assertEqual("bot-bottle-placeholder", dummy["tokens"]["session_token"])
self.assertEqual({}, dummy["tokens"]["future_object"])
self.assertEqual([], dummy["tokens"]["future_list"])
access_payload = _jwt_payload(dummy["tokens"]["access_token"])
self.assertEqual(
"bot-bottle-placeholder",
access_payload["custom_session"],
)
self.assertEqual({}, access_payload["future_nested"])
self.assertEqual([], access_payload["future_list"])
auth = access_payload["https://api.openai.com/auth"]
self.assertEqual("bot-bottle-placeholder", auth["session_context"])
self.assertEqual({}, auth["nested"])
if __name__ == "__main__":
unittest.main()
+32 -12
View File
@@ -14,6 +14,7 @@ import unittest
from pathlib import Path
from unittest import mock
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.docker.compose import (
@@ -32,6 +33,7 @@ from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
from bot_bottle.manifest import Manifest
from bot_bottle.pipelock import PipelockProxyPlan
from bot_bottle.supervise import SupervisePlan
from bot_bottle.workspace import workspace_plan
SLUG = "demo-abc12"
@@ -47,11 +49,10 @@ def _manifest(*, supervise: bool, with_git: bool, with_egress: bool) -> Manifest
if supervise:
bottle["supervise"] = True
if with_git:
bottle["git"] = {"remotes": {
"example.com": {
"Name": "upstream",
"Upstream": "ssh://git@example.com:22/x/y.git",
"IdentityFile": "/etc/hostname", # any existing file
bottle["git-gate"] = {"repos": {
"upstream": {
"url": "ssh://git@example.com:22/x/y.git",
"identity": "/etc/hostname", # any existing file
},
}}
if with_egress:
@@ -149,7 +150,6 @@ def _plan(
identity_file="/etc/hostname",
known_host_key="",
known_hosts_file=STATE / "git-gate" / "upstream-known_hosts",
extra_hosts={"example.com": "10.0.0.1"},
),)
routes: tuple[EgressRoute, ...] = ()
if with_egress:
@@ -162,8 +162,9 @@ def _plan(
roles=(),
),)
spec = _spec(supervise=supervise, with_git=with_git, with_egress=with_egress)
return DockerBottlePlan(
spec=_spec(supervise=supervise, with_git=with_git, with_egress=with_egress),
spec=spec,
stage_dir=STAGE,
slug=SLUG,
container_name=f"bot-bottle-{SLUG}",
@@ -180,6 +181,15 @@ def _plan(
egress_plan=_egress_plan(routes),
supervise_plan=_supervise_plan() if supervise else None,
use_runsc=False,
agent_provision=AgentProvisionPlan(
template="claude",
command="claude",
prompt_mode="append_file",
image="bot-bottle-claude:latest",
dockerfile="",
guest_env={},
),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
)
@@ -250,6 +260,20 @@ class TestAgentAlwaysPresent(unittest.TestCase):
s = bottle_plan_to_compose(_plan())["services"]["agent"]
self.assertIn("CLAUDE_CODE_OAUTH_TOKEN", s["environment"])
def test_agent_provider_env_uses_literal_values(self):
plan = _plan()
provision = AgentProvisionPlan(
template="codex",
command="codex",
prompt_mode="read_prompt_file",
image="bot-bottle-codex:latest",
dockerfile="",
guest_env={"CODEX_HOME": "/home/node/.codex"},
)
plan = type(plan)(**{**vars(plan), "agent_provision": provision})
s = bottle_plan_to_compose(plan)["services"]["agent"]
self.assertIn("CODEX_HOME=/home/node/.codex", s["environment"])
def test_agent_runsc_runtime(self):
plan = _plan()
plan = type(plan)(**{**vars(plan), "use_runsc": True})
@@ -414,12 +438,8 @@ class TestSidecarBundleShape(unittest.TestCase):
self.assertTrue(any("supervise/queue" in t or t.startswith("/run/supervise")
for t in targets))
def test_extra_hosts_emitted_for_git_upstreams(self):
def test_extra_hosts_omitted_for_git_upstreams(self):
sc = self._render(with_git=True)["services"]["sidecars"]
self.assertIn("example.com:10.0.0.1", sc.get("extra_hosts", []))
def test_extra_hosts_omitted_when_no_git(self):
sc = self._render()["services"]["sidecars"]
self.assertNotIn("extra_hosts", sc)
def test_agent_depends_on_bundle_only(self):
+49
View File
@@ -577,5 +577,54 @@ class TestEditInEditor(unittest.TestCase):
os.environ["EDITOR"] = original_editor
class TestCapabilityBlockSmolmachinesGuard(_FakeHomeMixin, unittest.TestCase):
"""approve() must refuse capability-block for smolmachines bottles and
pass it through for Docker bottles (PRD 0039)."""
def setUp(self):
self._setup_fake_home()
self._original_apply_capability = dashboard.apply_capability_change
dashboard.apply_capability_change = lambda slug, content: ("", content)
def tearDown(self):
dashboard.apply_capability_change = self._original_apply_capability
self._teardown_fake_home()
def _enqueue_capability(self, slug: str = "dev") -> "dashboard.QueuedProposal":
p = _proposal(slug=slug, tool=TOOL_CAPABILITY_BLOCK)
qdir = supervise.queue_dir_for_slug(slug)
qdir.mkdir(parents=True, exist_ok=True)
supervise.write_proposal(qdir, p)
return dashboard.QueuedProposal(proposal=p, queue_dir=qdir)
def _write_metadata(self, slug: str, compose_project: str) -> None:
from bot_bottle.backend.docker.bottle_state import BottleMetadata, write_metadata
write_metadata(BottleMetadata(
identity=slug,
agent_name="myagent",
cwd="",
copy_cwd=False,
started_at="2026-06-02T00:00:00+00:00",
compose_project=compose_project,
))
def test_smolmachines_bottle_raises_capability_apply_error(self):
self._write_metadata("dev", compose_project="")
qp = self._enqueue_capability("dev")
with self.assertRaises(CapabilityApplyError) as ctx:
dashboard.approve(qp)
self.assertIn("smolmachines", str(ctx.exception))
def test_docker_bottle_calls_apply_capability_change(self):
self._write_metadata("dev", compose_project="bot-bottle-dev")
qp = self._enqueue_capability("dev")
dashboard.approve(qp) # must not raise
def test_no_metadata_falls_through_to_docker_path(self):
# No metadata at all → assume Docker (backward-compatible).
qp = self._enqueue_capability("dev")
dashboard.approve(qp) # must not raise
if __name__ == "__main__":
unittest.main()
+145
View File
@@ -0,0 +1,145 @@
"""Unit: Docker launch teardown warning on ExitStack failure (issue #156).
When a callback registered in the ExitStack raises during teardown,
the teardown function must emit a WARNING-level message that includes
the container name and operation type, rather than silently discarding
the exception.
"""
from __future__ import annotations
import contextlib
import io
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import launch as launch_mod
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.egress import EgressPlan
from bot_bottle.git_gate import GitGatePlan
from bot_bottle.manifest import Manifest
from bot_bottle.pipelock import PipelockProxyPlan
from bot_bottle.workspace import workspace_plan
def _manifest() -> Manifest:
return Manifest.from_json_obj({
"bottles": {"dev": {}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def _plan(tmp: str) -> DockerBottlePlan:
stage = Path(tmp)
manifest = _manifest()
spec = BottleSpec(
manifest=manifest,
agent_name="demo",
copy_cwd=False,
user_cwd=tmp,
identity="test-teardown-00001",
)
return DockerBottlePlan(
spec=spec,
stage_dir=stage,
git_gate_plan=GitGatePlan(
slug="test-teardown-00001",
entrypoint_script=stage / "entrypoint.sh",
hook_script=stage / "hook.sh",
access_hook_script=stage / "access-hook.sh",
upstreams=(),
),
egress_plan=EgressPlan(
slug="test-teardown-00001",
routes_path=stage / "egress.yaml",
routes=(),
token_env_map={},
),
supervise_plan=None,
agent_provision=AgentProvisionPlan(
template="claude",
command="claude",
prompt_mode="append_file",
image="",
dockerfile="",
guest_env={},
),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
slug="test-teardown-00001",
container_name="bot-bottle-test-teardown-abc",
container_name_pinned=False,
image="bot-bottle-claude:latest",
derived_image="",
runtime_image="bot-bottle-claude:latest",
dockerfile_path="",
env_file=stage / "env",
forwarded_env={},
prompt_file=stage / "prompt.txt",
proxy_plan=PipelockProxyPlan(
yaml_path=stage / "pipelock.yaml",
slug="test-teardown-00001",
),
use_runsc=False,
)
class TestTeardownWarning(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.mkdtemp(prefix="docker-launch-teardown-test.")
def tearDown(self) -> None:
import shutil
shutil.rmtree(self._tmp, ignore_errors=True)
def test_teardown_failure_emits_warning_with_container_and_operation(self):
plan = _plan(self._tmp)
buf = io.StringIO()
with mock.patch.object(launch_mod.docker_mod, "build_image"), \
mock.patch.object(
launch_mod, "pipelock_tls_init",
return_value=(Path("/ca.crt"), Path("/ca.key")),
), \
mock.patch.object(
launch_mod, "egress_tls_init",
return_value=(Path("/egress_ca"), Path("/egress_cert")),
), \
mock.patch.object(
launch_mod.network_mod, "network_name_for_slug",
return_value="bb-internal-test",
), \
mock.patch.object(
launch_mod.network_mod, "network_egress_name_for_slug",
return_value="bb-egress-test",
), \
mock.patch.object(
launch_mod, "bottle_plan_to_compose",
return_value={"services": {"agent": {}}},
), \
mock.patch.object(
launch_mod, "write_compose_file",
return_value=Path("/tmp/compose.yml"),
), \
mock.patch.object(launch_mod, "compose_up"), \
mock.patch.object(launch_mod, "compose_dump_logs"), \
mock.patch.object(
launch_mod, "compose_down",
side_effect=RuntimeError("network remove failed"),
), \
contextlib.redirect_stderr(buf):
provision = mock.Mock(return_value=None)
with launch_mod.launch(plan, provision=provision):
pass
output = buf.getvalue()
self.assertIn("bot-bottle: warning:", output)
self.assertIn("bot-bottle-test-teardown-abc", output)
self.assertIn("compose-down", output)
if __name__ == "__main__":
unittest.main()
+37 -2
View File
@@ -13,6 +13,7 @@ import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.docker.provision import git as _git
@@ -20,20 +21,23 @@ from bot_bottle.egress import EgressPlan
from bot_bottle.git_gate import GitGatePlan
from bot_bottle.manifest import Manifest
from bot_bottle.pipelock import PipelockProxyPlan
from bot_bottle.workspace import workspace_plan
def _plan(*, git_user: dict | None = None,
copy_cwd: bool = False,
user_cwd: str = "/tmp/x",
stage_dir: Path | None = None) -> DockerBottlePlan:
bottle_json: dict = {}
if git_user is not None:
bottle_json["git"] = {"user": git_user}
bottle_json["git-gate"] = {"user": git_user}
manifest = Manifest.from_json_obj({
"bottles": {"dev": bottle_json},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
spec = BottleSpec(
manifest=manifest, agent_name="demo",
copy_cwd=False, user_cwd="/tmp/x",
copy_cwd=copy_cwd, user_cwd=user_cwd,
)
return DockerBottlePlan(
spec=spec,
@@ -66,6 +70,15 @@ def _plan(*, git_user: dict | None = None,
),
supervise_plan=None,
use_runsc=False,
agent_provision=AgentProvisionPlan(
template="claude",
command="claude",
prompt_mode="append_file",
image="bot-bottle-claude:latest",
dockerfile="",
guest_env={},
),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
)
@@ -97,6 +110,28 @@ class TestProvisionGitUser(unittest.TestCase):
)
self.assertEqual([], _git_config_calls(run))
def test_copies_cwd_git_to_workspace_plan_path(self):
cwd = self.stage / "cwd"
(cwd / ".git").mkdir(parents=True)
plan = _plan(copy_cwd=True, user_cwd=str(cwd), stage_dir=self.stage)
with patch.object(_git.subprocess, "run") as run:
_git._provision_cwd_git(plan, "bot-bottle-demo-abc12")
self.assertEqual(
[
"docker", "cp", f"{cwd}/.git",
"bot-bottle-demo-abc12:/home/node/workspace/.git",
],
run.call_args_list[0].args[0],
)
self.assertEqual(
[
"docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chown", "-R", "node:node", "/home/node/workspace/.git",
],
run.call_args_list[1].args[0],
)
def test_sets_name_and_email(self):
plan = _plan(
git_user={"name": "Eric Bauerfeld", "email": "eric@dideric.is"},
@@ -0,0 +1,191 @@
"""Unit: docker provider auth marker provisioning."""
from __future__ import annotations
import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle.agent_provider import (
AgentProvisionDir,
AgentProvisionFile,
AgentProvisionPlan,
)
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.docker.provision import provider_auth as _provider_auth
from bot_bottle.egress import EgressPlan
from bot_bottle.git_gate import GitGatePlan
from bot_bottle.manifest import Manifest
from bot_bottle.pipelock import PipelockProxyPlan
from bot_bottle.workspace import workspace_plan
def _plan(
*,
codex_auth_file: Path | None = None,
agent_provider_template: str = "codex",
) -> DockerBottlePlan:
manifest = Manifest.from_json_obj({
"bottles": {"dev": {"agent_provider": {"template": "codex"}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
spec = BottleSpec(
manifest=manifest,
agent_name="demo",
copy_cwd=False,
user_cwd="/tmp/x",
)
return DockerBottlePlan(
spec=spec,
stage_dir=Path("/tmp/stage"),
slug="demo-abc12",
container_name="bot-bottle-demo-abc12",
container_name_pinned=False,
image="bot-bottle-codex:latest",
derived_image="",
runtime_image="bot-bottle-codex:latest",
dockerfile_path="",
env_file=Path("/tmp/agent.env"),
forwarded_env={},
prompt_file=Path("/tmp/prompt.txt"),
proxy_plan=PipelockProxyPlan(
yaml_path=Path("/tmp/pipelock.yaml"),
slug="demo-abc12",
),
git_gate_plan=GitGatePlan(
slug="demo-abc12",
entrypoint_script=Path("/tmp/git-gate-entrypoint.sh"),
hook_script=Path("/tmp/git-gate-hook"),
access_hook_script=Path("/tmp/git-gate-access-hook"),
upstreams=(),
),
egress_plan=EgressPlan(
slug="demo-abc12",
routes_path=Path("/tmp/routes.yaml"),
routes=(),
token_env_map={},
),
supervise_plan=None,
use_runsc=False,
agent_provision=_agent_provision(
agent_provider_template, codex_auth_file=codex_auth_file,
),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
)
def _agent_provision(
template: str, *, codex_auth_file: Path | None = None,
) -> AgentProvisionPlan:
if template != "codex":
return AgentProvisionPlan(
template=template,
command=template,
prompt_mode="append_file",
image="",
dockerfile="",
guest_env={},
)
files = [
AgentProvisionFile(
Path("/tmp/codex-config.toml"),
"/home/node/.codex/config.toml",
),
]
if codex_auth_file is not None:
files.append(AgentProvisionFile(
codex_auth_file,
"/home/node/.codex/auth.json",
))
return AgentProvisionPlan(
template="codex",
command="codex",
prompt_mode="read_prompt_file",
image="bot-bottle-codex:latest",
dockerfile="",
guest_env={},
dirs=(AgentProvisionDir("/home/node/.codex"),),
files=tuple(files),
)
class TestProvisionProviderAuth(unittest.TestCase):
def test_noop_for_non_codex_provider(self):
with patch.object(_provider_auth.subprocess, "run") as run:
_provider_auth.provision_provider_auth(
_plan(agent_provider_template="claude"), "bot-bottle-demo-abc12",
)
self.assertEqual(0, run.call_count)
def test_codex_provider_trusts_launch_dir_without_auth_file(self):
with patch.object(_provider_auth.subprocess, "run") as run:
_provider_auth.provision_provider_auth(
_plan(), "bot-bottle-demo-abc12",
)
argvs = [call.args[0] for call in run.call_args_list]
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"mkdir", "-p", "/home/node/.codex"],
argvs,
)
trust_config = next(
a for a in argvs
if a[:2] == ["docker", "cp"] and a[2] == "/tmp/codex-config.toml"
)
self.assertEqual(
"bot-bottle-demo-abc12:/home/node/.codex/config.toml",
trust_config[3],
)
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chown", "node:node", "/home/node/.codex/config.toml"],
argvs,
)
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chmod", "600", "/home/node/.codex/config.toml"],
argvs,
)
def test_copies_dummy_auth_json_to_codex_home(self):
with patch.object(_provider_auth.subprocess, "run") as run:
_provider_auth.provision_provider_auth(
_plan(codex_auth_file=Path("/tmp/codex-auth.json")),
"bot-bottle-demo-abc12",
)
argvs = [call.args[0] for call in run.call_args_list]
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"mkdir", "-p", "/home/node/.codex"],
argvs,
)
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chown", "node:node", "/home/node/.codex"],
argvs,
)
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chmod", "700", "/home/node/.codex"],
argvs,
)
self.assertIn(
["docker", "cp", "/tmp/codex-auth.json",
"bot-bottle-demo-abc12:/home/node/.codex/auth.json"],
argvs,
)
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chown", "node:node", "/home/node/.codex/auth.json"],
argvs,
)
self.assertIn(
["docker", "exec", "-u", "0", "bot-bottle-demo-abc12",
"chmod", "600", "/home/node/.codex/auth.json"],
argvs,
)
if __name__ == "__main__":
unittest.main()
+58
View File
@@ -8,10 +8,13 @@ integration smoke."""
from __future__ import annotations
import subprocess
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle.backend.docker import util as docker_mod
from bot_bottle.workspace import WorkspacePlan
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess:
@@ -67,5 +70,60 @@ class TestSave(unittest.TestCase):
)
class TestBuildImageWithCwd(unittest.TestCase):
def test_uses_workspace_plan_paths(self):
with tempfile.TemporaryDirectory(prefix="bb-docker-cwd.") as tmp:
workspace = WorkspacePlan(
enabled=True,
host_path=Path(tmp),
guest_home="/guest/home",
guest_path="/guest/home/workspace",
workdir="/guest/home/workspace",
)
with patch.object(docker_mod.subprocess, "run") as run:
docker_mod.build_image_with_cwd("derived:tag", "base:tag", workspace)
argv = run.call_args.args[0]
dockerfile = run.call_args.kwargs["input"]
self.assertEqual(["docker", "build", "-t", "derived:tag", "-f", "-"], argv[:6])
self.assertTrue(argv[6].endswith("/context"))
self.assertIn("FROM base:tag\n", dockerfile)
self.assertIn(
"COPY --chown=node:node workspace/. /guest/home/workspace\n",
dockerfile,
)
self.assertIn("WORKDIR /guest/home/workspace\n", dockerfile)
def test_staged_context_includes_hidden_files_but_not_git_dir(self):
with tempfile.TemporaryDirectory(prefix="bb-docker-cwd.") as tmp:
root = Path(tmp)
(root / ".gitignore").write_text("*.pyc\n")
(root / ".dockerignore").write_text(".gitignore\n")
(root / ".env.example").write_text("SAFE=1\n")
(root / ".git").mkdir()
(root / ".git" / "config").write_text("[core]\n")
workspace = WorkspacePlan(
enabled=True,
host_path=root,
guest_home="/guest/home",
guest_path="/guest/home/workspace",
workdir="/guest/home/workspace",
)
def inspect_context(*args, **kwargs):
context = Path(args[0][-1])
staged = context / "workspace"
self.assertTrue((staged / ".gitignore").is_file())
self.assertTrue((staged / ".dockerignore").is_file())
self.assertTrue((staged / ".env.example").is_file())
self.assertFalse((staged / ".git").exists())
return _ok()
with patch.object(
docker_mod.subprocess, "run", side_effect=inspect_context,
):
docker_mod.build_image_with_cwd("derived:tag", "base:tag", workspace)
if __name__ == "__main__":
unittest.main()
+148 -14
View File
@@ -4,6 +4,8 @@ resolution (PRD 0017)."""
import unittest
from bot_bottle.egress import (
CODEX_HOST_CREDENTIAL_TOKEN_REF,
EgressRoute,
egress_manifest_routes,
egress_render_routes,
egress_resolve_token_values,
@@ -22,8 +24,19 @@ def _bottle(routes):
}).bottles["dev"]
class TestRoutesForBottle(unittest.TestCase):
def test_authenticated_route_gets_slot(self):
def _provider_route(host: str, token_ref: str, *, tls_passthrough: bool = False) -> EgressRoute:
return EgressRoute(
host=host,
auth_scheme="Bearer",
token_ref=token_ref,
tls_passthrough=tls_passthrough,
)
class TestManifestRouteLift(unittest.TestCase):
"""egress_manifest_routes is a pure lifter — no slot assignment."""
def test_authenticated_route_lifted_without_slot(self):
b = _bottle([{
"host": "api.github.com",
"auth": {"scheme": "Bearer", "token_ref": "GH_PAT"},
@@ -33,8 +46,8 @@ class TestRoutesForBottle(unittest.TestCase):
r = routes[0]
self.assertEqual("api.github.com", r.host)
self.assertEqual("Bearer", r.auth_scheme)
self.assertEqual("EGRESS_TOKEN_0", r.token_env)
self.assertEqual("GH_PAT", r.token_ref)
self.assertEqual("", r.token_env) # slot assigned later
self.assertEqual((), r.path_allowlist)
def test_unauthenticated_route_has_empty_auth_fields(self):
@@ -46,6 +59,20 @@ class TestRoutesForBottle(unittest.TestCase):
self.assertEqual("", r.token_ref)
self.assertEqual(("/x/",), r.path_allowlist)
class TestSlotAssignment(unittest.TestCase):
"""Slot assignment happens in egress_routes_for_bottle."""
def test_authenticated_route_gets_slot(self):
b = _bottle([{
"host": "api.github.com",
"auth": {"scheme": "Bearer", "token_ref": "GH_PAT"},
}])
routes = egress_routes_for_bottle(b)
r = routes[0]
self.assertEqual("EGRESS_TOKEN_0", r.token_env)
self.assertEqual("GH_PAT", r.token_ref)
def test_shared_token_ref_collapses_to_one_slot(self):
b = _bottle([
{"host": "api.github.com",
@@ -53,7 +80,7 @@ class TestRoutesForBottle(unittest.TestCase):
{"host": "github.com",
"auth": {"scheme": "Bearer", "token_ref": "GH_PAT"}},
])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
slots = {r.token_env for r in routes}
self.assertEqual({"EGRESS_TOKEN_0"}, slots)
@@ -64,7 +91,7 @@ class TestRoutesForBottle(unittest.TestCase):
{"host": "b.example",
"auth": {"scheme": "Bearer", "token_ref": "T2"}},
])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
slots = [r.token_env for r in routes]
self.assertEqual(["EGRESS_TOKEN_0", "EGRESS_TOKEN_1"], slots)
@@ -78,15 +105,14 @@ class TestRoutesForBottle(unittest.TestCase):
{"host": "b.example",
"auth": {"scheme": "Bearer", "token_ref": "T2"}},
])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
authed = [r.token_env for r in routes if r.token_env]
self.assertEqual(["EGRESS_TOKEN_0", "EGRESS_TOKEN_1"], authed)
self.assertEqual("", routes[1].token_env)
class TestRoutesForBottleUsesManifestOnly(unittest.TestCase):
"""The effective route table is exactly the manifest-declared
routes. Provider defaults are not injected implicitly."""
class TestRoutesForBottleManifestOnly(unittest.TestCase):
"""Without provider routes the effective table is exactly the manifest."""
def test_no_manifest_routes_means_no_effective_routes(self):
b = _bottle([])
@@ -107,6 +133,107 @@ class TestRoutesForBottleUsesManifestOnly(unittest.TestCase):
effective = [r.host for r in egress_routes_for_bottle(b)]
self.assertEqual(["x.example"], effective)
def test_tls_passthrough_lifted_from_manifest(self):
b = _bottle([{
"host": "api.openai.com",
"auth": {"scheme": "Bearer", "token_ref": "T"},
"pipelock": {"tls_passthrough": True},
}])
routes = egress_routes_for_bottle(b)
self.assertTrue(routes[0].tls_passthrough)
def test_tls_passthrough_false_by_default(self):
b = _bottle([{"host": "api.github.com"}])
routes = egress_routes_for_bottle(b)
self.assertFalse(routes[0].tls_passthrough)
class TestProviderRouteMerge(unittest.TestCase):
"""Provider routes win on host collision; manifest fills the rest."""
def test_provider_route_appended_when_not_in_manifest(self):
b = _bottle([])
pr = _provider_route("api.openai.com", "TOK")
routes = egress_routes_for_bottle(b, (pr,))
self.assertEqual(1, len(routes))
self.assertEqual("api.openai.com", routes[0].host)
self.assertEqual("Bearer", routes[0].auth_scheme)
self.assertEqual("EGRESS_TOKEN_0", routes[0].token_env)
self.assertEqual("TOK", routes[0].token_ref)
def test_unauthenticated_provider_route_appends_without_token_slot(self):
b = _bottle([])
pr = EgressRoute(host="api.openai.com", tls_passthrough=True)
routes = egress_routes_for_bottle(b, (pr,))
self.assertEqual(1, len(routes))
self.assertEqual("api.openai.com", routes[0].host)
self.assertEqual("", routes[0].auth_scheme)
self.assertEqual("", routes[0].token_env)
self.assertEqual("", routes[0].token_ref)
self.assertEqual({}, egress_token_env_map(routes))
def test_provider_route_wins_over_bare_manifest_route(self):
# Provisioned host wins outright; manifest path_allowlist is dropped.
b = _bottle([{"host": "api.openai.com", "path_allowlist": ["/v1/"]}])
pr = EgressRoute(host="api.openai.com", tls_passthrough=True)
routes = egress_routes_for_bottle(b, (pr,))
self.assertEqual(1, len(routes))
self.assertEqual("", routes[0].auth_scheme)
self.assertEqual("", routes[0].token_env)
self.assertEqual("", routes[0].token_ref)
self.assertTrue(routes[0].tls_passthrough)
self.assertEqual((), routes[0].path_allowlist)
self.assertEqual({}, egress_token_env_map(routes))
def test_two_provider_routes_with_same_token_ref_share_slot(self):
b = _bottle([])
routes = egress_routes_for_bottle(b, (
_provider_route("api.openai.com", CODEX_HOST_CREDENTIAL_TOKEN_REF),
_provider_route("chatgpt.com", CODEX_HOST_CREDENTIAL_TOKEN_REF),
))
self.assertEqual(["api.openai.com", "chatgpt.com"], [r.host for r in routes])
self.assertEqual("EGRESS_TOKEN_0", routes[0].token_env)
self.assertEqual("EGRESS_TOKEN_0", routes[1].token_env)
def test_provider_route_wins_over_authed_manifest_route(self):
# Provider wins even when manifest has its own auth for the host.
b = _bottle([{"host": "chatgpt.com",
"path_allowlist": ["/backend-api/"],
"auth": {"scheme": "Bearer", "token_ref": "OTHER"}}])
pr = _provider_route("chatgpt.com", CODEX_HOST_CREDENTIAL_TOKEN_REF)
routes = egress_routes_for_bottle(b, (pr,))
self.assertEqual(1, len(routes))
self.assertEqual("chatgpt.com", routes[0].host)
self.assertEqual("Bearer", routes[0].auth_scheme)
self.assertEqual("EGRESS_TOKEN_0", routes[0].token_env)
self.assertEqual(CODEX_HOST_CREDENTIAL_TOKEN_REF, routes[0].token_ref)
self.assertEqual((), routes[0].path_allowlist)
def test_manifest_route_preserved_for_non_provisioned_host(self):
b = _bottle([
{"host": "api.openai.com"},
{"host": "api.github.com",
"auth": {"scheme": "Bearer", "token_ref": "GH_PAT"}},
])
pr = _provider_route("api.openai.com", CODEX_HOST_CREDENTIAL_TOKEN_REF)
routes = egress_routes_for_bottle(b, (pr,))
hosts = [r.host for r in routes]
self.assertEqual(["api.openai.com", "api.github.com"], hosts)
self.assertEqual(CODEX_HOST_CREDENTIAL_TOKEN_REF, routes[0].token_ref)
self.assertEqual("GH_PAT", routes[1].token_ref)
def test_provider_route_tls_passthrough_set_on_appended_route(self):
b = _bottle([])
pr = _provider_route("api.openai.com", "TOK", tls_passthrough=True)
routes = egress_routes_for_bottle(b, (pr,))
self.assertTrue(routes[0].tls_passthrough)
def test_provider_route_tls_passthrough_wins_over_bare_manifest_route(self):
b = _bottle([{"host": "api.openai.com"}])
pr = _provider_route("api.openai.com", "TOK", tls_passthrough=True)
routes = egress_routes_for_bottle(b, (pr,))
self.assertTrue(routes[0].tls_passthrough)
class TestTokenEnvMap(unittest.TestCase):
def test_only_authenticated_routes_contribute(self):
@@ -115,7 +242,7 @@ class TestTokenEnvMap(unittest.TestCase):
"auth": {"scheme": "Bearer", "token_ref": "T1"}},
{"host": "passthrough.example"},
])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
m = egress_token_env_map(routes)
self.assertEqual({"EGRESS_TOKEN_0": "T1"}, m)
@@ -139,7 +266,7 @@ class TestRenderRoutes(unittest.TestCase):
"auth": {"scheme": "Bearer", "token_ref": "GH_PAT"},
"path_allowlist": ["/repos/x/"],
}])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
parsed = self._parsed(routes)
self.assertEqual(
[{
@@ -157,7 +284,7 @@ class TestRenderRoutes(unittest.TestCase):
# enforces both-or-neither, so emitting empty strings would
# round-trip as a partial pair and crash.
b = _bottle([{"host": "github.com", "path_allowlist": ["/x/"]}])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
entry = self._parsed(routes)[0]
self.assertNotIn("auth_scheme", entry)
self.assertNotIn("token_env", entry)
@@ -167,7 +294,7 @@ class TestRenderRoutes(unittest.TestCase):
"host": "api.anthropic.com",
"auth": {"scheme": "Bearer", "token_ref": "CL"},
}])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
self.assertNotIn("path_allowlist", self._parsed(routes)[0])
def test_empty_routes_round_trips(self):
@@ -186,7 +313,7 @@ class TestRenderRoutes(unittest.TestCase):
{"host": "github.com", "path_allowlist": ["/x/"]},
{"host": "api.anthropic.com"},
])
routes = egress_manifest_routes(b)
routes = egress_routes_for_bottle(b)
addon_routes = load_routes(egress_render_routes(routes))
self.assertEqual(3, len(addon_routes))
self.assertEqual("Bearer", addon_routes[0].auth_scheme)
@@ -217,6 +344,13 @@ class TestResolveTokenValues(unittest.TestCase):
{"GH_PAT": ""},
)
def test_codex_host_credential_ref_resolved_via_provisioned_env(self):
out = egress_resolve_token_values(
{"EGRESS_TOKEN_0": CODEX_HOST_CREDENTIAL_TOKEN_REF},
{CODEX_HOST_CREDENTIAL_TOKEN_REF: "codex-access-token"},
)
self.assertEqual({"EGRESS_TOKEN_0": "codex-access-token"}, out)
if __name__ == "__main__":
unittest.main()
+90
View File
@@ -4,7 +4,14 @@ These tests target `egress_addon_core` — the host-importable
half of the addon. The mitmproxy hook wrapper in
`egress_addon.py` is container-only and is not exercised here."""
import http.server
import subprocess
import tempfile
import threading
import time
import unittest
from pathlib import Path
from urllib.parse import urlsplit
from bot_bottle.egress_addon_core import (
Decision,
@@ -326,5 +333,88 @@ class TestIsGitPushRequest(unittest.TestCase):
self.assertFalse(is_git_push_request("/", ""))
class TestGitPushBlockFailFast(unittest.TestCase):
def test_real_git_push_fails_fast_when_egress_blocks_receive_pack(self):
"""A real git client should see egress's HTTPS-push 403 and exit.
The local server stands in for the egress proxy response after
CONNECT/TLS interception; git smart-HTTP uses the same paths over
plain HTTP here, which keeps this regression test hermetic.
"""
seen_paths: list[str] = []
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
self._handle()
def do_POST(self):
self._handle()
def _handle(self):
parsed = urlsplit(self.path)
seen_paths.append(self.path)
if is_git_push_request(parsed.path, parsed.query):
body = (
b"egress: git push over HTTPS is not supported; "
b"use the bottle.git SSH path (gitleaks-scanned by "
b"git-gate's pre-receive hook)."
)
self.send_response(403)
self.send_header("Content-Type", "text/plain; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
return
self.send_response(404)
self.send_header("Content-Length", "0")
self.end_headers()
def log_message(self, _fmt, *_args):
pass
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
with tempfile.TemporaryDirectory() as tmp:
repo = Path(tmp) / "repo"
repo.mkdir()
subprocess.run(["git", "init"], cwd=repo, check=True,
capture_output=True, text=True)
subprocess.run(["git", "config", "user.name", "test"],
cwd=repo, check=True)
subprocess.run(["git", "config", "user.email", "test@example.invalid"],
cwd=repo, check=True)
(repo / "README.md").write_text("test\n")
subprocess.run(["git", "add", "README.md"], cwd=repo, check=True)
subprocess.run(["git", "commit", "-m", "test"],
cwd=repo, check=True, capture_output=True, text=True)
remote = f"http://127.0.0.1:{server.server_port}/owner/repo.git"
subprocess.run(["git", "remote", "add", "origin", remote],
cwd=repo, check=True)
started = time.monotonic()
result = subprocess.run(
["git", "push", "origin", "HEAD:refs/heads/main"],
cwd=repo,
capture_output=True,
text=True,
timeout=5,
check=False,
)
elapsed = time.monotonic() - started
self.assertNotEqual(0, result.returncode)
self.assertLess(elapsed, 5)
self.assertTrue(
any("service=git-receive-pack" in p for p in seen_paths),
f"git did not request receive-pack capabilities; saw {seen_paths!r}",
)
self.assertIn("403", result.stderr)
if __name__ == "__main__":
unittest.main()
+55 -94
View File
@@ -9,14 +9,12 @@ from bot_bottle.git_gate import (
GitGate,
GitGatePlan,
GitGateUpstream,
git_gate_aggregate_extra_hosts,
git_gate_known_hosts_line,
git_gate_render_access_hook,
git_gate_render_entrypoint,
git_gate_render_hook,
git_gate_upstreams_for_bottle,
)
from bot_bottle.log import Die
from bot_bottle.manifest import Manifest
from tests.fixtures import fixture_minimal, fixture_with_git
@@ -46,86 +44,6 @@ class TestUpstreamsForBottle(unittest.TestCase):
self.assertEqual((), git_gate_upstreams_for_bottle(bottle))
class TestExtraHostsPlumbing(unittest.TestCase):
def test_upstream_carries_extra_hosts_from_manifest(self):
m = Manifest.from_json_obj({
"bottles": {
"dev": {
"git": {"remotes": {
"gitea.dideric.is": {
"Name": "bot-bottle",
"Upstream": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"IdentityFile": "/dev/null",
"ExtraHosts": {"gitea.dideric.is": "100.78.141.42"},
},
}},
},
},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
ups = git_gate_upstreams_for_bottle(m.bottles["dev"])
self.assertEqual(
{"gitea.dideric.is": "100.78.141.42"}, dict(ups[0].extra_hosts)
)
def test_aggregator_merges_distinct_hostnames(self):
ups = (
GitGateUpstream(
name="a", upstream_url="", upstream_host="", upstream_port="",
identity_file="", known_host_key="",
extra_hosts={"a.example": "10.0.0.1"},
),
GitGateUpstream(
name="b", upstream_url="", upstream_host="", upstream_port="",
identity_file="", known_host_key="",
extra_hosts={"b.example": "10.0.0.2"},
),
)
self.assertEqual(
{"a.example": "10.0.0.1", "b.example": "10.0.0.2"},
git_gate_aggregate_extra_hosts(ups),
)
def test_aggregator_allows_same_host_same_ip(self):
# Two entries listing the same host:ip is harmless duplication,
# not a conflict. The gate's /etc/hosts ends up with one line.
ups = (
GitGateUpstream(
name="a", upstream_url="", upstream_host="", upstream_port="",
identity_file="", known_host_key="",
extra_hosts={"gitea.dideric.is": "100.78.141.42"},
),
GitGateUpstream(
name="b", upstream_url="", upstream_host="", upstream_port="",
identity_file="", known_host_key="",
extra_hosts={"gitea.dideric.is": "100.78.141.42"},
),
)
self.assertEqual(
{"gitea.dideric.is": "100.78.141.42"},
git_gate_aggregate_extra_hosts(ups),
)
def test_aggregator_rejects_conflicting_ips(self):
ups = (
GitGateUpstream(
name="a", upstream_url="", upstream_host="", upstream_port="",
identity_file="", known_host_key="",
extra_hosts={"gitea.dideric.is": "100.78.141.42"},
),
GitGateUpstream(
name="b", upstream_url="", upstream_host="", upstream_port="",
identity_file="", known_host_key="",
extra_hosts={"gitea.dideric.is": "10.0.0.99"},
),
)
with self.assertRaises(Die):
git_gate_aggregate_extra_hosts(ups)
def test_aggregator_empty_is_empty(self):
self.assertEqual({}, git_gate_aggregate_extra_hosts(()))
class TestKnownHostsLine(unittest.TestCase):
def test_default_port_unbracketed(self):
line = git_gate_known_hosts_line("github.com", "22", "ssh-ed25519 AAAA")
@@ -158,19 +76,28 @@ class TestEntrypointRender(unittest.TestCase):
)
script = git_gate_render_entrypoint(ups)
self.assertIn("#!/bin/sh", script)
self.assertIn(
"init_repo 'bot-bottle' "
"'ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git'",
script,
# shlex.quote leaves safe strings unquoted; verify via token parse.
import shlex as _shlex
lines_with_init = [l for l in script.splitlines() if l.startswith("init_repo ")]
self.assertEqual(2, len(lines_with_init))
self.assertEqual(
["init_repo", "bot-bottle",
"ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git"],
_shlex.split(lines_with_init[0]),
)
self.assertIn(
"init_repo 'foo' 'ssh://git@github.com/didericis/foo.git'",
script,
self.assertEqual(
["init_repo", "foo", "ssh://git@github.com/didericis/foo.git"],
_shlex.split(lines_with_init[1]),
)
# Daemon line is what keeps PID 1 alive.
self.assertIn("exec git daemon", script)
self.assertIn("--enable=receive-pack", script)
self.assertIn("--timeout=15", script)
self.assertIn("--init-timeout=15", script)
self.assertIn("--base-path=/git", script)
# Smart HTTP receive-pack uses the same bare repos and hooks
# as git-daemon, so repos must opt in to HTTP pushes too.
self.assertIn("http.receivepack true", script)
# The access-hook is what makes fetch a mirror operation
# against the upstream (PRD 0008 v1.1).
self.assertIn("--access-hook=/etc/git-gate/access-hook", script)
@@ -185,6 +112,41 @@ class TestEntrypointRender(unittest.TestCase):
self.assertNotIn("init_repo '", script)
self.assertIn("exec git daemon", script)
def test_single_quote_in_upstream_url_is_escaped(self):
ups = (GitGateUpstream(
name="myrepo",
upstream_url="ssh://git@host/path'with'quotes.git",
upstream_host="host",
upstream_port="22",
identity_file="/key",
known_host_key="",
),)
script = git_gate_render_entrypoint(ups)
self.assertNotIn(
"init_repo 'myrepo' 'ssh://git@host/path'with'quotes.git'",
script,
)
self.assertIn("init_repo", script)
self.assertIn("path", script)
def test_space_and_semicolon_in_upstream_url_are_escaped(self):
import shlex as _shlex
raw_url = "ssh://git@host/path with spaces;evil.git"
ups = (GitGateUpstream(
name="myrepo",
upstream_url=raw_url,
upstream_host="host",
upstream_port="22",
identity_file="/key",
known_host_key="",
),)
script = git_gate_render_entrypoint(ups)
line = next(l for l in script.splitlines() if l.startswith("init_repo "))
tokens = _shlex.split(line)
self.assertEqual(3, len(tokens))
self.assertEqual("myrepo", tokens[1])
self.assertEqual(raw_url, tokens[2])
class TestHookRender(unittest.TestCase):
def test_pre_receive_hook_has_two_phases(self):
@@ -297,11 +259,10 @@ class TestPrepare(unittest.TestCase):
def test_prepare_skips_known_hosts_file_when_key_missing(self):
manifest = Manifest.from_json_obj({
"bottles": {"dev": {"git": {"remotes": {
"github.com": {
"Name": "foo",
"Upstream": "ssh://git@github.com/didericis/foo.git",
"IdentityFile": "/dev/null",
"bottles": {"dev": {"git-gate": {"repos": {
"foo": {
"url": "ssh://git@github.com/didericis/foo.git",
"identity": "/dev/null",
},
}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
+332
View File
@@ -0,0 +1,332 @@
"""Unit: smart-HTTP git-gate wrapper."""
import os
import subprocess
import tempfile
import threading
import unittest
import urllib.request
from pathlib import Path
from unittest import mock
from bot_bottle.git_http_backend import GitHttpHandler
class TestGitHttpBackend(unittest.TestCase):
def test_real_git_push_reaches_bare_repo(self):
from http.server import ThreadingHTTPServer
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
bare = root / "repo.git"
subprocess.run(["git", "init", "--bare", str(bare)],
check=True, capture_output=True, text=True)
subprocess.run(
["git", "-C", str(bare), "config", "http.receivepack", "true"],
check=True,
)
old_root = os.environ.get("GIT_PROJECT_ROOT")
os.environ["GIT_PROJECT_ROOT"] = str(root)
self.addCleanup(self._restore_env, old_root)
old_hook = os.environ.get("GIT_GATE_ACCESS_HOOK")
hook = root / "access-hook"
hook.write_text("#!/bin/sh\nexit 0\n")
hook.chmod(0o700)
os.environ["GIT_GATE_ACCESS_HOOK"] = str(hook)
self.addCleanup(self._restore_hook, old_hook)
server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
work = root / "work"
work.mkdir()
subprocess.run(["git", "init"], cwd=work, check=True,
capture_output=True, text=True)
subprocess.run(["git", "config", "user.name", "test"],
cwd=work, check=True)
subprocess.run(["git", "config", "user.email", "test@example.invalid"],
cwd=work, check=True)
(work / "README.md").write_text("test\n")
subprocess.run(["git", "add", "README.md"], cwd=work, check=True)
subprocess.run(["git", "commit", "-m", "init"], cwd=work,
check=True, capture_output=True, text=True)
url = f"http://127.0.0.1:{server.server_port}/repo.git"
subprocess.run(
["git", "push", url, "HEAD:refs/heads/main"],
cwd=work,
check=True,
capture_output=True,
text=True,
timeout=5,
)
pushed = subprocess.check_output(
["git", "-C", str(bare), "rev-parse", "refs/heads/main"],
text=True,
).strip()
head = subprocess.check_output(
["git", "-C", str(work), "rev-parse", "HEAD"],
text=True,
).strip()
self.assertEqual(head, pushed)
subprocess.run(
["git", "-C", str(bare), "symbolic-ref", "HEAD", "refs/heads/main"],
check=True,
)
clone = root / "clone"
subprocess.run(
["git", "clone", url, str(clone)],
check=True,
capture_output=True,
text=True,
timeout=5,
)
cloned = subprocess.check_output(
["git", "-C", str(clone), "rev-parse", "HEAD"],
text=True,
).strip()
self.assertEqual(head, cloned)
def test_post_forwards_git_cgi_headers(self):
from http.server import ThreadingHTTPServer
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
(root / "repo.git").mkdir()
old_root = os.environ.get("GIT_PROJECT_ROOT")
os.environ["GIT_PROJECT_ROOT"] = str(root)
self.addCleanup(self._restore_env, old_root)
server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
backend_response = (
b"Status: 200 OK\r\n"
b"Content-Type: application/x-git-upload-pack-result\r\n"
b"\r\n"
b"0000"
)
calls = [
subprocess.CompletedProcess(["hook"], 0, b"", b""),
subprocess.CompletedProcess(["git"], 0, backend_response, b""),
]
with mock.patch(
"bot_bottle.git_http_backend.subprocess.run",
side_effect=calls,
) as run:
request = urllib.request.Request(
f"http://127.0.0.1:{server.server_port}"
"/repo.git/git-upload-pack",
data=b"compressed",
headers={
"Accept": "application/x-git-upload-pack-result",
"Content-Encoding": "gzip",
"Content-Type": "application/x-git-upload-pack-request",
"Git-Protocol": "version=2",
"User-Agent": "git/test",
},
method="POST",
)
with urllib.request.urlopen(request, timeout=5) as response:
self.assertEqual(200, response.status)
self.assertEqual(b"0000", response.read())
env = run.call_args_list[1].kwargs["env"]
self.assertEqual("gzip", env["HTTP_CONTENT_ENCODING"])
self.assertEqual("version=2", env["HTTP_GIT_PROTOCOL"])
self.assertEqual(
"application/x-git-upload-pack-result",
env["HTTP_ACCEPT"],
)
self.assertEqual("git/test", env["HTTP_USER_AGENT"])
def test_access_hook_denial_is_logged_to_stdout(self):
"""When the access-hook exits non-zero we still return 403 to the
client, but the hook's stderr must also appear on the handler's
stdout so docker logs surface *why* otherwise the agent sees
the message and the operator just sees `403 -`."""
from http.server import ThreadingHTTPServer
import io
import sys
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
(root / "repo.git").mkdir()
old_root = os.environ.get("GIT_PROJECT_ROOT")
os.environ["GIT_PROJECT_ROOT"] = str(root)
self.addCleanup(self._restore_env, old_root)
server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
denial = b"git-gate: upstream fetch failed; refusing to serve stale data\n"
with mock.patch(
"bot_bottle.git_http_backend.subprocess.run",
return_value=subprocess.CompletedProcess(
["hook"], 1, b"", denial,
),
):
buf = io.StringIO()
with mock.patch.object(sys, "stdout", buf):
req = urllib.request.Request(
f"http://127.0.0.1:{server.server_port}"
"/repo.git/info/refs?service=git-upload-pack",
method="GET",
)
try:
urllib.request.urlopen(req, timeout=5)
self.fail("expected HTTPError 403")
except urllib.error.HTTPError as e:
self.assertEqual(403, e.code)
self.assertIn(b"upstream fetch failed", e.read())
logged = buf.getvalue()
self.assertIn("access-hook denied", logged)
self.assertIn("upstream fetch failed", logged)
def test_access_hook_denial_without_output_logs_exit_code(self):
"""If the hook exits non-zero but produces no stderr/stdout, the
log line should still say *something* the exit code instead
of silently emitting an empty line."""
from http.server import ThreadingHTTPServer
import io
import sys
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
(root / "repo.git").mkdir()
old_root = os.environ.get("GIT_PROJECT_ROOT")
os.environ["GIT_PROJECT_ROOT"] = str(root)
self.addCleanup(self._restore_env, old_root)
server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
with mock.patch(
"bot_bottle.git_http_backend.subprocess.run",
return_value=subprocess.CompletedProcess(
["hook"], 2, b"", b"",
),
):
buf = io.StringIO()
with mock.patch.object(sys, "stdout", buf):
req = urllib.request.Request(
f"http://127.0.0.1:{server.server_port}"
"/repo.git/info/refs?service=git-upload-pack",
method="GET",
)
try:
urllib.request.urlopen(req, timeout=5)
self.fail("expected HTTPError 403")
except urllib.error.HTTPError as e:
self.assertEqual(403, e.code)
logged = buf.getvalue()
self.assertIn("access-hook denied", logged)
self.assertIn("exit=2", logged)
@staticmethod
def _restore_env(value: str | None) -> None:
if value is None:
os.environ.pop("GIT_PROJECT_ROOT", None)
else:
os.environ["GIT_PROJECT_ROOT"] = value
@staticmethod
def _restore_hook(value: str | None) -> None:
if value is None:
os.environ.pop("GIT_GATE_ACCESS_HOOK", None)
else:
os.environ["GIT_GATE_ACCESS_HOOK"] = value
class TestContentLengthBounds(unittest.TestCase):
"""PRD 0041: malformed or oversized Content-Length is rejected before
git http-backend is invoked."""
def setUp(self):
from http.server import ThreadingHTTPServer
import tempfile, os
self._tmp = tempfile.mkdtemp()
os.environ["GIT_PROJECT_ROOT"] = self._tmp
self._server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
self._thread = threading.Thread(
target=self._server.serve_forever, daemon=True,
)
self._thread.start()
self._port = self._server.server_port
def tearDown(self):
self._server.shutdown()
self._server.server_close()
os.environ.pop("GIT_PROJECT_ROOT", None)
import shutil
shutil.rmtree(self._tmp, ignore_errors=True)
def _post(self, path: str, *, content_length_header: str,
body: bytes = b"x") -> int:
req = urllib.request.Request(
f"http://127.0.0.1:{self._port}{path}",
data=body,
method="POST",
)
req.add_header("Content-Length", content_length_header)
req.add_header("Content-Type", "application/x-git-receive-pack-request")
try:
with urllib.request.urlopen(req, timeout=3) as resp:
return resp.status
except urllib.error.HTTPError as e:
return e.code
def test_non_numeric_content_length_returns_400(self):
status = self._post("/repo.git/git-receive-pack",
content_length_header="abc")
self.assertEqual(400, status)
def test_negative_content_length_returns_400(self):
status = self._post("/repo.git/git-receive-pack",
content_length_header="-1")
self.assertEqual(400, status)
def test_oversized_content_length_returns_413(self):
# Declare 2 MiB — over the 1 MiB cap.
status = self._post("/repo.git/git-receive-pack",
content_length_header=str(2 * 1024 * 1024))
self.assertEqual(413, status)
def test_valid_small_body_passes_through(self):
# With a valid Content-Length the handler proceeds into
# git http-backend; that will fail (no real git repo) but the
# status won't be 400 or 413.
with mock.patch("bot_bottle.git_http_backend.subprocess.run") as run:
run.return_value = mock.Mock(
returncode=0,
stdout=(
b"Status: 200 OK\r\n"
b"Content-Type: application/x-git-receive-pack-result\r\n"
b"\r\n"
),
)
status = self._post("/repo.git/git-receive-pack",
content_length_header="1", body=b"x")
self.assertNotIn(status, (400, 413))
if __name__ == "__main__":
unittest.main()
+27 -33
View File
@@ -1,14 +1,14 @@
"""Unit: agent-level git.user overlay + provenance (PRD 0027, issue #94).
"""Unit: agent-level git-gate.user overlay + provenance (PRD 0027, PRD 0047).
An agent file may declare `git.user` (name/email). At
An agent file may declare `git-gate.user` (name/email). At
`Manifest.bottle_for()` it overlays the referenced bottle's
`git.user` per-field, agent-wins-on-non-empty. `git.remotes` is
`git-gate.user` per-field, agent-wins-on-non-empty. `git-gate.repos` is
rejected on agents. `Manifest.git_identity_summary()` reports the
effective identity with per-field `(agent)`/`(bottle)` provenance.
The `from_json_obj` path drives `Agent.from_dict` + `bottle_for`;
a temp-dir case locks the md loader (the `_AGENT_KEYS` allow + the
`git` threading into `agent_dict`)."""
`git-gate` threading into `agent_dict`)."""
from __future__ import annotations
@@ -34,10 +34,10 @@ def _error_message(callable_, *args, **kwargs) -> str:
def _manifest(*, bottle_user=None, agent_git=None) -> Manifest:
bottle: dict = {}
if bottle_user is not None:
bottle = {"git": {"user": bottle_user}}
bottle = {"git-gate": {"user": bottle_user}}
agent: dict = {"skills": [], "prompt": "", "bottle": "dev"}
if agent_git is not None:
agent["git"] = agent_git
agent["git-gate"] = agent_git
return Manifest.from_json_obj({
"bottles": {"dev": bottle},
"agents": {"impl": agent},
@@ -71,7 +71,6 @@ class TestAgentGitUserOverlay(unittest.TestCase):
def test_agent_identity_with_bottle_declaring_none(self):
m = _manifest(agent_git={"user": {"name": "a", "email": "a@b"}})
# The underlying bottle declares no identity; the merged one does.
self.assertTrue(m.bottles["dev"].git_user.is_empty())
self.assertFalse(m.bottle_for("impl").git_user.is_empty())
@@ -82,14 +81,10 @@ class TestAgentGitUserOverlay(unittest.TestCase):
self.assertEqual("b@c", u.email)
def test_bottle_for_returns_same_instance_when_no_overlay(self):
# No agent git.user → no replace(); the cached Bottle is
# returned as-is (identity check guards against churn).
m = _manifest(bottle_user={"name": "B"})
self.assertIs(m.bottles["dev"], m.bottle_for("impl"))
def test_bottle_for_returns_same_instance_when_overlay_is_noop(self):
# Agent restates exactly what the bottle already has → merged
# == bottle.git_user → same instance, no replace().
m = _manifest(
bottle_user={"name": "B", "email": "b@c"},
agent_git={"user": {"name": "B", "email": "b@c"}},
@@ -101,11 +96,11 @@ class TestAgentGitUserOverlay(unittest.TestCase):
"bottles": {"dev": {
"env": {"FOO": "bar"},
"supervise": True,
"git": {"user": {"name": "B"}},
"git-gate": {"user": {"name": "B"}},
}},
"agents": {"impl": {
"bottle": "dev", "skills": [], "prompt": "",
"git": {"user": {"name": "a"}},
"git-gate": {"user": {"name": "a"}},
}},
})
b = m.bottle_for("impl")
@@ -115,11 +110,11 @@ class TestAgentGitUserOverlay(unittest.TestCase):
class TestAgentGitUserRejections(unittest.TestCase):
def test_agent_remotes_dies_bottle_only(self):
def test_agent_repos_dies_bottle_only(self):
msg = _error_message(_manifest, agent_git={
"remotes": {"h": {"Name": "r", "Upstream": "ssh://x/y.git"}},
"repos": {"r": {"url": "ssh://git@x/y.git", "identity": "/dev/null"}},
})
self.assertIn("git.remotes", msg)
self.assertIn("git-gate.repos", msg)
self.assertIn("bottle-only", msg)
def test_agent_unknown_git_subkey_dies(self):
@@ -127,7 +122,6 @@ class TestAgentGitUserRejections(unittest.TestCase):
self.assertIn("not allowed at the agent level", msg)
def test_agent_git_user_both_empty_dies(self):
# Reuses GitUser.from_dict validation.
msg = _error_message(_manifest, agent_git={"user": {"name": "", "email": ""}})
self.assertIn("neither name nor email", msg)
@@ -164,7 +158,7 @@ class TestGitIdentitySummary(unittest.TestCase):
_BOTTLE_DEV = """
---
git:
git-gate:
user:
name: bottle-name
email: bottle@example.com
@@ -176,7 +170,7 @@ _BOTTLE_DEV = """
_AGENT_WITH_GIT = """
---
bottle: dev
git:
git-gate:
user:
name: agent-name
---
@@ -184,14 +178,14 @@ _AGENT_WITH_GIT = """
impl agent.
"""
_AGENT_WITH_REMOTES = """
_AGENT_WITH_REPOS = """
---
bottle: dev
git:
remotes:
h:
Name: r
Upstream: ssh://x/y.git
git-gate:
repos:
r:
url: ssh://git@x/y.git
identity: /dev/null
---
bad agent.
@@ -199,9 +193,9 @@ _AGENT_WITH_REMOTES = """
class TestAgentGitUserMdLoader(unittest.TestCase):
"""Locks the md path: `git` is an accepted agent key and threads
into the parsed Agent (not rejected as an unknown frontmatter
key), and agent `git.remotes` dies through the same loader."""
"""Locks the md path: `git-gate` is an accepted agent key and threads
into the parsed Agent (not rejected as an unknown frontmatter key),
and agent `git-gate.repos` dies through the same loader."""
def setUp(self) -> None:
self.home = Path(tempfile.mkdtemp(prefix="cb-home-"))
@@ -225,18 +219,18 @@ class TestAgentGitUserMdLoader(unittest.TestCase):
self._write("agents/impl.md", _AGENT_WITH_GIT)
m = Manifest.resolve(str(self.home))
u = m.bottle_for("impl").git_user
self.assertEqual("agent-name", u.name) # agent wins
self.assertEqual("bottle@example.com", u.email) # bottle falls through
self.assertEqual("agent-name", u.name)
self.assertEqual("bottle@example.com", u.email)
self.assertEqual(
"name=agent-name (agent), email=bottle@example.com (bottle)",
m.git_identity_summary("impl"),
)
def test_md_agent_remotes_dies(self):
def test_md_agent_repos_dies(self):
self._write("bottles/dev.md", _BOTTLE_DEV)
self._write("agents/impl.md", _AGENT_WITH_REMOTES)
self._write("agents/impl.md", _AGENT_WITH_REPOS)
msg = _error_message(Manifest.resolve, str(self.home))
self.assertIn("git.remotes", msg)
self.assertIn("git-gate.repos", msg)
self.assertIn("bottle-only", msg)
+66 -61
View File
@@ -29,6 +29,13 @@ def _provider_bottle(provider, routes):
}).bottles["dev"]
def _provider_config_bottle(agent_provider):
return Manifest.from_json_obj({
"bottles": {"dev": {"agent_provider": agent_provider}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}).bottles["dev"]
class TestMinimalRoute(unittest.TestCase):
def test_host_only(self):
b = _bottle([{"host": "api.example.com"}])
@@ -52,6 +59,58 @@ class TestMinimalRoute(unittest.TestCase):
_bottle([{"host": "x.example", "wat": "yes"}])
class TestAgentProviderHostCredentials(unittest.TestCase):
def test_forward_host_credentials_defaults_false(self):
b = _provider_config_bottle({"template": "codex"})
self.assertFalse(b.agent_provider.forward_host_credentials)
def test_forward_host_credentials_allowed_for_codex(self):
b = _provider_config_bottle({
"template": "codex",
"forward_host_credentials": True,
})
self.assertTrue(b.agent_provider.forward_host_credentials)
def test_forward_host_credentials_must_be_boolean(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
"template": "codex",
"forward_host_credentials": "yes",
})
def test_forward_host_credentials_rejected_for_claude(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
"template": "claude",
"forward_host_credentials": True,
})
def test_auth_token_defaults_empty(self):
b = _provider_config_bottle({"template": "claude"})
self.assertEqual("", b.agent_provider.auth_token)
def test_auth_token_allowed_for_claude(self):
b = _provider_config_bottle({
"template": "claude",
"auth_token": "BOT_BOTTLE_CLAUDE_OAUTH_TOKEN",
})
self.assertEqual("BOT_BOTTLE_CLAUDE_OAUTH_TOKEN", b.agent_provider.auth_token)
def test_auth_token_must_be_string(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
"template": "claude",
"auth_token": 42,
})
def test_auth_token_rejected_for_codex(self):
with self.assertRaises(ManifestError):
_provider_config_bottle({
"template": "codex",
"auth_token": "SOME_TOKEN",
})
class TestPathAllowlist(unittest.TestCase):
def test_optional(self):
b = _bottle([{"host": "x.example"}])
@@ -144,29 +203,12 @@ class TestRole(unittest.TestCase):
b = _bottle([{"host": "x.example"}])
self.assertEqual((), b.egress.routes[0].Role)
def test_string_normalizes_to_tuple(self):
b = _bottle([{
"host": "api.anthropic.com",
"role": "claude_code_oauth",
"auth": {"scheme": "Bearer", "token_ref": "T"},
}])
self.assertEqual(("claude_code_oauth",),
b.egress.routes[0].Role)
def test_list_supported(self):
b = _bottle([{
"host": "api.anthropic.com",
"role": ["claude_code_oauth"],
"auth": {"scheme": "Bearer", "token_ref": "T"},
}])
self.assertEqual(("claude_code_oauth",),
b.egress.routes[0].Role)
def test_unknown_role_rejected(self):
# The role enum is locked down — typos shouldn't silently
# become no-op markers.
with self.assertRaises(ManifestError):
_bottle([{"host": "x.example", "role": "totally-made-up"}])
def test_any_role_rejected(self):
# All former roles removed; the field is reserved for future use.
for role in ("claude_code_oauth", "codex_auth", "totally-made-up"):
with self.subTest(role=role):
with self.assertRaises(ManifestError):
_bottle([{"host": "x.example", "role": role}])
def test_non_string_role_rejected(self):
with self.assertRaises(ManifestError):
@@ -174,44 +216,7 @@ class TestRole(unittest.TestCase):
def test_list_with_non_string_item_rejected(self):
with self.assertRaises(ManifestError):
_bottle([{"host": "x.example",
"role": ["claude_code_oauth", 42]}])
def test_singleton_claude_code_oauth_enforced(self):
# Two routes both claiming the role would make "which one
# drives the placeholder env?" ambiguous.
with self.assertRaises(ManifestError):
_bottle([
{"host": "api.anthropic.com", "role": "claude_code_oauth",
"auth": {"scheme": "Bearer", "token_ref": "T1"}},
{"host": "api2.anthropic.example",
"role": "claude_code_oauth",
"auth": {"scheme": "Bearer", "token_ref": "T2"}},
])
def test_codex_auth_role_allowed_for_codex_provider(self):
b = _provider_bottle("codex", [{
"host": "api.openai.com",
"role": "codex_auth",
"auth": {"scheme": "Bearer", "token_ref": "OPENAI_TOKEN"},
}])
self.assertEqual(("codex_auth",), b.egress.routes[0].Role)
def test_claude_role_rejected_for_codex_provider(self):
with self.assertRaises(ManifestError):
_provider_bottle("codex", [{
"host": "api.anthropic.com",
"role": "claude_code_oauth",
"auth": {"scheme": "Bearer", "token_ref": "T"},
}])
def test_codex_role_rejected_for_default_claude_provider(self):
with self.assertRaises(ManifestError):
_bottle([{
"host": "api.openai.com",
"role": "codex_auth",
"auth": {"scheme": "Bearer", "token_ref": "T"},
}])
_bottle([{"host": "x.example", "role": ["x", 42]}])
class TestPipelockPolicy(unittest.TestCase):
+30 -45
View File
@@ -113,42 +113,30 @@ class TestExtendsEnvMerge(unittest.TestCase):
class TestExtendsGitMerge(unittest.TestCase):
"""git.user overlays by field; git.remotes merges by upstream
"""git-gate.user overlays by field; git-gate.repos merges by upstream
host, with child entries replacing duplicate hosts."""
_GIT_ENTRY_A = {
"Name": "a",
"Upstream": "ssh://git@host-a/a.git",
"IdentityFile": "/dev/null",
}
_GIT_ENTRY_B = {
"Name": "b",
"Upstream": "ssh://git@host-b/b.git",
"IdentityFile": "/dev/null",
}
_GIT_ENTRY_A = {"url": "ssh://git@host-a/a.git", "identity": "/dev/null"}
_GIT_ENTRY_B = {"url": "ssh://git@host-b/b.git", "identity": "/dev/null"}
def test_child_git_remotes_merge_with_parent(self):
def test_child_git_repos_merge_with_parent(self):
m = _build(
base={"git": {"remotes": {"host-a": self._GIT_ENTRY_A}}},
base={"git-gate": {"repos": {"a": self._GIT_ENTRY_A}}},
child={
"extends": "base",
"git": {"remotes": {"host-b": self._GIT_ENTRY_B}},
"git-gate": {"repos": {"b": self._GIT_ENTRY_B}},
},
)
names = [e.Name for e in m.bottles["child"].git]
self.assertEqual(["a", "b"], names)
def test_child_git_remote_replaces_same_host(self):
replacement = {
"Name": "a2",
"Upstream": "ssh://git@host-a/replacement.git",
"IdentityFile": "/dev/null",
}
def test_child_git_repo_replaces_same_host(self):
replacement = {"url": "ssh://git@host-a/replacement.git", "identity": "/dev/null"}
m = _build(
base={"git": {"remotes": {"host-a": self._GIT_ENTRY_A}}},
base={"git-gate": {"repos": {"a": self._GIT_ENTRY_A}}},
child={
"extends": "base",
"git": {"remotes": {"host-a": replacement}},
"git-gate": {"repos": {"a2": replacement}},
},
)
entries = m.bottles["child"].git
@@ -156,30 +144,30 @@ class TestExtendsGitMerge(unittest.TestCase):
self.assertEqual("a2", entries[0].Name)
self.assertEqual("replacement.git", entries[0].UpstreamPath)
def test_child_omits_git_inherits_full_list(self):
def test_child_omits_git_gate_inherits_full_list(self):
m = _build(
base={"git": {"remotes": {
"host-a": self._GIT_ENTRY_A,
"host-b": self._GIT_ENTRY_B,
base={"git-gate": {"repos": {
"a": self._GIT_ENTRY_A,
"b": self._GIT_ENTRY_B,
}}},
child={"extends": "base"},
)
names = [e.Name for e in m.bottles["child"].git]
self.assertEqual(["a", "b"], names)
def test_child_explicit_empty_git_clears_parent(self):
# `git.remotes: {}` is the documented way to say "drop
# the parent's remotes" rather than "inherit them".
def test_child_explicit_empty_repos_clears_parent(self):
# `git-gate.repos: {}` is the documented way to say "drop
# the parent's repos" rather than "inherit them".
m = _build(
base={"git": {"remotes": {"host-a": self._GIT_ENTRY_A}}},
child={"extends": "base", "git": {"remotes": {}}},
base={"git-gate": {"repos": {"a": self._GIT_ENTRY_A}}},
child={"extends": "base", "git-gate": {"repos": {}}},
)
self.assertEqual((), m.bottles["child"].git)
def test_child_git_user_inherits_parent_remotes(self):
def test_child_git_user_inherits_parent_repos(self):
m = _build(
base={"git": {"remotes": {"host-a": self._GIT_ENTRY_A}}},
child={"extends": "base", "git": {"user": {"name": "Child"}}},
base={"git-gate": {"repos": {"a": self._GIT_ENTRY_A}}},
child={"extends": "base", "git-gate": {"user": {"name": "Child"}}},
)
self.assertEqual(["a"], [e.Name for e in m.bottles["child"].git])
self.assertEqual("Child", m.bottles["child"].git_user.name)
@@ -209,12 +197,12 @@ class TestExtendsListsFullReplace(unittest.TestCase):
class TestExtendsGitUserOverlay(unittest.TestCase):
"""git.user: per-field overlay. Each non-empty field on child
"""git-gate.user: per-field overlay. Each non-empty field on child
wins; empties fall through to parent."""
def test_parent_full_child_omits(self):
m = _build(
base={"git": {"user": {"name": "Parent", "email": "p@x"}}},
base={"git-gate": {"user": {"name": "Parent", "email": "p@x"}}},
child={"extends": "base"},
)
u = m.bottles["child"].git_user
@@ -223,10 +211,10 @@ class TestExtendsGitUserOverlay(unittest.TestCase):
def test_child_overrides_both(self):
m = _build(
base={"git": {"user": {"name": "Parent", "email": "p@x"}}},
base={"git-gate": {"user": {"name": "Parent", "email": "p@x"}}},
child={
"extends": "base",
"git": {"user": {"name": "Child", "email": "c@x"}},
"git-gate": {"user": {"name": "Child", "email": "c@x"}},
},
)
u = m.bottles["child"].git_user
@@ -234,11 +222,9 @@ class TestExtendsGitUserOverlay(unittest.TestCase):
self.assertEqual("c@x", u.email)
def test_child_adds_email_inherits_name(self):
# Parent sets only name; child sets only email. Both end
# up populated on the child.
m = _build(
base={"git": {"user": {"name": "Parent"}}},
child={"extends": "base", "git": {"user": {"email": "c@x"}}},
base={"git-gate": {"user": {"name": "Parent"}}},
child={"extends": "base", "git-gate": {"user": {"email": "c@x"}}},
)
u = m.bottles["child"].git_user
self.assertEqual("Parent", u.name)
@@ -246,11 +232,10 @@ class TestExtendsGitUserOverlay(unittest.TestCase):
def test_child_overrides_only_email(self):
m = _build(
base={"git": {"user": {"name": "Parent", "email": "p@x"}}},
child={"extends": "base", "git": {"user": {"email": "c@x"}}},
base={"git-gate": {"user": {"name": "Parent", "email": "p@x"}}},
child={"extends": "base", "git-gate": {"user": {"email": "c@x"}}},
)
u = m.bottles["child"].git_user
# Child overrides email; name inherited from parent.
self.assertEqual("Parent", u.name)
self.assertEqual("c@x", u.email)
+180 -177
View File
@@ -1,39 +1,25 @@
"""Unit: Bottle.git manifest parsing + validation (PRD 0008)."""
"""Unit: git-gate.repos manifest parsing + validation (PRD 0047)."""
import unittest
from bot_bottle.manifest import ManifestError, Manifest
def _manifest(git_entries):
def _manifest(repos: dict) -> dict:
return {
"bottles": {"dev": {"git": {"remotes": {
_host_for(entry): entry for entry in git_entries
}}}},
"bottles": {"dev": {"git-gate": {"repos": repos}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}
def _host_for(entry):
upstream = entry.get("Upstream", "")
if "@a.example" in upstream:
return "a.example"
if "@b.example" in upstream:
return "b.example"
if "@github.com" in upstream:
return "github.com"
if "@gitea.dideric.is" in upstream:
return "gitea.dideric.is"
return "example.com"
class TestGitEntryParsing(unittest.TestCase):
def test_parses_minimal_entry(self):
m = Manifest.from_json_obj(_manifest([{
"Name": "bot-bottle",
"Upstream": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"IdentityFile": "/dev/null",
}]))
m = Manifest.from_json_obj(_manifest({
"bot-bottle": {
"url": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"identity": "/dev/null",
},
}))
entries = m.bottles["dev"].git
self.assertEqual(1, len(entries))
e = entries[0]
@@ -44,185 +30,145 @@ class TestGitEntryParsing(unittest.TestCase):
self.assertEqual("didericis/bot-bottle.git", e.UpstreamPath)
def test_default_port_is_22(self):
m = Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/didericis/foo.git",
"IdentityFile": "/dev/null",
}]))
m = Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://git@github.com/didericis/foo.git",
"identity": "/dev/null",
},
}))
e = m.bottles["dev"].git[0]
self.assertEqual("22", e.UpstreamPort)
self.assertEqual("github.com", e.UpstreamHost)
def test_known_host_key_optional(self):
m = Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
}]))
def test_host_key_optional(self):
m = Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
self.assertEqual("", m.bottles["dev"].git[0].KnownHostKey)
def test_missing_name_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
}]))
def test_host_key_stored(self):
m = Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
"host_key": "ssh-ed25519 AAAA",
},
}))
self.assertEqual("ssh-ed25519 AAAA", m.bottles["dev"].git[0].KnownHostKey)
def test_missing_upstream_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"IdentityFile": "/dev/null",
}]))
def test_repo_name_becomes_Name(self):
m = Manifest.from_json_obj(_manifest({
"my-repo": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
self.assertEqual("my-repo", m.bottles["dev"].git[0].Name)
def test_missing_identity_file_dies(self):
def test_missing_url_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
}]))
Manifest.from_json_obj(_manifest({
"foo": {"identity": "/dev/null"},
}))
def test_non_ssh_upstream_dies(self):
def test_missing_identity_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "https://github.com/didericis/foo.git",
"IdentityFile": "/dev/null",
}]))
Manifest.from_json_obj(_manifest({
"foo": {"url": "ssh://git@github.com/foo.git"},
}))
def test_scp_style_upstream_dies(self):
# SCP-style "git@host:path" is intentionally not supported in
# v1 — ssh:// only.
def test_unknown_key_in_entry_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "git@github.com:didericis/foo.git",
"IdentityFile": "/dev/null",
}]))
Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
"IdentityFile": "/dev/null", # old PascalCase key
},
}))
def test_upstream_without_user_dies(self):
def test_non_ssh_url_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://github.com/foo.git",
"IdentityFile": "/dev/null",
}]))
Manifest.from_json_obj(_manifest({
"foo": {
"url": "https://github.com/didericis/foo.git",
"identity": "/dev/null",
},
}))
def test_upstream_without_path_dies(self):
def test_scp_style_url_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com",
"IdentityFile": "/dev/null",
}]))
Manifest.from_json_obj(_manifest({
"foo": {
"url": "git@github.com:didericis/foo.git",
"identity": "/dev/null",
},
}))
def test_url_without_user_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://github.com/foo.git",
"identity": "/dev/null",
},
}))
def test_url_without_path_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://git@github.com",
"identity": "/dev/null",
},
}))
def test_non_numeric_port_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com:notaport/foo.git",
"IdentityFile": "/dev/null",
}]))
Manifest.from_json_obj(_manifest({
"foo": {
"url": "ssh://git@github.com:notaport/foo.git",
"identity": "/dev/null",
},
}))
class TestGitEntryExtraHosts(unittest.TestCase):
def test_extra_hosts_defaults_to_empty(self):
m = Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
}]))
self.assertEqual({}, dict(m.bottles["dev"].git[0].ExtraHosts))
def test_extra_hosts_parses_host_to_ip_map(self):
m = Manifest.from_json_obj(_manifest([{
"Name": "bot-bottle",
"Upstream": "ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git",
"IdentityFile": "/dev/null",
"ExtraHosts": {"gitea.dideric.is": "100.78.141.42"},
}]))
eh = dict(m.bottles["dev"].git[0].ExtraHosts)
self.assertEqual({"gitea.dideric.is": "100.78.141.42"}, eh)
def test_extra_hosts_must_be_object(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
"ExtraHosts": ["gitea.dideric.is", "100.78.141.42"],
}]))
def test_extra_hosts_ip_must_be_string(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
"ExtraHosts": {"gitea.dideric.is": 100},
}]))
def test_extra_hosts_empty_ip_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest([{
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
"ExtraHosts": {"gitea.dideric.is": ""},
}]))
def test_ip_literal_upstream(self):
m = Manifest.from_json_obj(_manifest({
"bot-bottle": {
"url": "ssh://git@100.78.141.42:30009/didericis/bot-bottle.git",
"identity": "/dev/null",
},
}))
e = m.bottles["dev"].git[0]
self.assertEqual("100.78.141.42", e.UpstreamHost)
self.assertEqual("30009", e.UpstreamPort)
self.assertEqual("bot-bottle", e.Name)
class TestGitEntryCrossValidation(unittest.TestCase):
def test_duplicate_name_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj({
"bottles": {"dev": {"git": {"remotes": {
"a.example": {
"Name": "foo",
"Upstream": "ssh://git@a.example/x.git",
"IdentityFile": "/dev/null",
},
"b.example": {
"Name": "foo",
"Upstream": "ssh://git@b.example/y.git",
"IdentityFile": "/dev/null",
},
}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def test_remote_key_must_match_upstream_host(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj({
"bottles": {"dev": {"git": {"remotes": {
"wrong.example": {
"Name": "foo",
"Upstream": "ssh://git@github.com/foo.git",
"IdentityFile": "/dev/null",
},
}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def test_remote_key_can_name_logical_host_for_ip_upstream(self):
def test_two_repos_different_hosts_both_parsed(self):
# Repo names come from dict keys; two distinct keys always produce
# two distinct entries (uniqueness is guaranteed at the YAML/dict level).
m = Manifest.from_json_obj({
"bottles": {"dev": {"git": {"remotes": {
"gitea.dideric.is": {
"Name": "bot-bottle",
"Upstream": "ssh://git@100.78.141.42:30009/didericis/bot-bottle.git",
"IdentityFile": "/dev/null",
"bottles": {"dev": {"git-gate": {"repos": {
"foo": {
"url": "ssh://git@a.example/x.git",
"identity": "/dev/null",
},
"bar": {
"url": "ssh://git@b.example/y.git",
"identity": "/dev/null",
},
}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
e = m.bottles["dev"].git[0]
self.assertEqual("gitea.dideric.is", e.RemoteKey)
self.assertEqual("100.78.141.42", e.UpstreamHost)
self.assertEqual("30009", e.UpstreamPort)
names = {e.Name for e in m.bottles["dev"].git}
self.assertEqual({"foo", "bar"}, names)
def test_legacy_ssh_field_dies_with_hint(self):
# PRD 0009: bottle.ssh is removed; manifests carrying it must
# fail loudly with a hint pointing at bottle.git.
with self.assertRaises(ManifestError):
Manifest.from_json_obj({
"bottles": {
@@ -239,25 +185,82 @@ class TestGitEntryCrossValidation(unittest.TestCase):
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def test_name_with_single_quote_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest({
"o'reilly": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
class TestEmptyGitField(unittest.TestCase):
def test_no_git_field_yields_empty_tuple(self):
def test_name_with_space_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest({
"my repo": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
def test_name_with_semicolon_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest({
"foo;bar": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
def test_name_with_dollar_dies(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj(_manifest({
"foo$bar": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
def test_valid_name_with_dots_and_hyphens_accepted(self):
m = Manifest.from_json_obj(_manifest({
"my.repo-name_1": {
"url": "ssh://git@github.com/foo.git",
"identity": "/dev/null",
},
}))
self.assertEqual("my.repo-name_1", m.bottles["dev"].git[0].Name)
def test_legacy_git_key_dies_with_hint(self):
msg = ""
try:
Manifest.from_json_obj({
"bottles": {"dev": {"git": {"remotes": {}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
except ManifestError as e:
msg = str(e)
self.assertIn("git-gate", msg)
self.assertIn("PRD 0047", msg)
class TestEmptyGitGateField(unittest.TestCase):
def test_no_git_gate_field_yields_empty_tuple(self):
m = Manifest.from_json_obj({
"bottles": {"dev": {}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
self.assertEqual((), m.bottles["dev"].git)
def test_git_object_type_required(self):
def test_git_gate_object_type_required(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj({
"bottles": {"dev": {"git": "not-a-list"}},
"bottles": {"dev": {"git-gate": "not-a-dict"}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def test_empty_remotes_yields_empty_tuple(self):
def test_empty_repos_yields_empty_tuple(self):
m = Manifest.from_json_obj({
"bottles": {"dev": {"git": {"remotes": {}}}},
"bottles": {"dev": {"git-gate": {"repos": {}}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
self.assertEqual((), m.bottles["dev"].git)
+5 -5
View File
@@ -1,4 +1,4 @@
"""Unit: Bottle git.user manifest parsing + validation (issue #86)."""
"""Unit: Bottle git-gate.user manifest parsing + validation (issue #86, PRD 0047)."""
import unittest
@@ -16,7 +16,7 @@ def _error_message(callable_, *args, **kwargs) -> str:
def _manifest(git_user):
return {
"bottles": {"dev": {"git": {"user": git_user}}},
"bottles": {"dev": {"git-gate": {"user": git_user}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}
@@ -75,13 +75,13 @@ class TestGitUserParsing(unittest.TestCase):
msg = _error_message(
Manifest.from_json_obj, _manifest({"name": 42}),
)
self.assertIn("git.user.name must be a string", msg)
self.assertIn("git-gate.user.name must be a string", msg)
def test_non_string_email_dies(self):
msg = _error_message(
Manifest.from_json_obj, _manifest({"email": ["x@y.z"]}),
)
self.assertIn("git.user.email must be a string", msg)
self.assertIn("git-gate.user.email must be a string", msg)
def test_legacy_top_level_git_user_dies(self):
msg = _error_message(
@@ -92,7 +92,7 @@ class TestGitUserParsing(unittest.TestCase):
},
)
self.assertIn("git_user", msg)
self.assertIn("git.user", msg)
self.assertIn("git-gate.user", msg)
class TestGitUserDirect(unittest.TestCase):
+74
View File
@@ -220,6 +220,80 @@ class TestAgentFileDoublesAsClaudeCodeSubagent(_ResolveCase):
self.assertEqual(("init-prd",), m.agents["implementer"].skills)
class TestManifestEntryPointParity(_ResolveCase):
"""The MD and JSON entry points share validation and composition
behavior for the same raw manifest shape."""
def test_agent_prompt_and_skills_match_json_entry(self):
_write(self.home_cb / "bottles" / "dev.md", _BOTTLE_DEV)
_write(self.home_cb / "agents" / "implementer.md", _AGENT_IMPL)
md_manifest = self.resolve()
json_manifest = Manifest.from_json_obj({
"bottles": {
"dev": {
"egress": {
"routes": [
{
"host": "api.anthropic.com",
"auth": {
"scheme": "Bearer",
"token_ref": "CLAUDE_CODE_OAUTH_TOKEN",
},
},
{"host": "example.com"},
],
},
},
},
"agents": {
"implementer": {
"bottle": "dev",
"skills": ["init-prd"],
"prompt": "You are a feature implementation agent.",
},
},
})
self.assertEqual(
md_manifest.agents["implementer"],
json_manifest.agents["implementer"],
)
self.assertEqual(
md_manifest.bottles["dev"].egress.routes,
json_manifest.bottles["dev"].egress.routes,
)
def test_json_agent_rejects_unknown_keys(self):
with self.assertRaises(ManifestError):
Manifest.from_json_obj({
"bottles": {"dev": {}},
"agents": {
"implementer": {
"bottle": "dev",
"skillz": ["init-prd"],
},
},
})
def test_json_agent_accepts_claude_code_passthrough_keys(self):
manifest = Manifest.from_json_obj({
"bottles": {"dev": {}},
"agents": {
"implementer": {
"name": "implementer",
"description": "Implements features against PRDs.",
"model": "opus",
"color": "blue",
"memory": "project",
"bottle": "dev",
},
},
})
self.assertEqual("dev", manifest.agents["implementer"].bottle)
class TestUnknownAgentKeyDies(_ResolveCase):
"""A typo'd / unknown frontmatter key on an agent file dies
rather than silently ignoring."""
+27
View File
@@ -5,6 +5,8 @@ git-gate (PRD 0008)."""
import unittest
from bot_bottle.agent_provider import CODEX_HOST_CREDENTIAL_HOSTS
from bot_bottle.egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
from bot_bottle.manifest import Manifest
from bot_bottle.pipelock import (
pipelock_effective_allowlist,
@@ -113,6 +115,31 @@ class TestTlsPassthrough(unittest.TestCase):
])))
self.assertEqual(["api.openai.com"], passthrough)
def test_forward_host_credentials_passes_through_codex_hosts(self):
# Egress injects the host bearer on the Codex API hosts; pipelock
# must pass them through or its header DLP blocks the injected JWT
# ("request header contains secret"). Provider routes carry
# tls_passthrough=True; pipelock reads this via egress_routes_for_bottle.
provider_routes = tuple(
EgressRoute(
host=host,
auth_scheme="Bearer",
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF,
tls_passthrough=True,
)
for host in CODEX_HOST_CREDENTIAL_HOSTS
)
passthrough = pipelock_effective_tls_passthrough(
_bottle({}), provider_routes,
)
self.assertEqual(["api.openai.com", "chatgpt.com"], passthrough)
def test_no_codex_passthrough_without_provider_routes(self):
passthrough = pipelock_effective_tls_passthrough(_bottle({
"agent_provider": {"template": "codex"},
}))
self.assertEqual([], passthrough)
class TestSsrfIpAllowlist(unittest.TestCase):
def test_default_empty(self):
+107
View File
@@ -19,6 +19,7 @@ from bot_bottle.pipelock import (
pipelock_build_config,
pipelock_render_yaml,
)
from bot_bottle.yaml_subset import parse_yaml_subset
from tests.fixtures import fixture_minimal
@@ -158,6 +159,51 @@ class TestRenderAndWrite(unittest.TestCase):
import shutil
shutil.rmtree(self.out_dir, ignore_errors=True)
def assert_render_semantics_match(self, cfg: dict[str, object]) -> None:
parsed = parse_yaml_subset(pipelock_render_yaml(cfg))
self.assertEqual(cfg["version"], parsed["version"])
self.assertEqual(cfg["mode"], parsed["mode"])
self.assertEqual(cfg["enforce"], parsed["enforce"])
parsed_allowlist = parsed["api_allowlist"]
if cfg["api_allowlist"] == [] and parsed_allowlist is None:
parsed_allowlist = []
self.assertEqual(cfg["api_allowlist"], parsed_allowlist)
self.assertEqual(cfg["forward_proxy"], parsed["forward_proxy"])
self.assertEqual(cfg["dlp"], parsed["dlp"])
self.assertEqual(
cfg["request_body_scanning"],
parsed["request_body_scanning"],
)
if "seed_phrase_detection" in cfg:
self.assertEqual(
cfg["seed_phrase_detection"],
parsed["seed_phrase_detection"],
)
else:
self.assertNotIn("seed_phrase_detection", parsed)
if "tls_interception" in cfg:
expected_tls = cast(dict[str, object], cfg["tls_interception"])
actual_tls = cast(dict[str, object], parsed["tls_interception"])
self.assertEqual(expected_tls["enabled"], actual_tls["enabled"])
self.assertEqual(expected_tls["ca_cert"], actual_tls["ca_cert"])
self.assertEqual(expected_tls["ca_key"], actual_tls["ca_key"])
expected_passthrough = expected_tls["passthrough_domains"]
if expected_passthrough:
self.assertEqual(
expected_passthrough,
actual_tls["passthrough_domains"],
)
else:
self.assertNotIn("passthrough_domains", actual_tls)
else:
self.assertNotIn("tls_interception", parsed)
if "ssrf" in cfg:
self.assertEqual(cfg["ssrf"], parsed["ssrf"])
else:
self.assertNotIn("ssrf", parsed)
def test_render_emits_required_top_level_keys(self):
"""One render-level smoke check: the serialized YAML is plausibly
the shape pipelock expects. We don't grep every key here — that's
@@ -175,6 +221,67 @@ class TestRenderAndWrite(unittest.TestCase):
self.assertNotIn("trusted_domains:", text)
self.assertNotIn("ssrf:", text)
def test_render_semantics_match_minimal_config(self):
cfg = pipelock_build_config(fixture_minimal().bottles["dev"])
self.assert_render_semantics_match(cfg)
def test_render_semantics_match_tls_with_empty_passthrough(self):
cfg = pipelock_build_config(
fixture_minimal().bottles["dev"],
ca_cert_path="/etc/pipelock-ca.pem",
ca_key_path="/etc/pipelock-ca-key.pem",
)
self.assert_render_semantics_match(cfg)
def test_render_semantics_match_all_optional_sections(self):
bottle = Manifest.from_json_obj({
"bottles": {"dev": {"egress": {"routes": [
{"host": "api.openai.com",
"pipelock": {"tls_passthrough": True}},
{"host": "gitea.dideric.is",
"pipelock": {"ssrf_ip_allowlist": ["100.78.141.42/32"]}},
]}}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}).bottles["dev"]
cfg = pipelock_build_config(
bottle,
ca_cert_path="/etc/pipelock-ca.pem",
ca_key_path="/etc/pipelock-ca-key.pem",
ssrf_ip_allowlist=("172.20.0.0/16",),
)
self.assert_render_semantics_match(cfg)
def test_render_rejects_missing_required_key(self):
cfg = pipelock_build_config(fixture_minimal().bottles["dev"])
del cfg["mode"]
with self.assertRaisesRegex(ValueError, r"config\.mode"):
pipelock_render_yaml(cfg)
def test_render_rejects_wrong_section_type(self):
cfg = pipelock_build_config(fixture_minimal().bottles["dev"])
cfg["dlp"] = []
with self.assertRaisesRegex(ValueError, r"config\.dlp.*mapping"):
pipelock_render_yaml(cfg)
def test_render_rejects_wrong_list_item_type(self):
cfg = pipelock_build_config(
fixture_minimal().bottles["dev"],
ca_cert_path="/etc/pipelock-ca.pem",
ca_key_path="/etc/pipelock-ca-key.pem",
)
tls = cast(dict[str, object], cfg["tls_interception"])
tls["passthrough_domains"] = ["api.openai.com", 3]
with self.assertRaisesRegex(
ValueError, r"tls_interception\.passthrough_domains",
):
pipelock_render_yaml(cfg)
def test_render_rejects_unsupported_top_level_section(self):
cfg = pipelock_build_config(fixture_minimal().bottles["dev"])
cfg["trusted_domains"] = []
with self.assertRaisesRegex(ValueError, r"config\.trusted_domains"):
pipelock_render_yaml(cfg)
def test_prepare_writes_file_at_mode_600(self):
plan = PipelockProxy().prepare(
fixture_minimal().bottles["dev"], "demo", self.out_dir
+248
View File
@@ -0,0 +1,248 @@
"""Unit: BottlePlan.print parity across Docker and smolmachines (PRD 0044).
Both backends inherit a single concrete print() from BottlePlan. These
tests verify that identical git_gate_plan and egress_plan inputs produce
identical preflight output regardless of backend-specific fields.
"""
from __future__ import annotations
import io
import sys
import tempfile
import unittest
from pathlib import Path
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.smolmachines.bottle_plan import SmolmachinesBottlePlan
from bot_bottle.egress import EgressPlan, EgressRoute
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
from bot_bottle.manifest import Manifest
from bot_bottle.pipelock import PipelockProxyPlan
from bot_bottle.workspace import workspace_plan
def _manifest() -> Manifest:
return Manifest.from_json_obj({
"bottles": {"dev": {}},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def _spec(manifest: Manifest, tmp: str) -> BottleSpec:
return BottleSpec(
manifest=manifest,
agent_name="demo",
copy_cwd=False,
user_cwd=tmp,
identity="test-00001",
)
def _git_gate_plan(tmp: str) -> GitGatePlan:
stage = Path(tmp)
return GitGatePlan(
slug="test-00001",
entrypoint_script=stage / "entrypoint.sh",
hook_script=stage / "hook.sh",
access_hook_script=stage / "access-hook.sh",
upstreams=(
GitGateUpstream(
name="myrepo",
upstream_url="ssh://git@gitea.example.com:30009/org/myrepo.git",
upstream_host="gitea.example.com",
upstream_port="30009",
identity_file="/dev/null",
known_host_key="ssh-ed25519 AAAA...",
),
),
)
def _egress_plan(tmp: str) -> EgressPlan:
return EgressPlan(
slug="test-00001",
routes_path=Path(tmp) / "egress.yaml",
routes=(
EgressRoute(
host="api.example.com",
path_allowlist=("/v1/",),
auth_scheme="bearer",
token_env="EGRESS_TOKEN_0",
token_ref="TOKEN",
),
EgressRoute(
host="static.example.com",
path_allowlist=("/",),
),
),
token_env_map={"EGRESS_TOKEN_0": "TOKEN"},
)
def _agent_provision() -> AgentProvisionPlan:
return AgentProvisionPlan(
template="claude",
command="claude",
prompt_mode="append_file",
image="",
dockerfile="",
guest_env={"HTTPS_PROXY": "http://127.0.0.1:9999"},
)
def _proxy_plan(tmp: str) -> PipelockProxyPlan:
return PipelockProxyPlan(
yaml_path=Path(tmp) / "pipelock.yaml",
slug="test-00001",
)
def _docker_plan(spec: BottleSpec, tmp: str) -> DockerBottlePlan:
stage = Path(tmp)
return DockerBottlePlan(
spec=spec,
stage_dir=stage,
git_gate_plan=_git_gate_plan(tmp),
egress_plan=_egress_plan(tmp),
supervise_plan=None,
agent_provision=_agent_provision(),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
slug="test-00001",
container_name="bot-bottle-test-00001",
container_name_pinned=False,
image="bot-bottle-claude:latest",
derived_image="",
runtime_image="bot-bottle-claude:latest",
dockerfile_path="",
env_file=stage / "env",
forwarded_env={},
prompt_file=stage / "prompt.txt",
proxy_plan=_proxy_plan(tmp),
use_runsc=False,
)
def _smolmachines_plan(spec: BottleSpec, tmp: str) -> SmolmachinesBottlePlan:
stage = Path(tmp)
return SmolmachinesBottlePlan(
spec=spec,
stage_dir=stage,
git_gate_plan=_git_gate_plan(tmp),
egress_plan=_egress_plan(tmp),
supervise_plan=None,
agent_provision=_agent_provision(),
workspace_plan=workspace_plan(spec, guest_home="/home/node"),
slug="test-00001",
bundle_subnet="10.99.0.0/24",
bundle_gateway="10.99.0.1",
bundle_ip="10.99.0.2",
machine_name="bot-bottle-test-00001",
agent_image_ref="bot-bottle-claude:latest",
guest_env={"HTTPS_PROXY": "http://127.0.0.1:9999"},
prompt_file=stage / "prompt.txt",
proxy_plan=_proxy_plan(tmp),
)
def _capture_print(plan: DockerBottlePlan | SmolmachinesBottlePlan) -> list[str]:
buf = io.StringIO()
orig = sys.stderr
sys.stderr = buf
try:
plan.print(remote_control=False)
finally:
sys.stderr = orig
return buf.getvalue().splitlines()
class TestGitGatePrintParity(unittest.TestCase):
"""Both backends render git gate entries as 'name → host:port'."""
def setUp(self) -> None:
self._tmp = tempfile.mkdtemp(prefix="plan-print-parity-")
manifest = _manifest()
spec = _spec(manifest, self._tmp)
self._docker_lines = _capture_print(_docker_plan(spec, self._tmp))
self._smol_lines = _capture_print(_smolmachines_plan(spec, self._tmp))
def _git_gate_lines(self, lines: list[str]) -> list[str]:
return [ln for ln in lines if "git gate" in ln]
def test_docker_renders_name_arrow_host_port(self) -> None:
git_lines = self._git_gate_lines(self._docker_lines)
self.assertEqual(1, len(git_lines))
self.assertIn("myrepo → gitea.example.com:30009", git_lines[0])
def test_smolmachines_renders_name_arrow_host_port(self) -> None:
git_lines = self._git_gate_lines(self._smol_lines)
self.assertEqual(1, len(git_lines))
self.assertIn("myrepo → gitea.example.com:30009", git_lines[0])
def test_git_gate_lines_match_across_backends(self) -> None:
self.assertEqual(
self._git_gate_lines(self._docker_lines),
self._git_gate_lines(self._smol_lines),
)
class TestEgressPrintParity(unittest.TestCase):
"""Both backends render egress with auth annotation where present."""
def setUp(self) -> None:
self._tmp = tempfile.mkdtemp(prefix="plan-print-parity-")
manifest = _manifest()
spec = _spec(manifest, self._tmp)
self._docker_lines = _capture_print(_docker_plan(spec, self._tmp))
self._smol_lines = _capture_print(_smolmachines_plan(spec, self._tmp))
def _egress_section(self, lines: list[str]) -> list[str]:
"""Return lines from the egress label through the last route entry.
print_multi renders the first route on the label line and
aligns additional routes as indented continuation lines
(no repeated label). Collect the label line plus every
non-blank, non-labelled line that follows before the next
top-level section begins."""
result: list[str] = []
collecting = False
indent_prefix = None
for ln in lines:
stripped = ln.lstrip()
if "egress" in stripped and ":" in stripped:
collecting = True
# Determine the continuation indent from this line's prefix.
idx = ln.index("egress")
indent_prefix = ln[:idx]
result.append(ln)
elif collecting:
if ln.startswith(indent_prefix) and "egress" not in ln and ":" not in ln.lstrip()[:20]:
result.append(ln)
else:
break
return result
def test_docker_includes_auth_annotation(self) -> None:
combined = "\n".join(self._egress_section(self._docker_lines))
self.assertIn("api.example.com [auth:bearer]", combined)
def test_smolmachines_includes_auth_annotation(self) -> None:
combined = "\n".join(self._egress_section(self._smol_lines))
self.assertIn("api.example.com [auth:bearer]", combined)
def test_unauthenticated_route_has_no_annotation(self) -> None:
full = "\n".join(self._docker_lines)
self.assertIn("static.example.com", full)
self.assertNotIn("static.example.com [auth:", full)
def test_egress_lines_match_across_backends(self) -> None:
self.assertEqual(
self._egress_section(self._docker_lines),
self._egress_section(self._smol_lines),
)
if __name__ == "__main__":
unittest.main()
+5 -5
View File
@@ -8,21 +8,21 @@ from bot_bottle.backend.print_util import visible_agent_env_names
class TestVisibleAgentEnvNames(unittest.TestCase):
def test_hides_codex_auth_placeholder(self):
def test_shows_all_when_no_hidden_names(self):
self.assertEqual(
["CUSTOM"],
["CUSTOM", "OPENAI_API_KEY"],
visible_agent_env_names(
["OPENAI_API_KEY", "CUSTOM"],
agent_provider_template="codex",
hidden_env_names=frozenset(),
),
)
def test_hides_only_active_provider_placeholder(self):
def test_hides_provider_placeholder(self):
self.assertEqual(
["CUSTOM", "OPENAI_API_KEY"],
visible_agent_env_names(
["CLAUDE_CODE_OAUTH_TOKEN", "OPENAI_API_KEY", "CUSTOM"],
agent_provider_template="claude",
hidden_env_names=frozenset({"CLAUDE_CODE_OAUTH_TOKEN"}),
),
)

Some files were not shown because too many files have changed in this diff Show More