Compare commits

...

49 Commits

Author SHA1 Message Date
didericis-codex fe82dbc7b8 refactor(smolmachines): drop obsolete docker-sidecar helpers
lint / lint (push) Successful in 2m5s
test / unit (pull_request) Successful in 1m2s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m11s
2026-07-09 18:14:56 +00:00
didericis-codex bfd3e659e8 refactor(smolmachines): inline sidecar proxy host
lint / lint (push) Successful in 2m1s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m10s
2026-07-09 13:53:44 -04:00
didericis-codex 9f0d56b75d test(smolmachines): cover sidecar vm launch paths 2026-07-09 13:53:44 -04:00
didericis-codex ce8e2b3874 feat: run smolmachines sidecar as vm 2026-07-09 13:53:44 -04:00
didericis-codex 46a77aec31 docs: draft smolmachines sidecar vm prd 2026-07-09 13:53:44 -04:00
didericis-claude fa7c6ab9d8 fix(tests): replace os.chmod with mock for unreadable-file tests
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m13s
lint / lint (push) Successful in 2m1s
test / unit (push) Successful in 59s
test / integration (push) Successful in 17s
test / coverage (push) Successful in 1m4s
Update Quality Badges / update-badges (push) Successful in 57s
CI runs as root, so chmod 0o000 doesn't prevent reads. Both
test_skips_unreadable_file (TestFindRepoBottleFile) and
test_returns_false_when_file_unreadable_on_write now mock
Path.read_text to raise OSError directly.
2026-07-09 17:35:28 +00:00
didericis-claude e27bd66080 feat(git-gate): show bottle filename in save-host-key prompt; public API
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Failing after 51s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Failing after 54s
DX improvement: find_repo_bottle_file() is called before the save
prompt so the user sees exactly which file will be updated:
  Save host_key for 'myrepo' to /home/.../bottles/dev.md? [y/N]
If no bottle file is found the prompt is skipped and a clear message
explains that the key is session-only.

Also:
- Use find_repo_bottle_file() inside find_and_update_bottle_file()
  instead of duplicating the scan logic
- Rename all module-level helpers to public names (no underscore prefix)
  since they are imported and tested externally
2026-07-09 17:26:25 +00:00
didericis-claude d496e30681 refactor(git-gate): replace regex frontmatter edit with parse→mutate→serialize
The regex-based _insert_host_key_in_frontmatter had a bug: child_indent
was overwritten on every line with indent > repo_indent (including
grandchildren), so host_key landed inside the key: block instead of at
the repo level.

Replace with a clean parse → mutate → re-serialize approach:
- Add serialize_yaml_subset() to yaml_subset.py (block-style, 2-space
  indent, round-trips through parse_yaml_subset)
- Replace _insert_host_key_in_frontmatter + _update_frontmatter_in_file
  with _add_host_key_to_frontmatter that parses the frontmatter dict,
  sets fm["git-gate"]["repos"][name]["host_key"], and re-serializes
- _find_and_update_bottle_file simplified to read file → delegate to
  _add_host_key_to_frontmatter → write if changed
2026-07-09 17:21:57 +00:00
didericis-claude 61740cdb6a fix(git-gate): accept any key type from ssh-keyscan, prefer ed25519
lint / lint (push) Successful in 2m0s
test / unit (pull_request) Successful in 53s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m6s
Servers that don't offer ed25519 (ecdsa-only or rsa-only gitea
instances) would raise "ssh-keyscan returned no ed25519 key" because
`-t ed25519` produced no output.

Remove the `-t ed25519` restriction: ssh-keyscan now returns all
supported key types and the function picks the best available via
_KEY_TYPE_PREFERENCE (ed25519 > ecdsa > rsa > first).

Also adds four new fetch_host_key tests covering multi-type preference
logic and updates the stale "no ed25519 key" assertion.
2026-07-09 16:45:47 +00:00
didericis-claude f9662c88a5 fix: pyright errors in test file, bump pyright floor to 1.1.411
lint / lint (push) Successful in 2m10s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Successful in 1m10s
- Remove unused `import dataclasses` and stray `die as real_die` import
- Replace lambda side_effect callbacks with list side_effect; lambdas
  with untyped parameters trigger reportUnknownLambdaType in pyright
  >=1.1.400 — side_effect=[...] is both cleaner and type-safe
- Bump pyright floor from >=1.1.300 to >=1.1.411 in requirements-dev.txt
  so CI installs a version that matches what the lint job runs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-09 16:11:14 +00:00
didericis-claude e89dffa899 fix: pyright/pylint clean-up and 100% branch coverage
lint / lint (push) Failing after 2m17s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
- Remove unused ManifestBottle/ManifestGitEntry imports
- Add check=False to subprocess.run (pylint subprocess-run-check)
- Add `from e` to TimeoutExpired re-raise (raise-missing-from)
- Catch UnicodeDecodeError alongside OSError/YamlSubsetError in
  _find_and_update_bottle_file
- Add 14 new test cases covering: malformed ssh-keyscan output lines,
  stderr in error messages, blank lines inside frontmatter blocks,
  repo entry with no children (child_indent fallback), missing closing
  `---` delimiter, files without git-gate section, non-dict repos
  section, invalid UTF-8 files, unreadable files, update producing no
  text change, _prompt_tty TTY read and stdin fallback, fetch failure
  in preflight, and the "persist failed / kept in memory" warning path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-09 16:03:39 +00:00
didericis-claude 21d03b7cc9 feat: preflight git-gate host key population
lint / lint (push) Failing after 2m3s
test / unit (pull_request) Successful in 1m3s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Failing after 1m11s
When a git-gate repo entry has no host_key configured, the prepare step
now either errors (headless) or fetches the key via ssh-keyscan, shows
it to the operator for confirmation, and optionally persists it to the
bottle config .md file on disk.

Closes #333

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-09 15:51:50 +00:00
Quality Badge Bot eb804f8674 chore: update quality badges
- Coverage: 84%
- Core coverage: 95%

[skip ci]
2026-07-09 05:00:17 +00:00
didericis b765f87fb5 test(smolmachines): verify Linux sandbox escape gate + fix platform check
lint / lint (push) Successful in 1m57s
test / unit (push) Successful in 56s
test / integration (push) Successful in 18s
test / coverage (push) Successful in 1m6s
Update Quality Badges / update-badges (push) Successful in 1m0s
Empirical verification on NixOS/Linux with /dev/kvm + smolvm 1.4.7:

  1. DB path confirmed: ~/.local/share/smolvm/server/smolvm.db exists,
     schema matches (vms table, data BLOB with allowed_cidrs JSON field).

  2. smolvm 1.4.7 still silently drops --allow-cidr with --from
     (allowed_cidrs=None in DB after create). force_allowlist patch
     is still necessary and correct.

  3. All 5 sandbox-escape attacks blocked with BOT_BOTTLE_BACKEND=smolmachines
     on Linux: hostname/IP allowlist, HTTP DLP, DNS exfil, git-gate gitleaks.

Changes:
- test_sandbox_escape: remove darwin-only guard; allow linux too
- contrib/claude/Dockerfile: add dnsutils (dig) for attack 4 DNS test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-09 00:54:50 -04:00
github-actions[bot] 1084d8b79d ci(prd): assign sequential numbers to new PRDs 2026-07-09 04:36:16 +00:00
didericis 36e62cd7e8 test: add unit tests to pass diff-coverage gate (91%)
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m4s
lint / lint (push) Successful in 1m56s
prd-number / assign-numbers (push) Successful in 19s
test / unit (push) Successful in 1m0s
test / integration (push) Successful in 19s
test / coverage (push) Successful in 1m5s
Update Quality Badges / update-badges (push) Failing after 59s
Cover the new production lines added on this branch:

- cli/__init__: MissingEnvVarError maps to return 1
- cli/start: no-agents-defined path, prepare_with_preflight calls
  render_preflight, _text_render_preflight backend name output
- env: resolve_env raises MissingEnvVarError for missing interpolated var
- smolmachines/launch: _proxy_host Linux/non-Linux branches,
  _discover_urls git-gate host and supervise URL stamping

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-09 00:19:24 -04:00
didericis 70e58f1333 test: fix test_cache_miss and remove unused re import
lint / lint (push) Successful in 2m8s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Failing after 1m6s
- test_smolmachines_launch_image: save mock now touches the tarball path
  so tarball.chmod(0o644) doesn't fail with FileNotFoundError
- test_smolmachines_launch (integration): remove unused `re` import
  (pyright reportUnusedImport)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-09 00:03:00 -04:00
didericis 3703b6f59f test: fix unit test failures after main rebase
lint / lint (push) Failing after 1m58s
test / unit (pull_request) Failing after 57s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Failing after 56s
- test_egress: expect MissingEnvVarError instead of Die for missing/empty token refs
- test_git_gate, test_git_gate_render_provision: expect MissingEnvVarError instead of RuntimeError for missing forge token
- test_smolmachines_provision: handle 2 exec calls (mkdir -p + chmod/update-ca-certificates) in provision_ca
- test_smolmachines_smolvm: update machine create/delete argv assertions to --name flag form (smolvm 1.4.7+)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 23:46:47 -04:00
didericis afd943c4f9 fix(smolmachines): fix CA provisioning and git-http startup on Linux
lint / lint (push) Failing after 2m2s
test / unit (pull_request) Failing after 56s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Failing after 53s
Two fixes for the Linux smolmachines backend:

- provision_ca: add mkdir -p before cp_in so the
  /usr/local/share/ca-certificates/ directory exists in the smolvm
  guest even when smolvm pack omits it (empty dir on Linux).
- git_http_backend: the fallback import in the except ImportError
  block tried `from git_gate import ...`, but git_gate.py is not
  deployed to the sidecar container and has its own package-relative
  imports; use the literal value 15 directly instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 22:06:28 -04:00
didericis 8f31741c9f refactor(errors): introduce MissingEnvVarError for missing host env vars
Replace die()/RuntimeError at the point of detection with a typed
MissingEnvVarError, and catch it in the CLI dispatcher alongside
ManifestError. Library code stays free of stderr output; the CLI
renders all user-facing error lines in one place.

Sites updated: egress.egress_resolve_token_values (unset/empty token),
git_gate._provision_dynamic_key and revoke_git_gate_provisioned_keys
(missing forge_token_env), env.resolve_env (unset interpolated var).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 22:05:49 -04:00
didericis 5bfe45bc1d fix(git-gate): use die() instead of RuntimeError for missing forge token env
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 22:05:03 -04:00
didericis 7aca7d5289 feat(start): show backend name in preflight output
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 22:01:58 -04:00
didericis f6d4eb9e3c fix(start): print error when no agents are defined
Previously start silently exited 0 when the manifest had no agents,
leaving the user with no feedback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 22:01:58 -04:00
didericis c5b9f05467 fix(smolmachines): use bridge gateway as TSI proxy host on Linux
On Linux the guest kernel's LOCAL routing table routes all
127.0.0.0/8 to the guest's own loopback interface (priority 0,
checked before any main-table route), so TSI never sees
connections to the per-bottle loopback alias — the fix_guest_
loopback_routing approach confirmed this at the kernel level.

Use the per-bottle docker bridge gateway (192.168.N.1) instead.
It is not a loopback address, so the guest routes it via eth0
and TSI intercepts it normally.  The TSI allowlist remains a
/32 that is distinct from the container IP (192.168.N.2), so
direct bypass to egress:9099 is still blocked by TSI.

Changes:
- Add _proxy_host() helper: returns bundle_gateway on Linux,
  loopback alias on macOS
- Thread proxy_host through _start_bundle, _discover_urls,
  _launch_vm, and _bundle_launch_spec (publish_host_ip)
- Remove _fix_guest_loopback_routing (no longer needed)
- Relax proxy-URL assertion in the integration test to accept
  any http://IP:port (with a comment explaining the difference)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-08 22:01:58 -04:00
didericis-codex 09a2329f31 test(smolmachines): annotate pyright test helpers 2026-07-08 22:01:35 -04:00
didericis 03c89dae81 feat(smolmachines): run backend on Linux
Port the smolmachines backend so BOT_BOTTLE_BACKEND=smolmachines
works on Linux (KVM), not just macOS:

- Preflight gates /dev/kvm presence + accessibility on Linux with
  actionable remediation (kvm module, kvm group).
- smolvm state-DB path is platform-derived (XDG on Linux).
- force_allowlist runs on both platforms and is fail-closed: it
  verifies the persisted TSI allowlist and dies rather than booting
  a VM whose egress confinement it can't confirm. Previously it
  no-oped on Linux, failing OPEN.
- allocate() does per-bottle 127.0.0.<N> scoping on Linux too (no
  ifconfig needed — all of 127/8 is already loopback); only
  ensure_pool's lo0 aliasing stays macOS-only.
- README documents Linux + NixOS host setup.

Linux/KVM integration (the sandbox-escape acceptance gate) is
pending verification on a NixOS host; unit tests cover the new
platform branches.

Issue: #283

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
2026-07-08 22:01:35 -04:00
didericis dd7232f368 docs(prd): add PRD for smolmachines backend on Linux
Design for porting the smolmachines backend off macOS-only: KVM
preflight, platform-aware smolvm state-DB path, fail-closed TSI
allowlist enforcement, and per-bottle loopback scoping on Linux.
NixOS is the primary validation target.

Issue: #283

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
2026-07-08 22:01:35 -04:00
didericis 76fdefded3 fix: repair codex resume paths
lint / lint (push) Successful in 1m55s
test / unit (push) Successful in 57s
test / integration (push) Successful in 19s
test / coverage (push) Successful in 1m6s
Update Quality Badges / update-badges (push) Successful in 1m1s
2026-07-08 12:38:15 -04:00
didericis 1085a2280d fix(macos_container): mount supervise db's parent dir, not the file
lint / lint (push) Successful in 1m55s
test / unit (push) Successful in 59s
test / integration (push) Successful in 19s
test / coverage (push) Successful in 1m8s
Update Quality Badges / update-badges (push) Successful in 57s
Apple's `container run --mount type=bind` only accepts directory
sources — a file source fails with "path '<file>' is not a
directory". Move the sqlite db into its own ~/.bot-bottle/db/
subdirectory so it can be bind-mounted the same way the CA/routes
mounts already are, without exposing the rest of ~/.bot-bottle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 15:48:36 -04:00
github-actions[bot] c97f4ecb8d ci(prd): assign sequential numbers to new PRDs 2026-07-06 19:29:43 +00:00
didericis-claude 142974a4b8 refactor: rename check_migrations → is_migrated
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m6s
lint / lint (push) Successful in 1m55s
prd-number / assign-numbers (push) Successful in 16s
test / unit (push) Successful in 54s
test / integration (push) Successful in 19s
test / coverage (push) Successful in 1m6s
Update Quality Badges / update-badges (push) Successful in 1m2s
Boolean-returning methods should read as predicates.
Renames DbStore.check_migrations, StoreManager.check_migrations,
and the test patch accordingly.
2026-07-06 19:19:49 +00:00
didericis-claude 041d9bbbf5 refactor: inline TableMigrations into store constructors
lint / lint (push) Successful in 1m59s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m4s
Move _MIGRATIONS from module scope into AuditStore.__init__ and
QueueStore.__init__, treating table schema as a hidden implementation
detail of each store class.
2026-07-06 19:12:45 +00:00
didericis-claude 9fb83ef1b0 refactor: rename _supervise_types → supervise_types (treat as public)
lint / lint (push) Successful in 2m1s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m6s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-06 18:53:53 +00:00
didericis-claude e7e8c7fdb4 refactor: extract supervise types to _supervise_types, eliminate get_supervise_mod()
Proposal, Response, AuditEntry, host_db_path, bot_bottle_root, and their
shared constants/helpers are moved to a new _supervise_types.py module.
queue_store and audit_store now import these directly instead of deferring
through get_supervise_mod() at call time.

host_db_path() in _supervise_types uses sys.modules[__name__].bot_bottle_root()
so monkey-patches on _supervise_types.bot_bottle_root propagate through it.
supervise.host_db_path() is re-defined to call bot_bottle_root() through
supervise's own globals, preserving patchability for callers that go through
the supervise module.

Test fixtures that create stores without an explicit db_path now patch
_sv_types.bot_bottle_root alongside supervise.bot_bottle_root. The server
test patches both flat and package _supervise_types since handle_tools_call
runs in the package context while the responder thread uses the flat module.

Dockerfile.sidecars gets a COPY for _supervise_types.py so the sidecar
bundle retains its flat import chain.

Follow-up to issuecomment-2872 on PR #320.
2026-07-06 18:53:25 +00:00
didericis-claude ed8fbfdfa8 feat: add StoreManager singleton for centralized store migration
lint / lint (push) Successful in 1m59s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Failing after 1m3s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-06 18:52:01 +00:00
didericis-claude 5fcca2060f feat(db_store): decouple migration from store construction
lint / lint (push) Failing after 2m4s
test / unit (pull_request) Successful in 1m2s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Failing after 1m9s
Remove auto-migration from DbStore.__init__; add explicit check_migrations()
and migrate() methods. The CLI now checks both stores on every startup and
prompts the user to confirm before migrating. supervise.prepare() calls
.migrate() directly now that __init__ no longer does it implicitly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-06 17:16:18 +00:00
didericis-claude a4d82e5ff2 refactor: extract TableMigrations and DbStore base class
lint / lint (push) Successful in 2m0s
test / unit (pull_request) Successful in 1m4s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m9s
Adds bot_bottle/migrations.py (TableMigrations) and bot_bottle/db_store.py
(DbStore) per PR review. Both stores now inherit from DbStore and hold a
TableMigrations instance instead of duplicating schema-version logic inline.
2026-07-02 21:04:34 +00:00
didericis-claude e8e4f6f7c7 refactor: address PR review — rename, move helpers, add migration runner
lint / lint (push) Successful in 2m4s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m5s
Per review #320 comments:

- Rename _sv() → get_supervise_mod() in both store files (review 206/211)
- Move _audit_entry_from_row onto AuditStore as _row_to_entry static method
  (review 208); move _proposal/_response_from_row onto QueueStore (review 211)
- Remove _host_db_path() free function; inline into __init__ (review 209/211)
- Add stdlib migration runner using a shared schema_versions table; each store
  tracks its own version under a module key so they can coexist in the same DB
  without clobbering a shared PRAGMA user_version (reviews 210/212/213)
- PRD: add goal 6 (migration runner), narrow non-goal to third-party ORM only
2026-07-02 03:27:02 +00:00
didericis-claude 5f0fc0d540 test: cover QueueStore/AuditStore guard branches and supervise bundle spec
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 52s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Successful in 1m2s
Add 8 tests covering the branches that were keeping diff-coverage below
90%: explicit db_path constructor arg, early-return guards when the DB
file is absent, _chmod OSError swallowing in both store classes, and the
supervise volume/env/daemon path in _bundle_launch_spec.

Diff-coverage rises from 89.2% to 94.6% (176/186 changed lines).
2026-07-02 02:59:26 +00:00
didericis-claude 244ad6a914 refactor: extract QueueStore and AuditStore to their own modules
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Failing after 59s
Moves _QueueStore → bot_bottle/queue_store.py (public QueueStore) and
_AuditStore → bot_bottle/audit_store.py (public AuditStore). Removes
the public queue_db_path() function; QueueStore resolves the DB path
via host_db_path() on the host, or via the SUPERVISE_DB_PATH env var
in the sidecar container (internal mechanism, not public API).

Adds queue_store.py and audit_store.py to Dockerfile.sidecars so the
sidecar bundle picks them up. Updates __all__ in supervise.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-01 21:45:08 +00:00
didericis-codex 29904609da fix(supervise): remove queue directory from db-backed flow
lint / lint (push) Successful in 2m4s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m10s
2026-07-01 19:50:38 +00:00
didericis-codex 3067b067d2 fix(supervise): store queue rows in host sqlite db
lint / lint (push) Successful in 2m5s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m2s
2026-07-01 19:33:43 +00:00
didericis-codex 212551df9a docs(prd): activate sqlite local storage
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m6s
2026-07-01 16:57:54 +00:00
didericis-codex f1b8bbdfa1 test(supervise): update edge cases for sqlite storage
lint / lint (push) Successful in 1m55s
test / unit (pull_request) Successful in 53s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m10s
2026-07-01 16:57:45 +00:00
didericis-codex 08918f9a8a feat(supervise): store queue and audit data in sqlite
lint / lint (push) Failing after 1m53s
test / unit (pull_request) Failing after 45s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Failing after 50s
2026-07-01 16:56:23 +00:00
didericis-codex 9af02831ea docs(prd): add sqlite local storage plan 2026-07-01 16:53:23 +00:00
Quality Badge Bot 5970b785aa chore: update quality badges
- Coverage: 83%
- Core coverage: 95%

[skip ci]
2026-07-01 16:51:08 +00:00
didericis 2f5cf81cf5 fix(git-gate): defer dynamic key provisioning
lint / lint (push) Successful in 1m59s
test / unit (push) Successful in 49s
test / integration (push) Successful in 23s
test / coverage (push) Successful in 1m0s
Update Quality Badges / update-badges (push) Successful in 53s
2026-07-01 12:45:46 -04:00
didericis 4a1e667306 fix(git-gate): inline GIT_GATE_TIMEOUT_SECS to fix git-http ImportError
lint / lint (push) Successful in 1m56s
test / unit (push) Successful in 48s
test / integration (push) Successful in 20s
test / coverage (push) Successful in 1m2s
Update Quality Badges / update-badges (push) Successful in 52s
git_http_backend.py is copied flat into the sidecar bundle image as a
standalone script, not as part of the bot_bottle package, and
git_gate.py/git_gate_render.py are never copied in. Its relative
import of GIT_GATE_TIMEOUT_SECS crashed the git-http daemon (port
9420) on every startup, silently leaving the smart-HTTP git-gate
transport down while the other sidecar daemons stayed up.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 11:53:26 -04:00
80 changed files with 4861 additions and 1357 deletions
+8 -2
View File
@@ -18,7 +18,7 @@
# /git-gate-entrypoint.sh docker-cp'd at start time # /git-gate-entrypoint.sh docker-cp'd at start time
# /git-gate/creds/* docker-cp'd at start time # /git-gate/creds/* docker-cp'd at start time
# /git/* bare repos, populated at runtime # /git/* bare repos, populated at runtime
# /run/supervise/queue/ bind-mounted at run time # /run/supervise/bot-bottle.db bind-mounted at run time
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir # /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
# #
# Exposed ports inside the container: # Exposed ports inside the container:
@@ -66,6 +66,12 @@ COPY bot_bottle/egress_dlp_config.py /app/egress_dlp_config.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/migrations.py /app/migrations.py
COPY bot_bottle/db_store.py /app/db_store.py
COPY bot_bottle/supervise_types.py /app/supervise_types.py
COPY bot_bottle/queue_store.py /app/queue_store.py
COPY bot_bottle/audit_store.py /app/audit_store.py
COPY bot_bottle/store_manager.py /app/store_manager.py
COPY bot_bottle/supervise.py /app/supervise.py COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
@@ -81,7 +87,7 @@ RUN mkdir -p \
/etc/git-gate \ /etc/git-gate \
/git-gate/creds \ /git-gate/creds \
/git \ /git \
/run/supervise/queue \ /run/supervise \
/home/mitmproxy/.mitmproxy /home/mitmproxy/.mitmproxy
# Documentation only — the compose renderer publishes whichever # Documentation only — the compose renderer publishes whichever
+19 -3
View File
@@ -6,7 +6,7 @@
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml) [![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen)](https://coverage.readthedocs.io/) [![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-96%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md) [![core coverage](https://img.shields.io/badge/core%20coverage-95%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data. **Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -25,7 +25,7 @@
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding. - **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required. - **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network. - **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. - **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. Runs on macOS (Hypervisor.framework) and Linux (KVM, `/dev/kvm`).
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`. - **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
## Architecture ## Architecture
@@ -71,10 +71,26 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
## Quickstart ## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus `smolvm` (macOS or Linux). The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend. Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
### smolmachines on Linux
The smolmachines backend runs on Linux as well as macOS. On Linux, `smolvm`/libkrun use KVM, so the host needs:
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
- **`smolvm`** on `PATH`: `curl -sSL https://smolmachines.com/install.sh | sh`.
- **Docker** for the sidecar bundle and image build, same as macOS.
Per-bottle isolation works the same as macOS without any `ifconfig`/sudo step — all of `127.0.0.0/8` is already loopback on Linux, so each bottle's sidecar bundle is published on its own `127.0.0.<N>` and TSI's allowlist is scoped to that `/32`.
```sh
BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>
```
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. If you run bottles from a Gitea Actions runner, use a `host`-label runner so Docker, `smolvm`, and `/dev/kvm` are all reachable from the job. `smolvm` isn't in nixpkgs — install the release binary (pin the version) and put it on the runner's `PATH`.
```sh ```sh
./cli.py start <agent> # builds the image on first run, drops you into claude ./cli.py start <agent> # builds the image on first run, drops you into claude
``` ```
+4
View File
@@ -227,6 +227,10 @@ class AgentProvider(ABC):
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
from .log import die from .log import die
cert_host_path, label = select_ca_cert(plan.egress_plan) cert_host_path, label = select_ca_cert(plan.egress_plan)
# Ensure the target directory exists. smolvm's pack step may not
# preserve the empty /usr/local/share/ca-certificates/ directory
# on Linux; mkdir -p is idempotent and safe for all backends.
bottle.exec("mkdir -p /usr/local/share/ca-certificates", user="root")
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH) bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
r = bottle.exec( r = bottle.exec(
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates", f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
+91
View File
@@ -0,0 +1,91 @@
"""SQLite-backed audit store for supervise (PRD 0013)."""
from __future__ import annotations
import sqlite3
from pathlib import Path
try:
from .supervise_types import AuditEntry, host_db_path
from .db_store import DbStore
from .migrations import TableMigrations
except ImportError:
from supervise_types import AuditEntry, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
class AuditStore(DbStore):
"""SQLite-backed persistent store for supervise audit entries."""
def __init__(self, db_path: Path | None = None) -> None:
# One entry per schema version: migrations[0] brings a fresh DB to
# version 1, [1] to version 2, etc. Add new entries at the end; never
# edit existing ones.
migrations = TableMigrations("audit_store", [
# v1 — initial schema
"""
CREATE TABLE IF NOT EXISTS supervise_audit_entries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT NOT NULL,
bottle_slug TEXT NOT NULL,
component TEXT NOT NULL,
operator_action TEXT NOT NULL,
operator_notes TEXT NOT NULL,
justification TEXT NOT NULL,
diff TEXT NOT NULL
)
""",
])
super().__init__(db_path or host_db_path(), migrations)
def write_audit_entry(self, entry: AuditEntry) -> Path:
with self._connect() as conn:
conn.execute(
"""
INSERT INTO supervise_audit_entries (
timestamp, bottle_slug, component, operator_action,
operator_notes, justification, diff
) VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(
entry.timestamp,
entry.bottle_slug,
entry.component,
entry.operator_action,
entry.operator_notes,
entry.justification,
entry.diff,
),
)
self._chmod()
return self.db_path
def read_audit_entries(self, component: str, slug: str) -> list[AuditEntry]:
if not self.db_path.is_file():
return []
with self._connect() as conn:
rows = conn.execute(
"""
SELECT * FROM supervise_audit_entries
WHERE component = ? AND bottle_slug = ?
ORDER BY id
""",
(component, slug),
).fetchall()
return [self._row_to_entry(row) for row in rows]
@staticmethod
def _row_to_entry(row: sqlite3.Row) -> AuditEntry:
return AuditEntry(
timestamp=row["timestamp"],
bottle_slug=row["bottle_slug"],
component=row["component"],
operator_action=row["operator_action"],
operator_notes=row["operator_notes"],
justification=row["justification"],
diff=row["diff"],
)
__all__ = ["AuditStore"]
+10
View File
@@ -75,6 +75,9 @@ class BottleSpec:
# Ordered bottle names selected at launch (issue #269). When non-empty # Ordered bottle names selected at launch (issue #269). When non-empty
# they are merged in order and replace the agent's `bottle:` field. # they are merged in order and replace the agent's `bottle:` field.
bottle_names: tuple[str, ...] = () bottle_names: tuple[str, ...] = ()
# True when launched via --headless (no TTY, no interactive prompts).
# The git-gate host-key preflight uses this to error rather than prompt.
headless: bool = False
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -300,6 +303,13 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
self._preflight() self._preflight()
from ..git_gate_host_key import preflight_host_keys
manifest = preflight_host_keys(
manifest,
headless=spec.headless,
home_md=spec.manifest.home_md,
)
manifest_bottle = manifest.bottle manifest_bottle = manifest.bottle
manifest_agent_provider = manifest_bottle.agent_provider manifest_agent_provider = manifest_bottle.agent_provider
agent_provider = get_provider(manifest_agent_provider.template) agent_provider = get_provider(manifest_agent_provider.template)
+4 -5
View File
@@ -34,7 +34,7 @@ from ...egress import (
from ...git_gate import GIT_GATE_HOSTNAME from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn from ...log import die, warn
from ...supervise import ( from ...supervise import (
QUEUE_DIR_IN_CONTAINER, DB_PATH_IN_CONTAINER,
SUPERVISE_HOSTNAME, SUPERVISE_HOSTNAME,
SUPERVISE_PORT, SUPERVISE_PORT,
) )
@@ -163,16 +163,15 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
if sp is not None: if sp is not None:
env += [ env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}", f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}", f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}", f"SUPERVISE_PORT={SUPERVISE_PORT}",
] ]
volumes.append({ volumes.append({
"type": "bind", "type": "bind",
"source": str(sp.queue_dir), "source": str(sp.db_path),
"target": QUEUE_DIR_IN_CONTAINER, "target": DB_PATH_IN_CONTAINER,
"read_only": False, "read_only": False,
}) })
internal_aliases = [EGRESS_HOSTNAME] internal_aliases = [EGRESS_HOSTNAME]
if gp.upstreams: if gp.upstreams:
internal_aliases.append(GIT_GATE_HOSTNAME) internal_aliases.append(GIT_GATE_HOSTNAME)
+9 -1
View File
@@ -37,7 +37,10 @@ from pathlib import Path
from typing import Callable, Generator from typing import Callable, Generator
from ...egress import egress_resolve_token_values from ...egress import egress_resolve_token_values
from ...git_gate import revoke_git_gate_provisioned_keys from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...log import info, warn from ...log import info, warn
from . import network as network_mod from . import network as network_mod
from . import util as docker_mod from . import util as docker_mod
@@ -118,6 +121,11 @@ def launch(
git_gate_plan = plan.git_gate_plan git_gate_plan = plan.git_gate_plan
if git_gate_plan.upstreams: if git_gate_plan.upstreams:
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
git_gate_plan,
git_gate_state_dir(plan.slug),
)
git_gate_plan = dataclasses.replace( git_gate_plan = dataclasses.replace(
git_gate_plan, git_gate_plan,
internal_network=internal_network, internal_network=internal_network,
+30 -4
View File
@@ -28,9 +28,12 @@ from ...egress import (
egress_resolve_token_values, egress_resolve_token_values,
egress_sidecar_env_entries, egress_sidecar_env_entries,
) )
from ...git_gate import revoke_git_gate_provisioned_keys from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...log import die, info, warn from ...log import die, info, warn
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
from ..docker.git_gate import ( from ..docker.git_gate import (
@@ -98,6 +101,8 @@ def launch(
egress_network = egress_network_name(plan.slug) egress_network = egress_network_name(plan.slug)
_create_networks(internal_network, egress_network, stack) _create_networks(internal_network, egress_network, stack)
plan = _provision_git_gate_keys(plan)
sidecar_name = sidecar_container_name(plan.slug) sidecar_name = sidecar_container_name(plan.slug)
container_mod.force_remove_container(sidecar_name) container_mod.force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network) _start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
@@ -241,6 +246,19 @@ def _stamp_agent_urls(
) )
def _provision_git_gate_keys(
plan: MacosContainerBottlePlan,
) -> MacosContainerBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None: def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan gp = plan.git_gate_plan
if not gp.upstreams: if not gp.upstreams:
@@ -361,7 +379,7 @@ def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
if plan.supervise_plan is not None: if plan.supervise_plan is not None:
env += [ env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}", f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}", f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}", f"SUPERVISE_PORT={SUPERVISE_PORT}",
] ]
return tuple(env) return tuple(env)
@@ -387,7 +405,15 @@ def _sidecar_mounts(
sp = plan.supervise_plan sp = plan.supervise_plan
if sp is not None: if sp is not None:
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False)) # `container run --mount type=bind` only accepts directory
# sources (a file source fails with "is not a directory") —
# mount db_path's dedicated parent dir instead of the file
# itself, same as the CA/routes mounts above.
mounts.append((
str(sp.db_path.parent),
str(Path(DB_PATH_IN_CONTAINER).parent),
False,
))
return tuple(mounts) return tuple(mounts)
+14
View File
@@ -47,10 +47,24 @@ _HOME_FOR = {
"root": "/root", "root": "/root",
} }
_DEFAULT_PATH_FOR = {
# Committed smolmachine snapshots are rebuilt from a rootfs tarball and
# lose Docker image ENV metadata. Restore the provider CLI path here so
# resumed Codex bottles can still find the per-user install.
"node": (
"/home/node/.local/bin:"
"/home/node/.codex/packages/standalone/current/bin:"
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
),
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
}
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]: def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
home = _HOME_FOR.get(user, f"/home/{user}") home = _HOME_FOR.get(user, f"/home/{user}")
out = [f"HOME={home}", f"USER={user}"] out = [f"HOME={home}", f"USER={user}"]
if "PATH" not in env:
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
for k, v in env.items(): for k, v in env.items():
out.append(f"{k}={v}") out.append(f"{k}={v}")
return out return out
+10 -23
View File
@@ -1,10 +1,8 @@
"""SmolmachinesBottlePlan — concrete BottlePlan for the smolmachines """SmolmachinesBottlePlan — concrete BottlePlan for the smolmachines
backend (PRD 0023). backend (PRD 0023).
Slug + bundle docker subnet / gateway / pinned IP + smolvm Slug + legacy bundle network coordinates + smolvm machine name +
machine name + agent `.smolmachine` artifact + per-bottle guest agent `.smolmachine` artifact + per-bottle guest env."""
env. Provisioning fields (CA cert path, prompt path, etc.) land
in chunk 4."""
from __future__ import annotations from __future__ import annotations
@@ -23,9 +21,10 @@ class SmolmachinesBottlePlan(BottlePlan):
`supervise_plan`, and `agent_provision` from BottlePlan.""" `supervise_plan`, and `agent_provision` from BottlePlan."""
slug: str slug: str
# Per-bottle docker subnet for the sidecar bundle container. # Legacy per-bottle bundle network coordinates. These remain on
# The bundle runs at `bundle_ip` (always `.2`); the gateway is # the plan while BundleLaunchSpec still carries the original shape,
# at `.1`. smolvm's TSI allowlist is set to `bundle_ip/32`. # but the smolmachines launch path exposes the sidecar VM through
# host-loopback forwarders instead of a Docker bridge IP.
bundle_subnet: str bundle_subnet: str
bundle_gateway: str bundle_gateway: str
bundle_ip: str bundle_ip: str
@@ -36,22 +35,10 @@ class SmolmachinesBottlePlan(BottlePlan):
# `--smolfile` is mutually exclusive with `--from`, and # `--smolfile` is mutually exclusive with `--from`, and
# `--from` is the path that avoids the registry-pull race). # `--from` is the path that avoids the registry-pull race).
guest_env: dict[str, str] guest_env: dict[str, str]
# Inner Plans for the sidecar bundle daemons. The same shape the # Agent-side endpoints. Empty at prepare time; launch populates
# docker backend uses — same `.prepare()` calls produced # these after sidecar VM bringup via `dataclasses.replace`.
# them — but our launch step doesn't populate the # Format: a `host:port` for git-gate (insteadOf URL prefix) +
# docker-specific network fields (internal_network, # full URLs for proxy / supervise.
# egress_network) because the smolmachines bundle isn't on
# docker's `--internal` + egress bridge topology; it's on a
# per-bottle bridge with a pinned IP. The unused fields stay
# at their dataclass defaults.
# Agent-side endpoints. On Docker Desktop the docker bridge
# IPs aren't reachable from the smolvm guest (TSI uses macOS
# networking; docker container IPs live in the daemon's VM),
# so the agent dials the bundle via host loopback +
# docker-published random ports. Empty at prepare time;
# launch populates these after bundle bringup via
# `dataclasses.replace`. Format: a `host:port` for git-gate
# (insteadOf URL prefix) + full URLs for proxy / supervise.
agent_proxy_url: str = "" agent_proxy_url: str = ""
agent_git_gate_host: str = "" agent_git_gate_host: str = ""
agent_supervise_url: str = "" agent_supervise_url: str = ""
@@ -1,20 +1,51 @@
"""Egress apply for the smolmachines backend. """Egress apply for the smolmachines backend.
The smolmachines sidecar bundle runs as a host-side Docker container, The smolmachines sidecar bundle runs as a sidecar smolVM. Route-file
so egress signalling is identical to the docker backend. inspection and reload signalling therefore go through ``smolvm machine
exec`` instead of Docker.
""" """
from __future__ import annotations from __future__ import annotations
from ..docker.egress_apply import ( # noqa: F401 from ...egress import EGRESS_ROUTES_IN_CONTAINER
DockerEgressApplicator, from ...log import warn
EgressApplyError, from ..egress_apply import EgressApplicator, EgressApplyError
applicator, from . import sidecar_bundle as _bundle
fetch_current_routes, from . import smolvm as _smolvm
)
def fetch_current_routes(slug: str) -> str:
machine = _bundle.bundle_machine_name(slug)
result = _smolvm.machine_exec(machine, ["cat", EGRESS_ROUTES_IN_CONTAINER])
if result.returncode != 0:
raise EgressApplyError(
f"could not read routes.yaml from {machine}: "
f"{(result.stderr or '').strip() or 'sidecar VM not running?'}"
)
return result.stdout
class SmolmachinesEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None:
machine = _bundle.bundle_machine_name(slug)
result = _smolvm.machine_exec(machine, ["sh", "-c", "kill -HUP 1"])
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'smolvm exec failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {machine}: "
f"{last_error or 'smolvm exec failed'}"
)
applicator = SmolmachinesEgressApplicator()
__all__ = [ __all__ = [
"DockerEgressApplicator", "SmolmachinesEgressApplicator",
"EgressApplyError", "EgressApplyError",
"applicator", "applicator",
"fetch_current_routes", "fetch_current_routes",
+6 -1
View File
@@ -31,6 +31,7 @@ from . import sidecar_bundle as _bundle
# matching the bundle container name pattern. We use the prefix # matching the bundle container name pattern. We use the prefix
# both as a filter and to strip back to the slug. # both as a filter and to strip back to the slug.
_VM_NAME_PREFIX = "bot-bottle-" _VM_NAME_PREFIX = "bot-bottle-"
_SIDECAR_VM_PREFIX = _bundle.bundle_machine_name("")
def enumerate_active() -> list[ActiveAgent]: def enumerate_active() -> list[ActiveAgent]:
@@ -54,7 +55,11 @@ def enumerate_active() -> list[ActiveAgent]:
for m in machines: for m in machines:
name = m.get("name") or "" name = m.get("name") or ""
state = m.get("state") or "" state = m.get("state") or ""
if state != "running" or not name.startswith(_VM_NAME_PREFIX): if (
state != "running"
or not name.startswith(_VM_NAME_PREFIX)
or name.startswith(_SIDECAR_VM_PREFIX)
):
continue continue
slug = name[len(_VM_NAME_PREFIX):] slug = name[len(_VM_NAME_PREFIX):]
metadata = read_metadata(slug) metadata = read_metadata(slug)
+138 -71
View File
@@ -1,12 +1,9 @@
"""End-to-end launch flow for the smolmachines backend """End-to-end launch flow for the smolmachines backend.
(PRD 0023 chunks 2d + 4b).
Brings up the per-bottle docker bridge + sidecar bundle (with Builds the sidecar bundle smolmachine, starts it as a sidecar VM
real daemons + their config files), creates + starts the smolvm with real daemons + their config files, creates + starts the agent
guest pointed at the bundle's pinned IP via TSI's smolVM, yields a `SmolmachinesBottle` handle, and tears everything
`--allow-cidr <bundle-ip>/32` allowlist, yields a down on context exit.
`SmolmachinesBottle` handle, tears everything down on context
exit.
The bundle's daemons consume the inner Plans the docker backend The bundle's daemons consume the inner Plans the docker backend
already produces: egress reads routes + CAs from the EgressPlan. already produces: egress reads routes + CAs from the EgressPlan.
@@ -27,7 +24,7 @@ from ...egress import (
egress_resolve_token_values, egress_resolve_token_values,
egress_sidecar_env_entries, egress_sidecar_env_entries,
) )
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde from ...util import expand_tilde
from ..docker import util as docker_mod from ..docker import util as docker_mod
from ..docker.egress import ( from ..docker.egress import (
@@ -41,7 +38,10 @@ from ..docker.git_gate import (
GIT_GATE_ENTRYPOINT_IN_CONTAINER, GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER, GIT_GATE_HOOK_IN_CONTAINER,
) )
from ...git_gate import revoke_git_gate_provisioned_keys from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...log import info, warn from ...log import info, warn
from ...bottle_state import ( from ...bottle_state import (
egress_state_dir, egress_state_dir,
@@ -49,6 +49,7 @@ from ...bottle_state import (
read_committed_image, read_committed_image,
) )
from . import loopback_alias as _loopback from . import loopback_alias as _loopback
from . import port_forward as _forward
from . import sidecar_bundle as _bundle from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm from . import smolvm as _smolvm
from .bottle import SmolmachinesBottle from .bottle import SmolmachinesBottle
@@ -68,9 +69,8 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
# Container-internal listening ports for each bundle daemon. The # Container-internal listening ports for each bundle daemon. The
# bundle publishes each one on a random host loopback port (see # sidecar VM publishes each one on a random host loopback port, and
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks # the launch flow wraps those raw ports with per-bottle forwarders.
# them up post-start.
_GIT_HTTP_PORT = 9420 _GIT_HTTP_PORT = 9420
_SUPERVISE_PORT = SUPERVISE_PORT _SUPERVISE_PORT = SUPERVISE_PORT
@@ -88,12 +88,12 @@ def launch(
try: try:
loopback_ip, network = _allocate_resources(plan, stack) loopback_ip, network = _allocate_resources(plan, stack)
plan = _mint_certs(plan) plan = _mint_certs(plan)
plan = _start_bundle(plan, network, loopback_ip, stack) proxy_host = loopback_ip
plan = _discover_urls(plan, loopback_ip) plan = _start_bundle(plan, network, proxy_host, stack)
agent_from_path = _agent_from_path(plan) agent_from_path = _agent_from_path(plan)
_launch_vm(plan, agent_from_path, loopback_ip, stack) _launch_vm(plan, agent_from_path, proxy_host, stack)
_init_vm(plan) _init_vm(plan)
bottle = SmolmachinesBottle( bottle = SmolmachinesBottle(
@@ -139,17 +139,17 @@ def _allocate_resources(
plan: SmolmachinesBottlePlan, plan: SmolmachinesBottlePlan,
stack: ExitStack, stack: ExitStack,
) -> tuple[str, str]: ) -> tuple[str, str]:
"""Reserve a loopback alias and create the per-bottle docker bridge. """Reserve a per-bottle host address.
macOS only routes 127.0.0.1 by default; the per-bottle alias The per-bottle address scopes TSI's allowlist to this bottle's
scopes TSI's allowlist to this bottle's published ports so the forwarder-published ports so the agent can't reach other bottles'
agent can't reach other bottles' or host services' ports on or host services. The returned network name remains in the bundle
loopback. No-op on Linux.""" spec for compatibility with older helper tests; no Docker sidecar
container is launched."""
del stack
_loopback.ensure_pool() _loopback.ensure_pool()
loopback_ip = _loopback.allocate(plan.slug) loopback_ip = _loopback.allocate(plan.slug)
network = _bundle.bundle_network_name(plan.slug) network = _bundle.bundle_network_name(plan.slug)
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
stack.callback(_bundle.remove_bundle_network, network)
return loopback_ip, network return loopback_ip, network
@@ -169,54 +169,86 @@ def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
def _start_bundle( def _start_bundle(
plan: SmolmachinesBottlePlan, plan: SmolmachinesBottlePlan,
network: str, network: str,
loopback_ip: str, proxy_host: str,
stack: ExitStack, stack: ExitStack,
) -> SmolmachinesBottlePlan: ) -> SmolmachinesBottlePlan:
"""Build the BundleLaunchSpec, resolve token env, start the """Build the BundleLaunchSpec, start the sidecar VM, wrap its raw
sidecar bundle container, and register teardown.""" smolVM-published loopback ports with per-bottle forwarders, stamp
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip) agent URLs from those forwarder ports, and register teardown."""
plan = _provision_git_gate_keys(plan)
bundle_spec = _bundle_launch_spec(plan, network, proxy_host)
token_env = _resolve_token_env(plan, dict(os.environ)) token_env = _resolve_token_env(plan, dict(os.environ))
_bundle.ensure_bundle_image(bundle_spec.image) artifact = _ensure_smolmachine(
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env}) bundle_spec.image,
stack.callback(_bundle.stop_bundle, plan.slug) dockerfile=_bundle.SIDECAR_BUNDLE_DOCKERFILE,
return plan )
launch = _bundle.start_bundle_vm(
bundle_spec,
from_path=artifact,
host_env={**os.environ, **token_env},
)
stack.callback(_bundle.stop_bundle_vm, plan.slug)
forward_specs = tuple(
_forward.ForwardSpec(
label=_label_for_port(container_port),
listen_host=proxy_host,
listen_port=0,
target_host="127.0.0.1",
target_port=host_port,
)
for container_port, host_port in launch.raw_ports.items()
)
handle = _forward.start_forwarder(forward_specs)
stack.callback(_forward.stop_forwarder, handle)
published_ports = {
_port_for_label(spec.label): spec.listen_port
for spec in handle.forwards
}
return _discover_urls(plan, proxy_host, published_ports)
def _provision_git_gate_keys(
plan: SmolmachinesBottlePlan,
) -> SmolmachinesBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _discover_urls( def _discover_urls(
plan: SmolmachinesBottlePlan, plan: SmolmachinesBottlePlan,
loopback_ip: str, proxy_host: str,
published_ports: dict[int, int],
) -> SmolmachinesBottlePlan: ) -> SmolmachinesBottlePlan:
"""Discover host-side ports for published container ports and """Stamp URLs + guest_env from per-bottle forwarder ports.
return the plan with URLs + guest_env stamped in.
Docker container IPs (192.168.x.x in the daemon's bridge) `proxy_host` is the host IP that both TSI's allowlist and the
aren't reachable from the smolvm guest on macOS — TSI uses forwarder listeners are keyed to. The raw smolVM-published ports
macOS networking, and macOS sees the daemon's bridge via the are intentionally not advertised to the agent.
published-port loopback forward only.
NO_PROXY includes the per-bottle loopback alias so the NO_PROXY includes `proxy_host` so supervise + git-gate URLs
supervise + git-gate URLs bypass HTTPS_PROXY.""" bypass HTTPS_PROXY."""
agent_facing_host_port = _bundle.bundle_host_port( agent_facing_host_port = published_ports[_EGRESS_PORT]
plan.slug, _EGRESS_PORT, host_ip=loopback_ip, agent_proxy_url = f"http://{proxy_host}:{agent_facing_host_port}"
)
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
agent_git_gate_host = "" agent_git_gate_host = ""
if plan.git_gate_plan.upstreams: if plan.git_gate_plan.upstreams:
git_gate_host_port = _bundle.bundle_host_port( git_gate_host_port = published_ports[_GIT_HTTP_PORT]
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip, agent_git_gate_host = f"{proxy_host}:{git_gate_host_port}"
)
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
agent_supervise_url = "" agent_supervise_url = ""
if plan.supervise_plan is not None: if plan.supervise_plan is not None:
supervise_host_port = _bundle.bundle_host_port( supervise_host_port = published_ports[_SUPERVISE_PORT]
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip, agent_supervise_url = f"http://{proxy_host}:{supervise_host_port}/"
)
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1") existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
no_proxy = f"{existing_no_proxy},{loopback_ip}" no_proxy = f"{existing_no_proxy},{proxy_host}"
guest_env = { guest_env = {
**plan.guest_env, **plan.guest_env,
"HTTPS_PROXY": agent_proxy_url, "HTTPS_PROXY": agent_proxy_url,
@@ -246,27 +278,31 @@ def _discover_urls(
def _launch_vm( def _launch_vm(
plan: SmolmachinesBottlePlan, plan: SmolmachinesBottlePlan,
agent_from_path: Path, agent_from_path: Path,
loopback_ip: str, proxy_host: str,
stack: ExitStack, stack: ExitStack,
) -> None: ) -> None:
"""Create, patch, and start the smolvm VM; register teardown. """Create, patch, and start the smolvm VM; register teardown.
--allow-cidr is the per-bottle loopback alias so the guest can --allow-cidr is `proxy_host/32` — the per-bottle loopback alias.
only reach this bottle's bundle ports. force_allowlist patches This ensures the guest can only reach sidecar forwarders published
smolvm 0.8.0's silent-drop of --allow-cidr when combined with on that IP, not host localhost or another bottle's alias.
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from force_allowlist confirms the allowlist persisted (patching smolvm
and --smolfile mutually exclusive.""" 0.8.0's silent-drop of --allow-cidr when combined with --from) and
fails closed if it can't. Smolfile isn't usable here — smolvm 0.8.0
makes --from and --smolfile mutually exclusive."""
tsi_cidr = f"{proxy_host}/32"
_smolvm.machine_create( _smolvm.machine_create(
plan.machine_name, plan.machine_name,
from_path=agent_from_path, from_path=agent_from_path,
allow_cidrs=[f"{loopback_ip}/32"], allow_cidrs=[tsi_cidr],
env=plan.guest_env, env=plan.guest_env,
) )
stack.callback(_smolvm.machine_delete, plan.machine_name) stack.callback(_smolvm.machine_delete, plan.machine_name)
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped # Confirm the booted VM's TSI allowlist will actually enforce the
# when combined with `--from`. Patch the persisted state DB # /32 before start (smolvm 0.8.0 silently drops `--allow-cidr`
# before start so the booted VM's TSI actually enforces. # with `--from`, so the persisted state DB is patched if needed).
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"]) # Fails closed if enforcement can't be confirmed.
_loopback.force_allowlist(plan.machine_name, [tsi_cidr])
_smolvm.machine_start(plan.machine_name) _smolvm.machine_start(plan.machine_name)
stack.callback(_smolvm.machine_stop, plan.machine_name) stack.callback(_smolvm.machine_stop, plan.machine_name)
@@ -275,7 +311,9 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
"""Repair filesystem ownership and wait for exec channel readiness. """Repair filesystem ownership and wait for exec channel readiness.
Ownership repair: smolvm's pack process remaps files to the host Ownership repair: smolvm's pack process remaps files to the host
invoker's uid (501 on macOS). /home/node must be node:node so invoker's uid (e.g. 501 on macOS, 1000 on Linux). The chowns use
names not numbers so they're correct on either. /home/node must
be node:node so
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
mode 1777 so non-root processes can create per-uid scratch dirs. mode 1777 so non-root processes can create per-uid scratch dirs.
All folded into one sh -c to avoid back-to-back exec calls All folded into one sh -c to avoid back-to-back exec calls
@@ -298,8 +336,30 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
_smolvm.wait_exec_ready(plan.machine_name) _smolvm.wait_exec_ready(plan.machine_name)
def _label_for_port(port: int) -> str:
if port == _EGRESS_PORT:
return "egress"
if port == _GIT_HTTP_PORT:
return "git-http"
if port == _SUPERVISE_PORT:
return "supervise"
return f"port-{port}"
def _port_for_label(label: str) -> int:
if label == "egress":
return _EGRESS_PORT
if label == "git-http":
return _GIT_HTTP_PORT
if label == "supervise":
return _SUPERVISE_PORT
if label.startswith("port-"):
return int(label.removeprefix("port-"))
raise ValueError(f"unknown sidecar forward label: {label}")
def _bundle_launch_spec( def _bundle_launch_spec(
plan: SmolmachinesBottlePlan, network: str, loopback_ip: str, plan: SmolmachinesBottlePlan, network: str, proxy_host: str,
) -> _bundle.BundleLaunchSpec: ) -> _bundle.BundleLaunchSpec:
"""Build a BundleLaunchSpec from the resolved inner Plans. """Build a BundleLaunchSpec from the resolved inner Plans.
@@ -352,14 +412,15 @@ def _bundle_launch_spec(
daemons.append("supervise") daemons.append("supervise")
env += [ env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}", f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}", f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}", f"SUPERVISE_PORT={SUPERVISE_PORT}",
] ]
volumes.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False)) volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
# Container ports the agent reaches from the smolvm guest — # Container ports the agent reaches from the smolvm guest —
# published on host loopback so the guest can dial via TSI + # published on `proxy_host` so the TSI allowlist and the docker
# macOS networking. Egress is always the agent's HTTP/HTTPS proxy. # port-forward bindings point at the same IP. Egress is always
# the agent's HTTP/HTTPS proxy.
ports_to_publish: list[int] = [_EGRESS_PORT] ports_to_publish: list[int] = [_EGRESS_PORT]
if gp.upstreams: if gp.upstreams:
ports_to_publish.append(_GIT_HTTP_PORT) ports_to_publish.append(_GIT_HTTP_PORT)
@@ -376,7 +437,7 @@ def _bundle_launch_spec(
environment=tuple(env), environment=tuple(env),
volumes=tuple(volumes), volumes=tuple(volumes),
ports_to_publish=tuple(ports_to_publish), ports_to_publish=tuple(ports_to_publish),
publish_host_ip=loopback_ip, publish_host_ip=proxy_host,
) )
@@ -450,6 +511,12 @@ def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
return sidecar return sidecar
tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar" tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar"
docker_mod.save(image_ref, str(tarball)) docker_mod.save(image_ref, str(tarball))
# On Linux, `docker save -o` writes the tarball with owner-only
# permissions (mode 600). The crane push container runs as UID
# 65532 (distroless nonroot) and can't read it through a bind
# mount unless world-read is set. The tarball is temporary and
# lives in ~/.cache, so 644 is safe.
tarball.chmod(0o644)
try: try:
with ephemeral_registry() as handle: with ephemeral_registry() as handle:
push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}" push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}"
@@ -33,10 +33,13 @@ sudo-add the missing pool on first use per boot — the aliases
persist on `lo0` until reboot, so subsequent launches don't persist on `lo0` until reboot, so subsequent launches don't
prompt. prompt.
Linux native daemons share the host's network namespace; the On Linux the whole `127.0.0.0/8` is already routed to `lo`, so
whole `127.0.0.0/8` is reachable by default and aliases are docker can publish a bundle's ports directly on `127.0.0.<N>`
unnecessary. The pool logic detects native-Linux and skips sudo with no `ifconfig`/sudo step. `ensure_pool` is therefore a no-op
entirely; the DB patch is also gated on macOS. on Linux, but per-bottle alias *allocation* and the TSI allowlist
DB patch run on both platforms — the isolation property is
identical, it's just cheaper to set up on Linux. The state-DB
path differs per platform (see `_smolvm_db_path`).
Allocation is coordinated by inspecting running bundle Allocation is coordinated by inspecting running bundle
containers' published host IPs — each bottle's bundle owns the containers' published host IPs — each bottle's bundle owns the
@@ -47,6 +50,7 @@ from __future__ import annotations
import fcntl import fcntl
import json import json
import os
import platform import platform
import re import re
import sqlite3 import sqlite3
@@ -57,20 +61,34 @@ from typing import Iterable
from ...log import die, info from ...log import die, info
# smolvm's persistent VM state on macOS — a SQLite DB whose `vms` def _smolvm_db_path() -> Path:
# table holds one JSON BLOB per machine. The Linux path is """smolvm's persistent VM state — a SQLite DB whose `vms` table
# different, but smolmachines is macOS-only in v1 (PRD 0023) so holds one JSON BLOB per machine. macOS stores it under
# we hard-code this. If the file moves under us we'll see a `Application Support`; Linux follows the XDG base-dir spec
# clear FileNotFoundError; not worth defensive cross-platform (`$XDG_DATA_HOME`, default `~/.local/share`).
# detection until the backend actually needs Linux.
_SMOLVM_DB_PATH = ( NOTE: the Linux location is inferred from smolvm's documented
Path.home() `~/.local/share` install layout and must be confirmed against a
/ "Library" real Linux smolvm install. If it's wrong, `force_allowlist`'s
/ "Application Support" fail-closed check turns it into a clear launch-time error rather
/ "smolvm" than a silent escape."""
/ "server" if platform.system() == "Darwin":
/ "smolvm.db" return (
) Path.home()
/ "Library"
/ "Application Support"
/ "smolvm"
/ "server"
/ "smolvm.db"
)
xdg_data = os.environ.get("XDG_DATA_HOME")
base = Path(xdg_data) if xdg_data else Path.home() / ".local" / "share"
return base / "smolvm" / "server" / "smolvm.db"
# Resolved once at import: the host platform doesn't change within a
# process. Tests patch this attribute directly.
_SMOLVM_DB_PATH = _smolvm_db_path()
# Sixteen aliases by default. Tunable for hosts that want more # Sixteen aliases by default. Tunable for hosts that want more
@@ -131,51 +149,74 @@ def ensure_pool() -> None:
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None: def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
"""Patch smolvm's persistent VM-state DB to set the machine's """Ensure the machine's persisted TSI allowlist equals
`allowed_cidrs` to the given list. Workaround for smolvm `allowed_cidrs`, failing **closed** if that can't be confirmed.
0.8.0's silent-drop of `--allow-cidr` when used with `--from`.
Must run AFTER `smolvm machine create` (the row has to Runs on both macOS and Linux. It exists because smolvm 0.8.0
exist) and BEFORE `smolvm machine start` (smolvm reads the silently drops `--allow-cidr` when combined with `--from`, so
row on start; in-flight VMs don't pick up changes). Once the allowlist has to be written into smolvm's persistent state
smolvm honors the CLI flag upstream this whole function is DB before `machine start`. Rather than assume the flag was
redundant — flag-respecting create + remove this call from dropped, we read the persisted row and only patch when it
launch. doesn't already match — so a newer smolvm that honors the flag
is left untouched.
No-op on non-macOS — the DB path differs and the Linux Must run AFTER `smolvm machine create` (the row has to exist)
smolmachines code path isn't exercised in v1.""" and BEFORE `smolvm machine start` (smolvm reads the row on
if not _is_macos(): start; in-flight VMs don't pick up changes).
return
Fail-closed: if the state DB is missing, the row is missing, or
the allowlist still doesn't match after patching, we `die()`
rather than boot a VM whose egress confinement we can't verify
— an unconfirmed allowlist is a sandbox-escape risk (the agent
VM could reach all of host loopback)."""
want = list(allowed_cidrs)
if not _SMOLVM_DB_PATH.is_file(): if not _SMOLVM_DB_PATH.is_file():
die( die(
f"smolvm state DB not found at {_SMOLVM_DB_PATH}. " f"smolvm state DB not found at {_SMOLVM_DB_PATH}; cannot "
f"smolvm 0.8.0 expected? `smolvm --version` to check." f"confirm the TSI allowlist is enforced. Refusing to launch "
f"(fail-closed). Check `smolvm --version` and the DB "
f"location for your platform."
) )
con = sqlite3.connect(str(_SMOLVM_DB_PATH)) con = sqlite3.connect(str(_SMOLVM_DB_PATH))
try: try:
cur = con.cursor() cfg = _read_machine_cfg(con, machine_name)
row = cur.execute( if cfg.get("allowed_cidrs") != want:
"SELECT data FROM vms WHERE name = ?", (machine_name,), cfg["allowed_cidrs"] = want
).fetchone() # Write as BLOB (the column type smolvm uses) — passing a
if row is None: # plain str makes sqlite store it as Text and smolvm then
die( # fails to read it.
f"smolvm DB has no row for machine {machine_name!r}" con.execute(
f"machine_create must run before force_allowlist." "UPDATE vms SET data = ? WHERE name = ?",
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
)
con.commit()
cfg = _read_machine_cfg(con, machine_name)
if cfg.get("allowed_cidrs") != want:
die(
f"could not enforce TSI allowlist {want!r} for machine "
f"{machine_name!r} (persisted value is "
f"{cfg.get('allowed_cidrs')!r}). Refusing to launch "
f"(fail-closed)."
) )
cfg = json.loads(row[0])
cfg["allowed_cidrs"] = list(allowed_cidrs)
# Write as BLOB (the column type smolvm uses) — passing a
# plain str makes sqlite store it as Text and smolvm then
# fails to read it.
cur.execute(
"UPDATE vms SET data = ? WHERE name = ?",
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
)
con.commit()
finally: finally:
con.close() con.close()
def _read_machine_cfg(con: sqlite3.Connection, machine_name: str) -> dict[str, object]:
"""Read + JSON-decode a machine's `data` BLOB from the smolvm
state DB. Dies (fail-closed) if the row is missing — the caller
can't confirm enforcement without it."""
row = con.execute(
"SELECT data FROM vms WHERE name = ?", (machine_name,),
).fetchone()
if row is None:
die(
f"smolvm DB has no row for machine {machine_name!r}"
f"machine_create must run before force_allowlist."
)
return json.loads(row[0])
def allocate(_slug: str) -> str: def allocate(_slug: str) -> str:
"""Pick the lowest-numbered alias from the pool not already """Pick the lowest-numbered alias from the pool not already
in use by a running smolmachines bundle. Bails when the pool in use by a running smolmachines bundle. Bails when the pool
@@ -184,16 +225,17 @@ def allocate(_slug: str) -> str:
used (no on-disk reservation, allocation is purely used (no on-disk reservation, allocation is purely
docker-state-driven). docker-state-driven).
On non-macOS the whole `127.0.0.0/8` is loopback by default; Runs on both platforms: the allocation logic (docker-state
`127.0.0.1` is fine to share and we skip the alias dance. inspection + the file lock) is platform-independent. macOS
This still returns a deterministic address so launch.py's needs `ensure_pool` to have aliased the addresses on `lo0`
callers don't have to branch on platform. first; on Linux all of `127.0.0.0/8` is already loopback, so
docker can publish on the chosen `127.0.0.<N>` with no setup.
Per-bottle scoping (so the agent can't reach other bottles' or
host services' loopback ports) therefore holds on both.
An exclusive file lock serialises concurrent calls so two An exclusive file lock serialises concurrent calls so two
simultaneous launches don't read the same docker state and simultaneous launches don't read the same docker state and
claim the same alias.""" claim the same alias."""
if not _is_macos():
return "127.0.0.1"
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True) _ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf: with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
fcntl.flock(lf, fcntl.LOCK_EX) fcntl.flock(lf, fcntl.LOCK_EX)
@@ -0,0 +1,245 @@
"""Per-bottle TCP forwarder for smolmachines sidecar VMs.
smolVM currently publishes guest ports on host loopback as
``HOST:GUEST`` port pairs, without an address-scoped bind. The agent
VM's TSI allowlist is IP-only, so bot-bottle exposes a second, stricter
surface: listeners bound only to the bottle's allocated host address,
forwarding to the raw smolVM-published loopback ports.
"""
from __future__ import annotations
import argparse
import json
import os
import selectors
import signal
import socket
import subprocess
import sys
import threading
from dataclasses import asdict, dataclass
from typing import Any, Iterable, Sequence
from ...log import die, warn
_BUFFER_SIZE = 64 * 1024
_LOOPBACK_TARGETS = {"127.0.0.1", "::1"}
_FORBIDDEN_LISTEN_HOSTS = {"", "0.0.0.0", "::", "127.0.0.1", "::1"}
@dataclass(frozen=True)
class ForwardSpec:
label: str
listen_host: str
listen_port: int
target_host: str
target_port: int
@dataclass(frozen=True)
class ForwarderHandle:
process: Any
forwards: tuple[ForwardSpec, ...]
def start_forwarder(specs: Sequence[ForwardSpec]) -> ForwarderHandle:
"""Start one helper process for a bottle and wait until all
listeners are bound. ``listen_port`` may be 0; the returned specs
contain the kernel-assigned ports printed by the helper."""
if not specs:
return ForwarderHandle(process=_CompletedProcess(), forwards=())
_validate_specs(specs)
argv = [
sys.executable,
"-c",
"from bot_bottle.backend.smolmachines.port_forward import main; "
"raise SystemExit(main())",
"--spec-json",
json.dumps([asdict(s) for s in specs], separators=(",", ":")),
]
proc = subprocess.Popen(
argv,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=None,
text=True,
env=os.environ,
)
assert proc.stdout is not None
line = proc.stdout.readline()
proc.stdout.close()
if not line:
proc.terminate()
proc.wait(timeout=5)
die("smolmachines forwarder failed before reporting readiness")
try:
payload = json.loads(line)
bound = tuple(ForwardSpec(**item) for item in payload["forwards"])
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
proc.terminate()
proc.wait(timeout=5)
die(f"smolmachines forwarder returned invalid readiness payload: {exc}")
return ForwarderHandle(process=proc, forwards=bound)
def stop_forwarder(handle: ForwarderHandle) -> None:
proc = handle.process
if proc.poll() is not None:
return
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait(timeout=5)
def _validate_specs(specs: Iterable[ForwardSpec]) -> None:
for spec in specs:
if spec.listen_host in _FORBIDDEN_LISTEN_HOSTS:
die(f"refusing unsafe smolmachines forwarder bind: {spec.listen_host!r}")
if spec.target_host not in _LOOPBACK_TARGETS:
die(f"refusing non-loopback smolmachines forwarder target: {spec.target_host!r}")
if not 0 <= spec.listen_port <= 65535:
die(f"invalid smolmachines forwarder listen port: {spec.listen_port}")
if not 1 <= spec.target_port <= 65535:
die(f"invalid smolmachines forwarder target port: {spec.target_port}")
class _CompletedProcess:
"""Popen-like no-op used when a bottle has no forwards."""
def poll(self) -> int:
return 0
def terminate(self) -> None:
return None
def kill(self) -> None:
return None
def wait(self, timeout: float | None = None) -> int:
del timeout
return 0
class _Forwarder:
def __init__(self, specs: Sequence[ForwardSpec]):
_validate_specs(specs)
self.specs = tuple(specs)
self.selector = selectors.DefaultSelector()
self.stop_event = threading.Event()
self.listeners: list[tuple[socket.socket, ForwardSpec]] = []
self.bound: list[ForwardSpec] = []
def start(self) -> None:
for spec in self.specs:
listener = socket.create_server(
(spec.listen_host, spec.listen_port),
reuse_port=False,
backlog=64,
)
listener.setblocking(False)
host, port = listener.getsockname()[:2]
bound = ForwardSpec(
label=spec.label,
listen_host=str(host),
listen_port=int(port),
target_host=spec.target_host,
target_port=spec.target_port,
)
self.listeners.append((listener, bound))
self.bound.append(bound)
self.selector.register(listener, selectors.EVENT_READ, bound)
def serve(self) -> None:
while not self.stop_event.is_set():
try:
events = self.selector.select(timeout=0.25)
except OSError:
return
for key, _ in events:
listener = key.fileobj
if not isinstance(listener, socket.socket):
continue
spec = key.data
try:
client, _ = listener.accept()
except OSError:
continue
threading.Thread(
target=self._handle_client,
args=(client, spec),
daemon=True,
).start()
def stop(self) -> None:
self.stop_event.set()
for listener, _ in self.listeners:
try:
self.selector.unregister(listener)
except Exception: # noqa: BLE001 - best effort shutdown
pass
listener.close()
self.selector.close()
def _handle_client(self, client: socket.socket, spec: ForwardSpec) -> None:
with client:
try:
target = socket.create_connection((spec.target_host, spec.target_port))
except OSError as exc:
warn(f"smolmachines forwarder {spec.label}: target connect failed: {exc}")
return
with target:
left = threading.Thread(target=_pipe, args=(client, target), daemon=True)
right = threading.Thread(target=_pipe, args=(target, client), daemon=True)
left.start()
right.start()
left.join()
right.join()
def _pipe(src: socket.socket, dst: socket.socket) -> None:
while True:
try:
data = src.recv(_BUFFER_SIZE)
except OSError:
break
if not data:
break
try:
dst.sendall(data)
except OSError:
break
try:
dst.shutdown(socket.SHUT_WR)
except OSError:
pass
def main(argv: Sequence[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--spec-json", required=True)
ns = parser.parse_args(argv)
specs = tuple(ForwardSpec(**item) for item in json.loads(ns.spec_json))
forwarder = _Forwarder(specs)
def request_stop(signum: int, _frame: object) -> None:
del signum
forwarder.stop()
signal.signal(signal.SIGTERM, request_stop)
signal.signal(signal.SIGINT, request_stop)
forwarder.start()
print(json.dumps({"forwards": [asdict(s) for s in forwarder.bound]}), flush=True)
try:
forwarder.serve()
finally:
forwarder.stop()
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -56,13 +56,11 @@ def resolve_plan(
git_gate_plan: GitGatePlan, git_gate_plan: GitGatePlan,
stage_dir: Path, stage_dir: Path,
) -> SmolmachinesBottlePlan: ) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker """Materialize the smolmachines plan. The agent `.smolmachine`
subnet + pinned IP are derived from the slug; the agent's artifact is built (or cache-hit) here so launch's
`.smolmachine` artifact is built (or cache-hit) here so `machine create --from` boots without a registry pull. Per-bottle
launch's `machine create --from` boots without a registry guest env lands on the plan for launch to pass straight through
pull. Per-bottle guest env + the TSI allow_cidrs land on the to `machine create` flags."""
plan for launch to pass straight through to
`machine create` flags."""
# ==== smolmachines specific setup ==== # ==== smolmachines specific setup ====
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug) subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
+75 -153
View File
@@ -1,39 +1,24 @@
"""Per-bottle sidecar bundle bringup for the smolmachines backend """Per-bottle sidecar bundle bringup for the smolmachines backend.
(PRD 0023).
Two docker resources per bottle live here: The sidecar bundle runs as its own smolVM. The agent VM reaches
bundle daemons through host-loopback ports published by that sidecar
- **A dedicated bridge network**, subnet derived from the slug. VM and wrapped by per-bottle address-bound forwarders."""
The bundle container gets a pinned IP at `<subnet>.2` so the
smolvm guest's TSI allowlist (`<bundle-ip>/32`) has a stable
target. Without pinning, we'd have to inspect the container's
assigned IP after start and feed it back into the Smolfile
— a race we can sidestep with `--ip`.
- **The bundle container itself**, running the PRD 0024 bundle
image (`bot-bottle-sidecars:latest` by default). Same
image, same daemons, same daemon-private env / bind-mounts
as the docker backend.
This module ships the lifecycle primitives only — create
network, start bundle, stop bundle, remove network — wrapped
around `subprocess.run(["docker", ...])`. Wiring them into the
launch flow + populating the `BundleLaunchSpec` from the inner
Plans (EgressPlan, …) lands in chunk 2d."""
from __future__ import annotations from __future__ import annotations
import subprocess import os
import socket
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Sequence from typing import Sequence
from ...log import die, warn from ...log import warn
from ..docker import util as docker_mod from ..docker import util as docker_mod
from ..docker.sidecar_bundle import ( from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE, SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE, SIDECAR_BUNDLE_IMAGE,
) )
from . import smolvm as _smolvm
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent) _REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
@@ -54,6 +39,11 @@ def bundle_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}" return f"bot-bottle-sidecars-{slug}"
def bundle_machine_name(slug: str) -> str:
"""Sidecar smolVM machine name."""
return bundle_container_name(slug)
@dataclass(frozen=True) @dataclass(frozen=True)
class BundleLaunchSpec: class BundleLaunchSpec:
"""Everything `start_bundle` needs to bring up one bundle """Everything `start_bundle` needs to bring up one bundle
@@ -70,10 +60,8 @@ class BundleLaunchSpec:
# supervisor inside the bundle reads it to skip # supervisor inside the bundle reads it to skip
# bottle-irrelevant daemons (e.g. supervise=False bottles). # bottle-irrelevant daemons (e.g. supervise=False bottles).
daemons_csv: str = "egress" daemons_csv: str = "egress"
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name # Plain "KEY=VALUE" strings + "KEY" bare names. Bare names inherit
# form inherits the value from the docker-run subprocess env, # from the host env passed to the sidecar VM launch.
# matching the docker backend's compose-up secret-forwarding
# pattern).
environment: Sequence[str] = field(default_factory=tuple) environment: Sequence[str] = field(default_factory=tuple)
# (host_path, container_path, read_only) bind mounts. # (host_path, container_path, read_only) bind mounts.
volumes: Sequence[tuple[str, str, bool]] = field(default_factory=tuple) volumes: Sequence[tuple[str, str, bool]] = field(default_factory=tuple)
@@ -92,6 +80,11 @@ class BundleLaunchSpec:
publish_host_ip: str = "127.0.0.1" publish_host_ip: str = "127.0.0.1"
@dataclass(frozen=True)
class BundleVmLaunch:
raw_ports: dict[int, int]
def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None: def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None:
"""Build the sidecar bundle image before `docker run`. """Build the sidecar bundle image before `docker run`.
@@ -107,136 +100,65 @@ def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None:
) )
def create_bundle_network(network_name: str, subnet: str, gateway: str) -> None: def allocate_raw_host_ports(container_ports: Sequence[int]) -> dict[int, int]:
"""`docker network create` with an explicit subnet + gateway """Reserve candidate host loopback ports for smolVM `-p HOST:GUEST`.
so the bundle's `--ip` lands on the address the Smolfile's
TSI allowlist points at. Idempotent on the caller's side — The sockets are closed before smolVM binds them, so this remains
`start_bundle` catches the "network exists" error and treats best-effort. smolVM failing to bind a selected port is fatal in
it as success (chunk-2d teardown is paired with each create). the caller's launch path."""
""" out: dict[int, int] = {}
result = subprocess.run( for container_port in container_ports:
["docker", "network", "create", with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
"--subnet", subnet, "--gateway", gateway, sock.bind(("127.0.0.1", 0))
network_name], out[container_port] = int(sock.getsockname()[1])
capture_output=True, text=True, check=False, return out
)
if result.returncode != 0:
# Already-exists is fine on a resume path; everything else
# is fatal — the bundle won't have an addressable network.
if "already exists" in (result.stderr or "").lower():
return
die(
f"docker network create {network_name} failed: "
f"{(result.stderr or '').strip()}"
)
def remove_bundle_network(network_name: str) -> None: def start_bundle_vm(
"""Idempotent: a missing network returns success.""" spec: BundleLaunchSpec,
result = subprocess.run( *,
["docker", "network", "rm", network_name], from_path: Path,
capture_output=True, text=True, check=False, host_env: dict[str, str] | None = None,
) ) -> BundleVmLaunch:
if result.returncode == 0: """Create and start the sidecar bundle as a smolVM.
return
if "no such network" in (result.stderr or "").lower():
return
# Network with attached containers is the common non-fatal
# case during a partial teardown — warn but don't die.
warn(
f"docker network rm {network_name} failed: "
f"{(result.stderr or '').strip()}"
)
smolVM's own published ports are raw host-loopback ports. The
def start_bundle(spec: BundleLaunchSpec, *, launch flow wraps them with per-bottle address-bound forwarders
env: dict[str, str] | None = None) -> None: before exposing anything to the agent VM."""
"""Bring the bundle container up on the per-bottle bridge with raw_ports = allocate_raw_host_ports(spec.ports_to_publish)
the pinned IP. Argv is built deterministically from `spec`; effective_host_env = host_env if host_env is not None else os.environ
`env` is the host subprocess env (forwarded values for any env: dict[str, str] = {"BOT_BOTTLE_SIDECAR_DAEMONS": spec.daemons_csv}
bare-name entries in `spec.environment`)."""
container = bundle_container_name(spec.slug)
argv = [
"docker", "run",
"--name", container,
"--detach",
"--rm",
"--network", spec.network_name,
"--ip", spec.bundle_ip,
"-e", f"BOT_BOTTLE_SIDECAR_DAEMONS={spec.daemons_csv}",
]
for entry in spec.environment: for entry in spec.environment:
argv += ["-e", entry] name, sep, value = entry.partition("=")
for host_path, container_path, read_only in spec.volumes: if sep:
suffix = ":ro" if read_only else "" env[name] = value
argv += ["-v", f"{host_path}:{container_path}{suffix}"] elif entry in effective_host_env:
# Loopback-only host port-forwards — the smolvm guest's TSI env[entry] = effective_host_env[entry]
# uses macOS networking, and macOS loopback is the only host name = bundle_machine_name(spec.slug)
# surface that round-trips into Docker Desktop's daemon VM. _smolvm.machine_create(
# Binds to the per-bottle alias so TSI's IP-only allowlist name,
# narrows reachability to this bottle's bundle only. from_path=from_path,
for port in spec.ports_to_publish: net=True,
argv += ["-p", f"{spec.publish_host_ip}::{port}"] env=env,
argv.append(spec.image) volumes=spec.volumes,
result = subprocess.run( ports=tuple(
argv, capture_output=True, text=True, (host_port, guest_port)
env=dict(env) if env is not None else None, check=False, for guest_port, host_port in raw_ports.items()
),
) )
if result.returncode != 0: _smolvm.machine_start(name)
die( _smolvm.wait_exec_ready(name)
f"docker run for bundle {container} failed: " return BundleVmLaunch(raw_ports=raw_ports)
f"{(result.stderr or '').strip()}"
)
def bundle_host_port( def stop_bundle_vm(slug: str) -> None:
slug: str, container_port: int, *, host_ip: str = "127.0.0.1", """Best-effort sidecar VM teardown."""
) -> int: name = bundle_machine_name(slug)
"""`docker port <bundle> <container_port>/tcp` → the random try:
host-side port docker assigned for the binding on `host_ip`. _smolvm.machine_stop(name)
Called after `start_bundle` on each container port listed in except _smolvm.SmolvmError as exc:
`BundleLaunchSpec.ports_to_publish` so the launch step can warn(f"smolvm machine stop {name} failed: {exc}")
build the agent's HTTPS_PROXY / GIT_GATE / SUPERVISE URLs in try:
`<host_ip>:<host port>` form.""" _smolvm.machine_delete(name)
container = bundle_container_name(slug) except _smolvm.SmolvmError as exc:
result = subprocess.run( warn(f"smolvm machine delete {name} failed: {exc}")
["docker", "port", container, f"{container_port}/tcp"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(
f"docker port {container} {container_port}/tcp failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
# Each line looks like `127.0.0.16:54321` — one per address
# family / host IP. Match on the expected host_ip prefix so
# bottles bound to per-bottle aliases pick the right line.
for raw in (result.stdout or "").splitlines():
line = raw.strip()
if line.startswith(f"{host_ip}:"):
_, _, port_str = line.rpartition(":")
try:
return int(port_str)
except ValueError:
die(f"unexpected `docker port` output: {line!r}")
die(
f"no port mapping on {host_ip} for {container} "
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
)
def stop_bundle(slug: str) -> None:
"""Idempotent: a missing container returns success."""
container = bundle_container_name(slug)
result = subprocess.run(
["docker", "rm", "-f", container],
capture_output=True, text=True, check=False,
)
if result.returncode == 0:
return
if "no such container" in (result.stderr or "").lower():
return
warn(
f"docker rm -f {container} failed: "
f"{(result.stderr or '').strip()}"
)
+20 -17
View File
@@ -113,13 +113,15 @@ def machine_create(
*, *,
image: str | None = None, image: str | None = None,
from_path: Path | None = None, from_path: Path | None = None,
net: bool = False,
allow_cidrs: Sequence[str] = (), allow_cidrs: Sequence[str] = (),
env: Mapping[str, str] | None = None, env: Mapping[str, str] | None = None,
ports: Sequence[tuple[int, int]] = (),
volumes: Sequence[tuple[str, str, bool]] = (),
) -> None: ) -> None:
"""`smolvm machine create NAME [--image IMG | --from PATH] """`smolvm machine create --name NAME [--image IMG | --from PATH]
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is positional [--allow-cidr CIDR ...] [-e K=V ...]`. NAME is passed as
(the CLI's exception to the `--name` pattern other `--name` (smolvm 1.4.7+; earlier versions took it positionally).
subcommands use).
`image` (registry ref like `alpine:latest`) and `from_path` `image` (registry ref like `alpine:latest`) and `from_path`
(a `.smolmachine` artifact) are mutually exclusive — one or (a `.smolmachine` artifact) are mutually exclusive — one or
@@ -132,25 +134,27 @@ def machine_create(
no-pull-at-start property. The flag form gives the same no-pull-at-start property. The flag form gives the same
result without the Smolfile complication. result without the Smolfile complication.
`--net` is sent explicitly when `allow_cidrs` is non-empty. `--net` is sent explicitly when `net=True` or `allow_cidrs` is
smolvm 0.8.0's docs say `--allow-cidr` implies `--net`, but non-empty. `--allow-cidr` implies `--net` per the CLI help, but
empirically the implication only fires when no `--from` is sending `--net` explicitly is harmless and ensures the guest has
set — `--from PATH --allow-cidr X/32` silently produces a network access even if that implication changes across versions."""
machine with `network: false` and no routes in the guest, so args: list[str] = ["machine", "create", "--name", name]
the agent can't reach the bundle's pinned IP."""
args: list[str] = ["machine", "create"]
if image is not None: if image is not None:
args += ["--image", image] args += ["--image", image]
if from_path is not None: if from_path is not None:
args += ["--from", str(from_path)] args += ["--from", str(from_path)]
if allow_cidrs: if net or allow_cidrs:
args.append("--net") args.append("--net")
for cidr in allow_cidrs: for cidr in allow_cidrs:
args += ["--allow-cidr", cidr] args += ["--allow-cidr", cidr]
for host_port, guest_port in ports:
args += ["-p", f"{host_port}:{guest_port}"]
for host_path, guest_path, read_only in volumes:
suffix = ":ro" if read_only else ""
args += ["-v", f"{host_path}:{guest_path}{suffix}"]
if env: if env:
for k, v in env.items(): for k, v in env.items():
args += ["-e", f"{k}={v}"] args += ["-e", f"{k}={v}"]
args.append(name)
_smolvm(*args) _smolvm(*args)
@@ -182,10 +186,9 @@ def machine_stop(name: str) -> None:
def machine_delete(name: str) -> None: def machine_delete(name: str) -> None:
"""`smolvm machine delete -f NAME`. NAME is positional. `-f` """`smolvm machine delete --name NAME -f`. `-f` skips the
skips the interactive confirmation — required for interactive confirmation — required for non-interactive teardown."""
non-interactive teardown.""" _smolvm("machine", "delete", "--name", name, "-f")
_smolvm("machine", "delete", "-f", name)
def machine_exec( def machine_exec(
+46 -14
View File
@@ -5,26 +5,58 @@ unit-tested without importing the docker subprocess paths."""
from __future__ import annotations from __future__ import annotations
import hashlib import hashlib
import os
import platform
import shutil import shutil
from ...log import die from ...log import die
# libkrun's Linux backend drives the guest through KVM, so the host
# must expose `/dev/kvm` and the invoking user must be able to open
# it. macOS uses Hypervisor.framework and needs no device node.
_KVM_DEVICE = "/dev/kvm"
def smolmachines_preflight() -> None: def smolmachines_preflight() -> None:
"""Ensure `smolvm` is on PATH before the launch flow runs. """Ensure the host can run the smolmachines backend before the
Called from `_resolve_plan`; gives the operator a clear launch flow starts. Called from `_resolve_plan`; surfaces a
install pointer rather than a cryptic FileNotFoundError clear, actionable error instead of a cryptic `smolvm` failure
later. `gvproxy` is no longer required — see the PRD's design deep in launch.
pivot section."""
if shutil.which("smolvm") is not None: Checks `smolvm` is on PATH (both platforms) and, on Linux,
return that `/dev/kvm` exists and is accessible. `gvproxy` is no
die( longer required — see the PRD's design pivot section."""
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on " if shutil.which("smolvm") is None:
"PATH. Install with: " die(
"curl -sSL https://smolmachines.com/install.sh | sh. " "BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
"To use the legacy Docker backend instead, set " "PATH. Install with: "
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker." "curl -sSL https://smolmachines.com/install.sh | sh. "
) "To use the legacy Docker backend instead, set "
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
)
if platform.system() == "Linux":
_preflight_kvm()
def _preflight_kvm() -> None:
"""Linux-only: libkrun needs `/dev/kvm`. Distinguish 'KVM not
enabled' from 'no permission' so the operator knows which to
fix."""
if not os.path.exists(_KVM_DEVICE):
die(
f"BOT_BOTTLE_BACKEND=smolmachines needs {_KVM_DEVICE} on "
"Linux but it is missing. Enable KVM: load the kvm-intel "
"or kvm-amd kernel module (and confirm virtualization is "
"enabled in BIOS/firmware). To use the legacy Docker "
"backend instead, set BOT_BOTTLE_BACKEND=docker."
)
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
die(
f"{_KVM_DEVICE} exists but is not readable/writable by the "
"current user. Add your user to the `kvm` group "
"(`sudo usermod -aG kvm \"$USER\"`) and re-login, or run "
"with access to the device."
)
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]: def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
+2 -3
View File
@@ -284,9 +284,8 @@ def git_gate_state_dir(identity: str) -> Path:
def supervise_state_dir(identity: str) -> Path: def supervise_state_dir(identity: str) -> Path:
"""State subdir reserved for supervise sidecar bind-mount sources. """State subdir reserved for supervise sidecar bind-mount sources.
The queue dir is intentionally NOT under here — it lives at Runtime queue/audit rows live in the host-level bot-bottle SQLite
~/.bot-bottle/queue/<slug>/ alongside the audit logs, so it database, so they survive state-dir cleanup."""
survives state-dir cleanup."""
return bottle_state_dir(identity) / _SUPERVISE_SUBDIR return bottle_state_dir(identity) / _SUPERVISE_SUBDIR
+18 -2
View File
@@ -7,8 +7,10 @@ from __future__ import annotations
import sys import sys
from ..errors import MissingEnvVarError
from ..log import Die, die, error from ..log import Die, die, error
from ..manifest import ManifestError from ..manifest import ManifestError
from ..store_manager import StoreManager
from ._common import PROG from ._common import PROG
from . import list as _list_mod from . import list as _list_mod
from .cleanup import cmd_cleanup from .cleanup import cmd_cleanup
@@ -74,11 +76,25 @@ def main(argv: list[str] | None = None) -> int:
if handler is None: if handler is None:
usage() usage()
die(f"unknown command: {command}") die(f"unknown command: {command}")
mgr = StoreManager.instance()
if not mgr.is_migrated():
sys.stderr.write("bot-bottle: database schema is out of date\n")
sys.stderr.write("Migrate now? [y/N] ")
sys.stderr.flush()
try:
answer = sys.stdin.readline().strip().lower()
except EOFError:
answer = ""
if answer != "y":
error("migration required — re-run and confirm to migrate")
return 1
mgr.migrate()
try: try:
return handler(rest) or 0 return handler(rest) or 0
except MissingEnvVarError as e:
error(str(e))
return 1
except ManifestError as e: except ManifestError as e:
# Manifest/config problems surface as a catchable exception;
# print the reason and exit non-zero (same UX die() used to give).
error(str(e)) error(str(e))
return 1 return 1
except Die as e: except Die as e:
+12 -3
View File
@@ -108,6 +108,13 @@ def cmd_start(argv: list[str]) -> int:
agent_name: str | None = args.name agent_name: str | None = args.name
if agent_name is None: if agent_name is None:
if not manifest.all_agent_names:
print(
"bot-bottle: no agents defined. "
"Add an agent to ~/.bot-bottle/agents/ or ./bot-bottle/agents/ to get started.",
file=sys.stderr,
)
return 1
agent_name = tui.filter_select( agent_name = tui.filter_select(
manifest.all_agent_names, manifest.all_agent_names,
title="Select agent", title="Select agent",
@@ -202,6 +209,7 @@ def _start_headless(
label=label, label=label,
color=args.color or "", color=args.color or "",
bottle_names=bottle_names, bottle_names=bottle_names,
headless=True,
) )
return _launch_bottle( return _launch_bottle(
spec, spec,
@@ -235,7 +243,7 @@ def prepare_with_preflight(
spec: BottleSpec, spec: BottleSpec,
*, *,
stage_dir: Path, stage_dir: Path,
render_preflight: Callable[[DockerBottlePlan], None], render_preflight: Callable[[DockerBottlePlan, str], None],
prompt_yes: Callable[[], bool], prompt_yes: Callable[[], bool],
dry_run: bool = False, dry_run: bool = False,
backend_name: str | None = None, backend_name: str | None = None,
@@ -256,7 +264,7 @@ def prepare_with_preflight(
plan = backend.prepare(spec, stage_dir=stage_dir) plan = backend.prepare(spec, stage_dir=stage_dir)
identity = _identity_from_plan(plan) identity = _identity_from_plan(plan)
render_preflight(plan) render_preflight(plan, backend.name)
if dry_run: if dry_run:
info("dry-run requested; not starting container.") info("dry-run requested; not starting container.")
@@ -383,8 +391,9 @@ def _text_prompt_yes() -> bool:
def _text_render_preflight(): def _text_render_preflight():
def _render(plan: DockerBottlePlan) -> None: def _render(plan: DockerBottlePlan, backend_name: str) -> None:
print(file=sys.stderr) print(file=sys.stderr)
print(f"backend: {backend_name}", file=sys.stderr)
print(_manifest_to_yaml(plan.manifest), file=sys.stderr) print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
return _render return _render
+9 -16
View File
@@ -45,7 +45,7 @@ from ..supervise import (
TOOL_EGRESS_BLOCK, TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW, TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW, TOOL_EGRESS_TOKEN_ALLOW,
list_pending_proposals, list_all_pending_proposals,
render_diff, render_diff,
write_audit_entry, write_audit_entry,
write_response, write_response,
@@ -63,10 +63,9 @@ _REPORT_ONLY_TOOLS: tuple[str, ...] = (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_AL
@dataclass(frozen=True) @dataclass(frozen=True)
class QueuedProposal: class QueuedProposal:
"""A pending proposal plus the queue dir it was found in.""" """A pending proposal from the supervise queue."""
proposal: Proposal proposal: Proposal
queue_dir: Path
# Errors any remediation engine may raise. Caught by the TUI key # Errors any remediation engine may raise. Caught by the TUI key
@@ -86,16 +85,11 @@ def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
def discover_pending() -> list[QueuedProposal]: def discover_pending() -> list[QueuedProposal]:
"""Walk ~/.bot-bottle/queue/* and collect pending proposals.""" """Collect pending proposals across bottles."""
queue_root = _supervise.bot_bottle_root() / "queue" out = [
if not queue_root.is_dir(): QueuedProposal(proposal=proposal)
return [] for proposal in list_all_pending_proposals()
out: list[QueuedProposal] = [] ]
for slug_dir in sorted(queue_root.iterdir()):
if not slug_dir.is_dir():
continue
for proposal in list_pending_proposals(slug_dir):
out.append(QueuedProposal(proposal=proposal, queue_dir=slug_dir))
out.sort(key=lambda q: q.proposal.arrival_timestamp) out.sort(key=lambda q: q.proposal.arrival_timestamp)
return out return out
@@ -118,7 +112,6 @@ def _detail_lines(
(f"tool: {p.tool}", 0), (f"tool: {p.tool}", 0),
(f"id: {p.id}", 0), (f"id: {p.id}", 0),
(f"arrived: {p.arrival_timestamp}", 0), (f"arrived: {p.arrival_timestamp}", 0),
(f"queue: {qp.queue_dir}", 0),
("", 0), ("", 0),
("justification:", 0), ("justification:", 0),
] ]
@@ -165,7 +158,7 @@ def approve(
notes=notes, notes=notes,
final_file=final_file, final_file=final_file,
) )
write_response(qp.queue_dir, response) write_response(qp.proposal.bottle_slug, response)
_write_audit( _write_audit(
qp, action=status, notes=notes, qp, action=status, notes=notes,
diff_before=diff_before, diff_after=diff_after, diff_before=diff_before, diff_after=diff_after,
@@ -179,7 +172,7 @@ def reject(qp: QueuedProposal, *, reason: str) -> None:
notes=reason, notes=reason,
final_file=None, final_file=None,
) )
write_response(qp.queue_dir, response) write_response(qp.proposal.bottle_slug, response)
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="") _write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
+1 -1
View File
@@ -21,7 +21,7 @@ FROM node:22-slim
# to it) works against egress's bumped TLS without the agent needing # to it) works against egress's bumped TLS without the agent needing
# local DNS. # local DNS.
RUN apt-get update \ RUN apt-get update \
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \ && apt-get install -y --no-install-recommends git ca-certificates curl ripgrep iproute2 dnsutils \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# App-specific deps. Python isn't required by claude-code itself # App-specific deps. Python isn't required by claude-code itself
+10 -3
View File
@@ -34,6 +34,12 @@ if TYPE_CHECKING:
_SUPERVISE_MCP_NAME = "supervise" _SUPERVISE_MCP_NAME = "supervise"
_CODEX_CLI = "/home/node/.codex/packages/standalone/current/bin/codex"
_CODEX_CLI_PATH = (
"/home/node/.local/bin:"
"/home/node/.codex/packages/standalone/current/bin:"
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
)
def _skills_dir(guest_home: str) -> str: def _skills_dir(guest_home: str) -> str:
@@ -50,7 +56,7 @@ def _prompt_path(guest_home: str) -> str:
_RUNTIME = AgentProviderRuntime( _RUNTIME = AgentProviderRuntime(
template="codex", template="codex",
command="codex", command=_CODEX_CLI,
image="bot-bottle-codex:latest", image="bot-bottle-codex:latest",
prompt_mode="read_prompt_file", prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",), bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
@@ -145,7 +151,8 @@ class CodexAgentProvider(AgentProvider):
"env", "env",
f"HOME={guest_home}", f"HOME={guest_home}",
f"CODEX_HOME={auth_dir}", f"CODEX_HOME={auth_dir}",
"codex", "login", "status", f"PATH={_CODEX_CLI_PATH}",
_CODEX_CLI, "login", "status",
), ( ), (
"codex host credentials: dummy auth was copied into the " "codex host credentials: dummy auth was copied into the "
"guest, but Codex did not accept it" "guest, but Codex did not accept it"
@@ -267,7 +274,7 @@ class CodexAgentProvider(AgentProvider):
return return
info(f"registering supervise MCP server in agent codex config → {supervise_url}") info(f"registering supervise MCP server in agent codex config → {supervise_url}")
r = bottle.exec( r = bottle.exec(
f"codex mcp add {_SUPERVISE_MCP_NAME} --url " f"{shlex.quote(_CODEX_CLI)} mcp add {_SUPERVISE_MCP_NAME} --url "
f"{shlex.quote(supervise_url)}", f"{shlex.quote(supervise_url)}",
user="node", user="node",
) )
+59
View File
@@ -0,0 +1,59 @@
"""Shared SQLite-backed store base class for bot-bottle (PRD 0013)."""
from __future__ import annotations
import sqlite3
from pathlib import Path
try:
from .migrations import TableMigrations
except ImportError:
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
class DbVersionError(Exception):
"""Raised when the on-disk schema is behind the current migration list."""
class DbStore:
"""Base for SQLite-backed stores. Subclasses resolve db_path then call super().__init__."""
def __init__(self, db_path: Path, migrations: TableMigrations) -> None:
self.db_path = db_path
self._migrations = migrations
self.db_path.parent.mkdir(parents=True, exist_ok=True)
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.db_path)
conn.row_factory = sqlite3.Row
return conn
def is_migrated(self) -> bool:
"""Return True if the DB is fully up-to-date, False if migration is needed."""
if not self.db_path.exists():
return False
try:
with self._connect() as conn:
row = conn.execute(
"SELECT version FROM schema_versions WHERE module = ?",
(self._migrations.schema_key,),
).fetchone()
except sqlite3.OperationalError:
return False
version = row[0] if row else 0
return version == len(self._migrations.migrations)
def migrate(self) -> None:
"""Apply any pending migrations and set permissions on the DB file."""
with self._connect() as conn:
self._migrations.apply(conn)
self._chmod()
def _chmod(self) -> None:
try:
self.db_path.chmod(0o600)
except OSError:
pass
__all__ = ["DbStore", "DbVersionError"]
+7 -4
View File
@@ -23,6 +23,7 @@ from .egress_addon_core import (
PathMatch as CorePathMatch, PathMatch as CorePathMatch,
Route, Route,
) )
from .errors import MissingEnvVarError
from .log import die from .log import die
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -354,16 +355,18 @@ def egress_resolve_token_values(
for token_env, token_ref in token_env_map.items(): for token_env, token_ref in token_env_map.items():
value = host_env.get(token_ref) value = host_env.get(token_ref)
if value is None: if value is None:
die( raise MissingEnvVarError(
token_ref,
f"egress: host env var '{token_ref}' is unset. Set it " f"egress: host env var '{token_ref}' is unset. Set it "
f"before launching, or remove the corresponding auth block " f"before launching, or remove the corresponding auth block "
f"from bottle.egress.routes." f"from bottle.egress.routes.",
) )
if not value: if not value:
die( raise MissingEnvVarError(
token_ref,
f"egress: host env var '{token_ref}' is empty. The " f"egress: host env var '{token_ref}' is empty. The "
f"egress will not inject an empty token; set it to " f"egress will not inject an empty token; set it to "
f"the real value or remove the route's auth block." f"the real value or remove the route's auth block.",
) )
out[token_env] = value out[token_env] = value
return out return out
+6 -9
View File
@@ -79,14 +79,13 @@ class EgressAddon:
# only — a restart re-prompts. Mutated only from the asyncio loop that # only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed. # runs the addon hooks, so no lock is needed.
self.safe_tokens: set[str] = set() self.safe_tokens: set[str] = set()
self._supervise_queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "").strip()
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip() self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ) self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True) self._reload(initial=True)
self._install_sighup() self._install_sighup()
def _supervise_available(self) -> bool: def _supervise_available(self) -> bool:
return bool(self._supervise_queue_dir and self._supervise_slug) return bool(self._supervise_slug)
def _reload(self, *, initial: bool = False) -> None: def _reload(self, *, initial: bool = False) -> None:
try: try:
@@ -393,9 +392,8 @@ class EgressAddon:
justification=_TOKEN_ALLOW_JUSTIFICATION, justification=_TOKEN_ALLOW_JUSTIFICATION,
current_file_hash=_sv.sha256_hex(payload), current_file_hash=_sv.sha256_hex(payload),
) )
queue_dir = Path(self._supervise_queue_dir)
try: try:
_sv.write_proposal(queue_dir, proposal) _sv.write_proposal(proposal)
except OSError as e: except OSError as e:
sys.stderr.write( sys.stderr.write(
f"egress: could not queue token-allow proposal: {e}; " f"egress: could not queue token-allow proposal: {e}; "
@@ -411,8 +409,8 @@ class EgressAddon:
**self._req_ctx(flow), **self._req_ctx(flow),
}) + "\n") }) + "\n")
response = await self._await_token_response(queue_dir, proposal.id) response = await self._await_token_response(proposal.id)
_sv.archive_proposal(queue_dir, proposal.id) _sv.archive_proposal(self._supervise_slug, proposal.id)
if response is not None and response.status in ( if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED, _sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
@@ -439,16 +437,15 @@ class EgressAddon:
async def _await_token_response( async def _await_token_response(
self, self,
queue_dir: Path,
proposal_id: str, proposal_id: str,
) -> "_sv.Response | None": ) -> "_sv.Response | None":
"""Poll the queue dir for the operator's response without blocking the """Poll the DB for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout.""" proxy event loop. Returns the Response, or None on timeout."""
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
deadline = loop.time() + self._token_allow_timeout deadline = loop.time() + self._token_allow_timeout
while True: while True:
try: try:
return _sv.read_response(queue_dir, proposal_id) return _sv.read_response(self._supervise_slug, proposal_id)
except (OSError, ValueError, KeyError): except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until # Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed. # the deadline, then fail closed.
+4 -2
View File
@@ -35,6 +35,7 @@ import re
import sys import sys
from dataclasses import dataclass, field from dataclasses import dataclass, field
from .errors import MissingEnvVarError
from .log import die from .log import die
from .manifest import Manifest from .manifest import Manifest
@@ -136,9 +137,10 @@ def resolve_env(manifest: Manifest) -> ResolvedEnv:
host_var = env_entry_interpolated_from(raw) host_var = env_entry_interpolated_from(raw)
host_value = os.environ.get(host_var, "") host_value = os.environ.get(host_var, "")
if not host_value: if not host_value:
die( raise MissingEnvVarError(
host_var,
f"env entry {name} is interpolated from ${host_var}, " f"env entry {name} is interpolated from ${host_var}, "
f"but ${host_var} is unset or empty in the host environment." f"but ${host_var} is unset or empty in the host environment.",
) )
forwarded[name] = host_value forwarded[name] = host_value
else: # literal else: # literal
+18
View File
@@ -0,0 +1,18 @@
"""bot-bottle application-level exception hierarchy.
Exceptions here are caught by the CLI dispatcher (``cli/__init__.py``)
and rendered as ``bot-bottle: error: `` lines same UX as ``die()``,
but without coupling library code to stderr output."""
from __future__ import annotations
class MissingEnvVarError(Exception):
"""Raised when a required host environment variable is unset or empty.
``var_name`` is the exact name of the missing variable so callers
can display it without re-parsing the message string."""
def __init__(self, var_name: str, message: str) -> None:
self.var_name = var_name
super().__init__(message)
+7 -11
View File
@@ -30,7 +30,6 @@ backend-specific and lives on concrete subclasses (see
from __future__ import annotations from __future__ import annotations
import dataclasses
from abc import ABC from abc import ABC
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
@@ -53,6 +52,7 @@ from .git_gate_render import (
_gitconfig_validate_value, _gitconfig_validate_value,
) )
from .git_gate_provision import ( from .git_gate_provision import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys, revoke_git_gate_provisioned_keys,
_provision_dynamic_key, _provision_dynamic_key,
_resolve_identity_file, _resolve_identity_file,
@@ -82,6 +82,7 @@ class GitGatePlan:
class GitGate(ABC): class GitGate(ABC):
"""The per-agent git-gate. Encapsulates the host-side prepare """The per-agent git-gate. Encapsulates the host-side prepare
(upstream lift + entrypoint/hook render); the sidecar's (upstream lift + entrypoint/hook render); the sidecar's
@@ -93,20 +94,14 @@ class GitGate(ABC):
entrypoint, pre-receive hook, and access-hook scripts (mode entrypoint, pre-receive hook, and access-hook scripts (mode
600) under `stage_dir`. Pure host-side, no docker subprocess. 600) under `stage_dir`. Pure host-side, no docker subprocess.
For `gitea` key entries, also generates and registers For `gitea` key entries, the returned upstream intentionally
a fresh deploy key via the forge API and writes the private key has an empty identity file. Backend launch fills that in after
+ key ID to `stage_dir`. the operator confirms the preflight.
Returned plan is incomplete: the launch step must fill Returned plan is incomplete: the launch step must fill
`internal_network` / `egress_network` via `dataclasses.replace` `internal_network` / `egress_network` via `dataclasses.replace`
before passing the plan to `.start`.""" before passing the plan to `.start`."""
upstreams_list = list(git_gate_upstreams_for_bottle(bottle)) upstreams = git_gate_upstreams_for_bottle(bottle)
for i, entry in enumerate(bottle.git):
upstreams_list[i] = dataclasses.replace(
upstreams_list[i],
identity_file=_resolve_identity_file(entry, slug, stage_dir),
)
upstreams = tuple(upstreams_list)
entrypoint = stage_dir / "git_gate_entrypoint.sh" entrypoint = stage_dir / "git_gate_entrypoint.sh"
entrypoint.write_text(git_gate_render_entrypoint(upstreams)) entrypoint.write_text(git_gate_render_entrypoint(upstreams))
entrypoint.chmod(0o600) entrypoint.chmod(0o600)
@@ -162,6 +157,7 @@ __all__ = [
"git_gate_render_entrypoint", "git_gate_render_entrypoint",
"git_gate_render_hook", "git_gate_render_hook",
"git_gate_render_access_hook", "git_gate_render_access_hook",
"provision_git_gate_dynamic_keys",
"revoke_git_gate_provisioned_keys", "revoke_git_gate_provisioned_keys",
"_gitconfig_validate_value", "_gitconfig_validate_value",
"_provision_dynamic_key", "_provision_dynamic_key",
+268
View File
@@ -0,0 +1,268 @@
"""Preflight host-key population for git-gate upstreams (issue #333).
When a git-gate repo entry lacks a `host_key`, this module either:
- headless: dies with a clear config error.
- interactive: fetches the key via ssh-keyscan, prompts the operator to
confirm, and optionally persists it to the bottle config file on disk.
Public entry point: `preflight_host_keys(manifest, headless=..., home_md=...)`.
"""
from __future__ import annotations
import dataclasses
import subprocess
import sys
from pathlib import Path
from typing import cast
from .log import die, info
from .manifest import Manifest
from .yaml_subset import YamlSubsetError, parse_frontmatter, serialize_yaml_subset
# Preferred key types, most secure first.
_KEY_TYPE_PREFERENCE = (
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"ssh-rsa",
)
def fetch_host_key(host: str, port: str) -> str:
"""Return an SSH public key for `host`:`port` via ssh-keyscan.
Returns the key in `<type> <base64-data>` format (the host prefix is
stripped so the result can be stored in `host_key` and later formatted
into a known_hosts line by `git_gate_known_hosts_line`).
Prefers ed25519 > ecdsa > rsa; falls back to the first key type
returned if none of the preferred types are present.
Raises `RuntimeError` on subprocess failure, timeout, or no result.
Uses only the Python stdlib (subprocess)."""
args = ["ssh-keyscan"]
if port and port != "22":
args += ["-p", port]
args.append(host)
try:
result = subprocess.run(
args, capture_output=True, text=True, timeout=15, check=False,
)
except OSError as e:
raise RuntimeError(
f"ssh-keyscan: could not launch for {host}:{port}: {e}"
) from e
except subprocess.TimeoutExpired as e:
raise RuntimeError(f"ssh-keyscan timed out for {host}:{port}") from e
# known_hosts format: "[host]:port type data" or "host type data"
# Strip the host/port prefix; collect "type -> type data" by type.
found: dict[str, str] = {}
for line in result.stdout.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
parts = line.split(None, 2)
if len(parts) == 3 and parts[1] not in found:
found[parts[1]] = f"{parts[1]} {parts[2]}"
for preferred in _KEY_TYPE_PREFERENCE:
if preferred in found:
return found[preferred]
if found:
return next(iter(found.values()))
raise RuntimeError(
f"ssh-keyscan returned no host key for {host}:{port}."
+ (f" stderr: {result.stderr.strip()!r}" if result.stderr.strip() else "")
)
# ---------------------------------------------------------------------------
# Frontmatter editing
# ---------------------------------------------------------------------------
def add_host_key_to_frontmatter(file_text: str, repo_name: str, host_key: str) -> str:
"""Return an updated copy of `file_text` with `host_key` set on the
named repo entry in the YAML frontmatter.
Parses the frontmatter into a dict, sets the key, and re-serializes.
Returns the original text unchanged when: the file has no frontmatter,
the git-gate.repos.<repo_name> entry is absent or already has a
host_key, or the frontmatter cannot be parsed."""
try:
fm, body = parse_frontmatter(file_text)
except YamlSubsetError:
return file_text
git_gate = fm.get("git-gate")
if not isinstance(git_gate, dict):
return file_text
repos = git_gate.get("repos")
if not isinstance(repos, dict):
return file_text
repo = repos.get(repo_name)
if not isinstance(repo, dict):
return file_text
if repo.get("host_key"):
return file_text
cast(dict[str, object], repo)["host_key"] = host_key
return f"---\n{serialize_yaml_subset(fm)}---\n{body}"
def find_repo_bottle_file(bottles_dir: Path, repo_name: str) -> Path | None:
"""Return the first `bottles_dir/*.md` that declares `repo_name` without
a `host_key`, without modifying anything. Returns None if not found."""
if not bottles_dir.is_dir():
return None
for path in sorted(bottles_dir.glob("*.md")):
try:
text = path.read_text(encoding="utf-8")
fm, _ = parse_frontmatter(text)
except (OSError, UnicodeDecodeError, YamlSubsetError):
continue
git_gate = fm.get("git-gate")
if not isinstance(git_gate, dict):
continue
repos = git_gate.get("repos")
if not isinstance(repos, dict):
continue
repo = repos.get(repo_name)
if not isinstance(repo, dict) or repo.get("host_key"):
continue
return path
return None
def find_and_update_bottle_file(
bottles_dir: Path, repo_name: str, host_key: str,
) -> bool:
"""Write `host_key` into the bottle file returned by `find_repo_bottle_file`.
Returns True on success, False when no suitable file is found or the
write fails."""
path = find_repo_bottle_file(bottles_dir, repo_name)
if path is None:
return False
try:
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
return False
updated = add_host_key_to_frontmatter(text, repo_name, host_key)
if updated == text:
return False
path.write_text(updated, encoding="utf-8")
info(f"wrote host_key for {repo_name!r} to {path}")
return True
# ---------------------------------------------------------------------------
# Interactive prompt helper
# ---------------------------------------------------------------------------
def prompt_tty(message: str) -> str:
"""Write `message` to stderr and read a line from /dev/tty (or stdin)."""
sys.stderr.write(message)
sys.stderr.flush()
try:
with open("/dev/tty", "r", encoding="utf-8") as tty:
return tty.readline().rstrip("\n")
except OSError:
return sys.stdin.readline().rstrip("\n")
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def preflight_host_keys(
manifest: Manifest,
*,
headless: bool,
home_md: Path | None,
) -> Manifest:
"""Ensure every git-gate repo entry has a `host_key` configured.
For entries whose `KnownHostKey` is empty:
- headless: calls `die()` with a clear message naming the repos.
- interactive: fetches the key via ssh-keyscan, shows it to the
operator, and requests confirmation. If accepted, optionally
persists it to the bottle config file on disk; the key is always
applied in memory for this launch regardless of the persistence
choice. Aborted confirmation calls `die()`.
Returns a (possibly updated) Manifest. If all entries already have
host keys the original manifest is returned unchanged."""
bottle = manifest.bottle
missing = [e for e in bottle.git if not e.KnownHostKey]
if not missing:
return manifest
if headless:
names = ", ".join(repr(e.Name) for e in missing)
die(
f"git-gate: no host_key configured for repo(s) {names}. "
f"Add host_key to each bottle git-gate.repos entry, or run "
f"interactively once to have it fetched and saved automatically."
)
bottles_dir = (home_md / "bottles") if home_md is not None else None
updated_entries = list(bottle.git)
for entry in missing:
host = entry.UpstreamHost
port = entry.UpstreamPort
label = f"git-gate.repos[{entry.Name!r}]"
info(f"{label}: no host_key configured; fetching from {host}:{port}")
try:
key = fetch_host_key(host, port)
except RuntimeError as e:
die(f"git-gate: {label}: {e}")
sys.stderr.write(f"\ngit-gate: host key for {label}:\n {key}\n\n")
confirm = prompt_tty("Is this host key correct? [y/N] ")
if confirm.strip().lower() not in ("y", "yes"):
die(f"git-gate: {label}: host key not confirmed; aborting launch")
if bottles_dir is not None:
target_file = find_repo_bottle_file(bottles_dir, entry.Name)
if target_file is not None:
save = prompt_tty(
f"Save host_key for {entry.Name!r} to {target_file}? [y/N] "
)
if save.strip().lower() in ("y", "yes"):
ok = find_and_update_bottle_file(bottles_dir, entry.Name, key)
if not ok:
sys.stderr.write(
f"git-gate: {label}: could not write to {target_file}; "
f"host_key kept in memory for this session only\n"
)
else:
sys.stderr.write(
f"git-gate: {label}: no bottle config file found for "
f"{entry.Name!r}; host_key kept in memory for this session only\n"
)
idx = next(i for i, e in enumerate(updated_entries) if e.Name == entry.Name)
updated_entries[idx] = dataclasses.replace(entry, KnownHostKey=key)
updated_bottle = dataclasses.replace(bottle, git=tuple(updated_entries))
return dataclasses.replace(manifest, bottle=updated_bottle)
__all__ = [
"fetch_host_key",
"preflight_host_keys",
"add_host_key_to_frontmatter",
"find_repo_bottle_file",
"find_and_update_bottle_file",
"prompt_tty",
]
+50 -4
View File
@@ -9,10 +9,17 @@ imported (`deploy_key_provisioner`) to keep its cost off the host path.
from __future__ import annotations from __future__ import annotations
import os import os
import dataclasses
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING
from .errors import MissingEnvVarError
from .log import info from .log import info
from .manifest import ManifestBottle, ManifestGitEntry from .manifest import ManifestBottle, ManifestGitEntry
from .git_gate_render import GitGateUpstream
if TYPE_CHECKING:
from .git_gate import GitGatePlan
def _provision_dynamic_key( def _provision_dynamic_key(
entry: ManifestGitEntry, entry: ManifestGitEntry,
@@ -28,9 +35,10 @@ def _provision_dynamic_key(
pk = entry.Key pk = entry.Key
token = os.environ.get(pk.forge_token_env) token = os.environ.get(pk.forge_token_env)
if token is None: if token is None:
raise RuntimeError( raise MissingEnvVarError(
pk.forge_token_env,
f"git-gate.repos[{entry.Name!r}] key.forge_token_env" f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set" f" = {pk.forge_token_env!r}: env var is not set",
) )
api_url = pk.api_url or f"https://{entry.UpstreamHost}" api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url) provisioner = get_provisioner(pk.provider, token, api_url)
@@ -72,10 +80,11 @@ def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) ->
key_id = id_file.read_text().strip() key_id = id_file.read_text().strip()
token = os.environ.get(pk.forge_token_env) token = os.environ.get(pk.forge_token_env)
if token is None: if token is None:
raise RuntimeError( raise MissingEnvVarError(
pk.forge_token_env,
f"git-gate.repos[{entry.Name!r}] key.forge_token_env" f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
f" = {pk.forge_token_env!r}: env var is not set;" f" = {pk.forge_token_env!r}: env var is not set;"
f" cannot revoke deploy key {key_id}" f" cannot revoke deploy key {key_id}",
) )
api_url = pk.api_url or f"https://{entry.UpstreamHost}" api_url = pk.api_url or f"https://{entry.UpstreamHost}"
provisioner = get_provisioner(pk.provider, token, api_url) provisioner = get_provisioner(pk.provider, token, api_url)
@@ -95,8 +104,45 @@ def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path)
return entry.IdentityFile return entry.IdentityFile
def provision_git_gate_dynamic_keys(
bottle: ManifestBottle,
plan: "GitGatePlan",
stage_dir: Path,
) -> "GitGatePlan":
"""Provision dynamic git-gate keys and return an updated plan.
This runs during backend launch, after the operator confirms the
preflight. Plan preparation intentionally stays side-effect-light:
dry-runs and aborted launches must not create remote deploy keys.
"""
if not plan.upstreams:
return plan
upstreams_by_name: dict[str, GitGateUpstream] = {
upstream.name: upstream for upstream in plan.upstreams
}
updated: list[GitGateUpstream] = []
for entry in bottle.git:
upstream = upstreams_by_name.get(entry.Name)
if upstream is None:
continue
if entry.Key.provider == "gitea":
identity_file = _provision_dynamic_key(entry, plan.slug, stage_dir)
upstream = dataclasses.replace(upstream, identity_file=identity_file)
updated.append(upstream)
if len(updated) != len(plan.upstreams):
updated_names = {u.name for u in updated}
for upstream in plan.upstreams:
if upstream.name not in updated_names:
updated.append(upstream)
return dataclasses.replace(plan, upstreams=tuple(updated))
__all__ = [ __all__ = [
"revoke_git_gate_provisioned_keys", "revoke_git_gate_provisioned_keys",
"provision_git_gate_dynamic_keys",
"_provision_dynamic_key", "_provision_dynamic_key",
"_resolve_identity_file", "_resolve_identity_file",
] ]
+48 -44
View File
@@ -228,19 +228,22 @@ supervise_gitleaks_allow() {
fi fi
proposal_id=$( proposal_id=$(
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY' PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
import datetime import datetime
import hashlib import hashlib
import json import json
import os import os
import sys import sys
import uuid
from pathlib import Path from pathlib import Path
try:
import supervise as _sv
except ImportError:
from bot_bottle import supervise as _sv
report_path = Path(sys.argv[1]) report_path = Path(sys.argv[1])
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "") slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
if not queue_dir or not slug: if not slug:
sys.exit(2) sys.exit(2)
try: try:
@@ -277,31 +280,19 @@ for i, finding in enumerate(raw, 1):
]) ])
payload = "\n".join(lines).rstrip() + "\n" payload = "\n".join(lines).rstrip() + "\n"
proposal_id = str(uuid.uuid4()) proposal = _sv.Proposal.new(
proposal = { bottle_slug=slug,
"id": proposal_id, tool=_sv.TOOL_GITLEAKS_ALLOW,
"bottle_slug": slug, proposed_file=payload,
"tool": "gitleaks-allow", justification=(
"proposed_file": payload,
"justification": (
"git-gate found gitleaks findings hidden by # gitleaks:allow; " "git-gate found gitleaks findings hidden by # gitleaks:allow; "
"approve only for dummy test fixtures or confirmed false positives" "approve only for dummy test fixtures or confirmed false positives"
), ),
"arrival_timestamp": datetime.datetime.now( current_file_hash=hashlib.sha256(payload.encode("utf-8")).hexdigest(),
datetime.timezone.utc now=datetime.datetime.now(datetime.timezone.utc),
).isoformat(), )
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(), _sv.write_proposal(proposal)
} print(proposal.id)
queue = Path(queue_dir)
queue.mkdir(parents=True, exist_ok=True)
path = queue / f"{proposal_id}.proposal.json"
tmp = path.with_suffix(path.suffix + ".tmp")
with tmp.open("w", encoding="utf-8") as f:
json.dump(proposal, f, indent=2)
f.write("\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
print(proposal_id)
PY PY
) )
rc=$? rc=$?
@@ -314,8 +305,7 @@ PY
return 1 return 1
fi fi
queue_dir=${SUPERVISE_QUEUE_DIR:-} slug=${SUPERVISE_BOTTLE_SLUG:-}
response_file="$queue_dir/${proposal_id}.response.json"
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300} timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
case "$timeout" in case "$timeout" in
''|*[!0-9]*) ''|*[!0-9]*)
@@ -327,26 +317,41 @@ PY
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2 echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
waited=0 waited=0
while [ "$waited" -lt "$timeout" ]; do while [ "$waited" -lt "$timeout" ]; do
if [ -f "$response_file" ]; then status=$(PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$slug" "$proposal_id" <<'PY'
status=$(python3 - "$response_file" <<'PY'
import json
import sys import sys
try: try:
with open(sys.argv[1], encoding="utf-8") as f: import supervise as _sv
raw = json.load(f) except ImportError:
except (OSError, json.JSONDecodeError): from bot_bottle import supervise as _sv
sys.exit(1)
status = raw.get("status") slug = sys.argv[1]
if not isinstance(status, str): try:
sys.exit(1) response = _sv.read_response(slug, sys.argv[2])
print(status) except FileNotFoundError:
sys.exit(2)
print(response.status)
PY PY
) || status="" )
rc=$?
if [ "$rc" -eq 2 ]; then
status=""
elif [ "$rc" -ne 0 ]; then
status="invalid"
fi
if [ -n "$status" ]; then
case "$status" in case "$status" in
approved|modified) approved|modified)
mkdir -p "$queue_dir/processed" PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$slug" "$proposal_id" <<'PY' || true
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true import sys
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
try:
import supervise as _sv
except ImportError:
from bot_bottle import supervise as _sv
_sv.archive_proposal(sys.argv[1], sys.argv[2])
PY
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2 echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
return 0 return 0
;; ;;
@@ -499,4 +504,3 @@ if ! git -C "$repo_dir" rev-parse --verify HEAD >/dev/null 2>&1; then
fi fi
exit 0 exit 0
""" """
+7 -2
View File
@@ -16,11 +16,16 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path from pathlib import Path
from urllib.parse import urlsplit from urllib.parse import urlsplit
from .git_gate import GIT_GATE_TIMEOUT_SECS
DEFAULT_PORT = 9420 DEFAULT_PORT = 9420
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
# imported: this module ships as a flat top-level sibling in the sidecar
# bundle image (see Dockerfile.sidecars), not as part of the bot_bottle
# package, so `bot_bottle.git_gate` and its dependency chain aren't
# available at runtime.
GIT_GATE_TIMEOUT_SECS = 15
# Bound memory use while still allowing ordinary git push packfiles. # Bound memory use while still allowing ordinary git push packfiles.
MAX_BODY_BYTES = 100 * 1024 * 1024 MAX_BODY_BYTES = 100 * 1024 * 1024
+37
View File
@@ -0,0 +1,37 @@
"""SQLite migration runner for bot-bottle stores."""
from __future__ import annotations
import sqlite3
class TableMigrations:
"""Runs a sequential list of DDL migrations tracked by schema_key in schema_versions."""
def __init__(self, schema_key: str, migrations: list[str]) -> None:
self.schema_key = schema_key
self.migrations = migrations
def apply(self, conn: sqlite3.Connection) -> None:
conn.execute(
"""
CREATE TABLE IF NOT EXISTS schema_versions (
module TEXT PRIMARY KEY,
version INTEGER NOT NULL DEFAULT 0
)
"""
)
row = conn.execute(
"SELECT version FROM schema_versions WHERE module = ?",
(self.schema_key,),
).fetchone()
version = row[0] if row else 0
for i, sql in enumerate(self.migrations[version:], start=version + 1):
conn.execute(sql)
conn.execute(
"INSERT OR REPLACE INTO schema_versions (module, version) VALUES (?, ?)",
(self.schema_key, i),
)
__all__ = ["TableMigrations"]
+215
View File
@@ -0,0 +1,215 @@
"""SQLite-backed queue store for supervise proposals and responses (PRD 0013)."""
from __future__ import annotations
import os
import sqlite3
from pathlib import Path
try:
from .supervise_types import Proposal, Response, host_db_path
from .db_store import DbStore
from .migrations import TableMigrations
except ImportError:
from supervise_types import Proposal, Response, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
class QueueStore(DbStore):
"""SQLite-backed persistent store for supervise proposals and responses."""
def __init__(self, queue_key: str, db_path: Path | None = None) -> None:
self.queue_key = queue_key
if db_path is not None:
resolved = db_path
else:
# In the sidecar container SUPERVISE_DB_PATH points at the
# bind-mounted host DB. On the host this env var is never set,
# so we always fall through to host_db_path().
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
resolved = Path(env_path) if env_path else host_db_path()
# One entry per schema version: migrations[0] brings a fresh DB to
# version 1, [1] to version 2, etc. Add new entries at the end; never
# edit existing ones.
migrations = TableMigrations("queue_store", [
# v1 — proposals table
"""
CREATE TABLE IF NOT EXISTS supervise_proposals (
queue_key TEXT NOT NULL,
id TEXT NOT NULL,
bottle_slug TEXT NOT NULL,
tool TEXT NOT NULL,
proposed_file TEXT NOT NULL,
justification TEXT NOT NULL,
arrival_timestamp TEXT NOT NULL,
current_file_hash TEXT NOT NULL,
archived INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (queue_key, id)
)
""",
# v2 — responses table
"""
CREATE TABLE IF NOT EXISTS supervise_responses (
queue_key TEXT NOT NULL,
proposal_id TEXT NOT NULL,
status TEXT NOT NULL,
notes TEXT NOT NULL,
final_file TEXT,
archived INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (queue_key, proposal_id)
)
""",
])
super().__init__(resolved, migrations)
def write_proposal(self, proposal: Proposal) -> Path:
with self._connect() as conn:
conn.execute(
"""
INSERT OR REPLACE INTO supervise_proposals (
queue_key, id, bottle_slug, tool, proposed_file, justification,
arrival_timestamp, current_file_hash, archived
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 0)
""",
(
self.queue_key,
proposal.id,
proposal.bottle_slug,
proposal.tool,
proposal.proposed_file,
proposal.justification,
proposal.arrival_timestamp,
proposal.current_file_hash,
),
)
self._chmod()
return self.db_path
def read_proposal(self, proposal_id: str) -> Proposal:
with self._connect() as conn:
row = conn.execute(
"""
SELECT * FROM supervise_proposals
WHERE queue_key = ? AND id = ? AND archived = 0
""",
(self.queue_key, proposal_id),
).fetchone()
if row is None:
raise FileNotFoundError(proposal_id)
return self._row_to_proposal(row)
def list_pending_proposals(self) -> list[Proposal]:
if not self.db_path.is_file():
return []
with self._connect() as conn:
rows = conn.execute(
"""
SELECT p.* FROM supervise_proposals p
WHERE p.archived = 0
AND p.queue_key = ?
AND NOT EXISTS (
SELECT 1 FROM supervise_responses r
WHERE r.queue_key = p.queue_key
AND r.proposal_id = p.id
AND r.archived = 0
)
ORDER BY p.arrival_timestamp, p.id
""",
(self.queue_key,),
).fetchall()
return [self._row_to_proposal(row) for row in rows]
def list_all_pending_proposals(self) -> list[Proposal]:
if not self.db_path.is_file():
return []
with self._connect() as conn:
rows = conn.execute(
"""
SELECT p.* FROM supervise_proposals p
WHERE p.archived = 0
AND NOT EXISTS (
SELECT 1 FROM supervise_responses r
WHERE r.queue_key = p.queue_key
AND r.proposal_id = p.id
AND r.archived = 0
)
ORDER BY p.arrival_timestamp, p.id
"""
).fetchall()
return [self._row_to_proposal(row) for row in rows]
def write_response(self, response: Response) -> Path:
with self._connect() as conn:
conn.execute(
"""
INSERT OR REPLACE INTO supervise_responses (
queue_key, proposal_id, status, notes, final_file, archived
) VALUES (?, ?, ?, ?, ?, 0)
""",
(
self.queue_key,
response.proposal_id,
response.status,
response.notes,
response.final_file,
),
)
self._chmod()
return self.db_path
def read_response(self, proposal_id: str) -> Response:
with self._connect() as conn:
row = conn.execute(
"""
SELECT * FROM supervise_responses
WHERE queue_key = ? AND proposal_id = ? AND archived = 0
""",
(self.queue_key, proposal_id),
).fetchone()
if row is None:
raise FileNotFoundError(proposal_id)
return self._row_to_response(row)
def archive_proposal(self, proposal_id: str) -> None:
if not self.db_path.is_file():
return
with self._connect() as conn:
conn.execute(
"""
UPDATE supervise_proposals SET archived = 1
WHERE queue_key = ? AND id = ?
""",
(self.queue_key, proposal_id),
)
conn.execute(
"""
UPDATE supervise_responses SET archived = 1
WHERE queue_key = ? AND proposal_id = ?
""",
(self.queue_key, proposal_id),
)
@staticmethod
def _row_to_proposal(row: sqlite3.Row) -> Proposal:
return Proposal(
id=row["id"],
bottle_slug=row["bottle_slug"],
tool=row["tool"],
proposed_file=row["proposed_file"],
justification=row["justification"],
arrival_timestamp=row["arrival_timestamp"],
current_file_hash=row["current_file_hash"],
)
@staticmethod
def _row_to_response(row: sqlite3.Row) -> Response:
return Response(
proposal_id=row["proposal_id"],
status=row["status"],
notes=row["notes"],
final_file=row["final_file"],
)
__all__ = ["QueueStore"]
+60
View File
@@ -0,0 +1,60 @@
"""Singleton manager for all bot-bottle SQLite stores (PRD 0013)."""
from __future__ import annotations
from pathlib import Path
try:
from .audit_store import AuditStore
from .queue_store import QueueStore
except ImportError:
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
_instance: StoreManager | None = None
class StoreManager:
"""Owns db_path and delegates migrate/is_migrated across all stores.
Use instance() for normal access. Call reset(db_path) in tests to swap
the singleton to a temp path, then reset() with no args to restore the
default."""
def __init__(self, db_path: Path | None = None) -> None:
if db_path is None:
# Lazy import to avoid a circular dependency: supervise imports
# StoreManager at module level; StoreManager must not import
# supervise at module level in return.
try:
from .supervise import host_db_path
except ImportError:
from supervise import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
db_path = host_db_path()
self.db_path = db_path
@classmethod
def instance(cls) -> StoreManager:
global _instance
if _instance is None:
_instance = cls()
return _instance
@classmethod
def reset(cls, db_path: Path | None = None) -> None:
"""Replace the singleton. Pass db_path for test isolation; omit to restore default."""
global _instance
_instance = cls(db_path)
def is_migrated(self) -> bool:
return (
QueueStore("", self.db_path).is_migrated()
and AuditStore(self.db_path).is_migrated()
)
def migrate(self) -> None:
QueueStore("", self.db_path).migrate()
AuditStore(self.db_path).migrate()
__all__ = ["StoreManager"]
+114 -351
View File
@@ -9,15 +9,14 @@ calls when it needs an operator-reviewed egress change:
Each tool call: the agent passes the full proposed file plus a Each tool call: the agent passes the full proposed file plus a
justification text. The sidecar validates the proposal syntactically, justification text. The sidecar validates the proposal syntactically,
writes it to the host's per-bottle queue dir, and holds the tool-call writes it to the host SQLite queue table, and holds the tool-call
connection open. The operator's supervise TUI connection open. The operator's supervise TUI
(bot_bottle.cli.supervise) sees the proposal, accepts (bot_bottle.cli.supervise) sees the proposal, accepts
approve / modify / reject, and writes a response file alongside the approve / modify / reject, and writes a response row. The sidecar sees
proposal. The sidecar sees the response and returns `{status, notes}` the response and returns `{status, notes}` to the agent.
to the agent.
This module defines the host-side library: dataclasses for the queue This module defines the host-side library: dataclasses for the queue
file shapes, queue read/write helpers, the audit log writer, and the record shapes, queue read/write helpers, the audit log writer, and the
diff renderer. The in-container sidecar lives in diff renderer. The in-container sidecar lives in
bot_bottle/supervise_server.py; the supervise daemon's container bot_bottle/supervise_server.py; the supervise daemon's container
lifecycle is owned by the sidecar bundle (PRD 0024). lifecycle is owned by the sidecar bundle (PRD 0024).
@@ -31,37 +30,56 @@ remediation engines that wire real config changes land in PRDs 0014,
from __future__ import annotations from __future__ import annotations
import dataclasses
import difflib import difflib
import hashlib import hashlib
import json
import os
import time import time
import uuid
from abc import ABC from abc import ABC
from dataclasses import dataclass from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
try:
from .supervise_types import (
ACTION_OPERATOR_EDIT,
AuditEntry,
HOST_DB_FILENAME,
Proposal,
Response,
STATUSES,
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOLS,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
bot_bottle_root,
)
except ImportError:
from supervise_types import ( # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
ACTION_OPERATOR_EDIT,
AuditEntry,
HOST_DB_FILENAME,
Proposal,
Response,
STATUSES,
STATUS_APPROVED,
STATUS_MODIFIED,
STATUS_REJECTED,
TOOLS,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
bot_bottle_root,
)
SUPERVISE_HOSTNAME = "supervise" SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100 SUPERVISE_PORT = 9100
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
# Written directly by the egress addon (not an agent-facing MCP tool) when an
# outbound DLP token block is routed to the operator for override (PRD 0062).
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
TOOLS: tuple[str, ...] = (
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
# The supervise sidecar uses these to query egress's # The supervise sidecar uses these to query egress's
# introspection endpoint for the `list-egress-routes` MCP # introspection endpoint for the `list-egress-routes` MCP
# tool. The hostname + port match egress's docker network # tool. The hostname + port match egress's docker network
@@ -77,236 +95,77 @@ COMPONENT_FOR_TOOL: dict[str, str] = {
TOOL_EGRESS_BLOCK: "egress", TOOL_EGRESS_BLOCK: "egress",
} }
STATUS_APPROVED = "approved" DB_PATH_IN_CONTAINER = "/run/supervise/bot-bottle.db"
STATUS_MODIFIED = "modified"
STATUS_REJECTED = "rejected"
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
# Operator-initiated audit entries (no tool call). PRD 0014's
# `routes edit <bottle>` verb writes entries with this action.
ACTION_OPERATOR_EDIT = "operator-edit"
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
DEFAULT_POLL_INTERVAL_SEC = 0.5 DEFAULT_POLL_INTERVAL_SEC = 0.5
# --- Paths ----------------------------------------------------------------- # --- Paths -----------------------------------------------------------------
def bot_bottle_root() -> Path:
return Path.home() / ".bot-bottle"
def queue_dir_for_slug(slug: str) -> Path:
return bot_bottle_root() / "queue" / slug
def audit_dir() -> Path: def audit_dir() -> Path:
return bot_bottle_root() / "audit" return bot_bottle_root() / "audit"
def host_db_path() -> Path:
# Calls bot_bottle_root() through this module's globals so that patches
# on supervise.bot_bottle_root propagate to callers going through
# supervise.host_db_path().
#
# Kept in its own "db" subdirectory (see supervise_types.host_db_path
# for why) — must stay in sync with that copy.
return bot_bottle_root() / "db" / HOST_DB_FILENAME
def audit_log_path(component: str, slug: str) -> Path: def audit_log_path(component: str, slug: str) -> Path:
return audit_dir() / f"{component}-{slug}.log" return audit_dir() / f"{component}-{slug}.log"
# --- Dataclasses ----------------------------------------------------------- try:
from .queue_store import QueueStore
from .audit_store import AuditStore
@dataclass(frozen=True) from .store_manager import StoreManager
class Proposal: except ImportError:
"""One pending tool-call from the agent. The sidecar writes one # Sidecar bundle: files are flat-copied under /app, not a package.
of these to the queue dir on a tool call; the operator's TUI from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
reads them; the sidecar polls for a matching Response.""" from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from store_manager import StoreManager # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
id: str
bottle_slug: str
tool: str
proposed_file: str
justification: str
arrival_timestamp: str
current_file_hash: str
@classmethod
def new(
cls,
*,
bottle_slug: str,
tool: str,
proposed_file: str,
justification: str,
current_file_hash: str,
now: datetime | None = None,
) -> "Proposal":
ts = (now or datetime.now(timezone.utc)).isoformat()
return cls(
id=str(uuid.uuid4()),
bottle_slug=bottle_slug,
tool=tool,
proposed_file=proposed_file,
justification=justification,
arrival_timestamp=ts,
current_file_hash=current_file_hash,
)
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
@classmethod
def from_dict(cls, raw: dict[str, object]) -> "Proposal":
tool = _require_str(raw, "tool")
if tool not in TOOLS:
raise ValueError(f"tool must be one of {TOOLS}; got {tool!r}")
return cls(
id=_require_str(raw, "id"),
bottle_slug=_require_str(raw, "bottle_slug"),
tool=tool,
proposed_file=_require_str(raw, "proposed_file"),
justification=_require_str(raw, "justification"),
arrival_timestamp=_require_str(raw, "arrival_timestamp"),
current_file_hash=_require_str(raw, "current_file_hash"),
)
@dataclass(frozen=True)
class Response:
"""The operator's decision on a proposal. The TUI writes one of
these to the queue dir; the sidecar reads it and returns the
`{status, notes}` pair to the agent's tool call.
`final_file` carries the file content the supervisor will
actually apply: for `approved`, equal to the proposal's
`proposed_file`; for `modified`, the operator's edited version
(the audit diff is current final_file, not current
proposed_file); for `rejected`, None."""
proposal_id: str
status: str
notes: str
final_file: str | None = None
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
@classmethod
def from_dict(cls, raw: dict[str, object]) -> "Response":
status = _require_str(raw, "status")
if status not in STATUSES:
raise ValueError(
f"response status must be one of {STATUSES}; got {status!r}"
)
final = raw.get("final_file")
if final is not None and not isinstance(final, str):
raise ValueError(
f"final_file must be a string or null; got {type(final).__name__}"
)
return cls(
proposal_id=_require_str(raw, "proposal_id"),
status=status,
notes=_require_str(raw, "notes"),
final_file=final,
)
@dataclass(frozen=True)
class AuditEntry:
"""One row of the per-bottle audit log. JSON-Lines, append-only."""
timestamp: str
bottle_slug: str
component: str
operator_action: str
operator_notes: str
justification: str
diff: str
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
# --- Queue I/O ------------------------------------------------------------- # --- Queue I/O -------------------------------------------------------------
def _proposal_filename(proposal_id: str) -> str: def write_proposal(proposal: Proposal) -> Path:
return f"{proposal_id}.proposal.json" """Persist `proposal` in the queue database, mode 0o600.
def _response_filename(proposal_id: str) -> str:
return f"{proposal_id}.response.json"
def _id_from_proposal_filename(path: Path) -> str | None:
name = path.name
if not name.endswith(".proposal.json"):
return None
return name[: -len(".proposal.json")]
def write_proposal(queue_dir: Path, proposal: Proposal) -> Path:
"""Persist `proposal` as JSON in the queue dir, mode 0o600.
Directory is created if missing.""" Directory is created if missing."""
queue_dir.mkdir(parents=True, exist_ok=True) return QueueStore(proposal.bottle_slug).write_proposal(proposal)
path = queue_dir / _proposal_filename(proposal.id)
payload = json.dumps(proposal.to_dict(), indent=2) + "\n"
_atomic_write(path, payload, mode=0o600)
return path
def read_proposal(queue_dir: Path, proposal_id: str) -> Proposal: def read_proposal(bottle_slug: str, proposal_id: str) -> Proposal:
path = queue_dir / _proposal_filename(proposal_id) return QueueStore(bottle_slug).read_proposal(proposal_id)
with path.open() as f:
raw = json.load(f)
if not isinstance(raw, dict):
raise ValueError(f"{path}: top-level must be an object")
return Proposal.from_dict(raw)
def list_pending_proposals(queue_dir: Path) -> list[Proposal]: def list_pending_proposals(bottle_slug: str) -> list[Proposal]:
"""All proposals in `queue_dir` that do not yet have a matching """All proposals for `bottle_slug` that do not yet have a matching
response file. Sorted by `arrival_timestamp` so the operator response. Sorted by `arrival_timestamp` so the operator
sees the queue FIFO.""" sees the queue FIFO."""
if not queue_dir.is_dir(): return QueueStore(bottle_slug).list_pending_proposals()
return []
out: list[Proposal] = []
for path in sorted(queue_dir.glob("*.proposal.json")):
proposal_id = _id_from_proposal_filename(path)
if proposal_id is None:
continue
if (queue_dir / _response_filename(proposal_id)).exists():
continue
try:
with path.open() as f:
raw = json.load(f)
except (OSError, json.JSONDecodeError):
continue
if not isinstance(raw, dict):
continue
try:
out.append(Proposal.from_dict(raw))
except (KeyError, ValueError):
continue
out.sort(key=lambda p: p.arrival_timestamp)
return out
def write_response(queue_dir: Path, response: Response) -> Path: def list_all_pending_proposals() -> list[Proposal]:
queue_dir.mkdir(parents=True, exist_ok=True) """All pending proposals across bottles, sorted FIFO."""
path = queue_dir / _response_filename(response.proposal_id) return QueueStore("").list_all_pending_proposals()
payload = json.dumps(response.to_dict(), indent=2) + "\n"
_atomic_write(path, payload, mode=0o600)
return path
def read_response(queue_dir: Path, proposal_id: str) -> Response: def write_response(bottle_slug: str, response: Response) -> Path:
path = queue_dir / _response_filename(proposal_id) return QueueStore(bottle_slug).write_response(response)
with path.open() as f:
raw = json.load(f)
if not isinstance(raw, dict): def read_response(bottle_slug: str, proposal_id: str) -> Response:
raise ValueError(f"{path}: top-level must be an object") return QueueStore(bottle_slug).read_response(proposal_id)
return Response.from_dict(raw)
def wait_for_response( def wait_for_response(
queue_dir: Path, bottle_slug: str,
proposal_id: str, proposal_id: str,
*, *,
poll_interval: float = DEFAULT_POLL_INTERVAL_SEC, poll_interval: float = DEFAULT_POLL_INTERVAL_SEC,
@@ -317,90 +176,35 @@ def wait_for_response(
which the wait raises TimeoutError. None waits forever the which the wait raises TimeoutError. None waits forever the
natural shape, since the operator's response time is unbounded. natural shape, since the operator's response time is unbounded.
Polls the filesystem so the implementation stays portable and Polls SQLite so the implementation stays portable and stdlib-only."""
stdlib-only.""" store = QueueStore(bottle_slug)
path = queue_dir / _response_filename(proposal_id)
while True: while True:
if path.exists(): try:
try: return store.read_response(proposal_id)
with path.open() as f: except FileNotFoundError:
raw = json.load(f) pass
except (OSError, json.JSONDecodeError):
raw = None
if isinstance(raw, dict):
try:
return Response.from_dict(raw)
except (KeyError, ValueError):
pass
if deadline is not None and time.monotonic() >= deadline: if deadline is not None and time.monotonic() >= deadline:
raise TimeoutError(f"no response for proposal {proposal_id!r}") raise TimeoutError(f"no response for proposal {proposal_id!r}")
time.sleep(poll_interval) time.sleep(poll_interval)
def archive_proposal(queue_dir: Path, proposal_id: str) -> None: def archive_proposal(bottle_slug: str, proposal_id: str) -> None:
"""Move both proposal and response files to `<queue_dir>/processed/`. """Mark both proposal and response rows processed.
Idempotent missing files are silently skipped.""" Idempotent missing rows are silently skipped."""
processed = queue_dir / "processed" QueueStore(bottle_slug).archive_proposal(proposal_id)
processed.mkdir(parents=True, exist_ok=True)
for name in (_proposal_filename(proposal_id), _response_filename(proposal_id)):
src = queue_dir / name
if src.exists():
src.rename(processed / name)
# --- Audit log ------------------------------------------------------------- # --- Audit log -------------------------------------------------------------
def write_audit_entry(entry: AuditEntry) -> Path: def write_audit_entry(entry: AuditEntry) -> Path:
"""Append `entry` as one JSON-Lines record to the per-bottle """Append `entry` to the host supervise audit table."""
audit log. Acquires an advisory exclusive lock so concurrent return AuditStore().write_audit_entry(entry)
writers don't interleave bytes."""
path = audit_log_path(entry.component, entry.bottle_slug)
path.parent.mkdir(parents=True, exist_ok=True)
line = json.dumps(entry.to_dict(), sort_keys=False) + "\n"
fd = os.open(path, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0o600)
try:
_try_flock(fd)
try:
os.write(fd, line.encode("utf-8"))
finally:
_try_funlock(fd)
finally:
os.close(fd)
return path
def read_audit_entries(component: str, slug: str) -> list[AuditEntry]: def read_audit_entries(component: str, slug: str) -> list[AuditEntry]:
"""Load all audit entries for the given component+slug. Empty """Load all audit entries for the given component+slug."""
list if the log doesn't exist.""" return AuditStore().read_audit_entries(component, slug)
path = audit_log_path(component, slug)
if not path.is_file():
return []
out: list[AuditEntry] = []
with path.open() as f:
for raw_line in f:
raw_line = raw_line.strip()
if not raw_line:
continue
try:
raw = json.loads(raw_line)
except json.JSONDecodeError:
continue
if not isinstance(raw, dict):
continue
try:
out.append(AuditEntry(
timestamp=_require_str(raw, "timestamp"),
bottle_slug=_require_str(raw, "bottle_slug"),
component=_require_str(raw, "component"),
operator_action=_require_str(raw, "operator_action"),
operator_notes=_require_str(raw, "operator_notes"),
justification=_require_str(raw, "justification"),
diff=_require_str(raw, "diff"),
))
except ValueError:
continue
return out
# --- Diff rendering -------------------------------------------------------- # --- Diff rendering --------------------------------------------------------
@@ -433,89 +237,47 @@ def sha256_hex(content: str) -> str:
class SupervisePlan: class SupervisePlan:
"""Output of Supervise.prepare; consumed by .start. """Output of Supervise.prepare; consumed by .start.
`queue_dir` is the host directory bind-mounted into the sidecar `db_path` is the host database bind-mounted into the sidecar at
at /run/supervise/queue. `internal_network` is empty at prepare /run/supervise/bot-bottle.db. `internal_network` is empty at
time; the backend's launch step fills it via dataclasses.replace prepare time; the backend's launch step fills it via
before calling .start.""" dataclasses.replace before calling .start."""
slug: str slug: str
queue_dir: Path db_path: Path
internal_network: str = "" internal_network: str = ""
class Supervise(ABC): class Supervise(ABC):
"""Per-bottle supervise sidecar. Encapsulates the host-side """Per-bottle supervise sidecar. Encapsulates host-side database
prepare (queue dir staging); the sidecar's start/stop lifecycle staging; the sidecar's start/stop lifecycle is backend-specific."""
is backend-specific."""
def prepare( def prepare(
self, self,
slug: str, slug: str,
stage_dir: Path, stage_dir: Path,
) -> SupervisePlan: ) -> SupervisePlan:
"""Stage the per-bottle queue dir on the host. Returns the """Stage the host database. Returns the plan; `internal_network`
plan; `internal_network` must be set by the launch step before must be set by the launch step before .start runs."""
.start runs."""
del stage_dir del stage_dir
queue_dir = queue_dir_for_slug(slug) mgr = StoreManager.instance()
queue_dir.mkdir(parents=True, exist_ok=True) mgr.migrate()
return SupervisePlan( return SupervisePlan(
slug=slug, slug=slug,
queue_dir=queue_dir, db_path=mgr.db_path,
) )
# --- Helpers ---------------------------------------------------------------
def _require_str(raw: dict[str, object], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str):
raise ValueError(f"missing or non-string field {key!r}")
return value
def _atomic_write(path: Path, content: str, *, mode: int) -> None:
"""Atomic: write to a sibling tmp file, fsync, rename."""
tmp = path.with_suffix(path.suffix + ".tmp")
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode)
try:
os.write(fd, content.encode("utf-8"))
os.fsync(fd)
finally:
os.close(fd)
os.replace(tmp, path)
try:
import fcntl as _fcntl
def _try_flock(fd: int) -> None: # type: ignore[reportRedeclaration]
try:
_fcntl.flock(fd, _fcntl.LOCK_EX)
except OSError:
pass
def _try_funlock(fd: int) -> None: # type: ignore[reportRedeclaration]
try:
_fcntl.flock(fd, _fcntl.LOCK_UN)
except OSError:
pass
except ImportError: # pragma: no cover — Windows path
def _try_flock(fd: int) -> None: # noqa: F841 — Windows fallback
return None
def _try_funlock(fd: int) -> None: # noqa: F841 — Windows fallback
return None
__all__ = [ __all__ = [
"ACTION_OPERATOR_EDIT", "ACTION_OPERATOR_EDIT",
"AuditEntry", "AuditEntry",
"AuditStore",
"COMPONENT_FOR_TOOL", "COMPONENT_FOR_TOOL",
"DEFAULT_POLL_INTERVAL_SEC", "DEFAULT_POLL_INTERVAL_SEC",
"DB_PATH_IN_CONTAINER",
"Proposal", "Proposal",
"QUEUE_DIR_IN_CONTAINER", "QueueStore",
"Response", "Response",
"StoreManager",
"STATUSES", "STATUSES",
"STATUS_APPROVED", "STATUS_APPROVED",
"STATUS_MODIFIED", "STATUS_MODIFIED",
@@ -536,8 +298,9 @@ __all__ = [
"audit_dir", "audit_dir",
"audit_log_path", "audit_log_path",
"bot_bottle_root", "bot_bottle_root",
"host_db_path",
"list_pending_proposals", "list_pending_proposals",
"queue_dir_for_slug", "list_all_pending_proposals",
"read_audit_entries", "read_audit_entries",
"read_proposal", "read_proposal",
"read_response", "read_response",
+9 -17
View File
@@ -7,14 +7,13 @@ config changes when stuck. The tools are `egress-allow`,
Each queued tool call: Each queued tool call:
1. Validates the proposed file syntactically. 1. Validates the proposed file syntactically.
2. Writes a Proposal to /run/supervise/queue/ (bind-mounted from 2. Writes a Proposal to the host SQLite database.
the host's ~/.bot-bottle/queue/<slug>/). 3. Blocks polling for a matching Response row.
3. Blocks polling for a matching Response file.
4. Returns the operator's `{status, notes}` to the agent. 4. Returns the operator's `{status, notes}` to the agent.
The bottle slug arrives via SUPERVISE_BOTTLE_SLUG env (stamped at The bottle slug arrives via SUPERVISE_BOTTLE_SLUG env (stamped at
container creation by the backend's start step). The queue dir comes container creation by the backend's start step). SUPERVISE_DB_PATH
from SUPERVISE_QUEUE_DIR (default `/run/supervise/queue`). points at the bind-mounted host database.
Speaks MCP over HTTP+JSON-RPC. Methods handled: Speaks MCP over HTTP+JSON-RPC. Methods handled:
@@ -42,7 +41,6 @@ import typing
import urllib.error import urllib.error
import urllib.request import urllib.request
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path
try: try:
# Same-directory imports inside the bundle container; these files are # Same-directory imports inside the bundle container; these files are
@@ -277,7 +275,6 @@ def validate_proposed_file(tool: str, content: str) -> None:
@dataclass(frozen=True) @dataclass(frozen=True)
class ServerConfig: class ServerConfig:
bottle_slug: str bottle_slug: str
queue_dir: Path
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS
@@ -376,7 +373,7 @@ def handle_tools_call(
current_file_hash=_sv.sha256_hex(proposed_file), current_file_hash=_sv.sha256_hex(proposed_file),
) )
try: try:
_sv.write_proposal(config.queue_dir, proposal) _sv.write_proposal(proposal)
except OSError as e: except OSError as e:
raise _RpcInternalError(f"failed to write proposal to queue: {e}") from e raise _RpcInternalError(f"failed to write proposal to queue: {e}") from e
sys.stderr.write( sys.stderr.write(
@@ -387,7 +384,7 @@ def handle_tools_call(
deadline = time.monotonic() + config.response_timeout_seconds deadline = time.monotonic() + config.response_timeout_seconds
try: try:
response = _sv.wait_for_response( response = _sv.wait_for_response(
config.queue_dir, config.bottle_slug,
proposal.id, proposal.id,
poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS, poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS,
deadline=deadline, deadline=deadline,
@@ -399,7 +396,7 @@ def handle_tools_call(
"isError": False, "isError": False,
} }
try: try:
_sv.archive_proposal(config.queue_dir, proposal.id) _sv.archive_proposal(config.bottle_slug, proposal.id)
except OSError as e: except OSError as e:
raise _RpcInternalError(f"failed to archive proposal: {e}") from e raise _RpcInternalError(f"failed to archive proposal: {e}") from e
@@ -539,7 +536,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer): class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
allow_reuse_address = True allow_reuse_address = True
daemon_threads = True daemon_threads = True
config: ServerConfig = ServerConfig(bottle_slug="", queue_dir=Path()) config: ServerConfig = ServerConfig(bottle_slug="")
# --- Entry point ----------------------------------------------------------- # --- Entry point -----------------------------------------------------------
@@ -548,21 +545,18 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
def serve( def serve(
*, *,
bottle_slug: str, bottle_slug: str,
queue_dir: Path,
port: int = _sv.SUPERVISE_PORT, port: int = _sv.SUPERVISE_PORT,
bind: str = "0.0.0.0", bind: str = "0.0.0.0",
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS, response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
) -> typing.NoReturn: ) -> typing.NoReturn:
queue_dir.mkdir(parents=True, exist_ok=True)
server = MCPServer((bind, port), MCPHandler) server = MCPServer((bind, port), MCPHandler)
server.config = ServerConfig( server.config = ServerConfig(
bottle_slug=bottle_slug, bottle_slug=bottle_slug,
queue_dir=queue_dir,
response_timeout_seconds=response_timeout_seconds, response_timeout_seconds=response_timeout_seconds,
) )
sys.stderr.write( sys.stderr.write(
f"supervise listening on {bind}:{port}; " f"supervise listening on {bind}:{port}; "
f"slug={bottle_slug!r}; queue={queue_dir}; " f"slug={bottle_slug!r}; "
f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type] f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type]
) )
sys.stderr.flush() sys.stderr.flush()
@@ -581,7 +575,6 @@ def main(argv: list[str]) -> int:
if not bottle_slug: if not bottle_slug:
sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n") sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n")
return 2 return 2
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT))) port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0") bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
try: try:
@@ -591,7 +584,6 @@ def main(argv: list[str]) -> int:
return 2 return 2
serve( serve(
bottle_slug=bottle_slug, bottle_slug=bottle_slug,
queue_dir=queue_dir,
port=port, port=port,
bind=bind, bind=bind,
response_timeout_seconds=response_timeout_seconds, response_timeout_seconds=response_timeout_seconds,
+189
View File
@@ -0,0 +1,189 @@
"""Shared types and path helpers for supervise stores (PRD 0013).
Extracted from supervise.py so queue_store and audit_store can import
Proposal, Response, AuditEntry, and host_db_path without creating a
circular import (supervise imports from queue_store/audit_store and
vice-versa).
Patching bot_bottle_root on this module propagates through host_db_path
because host_db_path looks up bot_bottle_root via sys.modules[__name__]
at call time rather than capturing it at import time.
"""
from __future__ import annotations
import dataclasses
import sys
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
HOST_DB_FILENAME = "bot-bottle.db"
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
TOOLS: tuple[str, ...] = (
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
STATUS_APPROVED = "approved"
STATUS_MODIFIED = "modified"
STATUS_REJECTED = "rejected"
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
ACTION_OPERATOR_EDIT = "operator-edit"
def bot_bottle_root() -> Path:
return Path.home() / ".bot-bottle"
def host_db_path() -> Path:
# Look up bot_bottle_root through this module's live namespace so that
# monkey-patches on supervise_types.bot_bottle_root take effect.
#
# Lives in its own "db" subdirectory, not directly under
# bot_bottle_root(), so backends that can only bind-mount
# directories (macos_container's `container run --mount`, which
# rejects file sources with "is not a directory") can share this
# one file with a sidecar without also exposing bot_bottle_root()'s
# other contents (git-gate keys, per-bottle state, etc.).
return sys.modules[__name__].bot_bottle_root() / "db" / HOST_DB_FILENAME
def _require_str(raw: dict[str, object], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str):
raise ValueError(f"missing or non-string field {key!r}")
return value
@dataclass(frozen=True)
class Proposal:
"""One pending tool-call from the agent."""
id: str
bottle_slug: str
tool: str
proposed_file: str
justification: str
arrival_timestamp: str
current_file_hash: str
@classmethod
def new(
cls,
*,
bottle_slug: str,
tool: str,
proposed_file: str,
justification: str,
current_file_hash: str,
now: datetime | None = None,
) -> "Proposal":
ts = (now or datetime.now(timezone.utc)).isoformat()
return cls(
id=str(uuid.uuid4()),
bottle_slug=bottle_slug,
tool=tool,
proposed_file=proposed_file,
justification=justification,
arrival_timestamp=ts,
current_file_hash=current_file_hash,
)
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
@classmethod
def from_dict(cls, raw: dict[str, object]) -> "Proposal":
tool = _require_str(raw, "tool")
if tool not in TOOLS:
raise ValueError(f"tool must be one of {TOOLS}; got {tool!r}")
return cls(
id=_require_str(raw, "id"),
bottle_slug=_require_str(raw, "bottle_slug"),
tool=tool,
proposed_file=_require_str(raw, "proposed_file"),
justification=_require_str(raw, "justification"),
arrival_timestamp=_require_str(raw, "arrival_timestamp"),
current_file_hash=_require_str(raw, "current_file_hash"),
)
@dataclass(frozen=True)
class Response:
"""The operator's decision on a proposal."""
proposal_id: str
status: str
notes: str
final_file: str | None = None
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
@classmethod
def from_dict(cls, raw: dict[str, object]) -> "Response":
status = _require_str(raw, "status")
if status not in STATUSES:
raise ValueError(
f"response status must be one of {STATUSES}; got {status!r}"
)
final = raw.get("final_file")
if final is not None and not isinstance(final, str):
raise ValueError(
f"final_file must be a string or null; got {type(final).__name__}"
)
return cls(
proposal_id=_require_str(raw, "proposal_id"),
status=status,
notes=_require_str(raw, "notes"),
final_file=final,
)
@dataclass(frozen=True)
class AuditEntry:
"""One row of the per-bottle audit log."""
timestamp: str
bottle_slug: str
component: str
operator_action: str
operator_notes: str
justification: str
diff: str
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
__all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"HOST_DB_FILENAME",
"Proposal",
"Response",
"STATUSES",
"STATUS_APPROVED",
"STATUS_MODIFIED",
"STATUS_REJECTED",
"TOOLS",
"TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK",
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_GITLEAKS_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"bot_bottle_root",
"host_db_path",
]
+107
View File
@@ -21,6 +21,11 @@ Public API:
For a Markdown file with YAML frontmatter delimited by `---` For a Markdown file with YAML frontmatter delimited by `---`
lines. Returns (frontmatter_dict, body_text). lines. Returns (frontmatter_dict, body_text).
serialize_yaml_subset(data) -> str
Serialize a dict (as produced by parse_yaml_subset) back to
block-style YAML text. The result ends with a newline and
can be parsed back by parse_yaml_subset.
What we accept (block-style): What we accept (block-style):
key: value # mapping entry, value is inline key: value # mapping entry, value is inline
@@ -576,3 +581,105 @@ def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
fm = parse_yaml_subset(fm_text) fm = parse_yaml_subset(fm_text)
body = text[body_start:] body = text[body_start:]
return fm, body return fm, body
# --- Serializer -------------------------------------------------------------
def _needs_quoting(s: str) -> bool:
"""True when the string must be single-quoted to survive a round-trip."""
if not s:
return True
if s in ("true", "false", "null", "~") or s in _RESERVED_BOOL_LIKE:
return True
if (
_INT_RX.match(s)
or _DATE_RX.match(s)
or _OCTAL_RX.match(s)
or _HEX_RX.match(s)
or _FLOAT_RX.match(s)
):
return True
# Characters that have special meaning at the start of a YAML value
if s[0] in ('"', "'", "[", "{", "!", "&", "*", "#", "|", ">", "%", "@", "`"):
return True
return False
def _yaml_scalar(v: object) -> str:
"""Serialize a scalar Python value to its YAML text form."""
if v is None:
return "null"
if isinstance(v, bool):
return "true" if v else "false"
if isinstance(v, int):
return str(v)
s = str(v)
if _needs_quoting(s):
return "'" + s.replace("'", "''") + "'"
return s
def _serialize_node(node: object, indent: int) -> list[str]:
"""Return lines (without trailing newlines) for `node` at `indent`.
Called only for non-empty dicts and lists (the caller guards with
`isinstance(val, (dict, list)) and val`), plus scalars at the leaf."""
prefix = " " * indent
if isinstance(node, dict):
out: list[str] = []
for key, val in node.items():
if isinstance(val, (dict, list)) and val:
out.append(f"{prefix}{key}:")
out.extend(_serialize_node(val, indent + 2))
else:
scalar = (
"{}" if isinstance(val, dict)
else "[]" if isinstance(val, list)
else _yaml_scalar(val)
)
out.append(f"{prefix}{key}: {scalar}")
return out
if isinstance(node, list):
out = []
for item in node:
if isinstance(item, dict) and item:
entries = list(item.items())
first_key, first_val = entries[0]
if isinstance(first_val, (dict, list)) and first_val:
out.append(f"{prefix}- {first_key}:")
out.extend(_serialize_node(first_val, indent + 4))
else:
scalar = (
"{}" if isinstance(first_val, dict)
else "[]" if isinstance(first_val, list)
else _yaml_scalar(first_val)
)
out.append(f"{prefix}- {first_key}: {scalar}")
cont = prefix + " "
for key, val in entries[1:]:
if isinstance(val, (dict, list)) and val:
out.append(f"{cont}{key}:")
out.extend(_serialize_node(val, indent + 4))
else:
scalar = (
"{}" if isinstance(val, dict)
else "[]" if isinstance(val, list)
else _yaml_scalar(val)
)
out.append(f"{cont}{key}: {scalar}")
else:
out.append(f"{prefix}- {_yaml_scalar(item)}")
return out
return [_yaml_scalar(node)] # pragma: no cover
def serialize_yaml_subset(data: dict[str, object]) -> str:
"""Serialize `data` (as produced by parse_yaml_subset) to YAML text.
Produces block-style output with 2-space indentation. The result ends
with a newline and can be parsed back by parse_yaml_subset. Keys are
emitted in iteration order (insertion order in Python 3.7+)."""
if not data:
return ""
return "\n".join(_serialize_node(data, 0)) + "\n"
+135
View File
@@ -0,0 +1,135 @@
# PRD 0067: SQLite local storage
- **Status:** Active
- **Author:** codex
- **Created:** 2026-07-01
- **Issue:** #319
## Summary
Add a small stdlib SQLite storage layer for bot-bottle host runtime state,
starting with the supervise queue and audit log. This replaces scattered JSON
queue files and JSONL audit logs with structured tables while preserving the
existing public supervise helper functions and sidecar queue mount contract.
## Problem
Bot-bottle currently stores supervise proposals and responses as individual JSON
files under `~/.bot-bottle/queue/<slug>/`, and audit entries as JSONL files
under `~/.bot-bottle/audit/`. That worked for the original interactive TUI, but
new forge-native orchestration needs durable, queryable local state for queues,
audit trails, watchdogs, and lifecycle records. PR #318 started introducing
SQLite-shaped boilerplate for forge state; the storage foundation should live in
its own PR so forge work can build on the shared runtime store instead of adding
one-off persistence.
## Goals / Success Criteria
1. Supervise proposals and responses are persisted through SQLite.
2. Audit entries are persisted through SQLite.
3. Supervise queue helpers use the bottle slug / queue key instead of a queue
directory path.
4. The sidecar receives the host database mount across docker, smolmachines,
and macOS-container backends.
5. The implementation stays stdlib-only.
6. Schema migrations use a `PRAGMA user_version` runner — no third-party deps.
7. Unit tests cover queue round-trips, pending discovery, response waits,
archive semantics, audit round-trips, and path creation.
## Non-goals
- Migrating old JSON queue files or JSONL audit logs.
- Adding forge orchestration state tables.
- Adding egress metering or budget tables.
- Changing the supervise TUI workflow or remediation behavior.
- Introducing a third-party ORM or migration library.
## Design
### Database locations
Queue and audit state use the host-level local database:
```text
~/.bot-bottle/bot-bottle.db
```
The supervise sidecar receives that database as a writable bind mount at
`/run/supervise/bot-bottle.db` and gets the path through `SUPERVISE_DB_PATH`.
No per-slug queue directory is mounted into the sidecar. This creates the shared
host database that later forge/native lifecycle work can extend in separate
PRDs.
### Tables
`supervise_proposals` lives in the host database:
```sql
CREATE TABLE supervise_proposals (
queue_key TEXT NOT NULL,
id TEXT NOT NULL,
bottle_slug TEXT NOT NULL,
tool TEXT NOT NULL,
proposed_file TEXT NOT NULL,
justification TEXT NOT NULL,
arrival_timestamp TEXT NOT NULL,
current_file_hash TEXT NOT NULL,
archived INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (queue_key, id)
);
```
`supervise_responses` lives in the host database:
```sql
CREATE TABLE supervise_responses (
queue_key TEXT NOT NULL,
proposal_id TEXT NOT NULL,
status TEXT NOT NULL,
notes TEXT NOT NULL,
final_file TEXT,
archived INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (queue_key, proposal_id)
);
```
`supervise_audit_entries` lives in the host database:
```sql
CREATE TABLE supervise_audit_entries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT NOT NULL,
bottle_slug TEXT NOT NULL,
component TEXT NOT NULL,
operator_action TEXT NOT NULL,
operator_notes TEXT NOT NULL,
justification TEXT NOT NULL,
diff TEXT NOT NULL
);
```
### Compatibility
The queue helpers take a bottle slug / queue key and perform equivalent
operations against `~/.bot-bottle/bot-bottle.db`:
- `list_pending_proposals` returns non-archived proposals without a non-archived
response, sorted by arrival time.
- `archive_proposal` marks matching proposal/response rows archived instead of
moving files into `processed/`.
- `wait_for_response` keeps the current polling behavior but polls SQLite.
The old audit path helpers (`audit_dir`, `audit_log_path`) stay available for
compatibility. `audit_log_path` no longer describes the active storage location;
callers should use `read_audit_entries`.
## Implementation chunks
1. Add SQLite store helpers for supervise queue and audit state.
2. Rewire `bot_bottle.supervise` queue/audit functions to the store.
3. Update supervise CLI discovery tests and queue/audit unit tests.
4. Run unit tests, pyright, and pylint for touched modules.
## Open questions
None.
+227
View File
@@ -0,0 +1,227 @@
# PRD 0068: smolmachines backend on Linux
- **Status:** Active
- **Author:** Claude
- **Created:** 2026-06-25
- **Issue:** #283
## Summary
Make the `smolmachines` backend (PRD 0023) runnable on Linux, not
just macOS. `smolvm` already supports Linux via KVM (`/dev/kvm`);
the gap is entirely in bot-bottle's host-side glue, which hard-codes
macOS assumptions in three places:
1. **Preflight** only checks that `smolvm` is on `PATH` — it never
checks the Linux KVM prerequisite, so a misconfigured host fails
deep in the launch flow with an opaque `smolvm` error.
2. **The TSI allowlist enforcement** (`force_allowlist`) — the
security property that confines the agent VM to its sidecar
bundle's `/32`**no-ops on Linux today, failing _open_**. The
smolvm state-DB path it patches is hard-coded to macOS's
`~/Library/Application Support/...`.
3. **Per-bottle loopback scoping** (`allocate`) returns the shared
`127.0.0.1` on Linux, which would let the agent VM reach every
service on host loopback — a downgrade from the per-bottle alias
isolation macOS gets.
This PRD closes all three so a bottle launched with
`BOT_BOTTLE_BACKEND=smolmachines` on Linux gets the same isolation
guarantee it gets on macOS, and documents the Linux/NixOS host
setup. The primary validation target is NixOS, but the changes are
distro-agnostic.
## Problem
The smolmachines backend runs each bottle's agent inside a libkrun
microVM via `smolvm`, with egress confined by TSI's `--allow-cidr`
allowlist set to a single `/32` — the sidecar bundle's loopback
address. Everything else (host loopback, LAN, internet) is denied at
the VMM layer. That security property is the entire reason the
backend exists.
libkrun runs on Hypervisor.framework (macOS) **and** KVM (Linux), and
`smolvm` ships Linux x86_64 / aarch64 builds that require `/dev/kvm`.
So the microVM layer already works on Linux. What does not work is
bot-bottle's host integration, which PRD 0023 explicitly scoped to
macOS-only for v1. Three concrete blockers:
- **No KVM preflight.** On a Linux host without `/dev/kvm` (kernel
module not loaded) or without access to it (user not in the `kvm`
group), the failure surfaces as a cryptic `smolvm` non-zero exit
mid-launch instead of an actionable message.
- **TSI enforcement fails open on Linux.** `force_allowlist`
early-returns on non-macOS. It exists because `smolvm` 0.8.0
silently drops `--allow-cidr` when combined with `--from`, so the
allowlist has to be patched into smolvm's persisted state DB before
`machine start`. On Linux that patch never runs **and** the DB path
is the macOS path, so the booted VM's TSI allowlist is whatever
smolvm defaulted to — potentially all of `127.0.0.0/8`. That is the
exact sandbox-escape the backend is supposed to prevent.
- **No per-bottle loopback isolation on Linux.** `allocate` returns
`127.0.0.1` on Linux. Even with a correct allowlist, `127.0.0.1/32`
is shared by every service on host loopback, so the agent could
reach other bottles' published ports and host services. On macOS
this is solved with per-bottle `127.0.0.16..31` aliases added via
`sudo ifconfig lo0 alias`. On Linux the whole `127.0.0.0/8` is
already routed to `lo`, so docker can publish to `127.0.0.<N>`
with **no `ifconfig`/sudo step at all** — the isolation is actually
cheaper to achieve than on macOS.
## Goals / Success Criteria
- `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` launches,
runs, and tears down a bottle on a Linux host with `/dev/kvm`.
- The TSI allowlist is enforced on Linux: PRD 0022's
`tests/integration/test_sandbox_escape.py` passes against
`BOT_BOTTLE_BACKEND=smolmachines` on Linux (the acceptance gate).
- Each Linux bottle is scoped to its own `127.0.0.<N>/32`, matching
the macOS per-bottle isolation property.
- A clear, actionable preflight error when `/dev/kvm` is missing or
inaccessible, with remediation (load `kvm-intel`/`kvm-amd`, join the
`kvm` group).
- **Fail-closed:** if bot-bottle cannot positively confirm the TSI
allowlist was persisted for a machine (DB missing, row missing,
patch didn't take), it `die()`s before `machine start` rather than
booting a VM with an unverified allowlist.
- macOS behavior is unchanged.
- README documents Linux + NixOS host setup.
## Non-goals
- Rootless / non-KVM fallbacks (e.g. software emulation). Linux
smolmachines requires `/dev/kvm`, full stop.
- Removing Docker as a host dependency — the sidecar bundle and
image-build pipeline still use Docker on Linux, same as macOS.
- Auto-installing `smolvm` or configuring KVM on the operator's
behalf. Preflight reports; the operator remediates.
- Nested-virtualization tuning for running the runner itself inside a
VM (documented as a caveat, not solved here).
## Design
### Platform detection
Reuse the existing `platform.system()` check already in
`loopback_alias.py` (`_is_macos()`). "Linux" is "not macOS" for every
branch below; no new third-platform path.
### Preflight: KVM gate (`util.smolmachines_preflight`)
After the existing `smolvm`-on-`PATH` check, add a Linux-only gate:
- `/dev/kvm` must exist → else `die()` with "enable KVM
(`kvm-intel`/`kvm-amd` kernel module)".
- `/dev/kvm` must be readable + writable by the current user
(`os.access(..., R_OK | W_OK)`) → else `die()` with "add your user
to the `kvm` group (and re-login)".
macOS is unaffected (Hypervisor.framework needs no device node).
### smolvm state-DB path (platform-aware)
`loopback_alias._SMOLVM_DB_PATH` becomes platform-derived:
- macOS: `~/Library/Application Support/smolvm/server/smolvm.db`
(unchanged).
- Linux: `$XDG_DATA_HOME/smolvm/server/smolvm.db`, defaulting to
`~/.local/share/smolvm/server/smolvm.db`.
> **Verification note:** the Linux DB location is inferred from
> smolvm's documented `~/.local/share` install layout and the XDG
> base-dir spec. It must be confirmed on a real Linux smolvm install;
> if smolvm uses a different path or schema, the fail-closed check
> below turns that into a clear `die()` at launch rather than a silent
> escape.
### TSI enforcement: cross-platform + fail-closed (`force_allowlist`)
Rework `force_allowlist(machine_name, allowed_cidrs)` to run on
**both** platforms and to fail closed:
1. Resolve the state DB; if the file is missing, `die()` (cannot
confirm enforcement → refuse to launch).
2. Read the machine's persisted row; if the row is missing, `die()`.
3. If the row's `allowed_cidrs` already equals the requested list
(e.g. a newer `smolvm` that honors `--allow-cidr` at create), do
nothing — no write.
4. Otherwise patch `allowed_cidrs` (the existing BLOB-encoded write)
and re-read.
5. If, after the patch, `allowed_cidrs` still does not equal the
requested list, `die()`.
This is robust across smolvm versions: it works whether `--allow-cidr`
is silently dropped (0.8.0) or honored (newer), and it never boots a
VM whose persisted allowlist it could not confirm. It is a strict
improvement on macOS too (today's code writes unconditionally and
never verifies).
> The persisted-row check confirms our write took, not that smolvm's
> runtime TSI enforces it. The runtime guarantee is covered by the
> sandbox-escape acceptance test; the persisted check is the cheap
> fail-closed guard at launch.
### Per-bottle loopback scoping on Linux (`allocate`)
`allocate` runs the same docker-state-driven allocation on Linux as on
macOS (`_allocate_locked`, the file lock, and `_aliases_in_use` via
`docker inspect` are all already cross-platform). The only macOS-only
step, `ensure_pool` (the `sudo ifconfig lo0 alias` dance), stays
macOS-only: on Linux `127.0.0.0/8` is already loopback, so docker can
publish bundle ports directly on `127.0.0.<N>` with no setup.
Net effect: Linux bottles get per-bottle `127.0.0.16..31/32` scoping
identical to macOS, without sudo.
### Launch flow
`launch.py` needs no structural change — `_allocate_resources` already
calls `ensure_pool()` (now a Linux no-op) then `allocate()` (now
per-bottle on Linux), and `_launch_vm` already calls
`force_allowlist()` (now active on Linux). Only the macOS-specific
docstrings are updated to describe the cross-platform behavior.
## Implementation chunks
1. **Preflight KVM gate**`util.smolmachines_preflight` +
unit tests for the missing-device and no-access branches.
2. **Platform-aware DB path + fail-closed `force_allowlist`**
`loopback_alias.py`; update/extend `TestForceAllowlist`.
3. **Cross-platform `allocate`** — drop the Linux early-return; update
`TestAllocate` / `TestAllocateLock` for the new Linux behavior.
4. **Docstring + comment cleanup** in `launch.py` and module headers.
5. **Docs** — README requirements + a Linux/NixOS host-setup section.
## Testing Strategy
- **Unit (CI, any OS):** the suite mocks `platform.system()` /
`subprocess` and patches `_SMOLVM_DB_PATH`, so the new Linux
branches are testable on the macOS/Linux CI runner without `smolvm`
or KVM. Covers: KVM preflight branches, fail-closed `force_allowlist`
(DB missing, row missing, patch-doesn't-take), per-bottle Linux
allocation + locking, platform-derived DB path.
- **Integration (Linux host with KVM — the acceptance gate):**
`tests/integration/test_sandbox_escape.py` against
`BOT_BOTTLE_BACKEND=smolmachines`. This cannot run on the macOS dev
box and must be executed on NixOS before merge.
## Open questions / verification pending
- **Confirm the Linux smolvm state-DB path and schema** on a real
install (the `~/.local/share/...` inference above).
- **Confirm whether the current smolvm Linux build still drops
`--allow-cidr` with `--from`** (the 0.8.0 bug). The fail-closed
design handles either answer, but knowing lets us drop the DB patch
if upstream fixed it.
- **Confirm docker publishing to `127.0.0.<N>` on Linux** behaves as
expected end-to-end with TSI (high confidence; standard loopback
behavior, but unverified on the target host).
## References
- PRD 0023 — smolmachines bottle backend (macOS v1).
- PRD 0022 — `test_sandbox_escape.py` acceptance gate.
- PRD 0024 — sidecar bundle image.
- smolvm: https://github.com/smol-machines/smolvm
@@ -0,0 +1,115 @@
# PRD prd-new: smolmachines sidecar VM
- **Status:** Active
- **Author:** codex
- **Created:** 2026-07-09
- **Issue:** #332
## Summary
Run the smolmachines backend's trusted sidecar bundle as its own smolVM instead
of a Docker container. A bottle then consists of an agent VM plus a sidecar VM,
with the agent VM's TSI allowlist limited to the per-bottle agent-facing
sidecar surface.
## Problem
The smolmachines backend currently runs the agent in smolVM but keeps
egress/git-gate/supervise in a Docker sidecar bundle. That hybrid launch path
keeps Docker in the trusted runtime path and leaves smolmachines with a
different isolation boundary than its agent VM design suggests.
The existing Docker sidecar path also uses Docker port publishing to bind only
agent-facing services to the per-bottle host address. TSI is IP-only, so the
smolVM replacement must preserve that scoping: publishing sidecar services on
generic host localhost would let the agent reach unrelated host services or
other bottle sidecars if those services share the allowed address.
## Goals / Success Criteria
1. `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` runs with both the
agent and sidecar as smolVMs.
2. The smolmachines launch path no longer uses `docker run` for the sidecar
bundle.
3. Agent-facing egress, git-gate, and supervise endpoints are exposed only
through the bottle's own published sidecar surface.
4. Internal sidecar-only ports are not reachable from the agent VM.
5. The agent VM TSI allowlist remains fail-closed and limited to the per-bottle
address.
6. Sidecar config, secrets, and state are delivered to the sidecar VM without
exposing provider or forge credentials to the agent VM.
7. Teardown removes both VMs and any host-side published port state.
## Non-goals
- Rewriting the egress, git-gate, or supervise daemons.
- Changing the Docker backend's sidecar bundle behavior.
- Replacing the existing agent image build and pack pipeline except where shared
helper extraction is needed for sidecar image packing.
- Weakening the current smolmachines TSI allowlist checks.
## Design
The sidecar VM continues to use the existing sidecar bundle image and
`/app/sidecar_init.py` supervisor. The launch flow builds the sidecar image,
packs it into a `.smolmachine` artifact, creates a per-bottle sidecar VM from
that artifact, passes the same daemon-selection environment the Docker bundle
uses today, mounts or copies the same daemon-private config/state inputs, starts
the sidecar VM, wraps the sidecar VM's raw smolVM-published ports with a local
per-bottle forwarder, then starts the agent VM with
`--allow-cidr <per-bottle-address>/32`.
smolVM currently exposes guest ports as `HOST:GUEST` port pairs bound on host
loopback. It does not expose an address-scoped bind like
`127.0.0.16:<port>:9099`. Because TSI is IP-only, bot-bottle must not advertise
those raw loopback ports directly to the agent. Instead, bot-bottle runs one
small stdlib Python TCP forwarder process per bottle:
```text
agent VM
-> TSI allows only <bottle-address>/32
-> bot-bottle forwarder bound to <bottle-address>:<random-port>
-> raw smolVM-published sidecar port on 127.0.0.1:<raw-port>
-> sidecar VM service
```
Agent-facing URLs are stamped from the forwarder ports, never the raw smolVM
ports:
- `HTTP_PROXY` / `HTTPS_PROXY` point at the published egress proxy port.
- `GIT_GATE_URL` points at the published git HTTP port when git-gate is enabled.
- `MCP_SUPERVISE_URL` points at the published supervise port when supervise is
enabled.
Internal sidecar-only services stay bound inside the sidecar VM and are not
published.
### Forwarder constraints
- Reject wildcard listener binds (`0.0.0.0`, `::`) and generic localhost
listener binds (`127.0.0.1`, `::1`) for agent-facing forwards.
- Accept only fixed loopback targets selected by launch; the client cannot
choose or influence the forwarding target.
- Forward bytes transparently without parsing HTTP, CONNECT, TLS, Git, or MCP.
- Log lifecycle and endpoint metadata only; never log payload bytes.
- Create forwards only for egress, git HTTP when enabled, and supervise when
enabled.
- Fail closed if any listener cannot bind exactly the requested per-bottle
address.
## Implementation chunks
1. Extract the existing image-to-smolmachine pack helper so agent and sidecar
artifacts share the same cache and registry path.
2. Add a strict stdlib host TCP forwarder for per-bottle address-bound
forwarding to raw smolVM-published sidecar ports.
3. Add a sidecar VM launch spec and lifecycle helpers: pack, create, start,
publish/discover ports, stop, delete.
4. Switch smolmachines launch from Docker sidecar bundle lifecycle to sidecar VM
plus forwarder lifecycle.
5. Update egress apply / reload paths for the sidecar VM supervisor.
6. Add unit tests for argv shape, URL stamping, teardown, and fail-closed
behavior.
7. Add or update integration coverage for agent-to-sidecar reachability, host
localhost denial, other-bottle alias denial, internal port denial, and
teardown cleanup.
+1 -1
View File
@@ -3,5 +3,5 @@
# These tools are used for code quality checks in CI/CD. # These tools are used for code quality checks in CI/CD.
pylint>=3.0.0 pylint>=3.0.0
pyright>=1.1.300 pyright>=1.1.411
coverage>=7.0.0 coverage>=7.0.0
+3 -3
View File
@@ -81,10 +81,10 @@ class TestSandboxEscape(unittest.TestCase):
# those are missing rather than die-ing inside backend.prepare. # those are missing rather than die-ing inside backend.prepare.
backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker") backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
if backend_name == "smolmachines": if backend_name == "smolmachines":
if sys.platform != "darwin": if sys.platform not in ("darwin", "linux"):
raise unittest.SkipTest( raise unittest.SkipTest(
"BOT_BOTTLE_BACKEND=smolmachines is macOS-only in " f"BOT_BOTTLE_BACKEND=smolmachines is not supported "
"v1 (libkrun TSI)" f"on {sys.platform} (macOS and Linux only)"
) )
if shutil.which("smolvm") is None: if shutil.which("smolvm") is None:
raise unittest.SkipTest( raise unittest.SkipTest(
@@ -1,111 +0,0 @@
"""Integration: PRD 0023 chunk 2c — bundle bringup on a per-bottle
docker bridge with the pinned IP.
End-to-end against the real docker daemon. Brings up just the
sidecar bundle on its own bridge, confirms the container lands at
the pinned IP, then tears down. Skipped under act_runner (docker
socket mount topology breaks bridge visibility) and when the
bundle image isn't available.
Full launch flow (smolvm + bundle + provisioning + the
localhost-reach / egress-port-bypass probes) lives in chunk 2d."""
from __future__ import annotations
import os
import subprocess
import time
import unittest
from bot_bottle.backend.smolmachines.sidecar_bundle import (
BundleLaunchSpec,
bundle_container_name,
bundle_network_name,
create_bundle_network,
remove_bundle_network,
start_bundle,
stop_bundle,
)
from tests._docker import skip_unless_docker
@skip_unless_docker()
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: docker socket mount topology breaks "
"in-process visibility of networks created on the host daemon",
)
class TestBundleBringup(unittest.TestCase):
def setUp(self):
self.slug = f"cb-test-bundle-{os.getpid()}-{int(time.time())}"
self.network = bundle_network_name(self.slug)
self.container = bundle_container_name(self.slug)
def tearDown(self):
stop_bundle(self.slug)
remove_bundle_network(self.network)
def _bundle_image_built(self) -> bool:
"""The bundle image (`bot-bottle-sidecars:latest`) is
built lazily by the docker backend's compose. If a
smolmachines-only operator hasn't run the docker backend
first, the image won't exist locally. Skip rather than
fail."""
r = subprocess.run(
["docker", "image", "inspect", "bot-bottle-sidecars:latest"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
)
return r.returncode == 0
def test_create_network_then_start_bundle_pins_ip(self):
if not self._bundle_image_built():
self.skipTest(
"bot-bottle-sidecars:latest not built; run a docker "
"bottle first or `docker build -f Dockerfile.sidecars .`"
)
# Pick a subnet unlikely to collide on the host. Last
# octet of the slug hash isn't deterministic across runs;
# we hardcode a high octet (.211) that the docker default
# bridges almost never use.
subnet = "192.168.211.0/24"
gateway = "192.168.211.1"
bundle_ip = "192.168.211.2"
create_bundle_network(self.network, subnet, gateway)
spec = BundleLaunchSpec(
slug=self.slug,
network_name=self.network,
subnet=subnet,
gateway=gateway,
bundle_ip=bundle_ip,
# Empty daemons_csv → init exits "no daemons selected"
# immediately. We just need the container to land on
# the network at the right IP before it exits.
daemons_csv="", # empty → init exits "no daemons selected"
)
start_bundle(spec)
# Inspect the container's IP on the per-bottle network.
r = subprocess.run(
["docker", "inspect",
"--format",
"{{(index .NetworkSettings.Networks \"" + self.network + "\").IPAddress}}",
self.container],
capture_output=True, text=True, check=False,
)
# Container may have exited (no daemons selected → exit 0).
# The inspect still works on exited containers as long as
# `--rm` hasn't fired yet, which is a race. Even if it has,
# the launch succeeded — the container existed, on the
# right network, at the right IP. We don't fail here on
# missing inspect.
if r.returncode == 0 and r.stdout.strip():
self.assertEqual(bundle_ip, r.stdout.strip(),
f"bundle landed at wrong IP: {r.stdout!r}")
if __name__ == "__main__":
unittest.main()
+19 -41
View File
@@ -1,10 +1,9 @@
"""Integration: PRD 0023 chunk 2d — end-to-end launch + exec """Integration: end-to-end smolmachines launch + exec round trip.
round trip + the acceptance probes.
The smoke confirms the launch flow (per-bottle docker bridge The smoke confirms the launch flow (sidecar bundle smolVM
sidecar bundle with host-loopback published ports smolvm guest host-loopback forwarders agent smolVM with TSI allowlist exec)
with TSI allowlist exec) plumbs together end to end. The probes confirm the plumbs together end to end. The probes confirm the security
security properties the design pivot was about: properties the design pivot was about:
- **localhost-reach probe** guest tries to dial a service - **localhost-reach probe** guest tries to dial a service
bound on the host's `127.0.0.1`. TSI's per-bottle loopback bound on the host's `127.0.0.1`. TSI's per-bottle loopback
@@ -14,14 +13,7 @@ security properties the design pivot was about:
the injected `HTTPS_PROXY`/`HTTP_PROXY` URL on the per-bottle the injected `HTTPS_PROXY`/`HTTP_PROXY` URL on the per-bottle
loopback alias, while direct egress with proxy vars unset fails. loopback alias, while direct egress with proxy vars unset fails.
- **egress-port-bypass probe** guest tries to dial Gated on macOS/Linux + smolvm + docker + not GITEA_ACTIONS the
`<bundle-ip>:9099` (egress's port). TSI permits the IP but
the bundle's egress daemon binds `127.0.0.1` inside its
container, so the connect refuses at the socket level. The
bind-address mitigation is what closes TSI's port-granularity
gap.
Gated on macOS + smolvm + docker + not GITEA_ACTIONS the
runner can't host libkrun-backed VMs.""" runner can't host libkrun-backed VMs."""
from __future__ import annotations from __future__ import annotations
@@ -65,8 +57,8 @@ def _minimal_manifest() -> ManifestIndex:
@skip_unless_docker() @skip_unless_docker()
@unittest.skipUnless( @unittest.skipUnless(
platform.system() == "Darwin", platform.system() in ("Darwin", "Linux"),
"smolvm is macOS-only for v1; Linux+KVM path is a future PRD", "smolvm requires macOS or Linux",
) )
@unittest.skipUnless( @unittest.skipUnless(
_smolvm_available(), _smolvm_available(),
@@ -114,8 +106,8 @@ class TestSmolmachinesLaunch(unittest.TestCase):
def test_localhost_reach_probe(self): def test_localhost_reach_probe(self):
# Agent dials a 127.0.0.1 service on the host. TSI's # Agent dials a 127.0.0.1 service on the host. TSI's
# allowlist contains only <bundle-ip>/32, so this must # allowlist contains only the per-bottle loopback alias, so
# refuse. We use a port unlikely to be bound on the host # this must refuse. We use a port unlikely to be bound on the host
# (high-numbered) so we're confirming TSI refusal, not # (high-numbered) so we're confirming TSI refusal, not
# just "no service listening." # just "no service listening."
r = self.bottle.exec( r = self.bottle.exec(
@@ -141,7 +133,15 @@ class TestSmolmachinesLaunch(unittest.TestCase):
proxies = [line.strip() for line in r.stdout.splitlines()] proxies = [line.strip() for line in r.stdout.splitlines()]
self.assertEqual(2, len(proxies), proxies) self.assertEqual(2, len(proxies), proxies)
self.assertEqual(proxies[0], proxies[1], proxies) self.assertEqual(proxies[0], proxies[1], proxies)
self.assertTrue(proxies[0].startswith("http://127."), proxies[0]) # macOS: proxy binds to the per-bottle loopback alias (127.x.x.x) so
# TSI can intercept guest connections to it. Linux: the guest kernel
# routes 127.0.0.0/8 to its own loopback (TSI never sees those), so
# the proxy instead binds to the per-bottle bridge gateway (192.168.x.1)
# which routes via eth0 and is intercepted by TSI normally.
self.assertRegex(
proxies[0], r"^http://\d+\.\d+\.\d+\.\d+:\d+$",
"expected proxy URL to be an http://IP:port address",
)
r = self.bottle.exec( r = self.bottle.exec(
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK" "curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
@@ -188,28 +188,6 @@ class TestSmolmachinesLaunch(unittest.TestCase):
self.assertEqual(0, r.returncode, msg=r.stderr) self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertEqual(_AGENT_PROMPT, r.stdout.rstrip("\n")) self.assertEqual(_AGENT_PROMPT, r.stdout.rstrip("\n"))
def test_egress_port_bypass_probe(self):
# Agent dials <bundle-ip>:9099 (egress's port). TSI
# permits the IP, but egress will bind 127.0.0.1:9099
# inside the bundle in chunk 3, so the connect refuses
# at the socket level. NOTE: in chunk 2d the bundle's
# daemons aren't running (daemons_csv=""), so nothing
# is listening on :9099 anyway — this test asserts the
# connect fails, which is the property chunk 3 will
# preserve once egress is actually running.
r = self.bottle.exec(
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
f"curl -s --show-error --max-time 3 http://{self.plan.bundle_ip}:9099 "
"2>&1 || true"
)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected egress port refusal; got: {r.stdout!r}",
)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
@@ -2,7 +2,7 @@
exercised against the real binary. exercised against the real binary.
The full machine-lifecycle round trip (create start exec The full machine-lifecycle round trip (create start exec
delete) is gated behind macOS + Darwin platform check and lives delete) is gated behind macOS/Linux platform check and lives
in chunk 2d's smoke. This file just verifies `is_available()` in chunk 2d's smoke. This file just verifies `is_available()`
correctly reports presence and `_smolvm()` can run a no-op correctly reports presence and `_smolvm()` can run a no-op
subcommand without errors enough to flag wrapper drift if subcommand without errors enough to flag wrapper drift if
@@ -23,8 +23,8 @@ from bot_bottle.backend.smolmachines.smolvm import is_available
"skipped under act_runner: smolvm not installed on the runner", "skipped under act_runner: smolvm not installed on the runner",
) )
@unittest.skipUnless( @unittest.skipUnless(
platform.system() == "Darwin", platform.system() in ("Darwin", "Linux"),
"smolvm is macOS-only for v1; Linux+KVM path is a future PRD", "smolvm requires macOS or Linux",
) )
@unittest.skipUnless( @unittest.skipUnless(
is_available(), is_available(),
+4 -1
View File
@@ -35,7 +35,10 @@ class TestAgentProviderRuntime(unittest.TestCase):
) )
config = Path(tmp, "codex-config.toml").read_text() config = Path(tmp, "codex-config.toml").read_text()
self.assertEqual("codex", plan.template) self.assertEqual("codex", plan.template)
self.assertEqual("codex", plan.command) self.assertEqual(
"/home/node/.codex/packages/standalone/current/bin/codex",
plan.command,
)
self.assertEqual("read_prompt_file", plan.prompt_mode) self.assertEqual("read_prompt_file", plan.prompt_mode)
self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile) self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile)
self.assertEqual( self.assertEqual(
+17
View File
@@ -12,11 +12,19 @@ from unittest.mock import patch
import bot_bottle.cli as climod import bot_bottle.cli as climod
from bot_bottle.cli import main from bot_bottle.cli import main
from bot_bottle.db_store import DbStore
from bot_bottle.log import Die from bot_bottle.log import Die
from bot_bottle.manifest import ManifestError from bot_bottle.manifest import ManifestError
from bot_bottle.store_manager import StoreManager
class TestMainDispatch(unittest.TestCase): class TestMainDispatch(unittest.TestCase):
def setUp(self) -> None:
patcher = patch.object(DbStore, "is_migrated", return_value=True)
self._mock_check = patcher.start()
self.addCleanup(patcher.stop)
self.addCleanup(StoreManager.reset)
def test_no_args_prints_usage_returns_2(self) -> None: def test_no_args_prints_usage_returns_2(self) -> None:
with patch("sys.stderr", io.StringIO()): with patch("sys.stderr", io.StringIO()):
self.assertEqual(2, main([])) self.assertEqual(2, main([]))
@@ -56,6 +64,15 @@ class TestMainDispatch(unittest.TestCase):
main(["x", "a", "b"]) main(["x", "a", "b"])
self.assertEqual([["a", "b"]], seen) self.assertEqual([["a", "b"]], seen)
def test_missing_env_var_error_maps_to_1(self) -> None:
from bot_bottle.errors import MissingEnvVarError
def boom(_rest: list[str]) -> int:
raise MissingEnvVarError("MY_VAR", "MY_VAR is not set")
with patch.dict(climod.COMMANDS, {"x": boom}), patch("sys.stderr", io.StringIO()):
self.assertEqual(1, main(["x"]))
def test_manifest_error_maps_to_1(self) -> None: def test_manifest_error_maps_to_1(self) -> None:
def boom(_rest: list[str]) -> int: def boom(_rest: list[str]) -> int:
raise ManifestError("bad manifest") raise ManifestError("bad manifest")
+51
View File
@@ -9,6 +9,7 @@ is created.
from __future__ import annotations from __future__ import annotations
import io
import os import os
import unittest import unittest
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
@@ -184,5 +185,55 @@ class TestCmdStartHeadless(unittest.TestCase):
self.assertEqual("docker", self._launch_mock.call_args[1]["backend_name"]) self.assertEqual("docker", self._launch_mock.call_args[1]["backend_name"])
class TestPrepareWithPreflight(unittest.TestCase):
"""prepare_with_preflight calls render_preflight with the plan and backend name."""
def test_render_preflight_called(self):
from pathlib import Path
mock_backend = MagicMock()
mock_plan = MagicMock()
mock_backend.prepare.return_value = mock_plan
mock_backend.name = "test-backend"
render = MagicMock()
with patch("bot_bottle.cli.start.get_bottle_backend", return_value=mock_backend), \
patch("bot_bottle.cli.start._identity_from_plan", return_value="id"), \
patch("bot_bottle.cli.start.info"):
start_mod.prepare_with_preflight(
MagicMock(),
stage_dir=Path("/tmp"),
render_preflight=render,
prompt_yes=MagicMock(),
dry_run=True,
)
render.assert_called_once_with(mock_plan, "test-backend")
class TestNoAgentsDefined(unittest.TestCase):
"""cmd_start prints an error and returns 1 when the manifest has no agents."""
def test_no_agents_defined_returns_1(self):
manifest = _make_manifest([], [])
with patch(
"bot_bottle.cli.start.ManifestIndex.resolve", return_value=manifest
), patch("sys.stderr", io.StringIO()) as err:
rc = start_mod.cmd_start([])
self.assertEqual(1, rc)
self.assertIn("no agents defined", err.getvalue())
class TestTextRenderPreflight(unittest.TestCase):
"""_text_render_preflight returns a renderer that prints the backend name."""
def test_backend_name_in_output(self):
render = start_mod._text_render_preflight()
plan = MagicMock()
with patch("bot_bottle.cli.start._manifest_to_yaml", return_value=""), \
patch("sys.stderr", io.StringIO()) as err:
render(plan, "my-backend")
self.assertIn("my-backend", err.getvalue())
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+3 -4
View File
@@ -107,7 +107,7 @@ def _egress_plan(
def _supervise_plan() -> SupervisePlan: def _supervise_plan() -> SupervisePlan:
return SupervisePlan( return SupervisePlan(
slug=SLUG, slug=SLUG,
queue_dir=STATE / "supervise" / "queue", db_path=STATE / "bot-bottle.db",
internal_network=f"bot-bottle-net-{SLUG}", internal_network=f"bot-bottle-net-{SLUG}",
) )
@@ -392,7 +392,7 @@ class TestSidecarBundleShape(unittest.TestCase):
sc = self._render(supervise=True)["services"]["sidecars"] sc = self._render(supervise=True)["services"]["sidecars"]
env_strings = sc["environment"] env_strings = sc["environment"]
self.assertIn(f"SUPERVISE_BOTTLE_SLUG={SLUG}", env_strings) self.assertIn(f"SUPERVISE_BOTTLE_SLUG={SLUG}", env_strings)
self.assertTrue(any(e.startswith("SUPERVISE_QUEUE_DIR=") for e in env_strings)) self.assertIn("SUPERVISE_DB_PATH=/run/supervise/bot-bottle.db", env_strings)
self.assertTrue(any(e.startswith("SUPERVISE_PORT=") for e in env_strings)) self.assertTrue(any(e.startswith("SUPERVISE_PORT=") for e in env_strings))
def test_volumes_always_includes_egress_ca(self): def test_volumes_always_includes_egress_ca(self):
@@ -408,8 +408,7 @@ class TestSidecarBundleShape(unittest.TestCase):
self.assertIn("/etc/egress", targets) self.assertIn("/etc/egress", targets)
self.assertIn("/git-gate-entrypoint.sh", targets) self.assertIn("/git-gate-entrypoint.sh", targets)
self.assertIn("/git-gate/creds/upstream-known_hosts", targets) self.assertIn("/git-gate/creds/upstream-known_hosts", targets)
self.assertTrue(any("supervise/queue" in t or t.startswith("/run/supervise") self.assertIn("/run/supervise/bot-bottle.db", targets)
for t in targets))
def test_extra_hosts_omitted_for_git_upstreams(self): def test_extra_hosts_omitted_for_git_upstreams(self):
sc = self._render(with_git=True)["services"]["sidecars"] sc = self._render(with_git=True)["services"]["sidecars"]
+1 -1
View File
@@ -74,7 +74,7 @@ def _plan(
if supervise: if supervise:
supervise_plan = SupervisePlan( supervise_plan = SupervisePlan(
slug="demo-abc12", slug="demo-abc12",
queue_dir=Path("/tmp/queue"), db_path=Path("/tmp/bot-bottle.db"),
) )
return DockerBottlePlan( return DockerBottlePlan(
spec=spec, spec=spec,
+33 -2
View File
@@ -77,7 +77,7 @@ def _plan(
if supervise: if supervise:
supervise_plan = SupervisePlan( supervise_plan = SupervisePlan(
slug="demo-abc12", slug="demo-abc12",
queue_dir=Path("/tmp/queue"), db_path=Path("/tmp/bot-bottle.db"),
) )
return DockerBottlePlan( return DockerBottlePlan(
spec=spec, spec=spec,
@@ -261,6 +261,36 @@ class TestCodexProvision(unittest.TestCase):
self.assertTrue(any("find" in s and "-delete" in s for s in scripts)) self.assertTrue(any("find" in s and "-delete" in s for s in scripts))
self.assertTrue(any("runuser" in s and "codex login status" in s for s in scripts)) self.assertTrue(any("runuser" in s and "codex login status" in s for s in scripts))
def test_forwarded_credentials_verify_sets_codex_path(self):
with tempfile.TemporaryDirectory(prefix="bb-codex-auth.") as tmp:
state_dir = Path(tmp)
prompt_file = state_dir / "prompt.txt"
prompt_file.write_text("")
with patch(
"bot_bottle.contrib.codex.agent_provider.codex_host_access_token",
return_value="token",
), patch(
"bot_bottle.contrib.codex.agent_provider.write_codex_dummy_auth_file",
):
provision = CodexAgentProvider().provision_plan(
dockerfile="",
state_dir=state_dir,
instance_name="bot-bottle-demo-abc12",
prompt_file=prompt_file,
forward_host_credentials=True,
host_env={},
)
verify_argv = provision.verify[0].argv
self.assertTrue(any(
item.startswith("PATH=/home/node/.local/bin:")
for item in verify_argv
))
self.assertIn(
"/home/node/.codex/packages/standalone/current/bin/codex",
verify_argv,
)
def test_dies_when_dir_creation_fails(self): def test_dies_when_dir_creation_fails(self):
provision = AgentProvisionPlan( provision = AgentProvisionPlan(
template="codex", command="codex", template="codex", command="codex",
@@ -301,7 +331,8 @@ class TestCodexSuperviseMcp(unittest.TestCase):
script = bottle.exec.call_args.args[0] script = bottle.exec.call_args.args[0]
self.assertEqual("node", bottle.exec.call_args.kwargs.get("user")) self.assertEqual("node", bottle.exec.call_args.kwargs.get("user"))
self.assertEqual( self.assertEqual(
f"codex mcp add supervise --url {_URL}", "/home/node/.codex/packages/standalone/current/bin/codex "
f"mcp add supervise --url {_URL}",
script, script,
) )
+3 -3
View File
@@ -19,7 +19,7 @@ from bot_bottle.egress import (
egress_sidecar_env_entries, egress_sidecar_env_entries,
egress_token_env_map, egress_token_env_map,
) )
from bot_bottle.log import Die from bot_bottle.errors import MissingEnvVarError
from bot_bottle.manifest import ManifestIndex from bot_bottle.manifest import ManifestIndex
from bot_bottle.yaml_subset import parse_yaml_subset from bot_bottle.yaml_subset import parse_yaml_subset
@@ -499,14 +499,14 @@ class TestResolveTokenValues(unittest.TestCase):
self.assertEqual({"EGRESS_TOKEN_0": "the-value"}, out) self.assertEqual({"EGRESS_TOKEN_0": "the-value"}, out)
def test_missing_token_ref_dies(self): def test_missing_token_ref_dies(self):
with self.assertRaises(Die): with self.assertRaises(MissingEnvVarError):
egress_resolve_token_values( egress_resolve_token_values(
{"EGRESS_TOKEN_0": "GH_PAT"}, {"EGRESS_TOKEN_0": "GH_PAT"},
{}, {},
) )
def test_empty_token_ref_dies(self): def test_empty_token_ref_dies(self):
with self.assertRaises(Die): with self.assertRaises(MissingEnvVarError):
egress_resolve_token_values( egress_resolve_token_values(
{"EGRESS_TOKEN_0": "GH_PAT"}, {"EGRESS_TOKEN_0": "GH_PAT"},
{"GH_PAT": ""}, {"GH_PAT": ""},
@@ -47,7 +47,6 @@ def _addon() -> EgressAddon:
a: EgressAddon = EgressAddon.__new__(EgressAddon) a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = Config(routes=(), log=LOG_FULL) a.config = Config(routes=(), log=LOG_FULL)
a.safe_tokens = set() a.safe_tokens = set()
a._supervise_queue_dir = ""
a._supervise_slug = "" a._supervise_slug = ""
a._token_allow_timeout = 300.0 a._token_allow_timeout = 300.0
return a return a
+3 -6
View File
@@ -212,7 +212,6 @@ def _addon(config: Config) -> EgressAddon:
a: EgressAddon = EgressAddon.__new__(EgressAddon) a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = config a.config = config
a.safe_tokens = set() a.safe_tokens = set()
a._supervise_queue_dir = ""
a._supervise_slug = "" a._supervise_slug = ""
a._token_allow_timeout = 300.0 a._token_allow_timeout = 300.0
a.routes_path = "/nonexistent/routes.yaml" a.routes_path = "/nonexistent/routes.yaml"
@@ -386,10 +385,10 @@ def _fake_sv(response_status: str | None) -> types.SimpleNamespace:
def _sha256_hex(_payload: Any) -> str: def _sha256_hex(_payload: Any) -> str:
return "hash" return "hash"
def _noop(_a: Any, _b: Any) -> None: def _noop(*_args: Any) -> None:
return None return None
def _read_response(_qd: Any, _pid: Any) -> Any: def _read_response(_slug: Any, _pid: Any) -> Any:
if response_status is None: if response_status is None:
raise OSError("not written yet") # forces poll -> timeout raise OSError("not written yet") # forces poll -> timeout
return types.SimpleNamespace(status=response_status) return types.SimpleNamespace(status=response_status)
@@ -409,7 +408,6 @@ def _fake_sv(response_status: str | None) -> types.SimpleNamespace:
class TestSuperviseBranch(unittest.TestCase): class TestSuperviseBranch(unittest.TestCase):
def _supervised_addon(self) -> EgressAddon: def _supervised_addon(self) -> EgressAddon:
addon = _addon(Config(routes=(Route(host="api.example.com"),))) addon = _addon(Config(routes=(Route(host="api.example.com"),)))
addon._supervise_queue_dir = "/tmp/egress-queue"
addon._supervise_slug = "test-bottle" addon._supervise_slug = "test-bottle"
addon._token_allow_timeout = 0.05 addon._token_allow_timeout = 0.05
return addon return addon
@@ -632,14 +630,13 @@ class TestRedactSurfaces(unittest.TestCase):
class TestSuperviseWriteFailure(unittest.TestCase): class TestSuperviseWriteFailure(unittest.TestCase):
def test_write_proposal_oserror_blocks(self) -> None: def test_write_proposal_oserror_blocks(self) -> None:
addon = _addon(Config(routes=(Route(host="api.example.com"),))) addon = _addon(Config(routes=(Route(host="api.example.com"),)))
addon._supervise_queue_dir = "/tmp/egress-queue"
addon._supervise_slug = "test-bottle" addon._supervise_slug = "test-bottle"
addon._token_allow_timeout = 0.05 addon._token_allow_timeout = 0.05
flow = _Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")) flow = _Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}"))
fake = _fake_sv("approved") fake = _fake_sv("approved")
def _raise(_qd: Any, _p: Any) -> None: def _raise(_p: Any) -> None:
raise OSError("disk full") raise OSError("disk full")
fake.write_proposal = _raise fake.write_proposal = _raise
+37 -3
View File
@@ -14,10 +14,12 @@ from bot_bottle.git_gate import (
git_gate_render_access_hook, git_gate_render_access_hook,
git_gate_render_entrypoint, git_gate_render_entrypoint,
git_gate_render_hook, git_gate_render_hook,
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys, revoke_git_gate_provisioned_keys,
_resolve_identity_file, _resolve_identity_file,
git_gate_upstreams_for_bottle, git_gate_upstreams_for_bottle,
) )
from bot_bottle.errors import MissingEnvVarError
from bot_bottle.manifest import ManifestIndex from bot_bottle.manifest import ManifestIndex
from tests.fixtures import fixture_minimal, fixture_with_git from tests.fixtures import fixture_minimal, fixture_with_git
@@ -209,12 +211,23 @@ class TestHookRender(unittest.TestCase):
# the suppressed findings for human approval. # the suppressed findings for human approval.
self.assertIn("--ignore-gitleaks-allow", hook) self.assertIn("--ignore-gitleaks-allow", hook)
self.assertIn("--report-format=json", hook) self.assertIn("--report-format=json", hook)
self.assertIn('"tool": "gitleaks-allow"', hook) self.assertIn("tool=_sv.TOOL_GITLEAKS_ALLOW", hook)
self.assertIn("SUPERVISE_QUEUE_DIR", hook) self.assertIn("_sv.write_proposal", hook)
self.assertIn("_sv.read_response", hook)
self.assertIn("SUPERVISE_BOTTLE_SLUG", hook) self.assertIn("SUPERVISE_BOTTLE_SLUG", hook)
self.assertIn("supervisor approved # gitleaks:allow", hook) self.assertIn("supervisor approved # gitleaks:allow", hook)
self.assertIn("supervisor rejected # gitleaks:allow", hook) self.assertIn("supervisor rejected # gitleaks:allow", hook)
def test_inline_gitleaks_allow_python_imports_work_in_sidecar_layout(self):
hook = git_gate_render_hook()
# The sidecar image copies supervise.py flat under /app, while
# host-side tests import it through the bot_bottle package.
# Hooks execute from the bare repo directory, so the embedded
# Python must include /app and support both import layouts.
self.assertIn('PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}"', hook)
self.assertIn("import supervise as _sv", hook)
self.assertIn("from bot_bottle import supervise as _sv", hook)
def test_inline_gitleaks_allow_fails_closed_without_supervisor(self): def test_inline_gitleaks_allow_fails_closed_without_supervisor(self):
hook = git_gate_render_hook() hook = git_gate_render_hook()
self.assertIn( self.assertIn(
@@ -371,6 +384,27 @@ class TestDynamicKeyProvisioning(unittest.TestCase):
self.assertEqual("/tmp/provisioned-key", _resolve_identity_file(entry, "demo", self.stage)) self.assertEqual("/tmp/provisioned-key", _resolve_identity_file(entry, "demo", self.stage))
mock_provision.assert_called_once() mock_provision.assert_called_once()
def test_prepare_defers_gitea_key_provisioning(self):
bottle = self._gitea_manifest().bottles["dev"]
with patch("bot_bottle.git_gate_provision._provision_dynamic_key") as mock_provision:
plan = _StubGate().prepare(bottle, "demo", self.stage)
mock_provision.assert_not_called()
self.assertEqual("", plan.upstreams[0].identity_file)
def test_launch_time_helper_provisions_gitea_keys(self):
bottle = self._gitea_manifest().bottles["dev"]
plan = _StubGate().prepare(bottle, "demo", self.stage)
with patch(
"bot_bottle.git_gate_provision._provision_dynamic_key",
return_value="/tmp/provisioned-key",
) as mock_provision:
updated = provision_git_gate_dynamic_keys(bottle, plan, self.stage)
mock_provision.assert_called_once_with(bottle.git[0], "demo", self.stage)
self.assertEqual("/tmp/provisioned-key", updated.upstreams[0].identity_file)
def test_revoke_skips_non_gitea_and_missing_id_file(self): def test_revoke_skips_non_gitea_and_missing_id_file(self):
revoke_git_gate_provisioned_keys(fixture_with_git().bottles["dev"], self.stage) revoke_git_gate_provisioned_keys(fixture_with_git().bottles["dev"], self.stage)
@@ -388,7 +422,7 @@ class TestDynamicKeyProvisioning(unittest.TestCase):
def test_revoke_missing_token_raises(self): def test_revoke_missing_token_raises(self):
bottle = self._gitea_manifest().bottles["dev"] bottle = self._gitea_manifest().bottles["dev"]
(self.stage / "repo-deploy-key-id").write_text("123\n") (self.stage / "repo-deploy-key-id").write_text("123\n")
with patch.dict("os.environ", {}, clear=True), self.assertRaises(RuntimeError) as cm: with patch.dict("os.environ", {}, clear=True), self.assertRaises(MissingEnvVarError) as cm:
revoke_git_gate_provisioned_keys(bottle, self.stage) revoke_git_gate_provisioned_keys(bottle, self.stage)
self.assertIn("env var is not set", str(cm.exception)) self.assertIn("env var is not set", str(cm.exception))
+698
View File
@@ -0,0 +1,698 @@
"""Unit tests for git_gate_host_key (issue #333)."""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest.mock import MagicMock, patch
from bot_bottle.git_gate_host_key import (
add_host_key_to_frontmatter,
find_and_update_bottle_file,
find_repo_bottle_file,
prompt_tty,
fetch_host_key,
preflight_host_keys,
)
from bot_bottle.manifest import Manifest, ManifestIndex
# ---------------------------------------------------------------------------
# fetch_host_key
# ---------------------------------------------------------------------------
class TestFetchHostKey(unittest.TestCase):
def _run_result(self, stdout: str, returncode: int = 0) -> MagicMock:
r = MagicMock()
r.stdout = stdout
r.stderr = ""
r.returncode = returncode
return r
def test_returns_type_and_key(self) -> None:
stdout = "gitea.example.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA\n"
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("gitea.example.com", "22")
self.assertEqual("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA", key)
def test_non_default_port_passes_p_flag(self) -> None:
stdout = "[gitea.example.com]:30009 ssh-ed25519 AAAA\n"
with patch("subprocess.run", return_value=self._run_result(stdout)) as mock_run:
fetch_host_key("gitea.example.com", "30009")
args = mock_run.call_args[0][0]
self.assertIn("-p", args)
self.assertIn("30009", args)
def test_default_port_omits_p_flag(self) -> None:
stdout = "github.com ssh-ed25519 AAAA\n"
with patch("subprocess.run", return_value=self._run_result(stdout)) as mock_run:
fetch_host_key("github.com", "22")
args = mock_run.call_args[0][0]
self.assertNotIn("-p", args)
def test_skips_comment_lines(self) -> None:
stdout = "# comment\ngithub.com ssh-ed25519 AAAA\n"
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("github.com", "22")
self.assertEqual("ssh-ed25519 AAAA", key)
def test_skips_lines_without_three_parts(self) -> None:
# A line with only two whitespace-separated tokens is not a valid
# known_hosts entry; it should be skipped and the function should
# raise rather than return a partial result.
stdout = "malformed-line\ngithub.com ssh-ed25519 AAAA\n"
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("github.com", "22")
self.assertEqual("ssh-ed25519 AAAA", key)
def test_raises_on_empty_output(self) -> None:
with patch("subprocess.run", return_value=self._run_result("")):
with self.assertRaises(RuntimeError) as cm:
fetch_host_key("example.com", "22")
self.assertIn("no host key", str(cm.exception))
def test_prefers_ed25519_over_ecdsa(self) -> None:
stdout = (
"gitea.example.com ecdsa-sha2-nistp256 BBBBB\n"
"gitea.example.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA\n"
)
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("gitea.example.com", "22")
self.assertEqual("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA", key)
def test_falls_back_to_ecdsa_without_ed25519(self) -> None:
stdout = "gitea.example.com ecdsa-sha2-nistp256 BBBBB\n"
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("gitea.example.com", "22")
self.assertEqual("ecdsa-sha2-nistp256 BBBBB", key)
def test_falls_back_to_rsa_without_ed25519_or_ecdsa(self) -> None:
stdout = "gitea.example.com ssh-rsa AAAAB3NzaC1yc2EAAAA\n"
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("gitea.example.com", "22")
self.assertEqual("ssh-rsa AAAAB3NzaC1yc2EAAAA", key)
def test_falls_back_to_first_for_unknown_type(self) -> None:
stdout = "gitea.example.com ssh-unknown CCCCC\n"
with patch("subprocess.run", return_value=self._run_result(stdout)):
key = fetch_host_key("gitea.example.com", "22")
self.assertEqual("ssh-unknown CCCCC", key)
def test_raises_includes_stderr_when_present(self) -> None:
r = self._run_result("")
r.stderr = "connection refused"
with patch("subprocess.run", return_value=r):
with self.assertRaises(RuntimeError) as cm:
fetch_host_key("example.com", "22")
self.assertIn("connection refused", str(cm.exception))
def test_raises_on_os_error(self) -> None:
with patch("subprocess.run", side_effect=OSError("not found")):
with self.assertRaises(RuntimeError) as cm:
fetch_host_key("example.com", "22")
self.assertIn("could not launch", str(cm.exception))
def test_raises_on_timeout(self) -> None:
import subprocess
with patch("subprocess.run", side_effect=subprocess.TimeoutExpired("ssh-keyscan", 15)):
with self.assertRaises(RuntimeError) as cm:
fetch_host_key("example.com", "22")
self.assertIn("timed out", str(cm.exception))
# ---------------------------------------------------------------------------
# prompt_tty
# ---------------------------------------------------------------------------
class TestPromptTty(unittest.TestCase):
def test_reads_from_tty_when_available(self) -> None:
import io
fake_tty = io.StringIO("yes\n")
with patch("builtins.open", return_value=fake_tty), patch("sys.stderr"):
result = prompt_tty("question: ")
self.assertEqual("yes", result)
def test_falls_back_to_stdin_on_os_error(self) -> None:
import io
with patch("builtins.open", side_effect=OSError("no tty")), \
patch("sys.stdin", io.StringIO("fallback\n")), \
patch("sys.stderr"):
result = prompt_tty("question: ")
self.assertEqual("fallback", result)
# ---------------------------------------------------------------------------
# add_host_key_to_frontmatter
# ---------------------------------------------------------------------------
class TestAddHostKeyToFrontmatter(unittest.TestCase):
_FILE = """\
---
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
---
# Body text here
"""
def test_inserts_host_key_at_repo_level(self) -> None:
result = add_host_key_to_frontmatter(self._FILE, "myrepo", "ssh-ed25519 AAAA")
self.assertIn("host_key: ssh-ed25519 AAAA", result)
# Must be a sibling of `url` and `key`, not nested inside `key`
lines = result.splitlines()
hk_line = next(l for l in lines if "host_key" in l)
url_line = next(l for l in lines if "url:" in l)
self.assertEqual(
len(hk_line) - len(hk_line.lstrip()),
len(url_line) - len(url_line.lstrip()),
)
def test_preserves_body(self) -> None:
result = add_host_key_to_frontmatter(self._FILE, "myrepo", "ssh-ed25519 AAAA")
self.assertIn("# Body text here", result)
def test_preserves_other_repos(self) -> None:
text = """\
---
git-gate:
repos:
first:
url: ssh://git@host/org/first.git
key:
provider: static
path: /dev/null
second:
url: ssh://git@host/org/second.git
key:
provider: static
path: /dev/null
---
"""
result = add_host_key_to_frontmatter(text, "first", "ssh-ed25519 AAAA")
self.assertIn("host_key: ssh-ed25519 AAAA", result)
self.assertIn("second:", result)
def test_no_change_when_host_key_already_present(self) -> None:
text = self._FILE.replace(
" path: /dev/null\n",
" path: /dev/null\n host_key: ssh-ed25519 EXISTING\n",
)
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 NEW")
self.assertNotIn("NEW", result)
def test_no_change_when_repo_not_found(self) -> None:
result = add_host_key_to_frontmatter(self._FILE, "other", "ssh-ed25519 AAAA")
self.assertEqual(self._FILE, result)
def test_no_change_without_frontmatter(self) -> None:
text = "No frontmatter here\n"
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
self.assertEqual(text, result)
def test_no_change_when_no_closing_delimiter(self) -> None:
text = "---\ngit-gate:\n repos:\n myrepo:\n url: x\n"
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
self.assertEqual(text, result)
def test_no_change_when_git_gate_missing(self) -> None:
text = "---\nenv:\n FOO: bar\n---\n"
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
self.assertEqual(text, result)
def test_no_change_when_repos_missing(self) -> None:
text = "---\ngit-gate:\n foo: bar\n---\n"
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
self.assertEqual(text, result)
def test_no_change_when_repo_entry_not_a_dict(self) -> None:
text = "---\ngit-gate:\n repos:\n - item\n---\n"
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
self.assertEqual(text, result)
# ---------------------------------------------------------------------------
# find_and_update_bottle_file
# ---------------------------------------------------------------------------
class TestFindAndUpdateBottleFile(unittest.TestCase):
def _write_bottle(self, d: str, name: str, content: str) -> Path:
path = Path(d) / f"{name}.md"
path.write_text(content, encoding="utf-8")
return path
def test_updates_file_with_matching_repo(self) -> None:
with tempfile.TemporaryDirectory() as d:
content = """\
---
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
---
"""
path = self._write_bottle(d, "dev", content)
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertTrue(result)
updated = path.read_text()
self.assertIn("host_key: ssh-ed25519 AAAA", updated)
def test_returns_false_when_no_matching_file(self) -> None:
with tempfile.TemporaryDirectory() as d:
content = """\
---
git-gate:
repos:
other:
url: ssh://git@host/org/other.git
key:
provider: static
path: /dev/null
---
"""
self._write_bottle(d, "dev", content)
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
def test_skips_file_that_already_has_host_key(self) -> None:
with tempfile.TemporaryDirectory() as d:
content = """\
---
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
host_key: ssh-ed25519 EXISTING
---
"""
path = self._write_bottle(d, "dev", content)
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 NEW")
self.assertFalse(result)
self.assertNotIn("NEW", path.read_text())
def test_returns_false_for_nonexistent_dir(self) -> None:
result = find_and_update_bottle_file(
Path("/nonexistent/dir"), "myrepo", "ssh-ed25519 AAAA"
)
self.assertFalse(result)
def test_skips_file_without_git_gate_section(self) -> None:
with tempfile.TemporaryDirectory() as d:
self._write_bottle(d, "plain", "---\nenv:\n FOO: bar\n---\n")
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
def test_skips_file_with_non_dict_repos(self) -> None:
with tempfile.TemporaryDirectory() as d:
self._write_bottle(d, "bad", "---\ngit-gate:\n repos:\n - item\n---\n")
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
def test_skips_invalid_utf8_file(self) -> None:
with tempfile.TemporaryDirectory() as d:
path = Path(d) / "broken.md"
path.write_bytes(b"\xff\xfe") # invalid UTF-8
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
def test_skips_unreadable_file(self) -> None:
import os
with tempfile.TemporaryDirectory() as d:
path = Path(d) / "unreadable.md"
path.write_text("---\n---\n")
os.chmod(path, 0o000)
try:
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
finally:
os.chmod(path, 0o644)
def test_returns_false_when_file_unreadable_on_write(self) -> None:
# find_repo_bottle_file succeeds, but the second read (for writing)
# raises OSError — e.g. file removed or permissions changed.
content = """\
---
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
---
"""
with tempfile.TemporaryDirectory() as d:
path = self._write_bottle(d, "dev", content)
with patch(
"bot_bottle.git_gate_host_key.find_repo_bottle_file",
return_value=path,
), patch.object(Path, "read_text", side_effect=OSError("Permission denied")):
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
def test_skips_when_update_produces_no_change(self) -> None:
# add_host_key_to_frontmatter returns the original text unchanged
# (e.g. because the entry is absent from the parsed structure).
# find_and_update_bottle_file must continue rather than write.
content = """\
---
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
---
"""
with tempfile.TemporaryDirectory() as d:
path = self._write_bottle(d, "dev", content)
with patch(
"bot_bottle.git_gate_host_key.add_host_key_to_frontmatter",
return_value=content, # no change
):
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
self.assertFalse(result)
self.assertNotIn("host_key", path.read_text())
# ---------------------------------------------------------------------------
# find_repo_bottle_file
# ---------------------------------------------------------------------------
class TestFindRepoBottleFile(unittest.TestCase):
_CONTENT = """\
---
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
---
"""
def _write(self, d: str, name: str, content: str) -> Path:
path = Path(d) / f"{name}.md"
path.write_text(content, encoding="utf-8")
return path
def test_returns_path_when_repo_found(self) -> None:
with tempfile.TemporaryDirectory() as d:
path = self._write(d, "dev", self._CONTENT)
result = find_repo_bottle_file(Path(d), "myrepo")
self.assertEqual(path, result)
def test_returns_none_when_no_matching_file(self) -> None:
with tempfile.TemporaryDirectory() as d:
self._write(d, "dev", self._CONTENT)
self.assertIsNone(find_repo_bottle_file(Path(d), "other"))
def test_returns_none_when_host_key_already_set(self) -> None:
content = self._CONTENT.replace(" path: /dev/null\n",
" path: /dev/null\n host_key: X\n")
with tempfile.TemporaryDirectory() as d:
self._write(d, "dev", content)
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
def test_returns_none_for_nonexistent_dir(self) -> None:
self.assertIsNone(find_repo_bottle_file(Path("/nonexistent"), "myrepo"))
def test_skips_unreadable_file(self) -> None:
with tempfile.TemporaryDirectory() as d:
(Path(d) / "unreadable.md").write_text(self._CONTENT)
with patch.object(Path, "read_text", side_effect=OSError("Permission denied")):
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
def test_skips_file_with_non_dict_git_gate(self) -> None:
with tempfile.TemporaryDirectory() as d:
(Path(d) / "dev.md").write_text("---\ngit-gate:\n - item\n---\n")
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
def test_skips_file_with_non_dict_repos(self) -> None:
with tempfile.TemporaryDirectory() as d:
(Path(d) / "dev.md").write_text("---\ngit-gate:\n repos:\n - item\n---\n")
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
def test_save_prompt_includes_filename(self) -> None:
"""preflight prompt must name the discovered bottle file."""
manifest = _manifest_with_git()
with tempfile.TemporaryDirectory() as home:
bottles_dir = Path(home) / "bottles"
bottles_dir.mkdir()
bottle_file = bottles_dir / "dev.md"
bottle_file.write_text(
"---\ngit-gate:\n repos:\n myrepo:\n"
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
" key:\n provider: static\n path: /dev/null\n---\n",
encoding="utf-8",
)
mock_prompt = MagicMock(side_effect=["y", "n"]) # confirm key, skip save
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty", mock_prompt,
), patch("sys.stderr"):
preflight_host_keys(manifest, headless=False, home_md=Path(home))
save_call = next(
c for c in mock_prompt.call_args_list if "Save" in c.args[0]
)
self.assertIn(str(bottle_file), save_call.args[0])
# ---------------------------------------------------------------------------
# preflight_host_keys
# ---------------------------------------------------------------------------
def _manifest_with_git(host_key: str = "") -> Manifest:
"""Build a Manifest with one git entry; host_key defaults to empty."""
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"git-gate": {
"repos": {
"myrepo": {
"url": "ssh://git@gitea.example.com:30009/org/myrepo.git",
"key": {"provider": "static", "path": "/dev/null"},
**({"host_key": host_key} if host_key else {}),
},
},
},
},
},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}).load_for_agent("demo")
class TestPreflightHostKeys(unittest.TestCase):
def test_no_op_when_all_keys_present(self) -> None:
manifest = _manifest_with_git("ssh-ed25519 AAAA")
result = preflight_host_keys(manifest, headless=False, home_md=None)
self.assertIs(result, manifest)
def test_headless_dies_when_key_missing(self) -> None:
manifest = _manifest_with_git()
with self.assertRaises(SystemExit):
preflight_host_keys(manifest, headless=True, home_md=None)
def test_interactive_fetches_and_confirms_key(self) -> None:
manifest = _manifest_with_git()
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty",
side_effect=["y", "n"], # confirm key, don't persist
), patch("sys.stderr"):
result = preflight_host_keys(manifest, headless=False, home_md=None)
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
def test_interactive_dies_when_key_not_confirmed(self) -> None:
manifest = _manifest_with_git()
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty",
return_value="n",
), patch("sys.stderr"), self.assertRaises(SystemExit):
preflight_host_keys(manifest, headless=False, home_md=None)
def test_interactive_persists_when_requested(self) -> None:
manifest = _manifest_with_git()
with tempfile.TemporaryDirectory() as home:
bottles_dir = Path(home) / "bottles"
bottles_dir.mkdir()
bottle_file = bottles_dir / "dev.md"
bottle_file.write_text(
"---\ngit-gate:\n repos:\n myrepo:\n"
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
" key:\n provider: static\n path: /dev/null\n---\n",
encoding="utf-8",
)
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty",
side_effect=["y", "y"], # confirm key, yes persist
), patch("sys.stderr"):
result = preflight_host_keys(
manifest, headless=False, home_md=Path(home),
)
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
written = bottle_file.read_text()
self.assertIn("host_key: ssh-ed25519 FETCHED", written)
def test_interactive_skips_persist_when_declined(self) -> None:
manifest = _manifest_with_git()
with tempfile.TemporaryDirectory() as home:
bottles_dir = Path(home) / "bottles"
bottles_dir.mkdir()
bottle_file = bottles_dir / "dev.md"
bottle_file.write_text(
"---\ngit-gate:\n repos:\n myrepo:\n"
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
" key:\n provider: static\n path: /dev/null\n---\n",
encoding="utf-8",
)
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty",
side_effect=["y", "n"], # confirm key, no persist
), patch("sys.stderr"):
result = preflight_host_keys(
manifest, headless=False, home_md=Path(home),
)
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
written = bottle_file.read_text()
self.assertNotIn("host_key", written)
def test_interactive_dies_when_fetch_fails(self) -> None:
manifest = _manifest_with_git()
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
side_effect=RuntimeError("connection refused"),
), patch("sys.stderr"), self.assertRaises(SystemExit):
preflight_host_keys(manifest, headless=False, home_md=None)
def test_interactive_warns_when_persist_file_not_found(self) -> None:
manifest = _manifest_with_git()
with tempfile.TemporaryDirectory() as home:
# bottles/ dir exists but does not contain the repo entry
bottles_dir = Path(home) / "bottles"
bottles_dir.mkdir()
import io
buf = io.StringIO()
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty",
side_effect=["y"], # confirm key only; no save prompt when no file found
), patch("sys.stderr", buf):
result = preflight_host_keys(
manifest, headless=False, home_md=Path(home),
)
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
self.assertIn("kept in memory", buf.getvalue())
def test_interactive_warns_when_write_fails_after_prompt(self) -> None:
# User says yes to save, but find_and_update_bottle_file fails.
manifest = _manifest_with_git()
with tempfile.TemporaryDirectory() as home:
bottles_dir = Path(home) / "bottles"
bottles_dir.mkdir()
bottle_file = bottles_dir / "dev.md"
bottle_file.write_text(
"---\ngit-gate:\n repos:\n myrepo:\n"
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
" key:\n provider: static\n path: /dev/null\n---\n",
encoding="utf-8",
)
import io
buf = io.StringIO()
with patch(
"bot_bottle.git_gate_host_key.fetch_host_key",
return_value="ssh-ed25519 FETCHED",
), patch(
"bot_bottle.git_gate_host_key.prompt_tty",
side_effect=["y", "y"], # confirm key, yes persist
), patch(
"bot_bottle.git_gate_host_key.find_and_update_bottle_file",
return_value=False,
), patch("sys.stderr", buf):
result = preflight_host_keys(
manifest, headless=False, home_md=Path(home),
)
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
self.assertIn("kept in memory", buf.getvalue())
def test_headless_names_all_missing_repos_in_error(self) -> None:
manifest = ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"git-gate": {
"repos": {
"alpha": {
"url": "ssh://git@host/org/alpha.git",
"key": {"provider": "static", "path": "/dev/null"},
},
"beta": {
"url": "ssh://git@host/org/beta.git",
"key": {"provider": "static", "path": "/dev/null"},
},
},
},
},
},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}).load_for_agent("demo")
import io
buf = io.StringIO()
with patch("sys.stderr", buf), self.assertRaises(SystemExit):
preflight_host_keys(manifest, headless=True, home_md=None)
if __name__ == "__main__":
unittest.main()
+3 -2
View File
@@ -13,6 +13,7 @@ from pathlib import Path
from typing import Any, cast from typing import Any, cast
from unittest.mock import patch from unittest.mock import patch
from bot_bottle.errors import MissingEnvVarError
from bot_bottle.git_gate import ( from bot_bottle.git_gate import (
_gitconfig_validate_value, _gitconfig_validate_value,
_provision_dynamic_key, _provision_dynamic_key,
@@ -120,7 +121,7 @@ class TestProvisionDynamicKey(unittest.TestCase):
patch.dict("os.environ", {}, clear=False): patch.dict("os.environ", {}, clear=False):
import os import os
os.environ.pop("GITEA_TOK", None) os.environ.pop("GITEA_TOK", None)
with self.assertRaises(RuntimeError): with self.assertRaises(MissingEnvVarError):
_provision_dynamic_key(_gitea_entry(), "s", Path(d)) _provision_dynamic_key(_gitea_entry(), "s", Path(d))
@@ -166,7 +167,7 @@ class TestRevokeProvisionedKeys(unittest.TestCase):
import os import os
os.environ.pop("GITEA_TOK", None) os.environ.pop("GITEA_TOK", None)
(Path(d) / "repo-deploy-key-id").write_text("kid123") (Path(d) / "repo-deploy-key-id").write_text("kid123")
with self.assertRaises(RuntimeError): with self.assertRaises(MissingEnvVarError):
revoke_git_gate_provisioned_keys(_bottle(_gitea_entry()), Path(d)) revoke_git_gate_provisioned_keys(_bottle(_gitea_entry()), Path(d))
+13
View File
@@ -30,6 +30,19 @@ class TestMacosContainerBottle(unittest.TestCase):
argv, argv,
) )
def test_agent_argv_accepts_absolute_provider_command(self):
command = "/home/node/.codex/packages/standalone/current/bin/codex"
bottle = MacosContainerBottle(
"bot-bottle-dev-abc",
lambda: None,
None,
agent_command=command,
)
with patch.dict(bottle_mod.os.environ, {}, clear=True):
argv = bottle.agent_argv(["run"])
self.assertIn(command, argv)
self.assertNotIn("codex", argv)
def test_agent_argv_includes_workdir(self): def test_agent_argv_includes_workdir(self):
bottle = MacosContainerBottle( bottle = MacosContainerBottle(
"bot-bottle-dev-abc", "bot-bottle-dev-abc",
+4 -2
View File
@@ -71,7 +71,9 @@ def _plan(
else: else:
git_gate_plan = SimpleNamespace(upstreams=()) git_gate_plan = SimpleNamespace(upstreams=())
supervise_plan = ( supervise_plan = (
SimpleNamespace(queue_dir=Path("/state/supervise/queue")) SimpleNamespace(
db_path=Path("/state/bot-bottle.db"),
)
if supervise else None if supervise else None
) )
agent_provision = SimpleNamespace( agent_provision = SimpleNamespace(
@@ -137,7 +139,7 @@ class TestMacosContainerLaunchArgv(unittest.TestCase):
argv, argv,
) )
self.assertIn( self.assertIn(
"type=bind,source=/state/supervise/queue,target=/run/supervise/queue", "type=bind,source=/state,target=/run/supervise",
argv, argv,
) )
+26 -1
View File
@@ -9,7 +9,11 @@ from __future__ import annotations
import unittest import unittest
from bot_bottle.manifest import ManifestBottle, ManifestIndex from unittest.mock import patch
from bot_bottle.env import resolve_env
from bot_bottle.errors import MissingEnvVarError
from bot_bottle.manifest import Manifest, ManifestBottle, ManifestIndex
from bot_bottle.manifest_agent import ( from bot_bottle.manifest_agent import (
ManifestAgent, ManifestAgent,
ManifestAgentProvider, ManifestAgentProvider,
@@ -238,5 +242,26 @@ class TestEagerIndexLookups(unittest.TestCase):
self.assertIsNone(m.git_identity_summary()) self.assertIsNone(m.git_identity_summary())
class TestResolveEnv(unittest.TestCase):
"""resolve_env raises MissingEnvVarError when an interpolated entry's
host var is unset."""
def _manifest(self, env_dict: dict[str, str]) -> Manifest:
idx = ManifestIndex.from_json_obj({
"bottles": {"dev": {"env": env_dict}},
"agents": {"demo": {"bottle": "dev"}},
})
return idx.load_for_agent("demo")
def test_missing_interpolated_host_var_raises(self):
manifest = self._manifest({"API_KEY": "${HOST_API_KEY}"})
with patch.dict("os.environ", {}, clear=False):
import os
os.environ.pop("HOST_API_KEY", None)
with self.assertRaises(MissingEnvVarError) as cm:
resolve_env(manifest)
self.assertIn("HOST_API_KEY", str(cm.exception))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+15 -2
View File
@@ -79,6 +79,11 @@ class TestClaudeArgvWrapped(unittest.TestCase):
"--", "--",
"runuser", "-u", "node", "--", "runuser", "-u", "node", "--",
"env", "HOME=/home/node", "USER=node", "env", "HOME=/home/node", "USER=node",
(
"PATH=/home/node/.local/bin:"
"/home/node/.codex/packages/standalone/current/bin:"
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
),
"claude", "claude",
], ],
argv, argv,
@@ -127,16 +132,24 @@ class TestClaudeArgvWrapped(unittest.TestCase):
self.assertIn("HTTPS_PROXY=http://127.0.0.1:1234", argv) self.assertIn("HTTPS_PROXY=http://127.0.0.1:1234", argv)
self.assertIn("NO_PROXY=localhost", argv) self.assertIn("NO_PROXY=localhost", argv)
def test_guest_env_path_overrides_default_path(self):
argv = _unwrap(_bottle(None, PATH="/custom/bin").agent_argv([]))
self.assertIn("PATH=/custom/bin", argv)
self.assertFalse(any(
item.startswith("PATH=/home/node/.local/bin")
for item in argv
))
def test_runuser_switch_precedes_claude(self): def test_runuser_switch_precedes_claude(self):
# The dashboard's `_build_resume_argv_with_fallback` finds # The dashboard's `_build_resume_argv_with_fallback` finds
# the `claude` token to split exec-framing from the claude # the `claude` token to split exec-framing from the claude
# tail. `runuser -u node --` must sit on the prefix side so # tail. `runuser -u node --` must sit on the prefix side so
# the shell wrap inherits the UID switch. # the shell wrap inherits the UID switch.
argv = _bottle().agent_argv([]) argv = _bottle().agent_argv([])
agent_idx = argv.index("claude") runuser_idx = argv.index("runuser")
self.assertEqual( self.assertEqual(
["runuser", "-u", "node", "--", "env"], ["runuser", "-u", "node", "--", "env"],
argv[agent_idx - 7:agent_idx - 2], argv[runuser_idx:runuser_idx + 5],
) )
def test_pi_provider_appends_system_prompt_without_print_mode(self): def test_pi_provider_appends_system_prompt_without_print_mode(self):
@@ -0,0 +1,58 @@
from __future__ import annotations
import unittest
from unittest.mock import patch
from bot_bottle.backend.egress_apply import EgressApplyError
from bot_bottle.backend.smolmachines import egress_apply
from bot_bottle.backend.smolmachines.smolvm import SmolvmRunResult
class TestFetchCurrentRoutes(unittest.TestCase):
def test_reads_routes_from_sidecar_vm(self):
with patch.object(
egress_apply._smolvm,
"machine_exec",
return_value=SmolvmRunResult(0, "routes", ""),
) as exec_:
self.assertEqual("routes", egress_apply.fetch_current_routes("dev-abc"))
exec_.assert_called_once_with(
"bot-bottle-sidecars-dev-abc",
["cat", "/etc/egress/routes.yaml"],
)
def test_read_failure_raises_apply_error(self):
with patch.object(
egress_apply._smolvm,
"machine_exec",
return_value=SmolvmRunResult(1, "", "nope"),
):
with self.assertRaises(EgressApplyError):
egress_apply.fetch_current_routes("dev-abc")
class TestReload(unittest.TestCase):
def test_sends_hup_to_sidecar_init(self):
with patch.object(
egress_apply._smolvm,
"machine_exec",
return_value=SmolvmRunResult(0, "", ""),
) as exec_:
egress_apply.SmolmachinesEgressApplicator()._signal_bundle_reload("dev-abc")
exec_.assert_called_once_with(
"bot-bottle-sidecars-dev-abc",
["sh", "-c", "kill -HUP 1"],
)
def test_reload_failure_raises_apply_error(self):
with patch.object(
egress_apply._smolvm,
"machine_exec",
return_value=SmolvmRunResult(1, "", "nope"),
):
with self.assertRaises(EgressApplyError):
egress_apply.SmolmachinesEgressApplicator()._signal_bundle_reload("dev-abc")
if __name__ == "__main__":
unittest.main()
+66
View File
@@ -0,0 +1,66 @@
"""Unit: smolmachines active-agent enumeration."""
from __future__ import annotations
import json
import subprocess
import unittest
from unittest.mock import patch
from bot_bottle.backend.smolmachines import enumerate as _enumerate
def _completed(
stdout: str,
returncode: int = 0,
) -> subprocess.CompletedProcess: # type: ignore[type-arg]
return subprocess.CompletedProcess(
args=[],
returncode=returncode,
stdout=stdout,
stderr="",
)
class TestEnumerateActive(unittest.TestCase):
def test_skips_sidecar_vms(self):
machines = [
{"name": "bot-bottle-demo-abc12", "state": "running"},
{"name": "bot-bottle-sidecars-demo-abc12", "state": "running"},
{"name": "bot-bottle-stopped-xyz", "state": "stopped"},
{"name": "other", "state": "running"},
]
class _Metadata:
agent_name = "demo"
started_at = "2026-07-09T00:00:00Z"
label = "Demo"
color = "green"
with patch(
"bot_bottle.backend.smolmachines.enumerate.subprocess.run",
return_value=_completed(json.dumps(machines)),
), patch(
"bot_bottle.backend.smolmachines.enumerate._query_bundle_services",
return_value={"demo-abc12": ("egress",)},
), patch(
"bot_bottle.backend.smolmachines.enumerate.read_metadata",
return_value=_Metadata(),
) as read_metadata:
active = _enumerate.enumerate_active()
self.assertEqual(1, len(active))
self.assertEqual("demo-abc12", active[0].slug)
self.assertEqual(("egress",), active[0].services)
read_metadata.assert_called_once_with("demo-abc12")
def test_invalid_smolvm_output_returns_empty(self):
with patch(
"bot_bottle.backend.smolmachines.enumerate.subprocess.run",
return_value=_completed("not json"),
):
self.assertEqual([], _enumerate.enumerate_active())
if __name__ == "__main__":
unittest.main()
+5 -1
View File
@@ -94,6 +94,10 @@ class TestEnsureSmolmachine(unittest.TestCase):
calls.append(name) calls.append(name)
return _f return _f
def save_and_record(image_ref: str, path: str) -> None:
Path(path).touch()
calls.append("save")
with patch.object( with patch.object(
_launch_mod.docker_mod, "build_image", _launch_mod.docker_mod, "build_image",
side_effect=record("build"), side_effect=record("build"),
@@ -102,7 +106,7 @@ class TestEnsureSmolmachine(unittest.TestCase):
return_value=f"sha256:{digest}fffffffffffffffff", return_value=f"sha256:{digest}fffffffffffffffff",
), patch.object( ), patch.object(
_launch_mod.docker_mod, "save", _launch_mod.docker_mod, "save",
side_effect=record("save"), side_effect=save_and_record,
) as save, patch.object( ) as save, patch.object(
_launch_mod, "ephemeral_registry", _launch_mod, "ephemeral_registry",
return_value=_Reg(), return_value=_Reg(),
+119 -16
View File
@@ -8,6 +8,7 @@ inspecting running bundle containers' port bindings."""
from __future__ import annotations from __future__ import annotations
import json import json
import os
import sqlite3 import sqlite3
import subprocess import subprocess
import tempfile import tempfile
@@ -112,9 +113,16 @@ class TestEnsurePool(unittest.TestCase):
class TestAllocate(unittest.TestCase): class TestAllocate(unittest.TestCase):
def test_returns_loopback_on_linux(self): def test_per_bottle_alias_on_linux(self):
with patch.object(loopback_alias, "_is_macos", return_value=False): # Linux gets the same per-bottle scoping as macOS (127/8 is
self.assertEqual("127.0.0.1", loopback_alias.allocate("demo")) # already loopback, so no ifconfig is needed). A fresh host
# with no running bundles allocates the first pool entry.
with tempfile.TemporaryDirectory() as tmp:
lock_path = Path(tmp) / "smolmachines.lock"
with patch.object(loopback_alias, "_is_macos", return_value=False), \
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
patch.object(loopback_alias, "_aliases_in_use", return_value=set()):
self.assertEqual("127.0.0.16", loopback_alias.allocate("demo"))
def test_picks_lowest_unused_on_macos(self): def test_picks_lowest_unused_on_macos(self):
# No bundles running -> first pool entry. # No bundles running -> first pool entry.
@@ -166,12 +174,25 @@ class TestAllocateLock(unittest.TestCase):
self.assertIn(fcntl_mod.LOCK_EX, flock_calls) self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
def test_no_lock_on_linux(self): def test_acquires_exclusive_lock_on_linux(self):
# Linux early-returns before touching the lock file. # Linux allocates per-bottle too, so it must take the same
with patch.object(loopback_alias, "_is_macos", return_value=False), \ # lock to serialise concurrent launches.
patch.object(loopback_alias.fcntl, "flock") as flock: import fcntl as fcntl_mod
loopback_alias.allocate("demo") flock_calls: list[int] = []
flock.assert_not_called()
def record_flock(fd, op): # type: ignore
flock_calls.append(op)
with tempfile.TemporaryDirectory() as tmp:
lock_path = Path(tmp) / "smolmachines.lock"
with patch.object(loopback_alias, "_is_macos", return_value=False), \
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
patch.object(loopback_alias, "_aliases_in_use", return_value=set()), \
patch.object(loopback_alias.fcntl, "flock",
side_effect=record_flock):
loopback_alias.allocate("demo")
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
def test_sequential_allocations_with_shared_lock_are_serialised(self): def test_sequential_allocations_with_shared_lock_are_serialised(self):
# Two sequential calls share the same lock file. The second # Two sequential calls share the same lock file. The second
@@ -241,10 +262,12 @@ class TestAliasInUseDetection(unittest.TestCase):
class TestForceAllowlist(unittest.TestCase): class TestForceAllowlist(unittest.TestCase):
"""Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`, """Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`, so
so `force_allowlist` opens the state DB directly and sets `force_allowlist` opens the state DB directly and sets the row's
the row's `allowed_cidrs` field. Round-trip tests against a `allowed_cidrs` field on both macOS and Linux. It is
real SQLite DB to lock down the BLOB encoding.""" fail-closed: it dies rather than launching a VM whose allowlist
it can't confirm. Round-trip tests against a real SQLite DB to
lock down the BLOB encoding."""
def setUp(self): def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="smolvm-db.") self._tmp = tempfile.TemporaryDirectory(prefix="smolvm-db.")
@@ -290,17 +313,67 @@ class TestForceAllowlist(unittest.TestCase):
self.assertEqual(4, cfg["cpus"]) self.assertEqual(4, cfg["cpus"])
self.assertTrue(cfg["network"]) self.assertTrue(cfg["network"])
def test_noop_on_linux(self): def test_patches_on_linux_too(self):
# force_allowlist no longer no-ops on Linux — the TSI
# allowlist must be enforced there as well.
with patch.object(loopback_alias, "_is_macos", return_value=False), \ with patch.object(loopback_alias, "_is_macos", return_value=False), \
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db): patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"]) loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
# DB row should be untouched.
con = sqlite3.connect(str(self.db)) con = sqlite3.connect(str(self.db))
cfg = json.loads(con.execute( cfg = json.loads(con.execute(
"SELECT data FROM vms WHERE name='demo-vm'", "SELECT data FROM vms WHERE name='demo-vm'",
).fetchone()[0]) ).fetchone()[0])
con.close() con.close()
self.assertIsNone(cfg["allowed_cidrs"]) self.assertEqual(["127.0.0.16/32"], cfg["allowed_cidrs"])
def test_skips_write_when_already_matching(self):
# A newer smolvm that honors --allow-cidr at create leaves the
# row already correct; force_allowlist must not rewrite it. We
# detect a no-write by comparing the raw BLOB byte-for-byte
# (a rewrite re-serialises the JSON, changing key order/bytes
# is not guaranteed, but mtime/identity isn't observable — so
# we assert the stored bytes are exactly what we pre-seeded).
seeded = json.dumps({
"name": "demo-vm", "cpus": 4, "mem": 8192,
"network": True, "allowed_cidrs": ["127.0.0.16/32"],
}).encode()
con = sqlite3.connect(str(self.db))
con.execute(
"UPDATE vms SET data=? WHERE name='demo-vm'",
(sqlite3.Binary(seeded),),
)
con.commit()
con.close()
with patch.object(loopback_alias, "_is_macos", return_value=True), \
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
con = sqlite3.connect(str(self.db))
stored = con.execute(
"SELECT data FROM vms WHERE name='demo-vm'").fetchone()[0]
con.close()
self.assertEqual(seeded, bytes(stored))
def test_dies_when_patch_does_not_take(self):
# If the persisted allowlist still doesn't match after the
# patch (e.g. wrong schema / smolvm stores it elsewhere),
# force_allowlist must fail closed rather than boot the VM.
original = loopback_alias._read_machine_cfg
def stale_cfg(con: sqlite3.Connection, name: str) -> dict[str, object]:
# Always report the un-patched row so the post-write
# verification never sees the requested cidrs.
cfg = original(con, name)
cfg["allowed_cidrs"] = None
return cfg
with patch.object(loopback_alias, "_is_macos", return_value=True), \
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db), \
patch.object(loopback_alias, "_read_machine_cfg", side_effect=stale_cfg), \
patch.object(loopback_alias, "die", side_effect=SystemExit("die")):
with self.assertRaises(SystemExit):
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
def test_dies_on_missing_db(self): def test_dies_on_missing_db(self):
with patch.object(loopback_alias, "_is_macos", return_value=True), \ with patch.object(loopback_alias, "_is_macos", return_value=True), \
@@ -323,5 +396,35 @@ class TestForceAllowlist(unittest.TestCase):
loopback_alias.force_allowlist("not-in-db", ["127.0.0.16/32"]) loopback_alias.force_allowlist("not-in-db", ["127.0.0.16/32"])
class TestSmolvmDbPath(unittest.TestCase):
"""The smolvm state-DB path is platform-derived: Application
Support on macOS, XDG data dir on Linux."""
def test_macos_path(self):
with patch.object(loopback_alias.platform, "system", return_value="Darwin"):
p = loopback_alias._smolvm_db_path()
self.assertEqual(
("Library", "Application Support", "smolvm", "server", "smolvm.db"),
p.parts[-5:],
)
def test_linux_default_xdg_path(self):
env = {k: v for k, v in os.environ.items() if k != "XDG_DATA_HOME"}
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
patch.dict(loopback_alias.os.environ, env, clear=True):
p = loopback_alias._smolvm_db_path()
self.assertEqual(
(".local", "share", "smolvm", "server", "smolvm.db"),
p.parts[-5:],
)
def test_linux_respects_xdg_data_home(self):
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
patch.dict(loopback_alias.os.environ,
{"XDG_DATA_HOME": "/custom/data"}, clear=False):
p = loopback_alias._smolvm_db_path()
self.assertEqual(Path("/custom/data/smolvm/server/smolvm.db"), p)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
@@ -0,0 +1,325 @@
from __future__ import annotations
import json
import socket
import subprocess
import threading
import unittest
from unittest.mock import MagicMock, patch
from bot_bottle.backend.smolmachines.port_forward import (
ForwardSpec,
ForwarderHandle,
_CompletedProcess,
_Forwarder,
_pipe,
main,
start_forwarder,
stop_forwarder,
)
def _echo_server() -> tuple[socket.socket, int]:
listener = socket.create_server(("127.0.0.1", 0))
port = int(listener.getsockname()[1])
def run() -> None:
with listener:
conn, _ = listener.accept()
with conn:
while True:
data = conn.recv(65536)
if not data:
return
conn.sendall(data)
threading.Thread(target=run, daemon=True).start()
return listener, port
class TestForwarderValidation(unittest.TestCase):
def test_empty_specs_returns_noop_handle(self):
handle = start_forwarder(())
self.assertEqual((), handle.forwards)
self.assertEqual(0, handle.process.poll())
stop_forwarder(handle)
def test_rejects_generic_localhost_listener(self):
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec(
label="egress",
listen_host="127.0.0.1",
listen_port=0,
target_host="127.0.0.1",
target_port=9099,
),
))
def test_rejects_wildcard_listener(self):
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec(
label="egress",
listen_host="0.0.0.0",
listen_port=0,
target_host="127.0.0.1",
target_port=9099,
),
))
def test_rejects_non_loopback_target(self):
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec(
label="egress",
listen_host="127.0.0.16",
listen_port=0,
target_host="192.0.2.10",
target_port=9099,
),
))
def test_rejects_invalid_ports(self):
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec("bad-listen", "127.0.0.2", 70000, "127.0.0.1", 9099),
))
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec("bad-target", "127.0.0.2", 0, "127.0.0.1", 0),
))
def test_completed_process_methods_are_noops(self):
proc = _CompletedProcess()
proc.terminate()
proc.kill()
self.assertEqual(0, proc.poll())
self.assertEqual(0, proc.wait(timeout=1))
class _FakeStdout:
def __init__(self, line: str):
self.line = line
self.closed = False
def readline(self) -> str:
return self.line
def close(self) -> None:
self.closed = True
class TestForwarderProcess(unittest.TestCase):
def test_start_parses_ready_payload(self):
payload = {
"forwards": [{
"label": "egress",
"listen_host": "127.0.0.2",
"listen_port": 61000,
"target_host": "127.0.0.1",
"target_port": 55000,
}],
}
proc = MagicMock()
proc.stdout = _FakeStdout(json.dumps(payload) + "\n")
with patch(
"bot_bottle.backend.smolmachines.port_forward.subprocess.Popen",
return_value=proc,
) as popen:
handle = start_forwarder((
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 55000),
))
self.assertIs(proc, handle.process)
self.assertEqual(61000, handle.forwards[0].listen_port)
self.assertTrue(proc.stdout.closed)
argv = popen.call_args.args[0]
self.assertIn("--spec-json", argv)
def test_start_dies_when_helper_exits_before_ready(self):
proc = MagicMock()
proc.stdout = _FakeStdout("")
with patch(
"bot_bottle.backend.smolmachines.port_forward.subprocess.Popen",
return_value=proc,
):
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 55000),
))
proc.terminate.assert_called_once()
proc.wait.assert_called_once_with(timeout=5)
def test_start_dies_on_invalid_ready_payload(self):
proc = MagicMock()
proc.stdout = _FakeStdout('{"wrong": []}\n')
with patch(
"bot_bottle.backend.smolmachines.port_forward.subprocess.Popen",
return_value=proc,
):
with self.assertRaises(SystemExit):
start_forwarder((
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 55000),
))
proc.terminate.assert_called_once()
proc.wait.assert_called_once_with(timeout=5)
def test_stop_kills_process_after_timeout(self):
proc = MagicMock()
proc.poll.return_value = None
proc.wait.side_effect = [subprocess.TimeoutExpired("cmd", 5), 0]
stop_forwarder(ForwarderHandle(process=proc, forwards=()))
proc.terminate.assert_called_once()
proc.kill.assert_called_once()
self.assertEqual(2, proc.wait.call_count)
class TestForwarding(unittest.TestCase):
def test_forwards_bytes_both_directions(self):
try:
probe = socket.create_server(("127.0.0.2", 0))
except OSError:
self.skipTest("127.0.0.2 is not bindable on this host")
else:
probe.close()
_listener, target_port = _echo_server()
handle = start_forwarder((
ForwardSpec(
label="egress",
listen_host="127.0.0.2",
listen_port=0,
target_host="127.0.0.1",
target_port=target_port,
),
))
try:
bound = handle.forwards[0]
with socket.create_connection((bound.listen_host, bound.listen_port)) as sock:
sock.sendall(b"hello")
self.assertEqual(b"hello", sock.recv(5))
finally:
stop_forwarder(handle)
class TestForwarderInProcess(unittest.TestCase):
def test_forwarder_serves_and_stops_in_process(self):
_listener, target_port = _echo_server()
forwarder = _Forwarder((
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", target_port),
))
forwarder.start()
thread = threading.Thread(target=forwarder.serve, daemon=True)
thread.start()
try:
bound = forwarder.bound[0]
with socket.create_connection((bound.listen_host, bound.listen_port)) as sock:
sock.sendall(b"ping")
self.assertEqual(b"ping", sock.recv(4))
finally:
forwarder.stop()
thread.join(timeout=2)
self.assertFalse(thread.is_alive())
def test_forwarder_logs_target_connect_failure(self):
forwarder = _Forwarder((
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 1),
))
forwarder.start()
thread = threading.Thread(target=forwarder.serve, daemon=True)
thread.start()
try:
bound = forwarder.bound[0]
with patch("bot_bottle.backend.smolmachines.port_forward.warn") as warn:
with socket.create_connection((bound.listen_host, bound.listen_port)):
pass
for _ in range(100):
if warn.called:
break
threading.Event().wait(0.01)
self.assertTrue(warn.called)
finally:
forwarder.stop()
thread.join(timeout=2)
def test_pipe_stops_when_recv_fails(self):
src, dst = socket.socketpair()
src.close()
try:
_pipe(src, dst)
finally:
dst.close()
def test_pipe_stops_when_send_fails(self):
src, src_peer = socket.socketpair()
_dst_peer, dst = socket.socketpair()
dst.close()
try:
src_peer.sendall(b"hello")
src_peer.shutdown(socket.SHUT_WR)
_pipe(src, dst)
finally:
src.close()
src_peer.close()
_dst_peer.close()
class TestMain(unittest.TestCase):
def test_main_starts_serves_and_stops_forwarder(self):
calls: list[str] = []
handlers = []
class FakeForwarder:
def __init__(self, specs): # type: ignore[no-untyped-def]
self.specs = specs
self.bound = specs
def start(self) -> None:
calls.append("start")
def serve(self) -> None:
calls.append("serve")
def stop(self) -> None:
calls.append("stop")
def record_signal(_signum, handler): # type: ignore[no-untyped-def]
handlers.append(handler)
spec = ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 9099)
with patch(
"bot_bottle.backend.smolmachines.port_forward._Forwarder",
FakeForwarder,
), patch(
"bot_bottle.backend.smolmachines.port_forward.signal.signal",
side_effect=record_signal,
), patch("builtins.print") as print_mock:
self.assertEqual(
0,
main(("--spec-json", json.dumps([{
"label": spec.label,
"listen_host": spec.listen_host,
"listen_port": spec.listen_port,
"target_host": spec.target_host,
"target_port": spec.target_port,
}]))),
)
handlers[0](15, None)
self.assertEqual(["start", "serve", "stop", "stop"], calls)
self.assertIn("forwards", print_mock.call_args.args[0])
if __name__ == "__main__":
unittest.main()
+157 -9
View File
@@ -9,6 +9,7 @@ from __future__ import annotations
import subprocess import subprocess
import tempfile import tempfile
import unittest import unittest
from contextlib import ExitStack
from dataclasses import replace from dataclasses import replace
from pathlib import Path from pathlib import Path
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
@@ -27,6 +28,7 @@ from bot_bottle.backend.smolmachines.bottle_plan import (
SmolmachinesBottlePlan, SmolmachinesBottlePlan,
) )
from bot_bottle.backend.smolmachines import launch as _launch from bot_bottle.backend.smolmachines import launch as _launch
from bot_bottle.backend.smolmachines import port_forward as _forward
from bot_bottle.backend.smolmachines.launch import _bundle_launch_spec from bot_bottle.backend.smolmachines.launch import _bundle_launch_spec
from bot_bottle.backend.util import AGENT_CA_PATH from bot_bottle.backend.util import AGENT_CA_PATH
from bot_bottle.egress import EgressPlan, EgressRoute from bot_bottle.egress import EgressPlan, EgressRoute
@@ -130,7 +132,7 @@ def _plan(
if supervise: if supervise:
supervise_plan = SupervisePlan( supervise_plan = SupervisePlan(
slug="demo-abc12", slug="demo-abc12",
queue_dir=Path("/tmp/queue"), db_path=Path("/tmp/bot-bottle.db"),
) )
return SmolmachinesBottlePlan( return SmolmachinesBottlePlan(
spec=spec, spec=spec,
@@ -276,8 +278,8 @@ class TestProvisionCA(unittest.TestCase):
str(self.egress_ca), str(self.egress_ca),
AGENT_CA_PATH, AGENT_CA_PATH,
) )
bottle.exec.assert_called_once() self.assertEqual(2, bottle.exec.call_count)
script = bottle.exec.call_args.args[0] script = bottle.exec.call_args_list[1].args[0]
self.assertIn("chmod 644", script) self.assertIn("chmod 644", script)
self.assertIn("update-ca-certificates", script) self.assertIn("update-ca-certificates", script)
self.assertEqual("root", bottle.exec.call_args.kwargs.get("user")) self.assertEqual("root", bottle.exec.call_args.kwargs.get("user"))
@@ -422,14 +424,17 @@ class TestBundleLaunchSpec(unittest.TestCase):
spec.environment, spec.environment,
) )
def test_supervise_adds_daemon_volume_and_env(self):
from bot_bottle.supervise import DB_PATH_IN_CONTAINER
plan = _plan(supervise=True)
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
self.assertIn("supervise", spec.daemons_csv)
self.assertIn(f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}", spec.environment)
self.assertIn(("/tmp/bot-bottle.db", DB_PATH_IN_CONTAINER, False), spec.volumes)
def test_canary_env_visible_to_smolvm_guest(self): def test_canary_env_visible_to_smolvm_guest(self):
plan = _plan(canary=True) plan = _plan(canary=True)
with patch.object( stamped = _launch._discover_urls(plan, "127.0.0.16", {9099: 65000})
_launch._bundle,
"bundle_host_port",
return_value="65000",
):
stamped = _launch._discover_urls(plan, "127.0.0.16")
self.assertEqual( self.assertEqual(
"fake-canary-value", "fake-canary-value",
@@ -498,5 +503,148 @@ class TestProvisionGitUser(unittest.TestCase):
self.assertIn("bot@example.com", calls[0][0]) self.assertIn("bot@example.com", calls[0][0])
class TestLaunchResourceWiring(unittest.TestCase):
def test_allocate_resources_uses_loopback_alias_and_bundle_name(self):
plan = _plan()
with patch(
"bot_bottle.backend.smolmachines.launch._loopback.ensure_pool",
) as ensure_pool, patch(
"bot_bottle.backend.smolmachines.launch._loopback.allocate",
return_value="127.0.0.16",
) as allocate:
loopback_ip, network = _launch._allocate_resources(plan, ExitStack())
ensure_pool.assert_called_once_with()
allocate.assert_called_once_with("demo-abc12")
self.assertEqual("127.0.0.16", loopback_ip)
self.assertEqual("bot-bottle-bundle-demo-abc12", network)
def test_start_bundle_wraps_raw_vm_ports_with_forwarder_ports(self):
plan = _plan(supervise=True)
plan = replace(
plan,
git_gate_plan=replace(
plan.git_gate_plan,
upstreams=(GitGateUpstream(
name="bot-bottle",
upstream_url="ssh://git@host/repo.git",
upstream_host="host",
upstream_port="22",
identity_file="/tmp/key",
known_host_key="",
),),
),
)
stack = MagicMock(spec=ExitStack)
raw_launch = MagicMock(raw_ports={9099: 55001, 9420: 55002, 9100: 55003})
handle = _forward.ForwarderHandle(
process=MagicMock(),
forwards=(
_forward.ForwardSpec("egress", "127.0.0.16", 65001, "127.0.0.1", 55001),
_forward.ForwardSpec("git-http", "127.0.0.16", 65002, "127.0.0.1", 55002),
_forward.ForwardSpec("supervise", "127.0.0.16", 65003, "127.0.0.1", 55003),
),
)
with patch(
"bot_bottle.backend.smolmachines.launch._provision_git_gate_keys",
return_value=plan,
), patch(
"bot_bottle.backend.smolmachines.launch._ensure_smolmachine",
return_value=Path("/cache/sidecar.smolmachine"),
) as ensure, patch(
"bot_bottle.backend.smolmachines.launch._bundle.start_bundle_vm",
return_value=raw_launch,
) as start_vm, patch(
"bot_bottle.backend.smolmachines.launch._forward.start_forwarder",
return_value=handle,
) as start_forwarder:
stamped = _launch._start_bundle(plan, "net", "127.0.0.16", stack)
ensure.assert_called_once()
start_vm.assert_called_once()
self.assertEqual(Path("/cache/sidecar.smolmachine"), start_vm.call_args.kwargs["from_path"])
specs = start_forwarder.call_args.args[0]
self.assertEqual(
(
_forward.ForwardSpec("egress", "127.0.0.16", 0, "127.0.0.1", 55001),
_forward.ForwardSpec("git-http", "127.0.0.16", 0, "127.0.0.1", 55002),
_forward.ForwardSpec("supervise", "127.0.0.16", 0, "127.0.0.1", 55003),
),
specs,
)
self.assertEqual("http://127.0.0.16:65001", stamped.agent_proxy_url)
self.assertEqual("127.0.0.16:65002", stamped.agent_git_gate_host)
self.assertEqual("http://127.0.0.16:65003/", stamped.agent_supervise_url)
self.assertEqual(2, stack.callback.call_count)
def test_init_vm_repairs_ownership_then_waits_for_exec(self):
plan = _plan()
with patch(
"bot_bottle.backend.smolmachines.launch._smolvm.machine_exec",
) as machine_exec, patch(
"bot_bottle.backend.smolmachines.launch._smolvm.wait_exec_ready",
) as wait_exec_ready:
_launch._init_vm(plan)
machine_exec.assert_called_once()
argv = machine_exec.call_args.args[1]
self.assertEqual(["sh", "-c"], argv[:2])
self.assertIn("chown -R node:node /home/node", argv[2])
wait_exec_ready.assert_called_once_with(plan.machine_name)
class TestPortLabels(unittest.TestCase):
def test_known_and_dynamic_port_labels_round_trip(self):
for port, label in (
(9099, "egress"),
(9420, "git-http"),
(9100, "supervise"),
(12345, "port-12345"),
):
self.assertEqual(label, _launch._label_for_port(port))
self.assertEqual(port, _launch._port_for_label(label))
def test_unknown_label_is_rejected(self):
with self.assertRaises(ValueError):
_launch._port_for_label("sidecar")
class TestDiscoverUrls(unittest.TestCase):
"""_discover_urls stamps git-gate host + supervise URL into the plan."""
def test_git_gate_host_set_when_upstreams_present(self):
plan = _plan()
plan = replace(
plan,
git_gate_plan=replace(
plan.git_gate_plan,
upstreams=(GitGateUpstream(
name="bot-bottle",
upstream_url="ssh://git@host/repo.git",
upstream_host="host",
upstream_port="22",
identity_file="/tmp/key",
known_host_key="",
),),
),
)
stamped = _launch._discover_urls(
plan,
"127.0.0.16",
{9099: 65000, 9420: 65001},
)
self.assertEqual("127.0.0.16:65001", stamped.agent_git_gate_host)
def test_supervise_url_set_when_supervise_present(self):
plan = _plan(supervise=True)
stamped = _launch._discover_urls(
plan,
"127.0.0.16",
{9099: 65000, 9100: 55556},
)
self.assertEqual("http://127.0.0.16:55556/", stamped.agent_supervise_url)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+91 -155
View File
@@ -1,9 +1,4 @@
"""Unit: bundle bringup primitives for the smolmachines backend """Unit: bundle bringup primitives for the smolmachines backend."""
(PRD 0023 chunk 2c).
Tests mock `subprocess.run` and assert on the docker argv shape.
The end-to-end integration smoke (real docker daemon, real
bundle image) lands in chunk 2d."""
from __future__ import annotations from __future__ import annotations
@@ -14,28 +9,15 @@ from unittest.mock import patch
from bot_bottle.backend.smolmachines.sidecar_bundle import ( from bot_bottle.backend.smolmachines.sidecar_bundle import (
BundleLaunchSpec, BundleLaunchSpec,
allocate_raw_host_ports,
bundle_container_name, bundle_container_name,
bundle_network_name, bundle_network_name,
create_bundle_network,
ensure_bundle_image, ensure_bundle_image,
remove_bundle_network, start_bundle_vm,
start_bundle, stop_bundle_vm,
stop_bundle,
) )
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess: # type: ignore
return subprocess.CompletedProcess(
args=[], returncode=0, stdout=stdout, stderr=stderr,
)
def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
return subprocess.CompletedProcess(
args=[], returncode=1, stdout="", stderr=stderr,
)
def _spec(**kwargs) -> BundleLaunchSpec: # type: ignore def _spec(**kwargs) -> BundleLaunchSpec: # type: ignore
defaults = dict( defaults = dict(
slug="demo-abc12", slug="demo-abc12",
@@ -68,123 +50,6 @@ class TestNamingHelpers(unittest.TestCase):
) )
class TestNetworkLifecycle(unittest.TestCase):
def _patch_run(self, **kwargs): # type: ignore
return patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
**kwargs,
)
def test_create_argv_explicit_subnet_and_gateway(self):
with self._patch_run(return_value=_ok()) as m:
create_bundle_network("nn", "192.168.50.0/24", "192.168.50.1")
self.assertEqual(
["docker", "network", "create",
"--subnet", "192.168.50.0/24",
"--gateway", "192.168.50.1",
"nn"],
m.call_args.args[0],
)
def test_create_treats_existing_network_as_success(self):
with self._patch_run(return_value=_fail("network nn already exists")):
# No SystemExit.
create_bundle_network("nn", "192.168.50.0/24", "192.168.50.1")
def test_create_other_failure_is_fatal(self):
with self._patch_run(return_value=_fail("invalid subnet")):
with self.assertRaises(SystemExit):
create_bundle_network("nn", "bogus", "bogus")
def test_remove_missing_network_is_idempotent(self):
# No SystemExit / no warn-and-continue noise; missing
# network is the expected case during a partial teardown.
with self._patch_run(return_value=_fail("Error: No such network: nn")):
remove_bundle_network("nn")
def test_remove_clean_returns_success(self):
with self._patch_run(return_value=_ok()):
remove_bundle_network("nn")
class TestStartBundle(unittest.TestCase):
def _patch_run(self):
return patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
return_value=_ok(),
)
def test_argv_pins_ip_on_network(self):
with self._patch_run() as m:
start_bundle(_spec())
argv = m.call_args.args[0]
# --network NETNAME --ip <bundle-ip> on the docker run.
self.assertIn("--network", argv)
self.assertIn("bot-bottle-bundle-demo-abc12", argv)
self.assertIn("--ip", argv)
self.assertIn("192.168.50.2", argv)
# Detached and auto-removed.
self.assertIn("--detach", argv)
self.assertIn("--rm", argv)
# Container name uses the per-slug bundle prefix.
i = argv.index("--name")
self.assertEqual("bot-bottle-sidecars-demo-abc12", argv[i + 1])
# Image at the end.
self.assertEqual("bot-bottle-sidecars:latest", argv[-1])
def test_daemons_env_passed_in(self):
with self._patch_run() as m:
start_bundle(_spec(daemons_csv="egress,supervise"))
argv = m.call_args.args[0]
self.assertIn("-e", argv)
self.assertIn(
"BOT_BOTTLE_SIDECAR_DAEMONS=egress,supervise",
argv,
)
def test_environment_entries_pass_through(self):
with self._patch_run() as m:
start_bundle(_spec(environment=(
"SUPERVISE_BOTTLE_SLUG=demo-abc12",
"EGRESS_TOKEN_0", # bare-name → host env inherit
)))
argv = m.call_args.args[0]
self.assertIn("SUPERVISE_BOTTLE_SLUG=demo-abc12", argv)
self.assertIn("EGRESS_TOKEN_0", argv)
def test_volumes_render_with_ro_flag(self):
with self._patch_run() as m:
start_bundle(_spec(volumes=(
("/host/egress-ca.pem", "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem", True),
("/host/queue", "/run/supervise/queue", False),
)))
argv = m.call_args.args[0]
self.assertIn(
"/host/egress-ca.pem:/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem:ro",
argv,
)
self.assertIn("/host/queue:/run/supervise/queue", argv)
def test_failure_dies(self):
with patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
return_value=_fail("invalid mount"),
):
with self.assertRaises(SystemExit):
start_bundle(_spec())
def test_host_env_inherited_to_subprocess(self):
# Bare-name entries in spec.environment rely on the docker
# subprocess being run with the host env. Confirm `env=`
# threads through.
with patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
return_value=_ok(),
) as m:
start_bundle(_spec(), env={"FOO": "bar"})
self.assertEqual({"FOO": "bar"}, m.call_args.kwargs["env"])
class TestEnsureBundleImage(unittest.TestCase): class TestEnsureBundleImage(unittest.TestCase):
def test_builds_sidecar_dockerfile_before_plain_docker_run(self): def test_builds_sidecar_dockerfile_before_plain_docker_run(self):
with patch( with patch(
@@ -200,26 +65,97 @@ class TestEnsureBundleImage(unittest.TestCase):
self.assertEqual("Dockerfile.sidecars", kwargs["dockerfile"]) self.assertEqual("Dockerfile.sidecars", kwargs["dockerfile"])
class TestStopBundle(unittest.TestCase): class TestStartBundleVm(unittest.TestCase):
def _patch_run(self, **kwargs): # type: ignore def test_allocate_raw_host_ports_returns_bindable_ports(self):
return patch( raw_ports = allocate_raw_host_ports((9099, 9420))
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
**kwargs,
)
def test_argv_force_removes(self): self.assertEqual({9099, 9420}, set(raw_ports))
with self._patch_run(return_value=_ok()) as m: for host_port in raw_ports.values():
stop_bundle("demo-abc12") self.assertIsInstance(host_port, int)
self.assertGreater(host_port, 0)
self.assertNotEqual(raw_ports[9099], raw_ports[9420])
def test_creates_sidecar_vm_with_ports_volumes_and_env(self):
calls: list[tuple[str, tuple[object, ...], dict[str, object]]] = []
def record(name): # type: ignore
def inner(*args, **kwargs): # type: ignore
calls.append((name, args, kwargs))
return inner
with patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.allocate_raw_host_ports",
return_value={9099: 55001, 9420: 55002},
), patch(
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_create",
side_effect=record("create"),
), patch(
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_start",
side_effect=record("start"),
), patch(
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.wait_exec_ready",
side_effect=record("wait"),
):
launch = start_bundle_vm(
_spec(
daemons_csv="egress,git-http",
environment=("FOO=bar", "TOKEN"),
volumes=(("/host/routes", "/etc/egress", True),),
ports_to_publish=(9099, 9420),
),
from_path=Path("/cache/sidecar.smolmachine"),
host_env={"TOKEN": "secret"},
)
self.assertEqual({9099: 55001, 9420: 55002}, launch.raw_ports)
create = calls[0]
self.assertEqual("create", create[0])
self.assertEqual(("bot-bottle-sidecars-demo-abc12",), create[1])
self.assertEqual(Path("/cache/sidecar.smolmachine"), create[2]["from_path"])
self.assertTrue(create[2]["net"])
self.assertEqual(((55001, 9099), (55002, 9420)), create[2]["ports"])
self.assertEqual((("/host/routes", "/etc/egress", True),), create[2]["volumes"])
self.assertEqual( self.assertEqual(
["docker", "rm", "-f", "bot-bottle-sidecars-demo-abc12"], {
m.call_args.args[0], "BOT_BOTTLE_SIDECAR_DAEMONS": "egress,git-http",
"FOO": "bar",
"TOKEN": "secret",
},
create[2]["env"],
) )
def test_missing_container_is_idempotent(self):
with self._patch_run(return_value=_fail( class TestStopBundleVm(unittest.TestCase):
"Error: No such container: bot-bottle-sidecars-demo-abc12" def test_stops_then_deletes_sidecar_vm(self):
)): with patch(
stop_bundle("demo-abc12") # no raise "bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_stop",
) as stop, patch(
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_delete",
) as delete:
stop_bundle_vm("demo-abc12")
stop.assert_called_once_with("bot-bottle-sidecars-demo-abc12")
delete.assert_called_once_with("bot-bottle-sidecars-demo-abc12")
def test_stop_and_delete_errors_are_warnings(self):
from bot_bottle.backend.smolmachines.smolvm import SmolvmError
result = subprocess.CompletedProcess(
args=[],
returncode=1,
stdout="",
stderr="failed",
)
with patch(
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_stop",
side_effect=SmolvmError(["smolvm", "machine", "stop"], result),
), patch(
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_delete",
side_effect=SmolvmError(["smolvm", "machine", "delete"], result),
), patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.warn",
) as warn:
stop_bundle_vm("demo-abc12")
self.assertEqual(2, warn.call_count)
if __name__ == "__main__": if __name__ == "__main__":
+22 -5
View File
@@ -79,7 +79,7 @@ class TestArgvShapes(unittest.TestCase):
with self._patch_run() as m: with self._patch_run() as m:
machine_create("agent-xyz") machine_create("agent-xyz")
self.assertEqual( self.assertEqual(
["smolvm", "machine", "create", "agent-xyz"], ["smolvm", "machine", "create", "--name", "agent-xyz"],
m.call_args.args[0], m.call_args.args[0],
) )
@@ -103,7 +103,24 @@ class TestArgvShapes(unittest.TestCase):
self.assertIn("192.168.50.2/32", argv) self.assertIn("192.168.50.2/32", argv)
self.assertIn("-e", argv) self.assertIn("-e", argv)
self.assertIn("HTTPS_PROXY=http://192.168.50.2:8888", argv) self.assertIn("HTTPS_PROXY=http://192.168.50.2:8888", argv)
self.assertEqual("agent-xyz", argv[-1]) self.assertIn("--name", argv)
self.assertIn("agent-xyz", argv)
def test_machine_create_with_net_ports_and_volumes(self):
with self._patch_run() as m:
machine_create(
"sidecar-xyz",
from_path=Path("/stage/sidecar.smolmachine"),
net=True,
ports=((55000, 9099),),
volumes=(("/host/routes", "/etc/egress", True),),
)
argv = m.call_args.args[0]
self.assertIn("--net", argv)
self.assertIn("-p", argv)
self.assertIn("55000:9099", argv)
self.assertIn("-v", argv)
self.assertIn("/host/routes:/etc/egress:ro", argv)
def test_machine_create_omits_net_when_no_allow_cidrs(self): def test_machine_create_omits_net_when_no_allow_cidrs(self):
with self._patch_run() as m: with self._patch_run() as m:
@@ -127,13 +144,13 @@ class TestArgvShapes(unittest.TestCase):
m.call_args.args[0], m.call_args.args[0],
) )
def test_machine_delete_uses_positional_name_and_force(self): def test_machine_delete_uses_name_flag_and_force(self):
# delete NAME is positional; -f required so no interactive # delete uses --name flag; -f required so no interactive
# confirmation blocks teardown. # confirmation blocks teardown.
with self._patch_run() as m: with self._patch_run() as m:
machine_delete("agent-xyz") machine_delete("agent-xyz")
self.assertEqual( self.assertEqual(
["smolvm", "machine", "delete", "-f", "agent-xyz"], ["smolvm", "machine", "delete", "--name", "agent-xyz", "-f"],
m.call_args.args[0], m.call_args.args[0],
) )
+63
View File
@@ -56,9 +56,14 @@ class TestBundleSubnet(unittest.TestCase):
class TestPreflight(unittest.TestCase): class TestPreflight(unittest.TestCase):
def test_smolvm_present_returns_none(self): def test_smolvm_present_returns_none(self):
# Pin macOS so the Linux KVM gate doesn't fire on a CI runner
# (ubuntu, no /dev/kvm) — this test isolates the PATH check.
with patch( with patch(
"bot_bottle.backend.smolmachines.util.shutil.which", "bot_bottle.backend.smolmachines.util.shutil.which",
return_value="/usr/local/bin/smolvm", return_value="/usr/local/bin/smolvm",
), patch(
"bot_bottle.backend.smolmachines.util.platform.system",
return_value="Darwin",
): ):
self.assertIsNone(smolmachines_preflight()) self.assertIsNone(smolmachines_preflight())
@@ -88,5 +93,63 @@ class TestPreflight(unittest.TestCase):
self.assertIn("BOT_BOTTLE_BACKEND=docker", msg) self.assertIn("BOT_BOTTLE_BACKEND=docker", msg)
class TestKvmPreflight(unittest.TestCase):
"""Linux-only KVM gate: smolvm needs /dev/kvm present and
accessible. macOS skips this entirely (Hypervisor.framework)."""
def _run(self, *, system: str, exists: bool, access: bool) -> None:
with patch(
"bot_bottle.backend.smolmachines.util.shutil.which",
return_value="/usr/bin/smolvm",
), patch(
"bot_bottle.backend.smolmachines.util.platform.system",
return_value=system,
), patch(
"bot_bottle.backend.smolmachines.util.os.path.exists",
return_value=exists,
), patch(
"bot_bottle.backend.smolmachines.util.os.access",
return_value=access,
):
return smolmachines_preflight()
def test_macos_skips_kvm_check(self):
# Even with /dev/kvm absent, macOS must not run the gate.
self.assertIsNone(self._run(system="Darwin", exists=False, access=False))
def test_linux_ok_returns_none(self):
self.assertIsNone(self._run(system="Linux", exists=True, access=True))
def test_linux_missing_device_dies(self):
with self.assertRaises(SystemExit):
self._run(system="Linux", exists=False, access=False)
def test_linux_no_access_dies(self):
with self.assertRaises(SystemExit):
self._run(system="Linux", exists=True, access=False)
def test_linux_missing_device_message(self):
import io
import sys
captured = io.StringIO()
with patch.object(sys, "stderr", captured):
with self.assertRaises(SystemExit):
self._run(system="Linux", exists=False, access=False)
msg = captured.getvalue()
self.assertIn("/dev/kvm", msg)
self.assertIn("kvm-intel", msg)
def test_linux_no_access_message(self):
import io
import sys
captured = io.StringIO()
with patch.object(sys, "stderr", captured):
with self.assertRaises(SystemExit):
self._run(system="Linux", exists=True, access=False)
msg = captured.getvalue()
self.assertIn("kvm", msg)
self.assertIn("group", msg)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+78 -40
View File
@@ -1,6 +1,5 @@
"""Unit: supervise queue + audit log + diff helpers (PRD 0013).""" """Unit: supervise queue + audit log + diff helpers (PRD 0013)."""
import json
import tempfile import tempfile
import threading import threading
import time import time
@@ -9,6 +8,9 @@ from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from bot_bottle import supervise from bot_bottle import supervise
from bot_bottle import supervise_types as sv_types
from bot_bottle.audit_store import AuditStore
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import ( from bot_bottle.supervise import (
AuditEntry, AuditEntry,
Proposal, Proposal,
@@ -19,7 +21,7 @@ from bot_bottle.supervise import (
TOOL_EGRESS_ALLOW, TOOL_EGRESS_ALLOW,
TOOL_GITLEAKS_ALLOW, TOOL_GITLEAKS_ALLOW,
archive_proposal, archive_proposal,
audit_log_path, host_db_path,
list_pending_proposals, list_pending_proposals,
read_audit_entries, read_audit_entries,
read_proposal, read_proposal,
@@ -112,32 +114,52 @@ class TestResponseRoundtrip(unittest.TestCase):
class TestQueueIO(unittest.TestCase): class TestQueueIO(unittest.TestCase):
def setUp(self): def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="bot-bottle-supervise-test.") self._tmp = tempfile.TemporaryDirectory(prefix="bot-bottle-supervise-test.")
self.queue_dir = Path(self._tmp.name) self._home_patch = self._patch_home(Path(self._tmp.name))
self.slug = "dev"
QueueStore("").migrate()
def tearDown(self): def tearDown(self):
self._home_patch()
self._tmp.cleanup() self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
def test_write_and_read_proposal(self): def test_write_and_read_proposal(self):
p = _proposal() p = _proposal()
path = write_proposal(self.queue_dir, p) path = write_proposal(p)
self.assertTrue(path.exists()) self.assertTrue(path.exists())
self.assertEqual(host_db_path(), path)
self.assertEqual(0o600, path.stat().st_mode & 0o777) self.assertEqual(0o600, path.stat().st_mode & 0o777)
loaded = read_proposal(self.queue_dir, p.id) loaded = read_proposal(self.slug, p.id)
self.assertEqual(p, loaded) self.assertEqual(p, loaded)
def test_list_pending_excludes_responded(self): def test_list_pending_excludes_responded(self):
a = _proposal(justification="first") a = _proposal(justification="first")
b = _proposal(justification="second") b = _proposal(justification="second")
write_proposal(self.queue_dir, a) write_proposal(a)
write_proposal(self.queue_dir, b) write_proposal(b)
write_response(self.queue_dir, Response( write_response(self.slug, Response(
proposal_id=a.id, status=STATUS_APPROVED, notes="", proposal_id=a.id, status=STATUS_APPROVED, notes="",
)) ))
pending = list_pending_proposals(self.queue_dir) pending = list_pending_proposals(self.slug)
self.assertEqual([b.id], [p.id for p in pending]) self.assertEqual([b.id], [p.id for p in pending])
def test_list_pending_returns_empty_for_missing_dir(self): def test_list_pending_returns_empty_for_missing_slug(self):
self.assertEqual([], list_pending_proposals(self.queue_dir / "nope")) self.assertEqual([], list_pending_proposals("nope"))
def test_list_pending_sorted_by_arrival(self): def test_list_pending_sorted_by_arrival(self):
# Fabricate two with explicit timestamps. # Fabricate two with explicit timestamps.
@@ -154,30 +176,30 @@ class TestQueueIO(unittest.TestCase):
now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc), now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc),
) )
# Write in reverse order. # Write in reverse order.
write_proposal(self.queue_dir, b) write_proposal(b)
write_proposal(self.queue_dir, a) write_proposal(a)
ordered = list_pending_proposals(self.queue_dir) ordered = list_pending_proposals(self.slug)
self.assertEqual([a.id, b.id], [p.id for p in ordered]) self.assertEqual([a.id, b.id], [p.id for p in ordered])
def test_write_and_read_response(self): def test_write_and_read_response(self):
r = Response(proposal_id="xyz", status=STATUS_REJECTED, notes="no") r = Response(proposal_id="xyz", status=STATUS_REJECTED, notes="no")
write_response(self.queue_dir, r) write_response(self.slug, r)
self.assertEqual(r, read_response(self.queue_dir, "xyz")) self.assertEqual(r, read_response(self.slug, "xyz"))
def test_wait_for_response_returns_when_file_appears(self): def test_wait_for_response_returns_when_file_appears(self):
p = _proposal() p = _proposal()
write_proposal(self.queue_dir, p) write_proposal(p)
def write_after_delay(): def write_after_delay():
time.sleep(0.05) time.sleep(0.05)
write_response(self.queue_dir, Response( write_response(self.slug, Response(
proposal_id=p.id, status=STATUS_APPROVED, notes="ok", proposal_id=p.id, status=STATUS_APPROVED, notes="ok",
)) ))
t = threading.Thread(target=write_after_delay) t = threading.Thread(target=write_after_delay)
t.start() t.start()
try: try:
r = wait_for_response(self.queue_dir, p.id, poll_interval=0.01) r = wait_for_response(self.slug, p.id, poll_interval=0.01)
finally: finally:
t.join() t.join()
self.assertEqual(STATUS_APPROVED, r.status) self.assertEqual(STATUS_APPROVED, r.status)
@@ -187,44 +209,51 @@ class TestQueueIO(unittest.TestCase):
deadline = time.monotonic() + 0.05 deadline = time.monotonic() + 0.05
with self.assertRaises(TimeoutError): with self.assertRaises(TimeoutError):
wait_for_response( wait_for_response(
self.queue_dir, "never", self.slug, "never",
poll_interval=0.01, deadline=deadline, poll_interval=0.01, deadline=deadline,
) )
def test_archive_proposal_moves_both_files(self): def test_archive_proposal_hides_rows(self):
p = _proposal() p = _proposal()
write_proposal(self.queue_dir, p) write_proposal(p)
write_response(self.queue_dir, Response( write_response(self.slug, Response(
proposal_id=p.id, status=STATUS_APPROVED, notes="", proposal_id=p.id, status=STATUS_APPROVED, notes="",
)) ))
archive_proposal(self.queue_dir, p.id) archive_proposal(self.slug, p.id)
self.assertFalse((self.queue_dir / f"{p.id}.proposal.json").exists()) self.assertEqual([], list_pending_proposals(self.slug))
self.assertFalse((self.queue_dir / f"{p.id}.response.json").exists()) with self.assertRaises(FileNotFoundError):
self.assertTrue((self.queue_dir / "processed" / f"{p.id}.proposal.json").exists()) read_response(self.slug, p.id)
self.assertTrue((self.queue_dir / "processed" / f"{p.id}.response.json").exists())
def test_archive_is_idempotent_on_missing_files(self): def test_archive_is_idempotent_on_missing_files(self):
# Should not raise. # Should not raise.
archive_proposal(self.queue_dir, "nope") archive_proposal(self.slug, "nope")
class TestAuditLog(unittest.TestCase): class TestAuditLog(unittest.TestCase):
def setUp(self): def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="bot-bottle-supervise-audit.") self._tmp = tempfile.TemporaryDirectory(prefix="bot-bottle-supervise-audit.")
self._home_patch = self._patch_home(Path(self._tmp.name)) self._home_patch = self._patch_home(Path(self._tmp.name))
AuditStore().migrate()
def tearDown(self): def tearDown(self):
self._home_patch() self._home_patch()
self._tmp.cleanup() self._tmp.cleanup()
def _patch_home(self, fake_home: Path): def _patch_home(self, fake_home: Path):
original = supervise.bot_bottle_root original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path: def fake_root() -> Path:
return fake_home / ".bot-bottle" return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment] supervise.bot_bottle_root = fake_root # type: ignore[assignment]
return lambda: setattr(supervise, "bot_bottle_root", original) sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
def test_write_then_read_single_entry(self): def test_write_then_read_single_entry(self):
e = AuditEntry( e = AuditEntry(
@@ -237,6 +266,7 @@ class TestAuditLog(unittest.TestCase):
diff="--- before\n+++ after\n", diff="--- before\n+++ after\n",
) )
path = write_audit_entry(e) path = write_audit_entry(e)
self.assertEqual(host_db_path(), path)
self.assertEqual(0o600, path.stat().st_mode & 0o777) self.assertEqual(0o600, path.stat().st_mode & 0o777)
loaded = read_audit_entries("cred-proxy", "dev") loaded = read_audit_entries("cred-proxy", "dev")
self.assertEqual([e], loaded) self.assertEqual([e], loaded)
@@ -252,12 +282,13 @@ class TestAuditLog(unittest.TestCase):
justification="", justification="",
diff="", diff="",
)) ))
path = audit_log_path("egress", "dev") entries = read_audit_entries("egress", "dev")
with path.open() as f: self.assertEqual(3, len(entries))
lines = [line for line in f if line.strip()] self.assertEqual(
self.assertEqual(3, len(lines)) ["2026-05-25T12:00:00+00:00", "2026-05-25T12:00:01+00:00",
for line in lines: "2026-05-25T12:00:02+00:00"],
self.assertTrue(json.loads(line)) # each line is valid JSON [entry.timestamp for entry in entries],
)
def test_separate_logs_per_component_slug(self): def test_separate_logs_per_component_slug(self):
write_audit_entry(AuditEntry( write_audit_entry(AuditEntry(
@@ -369,17 +400,24 @@ class TestSupervisePrepare(unittest.TestCase):
self._tmp.cleanup() self._tmp.cleanup()
def _patch_home(self, fake_home: Path): def _patch_home(self, fake_home: Path):
original = supervise.bot_bottle_root original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path: def fake_root() -> Path:
return fake_home / ".bot-bottle" return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment] supervise.bot_bottle_root = fake_root # type: ignore[assignment]
return lambda: setattr(supervise, "bot_bottle_root", original) sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
def test_prepare_creates_queue(self): def test_prepare_creates_queue(self):
plan = _StubSupervise().prepare("dev", self.stage_dir) plan = _StubSupervise().prepare("dev", self.stage_dir)
self.assertTrue(plan.queue_dir.is_dir()) self.assertTrue(plan.db_path.is_file())
self.assertEqual("dev", plan.slug) self.assertEqual("dev", plan.slug)
self.assertEqual("", plan.internal_network) self.assertEqual("", plan.internal_network)
+29 -29
View File
@@ -12,7 +12,10 @@ from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
from bot_bottle import supervise from bot_bottle import supervise
from bot_bottle import supervise_types as sv_types
from bot_bottle.audit_store import AuditStore
from bot_bottle.cli import supervise as supervise_cli from bot_bottle.cli import supervise as supervise_cli
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import ( from bot_bottle.supervise import (
Proposal, Proposal,
STATUS_APPROVED, STATUS_APPROVED,
@@ -52,13 +55,22 @@ class _FakeHomeMixin:
def _setup_fake_home(self): def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-test.") self._tmp = tempfile.TemporaryDirectory(prefix="supervise-test.")
original = supervise.bot_bottle_root original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path: def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle" return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment] supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore_home = lambda: setattr(supervise, "bot_bottle_root", original) sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
self._restore_home = restore
QueueStore("").migrate()
AuditStore().migrate()
def _teardown_fake_home(self): def _teardown_fake_home(self):
self._restore_home() self._restore_home()
@@ -77,9 +89,7 @@ class TestDiscoverPending(_FakeHomeMixin, unittest.TestCase):
def test_walks_all_slug_subdirs(self): def test_walks_all_slug_subdirs(self):
for slug in ("dev", "api"): for slug in ("dev", "api"):
qdir = supervise.queue_dir_for_slug(slug) supervise.write_proposal(_proposal(slug=slug))
qdir.mkdir(parents=True)
supervise.write_proposal(qdir, _proposal(slug=slug))
pending = supervise_cli.discover_pending() pending = supervise_cli.discover_pending()
self.assertEqual({"dev", "api"}, {qp.proposal.bottle_slug for qp in pending}) self.assertEqual({"dev", "api"}, {qp.proposal.bottle_slug for qp in pending})
@@ -97,18 +107,14 @@ class TestDiscoverPending(_FakeHomeMixin, unittest.TestCase):
now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc), now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc),
) )
for p in (late, early): for p in (late, early):
qdir = supervise.queue_dir_for_slug(p.bottle_slug) supervise.write_proposal(p)
qdir.mkdir(parents=True, exist_ok=True)
supervise.write_proposal(qdir, p)
pending = supervise_cli.discover_pending() pending = supervise_cli.discover_pending()
self.assertEqual([early.id, late.id], [qp.proposal.id for qp in pending]) self.assertEqual([early.id, late.id], [qp.proposal.id for qp in pending])
def test_excludes_already_responded(self): def test_excludes_already_responded(self):
p = _proposal() p = _proposal()
qdir = supervise.queue_dir_for_slug("dev") supervise.write_proposal(p)
qdir.mkdir(parents=True) supervise.write_response("dev", supervise.Response(
supervise.write_proposal(qdir, p)
supervise.write_response(qdir, supervise.Response(
proposal_id=p.id, status=STATUS_APPROVED, notes="", proposal_id=p.id, status=STATUS_APPROVED, notes="",
)) ))
self.assertEqual([], supervise_cli.discover_pending()) self.assertEqual([], supervise_cli.discover_pending())
@@ -123,10 +129,8 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
def _enqueue(self, tool: str = TOOL_EGRESS_ALLOW): def _enqueue(self, tool: str = TOOL_EGRESS_ALLOW):
p = _proposal(tool=tool) p = _proposal(tool=tool)
qdir = supervise.queue_dir_for_slug("dev") supervise.write_proposal(p)
qdir.mkdir(parents=True, exist_ok=True) return supervise_cli.QueuedProposal(proposal=p)
supervise.write_proposal(qdir, p)
return supervise_cli.QueuedProposal(proposal=p, queue_dir=qdir)
def test_approve_writes_response(self): def test_approve_writes_response(self):
qp = self._enqueue() qp = self._enqueue()
@@ -135,7 +139,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
return_value=("routes: []\n", "routes:\n - host: example.com\n"), return_value=("routes: []\n", "routes:\n - host: example.com\n"),
): ):
supervise_cli.approve(qp) supervise_cli.approve(qp)
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual(STATUS_APPROVED, resp.status) self.assertEqual(STATUS_APPROVED, resp.status)
self.assertIsNone(resp.final_file) self.assertIsNone(resp.final_file)
@@ -150,7 +154,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
final_file="routes:\n - host: edited.example.com\n", final_file="routes:\n - host: edited.example.com\n",
notes="tweaked", notes="tweaked",
) )
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual(STATUS_MODIFIED, resp.status) self.assertEqual(STATUS_MODIFIED, resp.status)
self.assertEqual("routes:\n - host: edited.example.com\n", resp.final_file) self.assertEqual("routes:\n - host: edited.example.com\n", resp.final_file)
self.assertEqual("tweaked", resp.notes) self.assertEqual("tweaked", resp.notes)
@@ -158,7 +162,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
def test_reject_writes_rejection(self): def test_reject_writes_rejection(self):
qp = self._enqueue() qp = self._enqueue()
supervise_cli.reject(qp, reason="nope") supervise_cli.reject(qp, reason="nope")
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual(STATUS_REJECTED, resp.status) self.assertEqual(STATUS_REJECTED, resp.status)
self.assertEqual("nope", resp.notes) self.assertEqual("nope", resp.notes)
@@ -181,36 +185,33 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
def test_approve_gitleaks_allow_leaves_response_for_gate(self): def test_approve_gitleaks_allow_leaves_response_for_gate(self):
qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW) qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW)
supervise_cli.approve(qp, notes="dummy fixture") supervise_cli.approve(qp, notes="dummy fixture")
# Gate polls the queue dir for the response; TUI must not archive it. # Gate polls the DB for the response; TUI must not archive it.
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual(STATUS_APPROVED, resp.status) self.assertEqual(STATUS_APPROVED, resp.status)
self.assertEqual("dummy fixture", resp.notes) self.assertEqual("dummy fixture", resp.notes)
self.assertFalse((qp.queue_dir / "processed").exists())
def test_tui_gitleaks_allow_requires_reason(self): def test_tui_gitleaks_allow_requires_reason(self):
qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW) qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW)
with patch.object(supervise_cli, "_prompt", return_value=""): with patch.object(supervise_cli, "_prompt", return_value=""):
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type] status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
self.assertEqual("approve aborted (empty reason)", status) self.assertEqual("approve aborted (empty reason)", status)
self.assertFalse((qp.queue_dir / "processed").exists())
def test_tui_gitleaks_allow_writes_reason(self): def test_tui_gitleaks_allow_writes_reason(self):
qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW) qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW)
with patch.object(supervise_cli, "_prompt", return_value="test fixture"): with patch.object(supervise_cli, "_prompt", return_value="test fixture"):
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type] status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
self.assertIn("approved gitleaks-allow", status) self.assertIn("approved gitleaks-allow", status)
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual("test fixture", resp.notes) self.assertEqual("test fixture", resp.notes)
def test_approve_token_allow_leaves_response_for_egress(self): def test_approve_token_allow_leaves_response_for_egress(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW) qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
supervise_cli.approve(qp, notes="false positive") supervise_cli.approve(qp, notes="false positive")
# The egress addon polls the queue dir for the response; the TUI must # The egress addon polls the DB for the response; the TUI must
# not archive it (the addon archives after reading). # not archive it (the addon archives after reading).
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual(STATUS_APPROVED, resp.status) self.assertEqual(STATUS_APPROVED, resp.status)
self.assertEqual("false positive", resp.notes) self.assertEqual("false positive", resp.notes)
self.assertFalse((qp.queue_dir / "processed").exists())
def test_token_allow_writes_no_audit_log(self): def test_token_allow_writes_no_audit_log(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW) qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
@@ -222,14 +223,13 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
with patch.object(supervise_cli, "_prompt", return_value=""): with patch.object(supervise_cli, "_prompt", return_value=""):
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type] status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
self.assertEqual("approve aborted (empty reason)", status) self.assertEqual("approve aborted (empty reason)", status)
self.assertFalse((qp.queue_dir / "processed").exists())
def test_tui_token_allow_writes_reason(self): def test_tui_token_allow_writes_reason(self):
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW) qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
with patch.object(supervise_cli, "_prompt", return_value="legit"): with patch.object(supervise_cli, "_prompt", return_value="legit"):
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type] status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
self.assertIn("approved egress-token-allow", status) self.assertIn("approved egress-token-allow", status)
resp = read_response(qp.queue_dir, qp.proposal.id) resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
self.assertEqual("legit", resp.notes) self.assertEqual("legit", resp.notes)
def test_suffix_for_token_allow_is_txt(self): def test_suffix_for_token_allow_is_txt(self):
+129 -57
View File
@@ -4,7 +4,6 @@ fallback paths."""
from __future__ import annotations from __future__ import annotations
import os
import tempfile import tempfile
import time import time
import unittest import unittest
@@ -12,14 +11,19 @@ from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
from bot_bottle import supervise from bot_bottle import supervise
from bot_bottle.audit_store import AuditStore
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import ( from bot_bottle.supervise import (
AuditEntry,
Proposal, Proposal,
STATUS_APPROVED,
TOOL_EGRESS_ALLOW, TOOL_EGRESS_ALLOW,
list_pending_proposals, list_pending_proposals,
read_audit_entries, read_audit_entries,
read_proposal, read_proposal,
read_response, read_response,
wait_for_response, wait_for_response,
write_audit_entry,
) )
@@ -37,58 +41,59 @@ class TestPathHelpers(unittest.TestCase):
def test_bot_bottle_root(self) -> None: def test_bot_bottle_root(self) -> None:
self.assertTrue(str(supervise.bot_bottle_root()).endswith(".bot-bottle")) self.assertTrue(str(supervise.bot_bottle_root()).endswith(".bot-bottle"))
def test_queue_dir_for_slug(self) -> None:
self.assertIn("slug", str(supervise.queue_dir_for_slug("slug")))
def test_id_from_non_proposal_filename(self) -> None:
self.assertIsNone(supervise._id_from_proposal_filename(Path("x.response.json")))
class TestReadMalformed(unittest.TestCase): class TestReadMalformed(unittest.TestCase):
def test_read_proposal_non_dict(self) -> None: def test_read_proposal_missing_row(self) -> None:
with tempfile.TemporaryDirectory() as d: with tempfile.TemporaryDirectory() as d:
(Path(d) / "p.proposal.json").write_text("[]") with patch.dict("os.environ", {"HOME": d}):
with self.assertRaises(ValueError): QueueStore("slug").migrate()
read_proposal(Path(d), "p") with self.assertRaises(FileNotFoundError):
read_proposal("slug", "p")
def test_read_response_non_dict(self) -> None: def test_read_response_missing_row(self) -> None:
with tempfile.TemporaryDirectory() as d: with tempfile.TemporaryDirectory() as d:
(Path(d) / "p.response.json").write_text("[]") with patch.dict("os.environ", {"HOME": d}):
with self.assertRaises(ValueError): QueueStore("slug").migrate()
read_response(Path(d), "p") with self.assertRaises(FileNotFoundError):
read_response("slug", "p")
def test_list_pending_skips_malformed(self) -> None: def test_list_pending_reads_db_only(self) -> None:
with tempfile.TemporaryDirectory() as d: with tempfile.TemporaryDirectory() as d:
qd = Path(d) with patch.dict("os.environ", {"HOME": d}):
(qd / "bad.proposal.json").write_text("{ not json") QueueStore("slug").migrate()
(qd / "arr.proposal.json").write_text("[]") supervise.write_proposal(_proposal())
(qd / "incomplete.proposal.json").write_text("{}") # from_dict raises pending = list_pending_proposals("slug")
supervise.write_proposal(qd, _proposal()) # one valid
pending = list_pending_proposals(qd)
self.assertEqual(1, len(pending)) self.assertEqual(1, len(pending))
self.assertEqual("slug", pending[0].bottle_slug) self.assertEqual("slug", pending[0].bottle_slug)
def test_list_pending_skips_when_response_present(self) -> None: def test_list_pending_skips_when_response_present(self) -> None:
with tempfile.TemporaryDirectory() as d: with tempfile.TemporaryDirectory() as d:
qd = Path(d) with patch.dict("os.environ", {"HOME": d}):
p = _proposal() QueueStore("slug").migrate()
supervise.write_proposal(qd, p) p = _proposal()
(qd / f"{p.id}.response.json").write_text("{}") # response exists -> skipped supervise.write_proposal(p)
self.assertEqual([], list_pending_proposals(qd)) supervise.write_response("slug", supervise.Response(
proposal_id=p.id,
status=STATUS_APPROVED,
notes="",
))
self.assertEqual([], list_pending_proposals("slug"))
class TestWaitForResponse(unittest.TestCase): class TestWaitForResponse(unittest.TestCase):
def test_malformed_response_then_timeout(self) -> None: def test_missing_response_times_out(self) -> None:
with tempfile.TemporaryDirectory() as d: with tempfile.TemporaryDirectory() as d:
(Path(d) / "p.response.json").write_text("{ not json") with patch.dict("os.environ", {"HOME": d}):
with self.assertRaises(TimeoutError): QueueStore("slug").migrate()
wait_for_response(Path(d), "p", deadline=time.monotonic()) with self.assertRaises(TimeoutError):
wait_for_response("slug", "p", deadline=time.monotonic())
def test_incomplete_response_then_timeout(self) -> None: def test_empty_db_response_does_not_count(self) -> None:
with tempfile.TemporaryDirectory() as d: with tempfile.TemporaryDirectory() as d:
(Path(d) / "p.response.json").write_text("{}") # dict but from_dict raises with patch.dict("os.environ", {"HOME": d}):
with self.assertRaises(TimeoutError): QueueStore("slug").migrate()
wait_for_response(Path(d), "p", deadline=time.monotonic()) with self.assertRaises(TimeoutError):
wait_for_response("slug", "p", deadline=time.monotonic())
class TestReadAuditEntries(unittest.TestCase): class TestReadAuditEntries(unittest.TestCase):
@@ -97,35 +102,102 @@ class TestReadAuditEntries(unittest.TestCase):
patch.dict("os.environ", {"HOME": home}): patch.dict("os.environ", {"HOME": home}):
self.assertEqual([], read_audit_entries("egress", "nope")) self.assertEqual([], read_audit_entries("egress", "nope"))
def test_skips_malformed_lines(self) -> None: def test_reads_entries_from_db(self) -> None:
with tempfile.TemporaryDirectory() as home, \ with tempfile.TemporaryDirectory() as home, \
patch.dict("os.environ", {"HOME": home}): patch.dict("os.environ", {"HOME": home}):
path = supervise.audit_log_path("egress", "slug") AuditStore().migrate()
path.parent.mkdir(parents=True, exist_ok=True) write_audit_entry(AuditEntry(
valid = ( timestamp="t",
'{"timestamp": "t", "bottle_slug": "slug", "component": "egress",' bottle_slug="slug",
' "operator_action": "approve", "operator_notes": "",' component="egress",
' "justification": "", "diff": ""}' operator_action="approve",
) operator_notes="",
path.write_text( justification="",
"\n" # blank line skipped diff="",
"{ not json\n" # JSONDecodeError skipped ))
"[]\n" # not a dict skipped write_audit_entry(AuditEntry(
"{}\n" # missing fields -> ValueError skipped timestamp="t",
+ valid + "\n" bottle_slug="other",
) component="egress",
operator_action="reject",
operator_notes="",
justification="",
diff="",
))
entries = read_audit_entries("egress", "slug") entries = read_audit_entries("egress", "slug")
self.assertEqual(1, len(entries)) self.assertEqual(1, len(entries))
self.assertEqual("approve", entries[0].operator_action) self.assertEqual("approve", entries[0].operator_action)
def test_legacy_audit_log_file_does_not_count(self) -> None:
with tempfile.TemporaryDirectory() as home, \
patch.dict("os.environ", {"HOME": home}):
path = supervise.audit_log_path("egress", "slug")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
'{"timestamp": "t", "bottle_slug": "slug", "component": "egress",'
' "operator_action": "approve", "operator_notes": "",'
' "justification": "", "diff": ""}\n'
)
entries = read_audit_entries("egress", "slug")
self.assertEqual([], entries)
class TestFlockFallback(unittest.TestCase):
def test_flock_on_closed_fd_is_swallowed(self) -> None: class TestStoreGuardBranches(unittest.TestCase):
# flock on a closed fd raises OSError(EBADF), which the helpers swallow. """Direct QueueStore / AuditStore construction and early-return guard branches."""
fd = os.open(os.devnull, os.O_RDONLY)
os.close(fd) def test_queue_store_explicit_db_path(self):
supervise._try_flock(fd) with tempfile.TemporaryDirectory() as d:
supervise._try_funlock(fd) db = Path(d) / "q.db"
store = QueueStore("key", db_path=db)
store.migrate()
self.assertTrue(db.is_file())
self.assertEqual(db, store.db_path)
def test_queue_store_missing_db_list_pending_returns_empty(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "q.db"
store = QueueStore("key", db_path=db)
store.migrate()
db.unlink()
self.assertEqual([], store.list_pending_proposals())
def test_queue_store_missing_db_list_all_returns_empty(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "q.db"
store = QueueStore("key", db_path=db)
store.migrate()
db.unlink()
self.assertEqual([], store.list_all_pending_proposals())
def test_queue_store_missing_db_archive_is_noop(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "q.db"
store = QueueStore("key", db_path=db)
store.migrate()
db.unlink()
store.archive_proposal("anything") # must not raise
def test_queue_store_chmod_oserror_is_swallowed(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "q.db"
store = QueueStore("key", db_path=db)
with patch("pathlib.Path.chmod", side_effect=OSError("ro")):
store.migrate() # must not raise
def test_audit_store_missing_db_read_returns_empty(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "a.db"
store = AuditStore(db_path=db)
store.migrate()
db.unlink()
self.assertEqual([], store.read_audit_entries("egress", "slug"))
def test_audit_store_chmod_oserror_is_swallowed(self):
with tempfile.TemporaryDirectory() as d:
db = Path(d) / "a.db"
store = AuditStore(db_path=db)
with patch("pathlib.Path.chmod", side_effect=OSError("ro")):
store.migrate() # must not raise
if __name__ == "__main__": if __name__ == "__main__":
+38 -18
View File
@@ -17,8 +17,12 @@ from unittest.mock import patch
# bare name `supervise`. # bare name `supervise`.
sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent / "bot_bottle")) sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent / "bot_bottle"))
import supervise as _sv # noqa: E402 # type: ignore import supervise as _sv # noqa: E402 # type: ignore
import queue_store as _qs # noqa: E402 # type: ignore
import audit_store as _as # noqa: E402 # type: ignore
import supervise_types as svt_flat # noqa: E402 # type: ignore
from bot_bottle import supervise_server # noqa: E402 from bot_bottle import supervise_server # noqa: E402
from bot_bottle import supervise_types as svt_pkg # noqa: E402
from bot_bottle.supervise_server import ( from bot_bottle.supervise_server import (
ERR_INTERNAL, ERR_INTERNAL,
ERR_INVALID_PARAMS, ERR_INVALID_PARAMS,
@@ -112,7 +116,7 @@ class TestRpcErrorTaxonomy(unittest.TestCase):
validate_proposed_file(_sv.TOOL_EGRESS_ALLOW, "routes: nope\n") validate_proposed_file(_sv.TOOL_EGRESS_ALLOW, "routes: nope\n")
def test_unknown_tool_in_tools_call_is_client_error(self): def test_unknown_tool_in_tools_call_is_client_error(self):
config = ServerConfig(bottle_slug="dev", queue_dir=Path("/unused")) config = ServerConfig(bottle_slug="dev")
with self.assertRaises(_RpcClientError) as cm: with self.assertRaises(_RpcClientError) as cm:
handle_tools_call({"name": "no-such-tool", "arguments": {}}, config) handle_tools_call({"name": "no-such-tool", "arguments": {}}, config)
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code) self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
@@ -122,9 +126,9 @@ class TestRpcInternalErrorOnIoFailure(unittest.TestCase):
def test_write_proposal_os_error_raises_internal(self): def test_write_proposal_os_error_raises_internal(self):
config = ServerConfig( config = ServerConfig(
bottle_slug="dev", bottle_slug="dev",
queue_dir=Path("/dev/null/cannot-exist"),
) )
with self.assertRaises(_RpcInternalError) as cm: with patch.object(_sv, "write_proposal", side_effect=OSError("disk full")), \
self.assertRaises(_RpcInternalError) as cm:
handle_tools_call( handle_tools_call(
{ {
"name": _sv.TOOL_EGRESS_ALLOW, "name": _sv.TOOL_EGRESS_ALLOW,
@@ -265,21 +269,43 @@ class TestHandleToolsList(unittest.TestCase):
class TestHandleToolsCall(unittest.TestCase): class TestHandleToolsCall(unittest.TestCase):
def setUp(self): def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-server-test.") self._tmp = tempfile.TemporaryDirectory(prefix="supervise-server-test.")
self.queue_dir = Path(self._tmp.name) self._home_patch = self._patch_home(Path(self._tmp.name))
self.config = ServerConfig(bottle_slug="dev", queue_dir=self.queue_dir) self.config = ServerConfig(bottle_slug="dev")
_qs.QueueStore("dev").migrate()
_as.AuditStore().migrate()
def tearDown(self): def tearDown(self):
self._home_patch()
self._tmp.cleanup() self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = _sv.bot_bottle_root
original_flat = svt_flat.bot_bottle_root
original_pkg = svt_pkg.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
_sv.bot_bottle_root = fake_root # type: ignore[assignment]
svt_flat.bot_bottle_root = fake_root # type: ignore[assignment]
svt_pkg.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
_sv.bot_bottle_root = original_sv # type: ignore[assignment]
svt_flat.bot_bottle_root = original_flat # type: ignore[assignment]
svt_pkg.bot_bottle_root = original_pkg # type: ignore[assignment]
return restore
def _respond_when_proposal_appears(self, status: str, notes: str = "") -> threading.Thread: def _respond_when_proposal_appears(self, status: str, notes: str = "") -> threading.Thread:
"""Background thread: poll the queue for a fresh proposal, write a """Background thread: poll the queue for a fresh proposal, write a
matching response. Returns the thread so the test can join it.""" matching response. Returns the thread so the test can join it."""
def runner(): def runner():
for _ in range(200): for _ in range(200):
pending = _sv.list_pending_proposals(self.queue_dir) pending = _sv.list_pending_proposals("dev")
if pending: if pending:
p = pending[0] p = pending[0]
_sv.write_response(self.queue_dir, _sv.Response( _sv.write_response("dev", _sv.Response(
proposal_id=p.id, status=status, notes=notes, proposal_id=p.id, status=status, notes=notes,
)) ))
return return
@@ -412,15 +438,11 @@ class TestHandleToolsCall(unittest.TestCase):
finally: finally:
responder.join() responder.join()
# No pending proposals left after archive. # No pending proposals left after archive.
self.assertEqual([], _sv.list_pending_proposals(self.queue_dir)) self.assertEqual([], _sv.list_pending_proposals("dev"))
# Both files moved to processed/.
processed = list((self.queue_dir / "processed").glob("*.json"))
self.assertEqual(2, len(processed))
def test_pending_response_times_out_without_archive(self): def test_pending_response_times_out_without_archive(self):
config = ServerConfig( config = ServerConfig(
bottle_slug="dev", bottle_slug="dev",
queue_dir=self.queue_dir,
response_timeout_seconds=0.05, response_timeout_seconds=0.05,
) )
result = handle_tools_call( result = handle_tools_call(
@@ -438,8 +460,7 @@ class TestHandleToolsCall(unittest.TestCase):
text = result["content"][0]["text"] # type: ignore[index] text = result["content"][0]["text"] # type: ignore[index]
self.assertIn("status: pending", text) self.assertIn("status: pending", text)
self.assertIn("proposal remains queued", text) self.assertIn("proposal remains queued", text)
self.assertEqual(1, len(_sv.list_pending_proposals(self.queue_dir))) self.assertEqual(1, len(_sv.list_pending_proposals("dev")))
self.assertFalse((self.queue_dir / "processed").exists())
class TestHandleListEgressRoutes(unittest.TestCase): class TestHandleListEgressRoutes(unittest.TestCase):
@@ -461,7 +482,7 @@ class TestHandleListEgressRoutes(unittest.TestCase):
with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()): with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()):
result = handle_list_egress_routes( result = handle_list_egress_routes(
{}, {},
ServerConfig(bottle_slug="dev", queue_dir=Path("/unused")), ServerConfig(bottle_slug="dev"),
) )
self.assertFalse(result["isError"]) # type: ignore[index] self.assertFalse(result["isError"]) # type: ignore[index]
@@ -476,7 +497,7 @@ class TestHandleListEgressRoutes(unittest.TestCase):
with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()): with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()):
result = handle_list_egress_routes( result = handle_list_egress_routes(
{}, {},
ServerConfig(bottle_slug="dev", queue_dir=Path("/unused")), ServerConfig(bottle_slug="dev"),
) )
self.assertTrue(result["isError"]) # type: ignore[index] self.assertTrue(result["isError"]) # type: ignore[index]
@@ -544,7 +565,6 @@ class TestHttpEndToEnd(unittest.TestCase):
def setUp(self): def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-http-test.") self._tmp = tempfile.TemporaryDirectory(prefix="supervise-http-test.")
self.queue_dir = Path(self._tmp.name)
# Pick a random port by binding to :0 first. # Pick a random port by binding to :0 first.
import socket import socket
s = socket.socket() s = socket.socket()
@@ -552,7 +572,7 @@ class TestHttpEndToEnd(unittest.TestCase):
self.port = s.getsockname()[1] self.port = s.getsockname()[1]
s.close() s.close()
self.server = MCPServer(("127.0.0.1", self.port), MCPHandler) self.server = MCPServer(("127.0.0.1", self.port), MCPHandler)
self.server.config = ServerConfig(bottle_slug="dev", queue_dir=self.queue_dir) self.server.config = ServerConfig(bottle_slug="dev")
self.thread = threading.Thread( self.thread = threading.Thread(
target=self.server.serve_forever, daemon=True, target=self.server.serve_forever, daemon=True,
) )
+137 -1
View File
@@ -6,7 +6,7 @@ import textwrap
import unittest import unittest
from bot_bottle.yaml_subset import YamlSubsetError from bot_bottle.yaml_subset import YamlSubsetError
from bot_bottle.yaml_subset import parse_frontmatter, parse_yaml_subset from bot_bottle.yaml_subset import parse_frontmatter, parse_yaml_subset, serialize_yaml_subset
def _y(s: str): def _y(s: str):
@@ -457,5 +457,141 @@ class TestEdgeAndErrorBranches(unittest.TestCase):
self.assertEqual(({}, ""), parse_frontmatter("")) self.assertEqual(({}, ""), parse_frontmatter(""))
# ---------------------------------------------------------------------------
# serialize_yaml_subset
# ---------------------------------------------------------------------------
class TestSerializeYamlSubset(unittest.TestCase):
def _roundtrip(self, text: str) -> None:
"""Parse then serialize then re-parse; result must equal original parse."""
parsed = parse_yaml_subset(text)
serialized = serialize_yaml_subset(parsed)
self.assertEqual(parsed, parse_yaml_subset(serialized))
# --- scalar values ---------------------------------------------------------
def test_string_bare(self) -> None:
self.assertEqual("key: value\n", serialize_yaml_subset({"key": "value"}))
def test_string_empty_is_quoted(self) -> None:
out = serialize_yaml_subset({"key": ""})
self.assertIn("key: ''", out)
self._roundtrip("key: ''\n")
def test_string_true_like_is_quoted(self) -> None:
out = serialize_yaml_subset({"key": "true"})
self.assertIn("'true'", out)
self.assertEqual({"key": "true"}, parse_yaml_subset(out))
def test_string_int_like_is_quoted(self) -> None:
out = serialize_yaml_subset({"key": "42"})
self.assertIn("'42'", out)
self.assertEqual({"key": "42"}, parse_yaml_subset(out))
def test_string_special_start_char_is_quoted(self) -> None:
for ch in ('"', "'", "[", "{", "!", "&", "*", "#", "|", ">"):
out = serialize_yaml_subset({"key": ch + "rest"})
self.assertIn("'", out, msg=f"expected quoting for {ch!r}")
def test_none_emits_null(self) -> None:
self.assertEqual("key: null\n", serialize_yaml_subset({"key": None}))
def test_bool_true(self) -> None:
self.assertEqual("key: true\n", serialize_yaml_subset({"key": True}))
def test_bool_false(self) -> None:
self.assertEqual("key: false\n", serialize_yaml_subset({"key": False}))
def test_int(self) -> None:
self.assertEqual("key: 42\n", serialize_yaml_subset({"key": 42}))
# --- nested dict -----------------------------------------------------------
def test_nested_dict(self) -> None:
data: dict[str, object] = {"outer": {"inner": "val"}}
out = serialize_yaml_subset(data)
self.assertIn("outer:", out)
self.assertIn(" inner: val", out)
self.assertEqual(data, parse_yaml_subset(out))
def test_empty_dict_value_inline(self) -> None:
out = serialize_yaml_subset({"key": {}})
self.assertIn("key: {}", out)
# --- lists -----------------------------------------------------------------
def test_list_of_scalars(self) -> None:
data: dict[str, object] = {"items": ["a", "b", "c"]}
out = serialize_yaml_subset(data)
self.assertEqual(data, parse_yaml_subset(out))
def test_empty_list_value_inline(self) -> None:
out = serialize_yaml_subset({"key": []})
self.assertIn("key: []", out)
def test_list_of_mappings(self) -> None:
data: dict[str, object] = {"items": [{"name": "alpha", "val": 1}, {"name": "beta", "val": 2}]}
out = serialize_yaml_subset(data)
self.assertEqual(data, parse_yaml_subset(out))
def test_list_item_mapping_with_nested_dict(self) -> None:
data: dict[str, object] = {"items": [{"name": "x", "sub": {"k": "v"}}]}
out = serialize_yaml_subset(data)
self.assertEqual(data, parse_yaml_subset(out))
def test_list_item_first_key_is_nested_dict(self) -> None:
# Covers the branch where the FIRST key of a mapping list item
# has a nested dict/list value (not a scalar).
data: dict[str, object] = {"items": [{"cfg": {"a": 1, "b": 2}, "name": "x"}]}
out = serialize_yaml_subset(data)
self.assertEqual(data, parse_yaml_subset(out))
def test_list_item_mapping_continuation_keys(self) -> None:
data: dict[str, object] = {"items": [{"a": 1, "b": 2, "c": 3}]}
out = serialize_yaml_subset(data)
self.assertEqual(data, parse_yaml_subset(out))
# --- empty / trivial -------------------------------------------------------
def test_empty_dict(self) -> None:
self.assertEqual("", serialize_yaml_subset({}))
# --- realistic bottle frontmatter round-trip --------------------------------
def test_bottle_frontmatter_round_trip(self) -> None:
text = textwrap.dedent("""\
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
""")
self._roundtrip(text)
def test_host_key_inserted_at_repo_level(self) -> None:
"""The canonical use-case: host_key lands as a sibling of url/key."""
from typing import cast
text = textwrap.dedent("""\
git-gate:
repos:
myrepo:
url: ssh://git@host/org/repo.git
key:
provider: static
path: /dev/null
""")
data = parse_yaml_subset(text)
repos = cast(dict[str, object], cast(dict[str, object], data["git-gate"])["repos"])
cast(dict[str, object], repos["myrepo"])["host_key"] = "ssh-ed25519 AAAA"
out = serialize_yaml_subset(data)
parsed_back = parse_yaml_subset(out)
repo = cast(dict[str, object], cast(dict[str, object], cast(dict[str, object], parsed_back["git-gate"])["repos"])["myrepo"])
self.assertEqual("ssh-ed25519 AAAA", repo["host_key"])
self.assertIn("provider", cast(dict[str, object], repo["key"]))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()