Proposal, Response, AuditEntry, host_db_path, bot_bottle_root, and their
shared constants/helpers are moved to a new _supervise_types.py module.
queue_store and audit_store now import these directly instead of deferring
through get_supervise_mod() at call time.
host_db_path() in _supervise_types uses sys.modules[__name__].bot_bottle_root()
so monkey-patches on _supervise_types.bot_bottle_root propagate through it.
supervise.host_db_path() is re-defined to call bot_bottle_root() through
supervise's own globals, preserving patchability for callers that go through
the supervise module.
Test fixtures that create stores without an explicit db_path now patch
_sv_types.bot_bottle_root alongside supervise.bot_bottle_root. The server
test patches both flat and package _supervise_types since handle_tools_call
runs in the package context while the responder thread uses the flat module.
Dockerfile.sidecars gets a COPY for _supervise_types.py so the sidecar
bundle retains its flat import chain.
Follow-up to issuecomment-2872 on PR #320.
Remove auto-migration from DbStore.__init__; add explicit check_migrations()
and migrate() methods. The CLI now checks both stores on every startup and
prompts the user to confirm before migrating. supervise.prepare() calls
.migrate() directly now that __init__ no longer does it implicitly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds bot_bottle/migrations.py (TableMigrations) and bot_bottle/db_store.py
(DbStore) per PR review. Both stores now inherit from DbStore and hold a
TableMigrations instance instead of duplicating schema-version logic inline.
Per review #320 comments:
- Rename _sv() → get_supervise_mod() in both store files (review 206/211)
- Move _audit_entry_from_row onto AuditStore as _row_to_entry static method
(review 208); move _proposal/_response_from_row onto QueueStore (review 211)
- Remove _host_db_path() free function; inline into __init__ (review 209/211)
- Add stdlib migration runner using a shared schema_versions table; each store
tracks its own version under a module key so they can coexist in the same DB
without clobbering a shared PRAGMA user_version (reviews 210/212/213)
- PRD: add goal 6 (migration runner), narrow non-goal to third-party ORM only
Moves _QueueStore → bot_bottle/queue_store.py (public QueueStore) and
_AuditStore → bot_bottle/audit_store.py (public AuditStore). Removes
the public queue_db_path() function; QueueStore resolves the DB path
via host_db_path() on the host, or via the SUPERVISE_DB_PATH env var
in the sidecar container (internal mechanism, not public API).
Adds queue_store.py and audit_store.py to Dockerfile.sidecars so the
sidecar bundle picks them up. Updates __all__ in supervise.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
git_http_backend.py is copied flat into the sidecar bundle image as a
standalone script, not as part of the bot_bottle package, and
git_gate.py/git_gate_render.py are never copied in. Its relative
import of GIT_GATE_TIMEOUT_SECS crashed the git-http daemon (port
9420) on every startup, silently leaving the smart-HTTP git-gate
transport down while the other sidecar daemons stayed up.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`--headless` is a non-interactive launch path for `cli.py start`:
agent, bottles, label, and color come from flags + manifest defaults
with no TUI selectors and no y/N preflight (auto-confirmed via a new
`assume_yes` param threaded into the shared `_launch_bottle` core).
- `--bottle` (repeatable) defaults to the agent's own `bottle:`;
`--label` defaults to the agent name and auto-uniquifies on slug
collision; `--color` defaults to none.
- `--prompt TEXT` is required in headless mode and is delivered to the
agent via a new `headless_prompt(prompt)` method on `AgentProvider`,
implemented for claude (`-p`), codex (positional), and pi (`-p`).
- The agent still execs on inherited stdio/PTY, so whatever allocates
the PTY drives the live session; only the launch chrome is headless.
- `--headless --dry-run` previews the resolved plan without launching.
Adds unit coverage in tests/unit/test_cli_start_headless.py and
headless_prompt tests for each provider. Also stubs headless_prompt on
the in-test AgentProvider subclasses so the unit suite collects cleanly.
Closes#315.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WL77TgFxKbs3cidGMG9dz7
Skill names become host/guest path segments interpolated into the
`bottle.exec` shell strings in each contrib provider's provision_skills.
They were validated only as strings, so a name with shell metacharacters
or path traversal could reach the command.
Layer two defenses:
- Primary: reject any skill name that isn't kebab-case
([a-z][a-z0-9-]*) at manifest load, reusing the convention already
enforced on bottle/agent filenames (new is_valid_entity_name helper
in manifest_schema). Fails loud and early, protecting every consumer
of the name — not just the exec call sites.
- Failsafe: shlex.quote the interpolated skills_dir / dst paths in the
claude, codex, and pi providers, so a future unvalidated field can't
inject shell metacharacters even if it bypasses the load-time check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
manifest.py imported the extends/loader resolvers, while those resolvers
needed ManifestBottle back from manifest.py — a true bidirectional cycle
papered over with in-function imports and TYPE_CHECKING guards (not clear
dependency inversion).
Extract ManifestBottle into a new leaf module manifest_bottle.py that depends
only on the other leaf modules (manifest_util/agent/egress/git/schema).
manifest.py re-exports ManifestBottle, so `from .manifest import ManifestBottle`
callers are unaffected. With the cycle gone:
- manifest_extends and manifest_loader import ManifestBottle from
manifest_bottle and their other deps from the real source modules, all at
top level (TYPE_CHECKING block removed).
- manifest.py imports the extends/loader/schema/yaml_subset/log helpers at
module top; all per-function lazy imports in the cluster are removed.
No behavior change; full unit suite green, pyright clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
- dlp_detectors._closest_pair: replace the O(n*m) cross product with an
O(n log n) sort + O(n) two-pointer merge, and early-out once a pair
falls within the proximity threshold. The inputs are attacker-controlled
response-body matches past the body-size cap, so the quadratic form was a
latent DoS. Extract _match_gap to share the span-gap calc with the caller.
- dlp_detectors._compute_encoded_variants: back the memo with a bounded
functools.lru_cache instead of an unbounded module dict, so a long-lived
proxy seeing rotating secrets evicts rather than growing without limit.
- supervise_server: extract the duplicated routes.yaml inputSchema into
_proposal_input_schema()/_ROUTES_YAML_DESCRIPTION so the egress-allow and
egress-block tools can't drift.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
`ManifestIndex.load_for_agent` was a ~100-line method branching across
the eager (from_json_obj) and lazy (from disk) resolution modes, with
the git-user merge tail duplicated in both branches. Split into
`_load_for_agent_eager` / `_load_for_agent_lazy` behind a small
dispatcher and extract the shared tail into
`_manifest_with_merged_git_user`. No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
Two per-request hot-path costs in the egress DLP scanner:
- `_encoded_variants` derived the full variant set (gzip + nine
encodings) for every provisioned secret on every redaction and
known-secret scan — once per host, path, header, and body. Cache it
per distinct secret; callers still get a fresh list so they can't
corrupt the shared cached tuple.
- `_find_partial_window` searched the text once per secret n-gram,
giving O(len(secret) * len(text)). Build the secret's n-gram set once
and sweep the text a single time: O(len(text)), no coverage loss.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
git_gate.py (699 LOC) mixed three responsibilities. Split into:
- git_gate_render.py — pure host-side rendering: the gate constants,
GitGateUpstream, gitconfig/known-hosts rendering, and the entrypoint /
pre-receive / access-hook script builders.
- git_gate_provision.py — the gitea deploy-key lifecycle
(_provision_dynamic_key / revoke / _resolve_identity_file).
- git_gate.py — the GitGate ABC + GitGatePlan, now 169 LOC, re-exporting
all moved names (see __all__) so the 19 importers are unchanged.
Host-side only (not flat-bundled), so no sidecar import shim. The one
test that patched the internal `_provision_dynamic_key` lookup is
repointed to its new module (public API unchanged). The two new modules
are added to scripts/critical-modules.txt so the decompose doesn't move
security code out of the measured core — critical aggregate stays 95%
(git_gate 100%, render 100%, provision 97%).
Closes#303
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
The interactive multiselect loop nested key dispatch up to six indent
levels deep — the worst offender being the space-bar toggle
(while > if focus > elif key > if filtered > if/else membership) and
the long order-mode elif chain inside the focus branch.
Extract two behaviour-identical helpers:
- `_toggle_membership(items, item)` collapses the add/remove if/else,
pulling the space branch back to four levels.
- `_handle_order_key(key, selected, order_cursor)` moves the entire
order-focus dispatch out of the loop, returning the new cursor.
No control-flow or key-binding changes; the loop's early returns and
focus toggling are untouched. (git_gate.py's deep-looking lines named
in the issue are multiline call-argument continuations already under
four levels of control nesting, so no change was warranted there.)
Closes#288
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
`egress_addon_core.py` mixed the per-route `dlp:` block parser
(`_parse_detectors` plus the detector-name and `outbound_on_match`
constants) in with the request-time scan/decision flow. Move that
config-parsing layer into a new stdlib-only `egress_dlp_config.py` as
`parse_dlp_block`, so the decision path in the core module reads
top-to-bottom without scrolling past config plumbing.
The constants and parser are re-exported from `egress_addon_core`
(and listed in `__all__`) so existing `from egress_addon_core import
ON_MATCH_*` / `OUTBOUND_DETECTOR_NAMES` callers are unchanged. The new
module ships flat into the sidecar bundle (Dockerfile.sidecars) and
uses the same flat/package import shim as its siblings. Pure refactor;
behavior and wire format unchanged.
Closes#287
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NkwFXLFff9PYPy4wgVBJp9
Three UX improvements requested in #270 review:
- filter_multiselect: Space toggles selection, Enter confirms (was both)
- bottle picker: bottles with extends chains show ancestry labels
(e.g. 'claude-dev <- bot-bottle-dev <- dev') for at-a-glance lineage
- preflight: replaces key-value summary with YAML of the resolved manifest
Tab switches focus to the selected-order panel; K/J shift the
highlighted item up/down; Space/Enter removes it. The filter list dims
while the order panel is active. Help line updates per focus mode.
- `bottle:` in agent frontmatter is now optional; agents without it
are portable and require bottles to be selected at launch.
- Adds `filter_multiselect` to `tui.py`: multi-select picker with
ordered selection list, Space/Enter to toggle, Ctrl-D to confirm.
- `ManifestIndex` gains `all_bottle_names` and `load_for_agent` accepts
`bottle_names: tuple[str, ...]` to merge bottles in order at runtime.
- `merge_bottles_runtime` in `manifest_extends.py` applies the same
field-merge rules as `extends:` to pre-resolved bottle objects.
- `BottleSpec` gains `bottle_names`; `_validate` and `write_launch_metadata`
thread it through so `resume` replays the same bottle configuration.
- `cmd_start` shows the bottle multiselect after agent selection,
pre-populated from the agent's `bottle:` field when present.
- Existing agents with `bottle:` declared continue to work unchanged.
Allow extends: to accept a list of bottle names in addition to a plain
string. Parents are resolved independently and folded left-to-right
into a single combined parent before the child is merged on top, so
orthogonal concerns (base env, networking, agent provider) can live in
separate bottles without forcing a linear chain.
Merge rules for the parent fold: env dict-merge with later winning on
collision; git-gate.user per-field overlay; git-gate.repos union by
name with later winning per-field on same name; egress.routes
concatenated; all scalar fields (supervise, agent_provider, egress.log)
use last-wins. The existing child-wins-over-all-parents rule is
unchanged. Cycle detection, diamond deduplication, and missing/invalid
parent errors all work across multi-parent graphs.
Closes#268
Closes#258.
`egress_render_routes` and `_render_match_entry` now pass all manifest
strings (host, auth_scheme, token_env, path/header values) through
`_yaml_str_escape` before interpolating into double-quoted YAML scalars,
preventing stray `"` or newlines from corrupting routes.yaml.
`git_gate_render_gitconfig` now calls `_gitconfig_validate_value` on
each Upstream value (and the derived alias) before writing the
`insteadOf` line, rejecting any value containing a newline that would
inject arbitrary gitconfig keys.
The constant now covers the daemon path, the HTTP backend access-hook,
and the git http-backend CGI subprocess, so 'daemon' in the name was
too narrow. Updated the comment to list all three current uses.
Closes#255. Without timeouts, a hung upstream during the access-hook
or git http-backend CGI call (git_http_backend.py) and a stalled Gitea
API during deploy-key provisioning (contrib/gitea/deploy_key_provisioner.py)
could wedge a sidecar indefinitely. Adds GIT_HTTP_BACKEND_TIMEOUT_SECS
(30s) to both subprocess.run calls in the HTTP backend, mirroring the
existing GIT_GATE_DAEMON_TIMEOUT_SECS on the daemon path. Adds
_API_TIMEOUT_SECS (30s) and _KEYGEN_TIMEOUT_SECS (10s) to the Gitea
provisioner's urlopen and ssh-keygen calls. Tests verify the timeout
values are forwarded in all four call sites.
Introduce _RpcClientError and _RpcInternalError as distinct subclasses
of _RpcError so the dispatcher can handle bad requests and server-side
faults differently — returning client errors verbatim and logging
internal faults with their cause before replying ERR_INTERNAL.
Wrap write_proposal and archive_proposal IO with _RpcInternalError
so OS failures surface through the typed path instead of the bare
Exception fallback. All existing raise _RpcError(...) call sites
converted to _RpcClientError.
Closes#253
An empty or non-numeric Status: header from git http-backend raised
ValueError/IndexError that escaped the handler thread. Wrap the parse
in a try/except and fall back to HTTP 500 instead.
Closes#254
`container build` resolves -f relative to the current working directory,
not the build context, so builds failed from any cwd other than the repo
root. Anchor a relative Dockerfile to the context before passing it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_log_request and _log_response wrote headers and bodies to stderr verbatim.
_log_request also included the sidecar-injected upstream Authorization value,
exposing live bearer tokens on every allowed request under LOG_FULL.
Apply redact_tokens to all header values and bodies in both log functions;
exclude the authorization header from _log_request entirely since its value
is always a live sidecar-injected credential by the time _log_request runs.
Closes#257
When a supervisor-approved safe-token exactly matched an env secret
(Pass 1), Passes 2 & 3 (alnum projection) still ran and re-blocked on
the same value. Track whether any variant was found-and-approved and
skip the projection passes for that secret in that case.
EgressPlan gains a `canary: str` field (default "") populated in Egress.prepare()
using secrets.token_urlsafe(32). Each launched bottle:
- sidecar receives EGRESS_TOKEN_CANARY=<value> (literal env entry, scanned by
existing known-secrets detector without any detector code changes)
- agent receives BOT_BOTTLE_CANARY=<value> (visible fake secret that signals
exfiltration with zero false positives if it appears in outbound traffic)
Docker compose and macos-container backends updated; smolmachines shares docker
compose and so picks this up automatically. Unit tests cover canary uniqueness,
detection via scan_known_secrets, and EgressPlan backward-compat default.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
didericis-claude
didericis-codex
didericis