Allow extends: to accept a list of bottle names in addition to a plain
string. Parents are resolved independently and folded left-to-right
into a single combined parent before the child is merged on top, so
orthogonal concerns (base env, networking, agent provider) can live in
separate bottles without forcing a linear chain.
Merge rules for the parent fold: env dict-merge with later winning on
collision; git-gate.user per-field overlay; git-gate.repos union by
name with later winning per-field on same name; egress.routes
concatenated; all scalar fields (supervise, agent_provider, egress.log)
use last-wins. The existing child-wins-over-all-parents rule is
unchanged. Cycle detection, diamond deduplication, and missing/invalid
parent errors all work across multi-parent graphs.
Closes#268
Closes#258.
`egress_render_routes` and `_render_match_entry` now pass all manifest
strings (host, auth_scheme, token_env, path/header values) through
`_yaml_str_escape` before interpolating into double-quoted YAML scalars,
preventing stray `"` or newlines from corrupting routes.yaml.
`git_gate_render_gitconfig` now calls `_gitconfig_validate_value` on
each Upstream value (and the derived alias) before writing the
`insteadOf` line, rejecting any value containing a newline that would
inject arbitrary gitconfig keys.
The constant now covers the daemon path, the HTTP backend access-hook,
and the git http-backend CGI subprocess, so 'daemon' in the name was
too narrow. Updated the comment to list all three current uses.
Closes#255. Without timeouts, a hung upstream during the access-hook
or git http-backend CGI call (git_http_backend.py) and a stalled Gitea
API during deploy-key provisioning (contrib/gitea/deploy_key_provisioner.py)
could wedge a sidecar indefinitely. Adds GIT_HTTP_BACKEND_TIMEOUT_SECS
(30s) to both subprocess.run calls in the HTTP backend, mirroring the
existing GIT_GATE_DAEMON_TIMEOUT_SECS on the daemon path. Adds
_API_TIMEOUT_SECS (30s) and _KEYGEN_TIMEOUT_SECS (10s) to the Gitea
provisioner's urlopen and ssh-keygen calls. Tests verify the timeout
values are forwarded in all four call sites.
Introduce _RpcClientError and _RpcInternalError as distinct subclasses
of _RpcError so the dispatcher can handle bad requests and server-side
faults differently — returning client errors verbatim and logging
internal faults with their cause before replying ERR_INTERNAL.
Wrap write_proposal and archive_proposal IO with _RpcInternalError
so OS failures surface through the typed path instead of the bare
Exception fallback. All existing raise _RpcError(...) call sites
converted to _RpcClientError.
Closes#253
An empty or non-numeric Status: header from git http-backend raised
ValueError/IndexError that escaped the handler thread. Wrap the parse
in a try/except and fall back to HTTP 500 instead.
Closes#254
`container build` resolves -f relative to the current working directory,
not the build context, so builds failed from any cwd other than the repo
root. Anchor a relative Dockerfile to the context before passing it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_log_request and _log_response wrote headers and bodies to stderr verbatim.
_log_request also included the sidecar-injected upstream Authorization value,
exposing live bearer tokens on every allowed request under LOG_FULL.
Apply redact_tokens to all header values and bodies in both log functions;
exclude the authorization header from _log_request entirely since its value
is always a live sidecar-injected credential by the time _log_request runs.
Closes#257
When a supervisor-approved safe-token exactly matched an env secret
(Pass 1), Passes 2 & 3 (alnum projection) still ran and re-blocked on
the same value. Track whether any variant was found-and-approved and
skip the projection passes for that secret in that case.
EgressPlan gains a `canary: str` field (default "") populated in Egress.prepare()
using secrets.token_urlsafe(32). Each launched bottle:
- sidecar receives EGRESS_TOKEN_CANARY=<value> (literal env entry, scanned by
existing known-secrets detector without any detector code changes)
- agent receives BOT_BOTTLE_CANARY=<value> (visible fake secret that signals
exfiltration with zero false positives if it appears in outbound traffic)
Docker compose and macos-container backends updated; smolmachines shares docker
compose and so picks this up automatically. Unit tests cover canary uniqueness,
detection via scan_known_secrets, and EgressPlan backward-compat default.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- _alnum_projection(): strip non-alphanumeric chars for separator-injection detection
- scan_known_secrets() gains two extra passes per secret after exact-variant matching:
alnum-projection exact match (catches hyphens/spaces between secret chars) and a
sliding-window partial-match scan (catches chunked substrings ≥ PARTIAL_MATCH_MIN_LEN)
- scan_known_secrets() accepts sensitive_prefixes param (default ("EGRESS_TOKEN_",))
so redact_tokens and call-sites can extend the scanned env-var prefix set
- scan_entropy() warn-only detector flagging windows with Shannon entropy ≥ 5.5 bits/char
- "entropy" added to OUTBOUND_DETECTOR_NAMES; scan_outbound opts it in only when
explicitly listed in dlp.outbound_detectors (never part of the default "all" set)
- scan_outbound reads BOT_BOTTLE_SENSITIVE_PREFIXES from environ to extend
scan_known_secrets beyond EGRESS_TOKEN_* without schema changes
- Binary bodies decoded via latin-1 fallback (bijective byte↔codepoint) instead
of utf-8 errors=replace, preserving ASCII secret strings in binary payloads
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gitea returns HTTP 422 when a deploy key title or public key content
already exists on the repo. The provisioner previously surfaced this
as a generic RuntimeError with the raw status code. Introduce
DeployKeyCollisionError (a RuntimeError subclass) in the base module
and detect 422 in GiteaDeployKeyProvisioner.create so callers can
catch collisions explicitly and the error message names the repo and
title involved.
The constant and its MCP tool name ("allow" → "egress-allow") were the
only supervise tools without an egress-scoped identifier, despite the
tool being egress-only (routes.yaml payload, COMPONENT_FOR_TOOL maps
it to "egress", always grouped with TOOL_EGRESS_BLOCK). The rename
brings it in line with TOOL_EGRESS_BLOCK and TOOL_EGRESS_TOKEN_ALLOW,
and adds TOOL_EGRESS_ALLOW and TOOL_EGRESS_BLOCK to __all__ (both were
previously absent).
Provider routes (the agent talking to its own LLM API — api.anthropic.com,
the Codex backend, etc.) carry the whole conversation payload, which is the
worst source of token-shaped false positives. egress_routes_for_bottle now
fills outbound_on_match=redact on any provider route that doesn't set it
explicitly, so a match there is scrubbed and forwarded rather than blocked
or queued for the operator. A provider that sets the policy keeps its
choice; manifest routes still default to supervise.
Tests: provider route gets redact default, explicit provider policy
preserved, manifest route unaffected. README + PRD 0062 updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
A 403 "egress DLP: URL-encoded CRLF (%0d%0a)" was firing on legitimate
requests (e.g. the Claude Code login flow) and bypassing the on-match
policy entirely, because CRLF blocks carry no matched value and were
routed straight to a hard 403.
Root cause: CRLF injection is only an attack in the request line and
headers. An HTTP body is delimited by Content-Length, so CRLF bytes in
the body cannot split the request — but the scan flattened the body into
the same blob it checked, so form-encoded / multi-line body content
(which legitimately contains %0d%0a) tripped it.
Fix:
- scan_outbound takes a crlf_text param; the addon scans CRLF only over
the body-excluded request line + headers. crlf_text=None keeps the
old full-blob behavior for host-side callers/tests; the websocket path
passes "" since a data frame is not a request line.
- The redact policy now also scrubs CRLF (new strip_crlf helper) from the
path and headers, so redact is a complete escape hatch and structural
CRLF in the URL/headers can be forwarded when a route opts into it.
Tests: strip_crlf unit tests; scan_outbound crlf_text body-exclusion and
backward-compat tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
Give each egress route a policy for what the proxy does when an outbound
DLP detector matches a token, defaulting to the supervise flow added in
the previous commit. The goal is cutting false-positive friction without
weakening default-deny.
- redact: scrub the matched value(s) from the body, non-host headers, and
path/query via redact_tokens, then re-scan. Forward if clean; fail
closed with a 403 if a match remains on a surface redaction can't
rewrite (the hostname, or a unicode-evasion token). For routes where a
token-shaped value is noise the upstream doesn't need.
- block: the original hard 403, never overridable.
- supervise (default, unset): hold the request for operator approval.
Structural blocks (CRLF, no safelist-able value) stay hard 403s under
every policy.
Threads outbound_on_match from the bottle manifest (manifest_egress)
through the resolved EgressRoute and rendered routes.yaml (egress.py) to
the addon's Route (egress_addon_core), and round-trips it via the
list-egress-routes introspection endpoint. The allow/egress-block tool
descriptions document the new key.
Tests: manifest parse/validation, core parse/validation, full
manifest->render->addon round-trip for redact. README + PRD 0062 updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
When the outbound DLP catches a token, route the block through the
existing supervisor approval queue instead of returning 403 outright.
The egress proxy holds the request open until the operator answers, then
remembers an approved value for the life of the proxy so the request --
and later ones carrying it -- flow through. Fails closed on rejection,
timeout, malformed response, or when supervise is disabled.
- ScanResult.matched carries the raw matched substring (sidecar-only;
never logged or written to the proposal). scan_outbound and the token
detectors take a safe_tokens set and skip approved values, continuing
past a safelisted match so a second secret in the same request is
still caught.
- New egress-token-allow proposal tool, written directly to the queue by
the addon (the gitleaks-allow pattern from PRD 0061). build_token_allow
_payload renders host/method/path/detector reason + redacted context.
- Async request hook polls the queue without stalling the proxy event
loop; EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS (default 300) bounds the wait.
- Supervisor TUI renders egress-token-allow like gitleaks-allow: report
only, modify unavailable, approval requires a recorded reason.
- Unit tests for the matched/safe-tokens plumbing, payload builder, tool
constant round-trip, and TUI paths; README + PRD 0062.
Closes#261.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
log.py was bare print-to-stderr wrappers with no levels or attributable
context (issue #252). Add:
- Ordered severities (debug/info/warn/error) gated by
BOT_BOTTLE_LOG_LEVEL (default info). debug is silent by default;
error always surfaces (nothing sits above it), so the fatal die path
stays visible regardless of configured level.
- An optional `context` mapping on every wrapper, rendered as a
parseable ` [k=v ...]` suffix (keys sorted; whitespace/quoted values
quoted) so failures can be filtered and correlated.
Default output with no context is byte-identical to the original lines,
so the 100+ existing single-string call sites are unaffected. Wires the
supervise crash path (the example the issue names) to attach error_type
and crash_log context. Adds test_log.py (backward-compat, context
rendering, level gating, die surfacing).
Closes#252.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
Issue #249: bottles should be supervised by default. Rather than
remove the flag (which would make supervision mandatory and is the
wrong plane for cost-control enforcement — see #251), keep the
opt-out and flip the default. Bottles that omit `supervise:` now get
the stuck-recovery sidecar; `supervise: false` still skips it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YcU7nerbg8cVj9R4EkpfLJ
The TUI was calling archive_proposal for gitleaks-allow immediately
after write_response, moving the response file to processed/ within
microseconds. The git-gate shell loop polls queue_dir for the response
file every second — it never sees it and hangs until timeout.
capability-block is handled by the MCP sidecar which archives after
reading; gitleaks-allow is handled by the shell gate which archives
after processing. Let the gate own the archive step.
On resume from a committed snapshot, smolvm's pack process remaps all
file uids to the host uid (501 on macOS). Files in /tmp that were
created during the session (e.g. /tmp/claude-1000 owned by node=uid
1000) get remapped to 501. Claude Code then refuses to use the temp
directory because it's owned by a different uid.
Two-part fix:
- Exclude ./tmp and ./var/tmp from the tar in _exec_tar_to_file.
Both directories are ephemeral; a resumed VM should start with clean
temp directories identical to a fresh VM.
- Add mkdir -p /tmp /var/tmp to _init_vm before chown/chmod, so the
directories are created if the committed snapshot omitted them.
Replace the Popen/stdout=PIPE approach with a write-then-copy
strategy that avoids binary-stdout piping through the smolvm exec
channel entirely:
1. Probe connectivity with `machine_exec(machine, ["true"])` first.
If this fails while an interactive session is running, the error
now says "concurrent exec not available" instead of the opaque
"<no stderr>".
2. Run `tar --create --gzip --file=/var/tmp/.bot-bottle-commit.tar.gz`
inside the VM via machine_exec (same mechanism used during
provisioning). tar writes to a file in the VM, not stdout, so
smolvm never has to transmit binary data over the exec channel.
3. Copy the compressed archive to the host with machine_cp.
4. Dockerfile switches to ADD rootfs.tar.gz / — Docker decompresses
gzip tarballs automatically.
smolvm machine exec requires stdout to be a pipe, not a regular
file descriptor. Passing stdout=file caused smolvm to return
non-zero with no stderr (the error was silently swallowed or went
to the regular-file fd instead of reaching us).
Switch _snapshot_running_vm to a new _exec_tar_to_file helper that
uses Popen with stdout=PIPE and streams the tar to disk via
shutil.copyfileobj. A background thread drains stderr concurrently
to prevent deadlock when the stderr pipe buffer fills while we are
writing stdout data.
The terminal-decoration wrapper script is invoked with sh -lc, which
sources login-shell init files (/etc/profile, ~/.profile) rather than
interactive-shell files (~/.zshrc). smolvm is typically installed via
homebrew whose PATH setup lands in ~/.zprofile or ~/.zshrc — not picked
up by sh -l — so pty_resize.py's Popen(["smolvm", ...]) raises
FileNotFoundError, pty_resize exits non-zero, and the trailing reset-
printf makes sh exit 0. The caller sees "session ended (exit 0)"
immediately with no agent output.
Use sh -c instead. The calling process (./cli.py) inherits the user's
interactive shell PATH where smolvm is present, confirmed by the
provision steps (machine_exec) succeeding before exec_agent is reached.
smolvm pack create --from-vm requires the VM to be stopped, and stopping
a smolmachines VM terminates any running interactive session.
Instead, mirror the macos-container approach: exec into the running VM as
root and stream the root filesystem via tar (smolvm machine exec -- tar),
build a Docker image from the archive, push to an ephemeral local registry,
and run smolvm pack create --image to produce the .smolmachine artifact.
The VM stays running throughout the commit.
Remove the stop-confirm prompt and machine_is_running check that were
added in the previous commit — neither is needed when we no longer stop.
smolvm pack create --from-vm requires the VM to be stopped. Add
machine_is_running() to smolvm.py (via machine ls --json state field),
and add the same confirm-stop flow to SmolmachinesFreezer that was
originally designed for macos-container: if running, prompt the user,
stop the VM, then pack. Already-stopped VMs are packed directly.
Apple Container removes containers when they stop, making the
stop-then-export flow impossible regardless of the --rm flag.
Replace `container export` (requires stopped container) with
`container exec --user root <name> tar --create ... --file=- --directory=/ .`
streamed to a temp file, then build the committed image from that archive
as before. The bottle stays running after commit, which is better UX.
Drop the stop-confirm prompt from MacosContainerFreezer since we no longer
need to stop the container at all.
container stop was removing the container immediately (due to --rm)
before container export could run. The force_remove_container teardown
callback on the ExitStack already handles cleanup on normal exit, so
--rm was redundant. Without it, the stopped container stays available
for container export to snapshot.
Freezer._freeze only ever used bottle.name, which is always
f"bot-bottle-{agent.slug}". Remove the Bottle parameter from
commit() and _freeze(), derive the container name from agent.slug
directly in each subclass, and delete the _NamedBottle stub that
existed solely to paper over this.
Adds a Freezer ABC (backend/freeze.py) that encapsulates the
stop-commit-mark-preserved flow for all backends, following the same
pattern as BottleBackend. Each backend gets its own Freezer subclass:
DockerFreezer — docker commit
MacosContainerFreezer — container export + image rebuild; prompts
to stop if the container is running
SmolmachinesFreezer — smolvm pack create --from-vm
The base class owns write_committed_image, mark_preserved, and the
resume hint. Subclasses implement _freeze() and optionally override
_export_hint() for migration instructions.
Freezer.commit(agent, bottle) is the primary entry point for use
within a live launch context. Freezer.commit_slug(slug) is a
convenience wrapper for cmd_commit, which no longer branches on
backend names itself.
get_freezer(backend_name) is the factory, analogous to
get_bottle_backend(). CommitCancelled is raised by MacosContainerFreezer
when the user declines the stop prompt; cmd_commit catches it and
returns 0.
`container export` requires the container to be stopped first. When a
running bottle is detected, prompt the user to confirm, stop the
container, then commit. Adds `container_is_running` and
`stop_container` helpers to the macos-container util.
Addresses #240 (comment)
didericis
didericis-codex
didericis-claude