Compare commits

..

3 Commits

Author SHA1 Message Date
didericis-claude 8a82861459 test: satisfy pyright strict annotations on the new supervise RPC tests
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 19s
test / unit (pull_request) Failing after 50s
lint / lint (push) Successful in 2m51s
test / integration-firecracker (pull_request) Successful in 3m34s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Add parameter/return annotations to the fake resolver's propose_supervise /
poll_supervise, annotate the test helpers and the shared _ARGS payload, and
narrow the None-able poll result — no behavior change. pylint 9.83, pyright
clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 02:20:56 +00:00
didericis-claude f0d2a4b855 fix(supervise): reach the queue over RPC, get bot-bottle.db off the data plane
lint / lint (push) Failing after 57s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / unit (pull_request) Failing after 37s
test / integration-docker (pull_request) Successful in 38s
test / integration-firecracker (pull_request) Successful in 3m16s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
PRD 0070's rule — only the orchestrator opens bot-bottle.db; the data
plane reaches state through the control-plane RPC — was not in force.
Three data-plane daemons held a direct read-write handle on the shared
SQLite file: the supervise MCP server, the egress DLP addon (the most
attack-exposed process, TLS-bumping hostile traffic), and the git-gate
pre-receive hook. An RCE in any of them could read every bottle's
plaintext identity_token and forge attribution fleet-wide (issue #469).

Add the agent half of the supervise flow to the control plane:

  POST /supervise/propose  -> queue a proposal, 201 {proposal_id}
  POST /supervise/poll     -> non-blocking decision poll, 200 {status,...}

Both attribute the caller by (source_ip, identity_token) exactly like
/resolve — never a caller-supplied slug — so a bottle can only ever queue
or read its own proposals even if the data plane is compromised. A decided
poll archives server-side, preserving the archive-after-read contract.

Data plane: the supervise server, egress addon, and git-gate hook now
queue/poll through PolicyResolver.propose_supervise / poll_supervise
instead of opening the DB. supervise_server keeps its ~30s grace window
by polling the RPC; egress keeps its safelist keyed by resolved bottle;
the git-gate hook gets (source_ip, identity_token) from the CGI env.

Packaging: drop the DB bind-mount and SUPERVISE_DB_PATH from the
data-plane containers/VMs (docker gateway + infra, macOS infra,
firecracker infra). The orchestrator remains the sole opener of the one
file via BOT_BOTTLE_ROOT / host_db_path().

Update PRD 0070: the rule is now in force; remove the transitional caveat.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 02:07:21 +00:00
didericis-claude 44122e2728 fix(egress): restart dead daemons and cap inbound scan body size (#455)
test / integration-docker (pull_request) Successful in 20s
lint / lint (push) Successful in 1m1s
test / unit (pull_request) Failing after 1m46s
test / integration-firecracker (pull_request) Successful in 3m22s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 10m4s
Two complementary fixes for the egress gateway OOM:

1. gateway_init: auto-restart any daemon that dies unexpectedly. The
   supervisor already had restart_daemon()/request_restart() logic; this
   wires it into tick() so an OOM-killed mitmdump is respawned without
   operator intervention. Implements the "eventual" failure policy
   described in the original module docstring.

2. egress_addon: cap response body bytes passed to the DLP inbound scan
   at EGRESS_INBOUND_SCAN_LIMIT_BYTES (default 1 MiB). mitmproxy buffers
   the full response before the hook fires; capping at scan time limits
   the additional amplification from decoded text and regex strings. Bodies
   over the limit emit an egress_scan_truncated log event. Set to 0 to
   disable the cap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-23 22:49:34 +00:00
7 changed files with 21 additions and 93 deletions
+1 -6
View File
@@ -14,7 +14,6 @@ the private orchestrator `_launch_bottle`.
from __future__ import annotations
import argparse
import io
import os
import shutil
import sys
@@ -196,11 +195,7 @@ def _start_headless(
path, so the agent still execs on the inherited stdio/PTY — an
orchestrator allocates that PTY and relays it to its
desktop/mobile clients."""
try:
stdin_fd = sys.stdin.fileno()
except io.UnsupportedOperation:
stdin_fd = -1
if not os.isatty(stdin_fd):
if not os.isatty(sys.stdin.fileno()):
die(
"--headless requires a PTY on stdin; run via:\n"
" script -q /dev/null ./cli.py start ..."
+6 -16
View File
@@ -9,7 +9,8 @@ Failure policy: when a child dies unexpectedly, the supervisor
restarts it automatically and logs the restart. The gateway stays
up; a temporary loss of one daemon (e.g. egress OOM-killed) is
recovered without manual container recreation. The supervisor
itself exits only when the operator sends SIGTERM/SIGINT.
itself exits only when (a) the operator sends SIGTERM/SIGINT, or
(b) every child has died.
Daemon subset is env-driven via `BOT_BOTTLE_GATEWAY_DAEMONS=egress`
for callers that don't use git-gate or supervise. Default: all
@@ -220,10 +221,9 @@ class _Supervisor:
"""One iteration of the watch loop. Returns True when every
child has exited and the supervisor can return.
A child dying unexpectedly is logged and restarted but does
NOT initiate shutdown — see the module docstring's
failure-policy section. Shutdown is signal-driven only."""
restarted_children = bool(self._restart_requested)
A child dying unexpectedly is logged but does NOT initiate
shutdown — see the module docstring's failure-policy
section. Shutdown is signal-driven only."""
self._drain_restart_requests()
for spec, p in self.procs:
@@ -237,13 +237,6 @@ class _Supervisor:
else:
_log(f"{spec.name} exited with code {rc}")
# Restart deaths discovered above before checking whether all
# processes are done. Deferring this until the next tick would make a
# single-daemon supervisor return True and exit with the restart still
# queued.
restarted_children |= bool(self._restart_requested)
self._drain_restart_requests()
if self.shutdown_at is not None:
elapsed = time.monotonic() - self.shutdown_at
if elapsed > _GRACE_SECONDS:
@@ -257,10 +250,7 @@ class _Supervisor:
)
self._sigkill_all()
done = (
not restarted_children
and all(p.poll() is not None for _, p in self.procs)
)
done = all(p.poll() is not None for _, p in self.procs)
if done:
for _, p in self.procs:
if p.stdout is not None:
+1 -1
View File
@@ -57,7 +57,7 @@ def _merge_two_bottles_runtime(base: "ManifestBottle", override: "ManifestBottle
n for n in override_repos_by_name if n not in base_repos_by_name
]
merged_git = tuple(
override_repos_by_name[n] if n in override_repos_by_name else base_repos_by_name[n]
override_repos_by_name.get(n, base_repos_by_name[n])
for n in merged_repos_names
)
+3 -9
View File
@@ -1,14 +1,8 @@
"""Foundational filesystem paths for bot-bottle.
`bot_bottle_root()` is the app data root per-bottle state, git-gate
keys, the gateway CA, and the shared SQLite DB all live under it. It
defaults to `~/.bot-bottle` and is overridable with the
**`BOT_BOTTLE_ROOT`** env var.
Note that the supervise queue and the audit log are *tables in the shared
DB*, not directories under the root see `queue_store.py` / `audit_store.py`.
The root held a `queue/` directory before the SQLite migration (PRD 0067);
nothing writes there now.
`bot_bottle_root()` is the app data root state, queue, audit logs,
git-gate keys, and the shared DB all live under it. It defaults to
`~/.bot-bottle` and is overridable with the **`BOT_BOTTLE_ROOT`** env var.
The env override is the single knob for redirecting the root: the test
suite points it at a throwaway dir instead of monkey-patching the function
-3
View File
@@ -69,9 +69,6 @@ class TestCmdStartHeadless(unittest.TestCase):
self._modal = patch.object(tui_mod, "name_color_modal").start()
patch.dict(os.environ, {}, clear=False).start()
os.environ.pop("BOT_BOTTLE_BACKEND", None)
# PTY check uses os.isatty(sys.stdin.fileno()); stub both so
# headless unit tests aren't blocked on a real TTY.
patch("bot_bottle.cli.start.os.isatty", return_value=True).start()
self.addCleanup(patch.stopall)
def _spec(self):
+10 -33
View File
@@ -162,17 +162,16 @@ class TestSupervisor(unittest.TestCase):
return sup.exit_code()
def test_all_children_succeed_returns_zero(self):
# `sh -c :` exits 0 immediately. Start shutdown before driving
# the loop so the intentionally short-lived fixtures are not
# treated as unexpected deaths and restarted.
# `sh -c :` exits 0 immediately. With the new failure
# policy a child dying doesn't trigger shutdown, so the
# loop only converges once BOTH have exited on their own.
# Both exit 0 → max(0, 0) = 0.
specs = [
_DaemonSpec("a", ("/bin/sh", "-c", ":")),
_DaemonSpec("b", ("/bin/sh", "-c", ":")),
]
sup = _Supervisor(specs)
sup.start_all()
time.sleep(0.1)
sup.request_shutdown(reason="test")
rc = self._drive(sup)
self.assertEqual(0, rc)
@@ -209,23 +208,6 @@ class TestSupervisor(unittest.TestCase):
sup.request_shutdown(reason="test-teardown")
self._drive(sup)
def test_single_daemon_crash_is_restarted_before_tick_completes(self):
specs = [_DaemonSpec("crasher", ("/bin/sh", "-c", "exit 1"))]
sup = _Supervisor(specs)
sup.start_all()
original_pid = sup.procs[0][1].pid
time.sleep(0.1)
done = sup.tick()
self.assertFalse(done)
self.assertNotEqual(original_pid, sup.procs[0][1].pid)
self.assertEqual(set(), sup._restart_requested)
self.assertIsNone(sup.shutdown_at)
sup.request_shutdown(reason="test-teardown")
self._drive(sup)
def test_crash_then_signal_surfaces_nonzero_exit_code(self):
# The crasher's exit code is what reaches the container
# exit even though shutdown was triggered by SIGTERM.
@@ -242,25 +224,20 @@ class TestSupervisor(unittest.TestCase):
rc = self._drive(sup)
self.assertEqual(1, rc)
def test_all_children_die_unattended_are_restarted(self):
def test_all_children_die_unattended_loop_converges(self):
# If nobody sends a signal but every child eventually
# dies on its own, the supervisor still exits — nothing
# left to supervise.
specs = [
_DaemonSpec("a", ("/bin/sh", "-c", "exit 0")),
_DaemonSpec("b", ("/bin/sh", "-c", "exit 2")),
]
sup = _Supervisor(specs)
sup.start_all()
original_pids = [p.pid for _, p in sup.procs]
time.sleep(0.1)
done = sup.tick()
self.assertFalse(done)
self.assertNotEqual(original_pids, [p.pid for _, p in sup.procs])
rc = self._drive(sup)
self.assertEqual(2, rc)
self.assertIsNone(sup.shutdown_at)
sup.request_shutdown(reason="test-teardown")
self._drive(sup)
def test_forward_signal_to_named_child(self):
# SIGHUP needs to reach mitmdump inside the bundle so
# routes.yaml reloads (egress_apply.py issues `docker kill
-25
View File
@@ -25,10 +25,6 @@ def _bottle(**kwargs: object) -> ManifestBottle:
return ManifestBottle.from_dict("test", kwargs)
def _git_repo(url: str) -> dict[str, object]:
return {"url": url, "key": {"provider": "gitea", "forge_token_env": "TOK"}}
class TestMergeBottlesRuntime(unittest.TestCase):
def test_single_bottle_returns_as_is(self):
b = _bottle(env={"FOO": "1"})
@@ -119,27 +115,6 @@ class TestMergeBottlesRuntime(unittest.TestCase):
self.assertEqual("2", result.env["B"])
self.assertEqual("3", result.env["C"])
def test_git_repo_only_in_override_does_not_raise(self):
# Regression for issue #457: override bottle declares a repo that the
# base doesn't have → KeyError on base_repos_by_name[n].
base = _bottle(env={"X": "base"})
override = _bottle(**{"git-gate": {"repos": {"myrepo": _git_repo("ssh://git@example.com/repo.git")}}})
result = merge_bottles_runtime([base, override])
self.assertIn("myrepo", [e.Name for e in result.git])
def test_git_repo_only_in_base_survives_override(self):
base = _bottle(**{"git-gate": {"repos": {"myrepo": _git_repo("ssh://git@example.com/repo.git")}}})
override = _bottle(env={"X": "override"})
result = merge_bottles_runtime([base, override])
self.assertIn("myrepo", [e.Name for e in result.git])
def test_git_repo_override_wins_by_name(self):
base = _bottle(**{"git-gate": {"repos": {"myrepo": _git_repo("ssh://git@base.example.com/repo.git")}}})
override = _bottle(**{"git-gate": {"repos": {"myrepo": _git_repo("ssh://git@override.example.com/repo.git")}}})
result = merge_bottles_runtime([base, override])
self.assertEqual(1, len(result.git))
self.assertEqual("ssh://git@override.example.com/repo.git", result.git[0].Upstream)
def test_empty_list_raises(self):
with self.assertRaises(ValueError):
merge_bottles_runtime([])