fix(gateway): persist mitmproxy CA on the host, not a named volume (#450)
lint / lint (push) Successful in 49s
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / integration-docker (pull_request) Successful in 12s
test / unit (pull_request) Successful in 36s
test / integration-firecracker (pull_request) Successful in 3m27s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped

The shared gateway self-generates a mitmproxy CA that every bottle installs
to trust its TLS interception. It was persisted on a Docker named volume,
which survives `docker rm` but is silently wiped by `docker volume prune` /
`docker system prune --volumes` during routine host maintenance. When that
happens the gateway mints a fresh CA on restart, and every already-running
bottle fails the TLS handshake even after it re-resolves and reconnects to
the moved gateway — a re-attachment blocker distinct from #443/#445.

Move CA persistence to a host bind-mount under the app-data root
(`bot_bottle_root()/gateway-ca`, via `host_gateway_ca_dir()`), mirroring how
the shared DB and control-plane token already live on the host. Docker never
prunes a path under the root, and it stays inspectable + rotatable from the
host. mitmproxy already adopts an existing CA and generates one only on first
run, so the bind-mount gives adopt-existing/generate-on-first-run for free.

Add an explicit rollover path: `rotate_gateway_ca()` clears the persisted CA
so the next start remints it, and `python -m bot_bottle.orchestrator.rotate_ca`
wires that together with dropping the running gateway container (whose
mitmproxy still holds the old CA in memory). Rotation stays an operator action
— it doesn't auto-re-provision running bottles, which re-attach to pick up the
new anchor.

Scope: the Docker infra/gateway path (the "infra container" in the report).
The macOS (`container`-only volume) and Firecracker (VM-attached ext4) backends
persist the CA differently and are unaffected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-21 18:16:28 +00:00
parent 4199de5e3e
commit f52ac0ebbf
8 changed files with 301 additions and 16 deletions
@@ -16,6 +16,7 @@ from bot_bottle.orchestrator.lifecycle import (
OrchestratorStartError,
source_hash,
)
from bot_bottle.paths import GATEWAY_CA_DIRNAME
from tests.unit import use_bottle_root
_URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen"
@@ -115,6 +116,14 @@ class TestOrchestratorService(unittest.TestCase):
# Gateway daemons + orchestrator explicitly opted in.
daemons_flag = "BOT_BOTTLE_GATEWAY_DAEMONS=egress,git-http,supervise,orchestrator"
self.assertIn("orchestrator", argv[argv.index(daemons_flag)])
# The mitmproxy CA persists on a HOST bind-mount under the app-data root
# (not a docker named volume `docker volume prune` would wipe — #450), so
# a restarted infra container keeps the CA every running bottle trusts.
ca_mounts = [a for a in argv if a.endswith(":/home/mitmproxy/.mitmproxy")]
self.assertEqual(1, len(ca_mounts))
src = ca_mounts[0].rsplit(":", 1)[0]
self.assertTrue(src.startswith(self._tmp.name), src)
self.assertTrue(src.endswith("/" + GATEWAY_CA_DIRNAME), src)
def test_ensure_running_builds_all_images(self) -> None:
calls: list[list[str]] = []