fix(gateway): persist git-gate state across gateway restarts
tracker-policy-pr / check-pr (pull_request) Successful in 12s
lint / lint (push) Successful in 59s
test / unit (pull_request) Successful in 53s
test / integration-docker (pull_request) Successful in 1m7s
test / coverage (pull_request) Successful in 19s

Per-bottle git-gate state (bare repos under /git/<id>, deploy creds under
/git-gate/creds/<id>) was provisioned once at bottle launch and lived only
in the gateway's ephemeral storage. A gateway rebuild/restart wiped it and
nothing re-provisioned already-running bottles, so their agents 404'd on
fetch/push. Same class of bug as the CA (#510); the orchestrator restores
only egress tokens, not git-gate declarations.

Persist the state on both backends, mirroring the CA-persistence approach:

- firecracker: attach a second persistent data drive (/dev/vdc) to the
  gateway VM and bind-mount its git/ + creds/ subdirs onto /git and
  /git-gate/creds in the gateway guest init, before the data plane starts.
  Generalize the VM config to a stable-ordered data_drives tuple (CA=vdb,
  git=vdc; orchestrator registry stays vdb).
- docker: bind-mount host dirs (host_gateway_git_dir / creds_dir, under the
  never-pruned app-data root) onto /git and /git-gate/creds, with
  BOT_BOTTLE_DOCKER_GIT_MOUNT / _CREDS_MOUNT env overrides so CI isolates
  them to per-run volumes it cleans up.

Teardown already rm -rf's /git/<id> + creds, so the persistent store
self-cleans over the normal lifecycle.

Closes #512

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-26 17:34:22 -04:00
parent 0ddaa95c14
commit dff811f190
13 changed files with 213 additions and 27 deletions
@@ -84,7 +84,7 @@ def _config(
vcpus: int,
mem_mib: int,
guest_mac: str,
data_drive: Path | None = None,
data_drives: tuple[Path, ...] = (),
extra_boot_args: str = "",
) -> dict[str, object]:
drives: list[dict[str, object]] = [
@@ -95,12 +95,15 @@ def _config(
"is_read_only": False,
}
]
# A second virtio-block device (guest /dev/vdb) — the infra VM's
# persistent registry "volume", a host-side ext4 file that outlives the
# ephemeral rootfs across VM restarts.
if data_drive is not None:
# Extra virtio-block devices — the infra VMs' persistent "volumes",
# host-side ext4 files that outlive the ephemeral rootfs across VM restarts.
# They appear to the guest as /dev/vdb, /dev/vdc, ... in list order (after
# the root /dev/vda), so callers must keep the order stable: the orchestrator
# attaches its registry (vdb); the gateway attaches its CA (vdb) then its
# git-gate state (vdc).
for i, data_drive in enumerate(data_drives):
drives.append({
"drive_id": "data",
"drive_id": f"data{i}",
"path_on_host": str(data_drive),
"is_root_device": False,
"is_read_only": False,
@@ -138,7 +141,7 @@ def boot(
mem_mib: int = 2048,
guest_mac: str = "06:00:AC:10:00:02",
detached: bool = False,
data_drive: Path | None = None,
data_drives: tuple[Path, ...] = (),
extra_boot_args: str = "",
) -> VmHandle:
"""Write the config and launch the VMM. Returns once the process is
@@ -155,7 +158,7 @@ def boot(
_config(
rootfs=rootfs, tap=tap, guest_ip=guest_ip, host_ip=host_ip,
pubkey=pubkey, vcpus=vcpus, mem_mib=mem_mib, guest_mac=guest_mac,
data_drive=data_drive, extra_boot_args=extra_boot_args,
data_drives=data_drives, extra_boot_args=extra_boot_args,
),
indent=2,
))