didericis/bot-bottle
1
0
Fork 0
Code Issues 33 Pull Requests 9 Actions Packages Releases Wiki Activity

PRD: reprovision on gateway startup #517

Merged
didericis-claude merged 1 commits from prd-gateway-reprovision-on-bringup into main 2026-07-27 21:53:58 -04:00
Conversation 3 Commits 1 Files Changed 69
+937 -4015
didericis-claude commented 2026-07-26 18:48:43 -04:00
Collaborator
Copy Link
Copy Source

Implements PRD 0081 (reconcile running bottles against a freshly-booted gateway instead of persisting gateway state on volumes). Supersedes the abandoned persistent-volume PRs #511 / #513.

In this PR

  • docs/prds/0081-*.md — the design.
  • OrchestratorCore.update_agent_secret + registry.store_agent_secret (per-key upsert) + POST /bottles/<id>/secret (cli-only) + client.update_agent_secret — update ONE egress token in place, the single-secret counterpart of reprovision_from_secret's restore-all. This is the reusable primitive extracted from the 2026-07-26 hotfix; unit-tested across service/store/server/client.
  • reconcile.py — attach_bottled_agents_to_gateway: on gateway cold boot, push the new mitmproxy CA into every live agent VM's trust store, re-provision git-gate repos/creds from the persisted upstreams.json snapshot, and restore egress tokens. Per-bottle failures are logged and skipped.
  • git_gate/provision.py — writes upstreams.json after key provisioning so the reconcile can reconstruct upstream state without the manifest.
  • Wired into FirecrackerInfraService.ensure_running() cold-boot path; BottleBackend base gets a no-op default.

Refs #516. Addresses the root problems in #510 and #512 (closed by the reconcile once it lands).

Implements PRD 0081 (reconcile running bottles against a freshly-booted gateway instead of persisting gateway state on volumes). Supersedes the abandoned persistent-volume PRs #511 / #513. **In this PR** - `docs/prds/0081-*.md` — the design. - `OrchestratorCore.update_agent_secret` + `registry.store_agent_secret` (per-key upsert) + `POST /bottles/<id>/secret` (cli-only) + `client.update_agent_secret` — update ONE egress token in place, the single-secret counterpart of `reprovision_from_secret`'s restore-all. This is the reusable primitive extracted from the 2026-07-26 hotfix; unit-tested across service/store/server/client. - `reconcile.py` — `attach_bottled_agents_to_gateway`: on gateway cold boot, push the new mitmproxy CA into every live agent VM's trust store, re-provision git-gate repos/creds from the persisted `upstreams.json` snapshot, and restore egress tokens. Per-bottle failures are logged and skipped. - `git_gate/provision.py` — writes `upstreams.json` after key provisioning so the reconcile can reconstruct upstream state without the manifest. - Wired into `FirecrackerInfraService.ensure_running()` cold-boot path; `BottleBackend` base gets a no-op default. Refs #516. Addresses the root problems in #510 and #512 (closed by the reconcile once it lands).
didericis force-pushed prd-gateway-reprovision-on-bringup from eb8ac7f011 to f9d0e15f13 2026-07-26 18:52:17 -04:00 Compare
didericis reviewed 2026-07-26 19:01:58 -04:00
docs/prds/0081-reprovision-gateway-state-on-bringup.md Outdated
@@ -0,0 +57,4 @@
- The docker / macOS backends. This lands the reconcile for Firecracker (the
backend that boots a fresh gateway rootfs each time); docker keeps its existing
host-bind-mounted CA until the same seam is extended there (follow-up). The
unification target is *per-service*, not *per-backend*.
didericis commented 2026-07-26 19:01:57 -04:00
Owner
Copy Link
Copy Source

No, target is for both. And it should be enforced on the base abc class for the backend/there should be something like a attach_bottled_agents_to_gateway method on the backend that reprovisions all the bottled agents.

No, target is for both. And it should be enforced on the base abc class for the backend/there should be something like a `attach_bottled_agents_to_gateway` method on the backend that reprovisions all the bottled agents.
didericis-claude commented 2026-07-26 19:06:28 -04:00
Author
Collaborator
Copy Link
Copy Source

Re: the docker/macOS non-goal (review comment on docs/prds/0081-...md) — agreed, made it target all backends, enforced on the ABC.

Updated in 4434752d:

  • Dropped the docker/macOS non-goal.
  • Design now leads with the contract: attach_bottled_agents_to_gateway() as an @abc.abstractmethod on BottleBackend (backend/base.py). Reconciling running bottles is a backend responsibility (only the backend can enumerate its agents and reach them — firecracker over SSH, docker/macOS over exec/cp), so every backend must implement it — cross-backend by construction, no per-backend follow-up. It reprovisions all registered bottles' CA + git-gate + egress tokens.
  • The host calls it on the gateway bring-up path (cold-boot branch only, never on adopt).
  • New open question: once docker's attach_bottled_agents_to_gateway pushes the CA to running agents, its host_gateway_ca_dir bind-mount is redundant — leaning toward dropping it so docker rotates like firecracker (one behaviour across backends).

The firecracker specifics (hook install, per-agent CA push, git-gate re-init, token reprovision) are now written as that backend's implementation of the abstract method.

Re: the docker/macOS non-goal (review comment on `docs/prds/0081-...md`) — agreed, made it target **all** backends, enforced on the ABC. Updated in 4434752d: - Dropped the docker/macOS non-goal. - Design now leads with the contract: **`attach_bottled_agents_to_gateway()` as an `@abc.abstractmethod` on `BottleBackend`** (`backend/base.py`). Reconciling running bottles is a backend responsibility (only the backend can enumerate its agents and reach them — firecracker over SSH, docker/macOS over `exec`/`cp`), so every backend must implement it — cross-backend by construction, no per-backend follow-up. It reprovisions all registered bottles' CA + git-gate + egress tokens. - The host calls it on the gateway **bring-up** path (cold-boot branch only, never on adopt). - New open question: once docker's `attach_bottled_agents_to_gateway` pushes the CA to running agents, its `host_gateway_ca_dir` bind-mount is redundant — leaning toward dropping it so docker rotates like firecracker (one behaviour across backends). The firecracker specifics (hook install, per-agent CA push, git-gate re-init, token reprovision) are now written as *that backend's implementation* of the abstract method.
didericis commented 2026-07-26 19:10:26 -04:00
Owner
Copy Link
Copy Source

New open question: once docker's attach_bottled_agents_to_gateway pushes the CA to running agents, its host_gateway_ca_dir bind-mount is redundant — leaning toward dropping it so docker rotates like firecracker (one behaviour across backends).

Yes, make it consistent across backends.

> New open question: once docker's attach_bottled_agents_to_gateway pushes the CA to running agents, its host_gateway_ca_dir bind-mount is redundant — leaning toward dropping it so docker rotates like firecracker (one behaviour across backends). Yes, make it consistent across backends.
didericis-claude changed title from PRD 0081: reprovision gateway-dependent state on gateway bring-up to PRD: reprovision on gateway startup 2026-07-26 22:10:34 -04:00
didericis force-pushed prd-gateway-reprovision-on-bringup from 4434752db4 to f565749f59 2026-07-27 10:59:53 -04:00 Compare
didericis-claude closed this pull request 2026-07-27 11:04:49 -04:00
didericis-claude reopened this pull request 2026-07-27 11:04:52 -04:00
didericis force-pushed prd-gateway-reprovision-on-bringup from f565749f59 to cf6bbc43da 2026-07-27 11:16:12 -04:00 Compare
didericis added 1 commit 2026-07-27 21:50:36 -04:00
feat(prd-0081): reprovision CA, git-gate, and egress tokens on gateway bring-up
prd-number-check / require-numbered-prds (pull_request) Successful in 13s
Details
tracker-policy-pr / check-pr (pull_request) Successful in 7s
Details
test / image-input-builds (pull_request) Successful in 41s
Details
test / unit (pull_request) Successful in 51s
Details
test / integration-docker (pull_request) Successful in 58s
Details
test / coverage (pull_request) Successful in 41s
Details
test / image-input-builds (push) Successful in 48s
Details
test / unit (push) Successful in 56s
Details
lint / lint (push) Successful in 1m2s
Details
Update Quality Badges / update-badges (push) Successful in 1m4s
Details
test / integration-docker (push) Failing after 2m53s
Details
test / coverage (push) Has been skipped
Details
dee0121e8d 
On a Firecracker gateway cold boot, reconcile every live agent VM against
the fresh gateway: push the new mitmproxy CA into each agent's trust store,
re-provision git-gate repos/creds from the persisted upstreams snapshot,
and restore egress tokens. Per-bottle failures are logged and skipped so
one unreachable VM does not block the rest.

New modules / changes:
- backend/firecracker/reconcile.py: attach_bottled_agents_to_gateway,
  _push_ca, _reprovision_git_gate, _guest_ip_from_config
- git_gate/provision.py: write upstreams.json after key provisioning so
  the bring-up reconcile can reconstruct the upstream table without the
  manifest
- backend/firecracker/infra.py: call attach_bottled_agents_to_gateway in
  the cold-boot branch of ensure_running()
- backend/base.py: no-op default on BottleBackend
- backend/firecracker/consolidated_launch.py: remove superseded
  _reprovision_running_bottles / _guest_ip_from_config
- orchestrator: OrchestratorCore.update_agent_secret + POST
  /bottles/<id>/secret + client.update_agent_secret (single-secret
  in-place update, the reusable primitive from the 2026-07-26 hotfix)
- tests: 15 new tests in test_firecracker_reconcile.py; updated
  test_firecracker_infra.py cold-boot cases; stale FC reprovision tests
  removed from test_backend_secret_reprovision.py

Closes #516.
didericis force-pushed prd-gateway-reprovision-on-bringup from 6487f0087a to dee0121e8d 2026-07-27 21:50:36 -04:00 Compare
didericis-claude merged commit dee0121e8d into main 2026-07-27 21:53:58 -04:00
didericis-claude deleted branch prd-gateway-reprovision-on-bringup 2026-07-27 21:54:01 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
didericis
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
Status/Needs Triage
Awaiting initial classification
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
didericis-claude (didericis (claude)) didericis-codex (didericis (codex)) didericis-gemma (gemma) didericis
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: didericis/bot-bottle#517
Delete Branch "prd-gateway-reprovision-on-bringup"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?