fix(supervise): make the response poll idempotent and slow it to 250ms
tracker-policy-pr / check-pr (pull_request) Successful in 15s
test / integration-docker (pull_request) Successful in 42s
test / unit (pull_request) Successful in 46s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m22s
test / coverage (pull_request) Successful in 23s
test / publish-infra (pull_request) Has been skipped

The control-plane supervise poll archived the proposal on the same call
that returned the decision, so a dropped connection lost the operator's
decision (the retry saw `unknown`). Poll no longer archives — a re-poll
returns the same decision. Decided proposals still drop off the operator's
pending list (a response row exists); their rows are reaped when the
bottle is torn down or reconciled.

Also raise the grace-window poll interval from 50ms to 250ms now that each
poll is an HTTP round trip rather than a local file read.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-24 06:26:06 +00:00
parent 1db2a9eb67
commit d8b61b3658
8 changed files with 90 additions and 26 deletions
+14 -7
View File
@@ -40,7 +40,7 @@ from ..supervise import (
STATUS_REJECTED,
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
archive_proposal,
archive_all_proposals,
list_all_pending_proposals,
read_proposal,
read_response,
@@ -136,6 +136,8 @@ class Orchestrator:
self._broker.submit(sign_request(req, self._secret))
self.registry.deregister(bottle_id)
self._tokens.pop(bottle_id, None)
# Reap any supervise proposals the gone bottle never acknowledged.
archive_all_proposals(bottle_id)
return True
def reconcile(
@@ -160,6 +162,8 @@ class Orchestrator:
live_source_ips, grace_seconds=grace_seconds)
for rec in reaped:
self._tokens.pop(rec.bottle_id, None)
# Reap any supervise proposals the gone bottle never acknowledged.
archive_all_proposals(rec.bottle_id)
return [rec.bottle_id for rec in reaped]
def tokens_for(self, bottle_id: str) -> dict[str, str]:
@@ -223,15 +227,19 @@ class Orchestrator:
return proposal.id
def supervise_poll_response(self, bottle_id: str, proposal_id: str) -> dict[str, object]:
"""Non-blocking poll of one of `bottle_id`'s queued proposals.
"""Non-blocking, **idempotent** poll of one of `bottle_id`'s proposals.
Returns `{"status": ...}`: a terminal decision (`approved`/`modified`/
`rejected`, with `notes` + `final_file`) once the operator has responded,
`pending` while it's still queued undecided, or `unknown` when no such
queued proposal exists for this bottle (wrong id, or already resolved and
read). A decided proposal is archived here — the same terminal step the
data plane used to run after reading the response — so `pending`
proposals stay visible to the operator until decided *and* polled.
queued proposal exists for this bottle (wrong id, or already archived).
Poll does **not** archive — a decided proposal keeps returning the same
decision so a client whose connection drops after the decision but before
it consumes the response still gets it on retry (issue #469 review). A
decided proposal is already excluded from the operator's pending list (a
response row exists), so it doesn't linger there; the row itself is reaped
when the bottle is torn down / reconciled.
Reads are scoped to `bottle_id` (the queue key), so a caller can never
read another bottle's proposal even with a guessed id."""
@@ -243,7 +251,6 @@ class Orchestrator:
except FileNotFoundError:
return {"status": POLL_STATUS_UNKNOWN}
return {"status": POLL_STATUS_PENDING}
archive_proposal(bottle_id, proposal_id)
return {
"status": response.status,
"notes": response.notes,