refactor(supervise): split the supervise plane by tier; per-service store managers
test / integration-docker (pull_request) Successful in 13s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / unit (pull_request) Successful in 42s
lint / lint (push) Failing after 54s
test / integration-firecracker (pull_request) Successful in 3m17s
test / coverage (pull_request) Successful in 17s
test / publish-infra (pull_request) Has been skipped

Give each service its own store package + manager, and cut the supervise module
along the control/data-plane boundary so nothing in the shared layer reaches up
into the orchestrator.

Stores, by owner:
  - bot_bottle/store/ keeps only the shared base (DbStore, migrations) and the
    concrete stores that aren't service-owned (audit_store, config_store).
  - bot_bottle/orchestrator/store/ now houses the orchestrator-owned stores —
    queue_store (supervise queue), secret_store, config_store — plus a new
    orchestrator store_manager that migrates them (composing audit/config
    downward from the base). The old shared store_manager is gone.

Supervise plane, by tier:
  - bot_bottle/supervisor/ (NEUTRAL, importable by every tier including the
    gateway): types.py (the Proposal/Response/AuditEntry wire types + the tool/
    status/poll constants + the shared daemon constants moved out of
    supervise.py) and plan.py (SupervisePlan, a pure DTO).
  - bot_bottle/orchestrator/supervisor/ (orchestrator-only): queue.py (the
    queue/audit I/O wrappers + render_diff + sha256_hex) and supervise.py (the
    Supervise lifecycle that stages the DB via the store manager). Its __init__
    re-exports the neutral vocabulary so orchestrator-side callers import from
    one place.

The gateway now imports only bot_bottle.supervisor.types (never
bot_bottle.supervise), so the data plane holds no code dependency on the
orchestrator — it reaches the queue over the control-plane RPC. This removes
the circular import that moving queue_store under orchestrator introduced
(supervise -> orchestrator -> service -> supervise).

supervise_types.py -> supervisor/types.py; supervise.py deleted (split). Full
unit suite green (2251).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-24 14:15:04 -04:00
parent f77023db1d
commit 44e2b5a897
52 changed files with 380 additions and 385 deletions
+16
View File
@@ -0,0 +1,16 @@
"""Neutral supervise vocabulary — importable by every tier.
The supervise plane spans the control plane (orchestrator) and the data plane
(gateway `supervise_server`): the gateway proposes/polls over RPC, the
orchestrator queues and applies operator decisions. Both sides must agree on
the wire types and constants, so those live here — a neutral package neither
tier's private code, importable by `bot_bottle.gateway` **and**
`bot_bottle.orchestrator` without either depending on the other.
* `types` — the `Proposal`/`Response`/`AuditEntry` dataclasses, the tool /
status / poll-status constants, and the shared daemon constants.
* `plan` — `SupervisePlan`, the launch-time staging DTO.
The orchestrator-only half (queue I/O, diff rendering, the `Supervise`
lifecycle) lives under `bot_bottle.orchestrator.supervisor`.
"""
+26
View File
@@ -0,0 +1,26 @@
"""`SupervisePlan` — the launch-time supervise staging DTO.
A pure value type (no store access), so it lives in the neutral package: the
backend builds it at launch and the orchestrator's `Supervise.prepare` returns
it. The behaviour that fills it in — staging the host database — is the
orchestrator's, in `bot_bottle.orchestrator.supervisor.supervise`.
"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
@dataclass(frozen=True)
class SupervisePlan:
"""Output of `Supervise.prepare`; consumed by `.start`.
`db_path` is the host database bind-mounted into the gateway at
/run/supervise/bot-bottle.db. `internal_network` is empty at prepare time;
the backend's launch step fills it via `dataclasses.replace` before calling
`.start`."""
slug: str
db_path: Path
internal_network: str = ""
+203
View File
@@ -0,0 +1,203 @@
"""Shared types for supervise stores (PRD 0013).
Extracted from supervise.py so queue_store and audit_store can import
Proposal, Response, and AuditEntry without creating a circular import
(supervise imports from queue_store/audit_store and vice-versa).
The path helpers (`bot_bottle_root`, `host_db_path`, `HOST_DB_FILENAME`)
live in `paths` — import them from there.
"""
from __future__ import annotations
import dataclasses
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
# Read-only agent tool: poll a queued proposal for the operator's decision
# without blocking or re-proposing. It never becomes a `Proposal.tool` (no
# queue record is created for it), so it is intentionally NOT in `TOOLS`.
TOOL_CHECK_PROPOSAL = "check-proposal"
TOOLS: tuple[str, ...] = (
TOOL_EGRESS_ALLOW,
TOOL_EGRESS_BLOCK,
TOOL_GITLEAKS_ALLOW,
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
)
STATUS_APPROVED = "approved"
STATUS_MODIFIED = "modified"
STATUS_REJECTED = "rejected"
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
# Non-terminal markers the control-plane supervise-poll RPC returns when a
# proposal has no operator decision yet (`PENDING`) or no queued proposal with
# that id exists for the calling bottle (`UNKNOWN`). They are never a
# `Response.status` — only the poll wire contract (see
# `Orchestrator.supervise_poll_response`) — so they are deliberately outside
# `STATUSES`.
POLL_STATUS_PENDING = "pending"
POLL_STATUS_UNKNOWN = "unknown"
ACTION_OPERATOR_EDIT = "operator-edit"
# Shared supervise-daemon constants. The gateway `supervise_server` and the
# backend read these, so they live with the wire types in the neutral package
# (never in the orchestrator's private code the data plane can't import).
SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100
# The supervise daemon queries egress's introspection endpoint for the
# `list-egress-routes` tool over loopback (egress runs alongside it in the
# gateway), stable across docker / firecracker / Apple Container backends.
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
DB_PATH_IN_CONTAINER = "/run/supervise/bot-bottle.db"
DEFAULT_POLL_INTERVAL_SEC = 0.5
COMPONENT_FOR_TOOL: dict[str, str] = {
TOOL_EGRESS_ALLOW: "egress",
TOOL_EGRESS_BLOCK: "egress",
}
def _require_str(raw: dict[str, object], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str):
raise ValueError(f"missing or non-string field {key!r}")
return value
@dataclass(frozen=True)
class Proposal:
"""One pending tool-call from the agent."""
id: str
bottle_slug: str
tool: str
proposed_file: str
justification: str
arrival_timestamp: str
current_file_hash: str
@classmethod
def new(
cls,
*,
bottle_slug: str,
tool: str,
proposed_file: str,
justification: str,
current_file_hash: str,
now: datetime | None = None,
) -> "Proposal":
ts = (now or datetime.now(timezone.utc)).isoformat()
return cls(
id=str(uuid.uuid4()),
bottle_slug=bottle_slug,
tool=tool,
proposed_file=proposed_file,
justification=justification,
arrival_timestamp=ts,
current_file_hash=current_file_hash,
)
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
@classmethod
def from_dict(cls, raw: dict[str, object]) -> "Proposal":
tool = _require_str(raw, "tool")
if tool not in TOOLS:
raise ValueError(f"tool must be one of {TOOLS}; got {tool!r}")
return cls(
id=_require_str(raw, "id"),
bottle_slug=_require_str(raw, "bottle_slug"),
tool=tool,
proposed_file=_require_str(raw, "proposed_file"),
justification=_require_str(raw, "justification"),
arrival_timestamp=_require_str(raw, "arrival_timestamp"),
current_file_hash=_require_str(raw, "current_file_hash"),
)
@dataclass(frozen=True)
class Response:
"""The operator's decision on a proposal."""
proposal_id: str
status: str
notes: str
final_file: str | None = None
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
@classmethod
def from_dict(cls, raw: dict[str, object]) -> "Response":
status = _require_str(raw, "status")
if status not in STATUSES:
raise ValueError(
f"response status must be one of {STATUSES}; got {status!r}"
)
final = raw.get("final_file")
if final is not None and not isinstance(final, str):
raise ValueError(
f"final_file must be a string or null; got {type(final).__name__}"
)
return cls(
proposal_id=_require_str(raw, "proposal_id"),
status=status,
notes=_require_str(raw, "notes"),
final_file=final,
)
@dataclass(frozen=True)
class AuditEntry:
"""One row of the per-bottle audit log."""
timestamp: str
bottle_slug: str
component: str
operator_action: str
operator_notes: str
justification: str
diff: str
def to_dict(self) -> dict[str, object]:
return dataclasses.asdict(self)
__all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"Proposal",
"Response",
"STATUSES",
"STATUS_APPROVED",
"STATUS_MODIFIED",
"STATUS_REJECTED",
"POLL_STATUS_PENDING",
"POLL_STATUS_UNKNOWN",
"TOOLS",
"TOOL_EGRESS_ALLOW",
"TOOL_EGRESS_BLOCK",
"TOOL_CHECK_PROPOSAL",
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_GITLEAKS_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"SUPERVISE_HOSTNAME",
"SUPERVISE_PORT",
"EGRESS_FORWARD_PROXY",
"EGRESS_INTROSPECT_URL",
"DB_PATH_IN_CONTAINER",
"DEFAULT_POLL_INTERVAL_SEC",
"COMPONENT_FOR_TOOL",
]