feat(orchestrator): registry co-tenants the shared bot-bottle.db (#352)
Per review: use one shared bot-bottle.db for all runtime state including the registry (DbStore namespaces by schema_key), so it's one queryable file for backup/console. default_db_path() -> host_db_path(). Drop the unilateral WAL flip — WAL on the shared DB affects supervise/audit and is finicky over guest shares, so it's a deliberate future change; keep a busy_timeout for lock contention. PRD State section updated: integrity now by SOLE ownership (only the orchestrator opens bot-bottle.db; data plane + console reach state via the control-plane RPC, never a file handle) rather than ro/rw mount-splitting, which one shared file can't do. Notes the transitional caveat that the supervise sidecar currently rw-mounts bot-bottle.db. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
@@ -261,7 +261,7 @@ piling methods on — the same discipline, recursed.
|
||||
The orchestrator is the natural owner of per-host **runtime state**:
|
||||
|
||||
- pool **slot leases** (which bottle holds slot *i*) — replaces today's
|
||||
`fcntl`-locked files with WAL-mode transactions;
|
||||
`fcntl`-locked files with SQLite transactions;
|
||||
- the **supervise approval queue** + remembered approvals;
|
||||
- the **live bottle registry** (source IP → bottle → policy/secrets refs),
|
||||
the lookup table the attribution invariant reads.
|
||||
@@ -273,38 +273,44 @@ Config splits into three tiers with different homes:
|
||||
|---|---|---|
|
||||
| Build-time constants | pool size, IP base, nft table | flat `.env` (PR #350) — must be readable by Nix eval + root bash, zero runtime |
|
||||
| User-authored config | bottle manifests, egress routes, secret refs | declarative files under `~/.bot-bottle/` — trust boundary at `$HOME`, git-trackable, "unknown keys die at load" |
|
||||
| Runtime state | slot leases, approvals, registry | **SQLite**, owned by the orchestrator |
|
||||
| Runtime state | slot leases, approvals, registry | one shared **`bot-bottle.db`**, solely owned by the orchestrator |
|
||||
|
||||
SQLite is right for the runtime tier (mutable, concurrent, queried) and
|
||||
wrong for the other two (Nix can't read it at eval time; it fights the
|
||||
declarative manifest trust model). Keep the tiers separate.
|
||||
|
||||
**The DB file is host-resident, not owned inside the orchestrator unit**
|
||||
(decided in review). Two reasons:
|
||||
**One shared `bot-bottle.db` for all runtime state** (decided in review).
|
||||
The registry co-tenants the existing host `bot-bottle.db` — the `DbStore`
|
||||
framework already namespaces each store by `schema_key`, so slot
|
||||
leases / approvals / registry share one file. One place to query, back up,
|
||||
and integrate a console against.
|
||||
|
||||
- **Durability across restarts.** Re-adoption sweeps the registry after an
|
||||
orchestrator restart, so the state *must* outlive the orchestrator
|
||||
instance's lifecycle. A host-resident file (the slice-1 default under
|
||||
`~/.cache/bot-bottle/orchestrator/`) is the simplest durable store; the
|
||||
orchestrator VM/container mounts it rather than carrying it.
|
||||
- **Integrity via access-scoping, not location.** Agents can't touch the
|
||||
DB directly regardless of where it lives — they're network-isolated in
|
||||
their bottles and only ever speak to the control/data plane. The real
|
||||
risk is a *compromised agent-facing data-plane service* (egress/git-gate,
|
||||
which parse hostile bytes) writing the registry and forging attribution.
|
||||
The control is a **read/write split**: writes (register/deregister) come
|
||||
only from the **control plane**; the **data plane gets the DB read-only**
|
||||
for attribution lookups. A host-resident file makes that split
|
||||
enforceable (mount `ro` into the data plane, `rw` into the control
|
||||
plane) — which owning it "entirely inside" a monolithic orchestrator
|
||||
would not.
|
||||
- **Host-resident, for durability.** Re-adoption sweeps the registry after
|
||||
an orchestrator restart, so state *must* outlive the orchestrator
|
||||
instance. The file lives on the host (`bot_bottle_root()/db/bot-bottle.db`);
|
||||
the orchestrator unit reaches it, it doesn't carry it.
|
||||
- **Integrity by sole ownership, not mount permissions.** Agents can't
|
||||
touch the DB directly wherever it lives (network-isolated in their
|
||||
bottles). The risk is a *compromised agent-facing data-plane service*
|
||||
(egress/git-gate, which parse hostile bytes) writing the registry and
|
||||
forging attribution. Because it's now one shared file, coarse `ro`/`rw`
|
||||
mount-splitting no longer isolates the registry — so the rule is stronger
|
||||
and simpler: **only the orchestrator (control plane) opens `bot-bottle.db`;
|
||||
the data plane and the console reach state through the control-plane RPC,
|
||||
never a direct file handle.** No agent-facing component gets the file, so
|
||||
none can forge attribution. (This supersedes the earlier `ro`-mount idea.)
|
||||
- *Transitional caveat:* today the per-bottle **supervise sidecar
|
||||
rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the
|
||||
pattern the orchestrator removes (supervise consolidates into the
|
||||
orchestrator; sidecar writes become RPC calls). Until that lands, don't
|
||||
put the attribution registry behind a data-plane-writable mount.
|
||||
|
||||
Implementation note for the VM slices: SQLite **WAL** over a guest share
|
||||
(virtiofs/9p) is finicky (the `-shm`/`-wal` files need real mmap/locking),
|
||||
so the durable-DB-on-host may want a small **host-side owner** the
|
||||
orchestrator reaches over the control-plane RPC, rather than a shared
|
||||
mount. Decide this at the Firecracker slice; `sqlite3` itself is stdlib, so
|
||||
"the host needs SQLite" is a non-cost.
|
||||
which is a second reason the DB wants a **host-side owner** the orchestrator
|
||||
reaches over the RPC rather than a shared mount into the VM. WAL on the
|
||||
shared DB is therefore a deliberate, tested future change — not enabled ad
|
||||
hoc. `sqlite3` itself is stdlib, so "the host needs SQLite" is a non-cost.
|
||||
|
||||
## Sequencing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user