feat(orchestrator): registry co-tenants the shared bot-bottle.db (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m5s

Per review: use one shared bot-bottle.db for all runtime state including
the registry (DbStore namespaces by schema_key), so it's one queryable
file for backup/console. default_db_path() -> host_db_path(). Drop the
unilateral WAL flip — WAL on the shared DB affects supervise/audit and is
finicky over guest shares, so it's a deliberate future change; keep a
busy_timeout for lock contention.

PRD State section updated: integrity now by SOLE ownership (only the
orchestrator opens bot-bottle.db; data plane + console reach state via the
control-plane RPC, never a file handle) rather than ro/rw mount-splitting,
which one shared file can't do. Notes the transitional caveat that the
supervise sidecar currently rw-mounts bot-bottle.db.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
2026-07-13 13:26:03 -04:00
parent ed7307e0e3
commit b174442c60
2 changed files with 46 additions and 33 deletions
+30 -24
View File
@@ -261,7 +261,7 @@ piling methods on — the same discipline, recursed.
The orchestrator is the natural owner of per-host **runtime state**:
- pool **slot leases** (which bottle holds slot *i*) — replaces today's
`fcntl`-locked files with WAL-mode transactions;
`fcntl`-locked files with SQLite transactions;
- the **supervise approval queue** + remembered approvals;
- the **live bottle registry** (source IP → bottle → policy/secrets refs),
the lookup table the attribution invariant reads.
@@ -273,38 +273,44 @@ Config splits into three tiers with different homes:
|---|---|---|
| Build-time constants | pool size, IP base, nft table | flat `.env` (PR #350) — must be readable by Nix eval + root bash, zero runtime |
| User-authored config | bottle manifests, egress routes, secret refs | declarative files under `~/.bot-bottle/` — trust boundary at `$HOME`, git-trackable, "unknown keys die at load" |
| Runtime state | slot leases, approvals, registry | **SQLite**, owned by the orchestrator |
| Runtime state | slot leases, approvals, registry | one shared **`bot-bottle.db`**, solely owned by the orchestrator |
SQLite is right for the runtime tier (mutable, concurrent, queried) and
wrong for the other two (Nix can't read it at eval time; it fights the
declarative manifest trust model). Keep the tiers separate.
**The DB file is host-resident, not owned inside the orchestrator unit**
(decided in review). Two reasons:
**One shared `bot-bottle.db` for all runtime state** (decided in review).
The registry co-tenants the existing host `bot-bottle.db` — the `DbStore`
framework already namespaces each store by `schema_key`, so slot
leases / approvals / registry share one file. One place to query, back up,
and integrate a console against.
- **Durability across restarts.** Re-adoption sweeps the registry after an
orchestrator restart, so the state *must* outlive the orchestrator
instance's lifecycle. A host-resident file (the slice-1 default under
`~/.cache/bot-bottle/orchestrator/`) is the simplest durable store; the
orchestrator VM/container mounts it rather than carrying it.
- **Integrity via access-scoping, not location.** Agents can't touch the
DB directly regardless of where it lives — they're network-isolated in
their bottles and only ever speak to the control/data plane. The real
risk is a *compromised agent-facing data-plane service* (egress/git-gate,
which parse hostile bytes) writing the registry and forging attribution.
The control is a **read/write split**: writes (register/deregister) come
only from the **control plane**; the **data plane gets the DB read-only**
for attribution lookups. A host-resident file makes that split
enforceable (mount `ro` into the data plane, `rw` into the control
plane) — which owning it "entirely inside" a monolithic orchestrator
would not.
- **Host-resident, for durability.** Re-adoption sweeps the registry after
an orchestrator restart, so state *must* outlive the orchestrator
instance. The file lives on the host (`bot_bottle_root()/db/bot-bottle.db`);
the orchestrator unit reaches it, it doesn't carry it.
- **Integrity by sole ownership, not mount permissions.** Agents can't
touch the DB directly wherever it lives (network-isolated in their
bottles). The risk is a *compromised agent-facing data-plane service*
(egress/git-gate, which parse hostile bytes) writing the registry and
forging attribution. Because it's now one shared file, coarse `ro`/`rw`
mount-splitting no longer isolates the registry — so the rule is stronger
and simpler: **only the orchestrator (control plane) opens `bot-bottle.db`;
the data plane and the console reach state through the control-plane RPC,
never a direct file handle.** No agent-facing component gets the file, so
none can forge attribution. (This supersedes the earlier `ro`-mount idea.)
- *Transitional caveat:* today the per-bottle **supervise sidecar
rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the
pattern the orchestrator removes (supervise consolidates into the
orchestrator; sidecar writes become RPC calls). Until that lands, don't
put the attribution registry behind a data-plane-writable mount.
Implementation note for the VM slices: SQLite **WAL** over a guest share
(virtiofs/9p) is finicky (the `-shm`/`-wal` files need real mmap/locking),
so the durable-DB-on-host may want a small **host-side owner** the
orchestrator reaches over the control-plane RPC, rather than a shared
mount. Decide this at the Firecracker slice; `sqlite3` itself is stdlib, so
"the host needs SQLite" is a non-cost.
which is a second reason the DB wants a **host-side owner** the orchestrator
reaches over the RPC rather than a shared mount into the VM. WAL on the
shared DB is therefore a deliberate, tested future change — not enabled ad
hoc. `sqlite3` itself is stdlib, so "the host needs SQLite" is a non-cost.
## Sequencing