bot-bottle/docs/prds/prd-new-sqlite-local-storage.md

PRD prd-new: SQLite local storage

• Status: Active
• Author: codex
• Created: 2026-07-01
• Issue: #319

Summary

Add a small stdlib SQLite storage layer for bot-bottle host runtime state, starting with the supervise queue and audit log. This replaces scattered JSON queue files and JSONL audit logs with structured tables while preserving the existing public supervise helper functions and sidecar queue mount contract.

Problem

Bot-bottle currently stores supervise proposals and responses as individual JSON files under ~/.bot-bottle/queue/<slug>/, and audit entries as JSONL files under ~/.bot-bottle/audit/. That worked for the original interactive TUI, but new forge-native orchestration needs durable, queryable local state for queues, audit trails, watchdogs, and lifecycle records. PR #318 started introducing SQLite-shaped boilerplate for forge state; the storage foundation should live in its own PR so forge work can build on the shared runtime store instead of adding one-off persistence.

Goals / Success Criteria

1. Supervise proposals and responses are persisted through SQLite.
2. Audit entries are persisted through SQLite.
3. Supervise queue helpers use the bottle slug / queue key instead of a queue directory path.
4. The sidecar receives the host database mount across docker, smolmachines, and macOS-container backends.
5. The implementation stays stdlib-only.
6. Schema migrations use a PRAGMA user_version runner — no third-party deps.
7. Unit tests cover queue round-trips, pending discovery, response waits, archive semantics, audit round-trips, and path creation.

Non-goals

• Migrating old JSON queue files or JSONL audit logs.
• Adding forge orchestration state tables.
• Adding egress metering or budget tables.
• Changing the supervise TUI workflow or remediation behavior.
• Introducing a third-party ORM or migration library.

Design

Database locations

Queue and audit state use the host-level local database:

~/.bot-bottle/bot-bottle.db

The supervise sidecar receives that database as a writable bind mount at /run/supervise/bot-bottle.db and gets the path through SUPERVISE_DB_PATH. No per-slug queue directory is mounted into the sidecar. This creates the shared host database that later forge/native lifecycle work can extend in separate PRDs.

Tables

supervise_proposals lives in the host database:

CREATE TABLE supervise_proposals (
    queue_key TEXT NOT NULL,
    id TEXT NOT NULL,
    bottle_slug TEXT NOT NULL,
    tool TEXT NOT NULL,
    proposed_file TEXT NOT NULL,
    justification TEXT NOT NULL,
    arrival_timestamp TEXT NOT NULL,
    current_file_hash TEXT NOT NULL,
    archived INTEGER NOT NULL DEFAULT 0,
    PRIMARY KEY (queue_key, id)
);

supervise_responses lives in the host database:

CREATE TABLE supervise_responses (
    queue_key TEXT NOT NULL,
    proposal_id TEXT NOT NULL,
    status TEXT NOT NULL,
    notes TEXT NOT NULL,
    final_file TEXT,
    archived INTEGER NOT NULL DEFAULT 0,
    PRIMARY KEY (queue_key, proposal_id)
);

supervise_audit_entries lives in the host database:

CREATE TABLE supervise_audit_entries (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    timestamp TEXT NOT NULL,
    bottle_slug TEXT NOT NULL,
    component TEXT NOT NULL,
    operator_action TEXT NOT NULL,
    operator_notes TEXT NOT NULL,
    justification TEXT NOT NULL,
    diff TEXT NOT NULL
);

Compatibility

The queue helpers take a bottle slug / queue key and perform equivalent operations against ~/.bot-bottle/bot-bottle.db:

• list_pending_proposals returns non-archived proposals without a non-archived response, sorted by arrival time.
• archive_proposal marks matching proposal/response rows archived instead of moving files into processed/.
• wait_for_response keeps the current polling behavior but polls SQLite.

The old audit path helpers (audit_dir, audit_log_path) stay available for compatibility. audit_log_path no longer describes the active storage location; callers should use read_audit_entries.

Implementation chunks

1. Add SQLite store helpers for supervise queue and audit state.
2. Rewire bot_bottle.supervise queue/audit functions to the store.
3. Update supervise CLI discovery tests and queue/audit unit tests.
4. Run unit tests, pyright, and pylint for touched modules.

Open questions

None.