bot-bottle/docs/prds/prd-new-sqlite-local-storage.md

PRD prd-new: SQLite local storage

• Status: Active
• Author: codex
• Created: 2026-07-01
• Issue: #319

Summary

Add a small stdlib SQLite storage layer for bot-bottle host runtime state, starting with the supervise queue and audit log. This replaces scattered JSON queue files and JSONL audit logs with structured tables while preserving the existing public supervise helper functions and sidecar queue mount contract.

Problem

Bot-bottle currently stores supervise proposals and responses as individual JSON files under ~/.bot-bottle/queue/<slug>/, and audit entries as JSONL files under ~/.bot-bottle/audit/. That worked for the original interactive TUI, but new forge-native orchestration needs durable, queryable local state for queues, audit trails, watchdogs, and lifecycle records. PR #318 started introducing SQLite-shaped boilerplate for forge state; the storage foundation should live in its own PR so forge work can build on the shared runtime store instead of adding one-off persistence.

Goals / Success Criteria

1. Supervise proposals and responses are persisted through SQLite.
2. Audit entries are persisted through SQLite.
3. Existing public supervise helpers keep their current call shape where practical: write_proposal, read_proposal, list_pending_proposals, write_response, read_response, wait_for_response, archive_proposal, write_audit_entry, and read_audit_entries.
4. The sidecar queue mount still works across docker, smolmachines, and macOS-container backends.
5. The implementation stays stdlib-only.
6. Unit tests cover queue round-trips, pending discovery, response waits, archive semantics, audit round-trips, and path creation.

Non-goals

• Migrating old JSON queue files or JSONL audit logs.
• Adding forge orchestration state tables.
• Adding egress metering or budget tables.
• Changing the supervise TUI workflow or remediation behavior.
• Introducing a third-party ORM or migration framework.

Design

Database locations

Queue state remains tied to the mounted per-bottle queue directory:

~/.bot-bottle/queue/<slug>/supervise.db

The supervise sidecar already receives that directory at /run/supervise/queue, so both the sidecar and host TUI can read and write the same SQLite file without changing backend mounts.

Audit state uses the host-level local database:

~/.bot-bottle/bot-bottle.db

This creates the shared host database that later forge/native lifecycle work can extend in separate PRDs.

Tables

supervise_proposals lives in the per-queue database:

CREATE TABLE supervise_proposals (
    id TEXT PRIMARY KEY,
    bottle_slug TEXT NOT NULL,
    tool TEXT NOT NULL,
    proposed_file TEXT NOT NULL,
    justification TEXT NOT NULL,
    arrival_timestamp TEXT NOT NULL,
    current_file_hash TEXT NOT NULL,
    archived INTEGER NOT NULL DEFAULT 0
);

supervise_responses lives in the same per-queue database:

CREATE TABLE supervise_responses (
    proposal_id TEXT PRIMARY KEY,
    status TEXT NOT NULL,
    notes TEXT NOT NULL,
    final_file TEXT,
    archived INTEGER NOT NULL DEFAULT 0
);

supervise_audit_entries lives in the host database:

CREATE TABLE supervise_audit_entries (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    timestamp TEXT NOT NULL,
    bottle_slug TEXT NOT NULL,
    component TEXT NOT NULL,
    operator_action TEXT NOT NULL,
    operator_notes TEXT NOT NULL,
    justification TEXT NOT NULL,
    diff TEXT NOT NULL
);

Compatibility

The existing helper functions keep accepting Path arguments for queue directories. Internally, they map the queue directory to supervise.db and perform equivalent operations:

• list_pending_proposals returns non-archived proposals without a non-archived response, sorted by arrival time.
• archive_proposal marks matching proposal/response rows archived instead of moving files into processed/.
• wait_for_response keeps the current polling behavior but polls SQLite.

The old path helpers (queue_dir_for_slug, audit_dir, audit_log_path) stay available for compatibility. audit_log_path no longer describes the active storage location; callers should use read_audit_entries.

Implementation chunks

1. Add SQLite store helpers for supervise queue and audit state.
2. Rewire bot_bottle.supervise queue/audit functions to the store.
3. Update supervise CLI discovery tests and queue/audit unit tests.
4. Run unit tests, pyright, and pylint for touched modules.

Open questions

None.