Compare commits

...

1 Commits

Author SHA1 Message Date
didericis-codex 016e59e029 docs(prd): tighten audit ordering and chain guarantees
prd-number-check / require-numbered-prds (pull_request) Failing after 12s
tracker-policy-pr / check-pr (pull_request) Successful in 15s
2026-07-26 17:44:41 +00:00
+54 -28
View File
@@ -37,7 +37,8 @@ emits into:
and signed-commit (#480), auth/authz, and audit self-events — each with
its trusted-vs-claimed fields and redaction rules.
5. A **stable export projection** (CloudEvents / OpenTelemetry Logs) and
the **#324 delivery contract** (payload, `(epoch, seq)` cursor, dedup,
the **#324 delivery contract** (payload, per-chain
`(host, epoch, seq)` cursor, dedup,
backpressure, retention ordering).
It is explicitly scheduled to land **immediately after the host
@@ -142,7 +143,7 @@ stable top level:
"epoch": 7, // writer-boot counter, bumped once per host-controller (writer) start
"seq": 1287, // monotonic sequence within this epoch (gap-detectable)
"segment": "20260726T000000Z", // journal segment id (rotation boundary); chain continues across segments
"prev": "<hex>", // hash of the previous record in the chain ("" for a segment genesis)
"prev": "<hex>", // prior record hash ("" only for the first record of a new chain)
"hash": "<hex>", // sha256(prev + canonical(this event with hash=""))
// ---- timestamps (host-owned) ----
@@ -270,7 +271,9 @@ The `hash` field is computed over the canonical form of the event **with
digest = sha256_hex(prev_hash + canonical({**event, "hash": ""}))
```
`prev` is the prior record's `hash`; a segment genesis uses `prev = ""`.
`prev` is the prior record's `hash`; only the first record of a brand-new
chain uses `prev = ""`. The first record of a rotated segment carries the
preceding segment's head, as specified under *Rotation* below.
Two exact rules pin the bytes so the chain is reproducible anywhere:
1. **Serialize the record with its own `hash` field set to `""`** (present,
@@ -296,7 +299,7 @@ serialization and linkage. An implementation is conformant iff it
reproduces these bytes and hashes.
```
# Record 0 — segment genesis (prev = "")
# Record 0 — brand-new chain genesis (prev = "")
canonical(record0, hash=""):
{"hash":"","id":"11111111-1111-4111-8111-111111111111","prev":"","seq":0,"type":"audit.segment_open"}
hash0 = sha256("" + canonical) =
@@ -319,17 +322,28 @@ bump that changes the bytes fails a golden test loudly.
### Ordering, idempotency, and duplicate handling
- **Ordering.** `(epoch, seq)` is a strict total order per host and, because
- **Ordering.** `(epoch, seq)` is a strict total order within one host's
native chain and, because
the writer is single, a strict order per bottle/activation within that
host — satisfying "at least strict causal order per activation/bottle".
`causation_id` records the explicit cause edges (a DAG) on top of the
total order, so a consumer can reconstruct request → decision → cutoff
even if unrelated events interleave between them.
even if unrelated events interleave between them. There is deliberately
no invented total order across imported host chains; a cross-host key is
`(host, epoch, seq)`, and consumers use correlation/causation edges where
causal ordering across hosts is known.
- **Idempotency.** `id` is the idempotency key. A producer that retries an
emit (e.g. after a writer restart mid-handoff) **reuses the same `id`**;
the writer drops a second append bearing an `id` already present in the
current segment's in-memory set, and the index `UPSERT`s by `id`, so a
duplicate never double-counts or forks the chain.
before appending, the writer checks a durable, host-wide id ledger that
spans every segment and survives restart/retention, and drops an id
already present. The ledger is operational metadata, not an audit source
of truth: after a crash it is reconciled from the journal before appends
resume, and retention preserves id tombstones after journal segments are
pruned. An in-memory set may cache the ledger but is never the authority.
The index additionally has a unique key on `id`; its `UPSERT` is
defensive and does not substitute for the pre-append check. Thus a
duplicate never enters the source-of-truth journal, double-counts, or
forks the chain.
- **Deduplication downstream.** Because `id` is stable across export and
replay, #324's cursor replay and any cross-host merge dedup on `id` — no
consumer needs to invent a second identity.
@@ -338,18 +352,22 @@ bump that changes the bytes fails a golden test loudly.
- **Rotation.** At a segment boundary the writer opens a new segment file,
sets its `segment` id, and carries the rotated-out segment's head as the
new genesis `prev` — so the chain is continuous *across* segments while
new segment's first `prev` — so the chain is continuous *across* segments
(`prev = ""` is reserved for the first record of a brand-new chain) while
each file stays independently openable. `verify` walks segments in order
and checks the head-to-genesis link at each seam.
and checks the preceding-head-to-first-record link at each seam.
- **Restart.** Covered above: read last line → adopt its `hash` as `prev`,
bump `epoch`, reset `seq`. The chain never restarts even though the
counters do.
- **Import.** `audit import <segment>` appends an externally supplied
segment (e.g. recovered from another host or a backup). Import verifies
the incoming chain in isolation first, then links it only if its genesis
`prev` matches a known head or is explicitly grafted; imported records
keep their original `id` (dedup) and are marked with their origin host so
attribution is not laundered.
the incoming chain in isolation first. A continuation whose first `prev`
matches a known head extends that chain. A foreign chain is registered as
a separate immutable chain namespace rather than rewriting or grafting
its records (which would invalidate their hashes). Imported records keep
their original `host`, `id`, `epoch`, and `seq`; the index keys their
native order by `(host, epoch, seq)` so attribution is not laundered and
tuples from different hosts cannot collide.
- **Truncation.** A crash can leave a partial final line; `verify` reports
it as `truncated-tail` (recoverable — replay resumes from the last intact
record). A chain that ends before a persisted head, or a missing interior
@@ -376,11 +394,13 @@ start the writer:
2. bumps `epoch` (persisted alongside the chain head) and resets `seq` to
0 for the new boot.
Total order is therefore `(epoch, seq)` — monotonic across restarts by
construction — with `ts_event`/`ts_recorded` for human reading and `ts_mono` for
sub-second ordering inside an epoch. A crash mid-append truncates at most
the last (partial) line; the verifier flags it and replay resumes from the
last intact record.
Total order within this host's native chain is therefore `(epoch, seq)` —
monotonic across restarts by construction — with
`ts_event`/`ts_recorded` for human reading and `ts_mono` for sub-second
ordering inside an epoch. Imported chains retain their own
`(host, epoch, seq)` order and do not acquire a fictional order relative to
the local chain. A crash mid-append truncates at most the last (partial)
line; the verifier flags it and replay resumes from the last intact record.
### Journal (source of truth) + SQLite index (rebuildable)
@@ -401,8 +421,11 @@ last intact record.
`outcome`, `sensitivity`, `correlation_id`, `causation_id`, plus two
event-specific projections promoted out of `payload` for query —
`repository` and `commit_sha` (populated for `forge.*`/`commit.*`, null
otherwise). The full canonical record is stored verbatim in a `raw` column
so the index never loses fidelity to the journal.
otherwise). The unique event id and native-order index are respectively
`id` and `(host, epoch, seq)`; a local-only `ingest_seq` provides stable
display order when a query intentionally mixes chains without pretending
that it is causal order. The full canonical record is stored verbatim in a
`raw` column so the index never loses fidelity to the journal.
**Local query surface** — `audit query`, no egress, no paid platform:
@@ -411,7 +434,7 @@ audit query \
[--since T] [--until T] [--type egress.*] [--host H] [--bottle B] \
[--activation A] [--agent SLUG] [--actor ID] [--outcome blocked] \
[--repository R] [--correlation-id C] [--commit SHA] \
[--follow BOTTLE] # a bottled agent's events in (epoch, seq) order
[--follow BOTTLE] # one host chain's events in (epoch, seq) order
[--json | --table]
audit verify [--segment S] # offline chain check; exit non-zero on any break
audit rebuild # drop + replay journal → index
@@ -578,9 +601,12 @@ PRD fixes the contract it depends on:
exact bytes the hash covers), optionally wrapped in the CloudEvents
projection whose `data` *is* that record. Either way the integrity fields
travel intact and the receiver can verify against the same bytes.
- **Cursor.** The export cursor is `(epoch, seq)` (equivalently the last
exported `hash`). It advances **only on acknowledgement**, so delivery is
at-least-once and gap-free; a crash re-sends from the last acked cursor.
- **Cursor.** Export maintains one cursor per native host chain:
`(host, epoch, seq)` (equivalently that chain's last exported `hash`).
It advances **only on acknowledgement**, so delivery is at-least-once and
gap-free; a crash re-sends from the last acked cursor. Imported foreign
chains use independent cursors and never share a bare `(epoch, seq)`
namespace with the local chain.
- **Idempotency / replay.** Dedup is on `id` (stable across replay), so
at-least-once delivery is safe — the receiver collapses re-sends.
- **Backpressure.** The outbox is the journal itself plus a cursor; when
@@ -683,8 +709,8 @@ Nothing in the contract is left undefined — only its code is deferred.
the `lifecycle.bottled_agent_*` leaf names. (Design → *The envelope* /
*Event registry*.)
- **Retention head-carry — yes.** When a journal segment is rotated out,
the new segment's genesis `prev` is the rotated-out head, so the verifier
still trusts the current head across a rotation. (Folds into the
the new segment's first record carries the rotated-out head as `prev`, so
the verifier still trusts the current head across a rotation. (Folds into the
retention follow-up.)
- **Redaction reuses the egress detectors — yes, scoped.** Reuse the
deterministic `dlp_detectors` (`scan_token_patterns` / `redact_tokens` /