Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 605146d287 | |||
| 27dea58ae1 | |||
| 5401f036a9 | |||
| 238f5f7614 |
@@ -212,6 +212,89 @@ jobs:
|
||||
name: firecracker-inputs
|
||||
path: /var/cache/bot-bottle-fc/dropbear
|
||||
|
||||
# Integration tests against the macOS Apple Container backend. Runs on a
|
||||
# self-hosted macOS runner (label `macos`) registered in HOST mode — Apple
|
||||
# Container needs the host `container` CLI + virtualization framework and
|
||||
# cannot run inside a Linux container, so this cannot reuse the KVM runner.
|
||||
#
|
||||
# Advisory only: workflow_dispatch (manual) exclusively — never push or
|
||||
# pull_request. A single non-redundant laptop that sleeps/roams must not run
|
||||
# unattended on every push to main, let alone block a PR merge, so this job is
|
||||
# deliberately NOT in the `coverage` job's `needs` and its coverage never
|
||||
# feeds the diff-coverage gate. Dispatch-only also means no fork PR (or any
|
||||
# push) ever executes on the host-mode runner.
|
||||
#
|
||||
# The infra container is a singleton (`bot-bottle-mac-infra`); the
|
||||
# `concurrency` group serializes runs so two never collide on it (#425), and
|
||||
# the always-run teardown removes it so a crashed run can't wedge the next.
|
||||
#
|
||||
# Runner prerequisites (provision once; see README "macOS Apple Container"):
|
||||
# the `container` CLI on PATH with `container system status` running, and a
|
||||
# Python >=3.11 with `coverage` importable on the launchd service PATH.
|
||||
integration-macos:
|
||||
runs-on: [self-hosted, macos]
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
concurrency:
|
||||
group: integration-macos-infra
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# Fail loudly if the backend this job promises isn't actually usable,
|
||||
# rather than letting every test silently `unittest.skip` and the job go
|
||||
# green on zero coverage. `backend status` exits non-zero (and prints the
|
||||
# per-check summary) when the `container` CLI or its system service is
|
||||
# missing — the same readiness check the skip guards gate on.
|
||||
- name: Preflight — Apple Container backend is ready
|
||||
run: |
|
||||
command -v container >/dev/null || {
|
||||
echo "container CLI not on PATH — provision the runner (README: macOS Apple Container)"; exit 1; }
|
||||
container system status || {
|
||||
echo "container system service not running — run 'container system start'"; exit 1; }
|
||||
python3 cli.py backend status --backend=macos-container
|
||||
|
||||
# `coverage` comes from the runner's provisioned Python (no pip install
|
||||
# into the host interpreter). Advisory job: report coverage in-line for
|
||||
# visibility but don't upload — it never feeds the combined gate.
|
||||
- name: Run integration tests (macos-container) with coverage
|
||||
env:
|
||||
BOT_BOTTLE_BACKEND: macos-container
|
||||
COVERAGE_FILE: ${{ github.workspace }}/.coverage.macos
|
||||
run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
|
||||
|
||||
- name: Report macos coverage
|
||||
env:
|
||||
COVERAGE_FILE: ${{ github.workspace }}/.coverage.macos
|
||||
run: python3 -m coverage report -m
|
||||
|
||||
# On failure, capture the infra containers' state and logs BEFORE the
|
||||
# teardown below removes them — otherwise a control-plane crash is
|
||||
# undiagnosable from CI, since `stop()` deletes the orchestrator (and its
|
||||
# logs) on every run. Best-effort: never let the diagnostics themselves
|
||||
# fail the job, and keep going if a container is already gone.
|
||||
- name: Dump infra diagnostics (on failure)
|
||||
if: failure()
|
||||
run: |
|
||||
set +e
|
||||
echo "=== containers ==="
|
||||
container ls -a | grep bot-bottle-mac || echo "(no bot-bottle-mac containers)"
|
||||
echo "=== networks ==="
|
||||
container network ls | grep bot-bottle-mac || echo "(no bot-bottle-mac networks)"
|
||||
for c in bot-bottle-mac-orchestrator bot-bottle-mac-infra; do
|
||||
echo "=== inspect $c ==="
|
||||
container inspect "$c" || echo "($c not found)"
|
||||
echo "=== logs $c ==="
|
||||
container logs "$c" || echo "($c logs unavailable)"
|
||||
done
|
||||
exit 0
|
||||
|
||||
# Remove the singleton infra container so a crashed or cancelled run
|
||||
# cannot leave `bot-bottle-mac-infra` wedged for the next job.
|
||||
- name: Teardown infra singleton
|
||||
if: always()
|
||||
run: python3 -c 'from bot_bottle.backend.macos_container.infra import MacosInfraService; MacosInfraService().stop()'
|
||||
|
||||
# Combined coverage gate: aggregates .coverage.* artifacts uploaded by each
|
||||
# test job, then runs the diff-coverage gate (new/changed lines >= 90%).
|
||||
#
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
|
||||
## Architecture
|
||||
|
||||
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
|
||||
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. git-gate runs over the gateway's consolidated `git-http` daemon (the legacy per-bottle `git://` daemon is not used on this backend); keys are provisioned dynamically at launch and revoked on teardown.
|
||||
|
||||
On the Firecracker backend, a bottle is an agent microVM plus a Docker gateway for egress, git-gate, and supervise. The VM reaches the gateway over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the gateway. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
|
||||
|
||||
@@ -75,6 +75,8 @@ On compatible macOS hosts, the default backend requires Apple's `container` CLI
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
|
||||
|
||||
> **CI (macOS Apple Container):** the `integration-macos` job (`.gitea/workflows/test.yml`) runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container` on a self-hosted macOS runner labelled `macos`, because Apple Container needs the host virtualization framework and cannot run in a Linux container (so it can't reuse the `kvm` runner). Provision an Apple Silicon host with the `container` CLI on `PATH` and `container system status` running, then register the runner in **host mode** (not docker mode) with the `macos` label — `brew install gitea-runner` (the `act_runner` rename). Give it a Python ≥ 3.11 with `coverage` importable on the launchd service's `PATH` (a launchd service doesn't inherit your shell profile, so pin `node` and the Python env explicitly). The job is **advisory** — `workflow_dispatch` (manual) only, never triggered by push or PR — since a single laptop that sleeps/roams must not block merges or churn on every push to main; its coverage doesn't feed the gate. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1.
|
||||
|
||||
### Containers inside a bottle
|
||||
|
||||
A bottle may set `nested_containers: true`. On the macOS backend this starts a
|
||||
|
||||
+12
-1
@@ -2,11 +2,22 @@
|
||||
|
||||
The test workflow lives at [`.gitea/workflows/test.yml`](../.gitea/workflows/test.yml).
|
||||
It runs the unit suite plus one integration job per backend
|
||||
(`integration-docker`, `integration-firecracker`) on:
|
||||
(`integration-docker`, `integration-firecracker`, `integration-macos`) on:
|
||||
|
||||
- every push to a branch with an open pull request, and
|
||||
- every push to `main`.
|
||||
|
||||
`integration-macos` is the exception: it is **advisory**, running only on
|
||||
`workflow_dispatch` (manual dispatch), never on push or pull requests. It targets the
|
||||
Apple Container backend on a self-hosted macOS runner (label `macos`,
|
||||
registered in host mode — Apple Container can't run in a Linux container, so it
|
||||
can't reuse the `kvm` runner). A single non-redundant laptop must not be able
|
||||
to block a PR merge, so the job stays out of the `coverage` job's `needs` and
|
||||
its coverage never feeds the diff-coverage gate. Because the infra container is
|
||||
a singleton (`bot-bottle-mac-infra`), the job declares a `concurrency` group
|
||||
and tears the container down on exit; keep runner concurrency at 1. See the
|
||||
README "macOS Apple Container" CI note for runner provisioning.
|
||||
|
||||
Each integration job selects its backend via `BOT_BOTTLE_BACKEND` and
|
||||
runs a **preflight** (`./cli.py backend status --backend=<name>`) that
|
||||
prints a clear per-check readiness summary and fails the job when the
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# PRD prd-new: macOS (Apple Container) CI runner
|
||||
|
||||
- **Status:** Draft
|
||||
- **Author:** Claude
|
||||
- **Created:** 2026-07-25
|
||||
- **Issue:** #426
|
||||
|
||||
## Summary
|
||||
|
||||
CI has no runner for the `macos-container` (Apple Container) backend.
|
||||
`.gitea/workflows/test.yml` exercises Docker (`ubuntu-latest`) and
|
||||
Firecracker (self-hosted `kvm`) but never the macOS backend. This PRD adds a
|
||||
self-hosted macOS runner (label `macos`) and an advisory `integration-macos`
|
||||
job that runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container`,
|
||||
so the backend that is the default on macOS stops shipping unexercised.
|
||||
|
||||
## Problem
|
||||
|
||||
The gap is not theoretical. `5ad3449` moved `bot_bottle` from flat files under
|
||||
`/app` into a pip-installed package but left init scripts spawning the
|
||||
supervisor as `python3 /app/gateway_init.py`, which no longer exists. Both the
|
||||
Firecracker and macOS backends carried the identical bug:
|
||||
|
||||
- **firecracker** — caught and fixed in `127ba49` because the KVM runner
|
||||
(added in `c193b04`, PR #349) runs that backend's integration suite.
|
||||
- **macos-container** — survived on `main` and only surfaced when a human ran
|
||||
`bot-bottle start` by hand.
|
||||
|
||||
The failure mode is expensive to debug: the supervisor never starts, so
|
||||
mitmdump never generates its CA, and launch dies downstream with
|
||||
`GatewayError: gateway CA not available`, which points at TLS rather than at
|
||||
the supervisor. Unit tests did not help — `test_macos_infra` asserted the
|
||||
substring `"gateway_init.py"`, which the *broken* path satisfies. (That
|
||||
specific assertion has since been tightened to the module form
|
||||
`bot_bottle.gateway_init`, matching its Firecracker twin, so the exact
|
||||
regression is now covered on `ubuntu-latest`. What remains missing is the
|
||||
end-to-end runner that would catch the *next* macOS-only launch regression.)
|
||||
|
||||
PR #470 (#414) already made the integration suite backend-agnostic:
|
||||
`skip_unless_selected_backend_available()` gates on the *selected* backend's
|
||||
own `is_backend_ready()` rather than `docker_available()`, and each
|
||||
integration job runs `./cli.py backend status --backend=<name>` as a preflight
|
||||
that fails loudly when the backend is missing. That is the machinery this job
|
||||
plugs into; this PRD supplies the runner and the job.
|
||||
|
||||
## Goals / Success criteria
|
||||
|
||||
- A macOS runner is registered and picks up jobs by the `macos` label.
|
||||
- An `integration-macos` job runs the integration suite against
|
||||
`BOT_BOTTLE_BACKEND=macos-container`.
|
||||
- The job **fails, not skips**, when the backend is unavailable on the runner
|
||||
(via the `backend status` preflight).
|
||||
- Reverting the `macos_container/infra.py` supervisor fix makes the job fail:
|
||||
the broken supervisor path throws `GatewayError` at bottle launch, which is
|
||||
`TestSandboxEscape.setUpClass`, failing the whole class before any individual
|
||||
attack runs.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Making `integration-macos` a **required** PR check. It runs on
|
||||
`workflow_dispatch` (manual dispatch) only — never on push or PRs. A single
|
||||
non-redundant laptop that sleeps and roams must never be able to block a PR
|
||||
merge or churn unattended on every push to main, and it is deliberately kept
|
||||
out of the `coverage` job's `needs` so the diff-coverage gate never depends on
|
||||
it.
|
||||
- Multi-machine or hosted macOS runners. Apple Container needs the host
|
||||
virtualization framework, so the runner must be a physical/VM macOS host on
|
||||
Apple Silicon — it cannot reuse the KVM runner or run in a Linux container.
|
||||
- Coverage aggregation from the macOS job into the combined gate (would couple
|
||||
the gate to the laptop).
|
||||
|
||||
## Design
|
||||
|
||||
### Runner (operational, provisioned once)
|
||||
|
||||
- Apple Silicon macOS host with Apple's `container` CLI installed and
|
||||
`container system status` reporting `running`.
|
||||
- Install the runner: `brew install gitea-runner` (the `act_runner` rename),
|
||||
registered in **host mode** with label `macos` — not docker mode, because
|
||||
Apple Container needs the host `container` CLI and virtualization framework,
|
||||
not a nested container.
|
||||
- A Python ≥ 3.11 with `coverage` importable on the runner's `PATH`. Because a
|
||||
launchd service does not inherit an interactive shell's `PATH`, pin `node`
|
||||
(for the JS `actions/*`) and the Python env explicitly in the service
|
||||
environment rather than relying on `nvm`/shell profile.
|
||||
- Concurrency 1. The infra container is a singleton (`bot-bottle-mac-infra`),
|
||||
so two simultaneous runs on one host collide (#425). The job also declares a
|
||||
`concurrency` group as belt-and-suspenders and tears the singleton down after
|
||||
each run.
|
||||
|
||||
### `integration-macos` job
|
||||
|
||||
Modeled on `integration-firecracker`:
|
||||
|
||||
- `runs-on: [self-hosted, macos]`.
|
||||
- `if:` `workflow_dispatch` only (advisory, manual dispatch; never push or PRs,
|
||||
so no fork-PR exposure, no merge-blocking, and no unattended runs on push).
|
||||
- `concurrency: { group: integration-macos-infra, cancel-in-progress: false }`
|
||||
to serialize runs against the singleton.
|
||||
- **Preflight** — `command -v container`, `container system status`, then
|
||||
`./cli.py backend status --backend=macos-container`; any failure exits
|
||||
non-zero so a misprovisioned runner fails loudly instead of silently
|
||||
skipping.
|
||||
- Run the integration suite under coverage with
|
||||
`BOT_BOTTLE_BACKEND=macos-container` and print a `coverage report -m` for
|
||||
visibility (no upload, not in the gate).
|
||||
- **Teardown** (`if: always()`) — `MacosInfraService().stop()` removes the
|
||||
singleton so a crashed run cannot wedge the next one.
|
||||
|
||||
### The `test_sandbox_escape` CI guard (the trap #470 left)
|
||||
|
||||
`TestSandboxEscape` is the only backend-agnostic integration test that boots a
|
||||
real bottle, so it is the one that would catch a macOS launch regression. It
|
||||
still carries a second guard that skips under `GITEA_ACTIONS` for every backend
|
||||
except `firecracker`:
|
||||
|
||||
```python
|
||||
@unittest.skipIf(
|
||||
os.environ.get("GITEA_ACTIONS") == "true"
|
||||
and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker",
|
||||
...,
|
||||
)
|
||||
```
|
||||
|
||||
The skip exists because the *containerized* `act_runner` (docker on
|
||||
`ubuntu-latest`) can't see a host bind mount and hides sibling-gateway network
|
||||
topology. Those constraints do not apply to a **host-mode** runner — neither
|
||||
the KVM host runner nor a macOS host runner is containerized. This PRD relaxes
|
||||
the guard to allow both host-mode backends (`firecracker`, `macos-container`)
|
||||
through while still skipping on the containerized Docker job. Without this
|
||||
change the macOS job would run green while skipping the exact test that proves
|
||||
the backend launches — the very false-green this issue is about.
|
||||
|
||||
## Open questions
|
||||
|
||||
- None known that block the job. git-gate is fully implemented on the macOS
|
||||
backend (the gateway's consolidated `git-http` daemon plus dynamic key
|
||||
provisioning/revocation), so `TestSandboxEscape` attack 5 — secret exfil
|
||||
pushed through git-gate, rejected by the gitleaks hook before the upstream
|
||||
push — runs the same as on the other backends. Any genuinely
|
||||
macOS-specific test adjustment would surface at first runner bring-up, but
|
||||
none is anticipated from the current backend implementation.
|
||||
@@ -1,463 +0,0 @@
|
||||
# PRD prd-new: Per-bottle signed commits & audit attribution
|
||||
|
||||
- **Status:** Draft
|
||||
- **Author:** didericis-claude
|
||||
- **Created:** 2026-07-25
|
||||
- **Issue:** #423
|
||||
|
||||
## Summary
|
||||
|
||||
Give each bottled agent a **per-activation signing key** so that every commit it
|
||||
produces is signed in the git-gate trust boundary (outside the bottle) and
|
||||
recorded in bot-bottle's own **host-owned audit store**, which is the portable
|
||||
source of truth. Each row cryptographically binds a commit's bytes (and their
|
||||
control-plane-recomputed SHA) to **access to that activation's signing key**, and
|
||||
binds the key to control-plane-owned activation metadata — bottle, host,
|
||||
manifest, agent, activation interval, retained public key — plus the commit's
|
||||
*claimed* author. The gate mints a short-lived Ed25519 key at spin-up, holds the
|
||||
private half in the sidecar `ssh-agent`, and forwards only `SSH_AUTH_SOCK` into
|
||||
the bottle. The gate rejects any commit it forwards that is not signed by the
|
||||
activation key; separately, the **control plane** independently recomputes each
|
||||
commit's object ID and verifies its signature before recording attribution — it
|
||||
never trusts a SHA, key, or verdict asserted by the gate.
|
||||
|
||||
This PRD deliberately does **not** enforce or vouch author/committer identity.
|
||||
Author/committer name/email are recorded as claims carried inside the signed
|
||||
object; making the gate reject a mismatching author/committer is a possible
|
||||
future add (see **Non-goals** and **Deferred: identity enforcement**). Push
|
||||
capability stays exactly as PRD 0048 deploy keys; forge subuser accounts,
|
||||
provisioned API tokens, and forge-side status/"Verified" badges remain out of
|
||||
scope (a future "forge actors" PRD).
|
||||
|
||||
Successor to:
|
||||
|
||||
- **PRD 0027 (agent git identity, #94)** / **ADR 0002** — established that
|
||||
`git-gate.user` name/email is *claimed, not vouched*. This PRD keeps that
|
||||
posture: it adds signed **provenance** and a durable host record, not identity
|
||||
enforcement.
|
||||
- **PRD 0048 (deploy-key provisioning, #169)** — the host-side mint-at-spin-up /
|
||||
revoke-at-teardown lifecycle the signing key follows. Deploy keys are
|
||||
unchanged.
|
||||
- **PRD 0070 (per-host orchestrator, #351)** — the orchestrator/control plane is
|
||||
the sole owner of `bot-bottle.db`; audit verification and recording live
|
||||
there, not in the data-plane gate (see **Trust boundary**).
|
||||
|
||||
## The guarantee
|
||||
|
||||
The crisp property this feature provides:
|
||||
|
||||
> The **host-owned audit store** binds a set of commit bytes — whose Git object
|
||||
> ID the control plane **recomputes** itself — to **access to this activation's
|
||||
> signing key**, and binds that key to **control-plane-owned activation
|
||||
> metadata**: bottle, host, manifest, agent, activation interval, retained public
|
||||
> key. An agent may author and sign arbitrary commit contents, but it cannot make
|
||||
> that signature verify as a *different* activation, and it cannot choose the
|
||||
> activation metadata the control plane records. The commit's author/committer
|
||||
> identity is **recorded as a claim**, not enforced or vouched. The forge remains
|
||||
> only the repository transport/capability layer.
|
||||
|
||||
What this does and does not prove (issue #423, comments #5554 / #5607 / #5608):
|
||||
|
||||
- It proves **access to activation *Y*'s signing key**: whoever assembled these
|
||||
commit bytes could sign with that key. Recomputing the object ID and verifying
|
||||
the embedded signature binds the SHA to activation *Y*, and the control plane's
|
||||
own records bind *Y*'s key to *Y*'s metadata.
|
||||
- It does **not** prove the commit was ever pushed, observed upstream, kept
|
||||
(vs. later reverted or dropped), or produced by the *agent* rather than by any
|
||||
other holder of the activation signing capability (the sidecar itself). The
|
||||
store deliberately makes no claim about publication or sole-agent authorship
|
||||
— the owner's requirement is attribution of *what manifest/agent/etc. was in
|
||||
use when a commit was signed*, not proof of where the commit went (#5607).
|
||||
- It does **not** make author/committer identity cryptographically vouched. The
|
||||
bottle chooses every byte sent through the forwarded agent, so a signature over
|
||||
`author Mallory <mallory@example>` is just as valid. Those fields are a claim
|
||||
carried inside the signed object and recorded as-is.
|
||||
- The binding is trustworthy because the **control plane** supplies the SHA (it
|
||||
recomputes it), the public key, and the activation metadata from its own state
|
||||
— never from a value the gateway asserts (see **Trust boundary**). The
|
||||
residual, by design: anything that holds the activation signing capability can
|
||||
produce commits that attribute to that activation. That is inherent to a
|
||||
binding on *activation-key access*, not a defect.
|
||||
|
||||
## Problem
|
||||
|
||||
An agent runs on the developer's machine as a *subrole*, scoped down per role.
|
||||
Locally that is fine because the machine is single-tenant. The git history an
|
||||
agent produces, however, is a durable artifact that outlives the session and
|
||||
can be pushed to shared repositories, and today bot-bottle offers no
|
||||
tamper-evidence over it:
|
||||
|
||||
- **No provenance.** Nothing ties a pushed commit to the bottle/activation that
|
||||
actually produced it. `git-gate.user` name/email is forgeable and cosmetic
|
||||
(ADR 0002); a commit could be produced anywhere.
|
||||
- **No durable, portable record.** There is no host-side ledger that says "SHA
|
||||
*X* was produced by agent *A* in bottle *B* on host *H* during interval
|
||||
*[t0,t1]*, signed by key *K*," independent of any forge and surviving key
|
||||
rotation.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- **Per-activation signing key.** A fresh Ed25519 keypair is minted host-side at
|
||||
each activation; the private half lives only in the sidecar `ssh-agent`, never
|
||||
in the bottle. Only `SSH_AUTH_SOCK` crosses the boundary.
|
||||
- **Signed commits with no SHA divergence.** Commits produced in the bottle are
|
||||
signed at commit time; the SHA the agent observes is the SHA that reaches the
|
||||
upstream through the gate.
|
||||
- **Gate rejects unsigned commits.** Before the gate forwards a push, every
|
||||
newly-introduced commit (those not already reachable from the advertised
|
||||
upstream refs) must verify against the activation public key; a push with any
|
||||
unsigned or wrong-key new commit is rejected, loudly, with the offending SHA.
|
||||
This is a **signature** check only — no author/committer matching.
|
||||
- **Control-plane-owned attribution.** The orchestrator/control plane (sole
|
||||
owner of `bot-bottle.db`, PRD 0070) recomputes each commit's object ID from the
|
||||
bytes, verifies the embedded signature against the activation public key it
|
||||
holds, and attaches activation metadata from its own state — accepting no SHA,
|
||||
key, verdict, or metadata asserted by the gateway. No upstream fetch is
|
||||
required.
|
||||
- **Host is the source of truth.** The audit record binds each recomputed SHA to
|
||||
the bottle, host, manifest, agent, activation interval, and retained public
|
||||
key, and records the commit's claimed author/committer.
|
||||
- **Verifiable after teardown.** The audit record retains the **full public
|
||||
key, fingerprint, principal, and validity interval** — enough to regenerate an
|
||||
allowed-signers file and run `git verify-commit` long after the activation
|
||||
ends and the key is gone.
|
||||
- **Reprovision-per-activation, fail-loud teardown.** The signing key is minted
|
||||
once per activation (persists across restarts within that activation) and
|
||||
discarded at teardown; deploy-key revocation continues to follow PRD 0048's
|
||||
fail-loud discipline.
|
||||
- **Push capability unchanged.** Forge access remains PRD 0048 deploy keys; no
|
||||
new forge API dependency beyond 0048's existing deploy-key registration.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Author/committer enforcement.** Explicitly out of scope for this PRD (issue
|
||||
#423, comment #5590). The gate does not reject a commit for carrying a foreign
|
||||
author or committer; those fields are recorded as claims. We rely on the
|
||||
cross-forge audit store of signed commits and the authors recorded there. See
|
||||
**Deferred: identity enforcement** for what a future add would look like.
|
||||
- **Cryptographically-vouched author identity.** Not claimed — see **The
|
||||
guarantee**.
|
||||
- **Forge subuser accounts / provisioned API tokens / PAT minting.** Dropped.
|
||||
Gitea's `POST /users/:name/tokens` requires Basic Auth *as the target user*
|
||||
(an admin PAT cannot mint one for another user; the only server-side path is
|
||||
the `gitea admin user generate-access-token` CLI), so a token-minting
|
||||
bootstrap is a design in its own right (issue #423, comments #5518 / #5554).
|
||||
This PRD needs no subrole API token, so that bootstrap problem does not arise.
|
||||
- **Forge-side attribution surfaces.** No commit-status badges, no forge
|
||||
"Verified" badge. The latter is doubly unsuitable: it renders dynamically
|
||||
against a *currently registered* key (so it would lie the moment a
|
||||
reprovisioned key is revoked), and on Gitea registering a signing key also
|
||||
grants push. Attribution lives in the host record and local `git
|
||||
verify-commit`, not the forge.
|
||||
- **Non-Gitea forges, dashboard UI for orphan cleanup, mid-session rotation,
|
||||
dirty-teardown reconciliation.** As before; a separate cleanup/sync pass
|
||||
handles orphans left by a crash or discarded snapshot.
|
||||
|
||||
## Scope narrowing
|
||||
|
||||
This PRD started as "forge subroles" (forge subuser accounts + provisioned API
|
||||
tokens + optional forge status posting + signing). Review (issue #423, comments
|
||||
#5518 → #5590) narrowed it in two steps:
|
||||
|
||||
1. **Dropped the forge-account and API-token machinery** (#5518 → #5556):
|
||||
the PAT bootstrap is not implementable as sketched (Basic-Auth-as-target-user
|
||||
constraint); the signature never vouched the author anyway; and making the
|
||||
host audit store the portable source of truth is a cleaner boundary that
|
||||
removes the forge-specific token lifecycle and commit-status dependence.
|
||||
2. **Dropped author/committer enforcement** (#5590): rely on the audit store of
|
||||
signed commits and the authors recorded there; gate enforcement of the
|
||||
identity fields is a possible future add, not part of this slice.
|
||||
|
||||
What remains is the core that stands on its own: **signed commits + a
|
||||
host-owned, independently-verified audit record.** Forge *actors* (a per-bottle
|
||||
account that comments/opens PRs) and *identity enforcement* (the gate rejecting a
|
||||
foreign author/committer) are each candidate future PRDs.
|
||||
|
||||
## Design
|
||||
|
||||
### Trust boundary (control plane vs data plane)
|
||||
|
||||
git-gate is the **data plane**: it parses hostile bytes from inside the bottle
|
||||
and forwards pushes. The orchestrator is the **control plane** and, per PRD
|
||||
0070, is the sole owner of `bot-bottle.db`. These are different trust boundaries,
|
||||
and the audit record must be anchored in the control plane:
|
||||
|
||||
- The gate performs a **synchronous pre-forward signature check** (below) and
|
||||
can reject a push before it reaches the upstream. This is a data-plane gate on
|
||||
what leaves the bottle, not the audit binding.
|
||||
- The **control plane** takes the commit bytes to attribute (gateway-delivered
|
||||
opaque bytes are fine), **recomputes the Git object ID**, **verifies the
|
||||
embedded signature** against the activation public key it minted and holds, and
|
||||
writes `attributed_commit` attaching metadata from its own state. It accepts
|
||||
**no** gateway-supplied `verified` flag, claimed SHA, public key, or activation
|
||||
identity.
|
||||
|
||||
The precise trust statement (issue #423, review by didericis-codex on d8362ec,
|
||||
resolved in #5608): the row binds *these commit bytes / this recomputed SHA* to
|
||||
*access to this activation's signing key*, and the control plane binds that key
|
||||
to the recorded activation metadata. It does **not** assert forge observation or
|
||||
that only the agent (not the signing sidecar) authored the commit — so this PRD
|
||||
does **not** claim a compromised gateway cannot obtain an attribution row.
|
||||
Because the sidecar holds the activation signing capability, a compromised
|
||||
gateway *can* assemble and sign a commit and have it attributed to that
|
||||
activation; what it cannot do is make the signature verify as a *different*
|
||||
activation or choose the metadata the control plane records. That residual is
|
||||
acceptable under the intended guarantee (#5607) and is why the guarantee is
|
||||
worded as activation-key access, not agent-only authorship or upstream
|
||||
publication. The gate therefore cannot stand in for host-side verification: the
|
||||
control plane recomputes the object ID and verifies the signature itself rather
|
||||
than trusting the gate's word.
|
||||
|
||||
### Identity model
|
||||
|
||||
Per **bottled agent** (agent definition ∘ sealed bottle), realized per
|
||||
activation:
|
||||
|
||||
| Part | Value | Source | Role |
|
||||
|------|-------|--------|------|
|
||||
| Signing key | one Ed25519 keypair | minted host-side per activation | signs every commit; private half sidecar-only; the anchor of provenance |
|
||||
| Author/committer | name + email | `git-gate.user` (PRD 0027 overlay) | written into commits and **recorded** as a claim; **not** enforced |
|
||||
|
||||
### Manifest surface
|
||||
|
||||
No new top-level keys and no `git-forge`/`forge-accounts` blocks. A single
|
||||
opt-in flag under the existing `git-gate` key turns on per-activation signing;
|
||||
`git-gate.user` (PRD 0027) supplies the author string as today.
|
||||
|
||||
```yaml
|
||||
git-gate:
|
||||
user: # PRD 0027 — author string; recorded, not enforced
|
||||
name: didericis-claude
|
||||
email: eric+claude@dideric.is
|
||||
signing:
|
||||
enabled: true # NEW — opt-in per-activation signing + audit
|
||||
repos:
|
||||
bot-bottle:
|
||||
url: ssh://git@100.78.141.42:30009/didericis/bot-bottle.git
|
||||
provisioned_key: # PRD 0048 — push capability, UNCHANGED
|
||||
provider: gitea
|
||||
token_env: GITEA_DEPLOY_TOKEN
|
||||
host_key: "ssh-ed25519 AAAA..."
|
||||
```
|
||||
|
||||
- `git-gate.signing.enabled: true` opts a bottle in. Without it, behavior is
|
||||
exactly as today. There is **no `enforce` sub-key** — this PRD does not enforce
|
||||
identity fields, so no knob is needed (and a knob that weakened a guarantee
|
||||
was flagged as a contradiction in review).
|
||||
- `git-gate.signing` is **bottle-only** (home-only policy), rejected at the
|
||||
agent level with a clear pointer. `git-gate.user` keeps its PRD 0027
|
||||
agent-overlay semantics.
|
||||
|
||||
### Signing: sign at commit time via a forwarded ssh-agent
|
||||
|
||||
The reason SHAs never diverge:
|
||||
|
||||
- The **sidecar** (the git-gate trust boundary) runs an `ssh-agent` holding the
|
||||
short-lived signing private key.
|
||||
- **Only `SSH_AUTH_SOCK`** is forwarded into the bottle — a bounded signing
|
||||
capability, not the key.
|
||||
- The provisioner writes the bottle `.gitconfig`:
|
||||
|
||||
```ini
|
||||
[commit]
|
||||
gpgsign = true
|
||||
[gpg]
|
||||
format = ssh
|
||||
[user]
|
||||
name = didericis-claude
|
||||
email = eric+claude@dideric.is
|
||||
signingkey = ssh-ed25519 AAAA... # activation signing PUBLIC key
|
||||
```
|
||||
|
||||
- `git commit` asks the forwarded agent to sign; the signature is embedded at
|
||||
object creation, so the agent-space SHA equals the pushed SHA. No transcoder,
|
||||
no SHA translation table.
|
||||
|
||||
### Gate pre-forward signature check (data plane)
|
||||
|
||||
The gate already fetches from upstream before every `upload-pack` and mirrors
|
||||
bidirectionally (PRD 0008). When `git-gate.signing.enabled` is set, after
|
||||
gitleaks and before forwarding a push upstream:
|
||||
|
||||
1. **Compute the newly-introduced set.** Commits reachable from the pushed ref
|
||||
tips but **not** reachable from any ref already advertised by the upstream
|
||||
(which the gate knows because it fetches upstream first) — equivalent to
|
||||
`git rev-list <new-tips> --not <all-known-upstream-refs>`. This excludes
|
||||
pulled/merged existing history; a merge commit the bottle creates is itself
|
||||
new and is checked, its already-upstream ancestors are not.
|
||||
2. **Verify each new commit's signature** against the activation public key. A
|
||||
commit that is unsigned or signed by any other key causes the push to be
|
||||
**rejected** with the offending SHA.
|
||||
3. No author/committer matching is performed.
|
||||
|
||||
This is a synchronous safety gate on what leaves the bottle; it is not the audit
|
||||
record.
|
||||
|
||||
### Control-plane verification & recording
|
||||
|
||||
For each commit to attribute (the gate hands the control plane the commit bytes;
|
||||
opaque gateway-delivered bytes are acceptable because nothing the gateway *says*
|
||||
about them is trusted), the orchestrator/control plane:
|
||||
|
||||
1. **Recomputes the Git object ID** from the bytes itself. The stored `sha` is
|
||||
this recomputed value, never a SHA the gateway claims.
|
||||
2. **Verifies the embedded signature** against the activation public key it
|
||||
minted and holds for that activation (via a generated allowed-signers file) —
|
||||
ignoring any `verified` flag, key, or activation identity supplied by the
|
||||
gateway.
|
||||
3. Writes `attributed_commit` only for bytes that pass, stamping the activation
|
||||
metadata (bottle/manifest/agent/host/interval) from its **own** state — not
|
||||
from anything the gateway provides — and recording the commit's claimed
|
||||
author/committer.
|
||||
|
||||
No upstream fetch is required: the guarantee is a byte↔activation-key binding, so
|
||||
the object does not need to come from the forge (issue #423, #5608). Bytes that
|
||||
do not verify against the activation key are **not** recorded as attributed (they
|
||||
may be logged as an anomaly instead).
|
||||
|
||||
### Audit trail
|
||||
|
||||
The host SQLite store (PRD 0067, `~/.bot-bottle/bot-bottle.db`, owned by the
|
||||
control plane per PRD 0070) records the signing-key lifecycle and per-commit
|
||||
attribution. Retention is the **full public key, fingerprint, principal, and
|
||||
validity interval** — enough to regenerate an allowed-signers file and verify
|
||||
commits after teardown (issue #423, comment #5554, resolution 3). Never any
|
||||
private key material.
|
||||
|
||||
```sql
|
||||
CREATE TABLE bottled_agent_activation (
|
||||
bottled_agent_slug TEXT NOT NULL,
|
||||
activation_id TEXT NOT NULL, -- one per activation cycle
|
||||
host TEXT NOT NULL,
|
||||
manifest_digest TEXT NOT NULL, -- ties the record to the sealed manifest
|
||||
agent TEXT NOT NULL,
|
||||
signing_pubkey TEXT NOT NULL, -- full ssh-ed25519 public key (for verify-commit)
|
||||
signing_fpr TEXT NOT NULL, -- SHA256:... fingerprint (stable handle)
|
||||
principal TEXT NOT NULL, -- allowed-signers principal, e.g. the author email
|
||||
valid_from TEXT NOT NULL,
|
||||
valid_until TEXT, -- NULL while active; set at teardown
|
||||
status TEXT NOT NULL, -- active | retired
|
||||
PRIMARY KEY (bottled_agent_slug, activation_id)
|
||||
);
|
||||
|
||||
CREATE TABLE attributed_commit (
|
||||
sha TEXT NOT NULL, -- control-plane-RECOMPUTED object ID, not gateway-claimed
|
||||
bottled_agent_slug TEXT NOT NULL,
|
||||
activation_id TEXT NOT NULL,
|
||||
repo TEXT NOT NULL,
|
||||
author_name TEXT NOT NULL, -- CLAIMED, recorded as-is (not enforced)
|
||||
author_email TEXT NOT NULL, -- CLAIMED
|
||||
committer_name TEXT NOT NULL, -- CLAIMED
|
||||
committer_email TEXT NOT NULL, -- CLAIMED
|
||||
observed_at TEXT NOT NULL,
|
||||
PRIMARY KEY (sha, repo)
|
||||
);
|
||||
```
|
||||
|
||||
Verification/allowed-signers generation is a stated part of the design: for a
|
||||
given SHA, join `attributed_commit → bottled_agent_activation`, emit
|
||||
`<principal> <signing_pubkey>` to a temporary allowed-signers file, and
|
||||
`git verify-commit` (or `ssh-keygen -Y verify`) against it. The
|
||||
`(pubkey, principal, valid_from/until)` tuple is exactly what that requires. The
|
||||
recorded author/committer columns are the *claim*; a consumer that wants to know
|
||||
"who says they wrote this" reads them, understanding they are unenforced.
|
||||
|
||||
### Credential lifecycle
|
||||
|
||||
Follows PRD 0048, minus the API-token kind (dropped):
|
||||
|
||||
- **Activation:** mint a fresh Ed25519 signing keypair; load the private half
|
||||
into the sidecar `ssh-agent`; write the public half into `.gitconfig` and the
|
||||
`bottled_agent_activation` row (`active`, `valid_from` set). Deploy keys are
|
||||
provisioned exactly as PRD 0048. Minting is **per activation** (a restart
|
||||
re-attaches the same key; a new activation mints a new key and retires the old
|
||||
row), so frozen snapshots don't accumulate live keys.
|
||||
- **Teardown (fail-loud):** revoke provisioned deploy keys via the forge API
|
||||
(0048); discard the signing key from the sidecar agent and set the activation
|
||||
row to `retired` with `valid_until`. The signing key was never on the forge,
|
||||
so there is nothing to revoke there — only the local retire. Deploy-key
|
||||
revocation failure halts teardown (0048); 404 = already-gone = success.
|
||||
- **Dirty teardown** is assumed handled; a separate cleanup/sync pass reconciles
|
||||
orphaned deploy keys.
|
||||
|
||||
## Deferred: identity enforcement
|
||||
|
||||
If a future PRD wants the gate to *enforce* that new commits carry the manifest
|
||||
identity, the natural shape is: extend the gate pre-forward check to also require
|
||||
each new commit's author **and** committer name/email to equal `git-gate.user`,
|
||||
rejecting mismatches — with the same control-plane re-verification before
|
||||
recording. This is deliberately left out now (issue #423, comment #5590); it is
|
||||
noted so the door stays open and the current schema (which records the claimed
|
||||
author/committer) already carries what such a check would compare against. Note
|
||||
that even then the property would be gate-*enforced*, not signature-*vouched*; a
|
||||
validating signing broker in front of the key would be required for the latter.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **This PRD.** Sets the (narrowed) design.
|
||||
2. **Manifest surface.** Add `git-gate.signing` (bottle-only; `enabled` only);
|
||||
reject it at the agent level. Unit tests for parse/validation and the
|
||||
agent-level rejection.
|
||||
3. **Signing pipeline.** Sidecar `ssh-agent` provisioning; forward
|
||||
`SSH_AUTH_SOCK` into the bottle across docker, smolmachines, macOS-container,
|
||||
and firecracker backends; emit the `commit.gpgsign` / `gpg.format=ssh` /
|
||||
`user.signingkey` gitconfig. Integration test: a bottle commit is
|
||||
`verify-commit`-valid and its SHA is unchanged through the gate; the private
|
||||
key is absent from the bottle.
|
||||
4. **Gate pre-forward signature check.** Compute the newly-introduced set
|
||||
(excluding upstream-reachable commits), verify each against the activation
|
||||
key, reject unsigned/wrong-key with the offending SHA. Tests: unsigned
|
||||
rejected; wrong-key rejected; pulled/merged upstream history passes; an
|
||||
all-signed push succeeds. A foreign-author commit that is correctly signed
|
||||
**passes the gate** (identity is not enforced here).
|
||||
5. **Control-plane verification + audit.** `bottled_agent_activation` /
|
||||
`attributed_commit` tables (PRD 0067 store, control-plane-owned per PRD 0070);
|
||||
the control plane recomputes each commit's object ID and verifies the
|
||||
signature before writing a row; retain full pubkey + fingerprint + principal +
|
||||
validity interval; record claimed author/committer; allowed-signers generation
|
||||
+ a post-teardown `verify-commit` helper. Tests: a gateway-claimed SHA/key/
|
||||
verdict is ignored — the row's `sha` is the recomputed ID and bytes not signed
|
||||
by the activation key produce **no** row.
|
||||
6. **Docs.** Glossary entry ("per-bottle signed commits"); README manifest
|
||||
section; ADR note that signing-enabled bottles gain signed *provenance* and a
|
||||
host-owned audit record while authorship stays *claimed* (ADR 0002 unchanged).
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Unit (must):** `git-gate.signing` parse/validation; agent-level `signing`
|
||||
rejection.
|
||||
- **Integration — signing (must):** end-to-end signed commit verifies with
|
||||
`git verify-commit`; SHA observed in the bottle equals the SHA upstream; the
|
||||
private key is absent from the bottle.
|
||||
- **Integration — gate check (must):** unsigned rejected; wrong-key rejected;
|
||||
the upstream-reachable exclusion (pull + merge human history and push a signed
|
||||
merge); a correctly-signed foreign-author commit **passes** (no identity
|
||||
enforcement); a clean all-signed push succeeds.
|
||||
- **Control plane (must):** the control plane recomputes the object ID and
|
||||
records a row for bytes genuinely signed by the activation key; a gateway-
|
||||
supplied SHA/key/verdict is ignored (the stored `sha` is the recomputed value);
|
||||
bytes signed by a foreign/invalid key produce **no** row.
|
||||
- **Lifecycle:** activation mints the key and writes an `active` row; teardown
|
||||
retires it (`valid_until`) and revokes deploy keys fail-loud; a restart
|
||||
re-attaches the same key (no new row); a fresh activation mints a new key and
|
||||
retires the old.
|
||||
- **Post-teardown verification:** regenerate the allowed-signers file from a
|
||||
`retired` row and confirm `verify-commit` still succeeds for an attributed SHA.
|
||||
|
||||
## Resolved: control-plane transport
|
||||
|
||||
Raised in review and **resolved** (issue #423, #5608): the commit object does not
|
||||
need to come from the forge, and reading the gateway-owned mirror is no stronger
|
||||
than accepting gateway-delivered bytes — both are fabricatable, and neither
|
||||
matters because the control plane trusts nothing the gateway *asserts*. The
|
||||
transport is therefore: the gate hands the control plane the raw commit bytes,
|
||||
the control plane **recomputes the object ID** and **verifies the signature**
|
||||
against the activation key, and stamps its own activation metadata. No upstream
|
||||
fetch. This is exactly what makes the byte↔activation-key binding sound
|
||||
regardless of transport.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Where the gate check slots into PRD 0008 ordering.** Modeled as a
|
||||
pre-forward step after gitleaks; confirm it composes with the existing
|
||||
access-hook / mirror ordering rather than needing a separate hook.
|
||||
@@ -67,14 +67,25 @@ _DUMMY_HOST_KEY = (
|
||||
)
|
||||
|
||||
|
||||
# Backends whose CI runner is HOST-mode (self-hosted), so the test process
|
||||
# and the backend share a host. The containerized act_runner (docker on
|
||||
# ubuntu-latest) is the one that can't see the host bind mount egress_tls_init
|
||||
# uses and hides sibling-gateway network topology; host-mode runners
|
||||
# (firecracker/KVM, macos-container) don't have those constraints, so the test
|
||||
# runs there. Keep this in sync with the `runs-on` labels in
|
||||
# .gitea/workflows/test.yml.
|
||||
_HOST_MODE_CI_BACKENDS = frozenset({"firecracker", "macos-container"})
|
||||
|
||||
|
||||
@skip_unless_selected_backend_available()
|
||||
@unittest.skipIf(
|
||||
os.environ.get("GITEA_ACTIONS") == "true"
|
||||
and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker",
|
||||
"skipped under act_runner unless BOT_BOTTLE_BACKEND=firecracker: "
|
||||
and os.environ.get("BOT_BOTTLE_BACKEND") not in _HOST_MODE_CI_BACKENDS,
|
||||
"skipped under the containerized act_runner (docker on ubuntu-latest): "
|
||||
"egress_tls_init uses a host bind mount the runner container can't "
|
||||
"see, and the network topology hides sibling-gateway visibility — "
|
||||
"these constraints don't apply on the self-hosted KVM runner",
|
||||
"these constraints don't apply on the self-hosted host-mode runners "
|
||||
"(firecracker/KVM, macos-container)",
|
||||
)
|
||||
class TestSandboxEscape(unittest.TestCase):
|
||||
"""End-to-end attacks against a real bottle. The bottle stays
|
||||
|
||||
Reference in New Issue
Block a user