ci(integration-macos): dump infra logs on failure before teardown

A control-plane failure (e.g. the orchestrator becoming unreachable at
bottle registration) is undiagnosable from CI as-is: the `if: always()`
teardown runs `MacosInfraService().stop()`, which deletes the
orchestrator and gateway containers — and their logs — on every run.

Add an `if: failure()` step that dumps `container ls`/`network ls`, plus
`inspect` and `logs` for `bot-bottle-mac-orchestrator` and
`bot-bottle-mac-infra`, ordered before the teardown so the evidence is
captured while the containers still exist. Best-effort (never fails the
job; tolerates an already-removed container).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.gitea/workflows/test.yml

@@ -212,6 +212,89 @@ jobs:
           name: firecracker-inputs
           path: /var/cache/bot-bottle-fc/dropbear
 
+  # Integration tests against the macOS Apple Container backend. Runs on a
+  # self-hosted macOS runner (label `macos`) registered in HOST mode — Apple
+  # Container needs the host `container` CLI + virtualization framework and
+  # cannot run inside a Linux container, so this cannot reuse the KVM runner.
+  #
+  # Advisory only: workflow_dispatch (manual) exclusively — never push or
+  # pull_request. A single non-redundant laptop that sleeps/roams must not run
+  # unattended on every push to main, let alone block a PR merge, so this job is
+  # deliberately NOT in the `coverage` job's `needs` and its coverage never
+  # feeds the diff-coverage gate. Dispatch-only also means no fork PR (or any
+  # push) ever executes on the host-mode runner.
+  #
+  # The infra container is a singleton (`bot-bottle-mac-infra`); the
+  # `concurrency` group serializes runs so two never collide on it (#425), and
+  # the always-run teardown removes it so a crashed run can't wedge the next.
+  #
+  # Runner prerequisites (provision once; see README "macOS Apple Container"):
+  # the `container` CLI on PATH with `container system status` running, and a
+  # Python >=3.11 with `coverage` importable on the launchd service PATH.
+  integration-macos:
+    runs-on: [self-hosted, macos]
+    if: github.event_name == 'workflow_dispatch'
+    concurrency:
+      group: integration-macos-infra
+      cancel-in-progress: false
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      # Fail loudly if the backend this job promises isn't actually usable,
+      # rather than letting every test silently `unittest.skip` and the job go
+      # green on zero coverage. `backend status` exits non-zero (and prints the
+      # per-check summary) when the `container` CLI or its system service is
+      # missing — the same readiness check the skip guards gate on.
+      - name: Preflight — Apple Container backend is ready
+        run: |
+          command -v container >/dev/null || {
+            echo "container CLI not on PATH — provision the runner (README: macOS Apple Container)"; exit 1; }
+          container system status || {
+            echo "container system service not running — run 'container system start'"; exit 1; }
+          python3 cli.py backend status --backend=macos-container
+
+      # `coverage` comes from the runner's provisioned Python (no pip install
+      # into the host interpreter). Advisory job: report coverage in-line for
+      # visibility but don't upload — it never feeds the combined gate.
+      - name: Run integration tests (macos-container) with coverage
+        env:
+          BOT_BOTTLE_BACKEND: macos-container
+          COVERAGE_FILE: ${{ github.workspace }}/.coverage.macos
+        run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
+
+      - name: Report macos coverage
+        env:
+          COVERAGE_FILE: ${{ github.workspace }}/.coverage.macos
+        run: python3 -m coverage report -m
+
+      # On failure, capture the infra containers' state and logs BEFORE the
+      # teardown below removes them — otherwise a control-plane crash is
+      # undiagnosable from CI, since `stop()` deletes the orchestrator (and its
+      # logs) on every run. Best-effort: never let the diagnostics themselves
+      # fail the job, and keep going if a container is already gone.
+      - name: Dump infra diagnostics (on failure)
+        if: failure()
+        run: |
+          set +e
+          echo "=== containers ==="
+          container ls -a | grep bot-bottle-mac || echo "(no bot-bottle-mac containers)"
+          echo "=== networks ==="
+          container network ls | grep bot-bottle-mac || echo "(no bot-bottle-mac networks)"
+          for c in bot-bottle-mac-orchestrator bot-bottle-mac-infra; do
+            echo "=== inspect $c ==="
+            container inspect "$c" || echo "($c not found)"
+            echo "=== logs $c ==="
+            container logs "$c" || echo "($c logs unavailable)"
+          done
+          exit 0
+
+      # Remove the singleton infra container so a crashed or cancelled run
+      # cannot leave `bot-bottle-mac-infra` wedged for the next job.
+      - name: Teardown infra singleton
+        if: always()
+        run: python3 -c 'from bot_bottle.backend.macos_container.infra import MacosInfraService; MacosInfraService().stop()'
+
   # Combined coverage gate: aggregates .coverage.* artifacts uploaded by each
   # test job, then runs the diff-coverage gate (new/changed lines >= 90%).
   #

README.md

@@ -30,7 +30,7 @@
 
 ## Architecture
 
-On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
+On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. git-gate runs over the gateway's consolidated `git-http` daemon (the legacy per-bottle `git://` daemon is not used on this backend); keys are provisioned dynamically at launch and revoked on teardown.
 
 On the Firecracker backend, a bottle is an agent microVM plus a Docker gateway for egress, git-gate, and supervise. The VM reaches the gateway over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the gateway. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
 
@@ -75,6 +75,8 @@ On compatible macOS hosts, the default backend requires Apple's `container` CLI
 
 Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
 
+> **CI (macOS Apple Container):** the `integration-macos` job (`.gitea/workflows/test.yml`) runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container` on a self-hosted macOS runner labelled `macos`, because Apple Container needs the host virtualization framework and cannot run in a Linux container (so it can't reuse the `kvm` runner). Provision an Apple Silicon host with the `container` CLI on `PATH` and `container system status` running, then register the runner in **host mode** (not docker mode) with the `macos` label — `brew install gitea-runner` (the `act_runner` rename). Give it a Python ≥ 3.11 with `coverage` importable on the launchd service's `PATH` (a launchd service doesn't inherit your shell profile, so pin `node` and the Python env explicitly). The job is **advisory** — `workflow_dispatch` (manual) only, never triggered by push or PR — since a single laptop that sleeps/roams must not block merges or churn on every push to main; its coverage doesn't feed the gate. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1.
+
 ### Containers inside a bottle
 
 A bottle may set `nested_containers: true`. On the macOS backend this starts a

docs/ci.md

@@ -2,11 +2,22 @@
 
 The test workflow lives at [`.gitea/workflows/test.yml`](../.gitea/workflows/test.yml).
 It runs the unit suite plus one integration job per backend
-(`integration-docker`, `integration-firecracker`) on:
+(`integration-docker`, `integration-firecracker`, `integration-macos`) on:
 
 - every push to a branch with an open pull request, and
 - every push to `main`.
 
+`integration-macos` is the exception: it is **advisory**, running only on
+`workflow_dispatch` (manual dispatch), never on push or pull requests. It targets the
+Apple Container backend on a self-hosted macOS runner (label `macos`,
+registered in host mode — Apple Container can't run in a Linux container, so it
+can't reuse the `kvm` runner). A single non-redundant laptop must not be able
+to block a PR merge, so the job stays out of the `coverage` job's `needs` and
+its coverage never feeds the diff-coverage gate. Because the infra container is
+a singleton (`bot-bottle-mac-infra`), the job declares a `concurrency` group
+and tears the container down on exit; keep runner concurrency at 1. See the
+README "macOS Apple Container" CI note for runner provisioning.
+
 Each integration job selects its backend via `BOT_BOTTLE_BACKEND` and
 runs a **preflight** (`./cli.py backend status --backend=<name>`) that
 prints a clear per-check readiness summary and fails the job when the

docs/prds/prd-new-macos-container-ci-runner.md

@@ -0,0 +1,142 @@
+# PRD prd-new: macOS (Apple Container) CI runner
+
+- **Status:** Draft
+- **Author:** Claude
+- **Created:** 2026-07-25
+- **Issue:** #426
+
+## Summary
+
+CI has no runner for the `macos-container` (Apple Container) backend.
+`.gitea/workflows/test.yml` exercises Docker (`ubuntu-latest`) and
+Firecracker (self-hosted `kvm`) but never the macOS backend. This PRD adds a
+self-hosted macOS runner (label `macos`) and an advisory `integration-macos`
+job that runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container`,
+so the backend that is the default on macOS stops shipping unexercised.
+
+## Problem
+
+The gap is not theoretical. `5ad3449` moved `bot_bottle` from flat files under
+`/app` into a pip-installed package but left init scripts spawning the
+supervisor as `python3 /app/gateway_init.py`, which no longer exists. Both the
+Firecracker and macOS backends carried the identical bug:
+
+- **firecracker** — caught and fixed in `127ba49` because the KVM runner
+  (added in `c193b04`, PR #349) runs that backend's integration suite.
+- **macos-container** — survived on `main` and only surfaced when a human ran
+  `bot-bottle start` by hand.
+
+The failure mode is expensive to debug: the supervisor never starts, so
+mitmdump never generates its CA, and launch dies downstream with
+`GatewayError: gateway CA not available`, which points at TLS rather than at
+the supervisor. Unit tests did not help — `test_macos_infra` asserted the
+substring `"gateway_init.py"`, which the *broken* path satisfies. (That
+specific assertion has since been tightened to the module form
+`bot_bottle.gateway_init`, matching its Firecracker twin, so the exact
+regression is now covered on `ubuntu-latest`. What remains missing is the
+end-to-end runner that would catch the *next* macOS-only launch regression.)
+
+PR #470 (#414) already made the integration suite backend-agnostic:
+`skip_unless_selected_backend_available()` gates on the *selected* backend's
+own `is_backend_ready()` rather than `docker_available()`, and each
+integration job runs `./cli.py backend status --backend=<name>` as a preflight
+that fails loudly when the backend is missing. That is the machinery this job
+plugs into; this PRD supplies the runner and the job.
+
+## Goals / Success criteria
+
+- A macOS runner is registered and picks up jobs by the `macos` label.
+- An `integration-macos` job runs the integration suite against
+  `BOT_BOTTLE_BACKEND=macos-container`.
+- The job **fails, not skips**, when the backend is unavailable on the runner
+  (via the `backend status` preflight).
+- Reverting the `macos_container/infra.py` supervisor fix makes the job fail:
+  the broken supervisor path throws `GatewayError` at bottle launch, which is
+  `TestSandboxEscape.setUpClass`, failing the whole class before any individual
+  attack runs.
+
+## Non-goals
+
+- Making `integration-macos` a **required** PR check. It runs on
+  `workflow_dispatch` (manual dispatch) only — never on push or PRs. A single
+  non-redundant laptop that sleeps and roams must never be able to block a PR
+  merge or churn unattended on every push to main, and it is deliberately kept
+  out of the `coverage` job's `needs` so the diff-coverage gate never depends on
+  it.
+- Multi-machine or hosted macOS runners. Apple Container needs the host
+  virtualization framework, so the runner must be a physical/VM macOS host on
+  Apple Silicon — it cannot reuse the KVM runner or run in a Linux container.
+- Coverage aggregation from the macOS job into the combined gate (would couple
+  the gate to the laptop).
+
+## Design
+
+### Runner (operational, provisioned once)
+
+- Apple Silicon macOS host with Apple's `container` CLI installed and
+  `container system status` reporting `running`.
+- Install the runner: `brew install gitea-runner` (the `act_runner` rename),
+  registered in **host mode** with label `macos` — not docker mode, because
+  Apple Container needs the host `container` CLI and virtualization framework,
+  not a nested container.
+- A Python ≥ 3.11 with `coverage` importable on the runner's `PATH`. Because a
+  launchd service does not inherit an interactive shell's `PATH`, pin `node`
+  (for the JS `actions/*`) and the Python env explicitly in the service
+  environment rather than relying on `nvm`/shell profile.
+- Concurrency 1. The infra container is a singleton (`bot-bottle-mac-infra`),
+  so two simultaneous runs on one host collide (#425). The job also declares a
+  `concurrency` group as belt-and-suspenders and tears the singleton down after
+  each run.
+
+### `integration-macos` job
+
+Modeled on `integration-firecracker`:
+
+- `runs-on: [self-hosted, macos]`.
+- `if:` `workflow_dispatch` only (advisory, manual dispatch; never push or PRs,
+  so no fork-PR exposure, no merge-blocking, and no unattended runs on push).
+- `concurrency: { group: integration-macos-infra, cancel-in-progress: false }`
+  to serialize runs against the singleton.
+- **Preflight** — `command -v container`, `container system status`, then
+  `./cli.py backend status --backend=macos-container`; any failure exits
+  non-zero so a misprovisioned runner fails loudly instead of silently
+  skipping.
+- Run the integration suite under coverage with
+  `BOT_BOTTLE_BACKEND=macos-container` and print a `coverage report -m` for
+  visibility (no upload, not in the gate).
+- **Teardown** (`if: always()`) — `MacosInfraService().stop()` removes the
+  singleton so a crashed run cannot wedge the next one.
+
+### The `test_sandbox_escape` CI guard (the trap #470 left)
+
+`TestSandboxEscape` is the only backend-agnostic integration test that boots a
+real bottle, so it is the one that would catch a macOS launch regression. It
+still carries a second guard that skips under `GITEA_ACTIONS` for every backend
+except `firecracker`:
+
+```python
+@unittest.skipIf(
+    os.environ.get("GITEA_ACTIONS") == "true"
+    and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker",
+    ...,
+)
+```
+
+The skip exists because the *containerized* `act_runner` (docker on
+`ubuntu-latest`) can't see a host bind mount and hides sibling-gateway network
+topology. Those constraints do not apply to a **host-mode** runner — neither
+the KVM host runner nor a macOS host runner is containerized. This PRD relaxes
+the guard to allow both host-mode backends (`firecracker`, `macos-container`)
+through while still skipping on the containerized Docker job. Without this
+change the macOS job would run green while skipping the exact test that proves
+the backend launches — the very false-green this issue is about.
+
+## Open questions
+
+- None known that block the job. git-gate is fully implemented on the macOS
+  backend (the gateway's consolidated `git-http` daemon plus dynamic key
+  provisioning/revocation), so `TestSandboxEscape` attack 5 — secret exfil
+  pushed through git-gate, rejected by the gitleaks hook before the upstream
+  push — runs the same as on the other backends. Any genuinely
+  macOS-specific test adjustment would surface at first runner bring-up, but
+  none is anticipated from the current backend implementation.

tests/integration/test_sandbox_escape.py

@@ -67,14 +67,25 @@ _DUMMY_HOST_KEY = (
 )
 
 
+# Backends whose CI runner is HOST-mode (self-hosted), so the test process
+# and the backend share a host. The containerized act_runner (docker on
+# ubuntu-latest) is the one that can't see the host bind mount egress_tls_init
+# uses and hides sibling-gateway network topology; host-mode runners
+# (firecracker/KVM, macos-container) don't have those constraints, so the test
+# runs there. Keep this in sync with the `runs-on` labels in
+# .gitea/workflows/test.yml.
+_HOST_MODE_CI_BACKENDS = frozenset({"firecracker", "macos-container"})
+
+
 @skip_unless_selected_backend_available()
 @unittest.skipIf(
     os.environ.get("GITEA_ACTIONS") == "true"
-    and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker",
-    "skipped under act_runner unless BOT_BOTTLE_BACKEND=firecracker: "
+    and os.environ.get("BOT_BOTTLE_BACKEND") not in _HOST_MODE_CI_BACKENDS,
+    "skipped under the containerized act_runner (docker on ubuntu-latest): "
     "egress_tls_init uses a host bind mount the runner container can't "
     "see, and the network topology hides sibling-gateway visibility — "
-    "these constraints don't apply on the self-hosted KVM runner",
+    "these constraints don't apply on the self-hosted host-mode runners "
+    "(firecracker/KVM, macos-container)",
 )
 class TestSandboxEscape(unittest.TestCase):
     """End-to-end attacks against a real bottle. The bottle stays