Reconciles the chunk-1 rebase onto the current main:
- `_network_container_ips` (added on main) raised the pre-rename
`ConsolidatedLaunchError`; renamed to `InfraLaunchError` to match the rest of
the rename.
- main grew `backend/base.py` to 563 lines; with the gateway-attach contract
(39 lines) it is 602, so the guardrail cap moves 600 -> 615. The reconcile
flow still lives in `backend/gateway_attach.py`, not base.py.
Refs #516, #519.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The gateway-attach reconcile (PRD 0081) gathered its resources via a fresh
default DockerInfraService(), so on a cold-boot bring-up of a NON-default
gateway instance — e.g. an isolated integration test's `-itest-` gateway — it
read the default `bot-bottle-orch-gateway`, which doesn't exist for that
instance, and setUpClass died with "No such container: bot-bottle-orch-gateway"
(the 0081 PR's integration-docker job).
Thread the infra service whose gateway just cold-booted from
DockerInfraService.ensure_running() into DockerBottleBackend so
_gateway_attach_resources() reads THAT gateway's CA. Ordinary construction (no
infra) still falls back to the per-host default singleton — production is
unchanged — and the resources-first / fail-hard reconcile ordering is kept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses review on #519 (@didericis 6143 + the fail-hard direction over the
codex skip).
Base owns the flow (6143 / ADR 0006). `attach_bottled_agents_to_gateway()` is
now a concrete method on `BottleBackend` that delegates to
`gateway_attach.reconcile_running_bottles`; backends override only three
primitives — `_gateway_attach_resources()`, `_running_bottles()`,
`_attach_bottle_to_gateway()`. The shared control flow + error policy live in
one place so a backend can't drift onto a bespoke loop or a silent skip. Keeps
the reconcile flow + `GatewayAttachResources` out of `backend/base.py` (the
size guardrail) in a dedicated `backend/gateway_attach.py`. New ADR 0006 records
the "shared behaviour in the base backend, subclasses override primitives"
theme.
Fail hard, never skip (the maintainer's direction over the codex review's
skip). Any attach failure now aborts bring-up instead of being logged and
skipped: a bottle that silently can't reach the fresh gateway (its egress just
starts failing TLS) is worse than a loud failure. Every bottle is attempted and
the failures are raised together (aggregate `InfraLaunchError`) so one bring-up
surfaces the full blast radius. Resource gathering (the CA fetch) also fails
hard. PRD 0081 goal + design updated to match (reversed from the earlier
"tolerate per-bottle failures" draft).
Bumps the base.py guardrail cap 580->600 for the new contract surface (the
delegator + 3 abstract primitives); the flow itself lives in gateway_attach.py.
Refs #516, #519.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mechanical rename addressing review 6141 on #519: "consolidated launch" was
unclear about what it composes. Rename the per-backend module
`consolidated_launch.py` -> `infra_launch.py` and the error
`ConsolidatedLaunchError` -> `InfraLaunchError` across all three backends and
their callers/tests. Unify the three identical per-backend error classes into
one `InfraLaunchError` defined in `backend/base.py` (re-exported by each
`infra_launch`) so the base backend can raise and catch a single shared type —
groundwork for the base owning the gateway-attach flow.
Add a glossary entry defining **infra** = the per-host gateway + orchestrator
service pair every bottle attaches to.
No behavior change.
Refs #516, #519.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `attach_bottled_agents_to_gateway()` as an `@abc.abstractmethod` on
`BottleBackend` and implement the first reconcile step across all three
backends: on a gateway cold boot, replace every already-running agent's
trusted CA with the freshly-minted gateway CA. Reconciling running bottles
is a backend responsibility (only the backend can enumerate its agents and
reach them — firecracker over SSH, docker/macOS over exec/cp), so making it
an abstract method keeps the fix cross-backend by construction.
The gateway rootfs is ephemeral, so a rebuild mints a new CA that every
running bottle distrusts (SSL verification fails — #510). This reconcile is
what distributes the fresh CA, so a routine gateway rebuild now doubles as a
free CA rotation. Per-bottle steps are best-effort: one unreachable or
malformed bottle is logged and skipped, never blocking the others.
Trigger — `Gateway.connect_to_orchestrator` now returns a cold-boot bool
(True when it actually (re)brought the gateway up, False when a healthy
current gateway was left untouched); each infra `ensure_running` gates the
reconcile on it, so it fires exactly on cold boot and never on an adopt.
Git-gate re-provision and egress-token restore fold into this same reconcile
on the same trigger in follow-up PRs (#516).
Refs #516. Addresses #510.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a single-secret counterpart to reprovision_from_secret's restore-all:
update ONE egress token for a running bottle without a relaunch, for
refreshing a short-lived host credential (e.g. the Codex access token)
whose launch-time snapshot has expired.
- registry_store.store_agent_secret: per-key upsert (delete+insert of the
one row), the counterpart of store_agent_secrets' replace-all.
- OrchestratorCore.update_agent_secret: set the in-memory token AND upsert
the re-encrypted row under the bottle's env_var_secret, leaving other
tokens untouched.
- POST /bottles/<id>/secret (cli-only) + client.update_agent_secret.
Refs #510, #512
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`container image tag` only accepts image-name[:tag] as its source and
rejects a bare 64-hex image ID ("cannot specify 64 byte hex string as
reference"), so pinning the agent image for the nested-containers layer
died before the build could start. Tag from the ref instead; the
existing post-tag inspection is what keeps the handoff fail-closed if
the ref moves between the two commands.
The unit test mocked subprocess and asserted the ID-as-source call
shape, so it never saw the CLI's rejection. It now pins the ref as the
source, and covers the bare-hex ID that `container image inspect`
actually reports plus the mid-flight tag move.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`doctor` told users to run `./cli.py backend setup`. cli.py is a four-line
wrapper at the repo root that calls bot_bottle.cli:main, and it does not ship —
[tool.setuptools.packages.find] includes only bot_bottle*, so anyone who
installed rather than cloned has no such file. Confirmed against a real
install: the user's tree contains exactly one executable, `bot-bottle`, and no
cli.py anywhere, while doctor recommended `./cli.py` three times. Invisible in
development, where ./cli.py works fine from a checkout, which is why only a
clean-install test surfaced it.
The two entry points are the same code, so the fix is to name the one that
always exists. 141 replacements across 45 files: the runtime messages that
caused this, plus README, docs, PRDs, research notes and test prose, so nothing
teaches the invocation a user cannot run.
scripts/demo.sh is deliberately untouched — it *executes* ./cli.py from a
checkout, where that is the correct and available path.
Verified end to end: a sandbox install now reports
"Run: bot-bottle backend setup --backend=firecracker", and bot-bottle is on
that user's PATH. Unit suite unchanged against the pre-existing baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
The install itself now works end to end on a fresh macOS account — venv built,
package installed from git, entry point linked — and `doctor` then died with an
unhandled traceback:
PermissionError: [Errno 13] Permission denied: 'ip'
netpool's probe helpers caught only FileNotFoundError. That is not the only way
a probe binary can be unavailable: when a name on PATH exists but this user
cannot execute it, exec fails with EACCES, and CPython reports that in
preference to the ENOENT from the other PATH entries. So `except
FileNotFoundError` misses it and the crash propagates all the way out of
`doctor`. Reproduced directly: a mode-000 file named `ip` on PATH yields
exactly the error above.
_run_ok's docstring already stated the intent — treat an unavailable binary as
failure rather than crashing — so this widens the catch to OSError to match
what it says. Any OSError means the probe could not run, which for a
fail-closed check is indistinguishable from "not present". The same narrow
catch is fixed in overlapping_routes and in the two docker probes
(compose ls, docker ps), which are the same shape and equally reachable.
Deliberately not touched: the FileNotFoundError catches around file I/O in
bottle_state and orchestrator/service, where the narrow exception is correct.
The harness also required `bot-bottle` on PATH before running doctor, which
could never be true: install.sh prints the PATH line rather than editing a
shell profile, by design, so on a fresh account the entry point is installed
and working but not on PATH. It now looks where the installer actually puts
it (~/.local/bin, then the venv) and notes when it's running by absolute path.
That was the harness failing a run for a reason the installer intends.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WEfZZhakx13bxTfXcZCoS5
Address review: the self-heal classified every non-absence `network inspect`
failure as a poisoned network and force-removed the shared gateway. But
inspect also fails on transient daemon/API errors, permission failures,
timeouts, or a bad context — destroying a healthy gateway on that guess would
tear the network out from under every live bottle.
Now the destructive path runs only for the known poison signature (docker's
`ParseAddr` error from the malformed `::1/64` IPv6 gateway). The absent case
(`No such network`) still just creates; any other inspect failure raises a
clear GatewayError without mutating shared state.
Adds a regression test asserting a generic inspect error issues neither
`docker rm --force` nor `docker network rm` and surfaces the error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR #515 stopped bot-bottle from *creating* a gateway network with a
malformed IPv6 subnet (--ipv6=false), but it can't recover a network that
is *already* poisoned. On a daemon that default-enables IPv6, the fixed-name
`bot-bottle-gateway` network gets an `fdd0::/64` subnet whose `::1/64`
gateway trips docker's own netip.ParseAddr, so `docker network inspect`/`ls`
exit non-zero — poisoning every command that reads networks.
Such a network can survive on a shared runner from a pre-fix or concurrent
launch. `_ensure_network` never healed it: its migrate/recreate branch only
ran when `network inspect` *succeeded*, but a poisoned network makes inspect
*fail*, so the code fell through to `network create`, which no-ops on
"already exists" — leaving the poison in place. The next subnet read then
failed with ConsolidatedLaunchError (test_multitenant_isolation), and a bare
`docker network ls` failed too (test_orphan_cleanup).
Fix, two parts:
- gateway.py: when `network inspect` fails for a reason other than "no such
network", treat the network as poisoned and force-remove + recreate it
IPv4-only. Absent-vs-poisoned is distinguished by the inspect stderr.
- test.yml: add an integration-docker preflight that drops the leftover
gateway network (and its container) before the suite, so direct `network
ls`/`inspect` calls in tests are clean even on a daemon where the in-code
heal can't run because inspect itself is what's broken.
Adds unit coverage for the poisoned-heal and the absent-create split.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On a docker daemon that default-enables IPv6 (default-address-pools),
creating the gateway network with only `--subnet` lets the daemon also
attach an fdd0::/64 IPv6 subnet. Its gateway is stored as `::1/64`,
which trips docker's own netip.ParseAddr in `network inspect`/`ls`:
ParseAddr("fdd0:0:0:6::1/64"): unexpected character, want colon
That poisons every `_network_cidr`/`network ls` read and fails the
docker integration suite intermittently (whichever run the runner's
IPv6 pool index lands on a broken network). bot-bottle attribution
pins IPv4 source IPs and has no IPv6 support, so pass `--ipv6=false`
explicitly at network create to keep the gateway network IPv4-only
regardless of the daemon default.
Note: an already-poisoned runner still needs a one-time
`docker network rm bot-bottle-gateway` (and possibly a daemon
restart) to clear the malformed network.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gitea AGit accepts pushes to refs/for/*, refs/draft/*, and
refs/for-review/* and opens pull requests backed by server-managed
refs/pull/<n>/head refs rather than ordinary refs/heads/* branches.
That breaks the git-gate branch workflow: follow-up commits can't be
pushed back through the branch, and Gitea rejects later direct updates
to the generated review ref, so recovery means recreating the PR.
Add a Phase 0 guard to the shared pre-receive hook that rejects
creation or update of those AGit review refs before any gitleaks scan
or upstream forward, with a message pointing callers at the
branch-backed PR workflow. Deletions (new == zero) stay allowed so
legacy AGit refs can still be cleaned up; normal branches and tags are
untouched.
Closes#506
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
didericis-claude
didericis
didericis-codex
Gitea Actions