didericis/bot-bottle
1
0
Fork 0
Code Issues 25 Pull Requests 9 Actions Packages Projects Releases Wiki Activity

feat(backend+orchestrator): slice 13d — docker cut-over to the consolidated gateway (e2e green) #379

Closed
didericis-claude wants to merge 3 commits from orchestrator-cutover into orchestrator-agent-compose
pull from: orchestrator-cutover
merge into: didericis:orchestrator-agent-compose
Conversation 1 Commits 3 Files Changed 14
+376 -356
didericis-claude commented 2026-07-13 23:08:12 -04:00
Collaborator
Copy Link
Copy Source

The keystone, validated end-to-end on real docker. test_sandbox_escape passes all 5 attacks (egress DLP + git-gate gitleaks) through the shared gateway — the entire consolidation (slices 11–13) works on the real host.

Two commits:

1. Containerize the orchestrator

Real-on-host testing found the host (NixOS) firewall drops container→host, so a host-process orchestrator is unreachable from the gateway. Fix = the PRD's virtualize the orchestrator: run it as a container on the shared network (gateway reaches it by name, container↔container). Register-only stub broker → no docker socket needed. Also: control plane returns 500 on a dispatch error instead of crashing the connection.

2. Cut DockerBottleBackend.launch over

Replaces the per-bottle bundle: launch_consolidated() (register + provision git-gate into the gateway + pinned source IP) → install the shared gateway CA → agent-only compose on the gateway network → point the agent's git-gate (http://<gw>:9420) + supervise (http://<gw>:9100) at the gateway → teardown = compose down + dereg/deprovision. Dropped per-bottle networks, per-bottle CA, and the bundle service.

Fixes only the real e2e could surface (unit mocks couldn't): the gateway needed the bottle-agnostic git hooks installed; source-IP allocation must read the actual container IPs on the network (the orchestrator container sits on it and was colliding).

Reviewer note

The bot-bottle-sidecars image must be rebuilt when its flat sources change — ensure_built only builds-if-missing, so a stale image silently runs the OLD single-tenant daemons (this bit me mid-validation). A content-hash/--rebuild path is a tracked follow-up.

Verification

pyright 0, pylint 9.83/10, unit suite green (1760; the 13 test_sidecar_init /bin/sleep errors are pre-existing NixOS noise), test_sandbox_escape green on real docker. Old bundle-launch unit tests rewritten against the new collaborators.

Stacked on #378.

🤖 Generated with Claude Code

**The keystone, validated end-to-end on real docker.** `test_sandbox_escape` passes all 5 attacks (egress DLP + git-gate gitleaks) through the shared gateway — the entire consolidation (slices 11–13) works on the real host. Two commits: ### 1. Containerize the orchestrator Real-on-host testing found the host (NixOS) firewall **drops container→host**, so a host-process orchestrator is unreachable from the gateway. Fix = the PRD's *virtualize the orchestrator*: run it as a container on the shared network (gateway reaches it **by name**, container↔container). Register-only stub broker → no docker socket needed. Also: control plane returns 500 on a dispatch error instead of crashing the connection. ### 2. Cut `DockerBottleBackend.launch` over Replaces the per-bottle bundle: `launch_consolidated()` (register + provision git-gate into the gateway + pinned source IP) → install the **shared gateway CA** → agent-only compose on the gateway network → point the agent's git-gate (`http://<gw>:9420`) + supervise (`http://<gw>:9100`) at the gateway → teardown = compose down + dereg/deprovision. Dropped per-bottle networks, per-bottle CA, and the bundle service. **Fixes only the real e2e could surface** (unit mocks couldn't): the gateway needed the bottle-agnostic git hooks installed; source-IP allocation must read the *actual* container IPs on the network (the orchestrator container sits on it and was colliding). ### Reviewer note The `bot-bottle-sidecars` image must be **rebuilt when its flat sources change** — `ensure_built` only builds-if-missing, so a stale image silently runs the OLD single-tenant daemons (this bit me mid-validation). A content-hash/`--rebuild` path is a tracked follow-up. ### Verification pyright **0**, pylint **9.83/10**, unit suite green (1760; the 13 `test_sidecar_init` `/bin/sleep` errors are pre-existing NixOS noise), **`test_sandbox_escape` green on real docker**. Old bundle-launch unit tests rewritten against the new collaborators. Stacked on #378. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
didericis-claude added 1 commit 2026-07-13 23:08:13 -04:00
feat(orchestrator): slice 13d(i) — containerize the orchestrator (validated on docker)
lint / lint (push) Successful in 2m5s
Details
test / unit (pull_request) Successful in 1m5s
Details
test / integration (pull_request) Successful in 21s
Details
test / coverage (pull_request) Successful in 1m13s
Details
a1acbc00ed 
Real-on-host testing surfaced two issues the unit-mocked slices couldn't:

1. The host (NixOS) firewall DROPS container->host traffic, so a host-process
   orchestrator is unreachable from the gateway container. Fix: run the
   orchestrator AS a container on the shared gateway network (PRD 0070's
   "virtualize the orchestrator") — the gateway reaches it by container name
   over docker DNS (container<->container, no firewall), and the host CLI
   reaches it via a published loopback port.
2. The control plane crashed the connection on a dispatch error (e.g. a
   broker failure) instead of returning 500.

Changes:
- lifecycle: OrchestratorProcess (host process) -> OrchestratorService
  (containers). Runs the control plane in the bundle image with the repo
  bind-mounted (orchestrator is stdlib-only), register-only stub broker so it
  needs NO docker socket (the backend launches agents; the host manages both
  containers). Registry DB persists via a host-root mount. ensure_running is
  an idempotent singleton over both containers.
- gateway: BOT_BOTTLE_ORCHESTRATOR_URL is now the orchestrator's *by-name*
  URL on the shared network (dropped the host.docker.internal hack).
- control_plane: _serve wraps dispatch — a failure returns 500, never crashes
  the connection.
- OrchestratorProcess default broker -> stub (register-only) for docker.

Validated live end-to-end: both containers up, gateway->orchestrator by name
OK, register -> resolve-by-source-IP returns the bottle's policy from inside
the gateway.

pyright 0 errors; pylint 9.83/10; unit suite green (1760 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
didericis added 1 commit 2026-07-13 23:28:39 -04:00
feat(backend): slice 13d — cut docker launch over to the consolidated gateway (e2e green)
lint / lint (push) Failing after 2m9s
Details
test / unit (pull_request) Successful in 1m5s
Details
test / integration (pull_request) Successful in 23s
Details
test / coverage (pull_request) Successful in 1m36s
Details
d65e97de73 
Rewire DockerBottleBackend.launch to the consolidated model, replacing the
per-bottle sidecar bundle. VALIDATED END-TO-END on real docker:
test_sandbox_escape passes all 5 attacks (egress DLP + git-gate gitleaks)
through the shared gateway.

launch now:
  - mints git-gate dynamic keys (if any), then launch_consolidated() to
    register the bottle + provision its git-gate state into the gateway and
    get the agent's attach context (pinned source IP, gateway address);
  - installs the SHARED gateway CA (gateway.ca_cert_pem) into the agent;
  - renders the agent-only consolidated compose on the gateway network at the
    pinned IP, proxied through the gateway;
  - points the agent's git-gate insteadOf (http://<gw>:9420) and supervise
    MCP (http://<gw>:9100) at the gateway (DockerBottlePlan.agent_git_gate_url
    / agent_supervise_url);
  - teardown = compose down + teardown_consolidated (dereg + deprovision).
Dropped the per-bottle networks, per-bottle egress CA, and bundle service.

Fixes surfaced by the real e2e (couldn't be caught by unit mocks):
  - provision_git_gate installs the bottle-agnostic pre-receive/access hooks
    into the gateway (were cp'd per-bundle before);
  - source-IP allocation reads the *actual* container IPs on the network
    (gateway + orchestrator + agents), not just gateway + registry — the
    orchestrator container sits on the network and was colliding.

Unit tests for the old bundle launch rewritten against the new collaborators.

NOTE for reviewers: the gateway/bundle image (bot-bottle-sidecars) must be
rebuilt when its flat sources change — `ensure_built` only builds-if-missing,
so a stale image silently runs the OLD single-tenant daemons. A content-hash
/ --rebuild path is a follow-up.

pyright 0 errors; pylint 9.83/10; unit suite green (1760 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise);
test_sandbox_escape green on real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
didericis-claude changed title from feat(orchestrator): slice 13d(i) — containerize the orchestrator (validated on docker) to feat(backend+orchestrator): slice 13d — docker cut-over to the consolidated gateway (e2e green) 2026-07-13 23:29:04 -04:00
didericis added 1 commit 2026-07-14 00:55:34 -04:00
fix(tests): annotate on_network param — pyright strict (CI lint)
lint / lint (push) Successful in 2m22s
Details
test / unit (pull_request) Successful in 1m2s
Details
test / integration (pull_request) Successful in 20s
Details
test / coverage (pull_request) Successful in 1m15s
Details
9d85541b67 
The cut-over's edit to test_consolidated_launch left the new on_network
kwarg untyped; pyright strict rejects it. No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
didericis-claude commented 2026-07-14 00:57:47 -04:00
Author
Collaborator
Copy Link
Copy Source

Collapsed into the single stack PR #380 (per-host orchestrator, PRD 0070). Closing this slice PR; review + merge happens on #380.

Collapsed into the single stack PR #380 (per-host orchestrator, PRD 0070). Closing this slice PR; review + merge happens on #380.
didericis-claude closed this pull request 2026-07-14 00:57:48 -04:00
Some checks are pending
lint / lint (push) Successful in 2m22s
Details
test / unit (pull_request) Successful in 1m2s
Details
test / integration (pull_request) Successful in 20s
Details
test / coverage (pull_request) Successful in 1m15s
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
didericis-claude (didericis (claude)) didericis-codex (didericis (codex)) didericis-gemma (gemma) didericis
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: didericis/bot-bottle#379
Delete Branch "orchestrator-cutover"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?