refactor(orchestrator): split conflated sidecar image into orchestrator + gateway images
One image — `bot-bottle-sidecars:latest`, built from `Dockerfile.sidecars` — served two unrelated roles: the egress/git-gate/supervise *data plane* and the orchestrator *control plane* (which ran the same image with the entrypoint overridden to `python3 -m bot_bottle.orchestrator`). The control plane is stdlib-only, so it needed none of the mitmproxy/git/gitleaks payload it was riding on — while being the most secret-dense process on the host (PRD 0070's "secret concentration"). Split into two purpose-built images: - `Dockerfile.gateway` -> `bot-bottle-gateway:latest` — the data plane (renamed from Dockerfile.sidecars; identical contents). - `Dockerfile.orchestrator` -> `bot-bottle-orchestrator:latest` — a lean `python:3.12-slim` runtime; the bind-mounted `bot_bottle` package supplies the code (so the #381 source-hash recreate semantics are unchanged). `OrchestratorService` now takes distinct `image` (control plane, default `ORCHESTRATOR_IMAGE`) and `gateway_image` (data plane, default `GATEWAY_IMAGE`) instead of feeding one `self.image` to both, and builds the lean image (build-if-missing) before starting the container. The per-bottle bundle constants in `backend/docker/sidecar_bundle.py` now alias the gateway constants so a bundle and the shared gateway can never drift onto different images. The `bot-bottle-sidecars` *image* name and `Dockerfile.sidecars` are gone; the per-bottle *container* name prefix (`bot-bottle-sidecars-<slug>`) is intentionally left for a separate change. Verified end-to-end: both images build; the lean image runs the control plane; `ensure_running` brings up the orchestrator on `bot-bottle-orchestrator:latest` and the gateway on `bot-bottle-gateway:latest` (distinct images) and reports healthy. Closes #384. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
@@ -1,4 +1,12 @@
|
|||||||
# Per-bottle sidecar bundle image (PRD 0024).
|
# Gateway data-plane image (PRD 0024 bundle shape; PRD 0070 gateway).
|
||||||
|
#
|
||||||
|
# The egress / git-gate / supervise *data plane* — one image, shared by
|
||||||
|
# the consolidated per-host gateway (PRD 0070) and, on backends that
|
||||||
|
# still run a per-bottle bundle, each bottle's sidecar. It is NOT the
|
||||||
|
# orchestrator control plane: that is the separate, lean
|
||||||
|
# `bot-bottle-orchestrator` image (Dockerfile.orchestrator, #384), which
|
||||||
|
# ships only python + the stdlib-only `bot_bottle` package and none of
|
||||||
|
# this image's mitmproxy / git / gitleaks payload.
|
||||||
#
|
#
|
||||||
# Collapses the prior per-sidecar images (egress, git-gate,
|
# Collapses the prior per-sidecar images (egress, git-gate,
|
||||||
# supervise) into one. A small stdlib-Python init supervisor at
|
# supervise) into one. A small stdlib-Python init supervisor at
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
# Orchestrator control-plane image (PRD 0070, #384).
|
||||||
|
#
|
||||||
|
# The per-host orchestrator runs `python3 -m bot_bottle.orchestrator`.
|
||||||
|
# The `bot_bottle` package is **stdlib-only** by design, so the control
|
||||||
|
# plane needs nothing but a Python runtime — none of the gateway's
|
||||||
|
# mitmproxy / git / gitleaks payload (that is the separate
|
||||||
|
# `bot-bottle-gateway` image, Dockerfile.gateway). Splitting them keeps
|
||||||
|
# the secret-dense control plane (it concentrates every bottle's egress
|
||||||
|
# tokens — see PRD 0070's "secret concentration") on a minimal
|
||||||
|
# dependency surface.
|
||||||
|
#
|
||||||
|
# The repo is bind-mounted read-only into the container at run time (see
|
||||||
|
# `orchestrator/lifecycle.py`), so the source is NOT copied in here: the
|
||||||
|
# image is just the runtime. `ensure_running` recreates the container
|
||||||
|
# only when the bind-mounted source hash changes (#381), which is why
|
||||||
|
# the code stays a mount rather than a baked layer.
|
||||||
|
|
||||||
|
FROM python:3.12-slim
|
||||||
|
|
||||||
|
# No third-party deps to install — stdlib only. Kept as an explicit,
|
||||||
|
# self-documenting stage so a future confinement step (baking the
|
||||||
|
# package, dropping the bind mount) has an obvious home.
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Documentation only; lifecycle.py overrides the entrypoint to
|
||||||
|
# `python3 -m bot_bottle.orchestrator` with the runtime flags.
|
||||||
|
ENTRYPOINT ["python3", "-m", "bot_bottle.orchestrator"]
|
||||||
@@ -1,30 +1,33 @@
|
|||||||
"""Sidecar bundle constants + helpers for the Docker backend
|
"""Sidecar bundle constants + helpers for the Docker backend
|
||||||
(PRD 0024).
|
(PRD 0024).
|
||||||
|
|
||||||
The bundle image (built by Dockerfile.sidecars, PRD 0024 chunk 1)
|
A per-bottle sidecar bundle runs egress + git-gate + supervise as one
|
||||||
runs egress + git-gate + supervise as one container per bottle
|
container under a small Python init supervisor. As of PRD 0024 chunk 5
|
||||||
under a small Python init supervisor. As of chunk 5 the bundle
|
the bundle is the only shape — the legacy four-sidecar topology and its
|
||||||
is the only shape — the legacy four-sidecar topology and its
|
`BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone.
|
||||||
`BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
|
|
||||||
|
The bundle's image is the **same data plane** as the consolidated
|
||||||
|
per-host gateway (PRD 0070), so it is one image with one name and one
|
||||||
|
Dockerfile: `bot-bottle-gateway` / `Dockerfile.gateway` (#384). These
|
||||||
|
constants alias the gateway's so a per-bottle bundle and the shared
|
||||||
|
gateway can never drift onto different images."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
from ...orchestrator.gateway import GATEWAY_DOCKERFILE, GATEWAY_IMAGE
|
||||||
|
|
||||||
|
|
||||||
# Bundle image. Defaults to a built-locally tag (built from the
|
# The per-bottle bundle image == the gateway data-plane image (aliased so
|
||||||
# repo's Dockerfile.sidecars via compose `build:`). Operators
|
# there is one source of truth; the `BOT_BOTTLE_GATEWAY_IMAGE` env override
|
||||||
# pinning to a published digest can override via env.
|
# on the gateway constant applies here too).
|
||||||
SIDECAR_BUNDLE_IMAGE = os.environ.get(
|
SIDECAR_BUNDLE_IMAGE = GATEWAY_IMAGE
|
||||||
"BOT_BOTTLE_SIDECAR_IMAGE",
|
SIDECAR_BUNDLE_DOCKERFILE = GATEWAY_DOCKERFILE
|
||||||
"bot-bottle-sidecars:latest",
|
|
||||||
)
|
|
||||||
|
|
||||||
SIDECAR_BUNDLE_DOCKERFILE = "Dockerfile.sidecars"
|
|
||||||
|
|
||||||
|
|
||||||
def sidecar_bundle_container_name(slug: str) -> str:
|
def sidecar_bundle_container_name(slug: str) -> str:
|
||||||
"""`bot-bottle-sidecars-<slug>`. Same prefix scheme as the
|
"""`bot-bottle-sidecars-<slug>`. Same prefix scheme as the
|
||||||
per-sidecar containers it replaces, so the dashboard's
|
per-sidecar containers it replaces, so the dashboard's
|
||||||
discovery-by-prefix logic keeps working."""
|
discovery-by-prefix logic keeps working. (The per-bottle *container*
|
||||||
|
name keeps the historical `sidecars-` prefix; only the *image* was
|
||||||
|
renamed to `bot-bottle-gateway` in #384.)"""
|
||||||
return f"bot-bottle-sidecars-{slug}"
|
return f"bot-bottle-sidecars-{slug}"
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ container.
|
|||||||
|
|
||||||
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
||||||
ships flat into the sidecar bundle image alongside this file —
|
ships flat into the sidecar bundle image alongside this file —
|
||||||
see `Dockerfile.sidecars`)."""
|
see `Dockerfile.gateway`)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -22,7 +22,7 @@ except ImportError: # pragma: no cover - host-side path
|
|||||||
from .yaml_subset import YamlSubsetError, parse_yaml_subset
|
from .yaml_subset import YamlSubsetError, parse_yaml_subset
|
||||||
|
|
||||||
# DLP detector-config parsing lives in a sibling module (also flat-bundled
|
# DLP detector-config parsing lives in a sibling module (also flat-bundled
|
||||||
# into the gateway — see Dockerfile.sidecars). Re-exported below so existing
|
# into the gateway — see Dockerfile.gateway). Re-exported below so existing
|
||||||
# `from egress_addon_core import ON_MATCH_*` callers keep working.
|
# `from egress_addon_core import ON_MATCH_*` callers keep working.
|
||||||
try:
|
try:
|
||||||
from egress_dlp_config import ( # type: ignore[import-not-found]
|
from egress_dlp_config import ( # type: ignore[import-not-found]
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ kept apart from the request-time scan/decision flow in `egress_addon_core`
|
|||||||
so each half reads top-to-bottom without scrolling past the other.
|
so each half reads top-to-bottom without scrolling past the other.
|
||||||
|
|
||||||
Stdlib-only; ships flat into the sidecar bundle image alongside
|
Stdlib-only; ships flat into the sidecar bundle image alongside
|
||||||
`egress_addon_core.py` — see `Dockerfile.sidecars`."""
|
`egress_addon_core.py` — see `Dockerfile.gateway`."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ from pathlib import Path
|
|||||||
from urllib.parse import urlsplit
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
# policy_resolver ships flat alongside this file in the sidecar bundle
|
# policy_resolver ships flat alongside this file in the sidecar bundle
|
||||||
# image (see Dockerfile.sidecars); the bot_bottle.* fallback is the
|
# image (see Dockerfile.gateway); the bot_bottle.* fallback is the
|
||||||
# host-side / test path. Mirrors egress_addon's import shape.
|
# host-side / test path. Mirrors egress_addon's import shape.
|
||||||
try:
|
try:
|
||||||
from policy_resolver import ( # type: ignore[import-not-found]
|
from policy_resolver import ( # type: ignore[import-not-found]
|
||||||
@@ -104,7 +104,7 @@ def resolve_sandbox_root(
|
|||||||
|
|
||||||
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
|
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
|
||||||
# imported: this module ships as a flat top-level sibling in the sidecar
|
# imported: this module ships as a flat top-level sibling in the sidecar
|
||||||
# bundle image (see Dockerfile.sidecars), not as part of the bot_bottle
|
# bundle image (see Dockerfile.gateway), not as part of the bot_bottle
|
||||||
# package, so `bot_bottle.git_gate` and its dependency chain aren't
|
# package, so `bot_bottle.git_gate` and its dependency chain aren't
|
||||||
# available at runtime.
|
# available at runtime.
|
||||||
GIT_GATE_TIMEOUT_SECS = 15
|
GIT_GATE_TIMEOUT_SECS = 15
|
||||||
|
|||||||
@@ -45,12 +45,12 @@ MITMPROXY_HOME = "/home/mitmproxy/.mitmproxy"
|
|||||||
GATEWAY_CA_VOLUME = "bot-bottle-gateway-mitmproxy"
|
GATEWAY_CA_VOLUME = "bot-bottle-gateway-mitmproxy"
|
||||||
GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem"
|
GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem"
|
||||||
|
|
||||||
# The real sidecar-bundle image + its Dockerfile. Kept as a local constant
|
# The gateway data-plane image + its Dockerfile. Kept as a local constant
|
||||||
# rather than imported from backend.docker.sidecar_bundle, which would drag
|
# rather than imported from backend.docker.sidecar_bundle, which would drag
|
||||||
# the whole backend layer into the lean orchestrator (see #359); unify when
|
# the whole backend layer into the lean orchestrator (see #359); unify when
|
||||||
# that lands. Env override matches the backend's BOT_BOTTLE_SIDECAR_IMAGE.
|
# that lands. Env override matches the backend's BOT_BOTTLE_GATEWAY_IMAGE.
|
||||||
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_SIDECAR_IMAGE", "bot-bottle-sidecars:latest")
|
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_GATEWAY_IMAGE", "bot-bottle-gateway:latest")
|
||||||
GATEWAY_DOCKERFILE = "Dockerfile.sidecars"
|
GATEWAY_DOCKERFILE = "Dockerfile.gateway"
|
||||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
|
||||||
|
|
||||||
@@ -86,8 +86,8 @@ class Gateway(abc.ABC):
|
|||||||
class DockerGateway(Gateway):
|
class DockerGateway(Gateway):
|
||||||
"""The consolidated gateway as a single, fixed-name Docker container.
|
"""The consolidated gateway as a single, fixed-name Docker container.
|
||||||
|
|
||||||
`image_ref` defaults to the real sidecar-bundle image; `ensure_built`
|
`image_ref` defaults to the gateway data-plane image; `ensure_built`
|
||||||
builds it from `Dockerfile.sidecars` when it's missing. (Note: slice 5
|
builds it from `Dockerfile.gateway` when it's missing. (Note: slice 5
|
||||||
builds + launches the bundle container; wiring its per-bottle,
|
builds + launches the bundle container; wiring its per-bottle,
|
||||||
source-IP-keyed config is a later slice — see PRD 0070.)"""
|
source-IP-keyed config is a later slice — see PRD 0070.)"""
|
||||||
|
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ names + the published port).
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import os
|
||||||
import time
|
import time
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.request
|
import urllib.request
|
||||||
@@ -25,11 +26,19 @@ from pathlib import Path
|
|||||||
from .. import log
|
from .. import log
|
||||||
from ..docker_cmd import run_docker
|
from ..docker_cmd import run_docker
|
||||||
from ..paths import bot_bottle_root
|
from ..paths import bot_bottle_root
|
||||||
from .gateway import GATEWAY_IMAGE, GATEWAY_NETWORK, DockerGateway
|
from .gateway import GATEWAY_IMAGE, GATEWAY_NETWORK, DockerGateway, GatewayError
|
||||||
|
|
||||||
DEFAULT_PORT = 8099
|
DEFAULT_PORT = 8099
|
||||||
ORCHESTRATOR_NAME = "bot-bottle-orchestrator"
|
ORCHESTRATOR_NAME = "bot-bottle-orchestrator"
|
||||||
ORCHESTRATOR_LABEL = "bot-bottle-orchestrator=1"
|
ORCHESTRATOR_LABEL = "bot-bottle-orchestrator=1"
|
||||||
|
# The control-plane's own runtime image — lean (python + the stdlib-only
|
||||||
|
# `bot_bottle` package, bind-mounted at run time), distinct from the heavy
|
||||||
|
# gateway data-plane image it used to borrow (#384). Env override for
|
||||||
|
# operators pinning a published build.
|
||||||
|
ORCHESTRATOR_IMAGE = os.environ.get(
|
||||||
|
"BOT_BOTTLE_ORCHESTRATOR_IMAGE", "bot-bottle-orchestrator:latest"
|
||||||
|
)
|
||||||
|
ORCHESTRATOR_DOCKERFILE = "Dockerfile.orchestrator"
|
||||||
# Baked onto the container as a label so `ensure_running` can tell whether the
|
# Baked onto the container as a label so `ensure_running` can tell whether the
|
||||||
# running process is executing the *current* bind-mounted source — see
|
# running process is executing the *current* bind-mounted source — see
|
||||||
# `_source_hash`.
|
# `_source_hash`.
|
||||||
@@ -37,7 +46,7 @@ ORCHESTRATOR_SOURCE_HASH_LABEL = "bot-bottle-orchestrator-source-hash"
|
|||||||
|
|
||||||
# The repo root is bind-mounted into the control-plane container so
|
# The repo root is bind-mounted into the control-plane container so
|
||||||
# `python -m bot_bottle.orchestrator` resolves the package (the orchestrator
|
# `python -m bot_bottle.orchestrator` resolves the package (the orchestrator
|
||||||
# is stdlib-only, so the bundle image's python is enough).
|
# is stdlib-only, so the lean orchestrator image's python is enough).
|
||||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||||
_APP_DIR = "/app"
|
_APP_DIR = "/app"
|
||||||
_ROOT_IN_CONTAINER = "/bot-bottle-root"
|
_ROOT_IN_CONTAINER = "/bot-bottle-root"
|
||||||
@@ -76,13 +85,19 @@ class OrchestratorService:
|
|||||||
*,
|
*,
|
||||||
port: int = DEFAULT_PORT,
|
port: int = DEFAULT_PORT,
|
||||||
network: str = GATEWAY_NETWORK,
|
network: str = GATEWAY_NETWORK,
|
||||||
image: str = GATEWAY_IMAGE,
|
image: str = ORCHESTRATOR_IMAGE,
|
||||||
|
gateway_image: str = GATEWAY_IMAGE,
|
||||||
repo_root: Path = _REPO_ROOT,
|
repo_root: Path = _REPO_ROOT,
|
||||||
host_root: Path | None = None,
|
host_root: Path | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self.port = port
|
self.port = port
|
||||||
self.network = network
|
self.network = network
|
||||||
|
# Two distinct images (#384): `image` is the lean control-plane
|
||||||
|
# runtime this container runs; `_gateway_image` is the heavy egress /
|
||||||
|
# git-gate / supervise data plane the gateway container runs. They
|
||||||
|
# were one conflated image before the split.
|
||||||
self.image = image
|
self.image = image
|
||||||
|
self._gateway_image = gateway_image
|
||||||
self._repo_root = repo_root
|
self._repo_root = repo_root
|
||||||
self._host_root = host_root or bot_bottle_root()
|
self._host_root = host_root or bot_bottle_root()
|
||||||
|
|
||||||
@@ -141,7 +156,29 @@ class OrchestratorService:
|
|||||||
)
|
)
|
||||||
|
|
||||||
def _gateway(self) -> DockerGateway:
|
def _gateway(self) -> DockerGateway:
|
||||||
return DockerGateway(self.image, network=self.network, orchestrator_url=self.internal_url)
|
return DockerGateway(
|
||||||
|
self._gateway_image, network=self.network, orchestrator_url=self.internal_url
|
||||||
|
)
|
||||||
|
|
||||||
|
def _ensure_orchestrator_image(self) -> None:
|
||||||
|
"""Build the lean control-plane image from `Dockerfile.orchestrator`
|
||||||
|
when it's missing (#384). Cheap — a `FROM python:*-slim` base with no
|
||||||
|
deps to install, so the layer cache makes rebuilds a no-op. Unlike the
|
||||||
|
gateway image this is build-if-missing, not build-every-time: the
|
||||||
|
control plane bind-mounts its source, so a code change is caught by the
|
||||||
|
source-hash recreate (below), not by an image rebuild."""
|
||||||
|
if run_docker(["docker", "image", "inspect", self.image]).returncode == 0:
|
||||||
|
return
|
||||||
|
argv = ["docker", "build", "-t", self.image,
|
||||||
|
"-f", str(self._repo_root / ORCHESTRATOR_DOCKERFILE),
|
||||||
|
str(self._repo_root)]
|
||||||
|
if os.environ.get("BOT_BOTTLE_NO_CACHE"):
|
||||||
|
argv.insert(2, "--no-cache")
|
||||||
|
proc = run_docker(argv)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise GatewayError(
|
||||||
|
f"orchestrator image build failed: {proc.stderr.strip()}"
|
||||||
|
)
|
||||||
|
|
||||||
def _orchestrator_source_current(self, current_hash: str) -> bool:
|
def _orchestrator_source_current(self, current_hash: str) -> bool:
|
||||||
"""True iff the running orchestrator container was created from the
|
"""True iff the running orchestrator container was created from the
|
||||||
@@ -181,6 +218,7 @@ class OrchestratorService:
|
|||||||
if self.is_healthy() and self._orchestrator_source_current(current_hash):
|
if self.is_healthy() and self._orchestrator_source_current(current_hash):
|
||||||
return self.url
|
return self.url
|
||||||
|
|
||||||
|
self._ensure_orchestrator_image()
|
||||||
log.info("starting orchestrator container", context={"name": ORCHESTRATOR_NAME})
|
log.info("starting orchestrator container", context={"name": ORCHESTRATOR_NAME})
|
||||||
self._run_orchestrator_container(current_hash)
|
self._run_orchestrator_container(current_hash)
|
||||||
|
|
||||||
@@ -204,6 +242,7 @@ __all__ = [
|
|||||||
"OrchestratorService",
|
"OrchestratorService",
|
||||||
"OrchestratorStartError",
|
"OrchestratorStartError",
|
||||||
"ORCHESTRATOR_NAME",
|
"ORCHESTRATOR_NAME",
|
||||||
|
"ORCHESTRATOR_IMAGE",
|
||||||
"DEFAULT_PORT",
|
"DEFAULT_PORT",
|
||||||
"DEFAULT_STARTUP_TIMEOUT_SECONDS",
|
"DEFAULT_STARTUP_TIMEOUT_SECONDS",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Per-bottle sidecar supervisor (PRD 0024 chunk 1).
|
"""Per-bottle sidecar supervisor (PRD 0024 chunk 1).
|
||||||
|
|
||||||
PID 1 inside the `bot-bottle-sidecars` bundle image. Spawns
|
PID 1 inside the `bot-bottle-gateway` data-plane image. Spawns
|
||||||
the configured daemons (egress, git-gate, supervise),
|
the configured daemons (egress, git-gate, supervise),
|
||||||
forwards SIGTERM/SIGINT to each child, and propagates per-daemon
|
forwards SIGTERM/SIGINT to each child, and propagates per-daemon
|
||||||
stdout+stderr to the container log with a `[name] ` prefix.
|
stdout+stderr to the container log with a `[name] ` prefix.
|
||||||
|
|||||||
@@ -50,7 +50,7 @@ from dataclasses import dataclass, replace
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
# Same-directory imports inside the bundle container; these files are
|
# Same-directory imports inside the bundle container; these files are
|
||||||
# COPYed flat under /app by Dockerfile.sidecars.
|
# COPYed flat under /app by Dockerfile.gateway.
|
||||||
from egress_addon_core import LOG_OFF, load_config
|
from egress_addon_core import LOG_OFF, load_config
|
||||||
from policy_resolver import PolicyResolveError, PolicyResolver
|
from policy_resolver import PolicyResolveError, PolicyResolver
|
||||||
import supervise as _sv
|
import supervise as _sv
|
||||||
|
|||||||
+1
-1
@@ -48,7 +48,7 @@ Discovery is invoked with `-t .` (top-level dir = repo root) so the
|
|||||||
the bottle's runtime, and creates zero Docker resources.
|
the bottle's runtime, and creates zero Docker resources.
|
||||||
- `test_orphan_cleanup.py` — `network_remove` is idempotent against
|
- `test_orphan_cleanup.py` — `network_remove` is idempotent against
|
||||||
missing resources, so the EXIT trap can call it unconditionally.
|
missing resources, so the EXIT trap can call it unconditionally.
|
||||||
- `test_sidecar_bundle_image.py` — builds Dockerfile.sidecars and
|
- `test_sidecar_bundle_image.py` — builds Dockerfile.gateway and
|
||||||
probes that gitleaks / mitmdump / supervise are all reachable
|
probes that gitleaks / mitmdump / supervise are all reachable
|
||||||
inside the bundle.
|
inside the bundle.
|
||||||
- `test_sidecar_bundle_compose.py` — end-to-end compose-up of an
|
- `test_sidecar_bundle_compose.py` — end-to-end compose-up of an
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ IMAGE = "busybox"
|
|||||||
class TestDockerGatewayImageExists(unittest.TestCase):
|
class TestDockerGatewayImageExists(unittest.TestCase):
|
||||||
def test_image_exists_true_for_present_false_for_absent(self) -> None:
|
def test_image_exists_true_for_present_false_for_absent(self) -> None:
|
||||||
# Ensure the tiny image is present (build_if_missing is disabled here
|
# Ensure the tiny image is present (build_if_missing is disabled here
|
||||||
# so this never triggers a Dockerfile.sidecars build).
|
# so this never triggers a Dockerfile.gateway build).
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
["docker", "pull", IMAGE],
|
["docker", "pull", IMAGE],
|
||||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ config (routes.yaml, etc) — that lands in chunk 2 when the
|
|||||||
renderer wires the bundle into compose. What we verify here is
|
renderer wires the bundle into compose. What we verify here is
|
||||||
the chunk-1 contract:
|
the chunk-1 contract:
|
||||||
|
|
||||||
- Dockerfile.sidecars builds (multi-stage works, base layers
|
- Dockerfile.gateway builds (multi-stage works, base layers
|
||||||
pull, COPYs resolve).
|
pull, COPYs resolve).
|
||||||
- gitleaks, mitmdump are at the documented paths and answer
|
- gitleaks, mitmdump are at the documented paths and answer
|
||||||
`--version`.
|
`--version`.
|
||||||
@@ -28,8 +28,8 @@ import unittest
|
|||||||
from tests._docker import skip_unless_docker
|
from tests._docker import skip_unless_docker
|
||||||
|
|
||||||
|
|
||||||
_IMAGE = "bot-bottle-sidecars-test:chunk1"
|
_IMAGE = "bot-bottle-gateway-test:chunk1"
|
||||||
_DOCKERFILE = "Dockerfile.sidecars"
|
_DOCKERFILE = "Dockerfile.gateway"
|
||||||
|
|
||||||
|
|
||||||
@skip_unless_docker()
|
@skip_unless_docker()
|
||||||
|
|||||||
@@ -297,8 +297,8 @@ class TestSidecarBundleShape(unittest.TestCase):
|
|||||||
|
|
||||||
def test_bundle_uses_bundle_image_and_dockerfile(self):
|
def test_bundle_uses_bundle_image_and_dockerfile(self):
|
||||||
sc = self._render()["services"]["sidecars"]
|
sc = self._render()["services"]["sidecars"]
|
||||||
self.assertEqual("bot-bottle-sidecars:latest", sc["image"])
|
self.assertEqual("bot-bottle-gateway:latest", sc["image"])
|
||||||
self.assertEqual("Dockerfile.sidecars", sc["build"]["dockerfile"])
|
self.assertEqual("Dockerfile.gateway", sc["build"]["dockerfile"])
|
||||||
|
|
||||||
def test_bundle_container_name_uses_sidecars_prefix(self):
|
def test_bundle_container_name_uses_sidecars_prefix(self):
|
||||||
sc = self._render()["services"]["sidecars"]
|
sc = self._render()["services"]["sidecars"]
|
||||||
|
|||||||
@@ -88,14 +88,14 @@ resolver #2
|
|||||||
"BOT_BOTTLE_MACOS_CONTAINER_DNS": "9.9.9.9",
|
"BOT_BOTTLE_MACOS_CONTAINER_DNS": "9.9.9.9",
|
||||||
}):
|
}):
|
||||||
util.build_image(
|
util.build_image(
|
||||||
"bot-bottle-sidecars:latest",
|
"bot-bottle-gateway:latest",
|
||||||
"/repo",
|
"/repo",
|
||||||
dockerfile="Dockerfile.sidecars",
|
dockerfile="Dockerfile.gateway",
|
||||||
)
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
[
|
[
|
||||||
"container", "build", "-t", "bot-bottle-sidecars:latest",
|
"container", "build", "-t", "bot-bottle-gateway:latest",
|
||||||
"--dns", "9.9.9.9", "-f", "/repo/Dockerfile.sidecars", "/repo",
|
"--dns", "9.9.9.9", "-f", "/repo/Dockerfile.gateway", "/repo",
|
||||||
],
|
],
|
||||||
run.call_args_list[-1].args[0],
|
run.call_args_list[-1].args[0],
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ def _proc(returncode: int = 0, stdout: str = "", stderr: str = "") -> Mock:
|
|||||||
|
|
||||||
class TestDockerGateway(unittest.TestCase):
|
class TestDockerGateway(unittest.TestCase):
|
||||||
def setUp(self) -> None:
|
def setUp(self) -> None:
|
||||||
self.sc = DockerGateway("bot-bottle-sidecars:latest")
|
self.sc = DockerGateway("bot-bottle-gateway:latest")
|
||||||
|
|
||||||
def test_default_name(self) -> None:
|
def test_default_name(self) -> None:
|
||||||
self.assertEqual(GATEWAY_NAME, self.sc.name)
|
self.assertEqual(GATEWAY_NAME, self.sc.name)
|
||||||
@@ -86,7 +86,7 @@ class TestDockerGateway(unittest.TestCase):
|
|||||||
runs = [c for c in calls if c[:2] == ["docker", "run"]]
|
runs = [c for c in calls if c[:2] == ["docker", "run"]]
|
||||||
self.assertEqual(1, len(runs))
|
self.assertEqual(1, len(runs))
|
||||||
self.assertIn(self.sc.name, runs[0])
|
self.assertIn(self.sc.name, runs[0])
|
||||||
self.assertIn("bot-bottle-sidecars:latest", runs[0])
|
self.assertIn("bot-bottle-gateway:latest", runs[0])
|
||||||
# Runs on the shared gateway network so agents can reach it by IP.
|
# Runs on the shared gateway network so agents can reach it by IP.
|
||||||
self.assertEqual(self.sc.network, runs[0][runs[0].index("--network") + 1])
|
self.assertEqual(self.sc.network, runs[0][runs[0].index("--network") + 1])
|
||||||
# Persists its CA on a named volume so agents keep trusting it.
|
# Persists its CA on a named volume so agents keep trusting it.
|
||||||
@@ -183,7 +183,7 @@ class TestDockerGatewayBuild(unittest.TestCase):
|
|||||||
builds = [c for c in calls if c[:2] == ["docker", "build"]]
|
builds = [c for c in calls if c[:2] == ["docker", "build"]]
|
||||||
self.assertEqual(1, len(builds))
|
self.assertEqual(1, len(builds))
|
||||||
self.assertIn(self.sc.image_ref, builds[0])
|
self.assertIn(self.sc.image_ref, builds[0])
|
||||||
self.assertTrue(any(a.endswith("Dockerfile.sidecars") for a in builds[0]))
|
self.assertTrue(any(a.endswith("Dockerfile.gateway") for a in builds[0]))
|
||||||
self.assertNotIn("--no-cache", builds[0])
|
self.assertNotIn("--no-cache", builds[0])
|
||||||
|
|
||||||
def test_ensure_built_no_cache_env_forces_full_rebuild(self) -> None:
|
def test_ensure_built_no_cache_env_forces_full_rebuild(self) -> None:
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from pathlib import Path
|
|||||||
from unittest.mock import MagicMock, Mock, patch
|
from unittest.mock import MagicMock, Mock, patch
|
||||||
|
|
||||||
from bot_bottle.orchestrator.lifecycle import (
|
from bot_bottle.orchestrator.lifecycle import (
|
||||||
|
ORCHESTRATOR_IMAGE,
|
||||||
ORCHESTRATOR_NAME,
|
ORCHESTRATOR_NAME,
|
||||||
ORCHESTRATOR_SOURCE_HASH_LABEL,
|
ORCHESTRATOR_SOURCE_HASH_LABEL,
|
||||||
OrchestratorService,
|
OrchestratorService,
|
||||||
@@ -121,6 +122,45 @@ class TestOrchestratorService(unittest.TestCase):
|
|||||||
self.assertIn("bot_bottle.orchestrator", argv)
|
self.assertIn("bot_bottle.orchestrator", argv)
|
||||||
self.assertEqual("127.0.0.1:8099:8099", argv[argv.index("--publish") + 1])
|
self.assertEqual("127.0.0.1:8099:8099", argv[argv.index("--publish") + 1])
|
||||||
|
|
||||||
|
def test_ensure_running_builds_lean_orchestrator_image_when_missing(self) -> None:
|
||||||
|
# The control plane runs its own lean image (#384), distinct from the
|
||||||
|
# gateway data plane — built from Dockerfile.orchestrator when absent.
|
||||||
|
calls: list[list[str]] = []
|
||||||
|
|
||||||
|
def fake(argv: list[str]) -> Mock:
|
||||||
|
calls.append(argv)
|
||||||
|
if argv[:2] == ["docker", "ps"]:
|
||||||
|
return _proc(stdout="") # orchestrator not running
|
||||||
|
if argv[:3] == ["docker", "image", "inspect"]:
|
||||||
|
return _proc(returncode=1) # image absent -> build
|
||||||
|
return _proc()
|
||||||
|
|
||||||
|
with patch(_URLOPEN, side_effect=[urllib.error.URLError("down"), _health(200)]), \
|
||||||
|
patch(_GATEWAY), patch(_RUN, side_effect=fake), patch(_SLEEP):
|
||||||
|
self.svc.ensure_running()
|
||||||
|
builds = [c for c in calls if c[:2] == ["docker", "build"]]
|
||||||
|
self.assertEqual(1, len(builds))
|
||||||
|
self.assertIn(ORCHESTRATOR_IMAGE, builds[0])
|
||||||
|
self.assertTrue(any(a.endswith("Dockerfile.orchestrator") for a in builds[0]))
|
||||||
|
# It is NOT the gateway image/dockerfile — the split is the point.
|
||||||
|
self.assertFalse(any("Dockerfile.gateway" in a for a in builds[0]))
|
||||||
|
|
||||||
|
def test_ensure_running_skips_orchestrator_image_build_when_present(self) -> None:
|
||||||
|
calls: list[list[str]] = []
|
||||||
|
|
||||||
|
def fake(argv: list[str]) -> Mock:
|
||||||
|
calls.append(argv)
|
||||||
|
if argv[:2] == ["docker", "ps"]:
|
||||||
|
return _proc(stdout="")
|
||||||
|
if argv[:3] == ["docker", "image", "inspect"]:
|
||||||
|
return _proc(returncode=0) # image present -> no build
|
||||||
|
return _proc()
|
||||||
|
|
||||||
|
with patch(_URLOPEN, side_effect=[urllib.error.URLError("down"), _health(200)]), \
|
||||||
|
patch(_GATEWAY), patch(_RUN, side_effect=fake), patch(_SLEEP):
|
||||||
|
self.svc.ensure_running()
|
||||||
|
self.assertEqual([], [c for c in calls if c[:2] == ["docker", "build"]])
|
||||||
|
|
||||||
def test_ensure_running_raises_on_timeout(self) -> None:
|
def test_ensure_running_raises_on_timeout(self) -> None:
|
||||||
with patch(_URLOPEN, side_effect=urllib.error.URLError("down")), \
|
with patch(_URLOPEN, side_effect=urllib.error.URLError("down")), \
|
||||||
patch(_GATEWAY), patch(_RUN, return_value=Mock(returncode=0, stderr="")), \
|
patch(_GATEWAY), patch(_RUN, return_value=Mock(returncode=0, stderr="")), \
|
||||||
|
|||||||
Reference in New Issue
Block a user