Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4cf57f55bb | |||
| 25113fae92 | |||
| a2edaf8694 | |||
| 14f19247c0 | |||
| 300b878288 | |||
| cbd92347d3 | |||
| 23e794f273 | |||
| f31be349ef | |||
| 4ba26b8813 | |||
| f600180861 |
@@ -9,9 +9,12 @@
|
||||
# Keeping the content in one place means future orchestrator deps (e.g.
|
||||
# iroh) are added here once, not duplicated per backend.
|
||||
#
|
||||
# It stays deliberately lean: only the pinned FastAPI/Uvicorn control-plane
|
||||
# stack is installed here — none of the gateway's mitmproxy/git/gitleaks
|
||||
# (that's Dockerfile.gateway) and no buildah (that's firecracker-only).
|
||||
# It stays deliberately lean: the control plane is **stdlib-only** today, so
|
||||
# no third-party payload — none of the gateway's mitmproxy/git/gitleaks
|
||||
# (that's Dockerfile.gateway) and no buildah (that's the firecracker
|
||||
# builder, and lives only in Dockerfile.orchestrator.fc). Keeping the
|
||||
# secret-dense control plane on a minimal dependency surface is the point
|
||||
# (PRD 0070's "secret concentration").
|
||||
#
|
||||
# Shares an exact multi-architecture Python/trixie manifest with the gateway
|
||||
# image. The version-qualified tag keeps the human-readable upstream version;
|
||||
@@ -22,11 +25,6 @@ FROM ${PYTHON_BASE_IMAGE}
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.orchestrator.lock /tmp/requirements.orchestrator.lock
|
||||
RUN pip install --no-cache-dir --require-hashes \
|
||||
-r /tmp/requirements.orchestrator.lock \
|
||||
&& rm /tmp/requirements.orchestrator.lock
|
||||
|
||||
# The orchestrator content. Baked so the image is self-contained (runs from
|
||||
# a built image, no runtime bind-mount); the docker backend may still
|
||||
# bind-mount /app for dev live-reload, which simply overlays this copy.
|
||||
|
||||
@@ -182,7 +182,9 @@ BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
|
||||
|
||||
## Manifest
|
||||
|
||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Bottles live in `~/.bot-bottle/bottles/`; agents may also be shipped by a repo at `<repo>/.bot-bottle/agents/<name>.md`.
|
||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Both bottles and agents are **home-only**: they live under `~/.bot-bottle/bottles/` and `~/.bot-bottle/agents/`. A `<repo>/.bot-bottle/agents/<name>.md` shipped by a workspace is ignored with a warning — since an agent may select a host identity and forge secret, checked-out content must not define one (PRD 0082). Keep repo-specific behavioral instructions in `AGENTS.md` instead.
|
||||
|
||||
Identity is **agent-owned**: the author name/email and named forge accounts live on the agent, not under `git-gate` (which now carries only Git transport policy). A bottle repo may optionally name one of the selected agent's forge aliases via `forge:`.
|
||||
|
||||
**Bottle** (`~/.bot-bottle/bottles/gitea-dev.md`):
|
||||
|
||||
@@ -190,37 +192,19 @@ Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle
|
||||
---
|
||||
extends: claude # inherit the Claude provider boundary
|
||||
|
||||
env:
|
||||
GIT_AUTHOR_NAME: didericis
|
||||
|
||||
git:
|
||||
user:
|
||||
name: "Eric Bauerfeld"
|
||||
email: "eric+claude@dideric.is"
|
||||
remotes:
|
||||
gitea.dideric.is:
|
||||
Name: bot-bottle
|
||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
||||
KnownHostKey: ssh-ed25519 AAAA...
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
inspect:
|
||||
auth:
|
||||
scheme: token # Bearer | token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
matches: # optional — restrict to specific paths/methods/headers
|
||||
- paths:
|
||||
- {type: prefix, value: /api/v1/}
|
||||
methods: [GET, POST, PATCH, DELETE]
|
||||
outbound_detectors: [token_patterns, known_secrets]
|
||||
inbound_detectors: false # disable response scanning for this host
|
||||
git-gate:
|
||||
repos:
|
||||
bot-bottle:
|
||||
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||
key:
|
||||
provider: gitea
|
||||
forge_token_env: GITEA_DEPLOY_TOKEN # deploy-key admin (push), PRD 0048
|
||||
host_key: "ssh-ed25519 AAAA..."
|
||||
forge: didericis-gitea # ← selects the agent's forge alias
|
||||
---
|
||||
|
||||
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
||||
gitea over SSH for push, token over HTTPS for the API.
|
||||
The `gitea-dev` bottle. Gitea over SSH for push; the API credential and
|
||||
workflow guidance come from the agent's `forge: didericis-gitea` association.
|
||||
````
|
||||
|
||||
**Agent** (`~/.bot-bottle/agents/gitea-helper.md`):
|
||||
@@ -230,11 +214,27 @@ gitea over SSH for push, token over HTTPS for the API.
|
||||
bottle: gitea-dev
|
||||
skills:
|
||||
- init-prd
|
||||
author:
|
||||
name: didericis-claude
|
||||
email: eric+claude@dideric.is
|
||||
forge-accounts:
|
||||
didericis-gitea:
|
||||
url: https://gitea.dideric.is/api/v1
|
||||
auth:
|
||||
type: token
|
||||
token_secret: GITEA_CLAUDE_TOKEN # host env var; value never enters the bottle
|
||||
---
|
||||
|
||||
You help maintain Gitea-hosted projects.
|
||||
````
|
||||
|
||||
`author` populates the bottle's `git config user.name/user.email`. When a
|
||||
selected repo names a `forge` alias, bot-bottle resolves the alias's
|
||||
`token_secret` from the host env into the egress proxy only (never the bottle),
|
||||
adds a scoped, proxy-authenticated route to the Gitea API origin, and appends
|
||||
non-secret forge workflow guidance to the agent's system prompt. Neither the
|
||||
token value nor its `token_secret` name appears in the bottle env or prompt.
|
||||
|
||||
**Egress route fields:**
|
||||
|
||||
| Field | Required | Description |
|
||||
|
||||
@@ -30,7 +30,6 @@ if TYPE_CHECKING:
|
||||
BottleImages,
|
||||
BottlePlan,
|
||||
BottleSpec,
|
||||
EnumerationError,
|
||||
ExecResult,
|
||||
)
|
||||
from .selection import (
|
||||
@@ -60,7 +59,6 @@ _LAZY_MODULES: dict[str, str] = {
|
||||
"BottleImages": "base",
|
||||
"BottleBackend": "base",
|
||||
"BackendStatus": "base",
|
||||
"EnumerationError": "base",
|
||||
"get_bottle_backend": "selection",
|
||||
"known_backend_names": "selection",
|
||||
"has_backend": "selection",
|
||||
@@ -102,7 +100,6 @@ __all__ = [
|
||||
"BottlePlan",
|
||||
"BottleSpec",
|
||||
"ExecResult",
|
||||
"EnumerationError",
|
||||
"CommitCancelled",
|
||||
"Freezer",
|
||||
"get_freezer",
|
||||
|
||||
@@ -42,10 +42,6 @@ class BackendStatus(enum.IntEnum):
|
||||
READY = 0
|
||||
|
||||
|
||||
class EnumerationError(RuntimeError):
|
||||
"""A backend could not produce an authoritative live-resource snapshot."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BottleSpec:
|
||||
"""CLI-supplied intent. Backend-agnostic — each backend's prepare
|
||||
@@ -172,10 +168,6 @@ class BottleCleanupPlan(ABC):
|
||||
"""True iff there is nothing to clean up; the CLI uses this to
|
||||
short-circuit before showing the y/N."""
|
||||
|
||||
@abstractmethod
|
||||
def intersect(self, current: "BottleCleanupPlan") -> "BottleCleanupPlan":
|
||||
"""Resources both displayed to the operator and currently removable."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExecResult:
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
"""Shared destructive-cleanup execution and failure accounting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from collections.abc import Sequence
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class CleanupError(RuntimeError):
|
||||
"""One or more approved cleanup mutations did not complete."""
|
||||
|
||||
|
||||
class CleanupFailures:
|
||||
"""Attempt every approved mutation, then fail with complete diagnostics."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._messages: list[str] = []
|
||||
|
||||
def run(self, argv: Sequence[str], description: str) -> None:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
list(argv), capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
self._messages.append(f"{description}: {exc}")
|
||||
return
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
self._messages.append(
|
||||
f"{description}: {detail or f'exit {result.returncode}'}"
|
||||
)
|
||||
|
||||
def remove_tree(self, path: Path, description: str) -> None:
|
||||
try:
|
||||
shutil.rmtree(path)
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except OSError as exc:
|
||||
self._messages.append(f"{description}: {exc}")
|
||||
|
||||
def raise_if_any(self) -> None:
|
||||
if self._messages:
|
||||
raise CleanupError("; ".join(self._messages))
|
||||
|
||||
|
||||
__all__ = ["CleanupError", "CleanupFailures"]
|
||||
@@ -46,22 +46,6 @@ class DockerBottleCleanupPlan(BottleCleanupPlan):
|
||||
and not self.orphan_state_dirs
|
||||
)
|
||||
|
||||
def intersect(self, current: BottleCleanupPlan) -> "DockerBottleCleanupPlan":
|
||||
if not isinstance(current, DockerBottleCleanupPlan):
|
||||
raise TypeError("cleanup plans must have the same backend type")
|
||||
return DockerBottleCleanupPlan(
|
||||
projects=tuple(x for x in self.projects if x in current.projects),
|
||||
stray_containers=tuple(
|
||||
x for x in self.stray_containers if x in current.stray_containers
|
||||
),
|
||||
stray_networks=tuple(
|
||||
x for x in self.stray_networks if x in current.stray_networks
|
||||
),
|
||||
orphan_state_dirs=tuple(
|
||||
x for x in self.orphan_state_dirs if x in current.orphan_state_dirs
|
||||
),
|
||||
)
|
||||
|
||||
def print(self) -> None:
|
||||
print(file=sys.stderr)
|
||||
for name in self.projects:
|
||||
|
||||
@@ -23,12 +23,11 @@ Active-agent enumeration lives in `backend/docker/enumerate.py`.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
from ...paths import bot_bottle_root
|
||||
from ...log import info
|
||||
from .. import EnumerationError
|
||||
from ..cleanup_control import CleanupFailures
|
||||
from ...log import info, warn
|
||||
from . import util as docker_mod
|
||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from ...bottle_state import bottle_state_dir, is_preserved
|
||||
@@ -37,17 +36,15 @@ from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
|
||||
|
||||
def _list_prefixed_containers() -> list[str]:
|
||||
"""All bot-bottle-prefixed containers, running or stopped."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "-a",
|
||||
"--filter", f"name=^{COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Names}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
raise EnumerationError(f"docker ps failed: {exc}") from exc
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "-a",
|
||||
"--filter", f"name=^{COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Names}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise EnumerationError(f"docker ps failed: {result.stderr.strip()}")
|
||||
warn(f"docker ps failed: {result.stderr.strip()}")
|
||||
return []
|
||||
out: list[str] = []
|
||||
for line in (result.stdout or "").splitlines():
|
||||
if not line:
|
||||
@@ -66,19 +63,15 @@ def _list_prefixed_networks() -> list[str]:
|
||||
to a compose project. Compose-managed networks have a
|
||||
`com.docker.compose.project` label; bare ones (from pre-compose
|
||||
code paths) don't."""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "network", "ls",
|
||||
"--filter", f"name={COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Name}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
raise EnumerationError(f"docker network ls failed: {exc}") from exc
|
||||
result = subprocess.run(
|
||||
["docker", "network", "ls",
|
||||
"--filter", f"name={COMPOSE_PROJECT_PREFIX}",
|
||||
"--format", "{{.Name}}\t{{.Label \"com.docker.compose.project\"}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise EnumerationError(
|
||||
f"docker network ls failed: {result.stderr.strip()}"
|
||||
)
|
||||
warn(f"docker network ls failed: {result.stderr.strip()}")
|
||||
return []
|
||||
out: list[str] = []
|
||||
for line in (result.stdout or "").splitlines():
|
||||
if not line:
|
||||
@@ -127,10 +120,7 @@ def prepare_cleanup() -> DockerBottleCleanupPlan:
|
||||
`enumerate_active_agents()` so the orphan-state-dir bucket
|
||||
doesn't include slugs whose non-docker bottle is still up."""
|
||||
docker_mod.require_docker()
|
||||
projects = list_compose_projects(
|
||||
warn_on_error=False,
|
||||
raise_on_error=True,
|
||||
)
|
||||
projects = list_compose_projects()
|
||||
project_set = set(projects)
|
||||
# Late import to avoid a circular at module-load time —
|
||||
# the backend package's __init__ imports this module.
|
||||
@@ -150,30 +140,40 @@ def cleanup(plan: DockerBottleCleanupPlan) -> None:
|
||||
"""Remove everything in the plan. Projects first (whose `compose
|
||||
down` reaps their containers + networks atomically), then stray
|
||||
legacy resources, then orphan state dirs."""
|
||||
failures = CleanupFailures()
|
||||
for project in plan.projects:
|
||||
info(f"docker compose down ({project})")
|
||||
failures.run(
|
||||
result = subprocess.run(
|
||||
["docker", "compose", "-p", project, "down", "--volumes"],
|
||||
f"docker compose down failed for {project}",
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
warn(
|
||||
f"compose down failed for {project}: "
|
||||
f"{result.stderr.strip()}"
|
||||
)
|
||||
|
||||
for name in plan.stray_containers:
|
||||
info(f"removing stray container {name}")
|
||||
failures.run(
|
||||
subprocess.run(
|
||||
["docker", "rm", "-f", name],
|
||||
f"removing stray container {name}",
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
for name in plan.stray_networks:
|
||||
info(f"removing stray network {name}")
|
||||
failures.run(
|
||||
subprocess.run(
|
||||
["docker", "network", "rm", name],
|
||||
f"removing stray network {name}",
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
for identity in plan.orphan_state_dirs:
|
||||
path = bottle_state_dir(identity)
|
||||
info(f"removing orphan state dir {path}")
|
||||
failures.remove_tree(path, f"removing orphan state dir {path}")
|
||||
failures.raise_if_any()
|
||||
try:
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
except OSError as e:
|
||||
warn(f"failed to remove {path}: {e}")
|
||||
|
||||
@@ -16,7 +16,6 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ...log import die, warn
|
||||
from ..base import EnumerationError
|
||||
|
||||
|
||||
# --- Lifecycle helpers (PRD 0018 chunk 3) ----------------------------------
|
||||
@@ -53,20 +52,19 @@ def slug_from_compose_project(project: str) -> str:
|
||||
|
||||
|
||||
def list_compose_projects(
|
||||
*,
|
||||
include_stopped: bool = True,
|
||||
warn_on_error: bool = True,
|
||||
raise_on_error: bool = False,
|
||||
*, include_stopped: bool = True, warn_on_error: bool = True,
|
||||
) -> list[str]:
|
||||
"""All compose project names starting with `bot-bottle-`.
|
||||
`include_stopped=True` (default) runs `docker compose ls --all`
|
||||
so exited projects appear too; pass False to get only projects
|
||||
with at least one running container.
|
||||
|
||||
Best-effort callers get ``[]`` on Docker errors or malformed output.
|
||||
Enumeration callers pass ``raise_on_error=True`` so a failed query is not
|
||||
reported as an authoritative empty result.
|
||||
"""
|
||||
Returns [] on docker daemon errors or malformed output rather
|
||||
than raising — callers should treat the empty list as "no
|
||||
projects discoverable", not "no projects exist". `warn_on_error`
|
||||
stays true for explicit operator commands like cleanup, but active
|
||||
discovery paths set it false so dashboard refreshes don't spam
|
||||
stderr while Docker Desktop is stopped."""
|
||||
argv = ["docker", "compose", "ls", "--format", "json"]
|
||||
if include_stopped:
|
||||
argv.insert(3, "--all")
|
||||
@@ -74,27 +72,19 @@ def list_compose_projects(
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True, check=False,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
if raise_on_error:
|
||||
raise EnumerationError(
|
||||
"docker compose ls failed: docker not found"
|
||||
) from exc
|
||||
except FileNotFoundError:
|
||||
# docker binary not on PATH — same shape as a daemon-down
|
||||
# error from the caller's POV: no projects discoverable.
|
||||
return []
|
||||
if result.returncode != 0:
|
||||
message = f"docker compose ls failed: {result.stderr.strip()}"
|
||||
if raise_on_error:
|
||||
raise EnumerationError(message)
|
||||
if warn_on_error:
|
||||
warn(message)
|
||||
warn(f"docker compose ls failed: {result.stderr.strip()}")
|
||||
return []
|
||||
try:
|
||||
projects = json.loads(result.stdout or "[]")
|
||||
except json.JSONDecodeError as e:
|
||||
message = f"docker compose ls returned malformed JSON: {e}"
|
||||
if raise_on_error:
|
||||
raise EnumerationError(message) from e
|
||||
if warn_on_error:
|
||||
warn(message)
|
||||
warn(f"docker compose ls returned malformed JSON: {e}")
|
||||
return []
|
||||
names: list[str] = []
|
||||
for p in projects:
|
||||
@@ -107,10 +97,7 @@ def list_compose_projects(
|
||||
|
||||
|
||||
def list_active_slugs(
|
||||
*,
|
||||
include_stopped: bool = False,
|
||||
warn_on_error: bool = True,
|
||||
raise_on_error: bool = False,
|
||||
*, include_stopped: bool = False, warn_on_error: bool = True,
|
||||
) -> list[str]:
|
||||
"""Slugs (project name minus prefix) of currently-running
|
||||
bottles. Used by the dashboard's operator-edit verbs to choose
|
||||
@@ -121,7 +108,6 @@ def list_active_slugs(
|
||||
for p in list_compose_projects(
|
||||
include_stopped=include_stopped,
|
||||
warn_on_error=warn_on_error,
|
||||
raise_on_error=raise_on_error,
|
||||
)
|
||||
) if slug
|
||||
)
|
||||
|
||||
@@ -68,11 +68,6 @@ def _network_container_ips(network: str) -> list[str]:
|
||||
"docker", "network", "inspect", "--format",
|
||||
"{{range .Containers}}{{.IPv4Address}} {{end}}", network,
|
||||
])
|
||||
if proc.returncode != 0:
|
||||
detail = proc.stderr.strip() or f"exit {proc.returncode}"
|
||||
raise ConsolidatedLaunchError(
|
||||
f"could not inspect addresses on gateway network {network}: {detail}"
|
||||
)
|
||||
ips: list[str] = []
|
||||
for entry in proc.stdout.split():
|
||||
ips.append(entry.split("/", 1)[0])
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""Active-agent enumeration for the docker backend.
|
||||
|
||||
Returns `ActiveAgent` records the CLI `active` command and the
|
||||
dashboard agents pane consume. Docker query failures raise rather
|
||||
than masquerading as an authoritative empty result.
|
||||
dashboard agents pane consume. Empty when docker isn't reachable
|
||||
— gated by `has_backend('docker')` at the cross-backend caller
|
||||
so this module trusts that docker is available when called.
|
||||
|
||||
The parser (`_parse_services_by_project`) is exposed for direct
|
||||
unit testing; the docker `docker ps` invocation is in
|
||||
@@ -12,18 +13,17 @@ from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from .. import ActiveAgent, EnumerationError
|
||||
from .. import ActiveAgent
|
||||
from ...bottle_state import read_metadata
|
||||
from .compose import compose_project_name, list_active_slugs
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
"""All currently-running docker-backed agents."""
|
||||
slugs = list_active_slugs(
|
||||
include_stopped=False,
|
||||
warn_on_error=False,
|
||||
raise_on_error=True,
|
||||
)
|
||||
"""All currently-running docker-backed agents. Caller is
|
||||
responsible for gating on `has_backend('docker')` if it
|
||||
matters; if docker is missing the `docker ps` call below
|
||||
returns an empty list silently."""
|
||||
slugs = list_active_slugs(include_stopped=False, warn_on_error=False)
|
||||
if not slugs:
|
||||
return []
|
||||
services_by_project = _query_services_by_project()
|
||||
@@ -74,8 +74,8 @@ def _query_services_by_project() -> dict[str, set[str]]:
|
||||
],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
raise EnumerationError("docker ps failed: docker not found") from exc
|
||||
except FileNotFoundError:
|
||||
return {}
|
||||
if r.returncode != 0:
|
||||
raise EnumerationError(f"docker ps failed: {r.stderr.strip()}")
|
||||
return {}
|
||||
return _parse_services_by_project(r.stdout or "")
|
||||
|
||||
@@ -27,11 +27,3 @@ class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not (self.vm_pids or self.run_dirs)
|
||||
|
||||
def intersect(self, current: BottleCleanupPlan) -> "FirecrackerBottleCleanupPlan":
|
||||
if not isinstance(current, FirecrackerBottleCleanupPlan):
|
||||
raise TypeError("cleanup plans must have the same backend type")
|
||||
return FirecrackerBottleCleanupPlan(
|
||||
vm_pids=tuple(x for x in self.vm_pids if x in current.vm_pids),
|
||||
run_dirs=tuple(x for x in self.run_dirs if x in current.run_dirs),
|
||||
)
|
||||
|
||||
@@ -11,9 +11,10 @@ Reaps *orphans* only — resources with no live VM behind them:
|
||||
— a VMM left lingering after its dir was removed.
|
||||
|
||||
A run dir with a *live* firecracker process is a running bottle and is
|
||||
left strictly alone: it is neither killed nor removed. Active-agent
|
||||
enumeration uses this same process snapshot, so cleanup and generic
|
||||
backend consumers agree about which bottles are running.
|
||||
left strictly alone: it is neither killed nor removed. (The backend's
|
||||
`enumerate_active` registry is still a stub — #354 — so a live process
|
||||
is the only reliable "this bottle is in use" signal we have. Once the
|
||||
registry lands, registry-orphaned-but-running VMs can be reaped too.)
|
||||
|
||||
TAP slots free themselves (the flock drops when the launcher exits), so
|
||||
there is nothing to reclaim there.
|
||||
@@ -21,16 +22,14 @@ there is nothing to reclaim there.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import info
|
||||
from .. import EnumerationError
|
||||
from ..cleanup_control import CleanupFailures
|
||||
from . import lifecycle_lock, util
|
||||
from . import util
|
||||
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
|
||||
|
||||
|
||||
@@ -38,7 +37,7 @@ def _run_root() -> Path:
|
||||
return util.cache_dir() / "run"
|
||||
|
||||
|
||||
def _run_dir_of(args: Sequence[str], run_root: Path) -> Path | None:
|
||||
def _run_dir_of(cmd: str, run_root: Path) -> Path | None:
|
||||
"""The bottle run dir a firecracker cmdline belongs to, or None.
|
||||
|
||||
A bottle VM is launched with `--config-file <run_root>/<slug>/config.json`,
|
||||
@@ -46,35 +45,15 @@ def _run_dir_of(args: Sequence[str], run_root: Path) -> Path | None:
|
||||
the run root. Anything else (a builder VM, the infra VM elsewhere) is
|
||||
not ours to reap here.
|
||||
"""
|
||||
for i, arg in enumerate(args):
|
||||
if arg == "--config-file" and i + 1 < len(args):
|
||||
parent = Path(args[i + 1]).parent
|
||||
toks = cmd.split()
|
||||
for i, tok in enumerate(toks):
|
||||
if tok == "--config-file" and i + 1 < len(toks):
|
||||
parent = Path(toks[i + 1]).parent
|
||||
if parent.parent == run_root:
|
||||
return parent
|
||||
return None
|
||||
|
||||
|
||||
def _decode_cmdline(raw: bytes) -> tuple[str, ...]:
|
||||
"""Decode Linux's NUL-delimited argv without losing embedded spaces."""
|
||||
return tuple(
|
||||
value.decode(errors="surrogateescape")
|
||||
for value in raw.split(b"\0") if value
|
||||
)
|
||||
|
||||
|
||||
def _process_args(pid: int) -> tuple[str, ...] | None:
|
||||
"""Read one process's lossless argv, or None when it exited meanwhile."""
|
||||
try:
|
||||
raw = Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not inspect Firecracker pid {pid}: {exc}"
|
||||
) from exc
|
||||
return _decode_cmdline(raw)
|
||||
|
||||
|
||||
def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
|
||||
"""Inspect running firecracker VMs under ``run_root``.
|
||||
|
||||
@@ -83,34 +62,23 @@ def _scan_processes(run_root: Path) -> tuple[set[str], list[int]]:
|
||||
* ``orphan_pids`` — firecracker pids whose run dir no longer exists
|
||||
(a lingering VMM to kill).
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pgrep", "firecracker"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not enumerate Firecracker processes: {exc}"
|
||||
) from exc
|
||||
if result.returncode == 1:
|
||||
# pgrep's documented "no processes matched" result.
|
||||
return set(), []
|
||||
result = subprocess.run(
|
||||
["pgrep", "-a", "firecracker"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or "").strip() or f"exit {result.returncode}"
|
||||
raise EnumerationError(
|
||||
f"could not enumerate Firecracker processes: {detail}"
|
||||
)
|
||||
return set(), []
|
||||
live: set[str] = set()
|
||||
orphan_pids: list[int] = []
|
||||
for line in result.stdout.splitlines():
|
||||
parts = line.split(None, 1)
|
||||
if len(parts) != 2:
|
||||
continue
|
||||
try:
|
||||
pid = int(line.strip())
|
||||
pid = int(parts[0])
|
||||
except ValueError:
|
||||
continue
|
||||
args = _process_args(pid)
|
||||
if args is None:
|
||||
continue
|
||||
run_dir = _run_dir_of(args, run_root)
|
||||
run_dir = _run_dir_of(parts[1], run_root)
|
||||
if run_dir is None:
|
||||
continue
|
||||
if run_dir.is_dir():
|
||||
@@ -146,44 +114,12 @@ def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
|
||||
|
||||
|
||||
def cleanup(plan: FirecrackerBottleCleanupPlan) -> None:
|
||||
"""Revalidate the preview under the launch lock, then remove its survivors."""
|
||||
with lifecycle_lock.hold():
|
||||
fresh = prepare_cleanup()
|
||||
approved_pids = set(plan.vm_pids).intersection(fresh.vm_pids)
|
||||
approved_dirs = set(plan.run_dirs).intersection(fresh.run_dirs)
|
||||
failures = CleanupFailures()
|
||||
for pid in sorted(approved_pids):
|
||||
_terminate_orphan(pid, _run_root())
|
||||
for path in sorted(approved_dirs):
|
||||
info(f"rm -rf {path}")
|
||||
failures.remove_tree(Path(path), f"removing Firecracker run dir {path}")
|
||||
failures.raise_if_any()
|
||||
|
||||
|
||||
def _terminate_orphan(pid: int, run_root: Path) -> None:
|
||||
"""Signal exactly the process identity that still owns an orphan config."""
|
||||
try:
|
||||
pidfd = os.pidfd_open(pid)
|
||||
except ProcessLookupError:
|
||||
return
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not pin Firecracker pid {pid} for cleanup: {exc}"
|
||||
) from exc
|
||||
try:
|
||||
try:
|
||||
raw = Path(f"/proc/{pid}/cmdline").read_bytes()
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except OSError as exc:
|
||||
raise EnumerationError(
|
||||
f"could not revalidate Firecracker pid {pid}: {exc}"
|
||||
) from exc
|
||||
args = _decode_cmdline(raw)
|
||||
run_dir = _run_dir_of(args, run_root)
|
||||
if run_dir is None or run_dir.is_dir():
|
||||
return
|
||||
for pid in plan.vm_pids:
|
||||
info(f"kill firecracker VM pid {pid}")
|
||||
signal.pidfd_send_signal(pidfd, signal.SIGTERM)
|
||||
finally:
|
||||
os.close(pidfd)
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
for path in plan.run_dirs:
|
||||
info(f"rm -rf {path}")
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
|
||||
@@ -1,32 +1,14 @@
|
||||
"""Active-agent enumeration for the Firecracker backend.
|
||||
|
||||
Running bottles are the Firecracker processes whose ``--config-file`` points
|
||||
at an existing per-bottle run directory. The same authoritative process scan
|
||||
protects cleanup from deleting live VMs; operational scan failures propagate
|
||||
as ``EnumerationError`` instead of masquerading as an empty host.
|
||||
The backend is disabled during the companion-container removal (#385) — it can't
|
||||
launch bottles, so there are none to enumerate. Real enumeration returns
|
||||
with the backend's consolidated relaunch (#354).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ...bottle_state import read_metadata
|
||||
from .. import ActiveAgent
|
||||
from .cleanup import live_run_dirs
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
out: list[ActiveAgent] = []
|
||||
for run_dir in live_run_dirs():
|
||||
slug = run_dir.name
|
||||
metadata = read_metadata(slug)
|
||||
out.append(ActiveAgent(
|
||||
backend_name="firecracker",
|
||||
slug=slug,
|
||||
agent_name=metadata.agent_name if metadata else "?",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
# Firecracker uses the shared gateway, so there are no
|
||||
# per-bottle gateway service containers to report.
|
||||
services=(),
|
||||
label=metadata.label if metadata else "",
|
||||
color=metadata.color if metadata else "",
|
||||
))
|
||||
return out
|
||||
return []
|
||||
|
||||
@@ -41,8 +41,6 @@ from ... import resources
|
||||
from ...log import die, info
|
||||
from . import util
|
||||
|
||||
ARTIFACT_HTTP_TIMEOUT_SECONDS = 30.0
|
||||
|
||||
# Bump if the on-disk artifact *format* changes (compression, layout) so a new
|
||||
# scheme can't collide with a cached/published artifact of the old one.
|
||||
_ARTIFACT_FORMAT = "1"
|
||||
@@ -56,7 +54,6 @@ _BUILD_INPUTS = {
|
||||
"image-build-args.json",
|
||||
"Dockerfile.orchestrator",
|
||||
"Dockerfile.orchestrator.fc",
|
||||
"requirements.orchestrator.lock",
|
||||
),
|
||||
"gateway": (
|
||||
"image-build-args.json",
|
||||
@@ -166,9 +163,7 @@ def _download(url: str, dest: Path) -> None:
|
||||
"""Stream `url` to `dest` (atomic via a `.part` sibling)."""
|
||||
tmp = dest.with_suffix(dest.suffix + ".part")
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
_open(url), timeout=ARTIFACT_HTTP_TIMEOUT_SECONDS,
|
||||
) as resp, open(tmp, "wb") as out:
|
||||
with urllib.request.urlopen(_open(url)) as resp, open(tmp, "wb") as out:
|
||||
shutil.copyfileobj(resp, out, _CHUNK)
|
||||
except urllib.error.HTTPError as e:
|
||||
tmp.unlink(missing_ok=True)
|
||||
|
||||
@@ -46,7 +46,7 @@ from ...log import die, info, warn
|
||||
from ...supervisor.types import SUPERVISE_PORT
|
||||
from ..docker.egress import EGRESS_PORT
|
||||
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
||||
from . import firecracker_vm, image_builder, isolation_probe, lifecycle_lock, netpool, util
|
||||
from . import firecracker_vm, image_builder, isolation_probe, netpool, util
|
||||
from .bottle import FirecrackerBottle
|
||||
from .bottle_plan import FirecrackerBottlePlan
|
||||
from ...orchestrator.store.config_store import resolve_teardown_timeout
|
||||
@@ -164,29 +164,25 @@ def launch(
|
||||
)
|
||||
|
||||
# Step 6: build the per-bottle rootfs + SSH key, then boot.
|
||||
# Cleanup takes the same lock while refreshing its process snapshot.
|
||||
# Hold it until the VMM exists so a newly-created run dir can never be
|
||||
# mistaken for an orphan in the build-before-boot window.
|
||||
with lifecycle_lock.hold():
|
||||
run_dir = util.cache_dir() / "run" / plan.slug
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
|
||||
# doesn't leak. Registered before vm.terminate below so it runs
|
||||
# *after* it (ExitStack is LIFO).
|
||||
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
|
||||
rootfs = run_dir / "rootfs.ext4"
|
||||
util.build_rootfs_ext4(agent_base, rootfs)
|
||||
private_key, pubkey = util.generate_keypair(run_dir)
|
||||
run_dir = util.cache_dir() / "run" / plan.slug
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
# Remove the run dir on teardown so the per-bottle rootfs.ext4 (~1G)
|
||||
# doesn't leak. Registered before vm.terminate below so it runs *after*
|
||||
# it (ExitStack is LIFO): the VM is gone before we rm its rootfs.
|
||||
stack.callback(lambda: shutil.rmtree(run_dir, ignore_errors=True))
|
||||
rootfs = run_dir / "rootfs.ext4"
|
||||
util.build_rootfs_ext4(agent_base, rootfs)
|
||||
private_key, pubkey = util.generate_keypair(run_dir)
|
||||
|
||||
vm = firecracker_vm.boot(
|
||||
name=plan.container_name,
|
||||
rootfs=rootfs,
|
||||
tap=slot.iface,
|
||||
guest_ip=slot.guest_ip,
|
||||
host_ip=slot.host_ip,
|
||||
pubkey=pubkey,
|
||||
run_dir=run_dir,
|
||||
)
|
||||
vm = firecracker_vm.boot(
|
||||
name=plan.container_name,
|
||||
rootfs=rootfs,
|
||||
tap=slot.iface,
|
||||
guest_ip=slot.guest_ip,
|
||||
host_ip=slot.host_ip,
|
||||
pubkey=pubkey,
|
||||
run_dir=run_dir,
|
||||
)
|
||||
stack.callback(vm.terminate)
|
||||
firecracker_vm.wait_for_ssh(vm, private_key)
|
||||
persist_env_var_secret(private_key, slot.guest_ip, ctx.env_var_secret)
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
"""Serialize Firecracker run-directory creation with orphan cleanup."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator
|
||||
|
||||
from . import util
|
||||
|
||||
|
||||
def _lock_path() -> Path:
|
||||
return util.cache_dir() / "run.lifecycle.lock"
|
||||
|
||||
|
||||
@contextmanager
|
||||
def hold() -> Generator[None]:
|
||||
"""Exclude cleanup while a launch directory lacks a visible VMM."""
|
||||
path = _lock_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("a", encoding="utf-8") as handle:
|
||||
fcntl.flock(handle, fcntl.LOCK_EX)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
fcntl.flock(handle, fcntl.LOCK_UN)
|
||||
|
||||
|
||||
__all__ = ["hold"]
|
||||
@@ -34,7 +34,6 @@ from pathlib import Path
|
||||
from . import infra_artifact, infra_vm, util
|
||||
|
||||
_CHUNK = 1 << 20
|
||||
_REGISTRY_HTTP_TIMEOUT_SECONDS = 30.0
|
||||
|
||||
_GZ_NAME = "rootfs.ext4.gz"
|
||||
_SHA_NAME = "rootfs.ext4.gz.sha256"
|
||||
@@ -92,9 +91,7 @@ def _put(url: str, body: "bytes | Path", token: str) -> None:
|
||||
req.add_header("Authorization", f"token {token}")
|
||||
req.add_header("Content-Type", "application/octet-stream")
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
req, timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
) as resp:
|
||||
with urllib.request.urlopen(req) as resp:
|
||||
print(f" uploaded {url} (HTTP {resp.status})")
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 409:
|
||||
@@ -115,9 +112,7 @@ def _delete(url: str, token: str) -> None:
|
||||
if token:
|
||||
req.add_header("Authorization", f"token {token}")
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
req, timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
):
|
||||
with urllib.request.urlopen(req):
|
||||
pass
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code != 404:
|
||||
@@ -156,10 +151,7 @@ def _try_download_published(role: str, role_dir: Path) -> str | None:
|
||||
version = _role_version(role)
|
||||
sha_url = infra_artifact.artifact_url(version, _SHA_NAME, role=role)
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
infra_artifact._open(sha_url),
|
||||
timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
):
|
||||
with urllib.request.urlopen(infra_artifact._open(sha_url)):
|
||||
pass
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
@@ -203,10 +195,7 @@ def _publish_bundle(role: str, role_dir: Path, token: str) -> str:
|
||||
# present, a re-publish is a no-op. Otherwise clear any partial upload left
|
||||
# by an interrupted prior attempt and upload the complete set.
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
infra_artifact._open(sha_url),
|
||||
timeout=_REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
) as resp:
|
||||
with urllib.request.urlopen(infra_artifact._open(sha_url)) as resp:
|
||||
remote_sha = resp.read().decode("utf-8").split()[0].strip().lower()
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code != 404:
|
||||
|
||||
@@ -25,11 +25,3 @@ class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not self.containers and not self.networks
|
||||
|
||||
def intersect(self, current: BottleCleanupPlan) -> "MacosContainerBottleCleanupPlan":
|
||||
if not isinstance(current, MacosContainerBottleCleanupPlan):
|
||||
raise TypeError("cleanup plans must have the same backend type")
|
||||
return MacosContainerBottleCleanupPlan(
|
||||
containers=tuple(x for x in self.containers if x in current.containers),
|
||||
networks=tuple(x for x in self.networks if x in current.networks),
|
||||
)
|
||||
|
||||
@@ -4,9 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from .. import EnumerationError
|
||||
from ..cleanup_control import CleanupFailures
|
||||
from ...log import info
|
||||
from ...log import info, warn
|
||||
from . import util as container_mod
|
||||
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
|
||||
|
||||
@@ -21,8 +19,8 @@ def _list_prefixed_containers() -> list[str]:
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
detail = result.stderr.strip() or f"exit {result.returncode}"
|
||||
raise EnumerationError(f"container list failed: {detail}")
|
||||
warn(f"container list failed: {result.stderr.strip()}")
|
||||
return []
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX)
|
||||
@@ -37,8 +35,7 @@ def _list_prefixed_networks() -> list[str]:
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
detail = result.stderr.strip() or f"exit {result.returncode}"
|
||||
raise EnumerationError(f"container network list failed: {detail}")
|
||||
return []
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX)
|
||||
@@ -54,17 +51,19 @@ def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
|
||||
|
||||
|
||||
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
|
||||
failures = CleanupFailures()
|
||||
for name in plan.containers:
|
||||
info(f"container delete --force {name}")
|
||||
failures.run(
|
||||
subprocess.run(
|
||||
["container", "delete", "--force", name],
|
||||
f"deleting container {name}",
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
for name in plan.networks:
|
||||
info(f"container network delete {name}")
|
||||
failures.run(
|
||||
subprocess.run(
|
||||
["container", "network", "delete", name],
|
||||
f"deleting network {name}",
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
failures.raise_if_any()
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import subprocess
|
||||
|
||||
from ...bottle_state import read_metadata
|
||||
from .. import ActiveAgent, EnumerationError
|
||||
from .. import ActiveAgent
|
||||
from .infra import INFRA_NAME, ORCHESTRATOR_NAME
|
||||
|
||||
# The name every agent container carries: `bot-bottle-<slug>`. Exported
|
||||
@@ -20,18 +20,17 @@ CONTAINER_NAME_PREFIX = "bot-bottle-"
|
||||
_INFRA_NAMES = frozenset({INFRA_NAME, ORCHESTRATOR_NAME})
|
||||
|
||||
|
||||
class EnumerationError(RuntimeError):
|
||||
"""container list failed; the resulting live set is not authoritative."""
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["container", "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
raise EnumerationError(
|
||||
"container list failed: container CLI not found"
|
||||
) from exc
|
||||
result = subprocess.run(
|
||||
["container", "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise EnumerationError(
|
||||
f"container list failed: "
|
||||
|
||||
@@ -118,7 +118,7 @@ class BottlePreparationPlanner:
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=provision,
|
||||
egress_plan=prepare_egress(bottle, slug, provision),
|
||||
egress_plan=prepare_egress(manifest, slug, provision),
|
||||
git_gate_plan=prepare_git_gate(bottle, slug),
|
||||
supervise_plan=prepare_supervise(bottle, slug),
|
||||
)
|
||||
|
||||
@@ -24,10 +24,11 @@ from ..bottle_state import (
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from ..egress import Egress, EgressPlan
|
||||
from ..egress import Egress, EgressPlan, egress_forge_routes
|
||||
from ..git_gate import GitGate, GitGatePlan
|
||||
from ..log import die
|
||||
from ..manifest import Manifest, ManifestBottle
|
||||
from ..manifest.forge import render_forge_guidance
|
||||
from ..supervisor.plan import SupervisePlan
|
||||
from ..orchestrator.supervisor import Supervisor
|
||||
from ..util import slugify
|
||||
@@ -71,12 +72,21 @@ def write_launch_metadata(
|
||||
|
||||
def prepare_agent_state_dir(slug: str, manifest: Manifest) -> tuple[Path, Path]:
|
||||
"""Create the agent state subdir, write the prompt file.
|
||||
Returns (agent_dir, prompt_file)."""
|
||||
Returns (agent_dir, prompt_file).
|
||||
|
||||
For repositories associated with a forge, appends generated, non-secret
|
||||
provider-specific workflow guidance to the prompt (PRD
|
||||
0082). The guidance carries neither the
|
||||
token value nor its `token_secret` name."""
|
||||
agent = manifest.agent
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
prompt_file.write_text(agent.prompt or "")
|
||||
prompt = agent.prompt or ""
|
||||
guidance = render_forge_guidance(manifest.forge_associations)
|
||||
if guidance:
|
||||
prompt = f"{prompt.rstrip()}\n\n{guidance}" if prompt.strip() else guidance
|
||||
prompt_file.write_text(prompt)
|
||||
prompt_file.chmod(0o600)
|
||||
return agent_dir, prompt_file
|
||||
|
||||
@@ -88,11 +98,18 @@ def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
|
||||
|
||||
|
||||
def prepare_egress(
|
||||
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
|
||||
manifest: Manifest, slug: str, provision: AgentProvisionPlan,
|
||||
) -> EgressPlan:
|
||||
"""Build the egress plan, adding a scoped, proxy-held Gitea API route for
|
||||
each forge alias referenced by a selected git-gate repo (PRD
|
||||
0082). The token is resolved from the host
|
||||
env at launch and never enters the bottle."""
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
|
||||
forge_routes = egress_forge_routes(manifest.forge_associations)
|
||||
return Egress().prepare(
|
||||
manifest.bottle, slug, egress_dir, provision.egress_routes, forge_routes,
|
||||
)
|
||||
|
||||
|
||||
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
|
||||
|
||||
@@ -22,7 +22,6 @@ from __future__ import annotations
|
||||
import sys
|
||||
|
||||
from ...backend import get_bottle_backend, has_backend, known_backend_names
|
||||
from ...backend.cleanup_control import CleanupError
|
||||
from ...log import info
|
||||
from ...util import read_tty_line
|
||||
|
||||
@@ -53,20 +52,10 @@ def cmd_cleanup(_argv: list[str]) -> int:
|
||||
info("cleanup: skipped")
|
||||
return 0
|
||||
|
||||
# Confirmation authorizes a fresh authoritative snapshot, not blind use of
|
||||
# identities that may have changed while the operator reviewed the preview.
|
||||
failures: list[str] = []
|
||||
for name, backend, displayed in prepared:
|
||||
current = backend.prepare_cleanup()
|
||||
approved = displayed.intersect(current)
|
||||
if approved.empty:
|
||||
for name, backend, plan in prepared:
|
||||
if plan.empty:
|
||||
continue
|
||||
try:
|
||||
backend.cleanup(approved)
|
||||
except CleanupError as exc:
|
||||
failures.append(f"{name}: {exc}")
|
||||
if failures:
|
||||
raise CleanupError("cleanup incomplete: " + "; ".join(failures))
|
||||
backend.cleanup(plan)
|
||||
info("cleanup: done")
|
||||
return 0
|
||||
|
||||
|
||||
@@ -128,7 +128,7 @@ def cmd_start(argv: list[str]) -> int:
|
||||
if not manifest.all_agent_names:
|
||||
print(
|
||||
"bot-bottle: no agents defined. "
|
||||
"Add an agent to ~/.bot-bottle/agents/ or ./bot-bottle/agents/ to get started.",
|
||||
"Add an agent to ~/.bot-bottle/agents/ to get started.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
@@ -383,12 +383,9 @@ def _peek_agent_bottle(manifest: ManifestIndex, agent_name: str) -> str:
|
||||
from ...manifest.loader import scan_agent_names
|
||||
from ...yaml_subset import YamlSubsetError, parse_frontmatter
|
||||
|
||||
# Agents are home-only (PRD 0082).
|
||||
home_agents = scan_agent_names(manifest.home_md / "agents")
|
||||
cwd_agents: dict[str, Path] = {}
|
||||
if manifest.cwd_md is not None:
|
||||
cwd_agents = scan_agent_names(manifest.cwd_md / "agents")
|
||||
merged = {**home_agents, **cwd_agents}
|
||||
path = merged.get(agent_name)
|
||||
path = home_agents.get(agent_name)
|
||||
if path is None:
|
||||
return ""
|
||||
try:
|
||||
@@ -488,13 +485,19 @@ def _manifest_to_yaml(manifest: Manifest) -> str:
|
||||
lines.append(" skills:")
|
||||
for s in agent.skills:
|
||||
lines.append(f" - {s}")
|
||||
if not agent.git_user.is_empty():
|
||||
lines.append(" git-gate:")
|
||||
lines.append(" user:")
|
||||
if agent.git_user.name:
|
||||
lines.append(f" name: {agent.git_user.name}")
|
||||
if agent.git_user.email:
|
||||
lines.append(f" email: {agent.git_user.email}")
|
||||
if agent.author is not None:
|
||||
lines.append(" author:")
|
||||
lines.append(f" name: {agent.author.name}")
|
||||
lines.append(f" email: {agent.author.email}")
|
||||
if agent.forge_accounts:
|
||||
lines.append(" forge-accounts:")
|
||||
for alias, acct in sorted(agent.forge_accounts.items()):
|
||||
lines.append(f" {alias}:")
|
||||
lines.append(f" url: {acct.url}")
|
||||
lines.append(" auth:")
|
||||
lines.append(f" type: {acct.auth_type}")
|
||||
# token_secret name is host config; show the name, never a value.
|
||||
lines.append(f" token_secret: {acct.token_secret}")
|
||||
|
||||
bottle = manifest.bottle
|
||||
lines.append("bottle:")
|
||||
@@ -510,20 +513,14 @@ def _manifest_to_yaml(manifest: Manifest) -> str:
|
||||
for k, v in sorted(bottle.env.items()):
|
||||
lines.append(f" {k}: {v}")
|
||||
|
||||
has_git_gate = not bottle.git_user.is_empty() or bottle.git
|
||||
if has_git_gate:
|
||||
if bottle.git:
|
||||
lines.append(" git-gate:")
|
||||
if not bottle.git_user.is_empty():
|
||||
lines.append(" user:")
|
||||
if bottle.git_user.name:
|
||||
lines.append(f" name: {bottle.git_user.name}")
|
||||
if bottle.git_user.email:
|
||||
lines.append(f" email: {bottle.git_user.email}")
|
||||
if bottle.git:
|
||||
lines.append(" repos:")
|
||||
for entry in bottle.git:
|
||||
lines.append(f" {entry.Name}:")
|
||||
lines.append(f" url: {entry.Upstream}")
|
||||
lines.append(" repos:")
|
||||
for entry in bottle.git:
|
||||
lines.append(f" {entry.Name}:")
|
||||
lines.append(f" url: {entry.Upstream}")
|
||||
if entry.Forge:
|
||||
lines.append(f" forge: {entry.Forge}")
|
||||
|
||||
if bottle.egress.routes:
|
||||
lines.append(" egress:")
|
||||
|
||||
@@ -32,6 +32,7 @@ if TYPE_CHECKING:
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
Egress,
|
||||
egress_agent_env_entries,
|
||||
egress_forge_routes,
|
||||
egress_gateway_env_entries,
|
||||
egress_manifest_routes,
|
||||
egress_render_routes,
|
||||
@@ -51,6 +52,7 @@ _LAZY: dict[str, str] = {
|
||||
"EGRESS_ROUTES_FILENAME": ".service",
|
||||
"EGRESS_ROUTES_IN_CONTAINER": ".service",
|
||||
"egress_agent_env_entries": ".service",
|
||||
"egress_forge_routes": ".service",
|
||||
"egress_gateway_env_entries": ".service",
|
||||
"egress_manifest_routes": ".service",
|
||||
"egress_render_routes": ".service",
|
||||
@@ -80,6 +82,7 @@ __all__ = [
|
||||
"Egress",
|
||||
"EgressPlan",
|
||||
"EgressRoute",
|
||||
"egress_forge_routes",
|
||||
"egress_manifest_routes",
|
||||
"egress_render_routes",
|
||||
"egress_resolve_token_values",
|
||||
|
||||
@@ -26,7 +26,7 @@ from ..log import die
|
||||
from .plan import EgressPlan, EgressRoute
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..manifest import ManifestBottle
|
||||
from ..manifest import ManifestBottle, ResolvedForgeAssociation
|
||||
|
||||
|
||||
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||
@@ -119,15 +119,61 @@ def egress_manifest_routes(
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def egress_forge_routes(
|
||||
associations: "tuple[ResolvedForgeAssociation, ...]",
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
"""Synthesize one inspected, token-authenticated egress route per distinct
|
||||
forge alias referenced by a selected git-gate repo (PRD
|
||||
0082).
|
||||
|
||||
The route is scoped to the canonical forge origin (host) and API prefix
|
||||
(`/api/v1`): the proxy injects the Gitea `token` scheme using the value of
|
||||
the host env var named by the account's `token_secret`, resolved at launch
|
||||
from the host environment — the token never enters the bottle. Aliases that
|
||||
canonicalize to the same host share a single route (deduplicated).
|
||||
|
||||
Composition (`resolve_forge_associations`) has already rejected referenced
|
||||
aliases that share a host but disagree on origin/auth/token_secret, so this
|
||||
host-dedup only ever collapses genuinely identical credentials — it never
|
||||
silently discards a distinct one."""
|
||||
out: list[EgressRoute] = []
|
||||
seen_hosts: set[str] = set()
|
||||
for assoc in associations:
|
||||
acct = assoc.account
|
||||
host_key = acct.host.lower()
|
||||
if host_key in seen_hosts:
|
||||
continue
|
||||
seen_hosts.add(host_key)
|
||||
out.append(EgressRoute(
|
||||
host=acct.host,
|
||||
matches=(CoreMatchEntry(
|
||||
paths=(CorePathMatch(type="prefix", value=acct.api_prefix),),
|
||||
),),
|
||||
auth_scheme=acct.auth_type,
|
||||
token_ref=acct.token_secret,
|
||||
inspect=True,
|
||||
))
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def egress_routes_for_bottle(
|
||||
bottle: ManifestBottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
forge_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
manifest = egress_manifest_routes(bottle)
|
||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
||||
merged = list(_default_provider_on_match(provider_routes)) + [
|
||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
||||
]
|
||||
# Provider routes (LLM API) default to redact-on-match; forge routes are
|
||||
# host-injected but keep the default DLP policy. Both take precedence over
|
||||
# a manifest route to the same host.
|
||||
reserved_hosts = (
|
||||
{pr.host.lower() for pr in provider_routes}
|
||||
| {fr.host.lower() for fr in forge_routes}
|
||||
)
|
||||
merged = (
|
||||
list(_default_provider_on_match(provider_routes))
|
||||
+ list(forge_routes)
|
||||
+ [r for r in manifest if r.host.lower() not in reserved_hosts]
|
||||
)
|
||||
return _assign_token_slots(merged)
|
||||
|
||||
|
||||
@@ -367,8 +413,9 @@ class Egress:
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
forge_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> EgressPlan:
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes, forge_routes)
|
||||
log = bottle.egress.Log
|
||||
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
|
||||
routes_path.write_text(egress_render_routes(routes, log=log))
|
||||
|
||||
@@ -136,18 +136,10 @@ def _pump(name: str, stream: IO[bytes]) -> None:
|
||||
"""Read lines from `stream`, prefix with `[name]`, write to
|
||||
stdout. Runs in its own thread per child; daemon=True so a
|
||||
blocked read doesn't keep the process alive after main exits."""
|
||||
try:
|
||||
for raw in iter(stream.readline, b""):
|
||||
line = raw.decode("utf-8", errors="replace").rstrip("\n")
|
||||
sys.stdout.write(f"[{name}] {line}\n")
|
||||
sys.stdout.flush()
|
||||
except (OSError, ValueError) as exc:
|
||||
# The manager closes a dead child's pipe after wait() and before a
|
||||
# restart. A pump can be between readline calls at that exact moment;
|
||||
# closed-stream errors are normal completion, not uncaught thread
|
||||
# failures. Preserve genuinely unexpected I/O diagnostics.
|
||||
if not stream.closed:
|
||||
_log(f"{name} output pump stopped: {type(exc).__name__}: {exc}")
|
||||
for raw in iter(stream.readline, b""):
|
||||
line = raw.decode("utf-8", errors="replace").rstrip("\n")
|
||||
sys.stdout.write(f"[{name}] {line}\n")
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
|
||||
|
||||
@@ -1,109 +0,0 @@
|
||||
"""Shared resource boundaries for gateway stdlib HTTP services."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import http.server
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Protocol
|
||||
|
||||
|
||||
class Readable(Protocol):
|
||||
def read(self, size: int = -1, /) -> bytes: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BodyReadError(Exception):
|
||||
status: int
|
||||
message: str
|
||||
|
||||
|
||||
def read_declared_body(
|
||||
stream: Readable,
|
||||
connection: socket.socket,
|
||||
raw_length: str | None,
|
||||
*,
|
||||
maximum: int,
|
||||
timeout_seconds: float,
|
||||
require_length: bool,
|
||||
) -> bytes:
|
||||
"""Validate and read exactly one declared body under a read deadline."""
|
||||
if raw_length is None:
|
||||
if require_length:
|
||||
raise BodyReadError(411, "Content-Length required")
|
||||
raw_length = "0"
|
||||
try:
|
||||
length = int(raw_length)
|
||||
except ValueError as exc:
|
||||
raise BodyReadError(400, "invalid Content-Length") from exc
|
||||
if length < 0:
|
||||
raise BodyReadError(400, "invalid Content-Length")
|
||||
if length > maximum:
|
||||
raise BodyReadError(413, "request body too large")
|
||||
previous_timeout = connection.gettimeout()
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
chunks: list[bytes] = []
|
||||
remaining = length
|
||||
try:
|
||||
while remaining:
|
||||
timeout = deadline - time.monotonic()
|
||||
if timeout <= 0:
|
||||
raise BodyReadError(408, "request body read timed out")
|
||||
connection.settimeout(timeout)
|
||||
chunk = stream.read(min(remaining, 64 * 1024))
|
||||
if not chunk:
|
||||
raise BodyReadError(400, "incomplete request body")
|
||||
chunks.append(chunk)
|
||||
remaining -= len(chunk)
|
||||
except TimeoutError as exc:
|
||||
raise BodyReadError(408, "request body read timed out") from exc
|
||||
finally:
|
||||
connection.settimeout(previous_timeout)
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
class BoundedThreadingHTTPServer(http.server.ThreadingHTTPServer):
|
||||
"""ThreadingHTTPServer with a hard cap on in-flight request threads."""
|
||||
|
||||
daemon_threads = True
|
||||
|
||||
def __init__( # pylint: disable=consider-using-with
|
||||
self, *args, max_workers: int = 32, **kwargs, # type: ignore[no-untyped-def]
|
||||
):
|
||||
if max_workers < 1:
|
||||
raise ValueError("max_workers must be positive")
|
||||
self._request_slots = threading.BoundedSemaphore(max_workers)
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
def process_request(
|
||||
self, request: Any, client_address: Any,
|
||||
) -> None:
|
||||
if not self._request_slots.acquire( # pylint: disable=consider-using-with
|
||||
blocking=False,
|
||||
):
|
||||
try:
|
||||
request.sendall(
|
||||
b"HTTP/1.1 503 Service Unavailable\r\n"
|
||||
b"Content-Length: 0\r\nConnection: close\r\n\r\n"
|
||||
)
|
||||
finally:
|
||||
self.shutdown_request(request)
|
||||
return
|
||||
try:
|
||||
super().process_request(request, client_address)
|
||||
except BaseException:
|
||||
self._request_slots.release()
|
||||
raise
|
||||
|
||||
def process_request_thread(
|
||||
self, request: Any, client_address: Any,
|
||||
) -> None:
|
||||
try:
|
||||
super().process_request_thread(request, client_address)
|
||||
finally:
|
||||
self._request_slots.release()
|
||||
|
||||
|
||||
__all__ = ["BodyReadError", "BoundedThreadingHTTPServer", "read_declared_body"]
|
||||
@@ -16,7 +16,7 @@ import typing
|
||||
from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
|
||||
|
||||
from bot_bottle.constants import IDENTITY_HEADER
|
||||
from bot_bottle.gateway.egress.dlp_detectors import redact_tokens
|
||||
from bot_bottle.gateway.egress.dlp_detectors import redact_tokens, strip_crlf
|
||||
from bot_bottle.gateway.egress.dlp_config import (
|
||||
DEFAULT_OUTBOUND_ON_MATCH,
|
||||
ON_MATCH_BLOCK,
|
||||
@@ -25,19 +25,19 @@ from bot_bottle.gateway.egress.dlp_config import (
|
||||
from bot_bottle.gateway.egress.context import resolve_client_context
|
||||
from bot_bottle.gateway.egress.dlp import (
|
||||
build_inbound_scan_text,
|
||||
build_outbound_scan_text,
|
||||
build_token_allow_payload,
|
||||
outbound_scan_headers,
|
||||
scan_inbound,
|
||||
scan_outbound,
|
||||
)
|
||||
from bot_bottle.gateway.egress.outbound_pipeline import redact_request, scan_request
|
||||
from bot_bottle.gateway.egress.matching import (
|
||||
decide,
|
||||
decide_git_fetch,
|
||||
is_git_fetch_request,
|
||||
is_git_push_request,
|
||||
match_route,
|
||||
)
|
||||
from bot_bottle.gateway.egress.request_pipeline import (
|
||||
evaluate_route_policy,
|
||||
git_block_reason,
|
||||
)
|
||||
from bot_bottle.gateway.egress.schema import route_to_yaml_dict
|
||||
from bot_bottle.gateway.egress.types import (
|
||||
LOG_BLOCKS,
|
||||
@@ -389,9 +389,19 @@ class EgressAddon:
|
||||
self._passthrough_conns.discard(conn_id)
|
||||
|
||||
async def request(self, flow: http.HTTPFlow) -> None:
|
||||
config, slug, env = self._request_context(flow)
|
||||
request_path, _, query = flow.request.path.partition("?")
|
||||
|
||||
# Reuse the context stashed by http_connect for HTTPS flows (one
|
||||
# orchestrator round-trip per connection). Plain-HTTP flows have no
|
||||
# prior CONNECT stash, so resolve now and stash for response/websocket.
|
||||
meta = getattr(flow, "metadata", None)
|
||||
if isinstance(meta, dict) and _FLOW_CTX_KEY in meta:
|
||||
config, slug, env = meta[_FLOW_CTX_KEY]
|
||||
self._request_token(flow) # strip identity headers; token already resolved
|
||||
else:
|
||||
config, slug, env = self._resolve_flow(flow)
|
||||
self._stash_flow_ctx(flow, config, slug, env)
|
||||
|
||||
# Introspection ("_egress.local/allowlist") reports the calling bottle's
|
||||
# own resolved routes — served after resolution so it reflects this
|
||||
# bottle's policy, not a stale global.
|
||||
@@ -412,66 +422,56 @@ class EgressAddon:
|
||||
# the path/query the git checks below rely on.
|
||||
request_path, _, query = flow.request.path.partition("?")
|
||||
|
||||
if not self._allow_git_request(flow, config, request_path, query):
|
||||
if is_git_push_request(request_path, query):
|
||||
self._block(
|
||||
flow,
|
||||
"egress: git push over HTTPS is not supported; "
|
||||
"use the bottle.git SSH path (gitleaks-scanned by "
|
||||
"git-gate's pre-receive hook).",
|
||||
ctx=self._req_ctx(flow),
|
||||
)
|
||||
return
|
||||
|
||||
self._apply_route_policy(flow, config, route, request_path, env)
|
||||
if is_git_fetch_request(request_path, query):
|
||||
git_decision = decide_git_fetch(
|
||||
config.routes, flow.request.pretty_host,
|
||||
)
|
||||
if git_decision.action == "block":
|
||||
self._block(
|
||||
flow,
|
||||
git_decision.reason,
|
||||
ctx=self._req_ctx(flow),
|
||||
)
|
||||
return
|
||||
|
||||
def _request_context(
|
||||
self, flow: http.HTTPFlow,
|
||||
) -> tuple[Config, str, "typing.Mapping[str, str]"]:
|
||||
"""Resolve one bottle context, reusing the HTTPS CONNECT snapshot."""
|
||||
meta = getattr(flow, "metadata", None)
|
||||
if isinstance(meta, dict) and _FLOW_CTX_KEY in meta:
|
||||
config, slug, env = meta[_FLOW_CTX_KEY]
|
||||
self._request_token(flow)
|
||||
return config, slug, env
|
||||
config, slug, env = self._resolve_flow(flow)
|
||||
self._stash_flow_ctx(flow, config, slug, env)
|
||||
return config, slug, env
|
||||
|
||||
def _allow_git_request(
|
||||
self, flow: http.HTTPFlow, config: Config,
|
||||
request_path: str, query: str,
|
||||
) -> bool:
|
||||
"""Apply the HTTPS Git push/fetch boundary before general routing."""
|
||||
reason = git_block_reason(
|
||||
config.routes, flow.request.pretty_host, request_path, query,
|
||||
)
|
||||
if not reason:
|
||||
return True
|
||||
self._block(flow, reason, ctx=self._req_ctx(flow))
|
||||
return False
|
||||
|
||||
def _apply_route_policy(
|
||||
self, flow: http.HTTPFlow, config: Config, route: Route | None,
|
||||
request_path: str, env: "typing.Mapping[str, str]",
|
||||
) -> None:
|
||||
"""Strip agent auth, evaluate the route, then inject gateway auth."""
|
||||
# Strip agent-set Authorization after DLP scan so smuggled tokens
|
||||
# are caught above; the route may inject gateway-owned auth below.
|
||||
# Routes with preserve_auth=True pass the header through as-is so the
|
||||
# agent's own credentials (e.g. registry bearer tokens) reach the upstream.
|
||||
result = evaluate_route_policy(
|
||||
config,
|
||||
route,
|
||||
host=flow.request.pretty_host,
|
||||
request_path=request_path,
|
||||
method=flow.request.method,
|
||||
headers=dict(flow.request.headers),
|
||||
env=env,
|
||||
)
|
||||
if result.strip_authorization:
|
||||
if route is None or not route.preserve_auth:
|
||||
flow.request.headers.pop("authorization", None)
|
||||
|
||||
if result.block_reason:
|
||||
self._block(flow, result.block_reason, ctx=self._req_ctx(flow))
|
||||
# Build headers mapping for match evaluation
|
||||
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
|
||||
|
||||
decision = decide(
|
||||
config.routes,
|
||||
flow.request.pretty_host,
|
||||
request_path,
|
||||
env,
|
||||
request_method=flow.request.method,
|
||||
request_headers=req_headers,
|
||||
deny_reason=config.deny_reason,
|
||||
)
|
||||
|
||||
if decision.action == "block":
|
||||
self._block(flow, decision.reason, ctx=self._req_ctx(flow))
|
||||
return
|
||||
|
||||
if result.inject_authorization is not None:
|
||||
flow.request.headers["authorization"] = result.inject_authorization
|
||||
if decision.inject_authorization is not None:
|
||||
flow.request.headers["authorization"] = decision.inject_authorization
|
||||
|
||||
if result.log_request:
|
||||
if config.log >= LOG_FULL:
|
||||
self._log_request(flow, env)
|
||||
|
||||
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
|
||||
@@ -495,12 +495,20 @@ class EgressAddon:
|
||||
Loops so the supervise policy can re-scan after each approval — a
|
||||
second, un-approved token in the same request is still caught."""
|
||||
while True:
|
||||
request_path, _, _ = flow.request.path.partition("?")
|
||||
result = scan_request(
|
||||
flow.request,
|
||||
route,
|
||||
env,
|
||||
safe_tokens=self._safe_tokens_for(slug),
|
||||
request_path, _, query = flow.request.path.partition("?")
|
||||
body = flow.request.get_text(strict=False) or ""
|
||||
headers = outbound_scan_headers(route, dict(flow.request.headers))
|
||||
scan_text = build_outbound_scan_text(
|
||||
flow.request.pretty_host, request_path, query, headers, body,
|
||||
)
|
||||
# CRLF is scanned only over the request line + headers, never the
|
||||
# body (see scan_outbound) — a body is not an injection vector.
|
||||
crlf_text = build_outbound_scan_text(
|
||||
flow.request.pretty_host, request_path, query, headers, "",
|
||||
)
|
||||
result = scan_outbound(
|
||||
route, scan_text, env,
|
||||
safe_tokens=self._safe_tokens_for(slug), crlf_text=crlf_text,
|
||||
)
|
||||
if result is None or result.severity != "block":
|
||||
return True
|
||||
@@ -510,7 +518,7 @@ class EgressAddon:
|
||||
# redact scrubs every detection (tokens and structural CRLF) and
|
||||
# forwards; it fails closed only if a match survives the scrub.
|
||||
if policy == ON_MATCH_REDACT:
|
||||
if redact_request(flow.request, route, env):
|
||||
if self._redact_outbound(flow, route, env):
|
||||
if self._flow_log(flow) >= LOG_BLOCKS:
|
||||
sys.stderr.write(json.dumps({
|
||||
"event": "egress_redacted",
|
||||
@@ -543,6 +551,41 @@ class EgressAddon:
|
||||
return False # _supervise_token_block wrote the 403 response
|
||||
# loop: the approved value is now in safe_tokens; re-scan.
|
||||
|
||||
def _redact_outbound(
|
||||
self, flow: http.HTTPFlow, route: Route, env: "typing.Mapping[str, str]",
|
||||
) -> bool:
|
||||
"""Scrub detected tokens (and CRLF injection sequences) from the mutable
|
||||
request surfaces (body, headers, path/query) and re-scan. `env` is the
|
||||
per-bottle env overlay. Returns True if the request is now clean; False
|
||||
if a block-severity match remains on a surface redaction cannot rewrite
|
||||
(the hostname) so the caller fails closed."""
|
||||
body = flow.request.get_text(strict=False)
|
||||
if body:
|
||||
redacted_body = redact_tokens(body, env=env)
|
||||
if redacted_body != body:
|
||||
flow.request.text = redacted_body
|
||||
for name, value in list(flow.request.headers.items()):
|
||||
if name.lower() == "host":
|
||||
continue # routing-critical; never a legitimate token
|
||||
redacted = strip_crlf(redact_tokens(value, env=env))
|
||||
if redacted != value:
|
||||
flow.request.headers[name] = redacted
|
||||
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=env))
|
||||
if redacted_path != flow.request.path:
|
||||
flow.request.path = redacted_path
|
||||
|
||||
request_path, _, query = flow.request.path.partition("?")
|
||||
new_body = flow.request.get_text(strict=False) or ""
|
||||
headers = outbound_scan_headers(route, dict(flow.request.headers))
|
||||
scan_text = build_outbound_scan_text(
|
||||
flow.request.pretty_host, request_path, query, headers, new_body,
|
||||
)
|
||||
crlf_text = build_outbound_scan_text(
|
||||
flow.request.pretty_host, request_path, query, headers, "",
|
||||
)
|
||||
result = scan_outbound(route, scan_text, env, crlf_text=crlf_text)
|
||||
return result is None or result.severity != "block"
|
||||
|
||||
async def _supervise_token_block(
|
||||
self,
|
||||
flow: http.HTTPFlow,
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
"""Outbound DLP request scanning and redaction for the egress pipeline."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import ItemsView, Mapping, Protocol
|
||||
|
||||
from .dlp import (
|
||||
build_outbound_scan_text,
|
||||
outbound_scan_headers,
|
||||
scan_outbound,
|
||||
)
|
||||
from .dlp_detectors import redact_tokens, strip_crlf
|
||||
from .types import Route, ScanResult
|
||||
|
||||
|
||||
class MutableHeaders(Protocol):
|
||||
def items(self) -> ItemsView[str, str]: ...
|
||||
def __getitem__(self, name: str, /) -> str: ...
|
||||
def __setitem__(self, name: str, value: str, /) -> None: ...
|
||||
|
||||
|
||||
class MutableRequest(Protocol):
|
||||
pretty_host: str
|
||||
path: str
|
||||
headers: MutableHeaders
|
||||
text: str
|
||||
|
||||
def get_text(self, strict: bool = False) -> str | None: ...
|
||||
|
||||
|
||||
def scan_request(
|
||||
request: MutableRequest,
|
||||
route: Route,
|
||||
env: Mapping[str, str],
|
||||
*,
|
||||
safe_tokens: set[str] | None = None,
|
||||
) -> ScanResult | None:
|
||||
"""Scan all mutable outbound request surfaces in their canonical order."""
|
||||
request_path, _, query = request.path.partition("?")
|
||||
headers = outbound_scan_headers(route, dict(request.headers.items()))
|
||||
body = request.get_text(strict=False) or ""
|
||||
scan_text = build_outbound_scan_text(
|
||||
request.pretty_host, request_path, query, headers, body,
|
||||
)
|
||||
# Bodies cannot alter HTTP framing, so CRLF detection is deliberately
|
||||
# restricted to the request line and headers.
|
||||
crlf_text = build_outbound_scan_text(
|
||||
request.pretty_host, request_path, query, headers, "",
|
||||
)
|
||||
return scan_outbound(
|
||||
route,
|
||||
scan_text,
|
||||
env,
|
||||
safe_tokens=safe_tokens,
|
||||
crlf_text=crlf_text,
|
||||
)
|
||||
|
||||
|
||||
def redact_request(
|
||||
request: MutableRequest,
|
||||
route: Route,
|
||||
env: Mapping[str, str],
|
||||
) -> bool:
|
||||
"""Redact mutable request surfaces and return whether the result is clean."""
|
||||
body = request.get_text(strict=False)
|
||||
if body:
|
||||
redacted_body = redact_tokens(body, env=env)
|
||||
if redacted_body != body:
|
||||
request.text = redacted_body
|
||||
for name, value in list(request.headers.items()):
|
||||
if name.lower() == "host":
|
||||
continue
|
||||
redacted = strip_crlf(redact_tokens(value, env=env))
|
||||
if redacted != value:
|
||||
request.headers[name] = redacted
|
||||
redacted_path = strip_crlf(redact_tokens(request.path, env=env))
|
||||
if redacted_path != request.path:
|
||||
request.path = redacted_path
|
||||
result = scan_request(request, route, env)
|
||||
return result is None or result.severity != "block"
|
||||
|
||||
|
||||
__all__ = ["MutableRequest", "redact_request", "scan_request"]
|
||||
@@ -1,92 +0,0 @@
|
||||
"""Framework-neutral request policy stages for the egress adapter.
|
||||
|
||||
The mitmproxy addon owns flow mutation and response construction. This module
|
||||
owns the ordered Git and route-policy decisions so those rules remain directly
|
||||
testable without a live proxy flow.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Mapping, Sequence
|
||||
|
||||
from .matching import (
|
||||
decide,
|
||||
decide_git_fetch,
|
||||
is_git_fetch_request,
|
||||
is_git_push_request,
|
||||
)
|
||||
from .types import LOG_FULL, Config, Route
|
||||
|
||||
GIT_PUSH_BLOCK_REASON = (
|
||||
"egress: git push over HTTPS is not supported; "
|
||||
"use the bottle.git SSH path (gitleaks-scanned by "
|
||||
"git-gate's pre-receive hook)."
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RoutePolicyResult:
|
||||
"""The flow mutations and outcome produced by general route policy."""
|
||||
|
||||
block_reason: str = ""
|
||||
strip_authorization: bool = False
|
||||
inject_authorization: str | None = None
|
||||
log_request: bool = False
|
||||
|
||||
|
||||
def git_block_reason(
|
||||
routes: Sequence[Route],
|
||||
host: str,
|
||||
request_path: str,
|
||||
query: str,
|
||||
) -> str:
|
||||
"""Return the HTTPS Git policy denial, or ``""`` when allowed."""
|
||||
if is_git_push_request(request_path, query):
|
||||
return GIT_PUSH_BLOCK_REASON
|
||||
if not is_git_fetch_request(request_path, query):
|
||||
return ""
|
||||
decision = decide_git_fetch(routes, host)
|
||||
return decision.reason if decision.action == "block" else ""
|
||||
|
||||
|
||||
def evaluate_route_policy(
|
||||
config: Config,
|
||||
route: Route | None,
|
||||
*,
|
||||
host: str,
|
||||
request_path: str,
|
||||
method: str,
|
||||
headers: Mapping[str, str],
|
||||
env: Mapping[str, str],
|
||||
) -> RoutePolicyResult:
|
||||
"""Evaluate authorization stripping, matching, injection, and logging."""
|
||||
strip_authorization = route is None or not route.preserve_auth
|
||||
effective_headers = {
|
||||
name.lower(): value
|
||||
for name, value in headers.items()
|
||||
if not (strip_authorization and name.lower() == "authorization")
|
||||
}
|
||||
decision = decide(
|
||||
config.routes,
|
||||
host,
|
||||
request_path,
|
||||
env,
|
||||
request_method=method,
|
||||
request_headers=effective_headers,
|
||||
deny_reason=config.deny_reason,
|
||||
)
|
||||
return RoutePolicyResult(
|
||||
block_reason=decision.reason if decision.action == "block" else "",
|
||||
strip_authorization=strip_authorization,
|
||||
inject_authorization=decision.inject_authorization,
|
||||
log_request=config.log >= LOG_FULL,
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"GIT_PUSH_BLOCK_REASON",
|
||||
"RoutePolicyResult",
|
||||
"evaluate_route_policy",
|
||||
"git_block_reason",
|
||||
]
|
||||
@@ -22,16 +22,11 @@ import os
|
||||
import subprocess
|
||||
import sys
|
||||
import typing
|
||||
from http.server import BaseHTTPRequestHandler
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from bot_bottle.constants import GIT_GATE_TIMEOUT_SECS, IDENTITY_HEADER
|
||||
from bot_bottle.gateway.bounded_http import (
|
||||
BodyReadError,
|
||||
BoundedThreadingHTTPServer,
|
||||
read_declared_body,
|
||||
)
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
|
||||
|
||||
|
||||
@@ -82,8 +77,6 @@ def resolve_sandbox_root(
|
||||
|
||||
# Bound memory use while still allowing ordinary git push packfiles.
|
||||
MAX_BODY_BYTES = 100 * 1024 * 1024
|
||||
REQUEST_BODY_TIMEOUT_SECONDS = 30.0
|
||||
MAX_REQUEST_WORKERS = 16
|
||||
|
||||
|
||||
class GitHttpHandler(BaseHTTPRequestHandler):
|
||||
@@ -191,31 +184,27 @@ class GitHttpHandler(BaseHTTPRequestHandler):
|
||||
value = self.headers.get(header)
|
||||
if value:
|
||||
env[variable] = value
|
||||
raw_length = self.headers.get("content-length", "0") or "0"
|
||||
try:
|
||||
body = read_declared_body(
|
||||
self.rfile,
|
||||
self.connection,
|
||||
self.headers.get("content-length"),
|
||||
maximum=MAX_BODY_BYTES,
|
||||
timeout_seconds=REQUEST_BODY_TIMEOUT_SECONDS,
|
||||
require_length=False,
|
||||
)
|
||||
except BodyReadError as exc:
|
||||
self.send_error(exc.status, exc.message)
|
||||
length = int(raw_length)
|
||||
except ValueError:
|
||||
self.send_error(400, "Bad Content-Length")
|
||||
return
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
["git", "http-backend"],
|
||||
input=body,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
timeout=GIT_GATE_TIMEOUT_SECS,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
self.log_message("git http-backend unavailable: %s", exc)
|
||||
self.send_error(503, "git backend unavailable")
|
||||
if length < 0:
|
||||
self.send_error(400, "Negative Content-Length")
|
||||
return
|
||||
if length > MAX_BODY_BYTES:
|
||||
self.send_error(413, "Request body too large")
|
||||
return
|
||||
body = self.rfile.read(length) if length else b""
|
||||
proc = subprocess.run(
|
||||
["git", "http-backend"],
|
||||
input=body,
|
||||
env=env,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
timeout=GIT_GATE_TIMEOUT_SECS,
|
||||
)
|
||||
self._write_cgi_response(proc.stdout)
|
||||
|
||||
def _repo_dir(self, sandbox_root: Path, path: str) -> Path | None:
|
||||
@@ -284,9 +273,7 @@ def main() -> int:
|
||||
"(no single-tenant flat-root fallback)\n"
|
||||
)
|
||||
return 1
|
||||
server = BoundedThreadingHTTPServer(
|
||||
("0.0.0.0", port), GitHttpHandler, max_workers=MAX_REQUEST_WORKERS,
|
||||
)
|
||||
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
|
||||
# Resolve each request's sandbox namespace by source IP against the
|
||||
# orchestrator control plane.
|
||||
server.policy_resolver = PolicyResolver(orch_url) # type: ignore[attr-defined]
|
||||
|
||||
@@ -1,92 +0,0 @@
|
||||
"""Framework-neutral MCP method and tool dispatch."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from typing import Callable, Protocol
|
||||
|
||||
from bot_bottle.gateway.egress.schema import load_config, route_to_yaml_dict
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
|
||||
from bot_bottle.supervisor import types as _sv
|
||||
|
||||
|
||||
class Request(Protocol):
|
||||
@property
|
||||
def method(self) -> str: ...
|
||||
|
||||
@property
|
||||
def params(self) -> dict[str, object]: ...
|
||||
|
||||
|
||||
class MethodNotFoundError(Exception):
|
||||
"""Raised when a JSON-RPC method has no MCP handler."""
|
||||
|
||||
|
||||
class RouteResolutionError(Exception):
|
||||
"""The caller's live route table could not be resolved authoritatively."""
|
||||
|
||||
|
||||
Handler = Callable[[dict[str, object]], object]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Handlers:
|
||||
initialize: Handler
|
||||
tools_list: Handler
|
||||
list_routes: Handler
|
||||
check_proposal: Handler
|
||||
propose: Handler
|
||||
|
||||
|
||||
def dispatch(request: Request, handlers: Handlers) -> object:
|
||||
"""Route one parsed request without depending on the HTTP server."""
|
||||
if request.method == "initialize":
|
||||
return handlers.initialize(request.params)
|
||||
if request.method == "notifications/initialized":
|
||||
return None
|
||||
if request.method == "tools/list":
|
||||
return handlers.tools_list(request.params)
|
||||
if request.method != "tools/call":
|
||||
raise MethodNotFoundError(request.method)
|
||||
|
||||
tool = request.params.get("name")
|
||||
if tool == _sv.TOOL_LIST_EGRESS_ROUTES:
|
||||
return handlers.list_routes(request.params)
|
||||
if tool == _sv.TOOL_CHECK_PROPOSAL:
|
||||
return handlers.check_proposal(request.params)
|
||||
return handlers.propose(request.params)
|
||||
|
||||
|
||||
def resolved_routes_payload(
|
||||
resolver: PolicyResolver,
|
||||
source_ip: str,
|
||||
identity_token: str,
|
||||
) -> dict[str, object]:
|
||||
"""Render an authoritatively resolved route table for the calling bottle."""
|
||||
try:
|
||||
policy, bottle_id, _tokens = resolver.resolve_policy_and_bottle_id(
|
||||
source_ip, identity_token,
|
||||
)
|
||||
except PolicyResolveError as exc:
|
||||
raise RouteResolutionError("orchestrator unavailable") from exc
|
||||
if not bottle_id:
|
||||
raise RouteResolutionError("request source is not attributed to a bottle")
|
||||
try:
|
||||
config = load_config(policy or "")
|
||||
except ValueError as exc:
|
||||
raise RouteResolutionError("resolved policy is invalid") from exc
|
||||
body = json.dumps(
|
||||
{"routes": [route_to_yaml_dict(route) for route in config.routes]},
|
||||
indent=2,
|
||||
)
|
||||
return {"content": [{"type": "text", "text": body}], "isError": False}
|
||||
|
||||
|
||||
__all__ = [
|
||||
"Handlers",
|
||||
"MethodNotFoundError",
|
||||
"RouteResolutionError",
|
||||
"dispatch",
|
||||
"resolved_routes_payload",
|
||||
]
|
||||
@@ -51,27 +51,17 @@ from __future__ import annotations
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
import socketserver
|
||||
import sys
|
||||
import time
|
||||
import typing
|
||||
from dataclasses import dataclass
|
||||
|
||||
from bot_bottle.constants import IDENTITY_HEADER
|
||||
from bot_bottle.gateway.bounded_http import (
|
||||
BodyReadError,
|
||||
BoundedThreadingHTTPServer,
|
||||
read_declared_body,
|
||||
)
|
||||
from bot_bottle.gateway.egress.schema import load_config
|
||||
from bot_bottle.gateway.egress.context import resolve_client_context
|
||||
from bot_bottle.gateway.egress.schema import load_config, route_to_yaml_dict
|
||||
from bot_bottle.gateway.egress.types import LOG_OFF
|
||||
from bot_bottle.gateway.policy_resolver import PolicyResolveError, PolicyResolver
|
||||
from bot_bottle.gateway.supervisor.mcp_dispatch import (
|
||||
Handlers as DispatchHandlers,
|
||||
MethodNotFoundError,
|
||||
RouteResolutionError,
|
||||
dispatch,
|
||||
resolved_routes_payload,
|
||||
)
|
||||
from bot_bottle.supervisor import types as _sv
|
||||
|
||||
|
||||
@@ -575,8 +565,6 @@ def format_unknown_proposal_text(proposal_id: str) -> str:
|
||||
# Max request body the server accepts. 1 MB is well above any realistic
|
||||
# routes.yaml proposal.
|
||||
MAX_BODY_BYTES = 1 * 1024 * 1024
|
||||
REQUEST_BODY_TIMEOUT_SECONDS = 10.0
|
||||
MAX_REQUEST_WORKERS = 32
|
||||
|
||||
|
||||
class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
@@ -599,18 +587,19 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
self._write_text(405, "use POST for MCP requests\n")
|
||||
|
||||
def do_POST(self) -> None:
|
||||
try:
|
||||
body = read_declared_body(
|
||||
self.rfile,
|
||||
self.connection,
|
||||
self.headers.get("Content-Length"),
|
||||
maximum=MAX_BODY_BYTES,
|
||||
timeout_seconds=REQUEST_BODY_TIMEOUT_SECONDS,
|
||||
require_length=True,
|
||||
)
|
||||
except BodyReadError as exc:
|
||||
self._write_text(exc.status, exc.message + "\n")
|
||||
length_header = self.headers.get("Content-Length")
|
||||
if length_header is None:
|
||||
self._write_text(411, "Content-Length required\n")
|
||||
return
|
||||
try:
|
||||
length = int(length_header)
|
||||
except ValueError:
|
||||
self._write_text(400, "invalid Content-Length\n")
|
||||
return
|
||||
if length < 0 or length > MAX_BODY_BYTES:
|
||||
self._write_text(413, "request body too large\n")
|
||||
return
|
||||
body = self.rfile.read(length)
|
||||
|
||||
try:
|
||||
req = parse_jsonrpc(body)
|
||||
@@ -622,11 +611,6 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
|
||||
try:
|
||||
result = self._dispatch(req, config)
|
||||
except MethodNotFoundError as e:
|
||||
self._write_jsonrpc(
|
||||
jsonrpc_error(req.id, ERR_METHOD_NOT_FOUND, f"method not found: {e}"),
|
||||
)
|
||||
return
|
||||
except _RpcClientError as e:
|
||||
self._write_jsonrpc(jsonrpc_error(req.id, e.code, e.message))
|
||||
return
|
||||
@@ -649,42 +633,41 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
self._write_jsonrpc(jsonrpc_result(req.id, result))
|
||||
|
||||
def _dispatch(self, req: JsonRpcRequest, config: ServerConfig) -> object:
|
||||
def check(params: dict[str, object]) -> object:
|
||||
return handle_check_proposal(
|
||||
params,
|
||||
resolver=self._resolver_or_fail(),
|
||||
source_ip=self.client_address[0],
|
||||
identity_token=self._identity_token(),
|
||||
)
|
||||
|
||||
def propose(params: dict[str, object]) -> object:
|
||||
return handle_tools_call(
|
||||
params,
|
||||
config,
|
||||
resolver=self._resolver_or_fail(),
|
||||
source_ip=self.client_address[0],
|
||||
identity_token=self._identity_token(),
|
||||
)
|
||||
|
||||
def list_routes(_params: dict[str, object]) -> object:
|
||||
try:
|
||||
return resolved_routes_payload(
|
||||
self._resolver_or_fail(),
|
||||
self.client_address[0],
|
||||
self._identity_token(),
|
||||
method = req.method
|
||||
if method == "initialize":
|
||||
return handle_initialize(req.params)
|
||||
if method == "notifications/initialized":
|
||||
return None # ack-only
|
||||
if method == "tools/list":
|
||||
return handle_tools_list(req.params)
|
||||
if method == "tools/call":
|
||||
# `list-egress-routes` is read-only introspection. The shared gateway
|
||||
# has no static route table (routes are resolved per request by
|
||||
# source IP), so answer it from the calling bottle's resolved policy.
|
||||
# Otherwise the agent sees an empty allowlist and composes an egress
|
||||
# proposal that *replaces* the live routes instead of extending them
|
||||
# — silently dropping base routes like api.anthropic.com on approval.
|
||||
if req.params.get("name") == _sv.TOOL_LIST_EGRESS_ROUTES:
|
||||
return self._resolved_routes_payload()
|
||||
resolver = self._resolver_or_fail()
|
||||
source_ip = self.client_address[0]
|
||||
token = self._identity_token()
|
||||
# `check-proposal` is a non-blocking read of the calling bottle's
|
||||
# own queue — attributed by (source_ip, identity_token) like a
|
||||
# proposal, but it never queues or blocks.
|
||||
if req.params.get("name") == _sv.TOOL_CHECK_PROPOSAL:
|
||||
return handle_check_proposal(
|
||||
req.params, resolver=resolver,
|
||||
source_ip=source_ip, identity_token=token,
|
||||
)
|
||||
except RouteResolutionError as exc:
|
||||
raise _RpcInternalError(
|
||||
f"could not resolve live egress routes: {exc}"
|
||||
) from exc
|
||||
|
||||
return dispatch(req, DispatchHandlers(
|
||||
initialize=handle_initialize,
|
||||
tools_list=handle_tools_list,
|
||||
list_routes=list_routes,
|
||||
check_proposal=check,
|
||||
propose=propose,
|
||||
))
|
||||
# The control plane attributes the proposal to the source-IP + token
|
||||
# resolved bottle, so the one shared queue holds each bottle's
|
||||
# proposal under its own id — no slug is asserted by this daemon.
|
||||
return handle_tools_call(
|
||||
req.params, config, resolver=resolver,
|
||||
source_ip=source_ip, identity_token=token,
|
||||
)
|
||||
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
|
||||
|
||||
def _identity_token(self) -> str:
|
||||
"""The agent's per-bottle identity token from the request header (the
|
||||
@@ -703,6 +686,20 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
raise _RpcInternalError("supervise server has no policy resolver")
|
||||
return resolver
|
||||
|
||||
def _resolved_routes_payload(self) -> dict[str, object]:
|
||||
"""The calling bottle's live egress routes as the `list-egress-routes`
|
||||
JSON payload, resolved by (source_ip, identity token). Fail-closed: an
|
||||
unattributed source or an unreachable orchestrator yields an empty route
|
||||
list (never another bottle's), courtesy of `resolve_client_context`."""
|
||||
resolver = self._resolver_or_fail()
|
||||
conf, _slug, _tokens = resolve_client_context(
|
||||
resolver, self.client_address[0], self._identity_token(),
|
||||
)
|
||||
body = json.dumps(
|
||||
{"routes": [route_to_yaml_dict(r) for r in conf.routes]}, indent=2,
|
||||
)
|
||||
return {"content": [{"type": "text", "text": body}], "isError": False}
|
||||
|
||||
def _write_jsonrpc(self, body: bytes) -> None:
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
@@ -722,7 +719,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
self.wfile.write(encoded)
|
||||
|
||||
|
||||
class MCPServer(BoundedThreadingHTTPServer):
|
||||
class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
||||
allow_reuse_address = True
|
||||
daemon_threads = True
|
||||
config: ServerConfig = ServerConfig()
|
||||
@@ -731,9 +728,6 @@ class MCPServer(BoundedThreadingHTTPServer):
|
||||
# closed per request (see `_resolver_or_fail`).
|
||||
policy_resolver: "PolicyResolver | None" = None
|
||||
|
||||
def __init__(self, *args, **kwargs): # type: ignore[no-untyped-def]
|
||||
super().__init__(*args, max_workers=MAX_REQUEST_WORKERS, **kwargs)
|
||||
|
||||
|
||||
# --- Entry point -----------------------------------------------------------
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
"""Manifest dataclasses (PRD 0011 layout).
|
||||
|
||||
Reads the per-file manifest tree:
|
||||
Reads the per-file manifest tree (home-only —
|
||||
PRD 0082):
|
||||
|
||||
$HOME/.bot-bottle/bottles/<name>.md — one bottle per file
|
||||
$HOME/.bot-bottle/agents/<name>.md — home-resident agents
|
||||
$CWD/.bot-bottle/agents/<name>.md — cwd-supplied agents
|
||||
$HOME/.bot-bottle/agents/<name>.md — agents
|
||||
|
||||
Each file is Markdown with YAML frontmatter. The frontmatter holds
|
||||
the structured config (see schema below); for agents the body is
|
||||
@@ -15,27 +15,38 @@ Bottle schema (frontmatter):
|
||||
extends: <bottle-name> # optional (PRD 0025)
|
||||
env: { <NAME>: <env-entry>, ... }
|
||||
git-gate: # optional (PRD 0047)
|
||||
user: { name: <str>, email: <str> } # optional
|
||||
repos: { <name>: <git-gate-entry>, ... } # optional
|
||||
# git-gate-entry keys: url, key, host_key, forge
|
||||
# `forge`: optional alias into the selected agent's forge-accounts
|
||||
egress: { routes: [ <egress-route>, ... ] }
|
||||
# route keys: host, matches, auth, role, dlp
|
||||
supervise: <bool> # optional (default true)
|
||||
nested_containers: <bool> # optional (default false)
|
||||
|
||||
Agent schema (frontmatter):
|
||||
bottle: <bottle-name> # required
|
||||
bottle: <bottle-name> # optional
|
||||
skills: [ <skill-name>, ... ] # optional
|
||||
git-gate:
|
||||
user: { name: <str>, email: <str> } # optional; overlays bottle
|
||||
author: # optional; agent git identity
|
||||
name: <str> # required when author is present
|
||||
email: <str> # required when author is present
|
||||
forge-accounts: # optional; alias -> forge account
|
||||
<alias>:
|
||||
url: <https Gitea /api/v1 base>
|
||||
auth: { type: token, token_secret: <host env var name> }
|
||||
# Claude Code subagent passthrough fields — accepted, ignored:
|
||||
name, description, model, color, memory
|
||||
|
||||
`author` populates the bottle's git user.name/user.email; `forge-accounts`
|
||||
maps a forge alias to a Gitea API origin plus a host token reference. Identity
|
||||
is agent-owned — `git-gate` is no longer accepted on an agent (git-gate.user
|
||||
moved to `author`; git-gate.repos is bottle-only).
|
||||
|
||||
The agent file's Markdown body is the system prompt (stripped).
|
||||
Unknown top-level frontmatter keys raise ManifestError with a hint.
|
||||
|
||||
Bottles can ONLY live under $HOME. A bottles/ dir under $CWD is a
|
||||
warn at load time and contributes nothing. The trust boundary is
|
||||
expressed as filesystem layout rather than resolver logic.
|
||||
Both bottles and agents can ONLY live under $HOME. An agents/ or bottles/
|
||||
dir under $CWD is a warn at load time and contributes nothing. The trust
|
||||
boundary is expressed as filesystem layout rather than resolver logic.
|
||||
|
||||
Two types are exported:
|
||||
|
||||
@@ -66,6 +77,11 @@ if TYPE_CHECKING:
|
||||
from .agent import ManifestAgent, ManifestAgentProvider
|
||||
from .bottle import ManifestBottle
|
||||
from .egress import EGRESS_AUTH_SCHEMES, ManifestEgressConfig, ManifestEgressRoute
|
||||
from .forge import (
|
||||
ManifestAuthor,
|
||||
ManifestForgeAccount,
|
||||
ResolvedForgeAssociation,
|
||||
)
|
||||
from .git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig
|
||||
|
||||
|
||||
@@ -81,6 +97,9 @@ _LAZY_MODULES: dict[str, str] = {
|
||||
"EGRESS_AUTH_SCHEMES": "egress",
|
||||
"ManifestEgressRoute": "egress",
|
||||
"ManifestEgressConfig": "egress",
|
||||
"ManifestAuthor": "forge",
|
||||
"ManifestForgeAccount": "forge",
|
||||
"ResolvedForgeAssociation": "forge",
|
||||
"ManifestGitEntry": "git",
|
||||
"ManifestGitUser": "git",
|
||||
"ManifestKeyConfig": "git",
|
||||
@@ -115,4 +134,7 @@ __all__ = [
|
||||
"EGRESS_AUTH_SCHEMES",
|
||||
"ManifestEgressRoute",
|
||||
"ManifestEgressConfig",
|
||||
"ManifestAuthor",
|
||||
"ManifestForgeAccount",
|
||||
"ResolvedForgeAssociation",
|
||||
]
|
||||
|
||||
@@ -3,11 +3,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import cast
|
||||
from typing import Mapping, cast
|
||||
|
||||
from ..agent_provider import PROVIDER_TEMPLATES
|
||||
from .util import ManifestError, as_json_object
|
||||
from .git import ManifestGitUser
|
||||
from .forge import ManifestAuthor, ManifestForgeAccount
|
||||
from .schema import AGENT_MODEL_KEYS, is_valid_entity_name
|
||||
|
||||
|
||||
@@ -119,15 +119,29 @@ class ManifestAgent:
|
||||
bottle: str = ""
|
||||
skills: tuple[str, ...] = ()
|
||||
prompt: str = ""
|
||||
# Per-agent git identity (issue #94). Overlays the referenced
|
||||
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
|
||||
# `user` is allowed at the agent level; `repos` stays bottle-only
|
||||
# because it carries credentials and host trust.
|
||||
git_user: ManifestGitUser = ManifestGitUser()
|
||||
# Agent-owned identity (PRD 0082).
|
||||
# `author` populates the bottle's git user.name/user.email;
|
||||
# `forge_accounts` maps a forge alias to a canonical Gitea API origin and
|
||||
# a host token reference. Both live only on the agent — never under
|
||||
# `git-gate`, which is bottle-only transport policy.
|
||||
author: ManifestAuthor | None = None
|
||||
forge_accounts: Mapping[str, ManifestForgeAccount] = field(
|
||||
default_factory=dict
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "ManifestAgent":
|
||||
d = as_json_object(raw, f"agent '{name}'")
|
||||
# git-gate is no longer accepted on an agent (checked before the
|
||||
# generic unknown-key error so the migration pointer is surfaced):
|
||||
# identity moved to `author`, and git-gate.repos is bottle-only.
|
||||
if "git-gate" in d:
|
||||
raise ManifestError(
|
||||
f"agent '{name}' has a 'git-gate' block, which is no longer "
|
||||
f"accepted on an agent (PRD 0082). "
|
||||
f"Move git-gate.user name/email into the 'author' block; "
|
||||
f"git-gate.repos stays on the bottle."
|
||||
)
|
||||
unknown = set(d.keys()) - AGENT_MODEL_KEYS
|
||||
if unknown:
|
||||
allowed = ", ".join(sorted(AGENT_MODEL_KEYS))
|
||||
@@ -191,24 +205,30 @@ class ManifestAgent:
|
||||
f"(was {type(prompt_raw).__name__})"
|
||||
)
|
||||
|
||||
# git-gate: agents may declare only `git-gate.user` (name/email).
|
||||
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
||||
git_user = ManifestGitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
||||
for k in gd:
|
||||
if k != "user":
|
||||
raise ManifestError(
|
||||
f"agent '{name}' git-gate.{k} is not allowed at the "
|
||||
f"agent level; only git-gate.user (name/email) may be "
|
||||
f"set on an agent. git-gate.repos is bottle-only "
|
||||
f"(it carries credentials and host trust)."
|
||||
)
|
||||
if "user" in gd:
|
||||
git_user = ManifestGitUser.from_dict(name, gd["user"])
|
||||
# author: agent-owned git identity (optional; both fields required
|
||||
# when present). Populates the bottle's user.name/user.email.
|
||||
author = (
|
||||
ManifestAuthor.from_dict(name, d["author"])
|
||||
if "author" in d else None
|
||||
)
|
||||
|
||||
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
|
||||
# forge-accounts: alias -> Gitea API origin + host token reference.
|
||||
forge_accounts: dict[str, ManifestForgeAccount] = {}
|
||||
forge_raw = d.get("forge-accounts")
|
||||
if forge_raw is not None:
|
||||
forge_d = as_json_object(forge_raw, f"agent '{name}' forge-accounts")
|
||||
for alias, entry in forge_d.items():
|
||||
forge_accounts[alias] = ManifestForgeAccount.from_dict(
|
||||
name, alias, entry,
|
||||
)
|
||||
|
||||
return cls(
|
||||
bottle=bottle,
|
||||
skills=skills,
|
||||
prompt=prompt,
|
||||
author=author,
|
||||
forge_accounts=forge_accounts,
|
||||
)
|
||||
|
||||
|
||||
def _parse_provider_settings(
|
||||
|
||||
@@ -107,11 +107,14 @@ class ManifestBottle:
|
||||
)
|
||||
env[var] = value
|
||||
|
||||
# `git_user` is now an internal resolved carrier populated from the
|
||||
# selected agent's `author` at composition time — it is never parsed
|
||||
# from the bottle manifest (PRD 0082).
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
git_user = ManifestGitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
git, git_user = parse_git_gate_config(name, git_raw)
|
||||
git = parse_git_gate_config(name, git_raw)
|
||||
|
||||
agent_provider = (
|
||||
ManifestAgentProvider.from_dict(name, d["agent_provider"])
|
||||
|
||||
@@ -210,7 +210,7 @@ def _fold_two_bottles(
|
||||
for n in names
|
||||
}
|
||||
if merged_repos_raw:
|
||||
merged_git, _ = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
|
||||
merged_git = parse_git_gate_config("_fold", {"repos": merged_repos_raw})
|
||||
else:
|
||||
merged_git = ()
|
||||
|
||||
|
||||
@@ -0,0 +1,312 @@
|
||||
"""Agent-owned author identity and forge accounts (PRD 0082).
|
||||
|
||||
`ManifestAuthor` is the agent's git author identity (name/email) — it moved off
|
||||
`git-gate.user` (PRD 0027 / ADR 0002) and onto the trusted agent definition.
|
||||
|
||||
`ManifestForgeAccount` maps a **forge alias** to a canonical HTTPS Gitea API
|
||||
origin plus the *name* of a host env var holding the operator-provided API
|
||||
token. The token value never lives on the manifest; the host resolves it only
|
||||
when a selected bottle repository references the alias, and hands it to the
|
||||
egress proxy — never to the bottle.
|
||||
|
||||
`ResolvedForgeAssociation` is the composed view: one distinct forge alias that
|
||||
at least one selected git-gate repository points at, with the repository names
|
||||
that reference it. The proxy-provisioning and prompt-guidance steps consume it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import urllib.parse
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .schema import is_valid_entity_name
|
||||
from .util import ManifestError, as_json_object
|
||||
|
||||
# Only the Gitea `/api/v1` base is supported today. A future provider adds a
|
||||
# validator + auth scheme + guidance renderer in code rather than accepting a
|
||||
# repository-supplied prompt or path (PRD non-goal: provider-generic prompts).
|
||||
GITEA_API_PREFIX = "/api/v1"
|
||||
|
||||
# Auth schemes an agent forge account may declare. Gitea uses `token`.
|
||||
FORGE_AUTH_TYPES = ("token",)
|
||||
|
||||
|
||||
def canonicalize_forge_url(
|
||||
agent_name: str, alias: str, url: object,
|
||||
) -> tuple[str, str, str, str]:
|
||||
"""Validate and canonicalize a forge account's API base URL.
|
||||
|
||||
Returns `(canonical, origin, host, api_prefix)` where `origin` is
|
||||
`https://host[:port]` and `canonical` is `origin + api_prefix`.
|
||||
|
||||
Fails closed on: non-string, non-`https`, embedded userinfo, query, or
|
||||
fragment, a missing host, or any path other than the supported Gitea API
|
||||
base (`/api/v1`, trailing slash tolerated)."""
|
||||
label = f"agent '{agent_name}' forge-accounts.{alias}.url"
|
||||
if not isinstance(url, str) or not url:
|
||||
raise ManifestError(f"{label} is required (non-empty string)")
|
||||
try:
|
||||
parts = urllib.parse.urlsplit(url)
|
||||
except ValueError as e:
|
||||
raise ManifestError(f"{label} is not a valid URL ({e}): {url!r}") from e
|
||||
|
||||
if parts.scheme != "https":
|
||||
raise ManifestError(
|
||||
f"{label} must use https (got scheme {parts.scheme!r} in {url!r})"
|
||||
)
|
||||
if parts.username or parts.password:
|
||||
raise ManifestError(
|
||||
f"{label} must not embed userinfo (user:pass@); the token is held "
|
||||
f"host-side via auth.token_secret. Got {url!r}"
|
||||
)
|
||||
if parts.query:
|
||||
raise ManifestError(f"{label} must not contain a query string: {url!r}")
|
||||
if parts.fragment:
|
||||
raise ManifestError(f"{label} must not contain a fragment: {url!r}")
|
||||
|
||||
host = parts.hostname
|
||||
if not host:
|
||||
raise ManifestError(f"{label} must include a hostname: {url!r}")
|
||||
|
||||
path = parts.path.rstrip("/")
|
||||
if path != GITEA_API_PREFIX:
|
||||
raise ManifestError(
|
||||
f"{label} path must be the Gitea API base '{GITEA_API_PREFIX}' "
|
||||
f"(got {parts.path!r} in {url!r}); other providers and API paths "
|
||||
f"are not yet supported."
|
||||
)
|
||||
|
||||
host = host.lower()
|
||||
netloc = host if parts.port is None else f"{host}:{parts.port}"
|
||||
origin = f"https://{netloc}"
|
||||
return f"{origin}{path}", origin, host, path
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManifestAuthor:
|
||||
"""The agent's git author identity. Both fields are required and
|
||||
non-empty — an author that is declared must fully identify the agent.
|
||||
Populates `user.name` / `user.email` inside the bottle."""
|
||||
|
||||
name: str
|
||||
email: str
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, agent_name: str, raw: object) -> "ManifestAuthor":
|
||||
d = as_json_object(raw, f"agent '{agent_name}' author")
|
||||
for k in d:
|
||||
if k not in {"name", "email"}:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author has unknown key {k!r}; "
|
||||
f"allowed: name, email"
|
||||
)
|
||||
name = d.get("name")
|
||||
if not isinstance(name, str) or not name:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.name must be a non-empty string"
|
||||
)
|
||||
email = d.get("email")
|
||||
if not isinstance(email, str) or not email:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.email must be a non-empty string"
|
||||
)
|
||||
# Git identity validation: reject values that would break the
|
||||
# `git config user.email` line or smuggle a second config directive.
|
||||
if any(c in email for c in ("\n", "\r", " ", "\t")):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.email must not contain whitespace "
|
||||
f"or newlines (got {email!r})"
|
||||
)
|
||||
if any(c in name for c in ("\n", "\r")):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' author.name must not contain newlines "
|
||||
f"(got {name!r})"
|
||||
)
|
||||
return cls(name=name, email=email)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManifestForgeAccount:
|
||||
"""One forge alias on an agent: a canonical Gitea API origin plus the
|
||||
host env var name that holds the agent-specific API token. The
|
||||
authenticated account is whoever owns that token — there is no separate
|
||||
account-name field."""
|
||||
|
||||
alias: str
|
||||
url: str # canonical https base incl. /api/v1, no trailing slash
|
||||
origin: str # https://host[:port]
|
||||
host: str # lowercased hostname
|
||||
api_prefix: str # /api/v1
|
||||
auth_type: str # "token"
|
||||
token_secret: str # host env var name holding the API token
|
||||
|
||||
@classmethod
|
||||
def from_dict(
|
||||
cls, agent_name: str, alias: str, raw: object,
|
||||
) -> "ManifestForgeAccount":
|
||||
if not is_valid_entity_name(alias):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' forge-accounts key {alias!r} is not a "
|
||||
f"valid alias; must match [a-z][a-z0-9-]*"
|
||||
)
|
||||
label = f"agent '{agent_name}' forge-accounts.{alias}"
|
||||
d = as_json_object(raw, label)
|
||||
for k in d:
|
||||
if k not in {"url", "auth"}:
|
||||
raise ManifestError(
|
||||
f"{label} has unknown key {k!r}; allowed: url, auth"
|
||||
)
|
||||
|
||||
canonical, origin, host, api_prefix = canonicalize_forge_url(
|
||||
agent_name, alias, d.get("url"),
|
||||
)
|
||||
|
||||
if "auth" not in d:
|
||||
raise ManifestError(f"{label} missing required 'auth' block")
|
||||
auth = as_json_object(d.get("auth"), f"{label}.auth")
|
||||
for k in auth:
|
||||
if k not in {"type", "token_secret"}:
|
||||
raise ManifestError(
|
||||
f"{label}.auth has unknown key {k!r}; allowed: type, "
|
||||
f"token_secret"
|
||||
)
|
||||
auth_type = auth.get("type")
|
||||
if auth_type not in FORGE_AUTH_TYPES:
|
||||
raise ManifestError(
|
||||
f"{label}.auth.type must be one of "
|
||||
f"{', '.join(FORGE_AUTH_TYPES)} (got {auth_type!r})"
|
||||
)
|
||||
token_secret = auth.get("token_secret")
|
||||
if not isinstance(token_secret, str) or not token_secret:
|
||||
raise ManifestError(
|
||||
f"{label}.auth.token_secret must be a non-empty string (the "
|
||||
f"name of a host env var holding the API token)"
|
||||
)
|
||||
return cls(
|
||||
alias=alias,
|
||||
url=canonical,
|
||||
origin=origin,
|
||||
host=host,
|
||||
api_prefix=api_prefix,
|
||||
auth_type=str(auth_type),
|
||||
token_secret=token_secret,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ResolvedForgeAssociation:
|
||||
"""A distinct forge alias referenced by one or more selected git-gate
|
||||
repositories. `repo_names` lists the git-gate repo names pointing at
|
||||
`account.alias` (sorted, deduplicated)."""
|
||||
|
||||
account: ManifestForgeAccount
|
||||
repo_names: tuple[str, ...]
|
||||
|
||||
@property
|
||||
def alias(self) -> str:
|
||||
return self.account.alias
|
||||
|
||||
|
||||
def resolve_forge_associations(
|
||||
agent_name: str,
|
||||
forge_accounts: "dict[str, ManifestForgeAccount]",
|
||||
git_entries: "tuple[object, ...]",
|
||||
) -> tuple[ResolvedForgeAssociation, ...]:
|
||||
"""Compose the agent's forge accounts with the effective bottle's
|
||||
git-gate repos. Each git entry that declares a `forge` alias must match a
|
||||
forge account on the selected agent — otherwise fail closed before launch.
|
||||
|
||||
Returns one association per distinct referenced alias, carrying the repo
|
||||
names that reference it. Unreferenced accounts produce nothing (no token
|
||||
is resolved and no route is created for them)."""
|
||||
by_alias: dict[str, list[str]] = {}
|
||||
for entry in git_entries:
|
||||
alias = getattr(entry, "Forge", "")
|
||||
if not alias:
|
||||
continue
|
||||
if alias not in forge_accounts:
|
||||
available = ", ".join(sorted(forge_accounts)) or "(none)"
|
||||
raise ManifestError(
|
||||
f"git-gate.repos['{getattr(entry, 'Name', '?')}'].forge "
|
||||
f"references forge alias {alias!r}, which is not defined on "
|
||||
f"agent '{agent_name}'. Available forge-accounts: {available}"
|
||||
)
|
||||
by_alias.setdefault(alias, []).append(getattr(entry, "Name", ""))
|
||||
|
||||
associations = tuple(
|
||||
ResolvedForgeAssociation(
|
||||
account=forge_accounts[alias],
|
||||
repo_names=tuple(sorted(set(names))),
|
||||
)
|
||||
for alias, names in sorted(by_alias.items())
|
||||
)
|
||||
|
||||
# The egress proxy routes by host, so two referenced aliases that resolve
|
||||
# to the same host must agree on the credential and target — otherwise the
|
||||
# generated plan would authenticate every call to that host as whichever
|
||||
# alias happened to win, silently acting as the wrong forge account. Fail
|
||||
# closed here (before the bottle is created) rather than dropping a
|
||||
# credential at route-synthesis time.
|
||||
by_host: dict[str, ManifestForgeAccount] = {}
|
||||
for assoc in associations:
|
||||
acct = assoc.account
|
||||
prev = by_host.get(acct.host)
|
||||
if prev is None:
|
||||
by_host[acct.host] = acct
|
||||
continue
|
||||
if (prev.origin, prev.api_prefix, prev.auth_type, prev.token_secret) != (
|
||||
acct.origin, acct.api_prefix, acct.auth_type, acct.token_secret
|
||||
):
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' forge-accounts '{prev.alias}' and "
|
||||
f"'{acct.alias}' both resolve to host '{acct.host}' but differ "
|
||||
f"in origin/auth/token_secret. The egress proxy routes by host, "
|
||||
f"so the intended credential would be ambiguous — every request "
|
||||
f"to '{acct.host}' would authenticate as one account. Give the "
|
||||
f"aliases distinct hosts, or make their url/auth identical."
|
||||
)
|
||||
|
||||
return associations
|
||||
|
||||
|
||||
def render_forge_guidance(
|
||||
associations: tuple[ResolvedForgeAssociation, ...],
|
||||
) -> str:
|
||||
"""Render the non-secret, provider-specific forge workflow guidance
|
||||
appended to the agent's system prompt for associated repositories only.
|
||||
|
||||
Derived entirely from validated typed fields — never repository-supplied
|
||||
Markdown. Contains neither the token value nor its `token_secret` name.
|
||||
Returns "" when there are no associations (no section is generated)."""
|
||||
if not associations:
|
||||
return ""
|
||||
lines: list[str] = [
|
||||
"## Forge access (managed by bot-bottle)",
|
||||
"",
|
||||
"bot-bottle authenticates the forge API requests below for you "
|
||||
"through the egress proxy. Do not read, print, or manually attach "
|
||||
"an authorization token — the proxy injects it. Never place a token "
|
||||
"in a request.",
|
||||
]
|
||||
for assoc in associations:
|
||||
acct = assoc.account
|
||||
for repo in assoc.repo_names:
|
||||
lines.extend([
|
||||
"",
|
||||
f"### Repository `{repo}` (forge alias `{acct.alias}`)",
|
||||
f"- Forge API base URL: `{acct.url}`.",
|
||||
f"- This git-gate repository (`{repo}`) is tied to forge "
|
||||
f"`{acct.alias}`; use its API for forge actions on this repo.",
|
||||
f"- Call the API at `{acct.url}` over HTTPS through the proxy. "
|
||||
"The proxy adds authentication; you never supply a token "
|
||||
"yourself.",
|
||||
"- Git pushes still use the git-gate remote, not the API.",
|
||||
"- To propose changes: push a normal branch "
|
||||
"(`refs/heads/<branch>`) through the git-gate remote, then open "
|
||||
"a branch-backed pull request through the API.",
|
||||
"- Do NOT push AGit review refs (`refs/for/*`, `refs/draft/*`, "
|
||||
"`refs/for-review/*`); they are prohibited here.",
|
||||
"- Use the API for reviews and comments, and verify the "
|
||||
"returned object state before claiming the task is complete.",
|
||||
])
|
||||
return "\n".join(lines) + "\n"
|
||||
+38
-12
@@ -117,6 +117,11 @@ class ManifestGitEntry:
|
||||
UpstreamHost: str = ""
|
||||
UpstreamPort: str = ""
|
||||
UpstreamPath: str = ""
|
||||
# Optional forge alias (PRD 0082). When
|
||||
# set, it must match a `forge-accounts` alias on the selected agent; the
|
||||
# composition enables a scoped proxy-held API credential and forge
|
||||
# workflow guidance for this repo. Empty = no forge association.
|
||||
Forge: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_repos_entry(
|
||||
@@ -139,10 +144,10 @@ class ManifestGitEntry:
|
||||
label = f"git-gate.repos[{repo_name!r}]"
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
||||
for k in d:
|
||||
if k not in {"url", "key", "host_key"}:
|
||||
if k not in {"url", "key", "host_key", "forge"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
||||
f"allowed: url, key, host_key"
|
||||
f"allowed: url, key, host_key, forge"
|
||||
)
|
||||
upstream = d.get("url")
|
||||
if not isinstance(upstream, str) or not upstream:
|
||||
@@ -150,6 +155,21 @@ class ManifestGitEntry:
|
||||
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
||||
)
|
||||
|
||||
forge = d.get("forge", "")
|
||||
if not isinstance(forge, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} forge must be a string "
|
||||
f"(was {type(forge).__name__})"
|
||||
)
|
||||
if forge and not _GIT_NAME_RE.match(forge):
|
||||
# forge aliases follow the kebab-case identifier grammar; the
|
||||
# cross-check against the agent's forge-accounts happens at
|
||||
# composition time (it needs the resolved agent).
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} forge {forge!r} is not a "
|
||||
f"valid forge alias; allowed characters: A-Z a-z 0-9 . _ -"
|
||||
)
|
||||
|
||||
if "key" not in d:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} missing required 'key' block"
|
||||
@@ -176,6 +196,7 @@ class ManifestGitEntry:
|
||||
UpstreamHost=host,
|
||||
UpstreamPort=port,
|
||||
UpstreamPath=path,
|
||||
Forge=forge,
|
||||
)
|
||||
|
||||
|
||||
@@ -286,21 +307,26 @@ class ManifestGitUser:
|
||||
def parse_git_gate_config(
|
||||
bottle_name: str,
|
||||
raw: object,
|
||||
) -> tuple[tuple[ManifestGitEntry, ...], ManifestGitUser]:
|
||||
) -> tuple[ManifestGitEntry, ...]:
|
||||
"""Parse `git-gate` on a bottle. Only `repos` is accepted; `git-gate.user`
|
||||
moved to the agent's `author` block (PRD 0082)."""
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
||||
if "user" in d:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate.user is no longer supported "
|
||||
f"(PRD 0082). Move name/email into "
|
||||
f"the selected home agent's 'author' block:\n"
|
||||
f" author:\n name: <name>\n email: <email>\n"
|
||||
f"Identity is agent-owned; git-gate now carries only transport "
|
||||
f"policy (repos)."
|
||||
)
|
||||
for k in d:
|
||||
if k not in {"user", "repos"}:
|
||||
if k != "repos":
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
||||
f"allowed: user, repos"
|
||||
f"allowed: repos"
|
||||
)
|
||||
|
||||
git_user = (
|
||||
ManifestGitUser.from_dict(bottle_name, d["user"])
|
||||
if "user" in d
|
||||
else ManifestGitUser()
|
||||
)
|
||||
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
repos_raw = d.get("repos")
|
||||
if repos_raw is not None:
|
||||
@@ -311,4 +337,4 @@ def parse_git_gate_config(
|
||||
)
|
||||
validate_unique_git_names(bottle_name, git)
|
||||
|
||||
return git, git_user
|
||||
return git
|
||||
|
||||
@@ -19,6 +19,7 @@ from .util import ManifestError, as_json_object
|
||||
from .agent import ManifestAgent
|
||||
from .bottle import ManifestBottle
|
||||
from .extends import merge_bottles_runtime, resolve_bottles
|
||||
from .forge import ResolvedForgeAssociation, resolve_forge_associations
|
||||
from .git import ManifestGitUser
|
||||
from .loader import (
|
||||
check_stale_json,
|
||||
@@ -37,30 +38,50 @@ def _section_dict(value: object, label: str) -> dict[str, object]:
|
||||
return as_json_object(value, label)
|
||||
|
||||
|
||||
def _merge_git_user(
|
||||
agent_user: ManifestGitUser, base_user: ManifestGitUser
|
||||
) -> ManifestGitUser:
|
||||
"""Merge the agent's git.user over the bottle's, agent-wins-on-non-empty."""
|
||||
if agent_user.is_empty():
|
||||
return base_user
|
||||
return ManifestGitUser(
|
||||
name=agent_user.name or base_user.name,
|
||||
email=agent_user.email or base_user.email,
|
||||
def _warn_ignored_cwd_dir(cwd_dir: Path, kind: str, home_path: str) -> None:
|
||||
"""Warn (once) that manifest files of `kind` under `$CWD/.bot-bottle/`
|
||||
are ignored — the filesystem layout IS the trust boundary. `kind` is the
|
||||
subdir name (`bottles`/`agents`); `home_path` is where they belong."""
|
||||
stale = cwd_dir / kind
|
||||
if not stale.is_dir():
|
||||
return
|
||||
files = sorted(stale.glob("*.md"))
|
||||
if not files:
|
||||
return
|
||||
names = ", ".join(p.name for p in files)
|
||||
warn(
|
||||
f"ignoring {kind[:-1]} file(s) under {stale}: {names}. "
|
||||
f"{kind.capitalize()} can only live under {home_path} "
|
||||
f"(PRD 0082). Move them or delete."
|
||||
)
|
||||
|
||||
|
||||
def _manifest_with_merged_git_user(
|
||||
agent: "ManifestAgent", raw_bottle: "ManifestBottle"
|
||||
def _compose_manifest(
|
||||
agent_name: str,
|
||||
agent: "ManifestAgent",
|
||||
raw_bottle: "ManifestBottle",
|
||||
) -> "Manifest":
|
||||
"""Build the single-value Manifest, overlaying the agent's git-gate.user
|
||||
onto the bottle (agent wins on non-empty, per-field). Shared by the eager
|
||||
and lazy load_for_agent paths."""
|
||||
merged = _merge_git_user(agent.git_user, raw_bottle.git_user)
|
||||
bottle = (
|
||||
raw_bottle if merged == raw_bottle.git_user
|
||||
else replace(raw_bottle, git_user=merged)
|
||||
"""Build the single-value Manifest from the selected agent and its
|
||||
effective bottle (PRD 0082):
|
||||
|
||||
- the agent's `author` populates the bottle's git user.name/user.email;
|
||||
- each git-gate repo's `forge` alias is resolved against the agent's
|
||||
`forge-accounts` (failing closed on an unknown alias) into the
|
||||
Manifest's forge associations.
|
||||
|
||||
Shared by the eager (from_json_obj) and lazy (from_md_dirs) paths."""
|
||||
identity = (
|
||||
ManifestGitUser(name=agent.author.name, email=agent.author.email)
|
||||
if agent.author is not None else ManifestGitUser()
|
||||
)
|
||||
return Manifest(agent=agent, bottle=bottle)
|
||||
bottle = (
|
||||
raw_bottle if identity == raw_bottle.git_user
|
||||
else replace(raw_bottle, git_user=identity)
|
||||
)
|
||||
associations = resolve_forge_associations(
|
||||
agent_name, dict(agent.forge_accounts), bottle.git,
|
||||
)
|
||||
return Manifest(agent=agent, bottle=bottle, forge_associations=associations)
|
||||
|
||||
|
||||
def _resolve_effective_bottle_eager(
|
||||
@@ -121,26 +142,28 @@ def _resolve_effective_bottle_lazy(
|
||||
class Manifest:
|
||||
"""Single-agent/bottle value type. Returned by ManifestIndex.load_for_agent().
|
||||
|
||||
`bottle` is the effective bottle with the agent's git-gate.user already
|
||||
overlaid per-field (agent wins on non-empty). Backends and provisioners
|
||||
use this directly — no agent_name lookup needed."""
|
||||
`bottle` is the effective bottle with the agent's `author` already
|
||||
populated into its git identity. `forge_associations` holds the distinct
|
||||
forge aliases referenced by the effective bottle's git-gate repos, resolved
|
||||
against the agent's `forge-accounts`. Backends and provisioners use this
|
||||
directly — no agent_name lookup needed."""
|
||||
|
||||
agent: ManifestAgent
|
||||
bottle: ManifestBottle
|
||||
forge_associations: tuple[ResolvedForgeAssociation, ...] = ()
|
||||
|
||||
def git_identity_summary(self) -> str | None:
|
||||
"""One-line effective git identity with per-field provenance, e.g.
|
||||
`name=claude (agent), email=eric@dideric.is (bottle)`.
|
||||
Returns None when neither agent nor bottle sets an identity."""
|
||||
over = self.agent.git_user # agent's declared git_user (pre-merge)
|
||||
merged = self.bottle.git_user # effective git_user (post-merge)
|
||||
if merged.is_empty():
|
||||
"""One-line effective git identity, e.g.
|
||||
`name=claude, email=eric@dideric.is`. Sourced from the agent's
|
||||
`author` block. Returns None when the agent declares no author."""
|
||||
gu = self.bottle.git_user
|
||||
if gu.is_empty():
|
||||
return None
|
||||
parts: list[str] = []
|
||||
if merged.name:
|
||||
parts.append(f"name={merged.name} ({'agent' if over.name else 'bottle'})")
|
||||
if merged.email:
|
||||
parts.append(f"email={merged.email} ({'agent' if over.email else 'bottle'})")
|
||||
if gu.name:
|
||||
parts.append(f"name={gu.name}")
|
||||
if gu.email:
|
||||
parts.append(f"email={gu.email}")
|
||||
return ", ".join(parts)
|
||||
|
||||
|
||||
@@ -164,15 +187,15 @@ class ManifestIndex:
|
||||
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "ManifestIndex":
|
||||
"""Walk the per-file manifest tree and build a ManifestIndex.
|
||||
|
||||
Layout (PRD 0011):
|
||||
Layout:
|
||||
$HOME/.bot-bottle/bottles/<name>.md — bottles (home-only)
|
||||
$HOME/.bot-bottle/agents/<name>.md — home agents
|
||||
$CWD/.bot-bottle/agents/<name>.md — cwd agents
|
||||
$HOME/.bot-bottle/agents/<name>.md — agents (home-only)
|
||||
|
||||
Cwd agents merge into the home agents on the same name
|
||||
(cwd wins). A bottles/ subdir under $CWD is logged as a
|
||||
warning and ignored — the filesystem layout IS the trust
|
||||
boundary.
|
||||
Both agents and bottles are home-only
|
||||
(PRD 0082): a `bottles/` or `agents/`
|
||||
subdir under $CWD is logged as a warning and ignored — the filesystem
|
||||
layout IS the trust boundary, since an agent may now select a host
|
||||
identity and forge secret.
|
||||
|
||||
If `missing_ok` is true, a missing `$HOME/.bot-bottle/`
|
||||
returns an empty index instead of dying. This is for
|
||||
@@ -223,17 +246,12 @@ class ManifestIndex:
|
||||
Used by tests to build a ManifestIndex from fixture directories
|
||||
without touching `os.environ`."""
|
||||
if cwd_dir is not None:
|
||||
stale_bottles = cwd_dir / "bottles"
|
||||
if stale_bottles.is_dir():
|
||||
files = sorted(stale_bottles.glob("*.md"))
|
||||
if files:
|
||||
names = ", ".join(p.name for p in files)
|
||||
warn(
|
||||
f"ignoring bottle file(s) under "
|
||||
f"{stale_bottles}: {names}. Bottles can only "
|
||||
f"live under $HOME/.bot-bottle/bottles/ "
|
||||
f"(PRD 0011). Move them or delete."
|
||||
)
|
||||
_warn_ignored_cwd_dir(cwd_dir, "bottles", "$HOME/.bot-bottle/bottles/")
|
||||
# Agents became home-only in
|
||||
# PRD 0082: a cwd agent file that
|
||||
# once shadowed a home agent could select a host identity/secret,
|
||||
# so it is now ignored with a migration pointer.
|
||||
_warn_ignored_cwd_dir(cwd_dir, "agents", "$HOME/.bot-bottle/agents/")
|
||||
return cls(bottles={}, agents={}, home_md=home_dir, cwd_md=cwd_dir)
|
||||
|
||||
@classmethod
|
||||
@@ -275,13 +293,12 @@ class ManifestIndex:
|
||||
|
||||
In names-only mode (from resolve/from_md_dirs) this scans agent
|
||||
filenames without reading their content. In eager mode (from
|
||||
from_json_obj) it returns the pre-parsed agents' names."""
|
||||
from_json_obj) it returns the pre-parsed agents' names.
|
||||
|
||||
Agents are home-only (PRD 0082): cwd
|
||||
agent files never contribute names."""
|
||||
if self.home_md is not None:
|
||||
home_names = set(scan_agent_names(self.home_md / "agents").keys())
|
||||
cwd_names: set[str] = set()
|
||||
if self.cwd_md is not None:
|
||||
cwd_names = set(scan_agent_names(self.cwd_md / "agents").keys())
|
||||
return sorted(home_names | cwd_names)
|
||||
return sorted(scan_agent_names(self.home_md / "agents").keys())
|
||||
return sorted(self.agents.keys())
|
||||
|
||||
def load_for_agent(
|
||||
@@ -326,7 +343,7 @@ class ManifestIndex:
|
||||
raw_bottle = _resolve_effective_bottle_eager(
|
||||
agent_name, agent, bottle_names, self.bottles
|
||||
)
|
||||
return _manifest_with_merged_git_user(agent, raw_bottle)
|
||||
return _compose_manifest(agent_name, agent, raw_bottle)
|
||||
|
||||
def _load_for_agent_lazy(
|
||||
self, agent_name: str, bottle_names: tuple[str, ...]
|
||||
@@ -334,20 +351,17 @@ class ManifestIndex:
|
||||
"""Lazy path (resolve/from_md_dirs): read and parse the agent file and
|
||||
its bottle chain from disk for the first time here."""
|
||||
assert self.home_md is not None # guaranteed by load_for_agent dispatch
|
||||
# Locate the agent file; cwd wins over home on name collision.
|
||||
# Agents are home-only (PRD 0082):
|
||||
# a cwd agent file must not select a host identity or forge secret.
|
||||
home_agents = scan_agent_names(self.home_md / "agents")
|
||||
cwd_agents: dict[str, Path] = {}
|
||||
if self.cwd_md is not None:
|
||||
cwd_agents = scan_agent_names(self.cwd_md / "agents")
|
||||
merged_agents = {**home_agents, **cwd_agents}
|
||||
|
||||
if agent_name not in merged_agents:
|
||||
available = ", ".join(sorted(merged_agents.keys())) or "(none)"
|
||||
if agent_name not in home_agents:
|
||||
available = ", ".join(sorted(home_agents.keys())) or "(none)"
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' not defined. Available: {available}"
|
||||
)
|
||||
|
||||
agent_path = merged_agents[agent_name]
|
||||
agent_path = home_agents[agent_name]
|
||||
try:
|
||||
fm, body = parse_frontmatter(agent_path.read_text())
|
||||
except OSError as e:
|
||||
@@ -374,15 +388,18 @@ class ManifestIndex:
|
||||
}
|
||||
if agent_bottle:
|
||||
agent_dict["bottle"] = agent_bottle
|
||||
if "git-gate" in fm:
|
||||
agent_dict["git-gate"] = fm["git-gate"]
|
||||
# Surface agent-owned identity keys (and any stale git-gate, so
|
||||
# ManifestAgent.from_dict raises the migration error).
|
||||
for key in ("author", "forge-accounts", "git-gate"):
|
||||
if key in fm:
|
||||
agent_dict[key] = fm[key]
|
||||
# Pass the effective bottle name as the known-bottles set so agents
|
||||
# that have bottle: set are validated; agents without bottle: pass {}
|
||||
# since bottle_names were already resolved above.
|
||||
known = {effective_bottle_name} if effective_bottle_name else set()
|
||||
agent = ManifestAgent.from_dict(agent_name, agent_dict, known)
|
||||
|
||||
return _manifest_with_merged_git_user(agent, raw_bottle)
|
||||
return _compose_manifest(agent_name, agent, raw_bottle)
|
||||
|
||||
def has_agent(self, name: str) -> bool:
|
||||
return name in self.agents
|
||||
@@ -394,13 +411,9 @@ class ManifestIndex:
|
||||
if self.has_agent(name):
|
||||
return
|
||||
if self.home_md is not None:
|
||||
# Names-only mode: check file existence without parsing.
|
||||
home_path = self.home_md / "agents" / f"{name}.md"
|
||||
cwd_path = (
|
||||
self.cwd_md / "agents" / f"{name}.md"
|
||||
if self.cwd_md else None
|
||||
)
|
||||
if home_path.is_file() or (cwd_path and cwd_path.is_file()):
|
||||
# Names-only mode: check home file existence without parsing.
|
||||
# Agents are home-only; a cwd agent file is never selectable.
|
||||
if (self.home_md / "agents" / f"{name}.md").is_file():
|
||||
return
|
||||
available = ", ".join(self.all_agent_names) or "(none)"
|
||||
raise ManifestError(
|
||||
|
||||
@@ -22,7 +22,10 @@ BOTTLE_KEYS = frozenset(
|
||||
}
|
||||
)
|
||||
AGENT_KEYS_REQUIRED: frozenset[str] = frozenset()
|
||||
AGENT_KEYS_OPTIONAL = frozenset({"bottle", "skills", "git-gate"})
|
||||
# `author` / `forge-accounts` are agent-owned identity (PRD
|
||||
# 0082). `git-gate` is no longer accepted on an
|
||||
# agent: `git-gate.user` moved to `author`, and `git-gate.repos` is bottle-only.
|
||||
AGENT_KEYS_OPTIONAL = frozenset({"bottle", "skills", "author", "forge-accounts"})
|
||||
|
||||
# Claude Code subagent fields bot-bottle ignores at launch but does
|
||||
# not reject. This lets the same file double as
|
||||
|
||||
@@ -44,7 +44,7 @@ if TYPE_CHECKING:
|
||||
from ..gateway import Gateway, GatewayError
|
||||
from .lifecycle import Orchestrator
|
||||
from .service import OrchestratorCore
|
||||
from .server import OrchestratorServer, create_app, make_server
|
||||
from .server import OrchestratorServer, dispatch, make_server
|
||||
|
||||
|
||||
# Facade name -> submodule that defines it. Lazy so importing a leaf (or the
|
||||
@@ -67,8 +67,8 @@ _LAZY: dict[str, str] = {
|
||||
"GatewayError": "..gateway",
|
||||
"Orchestrator": ".lifecycle",
|
||||
"OrchestratorCore": ".service",
|
||||
"create_app": ".server",
|
||||
"OrchestratorServer": ".server",
|
||||
"dispatch": ".server",
|
||||
"make_server": ".server",
|
||||
}
|
||||
|
||||
@@ -100,7 +100,7 @@ __all__ = [
|
||||
"GatewayError",
|
||||
"Orchestrator",
|
||||
"OrchestratorCore",
|
||||
"create_app",
|
||||
"OrchestratorServer",
|
||||
"dispatch",
|
||||
"make_server",
|
||||
]
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""Run the orchestrator control plane as a plain process (PRD 0070 dev-harness).
|
||||
|
||||
BOT_BOTTLE_ORCHESTRATOR_TOKEN=<signing-key> \
|
||||
python -m bot_bottle.orchestrator [--host H] [--port P] [--db PATH]
|
||||
python -m bot_bottle.orchestrator [--host H] [--port P] [--db PATH]
|
||||
|
||||
The PRD sequences the orchestrator as a plain-process dev-harness first, so
|
||||
the consolidation core (registry + attribution + HTTP control plane + live
|
||||
@@ -17,13 +16,12 @@ import secrets
|
||||
from pathlib import Path
|
||||
|
||||
from .. import log
|
||||
from ..trust_domain import CONTROL_PLANE
|
||||
from .broker import LaunchBroker, StubBroker
|
||||
from .docker_broker import DockerBroker
|
||||
from .server import make_server
|
||||
from .service import OrchestratorCore
|
||||
from .store.store_manager import StoreManager
|
||||
from .broker import LaunchBroker, StubBroker
|
||||
from .server import make_server
|
||||
from .docker_broker import DockerBroker
|
||||
from .store.registry_store import RegistryStore, default_db_path
|
||||
from .service import OrchestratorCore
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
@@ -40,11 +38,6 @@ def main(argv: list[str] | None = None) -> int:
|
||||
help="launch broker: 'stub' records requests; 'docker' runs containers",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
if not CONTROL_PLANE.key_from_env():
|
||||
log.die(
|
||||
f"{CONTROL_PLANE.key_env} is required; refusing to start the "
|
||||
"orchestrator without caller authentication"
|
||||
)
|
||||
|
||||
registry = RegistryStore(args.db)
|
||||
registry.migrate()
|
||||
@@ -62,14 +55,17 @@ def main(argv: list[str] | None = None) -> int:
|
||||
orchestrator = OrchestratorCore(registry, broker, secret)
|
||||
|
||||
server = make_server(orchestrator, host=args.host, port=args.port)
|
||||
bound_host, bound_port = server.server_address[0], server.server_address[1]
|
||||
log.info(
|
||||
"orchestrator control plane listening",
|
||||
context={"host": args.host, "port": args.port, "db": str(registry.db_path)},
|
||||
context={"host": bound_host, "port": bound_port, "db": str(registry.db_path)},
|
||||
)
|
||||
try:
|
||||
server.run()
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
log.info("orchestrator shutting down")
|
||||
finally:
|
||||
server.server_close()
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
@@ -1,339 +0,0 @@
|
||||
"""FastAPI control-plane routes for the orchestrator."""
|
||||
# pyright: reportUnusedFunction=false
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import math
|
||||
import sys
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import BaseModel, ConfigDict, StrictStr
|
||||
from starlette.types import ASGIApp, Message, Receive, Scope, Send
|
||||
|
||||
from ..orchestrator_auth import ROLE_CLI, ROLES
|
||||
from ..supervisor.types import TOOLS
|
||||
from ..trust_domain import CONTROL_PLANE
|
||||
from .http_contract import (
|
||||
MAX_BODY_BYTES,
|
||||
ORCHESTRATOR_AUTH_HEADER,
|
||||
REQUEST_BODY_TIMEOUT_SECONDS,
|
||||
)
|
||||
from .service import OrchestratorCore
|
||||
|
||||
_GATEWAY_ROUTES = frozenset({
|
||||
("POST", "/resolve"),
|
||||
("POST", "/supervise/propose"),
|
||||
("POST", "/supervise/poll"),
|
||||
})
|
||||
|
||||
|
||||
class _StrictModel(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore", strict=True)
|
||||
|
||||
|
||||
class LaunchBody(_StrictModel):
|
||||
source_ip: StrictStr
|
||||
image_ref: StrictStr = ""
|
||||
metadata: StrictStr = ""
|
||||
policy: StrictStr = ""
|
||||
tokens: dict[StrictStr, StrictStr] = {}
|
||||
env_var_secret: StrictStr = ""
|
||||
|
||||
|
||||
class PolicyBody(_StrictModel):
|
||||
policy: StrictStr
|
||||
|
||||
|
||||
class ReprovisionBody(_StrictModel):
|
||||
env_var_secret: StrictStr
|
||||
|
||||
|
||||
class ReconcileBody(_StrictModel):
|
||||
live_source_ips: list[StrictStr]
|
||||
grace_seconds: float | None = None
|
||||
|
||||
|
||||
class IdentityBody(_StrictModel):
|
||||
source_ip: StrictStr
|
||||
identity_token: StrictStr = ""
|
||||
|
||||
|
||||
class AttributeBody(_StrictModel):
|
||||
source_ip: StrictStr
|
||||
identity_token: StrictStr
|
||||
|
||||
|
||||
class RespondBody(_StrictModel):
|
||||
proposal_id: StrictStr
|
||||
bottle_slug: StrictStr
|
||||
decision: StrictStr
|
||||
notes: StrictStr = ""
|
||||
final_file: StrictStr | None = None
|
||||
|
||||
|
||||
class ProposeBody(IdentityBody):
|
||||
tool: StrictStr
|
||||
proposed_file: StrictStr
|
||||
justification: StrictStr
|
||||
|
||||
|
||||
class PollBody(IdentityBody):
|
||||
proposal_id: StrictStr
|
||||
|
||||
|
||||
class ControlPlaneBoundary:
|
||||
"""Reject unauthenticated and oversized requests before reading a body."""
|
||||
|
||||
def __init__(self, app: ASGIApp, signing_key: str) -> None:
|
||||
self.app = app
|
||||
self.signing_key = signing_key
|
||||
|
||||
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
|
||||
if scope["type"] != "http":
|
||||
await self.app(scope, receive, send)
|
||||
return
|
||||
method = scope["method"]
|
||||
route = scope["path"].rstrip("/") or "/"
|
||||
if not (method == "GET" and route == "/health"):
|
||||
headers = dict(scope["headers"])
|
||||
presented = headers.get(
|
||||
ORCHESTRATOR_AUTH_HEADER.encode(), b"",
|
||||
).decode(errors="ignore")
|
||||
role = CONTROL_PLANE.verify(presented, self.signing_key)
|
||||
if role is None:
|
||||
await self._reject(
|
||||
scope, send, 401, "control-plane authentication required",
|
||||
)
|
||||
return
|
||||
allowed = ROLES if (method, route) in _GATEWAY_ROUTES else {ROLE_CLI}
|
||||
if role not in allowed:
|
||||
await self._reject(scope, send, 403, "insufficient role for this route")
|
||||
return
|
||||
scope.setdefault("state", {})["role"] = role
|
||||
raw_length = dict(scope["headers"]).get(b"content-length")
|
||||
if raw_length is not None:
|
||||
try:
|
||||
length = int(raw_length)
|
||||
except ValueError:
|
||||
await self._reject(scope, send, 400, "invalid Content-Length")
|
||||
return
|
||||
if length < 0:
|
||||
await self._reject(scope, send, 400, "invalid Content-Length")
|
||||
return
|
||||
if length > MAX_BODY_BYTES:
|
||||
await self._reject(scope, send, 413, "request body too large")
|
||||
return
|
||||
try:
|
||||
body = await self._read_body(receive)
|
||||
except _BodyTooLarge:
|
||||
await self._reject(scope, send, 413, "request body too large")
|
||||
return
|
||||
except TimeoutError:
|
||||
await self._reject(scope, send, 408, "request body read timed out")
|
||||
return
|
||||
try:
|
||||
await self.app(scope, self._replay_body(body), send)
|
||||
except Exception as exc: # noqa: BLE001 - redact control-plane failures
|
||||
sys.stderr.write(
|
||||
f"orchestrator: {method} {route} failed "
|
||||
f"[error_type={type(exc).__name__}]\n"
|
||||
)
|
||||
sys.stderr.flush()
|
||||
await self._reject(scope, send, 500, "internal error")
|
||||
|
||||
@staticmethod
|
||||
async def _reject(
|
||||
scope: Scope, send: Send, status: int, error: str,
|
||||
) -> None:
|
||||
response = JSONResponse({"error": error}, status_code=status)
|
||||
await response(scope, ControlPlaneBoundary._empty_receive, send)
|
||||
|
||||
@staticmethod
|
||||
async def _empty_receive() -> Message:
|
||||
return {"type": "http.disconnect"}
|
||||
|
||||
@staticmethod
|
||||
async def _read_body(receive: Receive) -> bytes:
|
||||
body = bytearray()
|
||||
async with asyncio.timeout(REQUEST_BODY_TIMEOUT_SECONDS):
|
||||
while True:
|
||||
message = await receive()
|
||||
if message["type"] != "http.request":
|
||||
break
|
||||
body.extend(message.get("body", b""))
|
||||
if len(body) > MAX_BODY_BYTES:
|
||||
raise _BodyTooLarge
|
||||
if not message.get("more_body", False):
|
||||
break
|
||||
return bytes(body)
|
||||
|
||||
@staticmethod
|
||||
def _replay_body(body: bytes) -> Receive:
|
||||
sent = False
|
||||
|
||||
async def replay() -> Message:
|
||||
nonlocal sent
|
||||
if sent:
|
||||
return {"type": "http.disconnect"}
|
||||
sent = True
|
||||
return {"type": "http.request", "body": body, "more_body": False}
|
||||
|
||||
return replay
|
||||
|
||||
|
||||
class _BodyTooLarge(Exception):
|
||||
"""The streamed request exceeded the control-plane body limit."""
|
||||
|
||||
|
||||
def _required(value: str, name: str) -> str:
|
||||
if not value:
|
||||
raise HTTPException(400, f"{name} (string) is required")
|
||||
return value
|
||||
|
||||
|
||||
def create_app(orch: OrchestratorCore, *, signing_key: str) -> FastAPI:
|
||||
"""Build the authenticated orchestrator ASGI application."""
|
||||
key = signing_key.strip()
|
||||
if not key:
|
||||
raise ValueError(
|
||||
"orchestrator control-plane signing key is required; "
|
||||
"refusing to start without caller authentication"
|
||||
)
|
||||
app = FastAPI(
|
||||
title="bot-bottle orchestrator",
|
||||
docs_url=None,
|
||||
redoc_url=None,
|
||||
openapi_url=None,
|
||||
)
|
||||
app.add_middleware(ControlPlaneBoundary, signing_key=key)
|
||||
|
||||
@app.get("/health")
|
||||
def health() -> dict[str, str]:
|
||||
return {"status": "ok"}
|
||||
|
||||
@app.get("/gateway")
|
||||
def gateway() -> dict[str, object]:
|
||||
return orch.gateway_status()
|
||||
|
||||
@app.get("/bottles")
|
||||
def bottles() -> dict[str, object]:
|
||||
return {"bottles": [record.redacted() for record in orch.registry.all()]}
|
||||
|
||||
@app.post("/bottles", status_code=201)
|
||||
def launch(body: LaunchBody) -> dict[str, str]:
|
||||
rec = orch.launch_bottle(
|
||||
_required(body.source_ip, "source_ip"),
|
||||
image_ref=body.image_ref,
|
||||
metadata=body.metadata,
|
||||
policy=body.policy,
|
||||
tokens=dict(body.tokens),
|
||||
env_var_secret=body.env_var_secret,
|
||||
)
|
||||
return {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
|
||||
|
||||
@app.put("/bottles/{bottle_id}/policy")
|
||||
def set_policy(bottle_id: str, body: PolicyBody) -> dict[str, object]:
|
||||
if orch.set_policy(bottle_id, body.policy):
|
||||
return {"updated": True}
|
||||
raise HTTPException(404, "no such bottle")
|
||||
|
||||
@app.post("/bottles/{bottle_id}/reprovision_gateway")
|
||||
def reprovision(bottle_id: str, body: ReprovisionBody) -> dict[str, object]:
|
||||
secret = _required(body.env_var_secret, "env_var_secret")
|
||||
if orch.reprovision_from_secret(bottle_id, secret):
|
||||
return {"reprovisioned": True}
|
||||
raise HTTPException(404, "no stored secrets for this bottle")
|
||||
|
||||
@app.delete("/bottles/{bottle_id}")
|
||||
def teardown(bottle_id: str) -> dict[str, object]:
|
||||
if orch.teardown_bottle(bottle_id):
|
||||
return {"torn_down": True}
|
||||
raise HTTPException(404, "no such bottle")
|
||||
|
||||
@app.post("/reconcile")
|
||||
def reconcile(body: ReconcileBody) -> dict[str, object]:
|
||||
if any(not ip for ip in body.live_source_ips):
|
||||
raise HTTPException(400, "live_source_ips must contain non-empty strings")
|
||||
kwargs: dict[str, float] = {}
|
||||
if body.grace_seconds is not None:
|
||||
if not math.isfinite(body.grace_seconds) or body.grace_seconds < 0:
|
||||
raise HTTPException(
|
||||
400, "grace_seconds must be a non-negative finite number",
|
||||
)
|
||||
kwargs["grace_seconds"] = body.grace_seconds
|
||||
return {"reaped": orch.reconcile(body.live_source_ips, **kwargs)}
|
||||
|
||||
@app.post("/attribute")
|
||||
def attribute(body: AttributeBody) -> dict[str, str]:
|
||||
rec = orch.attribute(body.source_ip, body.identity_token)
|
||||
if rec is None:
|
||||
raise HTTPException(403, "unattributed")
|
||||
return {"bottle_id": rec.bottle_id}
|
||||
|
||||
@app.get("/supervise/proposals")
|
||||
def proposals() -> dict[str, object]:
|
||||
return {"proposals": orch.supervise_pending()}
|
||||
|
||||
@app.post("/supervise/respond")
|
||||
def respond(body: RespondBody) -> dict[str, object]:
|
||||
ok, error = orch.supervise_respond(
|
||||
_required(body.proposal_id, "proposal_id"),
|
||||
bottle_slug=_required(body.bottle_slug, "bottle_slug"),
|
||||
decision=_required(body.decision, "decision"),
|
||||
notes=body.notes,
|
||||
final_file=body.final_file,
|
||||
)
|
||||
if not ok:
|
||||
raise HTTPException(409, error)
|
||||
return {"responded": True}
|
||||
|
||||
@app.post("/supervise/propose", status_code=201)
|
||||
def propose(body: ProposeBody) -> dict[str, str]:
|
||||
source_ip = _required(body.source_ip, "source_ip")
|
||||
if body.tool not in TOOLS:
|
||||
raise HTTPException(400, f"tool (string) must be one of {TOOLS}")
|
||||
rec = orch.resolve(source_ip, body.identity_token)
|
||||
if rec is None:
|
||||
raise HTTPException(403, "unattributed")
|
||||
proposal_id = orch.supervise_queue_proposal(
|
||||
rec.bottle_id,
|
||||
tool=body.tool,
|
||||
proposed_file=_required(body.proposed_file, "proposed_file"),
|
||||
justification=_required(body.justification, "justification"),
|
||||
)
|
||||
return {"proposal_id": proposal_id}
|
||||
|
||||
@app.post("/supervise/poll")
|
||||
def poll(body: PollBody) -> dict[str, object]:
|
||||
rec = orch.resolve(
|
||||
_required(body.source_ip, "source_ip"), body.identity_token,
|
||||
)
|
||||
if rec is None:
|
||||
raise HTTPException(403, "unattributed")
|
||||
return orch.supervise_poll_response(
|
||||
rec.bottle_id, _required(body.proposal_id, "proposal_id"),
|
||||
)
|
||||
|
||||
@app.post("/resolve")
|
||||
def resolve(body: IdentityBody) -> dict[str, object]:
|
||||
rec = orch.resolve(
|
||||
_required(body.source_ip, "source_ip"), body.identity_token,
|
||||
)
|
||||
if rec is None:
|
||||
raise HTTPException(403, "unattributed")
|
||||
return {
|
||||
"bottle_id": rec.bottle_id,
|
||||
"policy": rec.policy,
|
||||
"tokens": orch.tokens_for(rec.bottle_id),
|
||||
}
|
||||
|
||||
return app
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ControlPlaneBoundary",
|
||||
"MAX_BODY_BYTES",
|
||||
"ORCHESTRATOR_AUTH_HEADER",
|
||||
"create_app",
|
||||
]
|
||||
@@ -21,7 +21,7 @@ from dataclasses import dataclass
|
||||
from ..log import debug
|
||||
from ..orchestrator_auth import ROLE_CLI
|
||||
from ..trust_domain import CONTROL_PLANE
|
||||
from .http_contract import ORCHESTRATOR_AUTH_HEADER
|
||||
from .server import ORCHESTRATOR_AUTH_HEADER
|
||||
|
||||
DEFAULT_TIMEOUT_SECONDS = 5.0
|
||||
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
"""Dependency-free constants shared by orchestrator HTTP clients and server."""
|
||||
|
||||
ORCHESTRATOR_AUTH_HEADER = "x-bot-bottle-orchestrator-auth"
|
||||
MAX_BODY_BYTES = 1 * 1024 * 1024
|
||||
REQUEST_BODY_TIMEOUT_SECONDS = 10.0
|
||||
|
||||
__all__ = [
|
||||
"MAX_BODY_BYTES",
|
||||
"ORCHESTRATOR_AUTH_HEADER",
|
||||
"REQUEST_BODY_TIMEOUT_SECONDS",
|
||||
]
|
||||
@@ -1,80 +1,464 @@
|
||||
"""Uvicorn transport for the FastAPI orchestrator control plane."""
|
||||
"""Orchestrator HTTP control plane (PRD 0070).
|
||||
|
||||
The backend-agnostic control-plane RPC (CLI / console -> orchestrator) over
|
||||
**HTTP** — the universal transport chosen in 0070 (works on every host; no
|
||||
vsock / unix-socket portability caveats):
|
||||
|
||||
GET /health -> 200 {"status": "ok"}
|
||||
GET /gateway -> 200 {"configured", ["name","running"]}
|
||||
GET /bottles -> 200 {"bottles": [ <redacted record>, ...]}
|
||||
POST /bottles -> 201 {"bottle_id","identity_token"} (launch)
|
||||
body: {"source_ip", ["image_ref"],
|
||||
["metadata"], ["policy"],
|
||||
["tokens"], ["env_var_secret"]}
|
||||
PUT /bottles/<bottle_id>/policy -> 200 {"updated": true} | 404 (live reload)
|
||||
body: {"policy"}
|
||||
POST /bottles/<bottle_id>/reprovision_gateway
|
||||
-> 200 {"reprovisioned": true} | 404
|
||||
body: {"env_var_secret"}
|
||||
DELETE /bottles/<bottle_id> -> 200 {"torn_down": true} | 404 (teardown)
|
||||
POST /reconcile -> 200 {"reaped": [bottle_id, ...]}
|
||||
body: {"live_source_ips": [...],
|
||||
["grace_seconds"]}
|
||||
POST /attribute -> 200 {"bottle_id"} | 403
|
||||
POST /resolve -> 200 {"bottle_id","policy"} | 403
|
||||
body: {"source_ip","identity_token"}
|
||||
GET /supervise/proposals -> 200 {"proposals": [ <proposal>, ...]}
|
||||
POST /supervise/respond -> 200 {"responded": true} | 409 (operator)
|
||||
body: {"proposal_id","bottle_slug",
|
||||
"decision", ["notes"],["final_file"]}
|
||||
POST /supervise/propose -> 201 {"proposal_id"} | 403 (agent)
|
||||
body: {"source_ip","identity_token",
|
||||
"tool","proposed_file","justification"}
|
||||
POST /supervise/poll -> 200 {"status", ["notes"],["final_file"]} | 403
|
||||
body: {"source_ip","identity_token",
|
||||
"proposal_id"}
|
||||
|
||||
The `/supervise/propose` + `/supervise/poll` pair is the **agent** half of the
|
||||
supervise flow: the data plane (supervise / egress / git-gate) queues a proposal
|
||||
and polls for its response over RPC instead of opening `bot-bottle.db` directly.
|
||||
`poll` is idempotent — it never archives, so a dropped connection can't lose an
|
||||
operator decision (the row is reaped when the bottle is torn down / reconciled).
|
||||
Both attribute the caller by `(source_ip, identity_token)` exactly like
|
||||
`/resolve`, so a bottle can only ever queue or read its own proposals.
|
||||
|
||||
`POST /bottles` / `DELETE` drive the full launch lifecycle: they mint (or
|
||||
tear down) the bottle in the registry AND broker the backend-native launch
|
||||
via the orchestrator. Register/deregister without a launch are internal to
|
||||
`OrchestratorCore`, not exposed here.
|
||||
|
||||
Routing/handling is the pure function `dispatch()` so it is unit-testable
|
||||
without a socket; `Handler` / `OrchestratorServer` / `make_server` are a
|
||||
thin stdlib adapter around it. Listing redacts identity tokens — they are
|
||||
returned only once, to the caller that launches the bottle.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import http.server
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import socket
|
||||
import threading
|
||||
|
||||
import uvicorn
|
||||
import socketserver
|
||||
import sys
|
||||
import typing
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from ..orchestrator_auth import ROLE_CLI, ROLES
|
||||
from ..trust_domain import CONTROL_PLANE
|
||||
from .api import create_app
|
||||
from .http_contract import MAX_BODY_BYTES, ORCHESTRATOR_AUTH_HEADER
|
||||
from ..supervisor.types import TOOLS
|
||||
from .service import OrchestratorCore
|
||||
|
||||
MAX_REQUESTS = 32
|
||||
KEEP_ALIVE_TIMEOUT_SECONDS = 10
|
||||
# JSON body payload type (parsed request / rendered response).
|
||||
Json = dict[str, object]
|
||||
|
||||
# The request header carrying the caller's role-scoped control-plane token (a
|
||||
# signed JWT naming the caller's role — see orchestrator_auth). The role gates which
|
||||
# routes the caller may reach: the data plane holds a `gateway` token good only
|
||||
# for the agent-facing lookups; the host CLI holds a `cli` token for the
|
||||
# operator/mutating routes. An agent that can merely *reach* the port holds no
|
||||
# token at all, and a compromised gateway holds only `gateway` — neither can
|
||||
# drive the operator routes (approve proposals, rewrite policy, read tokens).
|
||||
ORCHESTRATOR_AUTH_HEADER = "x-bot-bottle-orchestrator-auth"
|
||||
|
||||
# The routes the data plane (role `gateway`) is allowed to reach — exactly the
|
||||
# per-request lookups PolicyResolver makes. Every other authenticated route is
|
||||
# operator-only. `cli` is a superset role: it may reach any route.
|
||||
_GATEWAY_ROUTES: frozenset[tuple[str, str]] = frozenset({
|
||||
("POST", "/resolve"),
|
||||
("POST", "/supervise/propose"),
|
||||
("POST", "/supervise/poll"),
|
||||
})
|
||||
|
||||
|
||||
class OrchestratorServer:
|
||||
"""Small lifecycle wrapper around Uvicorn with an eagerly bound socket."""
|
||||
def _allowed_roles(method: str, route: str) -> frozenset[str]:
|
||||
"""The roles permitted on `(method, route)`: `gateway` or `cli` on the
|
||||
data-plane routes, `cli`-only everywhere else."""
|
||||
if (method, route) in _GATEWAY_ROUTES:
|
||||
return ROLES
|
||||
return frozenset({ROLE_CLI})
|
||||
|
||||
def __init__(self, config: uvicorn.Config) -> None:
|
||||
self._server = uvicorn.Server(config)
|
||||
self._stopped = threading.Event()
|
||||
self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
self._socket.bind((config.host, config.port))
|
||||
self._socket.listen(config.backlog)
|
||||
self.server_address = self._socket.getsockname()
|
||||
|
||||
def run(self) -> None:
|
||||
def _parse_json_object(body: bytes) -> Json:
|
||||
"""Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
|
||||
if not body:
|
||||
return {}
|
||||
obj = json.loads(body) # raises json.JSONDecodeError (a ValueError)
|
||||
if not isinstance(obj, dict):
|
||||
raise ValueError("request body must be a JSON object")
|
||||
return obj
|
||||
|
||||
|
||||
def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
|
||||
orch: OrchestratorCore, method: str, path: str, body: bytes, *, role: str | None = ROLE_CLI,
|
||||
) -> tuple[int, Json]:
|
||||
"""Route one control-plane request to a (status, payload) pair. Pure —
|
||||
no I/O beyond the orchestrator — so it is fully testable without a socket.
|
||||
|
||||
`role` is the caller's verified control-plane role (`gateway` or `cli`), or
|
||||
None for an unauthenticated request; an open-mode server (no signing key
|
||||
configured — see `OrchestratorServer`) passes `cli`. Every route except
|
||||
`GET /health` requires a role: a missing role is 401, and a role that
|
||||
doesn't cover the route is 403 — so a `gateway` data-plane token can reach
|
||||
`/resolve` + `/supervise/{propose,poll}` but not the operator routes
|
||||
(rewrite policy, read injected tokens, approve its own supervise proposals).
|
||||
The source-IP + identity-token checks inside `/resolve` and `/attribute`
|
||||
authenticate the *bottle* a request is about, not the *caller*, so this role
|
||||
gate is what protects the caller-privileged routes. Defaults `cli` so unit
|
||||
tests of the routing logic don't have to thread it through."""
|
||||
route = urlsplit(path).path.rstrip("/") or "/"
|
||||
|
||||
if method == "GET" and route == "/health":
|
||||
return 200, {"status": "ok"}
|
||||
|
||||
# Role gate — every route below is a trusted-caller operation. Deny before
|
||||
# touching the registry / broker / supervise store.
|
||||
if role is None:
|
||||
return 401, {"error": "control-plane authentication required"}
|
||||
if role not in _allowed_roles(method, route):
|
||||
return 403, {"error": "insufficient role for this route"}
|
||||
|
||||
if method == "GET" and route == "/gateway":
|
||||
return 200, orch.gateway_status()
|
||||
|
||||
if method == "GET" and route == "/bottles":
|
||||
return 200, {"bottles": [r.redacted() for r in orch.registry.all()]}
|
||||
|
||||
if method == "POST" and route == "/bottles":
|
||||
try:
|
||||
self._server.run(sockets=[self._socket])
|
||||
finally:
|
||||
self._stopped.set()
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
source_ip = data.get("source_ip")
|
||||
if not isinstance(source_ip, str) or not source_ip:
|
||||
return 400, {"error": "source_ip (string) is required"}
|
||||
image_ref = data.get("image_ref")
|
||||
metadata = data.get("metadata")
|
||||
policy = data.get("policy")
|
||||
raw_tokens = data.get("tokens")
|
||||
tokens = {
|
||||
k: v for k, v in raw_tokens.items() if isinstance(k, str) and isinstance(v, str)
|
||||
} if isinstance(raw_tokens, dict) else {}
|
||||
env_var_secret = data.get("env_var_secret", "")
|
||||
rec = orch.launch_bottle(
|
||||
source_ip,
|
||||
image_ref=image_ref if isinstance(image_ref, str) else "",
|
||||
metadata=metadata if isinstance(metadata, str) else "",
|
||||
policy=policy if isinstance(policy, str) else "",
|
||||
tokens=tokens,
|
||||
env_var_secret=env_var_secret if isinstance(env_var_secret, str) else "",
|
||||
)
|
||||
return 201, {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
|
||||
|
||||
def serve_forever(self) -> None:
|
||||
self.run()
|
||||
if method == "PUT" and route.startswith("/bottles/") and route.endswith("/policy"):
|
||||
bottle_id = route[len("/bottles/"):-len("/policy")]
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
policy = data.get("policy")
|
||||
if not isinstance(policy, str):
|
||||
return 400, {"error": "policy (string) is required"}
|
||||
if orch.set_policy(bottle_id, policy):
|
||||
return 200, {"updated": True}
|
||||
return 404, {"error": "no such bottle"}
|
||||
|
||||
def shutdown(self) -> None:
|
||||
self._server.should_exit = True
|
||||
self._stopped.wait(timeout=5)
|
||||
if (
|
||||
method == "POST"
|
||||
and route.startswith("/bottles/")
|
||||
and route.endswith("/reprovision_gateway")
|
||||
):
|
||||
bottle_id = route[len("/bottles/") : -len("/reprovision_gateway")]
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
env_var_secret = data.get("env_var_secret")
|
||||
if not isinstance(env_var_secret, str) or not env_var_secret:
|
||||
return 400, {"error": "env_var_secret (string) is required"}
|
||||
if orch.reprovision_from_secret(bottle_id, env_var_secret):
|
||||
return 200, {"reprovisioned": True}
|
||||
return 404, {"error": "no stored secrets for this bottle"}
|
||||
|
||||
def server_close(self) -> None:
|
||||
self._socket.close()
|
||||
if method == "DELETE" and route.startswith("/bottles/"):
|
||||
bottle_id = route[len("/bottles/"):]
|
||||
if orch.teardown_bottle(bottle_id):
|
||||
return 200, {"torn_down": True}
|
||||
return 404, {"error": "no such bottle"}
|
||||
|
||||
if method == "POST" and route == "/reconcile":
|
||||
# Host-driven self-heal: the caller enumerates its live bottles (only
|
||||
# the host can see the backend) and the orchestrator drops rows for
|
||||
# every other active bottle. Trusted-caller only — an agent that could
|
||||
# reach this would be able to unregister its neighbours.
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
raw_ips = data.get("live_source_ips")
|
||||
if not isinstance(raw_ips, list):
|
||||
return 400, {"error": "live_source_ips (list of strings) is required"}
|
||||
if any(not isinstance(ip, str) or not ip for ip in raw_ips):
|
||||
return 400, {"error": "live_source_ips must contain non-empty strings"}
|
||||
live = raw_ips
|
||||
grace = data.get("grace_seconds")
|
||||
kwargs: dict[str, float] = {}
|
||||
if grace is not None:
|
||||
if isinstance(grace, bool) or not isinstance(grace, (int, float)):
|
||||
return 400, {"error": "grace_seconds must be a non-negative finite number"}
|
||||
parsed_grace = float(grace)
|
||||
if not math.isfinite(parsed_grace) or parsed_grace < 0:
|
||||
return 400, {"error": "grace_seconds must be a non-negative finite number"}
|
||||
kwargs["grace_seconds"] = parsed_grace
|
||||
return 200, {"reaped": orch.reconcile(live, **kwargs)}
|
||||
|
||||
if method == "POST" and route == "/attribute":
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
source_ip = data.get("source_ip")
|
||||
token = data.get("identity_token")
|
||||
if not isinstance(source_ip, str) or not isinstance(token, str):
|
||||
return 400, {"error": "source_ip and identity_token (strings) required"}
|
||||
rec = orch.attribute(source_ip, token)
|
||||
if rec is None:
|
||||
return 403, {"error": "unattributed"}
|
||||
return 200, {"bottle_id": rec.bottle_id}
|
||||
|
||||
if method == "GET" and route == "/supervise/proposals":
|
||||
# Operator TUI: pending supervise proposals across all bottles.
|
||||
return 200, {"proposals": orch.supervise_pending()}
|
||||
|
||||
if method == "POST" and route == "/supervise/respond":
|
||||
# Operator decision: apply (approve/modify rewrites egress policy),
|
||||
# write the queued response, audit — all server-side on the one DB.
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
proposal_id = data.get("proposal_id")
|
||||
bottle_slug = data.get("bottle_slug")
|
||||
decision = data.get("decision")
|
||||
if not (isinstance(proposal_id, str) and proposal_id):
|
||||
return 400, {"error": "proposal_id (string) is required"}
|
||||
if not (isinstance(bottle_slug, str) and bottle_slug):
|
||||
return 400, {"error": "bottle_slug (string) is required"}
|
||||
if not (isinstance(decision, str) and decision):
|
||||
return 400, {"error": "decision (string) is required"}
|
||||
notes = data.get("notes")
|
||||
final_file = data.get("final_file")
|
||||
ok, err = orch.supervise_respond(
|
||||
proposal_id,
|
||||
bottle_slug=bottle_slug,
|
||||
decision=decision,
|
||||
notes=notes if isinstance(notes, str) else "",
|
||||
final_file=final_file if isinstance(final_file, str) else None,
|
||||
)
|
||||
if ok:
|
||||
return 200, {"responded": True}
|
||||
return 409, {"error": err}
|
||||
|
||||
if method == "POST" and route == "/supervise/propose":
|
||||
# Agent half: queue a proposal, attributed to the caller resolved from
|
||||
# (source_ip, identity_token) — never a caller-supplied slug — so the
|
||||
# data plane can't forge attribution. Fail-closed 403 when unattributed.
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
source_ip = data.get("source_ip")
|
||||
token = data.get("identity_token")
|
||||
tool = data.get("tool")
|
||||
proposed_file = data.get("proposed_file")
|
||||
justification = data.get("justification")
|
||||
if not isinstance(source_ip, str) or not source_ip:
|
||||
return 400, {"error": "source_ip (string) is required"}
|
||||
if not isinstance(tool, str) or tool not in TOOLS:
|
||||
return 400, {"error": f"tool (string) must be one of {TOOLS}"}
|
||||
if not isinstance(proposed_file, str) or not proposed_file:
|
||||
return 400, {"error": "proposed_file (string) is required"}
|
||||
if not isinstance(justification, str) or not justification:
|
||||
return 400, {"error": "justification (string) is required"}
|
||||
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
|
||||
if rec is None:
|
||||
return 403, {"error": "unattributed"}
|
||||
proposal_id = orch.supervise_queue_proposal(
|
||||
rec.bottle_id, tool=tool, proposed_file=proposed_file,
|
||||
justification=justification,
|
||||
)
|
||||
return 201, {"proposal_id": proposal_id}
|
||||
|
||||
if method == "POST" and route == "/supervise/poll":
|
||||
# Agent half: non-blocking read of the caller's own proposal decision.
|
||||
# Attributed like /propose, and scoped to the resolved bottle id, so a
|
||||
# guessed proposal_id can never read another bottle's response.
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
source_ip = data.get("source_ip")
|
||||
token = data.get("identity_token")
|
||||
proposal_id = data.get("proposal_id")
|
||||
if not isinstance(source_ip, str) or not source_ip:
|
||||
return 400, {"error": "source_ip (string) is required"}
|
||||
if not isinstance(proposal_id, str) or not proposal_id:
|
||||
return 400, {"error": "proposal_id (string) is required"}
|
||||
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
|
||||
if rec is None:
|
||||
return 403, {"error": "unattributed"}
|
||||
return 200, orch.supervise_poll_response(rec.bottle_id, proposal_id)
|
||||
|
||||
if method == "POST" and route == "/resolve":
|
||||
# The per-request lookup the multi-tenant gateway makes: returns the
|
||||
# bottle's policy. Requires a matching (source_ip, identity_token)
|
||||
# pair — a missing/empty/mismatched token fail-closes (403), no
|
||||
# source-IP-only fallback.
|
||||
try:
|
||||
data = _parse_json_object(body)
|
||||
except ValueError as e:
|
||||
return 400, {"error": f"invalid JSON: {e}"}
|
||||
source_ip = data.get("source_ip")
|
||||
token = data.get("identity_token")
|
||||
if not isinstance(source_ip, str) or not source_ip:
|
||||
return 400, {"error": "source_ip (string) is required"}
|
||||
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
|
||||
if rec is None:
|
||||
return 403, {"error": "unattributed"}
|
||||
# tokens are the in-memory per-bottle egress auth values the gateway
|
||||
# injects; served here, never persisted.
|
||||
return 200, {
|
||||
"bottle_id": rec.bottle_id,
|
||||
"policy": rec.policy,
|
||||
"tokens": orch.tokens_for(rec.bottle_id),
|
||||
}
|
||||
|
||||
return 404, {"error": "not found"}
|
||||
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
"""Thin stdlib adapter: read the body, call `dispatch`, write JSON."""
|
||||
|
||||
# Quiet by default (the orchestrator has its own logging); opt back into
|
||||
# stdlib access logging with BOT_BOTTLE_ORCHESTRATOR_DEBUG.
|
||||
def log_message(self, format: str, *args: typing.Any) -> None: # noqa: A002
|
||||
if os.environ.get("BOT_BOTTLE_ORCHESTRATOR_DEBUG"):
|
||||
super().log_message(format, *args)
|
||||
|
||||
def _serve(self, method: str) -> None:
|
||||
"""Read the request body, dispatch it, and write the JSON reply. A
|
||||
dispatch failure (e.g. a broker error) returns a 500 rather than
|
||||
crashing the connection, so one bad request can't take the control
|
||||
plane down for the caller."""
|
||||
server = self.server
|
||||
assert isinstance(server, OrchestratorServer)
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
body = self.rfile.read(length) if length > 0 else b""
|
||||
role = server.role_for(self.headers.get(ORCHESTRATOR_AUTH_HEADER, ""))
|
||||
try:
|
||||
status, payload = dispatch(
|
||||
server.orchestrator, method, self.path, body, role=role)
|
||||
except Exception as e: # noqa: BLE001 — the control plane must stay up
|
||||
# Do not echo exception messages to the caller or logs: broker and
|
||||
# persistence exceptions can contain request data. The operation,
|
||||
# route, and exception type are enough to correlate a traceback.
|
||||
sys.stderr.write(
|
||||
f"orchestrator: {method} {self.path} failed "
|
||||
f"[error_type={type(e).__name__}]\n"
|
||||
)
|
||||
sys.stderr.flush()
|
||||
status, payload = 500, {"error": "internal error"}
|
||||
data = json.dumps(payload).encode()
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
def do_GET(self) -> None:
|
||||
self._serve("GET")
|
||||
|
||||
def do_POST(self) -> None:
|
||||
self._serve("POST")
|
||||
|
||||
def do_PUT(self) -> None:
|
||||
self._serve("PUT")
|
||||
|
||||
def do_DELETE(self) -> None:
|
||||
self._serve("DELETE")
|
||||
|
||||
|
||||
class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
||||
"""Threading HTTP server that carries the orchestrator for its handlers.
|
||||
|
||||
Holds the per-host control-plane *signing key* (from
|
||||
`$BOT_BOTTLE_ORCHESTRATOR_TOKEN`, injected by the launcher into the
|
||||
orchestrator process only) and verifies each request's role-scoped token
|
||||
against it. When a key is set, every route but `/health` requires a valid
|
||||
token whose role covers the route; when it is unset the server runs **open**
|
||||
(full `cli` access) and says so loudly at startup — a fail-visible fallback
|
||||
for tests and any backend that hasn't wired the key yet (e.g. Firecracker,
|
||||
whose nft boundary already blocks agents from the control-plane port)."""
|
||||
|
||||
daemon_threads = True
|
||||
allow_reuse_address = True
|
||||
|
||||
def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None:
|
||||
self.orchestrator = orchestrator
|
||||
# The control-plane trust domain's signing key, as injected into THIS
|
||||
# (the owning) process by the launcher (#476). Unset → open mode below.
|
||||
self._signing_key = CONTROL_PLANE.key_from_env()
|
||||
if not self._signing_key:
|
||||
sys.stderr.write(
|
||||
"orchestrator: WARNING — no control-plane signing key "
|
||||
f"(${CONTROL_PLANE.key_env}); running WITHOUT caller "
|
||||
"authentication. Any client that can reach this port can drive "
|
||||
"it. Backends that put the control plane on an agent-reachable "
|
||||
"network MUST set this.\n"
|
||||
)
|
||||
sys.stderr.flush()
|
||||
super().__init__(address, Handler)
|
||||
|
||||
def role_for(self, presented: str) -> str | None:
|
||||
"""The role the request is authorized as, or None if unauthenticated.
|
||||
Open mode (no signing key) grants full `cli` access — the fail-visible
|
||||
fallback. Otherwise verify the presented signed token; a missing/invalid
|
||||
token yields None (→ 401), a valid one yields its `gateway`/`cli`
|
||||
role (→ per-route 401/403 in `dispatch`)."""
|
||||
if not self._signing_key:
|
||||
return ROLE_CLI
|
||||
return CONTROL_PLANE.verify(presented, self._signing_key)
|
||||
|
||||
|
||||
def make_server(
|
||||
orchestrator: OrchestratorCore,
|
||||
host: str = "127.0.0.1",
|
||||
port: int = 0,
|
||||
*,
|
||||
signing_key: str | None = None,
|
||||
orchestrator: OrchestratorCore, host: str = "127.0.0.1", port: int = 0
|
||||
) -> OrchestratorServer:
|
||||
"""Build a bounded Uvicorn server around the orchestrator application."""
|
||||
key = CONTROL_PLANE.key_from_env() if signing_key is None else signing_key
|
||||
app = create_app(orchestrator, signing_key=key)
|
||||
config = uvicorn.Config(
|
||||
app,
|
||||
host=host,
|
||||
port=port,
|
||||
access_log=bool(os.environ.get("BOT_BOTTLE_ORCHESTRATOR_DEBUG")),
|
||||
log_level="info",
|
||||
limit_concurrency=MAX_REQUESTS,
|
||||
timeout_keep_alive=KEEP_ALIVE_TIMEOUT_SECONDS,
|
||||
server_header=False,
|
||||
)
|
||||
return OrchestratorServer(config)
|
||||
"""Build (but do not start) a control-plane server. `port=0` binds an
|
||||
ephemeral port — read `server.server_address` for the actual one."""
|
||||
return OrchestratorServer((host, port), orchestrator)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"KEEP_ALIVE_TIMEOUT_SECONDS",
|
||||
"MAX_BODY_BYTES",
|
||||
"MAX_REQUESTS",
|
||||
"dispatch", "Handler", "OrchestratorServer", "make_server", "Json",
|
||||
"ORCHESTRATOR_AUTH_HEADER",
|
||||
"OrchestratorServer",
|
||||
"create_app",
|
||||
"make_server",
|
||||
]
|
||||
|
||||
@@ -371,12 +371,10 @@ class OrchestratorCore:
|
||||
if not encrypted:
|
||||
return False
|
||||
try:
|
||||
decrypted = {
|
||||
k: decrypt_value(env_var_secret, v) for k, v in encrypted.items()
|
||||
}
|
||||
self._tokens[bottle_id] = {k: decrypt_value(env_var_secret, v)
|
||||
for k, v in encrypted.items()}
|
||||
except ValueError:
|
||||
return False
|
||||
self._tokens[bottle_id] = decrypted
|
||||
return True
|
||||
|
||||
# --- consolidated gateway ----------------------------------------------
|
||||
|
||||
@@ -129,10 +129,6 @@ _MIGRATIONS = TableMigrations(
|
||||
# v5 — index for fast per-bottle lookups and bulk DELETE on teardown.
|
||||
"CREATE INDEX IF NOT EXISTS idx_bottled_agent_secrets_id "
|
||||
"ON bottled_agent_secrets (bottled_agent_id, type)",
|
||||
# v6 — unauthenticated legacy ciphertext must never be selected by
|
||||
# attacker-controlled blob contents. Existing local agents are
|
||||
# intentionally reprovisioned instead of retaining downgrade support.
|
||||
"DELETE FROM bottled_agent_secrets",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@@ -12,15 +12,9 @@ reattachment path reads ENV_VAR_SECRET from the running agent container via
|
||||
``POST /bottles/<id>/reprovision_gateway``; the orchestrator decrypts the
|
||||
stored rows and re-populates ``_tokens``.
|
||||
|
||||
Encryption scheme: encrypt-then-MAC using independent HMAC-SHA256-derived
|
||||
encryption and authentication subkeys (stdlib-only, no external deps). Each
|
||||
value is encrypted independently. New output blobs are:
|
||||
|
||||
``version || nonce (16 bytes) || ciphertext || tag (32 bytes)``
|
||||
|
||||
encoded as URL-safe base64 (no padding). Unversioned legacy ciphertext is
|
||||
rejected; the registry migration clears those rows rather than allowing blob
|
||||
contents to select an unauthenticated decoder.
|
||||
Encryption scheme: HMAC-SHA256 used as a PRF in CTR mode (stdlib-only,
|
||||
no external deps). Each value is encrypted independently. The output blob is
|
||||
``nonce (16 bytes) || ciphertext`` encoded as URL-safe base64 (no padding).
|
||||
|
||||
keystream_block_i = HMAC-SHA256(key, nonce || i.to_bytes(4, "big"))
|
||||
ciphertext_i = plaintext_i XOR keystream_block_i[:len(plaintext_i)]
|
||||
@@ -36,8 +30,6 @@ import secrets
|
||||
_KEY_BYTES = 32 # 256-bit key from ENV_VAR_SECRET
|
||||
_NONCE_BYTES = 16 # 128-bit random nonce per encrypt call
|
||||
_BLOCK = 32 # HMAC-SHA256 output width == one keystream block
|
||||
_TAG_BYTES = 32
|
||||
_VERSION = b"BBSE1"
|
||||
|
||||
# Env-var name the agent container receives at startup.
|
||||
ENV_VAR_SECRET_NAME = "ENV_VAR_SECRET"
|
||||
@@ -49,13 +41,7 @@ def new_env_var_secret() -> str:
|
||||
|
||||
|
||||
def _b64dec(s: str) -> bytes:
|
||||
return base64.b64decode(
|
||||
s + "=" * (-len(s) % 4), altchars=b"-_", validate=True,
|
||||
)
|
||||
|
||||
|
||||
def _subkey(key: bytes, purpose: bytes) -> bytes:
|
||||
return hmac.new(key, b"bot-bottle-secret-store:" + purpose, hashlib.sha256).digest()
|
||||
return base64.urlsafe_b64decode(s + "=" * (-len(s) % 4))
|
||||
|
||||
|
||||
def _keystream(key: bytes, nonce: bytes, block_index: int) -> bytes:
|
||||
@@ -67,53 +53,37 @@ def _keystream(key: bytes, nonce: bytes, block_index: int) -> bytes:
|
||||
def encrypt_value(secret_b64: str, plaintext: str) -> str:
|
||||
"""Encrypt a single string value with *secret_b64* (the ENV_VAR_SECRET).
|
||||
|
||||
Returns a URL-safe base64 authenticated blob suitable for
|
||||
Returns a URL-safe base64 blob ``nonce || ciphertext`` suitable for
|
||||
the ``bottled_agent_secrets.value`` column."""
|
||||
key = _b64dec(secret_b64)
|
||||
encryption_key = _subkey(key, b"encryption")
|
||||
authentication_key = _subkey(key, b"authentication")
|
||||
pt = plaintext.encode()
|
||||
nonce = secrets.token_bytes(_NONCE_BYTES)
|
||||
ct = bytearray()
|
||||
for i in range(0, len(pt), _BLOCK):
|
||||
chunk = pt[i : i + _BLOCK]
|
||||
ks = _keystream(encryption_key, nonce, i // _BLOCK)[: len(chunk)]
|
||||
ks = _keystream(key, nonce, i)[: len(chunk)]
|
||||
ct.extend(p ^ k for p, k in zip(chunk, ks))
|
||||
authenticated = _VERSION + nonce + bytes(ct)
|
||||
tag = hmac.new(authentication_key, authenticated, hashlib.sha256).digest()
|
||||
return base64.urlsafe_b64encode(authenticated + tag).rstrip(b"=").decode()
|
||||
return base64.urlsafe_b64encode(nonce + bytes(ct)).rstrip(b"=").decode()
|
||||
|
||||
|
||||
def decrypt_value(secret_b64: str, blob_b64: str) -> str:
|
||||
"""Decrypt a blob produced by :func:`encrypt_value`.
|
||||
|
||||
Returns the original plaintext string. Raises ``ValueError`` for malformed
|
||||
input, authentication failure, or a key mismatch."""
|
||||
input or a key mismatch (wrong key produces garbage, not an error, unless
|
||||
the plaintext is non-UTF-8 — treat all such failures as wrong key)."""
|
||||
key = _b64dec(secret_b64)
|
||||
try:
|
||||
blob = _b64dec(blob_b64)
|
||||
except (ValueError, TypeError) as exc:
|
||||
except Exception as exc:
|
||||
raise ValueError(f"invalid ciphertext blob: {exc}") from exc
|
||||
if not blob.startswith(_VERSION):
|
||||
raise ValueError("unsupported ciphertext format")
|
||||
minimum = len(_VERSION) + _NONCE_BYTES + _TAG_BYTES
|
||||
if len(blob) < minimum:
|
||||
if len(blob) < _NONCE_BYTES:
|
||||
raise ValueError("ciphertext blob too short")
|
||||
authenticated, supplied_tag = blob[:-_TAG_BYTES], blob[-_TAG_BYTES:]
|
||||
authentication_key = _subkey(key, b"authentication")
|
||||
expected_tag = hmac.new(
|
||||
authentication_key, authenticated, hashlib.sha256,
|
||||
).digest()
|
||||
if not hmac.compare_digest(supplied_tag, expected_tag):
|
||||
raise ValueError("ciphertext authentication failed")
|
||||
nonce_start = len(_VERSION)
|
||||
nonce = blob[nonce_start : nonce_start + _NONCE_BYTES]
|
||||
ciphertext = blob[nonce_start + _NONCE_BYTES : -_TAG_BYTES]
|
||||
encryption_key = _subkey(key, b"encryption")
|
||||
nonce, ciphertext = blob[:_NONCE_BYTES], blob[_NONCE_BYTES:]
|
||||
pt = bytearray()
|
||||
for i in range(0, len(ciphertext), _BLOCK):
|
||||
chunk = ciphertext[i : i + _BLOCK]
|
||||
ks = _keystream(encryption_key, nonce, i // _BLOCK)[: len(chunk)]
|
||||
ks = _keystream(key, nonce, i)[: len(chunk)]
|
||||
pt.extend(c ^ k for c, k in zip(chunk, ks))
|
||||
try:
|
||||
return bytes(pt).decode()
|
||||
@@ -121,9 +91,4 @@ def decrypt_value(secret_b64: str, blob_b64: str) -> str:
|
||||
raise ValueError(f"decryption produced non-UTF-8 output (wrong key?): {exc}") from exc
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ENV_VAR_SECRET_NAME",
|
||||
"new_env_var_secret",
|
||||
"encrypt_value",
|
||||
"decrypt_value",
|
||||
]
|
||||
__all__ = ["ENV_VAR_SECRET_NAME", "new_env_var_secret", "encrypt_value", "decrypt_value"]
|
||||
|
||||
@@ -44,7 +44,6 @@ BUNDLED_RESOURCES: tuple[str, ...] = (
|
||||
"Dockerfile.orchestrator",
|
||||
"Dockerfile.orchestrator.fc",
|
||||
"requirements.gateway.lock",
|
||||
"requirements.orchestrator.lock",
|
||||
"nix/firecracker-netpool.nix",
|
||||
"scripts/firecracker-netpool.sh",
|
||||
)
|
||||
|
||||
@@ -40,7 +40,7 @@ from .paths import (
|
||||
|
||||
class ProvisioningError(RuntimeError):
|
||||
"""A control-plane auth invariant would be violated (e.g. starting the
|
||||
orchestrator without its signing key)."""
|
||||
orchestrator without its signing key — which would run OPEN)."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -67,8 +67,9 @@ class TrustDomain:
|
||||
|
||||
def key_from_env(self, environ: Mapping[str, str] | None = None) -> str:
|
||||
"""The signing key as the owning process sees it — read from `key_env`
|
||||
(default `os.environ`). ``""`` when unset; owning services reject that
|
||||
value rather than start without authentication."""
|
||||
(default `os.environ`). "" when unset; the caller decides whether that is
|
||||
fatal (`ControlPlaneProvisioning`) or the open-mode fallback
|
||||
(`OrchestratorServer`)."""
|
||||
env = os.environ if environ is None else environ
|
||||
return env.get(self.key_env, "").strip()
|
||||
|
||||
|
||||
@@ -1,9 +1,19 @@
|
||||
# PRD 0011: Per-file Markdown manifest
|
||||
|
||||
- **Status:** Active
|
||||
- **Status:** Active (agent cwd-discovery superseded)
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-05-24
|
||||
|
||||
> **Superseded in part by PRD 0082.**
|
||||
> The `$CWD/.bot-bottle/agents/<name>.md` discovery/override path described
|
||||
> below is removed: agents are now **home-only**, like bottles. Once an agent
|
||||
> definition can select a host identity (`author`) and a host forge secret
|
||||
> (`forge-accounts`), letting checked-out workspace content define or override
|
||||
> an agent would let untrusted content select host credentials. A cwd
|
||||
> `agents/` (or `bottles/`) directory is now warned-about and ignored. The
|
||||
> filesystem-layout trust boundary still holds — it just admits nothing from
|
||||
> `$CWD`.
|
||||
|
||||
## Summary
|
||||
|
||||
Replace the single-file `bot-bottle.json` manifest with a
|
||||
|
||||
@@ -56,7 +56,8 @@ key.
|
||||
- Rewriting the HMAC primitive: `orchestrator_auth.mint/verify` gain an optional
|
||||
`roles=` arg (default unchanged) so a key can carry a different role set;
|
||||
nothing else changes.
|
||||
- Network topology or the plane split (#469).
|
||||
- Network topology, the plane split (#469), or the server's open-mode fallback
|
||||
for tests.
|
||||
|
||||
## Design
|
||||
|
||||
|
||||
@@ -1,197 +0,0 @@
|
||||
# PRD 0082: Authoritative failure boundaries
|
||||
|
||||
- **Status:** Draft
|
||||
- **Author:** codex
|
||||
- **Created:** 2026-07-27
|
||||
- **Issue:** #444
|
||||
|
||||
## Summary
|
||||
|
||||
Make every security- or lifecycle-sensitive snapshot distinguish authoritative
|
||||
empty state from unavailable state, and make every destructive or
|
||||
resource-consuming boundary revalidate the assumptions it acts on. This
|
||||
finishes the focused quality work begun under #444 without broad rewrites:
|
||||
cleanup cannot act on stale identities, policy introspection cannot publish a
|
||||
fabricated empty policy, gateway servers bound untrusted work, and daemon
|
||||
shutdown does not emit uncaught background-thread failures. Shared
|
||||
control-plane storage and gateway credential provisioning also enforce their
|
||||
filesystem security contract before sensitive data is written.
|
||||
|
||||
## Problem
|
||||
|
||||
Several paths are individually fail-closed but compose into unsafe or
|
||||
misleading behavior:
|
||||
|
||||
1. `cleanup` prepares a plan, waits indefinitely for operator confirmation,
|
||||
then kills stored PIDs and removes stored paths without checking that those
|
||||
identities still describe the same orphan. A PID may be reused or a run
|
||||
directory may become active during the prompt.
|
||||
2. The supervisor reuses egress's deny-all fallback for
|
||||
`list-egress-routes`. Deny-all is correct for enforcement, but presenting it
|
||||
as a successful empty route table can cause a later replace-all proposal to
|
||||
discard live routes.
|
||||
3. The supervisor and Git HTTP services accept bounded declared body sizes but
|
||||
use blocking reads and unbounded request threads. An untrusted bottle can
|
||||
exhaust the shared gateway with slow or parallel requests.
|
||||
4. macOS cleanup enumerates containers and networks independently and treats a
|
||||
failed query as an empty class, so a partial snapshot can still become a
|
||||
destructive plan.
|
||||
5. Gateway log-pump threads race stream closure during shutdown and emit
|
||||
uncaught exceptions even when shutdown otherwise succeeds.
|
||||
6. Firecracker discovers VMs through whitespace-split `pgrep -a` output.
|
||||
A configured cache path containing spaces can hide a live VM from the
|
||||
snapshot and make its run directory appear orphaned.
|
||||
7. Docker cleanup asks compose for its project snapshot in best-effort mode.
|
||||
A transient query failure can therefore become an empty stopped-project
|
||||
set and authorize deletion of associated state directories.
|
||||
8. Firecracker artifact downloads and registry publication have no network
|
||||
deadline, so an unresponsive registry can hold setup or release work
|
||||
indefinitely.
|
||||
9. Authenticated secret blobs select the unauthenticated legacy decoder when
|
||||
their in-band version prefix is changed, allowing storage tampering to
|
||||
bypass tag verification.
|
||||
10. Cleanup executes the entire post-confirmation snapshot rather than the
|
||||
intersection with what the operator saw, and mutation failures are not
|
||||
reflected in the command result.
|
||||
11. Git smart-HTTP can retain sixteen 100 MiB request bodies concurrently,
|
||||
cleanup mutations have no subprocess deadline, and Firecracker signalling
|
||||
failures bypass shared mutation accounting.
|
||||
12. SQLite creates the shared control-plane database before its mode is
|
||||
restricted, then suppresses permission-repair failures. Gateway transports
|
||||
also differ in whether copied deploy-key modes are preserved.
|
||||
|
||||
These are one design problem: state used to authorize deletion, replacement,
|
||||
or resource allocation must be authoritative at the point of use.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- Cleanup never signals a PID or recursively deletes a path solely because it
|
||||
appeared in a pre-confirmation snapshot.
|
||||
- Firecracker cleanup proves immediately before action that a PID is still the
|
||||
same Firecracker process and that a run directory is still orphaned.
|
||||
- Firecracker process discovery reads NUL-delimited argv from `/proc`; paths
|
||||
are never reconstructed from whitespace-delimited process listings.
|
||||
- All backend cleanup discovery primitives raise a typed enumeration error on
|
||||
operational failure. No backend may independently continue from a partial
|
||||
snapshot.
|
||||
- Shared cleanup control flow lives in the backend layer; concrete backends
|
||||
override resource-specific discovery and validation primitives rather than
|
||||
each implementing a bespoke failure policy.
|
||||
- `list-egress-routes` returns an MCP error when attribution or policy
|
||||
resolution is unavailable. A genuine, authoritatively resolved empty policy
|
||||
remains a successful empty list.
|
||||
- Supervisor and Git HTTP request bodies have total read deadlines, and each
|
||||
service bounds concurrent request work. Limits apply to authenticated
|
||||
callers because bottles themselves are untrusted.
|
||||
- Gateway child-output pumping treats expected stream closure during shutdown
|
||||
as completion while preserving diagnostics for unexpected failures.
|
||||
- Artifact pull, existence-check, and publication requests use explicit
|
||||
network deadlines.
|
||||
- Persisted secrets accept only the authenticated format. The schema migration
|
||||
intentionally clears legacy rows; local agents are reprovisioned rather
|
||||
than retaining a ciphertext-controlled downgrade path.
|
||||
- Cleanup executes only resources present in both the displayed and current
|
||||
authoritative plans, attempts every approved mutation, and returns failure
|
||||
when any mutation does not complete.
|
||||
- Git request bodies spool to disk behind a separate heavy-work semaphore;
|
||||
cleanup commands have configurable deadlines; Firecracker signalling
|
||||
failures aggregate while identity-verification uncertainty still aborts.
|
||||
- The shared database directory and file are private before SQLite writes any
|
||||
control-plane state; an inability to enforce those modes aborts startup.
|
||||
- Gateway credential directories and files receive explicit private modes
|
||||
inside the gateway, independent of Docker, Apple Container, or SSH copy
|
||||
semantics.
|
||||
- Unit tests cover PID/path reuse, partial backend enumeration, transient
|
||||
policy resolution failure, slow bodies, concurrency saturation, and stream
|
||||
closure races.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Further decomposition solely to reduce module line counts.
|
||||
- Replacing gateway stdlib HTTP services with a web framework.
|
||||
- Changing egress matching, DLP decisions, proposal semantics, or backend
|
||||
launch behavior beyond the synchronization required for safe cleanup.
|
||||
- Making cleanup silently skip uncertain resources. Uncertainty is an
|
||||
operator-visible failure.
|
||||
|
||||
## Design
|
||||
|
||||
### Shared backend control flow
|
||||
|
||||
Follow the backend architecture rule used by gateway attachment: shared
|
||||
behavior lives above concrete backends; subclasses provide primitives, not
|
||||
control flow.
|
||||
|
||||
Cleanup remains previewable, but confirmation authorizes a *new authoritative
|
||||
evaluation*, not blind execution of the displayed object. The shared flow:
|
||||
|
||||
1. asks each available backend for a preview;
|
||||
2. displays the union and asks for confirmation;
|
||||
3. refreshes each non-empty backend plan;
|
||||
4. validates destructive identities immediately before action;
|
||||
5. aborts loudly if the refreshed plan or any identity cannot be proven safe.
|
||||
|
||||
Backend-specific primitives define how to identify a resource. Firecracker
|
||||
uses process start identity plus canonical config/run paths; container
|
||||
backends use authoritative CLI queries and stable resource names/labels.
|
||||
|
||||
Container engines expose destructive name-based commands without a portable
|
||||
compare-and-delete operation. Cleanup therefore refreshes after confirmation
|
||||
and requires every discovery query to succeed, minimizing but not claiming to
|
||||
eliminate the final name-reuse race. A future engine-specific stable-ID
|
||||
primitive may close that residual window without moving control flow back
|
||||
into each backend.
|
||||
|
||||
### Enforcement state versus introspection state
|
||||
|
||||
Egress enforcement retains its deny-all fallback because uncertainty must not
|
||||
grant network access. Supervisor introspection uses a strict resolver path:
|
||||
unattributed callers and resolver failures become typed MCP errors, while a
|
||||
successfully resolved policy containing zero routes returns `routes: []`.
|
||||
|
||||
### Gateway resource boundaries
|
||||
|
||||
Both stdlib servers set a per-connection body deadline before reading and use a
|
||||
bounded request executor or semaphore. Saturated capacity fails quickly with a
|
||||
service-unavailable response. Existing size caps remain independent:
|
||||
supervisor proposals retain the 1 MiB cap and Git pack requests retain their
|
||||
larger protocol-appropriate cap.
|
||||
|
||||
### Shutdown diagnostics
|
||||
|
||||
The gateway output pump catches only stream-closure exceptions expected after
|
||||
the supervisor closes child pipes. Other I/O failures remain visible and are
|
||||
reported through the supervisor's normal diagnostic channel.
|
||||
|
||||
### Shared filesystem security
|
||||
|
||||
The common SQLite store owns database creation for every backend. It creates
|
||||
the parent directory and an empty database with private modes before opening
|
||||
SQLite, repairs existing modes, verifies the resulting state, and propagates
|
||||
every enforcement failure. Backend launchers do not duplicate this policy.
|
||||
|
||||
The backend-neutral gateway provisioner likewise applies directory and file
|
||||
modes after transport copies complete. This avoids relying on copy behavior
|
||||
that differs among Docker, Apple Container, and Firecracker's SSH transport.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. Existing fail-closed security and backend enumeration fixes.
|
||||
2. FastAPI orchestrator transport and bounded control-plane bodies.
|
||||
3. Egress request-policy and outbound-DLP pipeline extraction.
|
||||
4. Supervisor MCP dispatch extraction.
|
||||
5. Shared cleanup refresh/revalidation plus authoritative macOS discovery.
|
||||
6. Strict supervisor introspection and bounded supervisor/Git HTTP work.
|
||||
7. Gateway shutdown log-pump closure handling.
|
||||
8. Lossless Firecracker process identities, authoritative Docker cleanup
|
||||
queries, and bounded Firecracker artifact transfers.
|
||||
9. Mandatory authenticated secret storage, shared cleanup-plan intersection
|
||||
and mutation accounting, and contained Git backend process failures.
|
||||
10. Disk-spooled and separately bounded Git bodies, cleanup command deadlines,
|
||||
and classified Firecracker signalling failures.
|
||||
11. Fail-closed shared database creation and backend-neutral gateway credential
|
||||
permissions.
|
||||
|
||||
## Open questions
|
||||
|
||||
None.
|
||||
@@ -0,0 +1,391 @@
|
||||
# PRD 0082: Trusted agent forge identity and guidance
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Author:** didericis-claude
|
||||
- **Created:** 2026-07-25
|
||||
- **Issue:** #423
|
||||
|
||||
## Summary
|
||||
|
||||
Move author identity and named forge configurations into host-trusted agent
|
||||
definitions. A bottle may optionally associate a git-gate repository with one
|
||||
of the selected agent's forge aliases. When associated, bot-bottle gives the
|
||||
agent non-secret, provider-specific system guidance for that repository and
|
||||
routes authenticated forge API calls through the egress proxy without exposing
|
||||
the forge token to the bottle. The authenticated account is inferred from the
|
||||
identity that owns that token; it is not separately declared in the manifest.
|
||||
|
||||
Repository-local agent and bottle definitions are no longer discovered. Once
|
||||
agent definitions can select a host forge credential, allowing checked-out
|
||||
repository content to define or override an agent would let untrusted workspace
|
||||
content select host identities and secrets.
|
||||
|
||||
This PRD is deliberately limited to identity ownership, manifest trust, forge
|
||||
API access, and generated guidance. Per-activation signing, signature
|
||||
enforcement, commit attribution, and the commit audit model move to a follow-up
|
||||
PRD.
|
||||
|
||||
Successor to:
|
||||
|
||||
- **PRD 0011 (per-file manifests)** — allowed repository-local agent files to
|
||||
override home agents. This PRD removes that trust path: agents and bottles are
|
||||
loaded only from the host-owned `~/.bot-bottle` tree.
|
||||
- **PRD 0027 (agent git identity, #94)** / **ADR 0002** — put name/email in
|
||||
`git-gate.user` while keeping them claimed rather than vouched. This PRD moves
|
||||
those agent properties out of git-gate and into the trusted agent definition.
|
||||
- **PRD 0048 (deploy-key provisioning, #169)** — remains the Git push
|
||||
capability. A forge actor token is a separate API credential and never
|
||||
replaces a deploy key.
|
||||
|
||||
## Problem
|
||||
|
||||
### Forge workflow context is missing
|
||||
|
||||
The agent prompt does not know which forge backs a git-gate repository, which
|
||||
API base URL to use, or that authenticated requests must go through the egress
|
||||
proxy. Bespoke prompt text has drifted between agents. Missing guidance has
|
||||
already caused incorrect PR behavior, including attempts to use Gitea AGit
|
||||
review refs instead of a normal branch-backed pull request.
|
||||
|
||||
### Identity is owned by the wrong layer
|
||||
|
||||
`git-gate.user` puts an agent property on a repository transport component. An
|
||||
author identity and set of forge credentials should follow the agent across
|
||||
bottles and repositories. Git-gate should own Git transport policy, not decide
|
||||
who the agent is.
|
||||
|
||||
### Repository-local agents become a credential-selection path
|
||||
|
||||
Today `$CWD/.bot-bottle/agents/*.md` can define new agents and override
|
||||
home-resident agents. If an agent definition may reference an operator-provided
|
||||
forge token, a malicious repository could select a host credential merely by
|
||||
being the current workspace. Repository-local bottles are already ignored;
|
||||
agents need the same host-only boundary.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- **Agent-owned identity.** Author name/email and named forge configurations
|
||||
live on the agent definition, not under `git-gate`.
|
||||
- **Trusted definitions only.** Agent and bottle files are discovered only
|
||||
under `~/.bot-bottle/{agents,bottles}`. Repository-local definitions never
|
||||
contribute names, defaults, or overrides.
|
||||
- **Optional repository association.** A bottle repository may name one forge
|
||||
alias from the selected agent. Repositories without `forge` retain current
|
||||
behavior and generate no forge guidance or credential route.
|
||||
- **Proxy-held forge credential.** The host resolves the selected forge
|
||||
alias's token reference and gives the value only to the egress proxy. The
|
||||
bottle receives neither the token nor a credential file containing it.
|
||||
- **Forge-aware system guidance.** For associated repositories, bot-bottle
|
||||
generates provider-specific instructions describing the API base URL,
|
||||
repository/account association, proxy-authenticated access, and correct PR
|
||||
workflow.
|
||||
- **No secret prompt material.** Neither token values nor token
|
||||
environment-variable names appear in the prompt or bottle environment.
|
||||
- **Fail closed.** Unknown account references, unsupported/non-HTTPS URLs,
|
||||
missing host secrets, and misplaced agent/bottle fields fail before creating
|
||||
the bottle.
|
||||
- **Push capability unchanged.** Git transport remains PRD 0048 deploy keys.
|
||||
The forge actor token is only for API actions such as opening, reviewing, and
|
||||
commenting on pull requests.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Commit signing or signature enforcement.** No signing key is minted and
|
||||
git-gate does not verify commit signatures in this PRD.
|
||||
- **Commit attribution or audit tables.** There is no trustworthy commit
|
||||
observation event in this slice. A follow-up signing PRD owns activation keys,
|
||||
control-plane verification, and any configured-author-versus-claimed-author
|
||||
audit model.
|
||||
- **Cryptographically vouched author identity.** `author` configures Git and
|
||||
identifies the agent actor, but commit author/committer fields remain claims
|
||||
under ADR 0002.
|
||||
- **Forge account or token minting.** The operator creates the agent-specific
|
||||
account/token out of band. Bot-bottle references an existing host secret; it
|
||||
does not create, rotate, or revoke that credential.
|
||||
- **Forge-side commit attribution surfaces.** No commit status, signing-key
|
||||
registration, or "Verified" badge.
|
||||
- **Provider-generic arbitrary prompts.** Gitea is the first supported forge.
|
||||
A future provider adds typed validation and generated guidance in code rather
|
||||
than accepting repository-supplied prompt text.
|
||||
|
||||
## Design
|
||||
|
||||
### Ownership model
|
||||
|
||||
The resolved bottled agent is an agent definition composed with a bottle:
|
||||
|
||||
| Part | Source | Role |
|
||||
|------|--------|------|
|
||||
| Author identity | agent `author` | configures Git name/email and identifies the agent's claimed author |
|
||||
| Forge configurations | agent `forge-accounts` | maps forge aliases to API origins and host token references available to this agent; token ownership determines account identity |
|
||||
| Git repository | bottle `git-gate.repos` | configures Git transport, host verification, and push capability |
|
||||
| Repository/forge association | bottle repo `forge` | optionally selects an agent forge alias for guidance and API access |
|
||||
|
||||
This boundary keeps identity on the agent, capability/policy on the bottle, and
|
||||
transport enforcement in git-gate.
|
||||
|
||||
### Agent manifest
|
||||
|
||||
Agent identity and forge accounts live only in
|
||||
`~/.bot-bottle/agents/<name>.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
author:
|
||||
name: didericis-claude
|
||||
email: eric+claude@dideric.is
|
||||
forge-accounts:
|
||||
didericis-gitea:
|
||||
auth:
|
||||
type: token
|
||||
token_secret: GITEA_CLAUDE_TOKEN
|
||||
url: https://gitea.dideric.is/api/v1
|
||||
---
|
||||
```
|
||||
|
||||
`author` contains:
|
||||
|
||||
- `name`: non-empty string;
|
||||
- `email`: non-empty string passing the existing Git identity validation.
|
||||
|
||||
The resolved values populate `user.name` and `user.email`. Existing
|
||||
`git-gate.user` fields fail with migration guidance to move the values into the
|
||||
selected home agent's `author` block. There is no compatibility period where a
|
||||
bottle identity silently overrides the agent identity.
|
||||
|
||||
`forge-accounts` is a map whose keys are **forge aliases** and follow the
|
||||
manifest's existing kebab-case identifier grammar
|
||||
(`[a-z][a-z0-9-]*`). Each forge entry contains:
|
||||
|
||||
- `url`: a canonical HTTPS Gitea API base URL;
|
||||
- `auth.type`: `token` in this slice;
|
||||
- `auth.token_secret`: the name of a host environment variable containing the
|
||||
operator-provided, agent-specific API token.
|
||||
|
||||
The token's owner determines the authenticated forge account; there is no
|
||||
separate account-name field. The token secret name is host configuration, not
|
||||
bottle configuration. The token value is resolved only if a selected bottle
|
||||
repository references the forge alias.
|
||||
|
||||
### Bottle manifest
|
||||
|
||||
Repository policy remains in `~/.bot-bottle/bottles/<name>.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
bot-bottle:
|
||||
url: ssh://git@100.78.141.42:30009/didericis/bot-bottle.git
|
||||
provisioned_key:
|
||||
provider: gitea
|
||||
token_env: GITEA_DEPLOY_TOKEN
|
||||
host_key: "ssh-ed25519 AAAA..."
|
||||
forge: didericis-gitea
|
||||
---
|
||||
```
|
||||
|
||||
`git-gate.repos.<name>.forge` is optional:
|
||||
|
||||
- When absent, the repository behaves exactly as it does today. Bot-bottle does
|
||||
not resolve a forge actor token and does not add forge-specific instructions
|
||||
for that repository.
|
||||
- When present, it must match a forge alias in `forge-accounts` on the selected
|
||||
agent.
|
||||
The resolved association enables a scoped proxy credential route and adds the
|
||||
repository/forge relationship to generated system guidance.
|
||||
|
||||
`provisioned_key.token_env` remains the deploy-key administration credential
|
||||
from PRD 0048. It is separate from `forge-accounts.*.auth.token_secret`: the
|
||||
former provisions Git push capability, while the latter performs API actions as
|
||||
the agent.
|
||||
|
||||
`author` and `forge-accounts` are agent-only. `git-gate.repos`, including
|
||||
`forge`, is bottle-only. Validation errors point to the correct file and trust
|
||||
domain rather than ignoring misplaced keys.
|
||||
|
||||
### Definition trust and discovery
|
||||
|
||||
Only the host-owned manifest tree is authoritative:
|
||||
|
||||
- Agents: `~/.bot-bottle/agents/*.md`
|
||||
- Bottles: `~/.bot-bottle/bottles/*.md`
|
||||
|
||||
`$CWD/.bot-bottle/agents/*.md` no longer contributes new agents and no longer
|
||||
overrides a home agent. `$CWD/.bot-bottle/bottles/*.md` remains unusable. If
|
||||
either repository-local directory contains manifest files, bot-bottle emits a
|
||||
warning that they are ignored and points to the corresponding home path.
|
||||
|
||||
Every discovery and resolution surface uses the same home-only index:
|
||||
|
||||
- agent enumeration and selectors;
|
||||
- `require_agent`;
|
||||
- lazy `load_for_agent`;
|
||||
- default-agent/default-bottle resolution;
|
||||
- dashboard and headless launch paths.
|
||||
|
||||
There must be no alternate direct-path load that can still select a workspace
|
||||
definition.
|
||||
|
||||
Workspace instructions remain repository content (for example `AGENTS.md`),
|
||||
but runtime policy, host secret references, and actor identity do not.
|
||||
Programmatic in-memory manifests remain available for tests and trusted internal
|
||||
composition; they are not a filesystem discovery path.
|
||||
|
||||
### Forge URL validation
|
||||
|
||||
The host parses and canonicalizes each referenced forge alias's URL before
|
||||
creating any runtime resources:
|
||||
|
||||
- scheme must be `https`;
|
||||
- userinfo, query, and fragment are forbidden;
|
||||
- hostname must be present;
|
||||
- the path must be a supported Gitea API base (initially `/api/v1`, with
|
||||
normalization of a trailing slash);
|
||||
- visually different inputs that canonicalize to the same origin/prefix are
|
||||
deduplicated;
|
||||
- unsupported providers or path shapes fail closed.
|
||||
|
||||
The provider is determined by typed support in bot-bottle, not by prompt text
|
||||
from a repository. Adding another provider requires a validator, auth scheme,
|
||||
and guidance renderer.
|
||||
|
||||
### Proxy credential provisioning
|
||||
|
||||
For each distinct forge alias referenced by at least one selected bottle
|
||||
repository, the host:
|
||||
|
||||
1. Resolves `auth.token_secret` from the host environment and rejects a missing
|
||||
or empty value.
|
||||
2. Copies the token value only into the egress proxy's credential environment.
|
||||
3. Adds an inspected route scoped to the canonical forge origin and API prefix.
|
||||
4. Configures the provider authentication scheme (`token` for Gitea) so the
|
||||
proxy injects authentication.
|
||||
|
||||
The bottle makes an unauthenticated request to the configured HTTPS API URL.
|
||||
The token is not copied into the bottle, `.gitconfig`, generated prompt,
|
||||
workspace, or process environment visible to the agent.
|
||||
|
||||
If several repositories reference the same forge alias, they share one
|
||||
credential route. Unreferenced forge aliases resolve no secret and create no
|
||||
route.
|
||||
|
||||
### Generated system guidance
|
||||
|
||||
Bot-bottle appends a generated, non-secret section to its existing system
|
||||
prompt file. It is derived from validated typed fields, not copied Markdown from
|
||||
a repository.
|
||||
|
||||
For each associated repository, Gitea guidance includes:
|
||||
|
||||
- forge alias (for example `didericis-gitea`);
|
||||
- API base URL (for example `https://gitea.dideric.is/api/v1`);
|
||||
- the git-gate repository name tied to that forge;
|
||||
- the instruction to call the configured HTTPS API through the proxy without
|
||||
reading, printing, or manually attaching an authorization token;
|
||||
- the distinction that Git pushes still use the git-gate remote;
|
||||
- the requirement to create/update a normal `refs/heads/<branch>` and open a
|
||||
branch-backed pull request through the API;
|
||||
- the prohibition on pushing `refs/for/*`, `refs/draft/*`, or
|
||||
`refs/for-review/*`;
|
||||
- the instruction to use the API for reviews/comments and verify returned
|
||||
object state before claiming completion.
|
||||
|
||||
The prompt contains neither the token value nor its `token_secret` name.
|
||||
Repositories without `forge` are omitted from this section. If no selected
|
||||
repository has a forge association, no forge guidance section is generated.
|
||||
|
||||
### Failure and lifecycle behavior
|
||||
|
||||
Forge configuration is validated before bottle creation. A bad association or
|
||||
credential must not leave a partially-created bottle or proxy.
|
||||
|
||||
The operator-owned token is not minted, rotated, or revoked by bot-bottle. On
|
||||
teardown, stopping the egress proxy discards the activation's in-memory/runtime
|
||||
copy. Later activations resolve the current host secret again.
|
||||
|
||||
Logs may include the forge alias, canonical API origin, and repository name.
|
||||
They must never contain the token value. Errors for missing secrets name the
|
||||
configuration field and host environment variable, but do not print any value.
|
||||
|
||||
## Migration
|
||||
|
||||
This change is intentionally breaking at the manifest trust boundary:
|
||||
|
||||
1. Move each home bottle/agent `git-gate.user.name` and `.email` into the
|
||||
corresponding home agent's `author`.
|
||||
2. Add agent-specific `forge-accounts` only to home agent definitions.
|
||||
3. Add optional `forge` associations to home bottle repository entries.
|
||||
4. Move any `$CWD/.bot-bottle/agents/*.md` that should remain selectable into
|
||||
`~/.bot-bottle/agents/`. Repository copies are ignored thereafter.
|
||||
5. Keep repository-specific behavioral instructions in `AGENTS.md` or another
|
||||
workspace instruction file; do not put runtime identity or secret references
|
||||
there.
|
||||
|
||||
Errors and warnings link to this migration rather than silently changing which
|
||||
identity or definition is active.
|
||||
|
||||
## Follow-up: signed commits and attribution
|
||||
|
||||
A separate PRD will consume the trusted agent `author` introduced here and own:
|
||||
|
||||
- per-activation signing key minting and sidecar isolation;
|
||||
- commit-time signing through a forwarded signing capability;
|
||||
- git-gate rejection of unsigned/wrong-key new commits;
|
||||
- independent control-plane object-ID recomputation and signature verification;
|
||||
- activation and attributed-commit audit tables;
|
||||
- storage of configured agent author separately from each commit's unenforced
|
||||
claimed author/committer;
|
||||
- post-teardown verification and key-retention policy.
|
||||
|
||||
That PRD must not reintroduce identity under git-gate or expand repository-local
|
||||
manifest trust.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **This PRD.** Establish the identity, forge, and filesystem trust boundary.
|
||||
2. **Home-only definitions.** Remove cwd agents from discovery, override,
|
||||
enumeration, lazy loading, defaults, and selectors. Warn on ignored cwd
|
||||
agent/bottle files and provide migration guidance.
|
||||
3. **Manifest schema.** Add agent-only `author` and `forge-accounts`; remove
|
||||
`git-gate.user`; add optional bottle-only
|
||||
`git-gate.repos.<name>.forge`. Validate account names, composition
|
||||
references, field placement, and Gitea API URLs.
|
||||
4. **Proxy provisioning.** Lazily resolve only referenced token secrets and
|
||||
create scoped authenticated Gitea API routes without putting credentials in
|
||||
the bottle.
|
||||
5. **Prompt generation.** Render typed Gitea/repository workflow guidance into
|
||||
the existing bot-bottle prompt path for associated repositories only.
|
||||
6. **Docs and migration.** Update README examples, PRD 0011-facing discovery
|
||||
documentation, agent/bottle schema docs, and error guidance.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Trust boundary:** cwd agent files are ignored with a warning, cannot
|
||||
override a home agent, are absent from enumeration/selectors/defaults, and
|
||||
cannot be loaded by name. Cwd bottle behavior remains home-only.
|
||||
- **Agent schema:** `author` and `forge-accounts` parse; malformed identities,
|
||||
non-kebab account names, unknown fields, invalid auth types, and misplaced
|
||||
git-gate fields fail clearly.
|
||||
- **Bottle schema:** `forge` is optional; absent associations preserve current
|
||||
behavior; present associations resolve against the selected agent; unknown or
|
||||
misplaced associations fail before launch.
|
||||
- **URL validation:** HTTPS Gitea API bases pass and canonicalize; HTTP,
|
||||
userinfo, query, fragment, missing host, unsupported paths, and unsupported
|
||||
providers fail closed.
|
||||
- **Secret handling:** only referenced accounts resolve environment secrets;
|
||||
missing/empty secrets fail before runtime creation; token values are absent
|
||||
from bottle env, prompt, generated config, logs, and workspace; token secret
|
||||
names are absent from the bottle and prompt.
|
||||
- **Proxy behavior:** associated API requests receive proxy-injected Gitea
|
||||
authentication scoped to the configured origin/prefix; unrelated hosts and
|
||||
paths receive no credential.
|
||||
- **Prompt behavior:** guidance names account/API/repository associations,
|
||||
branch-backed PR workflow, prohibited AGit refs, and mutation verification;
|
||||
repositories without `forge` are omitted; no associations means no section.
|
||||
- **Migration:** legacy `git-gate.user` fails with an `author` migration pointer;
|
||||
ignored cwd definitions warn with the target home path.
|
||||
|
||||
## Open questions
|
||||
|
||||
- None.
|
||||
@@ -5,10 +5,9 @@ model: opus
|
||||
bottle: dev
|
||||
skills:
|
||||
- init-prd
|
||||
git-gate:
|
||||
user:
|
||||
name: implementer-bot
|
||||
email: eric+implementer@dideric.is
|
||||
author:
|
||||
name: implementer-bot
|
||||
email: eric+implementer@dideric.is
|
||||
---
|
||||
|
||||
You are a feature-implementation agent running inside an ephemeral
|
||||
|
||||
@@ -2,12 +2,9 @@
|
||||
# The bot-bottle project itself has no runtime dependencies.
|
||||
# These tools are used for code quality checks in CI/CD.
|
||||
|
||||
-r requirements.orchestrator.in
|
||||
pylint>=3.0.0
|
||||
pyright>=1.1.411
|
||||
coverage>=7.0.0
|
||||
# PEP 517 build front-end used by tests/unit/test_wheel_install.py to build and
|
||||
# install a real wheel (proves the installed distribution is self-contained).
|
||||
build>=1.0.0
|
||||
# FastAPI's in-process TestClient transport.
|
||||
httpx>=0.28.0
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
# Runtime dependencies baked only into the orchestrator images.
|
||||
fastapi==0.140.0
|
||||
uvicorn==0.51.0
|
||||
@@ -1,182 +0,0 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile with Python 3.13
|
||||
# by the following command:
|
||||
#
|
||||
# pip-compile --generate-hashes --output-file=requirements.orchestrator.lock requirements.orchestrator.in
|
||||
#
|
||||
annotated-doc==0.0.4 \
|
||||
--hash=sha256:571ac1dc6991c450b25a9c2d84a3705e2ae7a53467b5d111c24fa8baabbed320 \
|
||||
--hash=sha256:fbcda96e87e9c92ad167c2e53839e57503ecfda18804ea28102353485033faa4
|
||||
# via fastapi
|
||||
annotated-types==0.8.0 \
|
||||
--hash=sha256:13b2beaad985e05e2d6407ee4c4f35590b11f8d693a258a561055cac8f64cab7 \
|
||||
--hash=sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0
|
||||
# via pydantic
|
||||
anyio==4.14.2 \
|
||||
--hash=sha256:9f505dda5ac9f0c8309b5e8bd445a8c2bf7246f3ce950121e45ea15bc41d1494 \
|
||||
--hash=sha256:cfa139f3ed1a23ee8f88a145ddb5ac7605b8bbfd8592baacd7ce3d8bb4313c7f
|
||||
# via starlette
|
||||
click==8.4.2 \
|
||||
--hash=sha256:9a6cea6e60b17ebe0a44c5cc636d94f09bd66142c1cd7d8b4cd731c4917a15f6 \
|
||||
--hash=sha256:e6f9f66136c816745b9d65817da91d61d957fb16e02e4dcd0552553c5a197b76
|
||||
# via uvicorn
|
||||
fastapi==0.140.0 \
|
||||
--hash=sha256:e951c0a0d9540bf5d9a2a9e078fd415da2ab7e312d435139e7d9e2e7fe9f0b23 \
|
||||
--hash=sha256:f338951b82fd74ca8f843163aec43ea1a1ce84d515415a50fa98fa25572a5544
|
||||
# via -r requirements.orchestrator.in
|
||||
h11==0.16.0 \
|
||||
--hash=sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1 \
|
||||
--hash=sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86
|
||||
# via uvicorn
|
||||
idna==3.18 \
|
||||
--hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \
|
||||
--hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848
|
||||
# via anyio
|
||||
pydantic==2.13.4 \
|
||||
--hash=sha256:45a282cde31d808236fd7ea9d919b128653c8b38b393d1c4ab335c62924d9aba \
|
||||
--hash=sha256:c40756b57adaa8b1efeeced5c196f3f3b7c435f90e84ea7f443901bec8099ef6
|
||||
# via fastapi
|
||||
pydantic-core==2.46.4 \
|
||||
--hash=sha256:00c603d540afdd6b80eb39f078f33ebd46211f02f33e34a32d9f053bba711de0 \
|
||||
--hash=sha256:0186750b482eefa11d7f435892b09c5c606193ef3375bcf94aa00ae6bfb66262 \
|
||||
--hash=sha256:041bde0a48fd37cf71cab1c9d56d3e8625a3793fef1f7dd232b3ff37e978ecda \
|
||||
--hash=sha256:0c563b08bca408dc7f65f700633d8442fffb2421fc47b8101377e9fd65051ff0 \
|
||||
--hash=sha256:0cbe8b01f948de4286c74cdd6c667aceb38f5c1e26f0693b3983d9d74887c65e \
|
||||
--hash=sha256:0ce40cd7b21210e99342afafbd4d0f76d784eb5b1d60f3bdc566be4983c6c73b \
|
||||
--hash=sha256:0e96592440881c74a213e5ad528e2b24d3d4f940de2766bed9010ab1d9e51594 \
|
||||
--hash=sha256:10e17cbb10a330363733efc4d7c4d0dd827ac0909b8f6a6542298fed1ea62f29 \
|
||||
--hash=sha256:133878133d271ade3d41d1bfb2a45ec38dbdbda40bc065921c6b04e4630127e2 \
|
||||
--hash=sha256:14d4edf427bdcf950a8a02d7cb44a08614388dd6e1bdcbf4f67504fa7887da9c \
|
||||
--hash=sha256:14f4c5d6db102bd796a627bbb3a17b4cf4574b9ae861d8b7c9a9661c6dd3362d \
|
||||
--hash=sha256:17299feefe090f2caa5b8e37222bb5f663e4935a8bfa6931d4102e5df1a9f398 \
|
||||
--hash=sha256:184c081504d17f1c1066e430e117142b2c77d9448a97f7b65c6ac9fd9aee238d \
|
||||
--hash=sha256:18e5ceec2ab67e6d5f1a9085e5a24c9c4e2ac4545730bfe668680bca05e555f3 \
|
||||
--hash=sha256:19e51f073cd3df251856a8a4189fbdf1de4012c3ebacfb1884f94f1eb406079f \
|
||||
--hash=sha256:1a7dd0b3ee80d90150e3495a3a13ac34dbcbfd4f012996a6a1d8900e91b5c0fb \
|
||||
--hash=sha256:1d8ba486450b14f3b1d63bc521d410ec7565e52f887b9fb671791886436a42f7 \
|
||||
--hash=sha256:2108ba5c1c1eca18030634489dc544844144ee36357f2f9f780b93e7ddbb44b5 \
|
||||
--hash=sha256:228ee9bae8bef5b1e97ec58302f80357c37199e0d0a99174e138d28e6957b9d9 \
|
||||
--hash=sha256:23ace664830ee0bfe014a0c7bc248b1f7f25ed7ad103852c317624a1083af462 \
|
||||
--hash=sha256:2412e734dcb48da14d4e4006b82b46b74f2518b8a26ee7e58c6844a6cd6d03c4 \
|
||||
--hash=sha256:29c61fc04a3d840155ff08e475a04809278972fe6aef51e2720554e96367e34b \
|
||||
--hash=sha256:2f84c03c8607173d16b5a854ec68a2f9079ae03237a54fb506d13af47e1d018d \
|
||||
--hash=sha256:3009f12e4e90b7f88b4f9adb1b0c4a3d58fe7820f3238c190047209d148026df \
|
||||
--hash=sha256:3245406455a5d98187ec35530fd772b1d799b26667980872c8d4614991e2c4a2 \
|
||||
--hash=sha256:3447661d99f75a3683a4cf5c87da72f2161964611864dbbeac7fbb118bb4bfc0 \
|
||||
--hash=sha256:372429a130e469c9cd698925ce5fc50940b7a1336b0d82038e63d5bbc4edc519 \
|
||||
--hash=sha256:395aebd9183f9d112f569aeb5b2214d1a10a33bec8456447f7fbdfa51d38d4cd \
|
||||
--hash=sha256:3a233125ac121aa3ffba9a2b59edfc4a985a76092dc8279586ab4b71390875e7 \
|
||||
--hash=sha256:3be77f45df024d789a672ae34f8b06fb346c4f9f46ea714956660ea4862e89ac \
|
||||
--hash=sha256:3bf92c5d0e00fefaab325a4d27828fe6b6e2a21848686b5b60d2d9eeb09d76c6 \
|
||||
--hash=sha256:3ecbc122d18468d06ca279dc26a8c2e2d5acb10943bb35e36ae92096dc3b5565 \
|
||||
--hash=sha256:3fb702cd90b0446a3a1c5e470bfa0dd23c0233b676a9099ddcc964fa6ca13898 \
|
||||
--hash=sha256:428e04521a40150c85216fc8b85e8d39fece235a9cf5e383761238c7fa9b96fb \
|
||||
--hash=sha256:432c179df7874eeb73307aad2df0755e1ae0efa61ff0ea89b93e194411ae3928 \
|
||||
--hash=sha256:4a05d69cba51d852c5c3e92758653245a50c0b646ced0cf05bd793ed592839d6 \
|
||||
--hash=sha256:4c63ebc82684aa89d9a3bcbd13d515b3be44250dc68dd3bd81526c1cb31286c3 \
|
||||
--hash=sha256:4fc73cb559bdb54b1134a706a2802a4cddd27a0633f5abb7e53056268751ac6a \
|
||||
--hash=sha256:4fcbe087dbc2068af7eda3aa87634eba216dbda64d1ae73c8684b621d33f6596 \
|
||||
--hash=sha256:56cb4851bcaf3d117eddcef4fe66afd750a50274b0da8e22be256d10e5611987 \
|
||||
--hash=sha256:5855698a4856556d86e8e6cd8434bc3ac0314ee8e12089ae0e143f64c6256e4e \
|
||||
--hash=sha256:5a4330cdbc57162e4b3aa303f588ba752257694c9c9be3e7ebb11b4aca659b5d \
|
||||
--hash=sha256:5b712b53160b79a5850310b912a5ef8e57e56947c8ad690c227f5c9d7e561712 \
|
||||
--hash=sha256:5d5902252db0d3cedf8d4a1bc68f70eeb430f7e4c7104c8c476753519b423008 \
|
||||
--hash=sha256:617d7e2ca7dcb8c5cf6bcb8c59b8832c94b36196bbf1cbd1bfb56ed341905edd \
|
||||
--hash=sha256:62f875393d7f270851f20523dd2e29f082bcc82292d66db2b64ea71f64b6e1c1 \
|
||||
--hash=sha256:633147d34cf4550417f12e2b1a0383973bdf5cdfde212cb09e9a581cf10820be \
|
||||
--hash=sha256:66ce7632c22d837c95301830e111ad0128a32b8207533b60896a96c4915192ea \
|
||||
--hash=sha256:6b3ace8194b0e5204818c92802dcdca7fc6d88aabbb799d7c795540d9cd6d292 \
|
||||
--hash=sha256:6f2eeda33a839975441c86a4119e1383c50b47faf0cbb5176985565c6bb02c33 \
|
||||
--hash=sha256:7027560ee92211647d0d34e3f7cd6f50da56399d26a9c8ad0da286d3869a53f3 \
|
||||
--hash=sha256:7283d57845ecf5a163403eb0702dfc220cc4fbdd18919cb5ccea4f95ee1cdab4 \
|
||||
--hash=sha256:7a5f930472650a82629163023e630d160863fce524c616f4e5186e5de9d9a49b \
|
||||
--hash=sha256:7bfb192b3f4b9e8a89b6277b6ce787564f62cfd272055f6e685726b111dc7826 \
|
||||
--hash=sha256:811ff8e9c313ab425368bcbb36e5c4ebd7108c2bbf4e4089cfbb0b01eff63fac \
|
||||
--hash=sha256:8233f2947cf85404441fd7e0085f53b10c93e0ee78611099b5c7237e36aacbf7 \
|
||||
--hash=sha256:82cf5301172168103724d49a1444d3378cb20cdee30b116a1bd6031236298a5d \
|
||||
--hash=sha256:8358a950c8909158e3df31538a7e4edc2d7265a7c54b47f0864d9e5bae9dcebf \
|
||||
--hash=sha256:85bb3611ff1802f3ee7fdd7dbff26b56f343fb432d57a4728fdd49b6ef35e2f4 \
|
||||
--hash=sha256:86e1a4418c6cd97d60c95c71164158eaf7324fae7b0923264016baa993eba6fc \
|
||||
--hash=sha256:8b9bab013d1c7a79d3501ff86d0bc9c31bf587db4551677b96bec07df78c6b15 \
|
||||
--hash=sha256:8c5dac79fa1614d1e06ca695109c6105923bd9c7d1d6c918d4e637b7e6b32fd3 \
|
||||
--hash=sha256:8d0820e8192167f80d88d64038e609c31452eeca865b4e1d9950a27a4609b00b \
|
||||
--hash=sha256:8daafc69c93ee8a0204506a3b6b30f586ef54028f52aeeeb5c4cfc5184fd5914 \
|
||||
--hash=sha256:9037063db01f09b09e237c282b6792bd4da634b5402c4e7f0c61effed7701a04 \
|
||||
--hash=sha256:905a0ed8ea6f2d61c1738835f99b699348d7857379083e5fc497fa0c967a407c \
|
||||
--hash=sha256:90884113d8b48f760e9587002789ddd741e76ab9f89518cd1e43b1f1a52ec44b \
|
||||
--hash=sha256:91a06d2e259ecfbd8c901d70c3c507900458498142b3026a296b7de4d1322cc9 \
|
||||
--hash=sha256:926c9541b14b12b1681dca8a0b75feb510b06c6341b70a8e500c2fdcff837cce \
|
||||
--hash=sha256:9401557acd873c3a7f3eb9383edef8ac4968f9510e340f4808d427e75667e7b4 \
|
||||
--hash=sha256:9551187363ffc0de2a00b2e47c25aeaeb1020b69b668762966df15fc5659dd5a \
|
||||
--hash=sha256:962ccbab7b642487b1d8b7df90ef677e03134cf1fd8880bf698649b22a69371f \
|
||||
--hash=sha256:97e7cf2be5c77b7d1a9713a05605d49460d02c6078d38d8bef3cbe323c548424 \
|
||||
--hash=sha256:9aa768456404a8bf48a4406685ac2bec8e72b62c69313734fa3b73cf33b3a894 \
|
||||
--hash=sha256:9bc519fbf2b7578398853d815009ae5e4d4603d12f4e3f91da8c06852d3da3e9 \
|
||||
--hash=sha256:9d56801be94b86a9da183e5f3766e6310752b99ff647e38b09a9500d88e46e76 \
|
||||
--hash=sha256:9f444c499b3eefd3a92e348059471ea0c3a6e303d9c1cec09fa748fd9f895201 \
|
||||
--hash=sha256:9fa8ae11da9e2b3126c6426f147e0fba88d96d65921799bb30c6abd1cb2c97fb \
|
||||
--hash=sha256:a0f62d0a58f4e7da165457e995725421e0064f2255d8eccebc49f41bbc23b109 \
|
||||
--hash=sha256:a396dcc17e5a0b164dbe026896245a4fa9ff402edca1dff0be3d53a517f74de4 \
|
||||
--hash=sha256:aaa2a54443eff1950ba5ddc6b6ccda0d9c84a364276a62f969bdf2a390650848 \
|
||||
--hash=sha256:ad785e92e6dc634c21555edc8bd6b64957ab844541bcb96a1366c202951ae526 \
|
||||
--hash=sha256:af8244b2bef6aaad6d92cda81372de7f8c8d36c9f0c3ea36e827c60e7d9467a0 \
|
||||
--hash=sha256:b078afbc25f3a1436c7a1d2cd3e322497ee99615ba97c563566fdf46aff1ee01 \
|
||||
--hash=sha256:b2f69dec1725e79a012d920df1707de5caf7ed5e08f3be4435e25803efc47458 \
|
||||
--hash=sha256:b8458003118a712e66286df6a707db01c52c0f52f7db8e4a38f0da1d3b94fc4e \
|
||||
--hash=sha256:bb63e0198ca18aad131c089b9204c23079c3afa95487e561f4c522d519e55aba \
|
||||
--hash=sha256:bfec22eab3c8cc2ceec0248aec886624116dc079afa027ecc8ad4a7e62010f8a \
|
||||
--hash=sha256:c1747f85cee84c26985853c6f3d9bd3e75da5212912443fa111c113b9c246f39 \
|
||||
--hash=sha256:c1b3f518abeca3aa13c712fd202306e145abf59a18b094a6bafb2d2bbf59192c \
|
||||
--hash=sha256:c50f2528cf200c5eed56faf3f4e22fcd5f38c157a8b78576e6ba3168ec35f000 \
|
||||
--hash=sha256:c68fcd102d71ea85c5b2dfac3f4f8476eff42a9e078fd5faefff6d145063536b \
|
||||
--hash=sha256:c7a7bd4e39e8e4c12c39cd480356842b6a8a06e41b23a55a5e3e191718838ddf \
|
||||
--hash=sha256:c94f0688e7b8d0a67abf40e57a7eaaecd17cc9586706a31b76c031f63df052b4 \
|
||||
--hash=sha256:cbaf13819775b7f769bf4a1f066cb6df7a28d4480081a589828ef190226881cd \
|
||||
--hash=sha256:cd2213145bcc2ba85884d0ac63d222fece9209678f77b9b4d76f054c561adb28 \
|
||||
--hash=sha256:ce5c1d2a8b27468f433ca974829c44060b8097eedc39933e3c206a90ee49c4a9 \
|
||||
--hash=sha256:d396ec2b979760aaf3218e76c24e65bd0aca24983298653b3a9d7a45f9e47b30 \
|
||||
--hash=sha256:d51026d73fcfd93610abc7b27789c26b313920fcfb20e27462d74a7f8b06e983 \
|
||||
--hash=sha256:d80ee3d731373b24cebbc10d689ca4ee1875caf0d5703a245db18efd4dd37fc1 \
|
||||
--hash=sha256:d995260fdf4e1db774581b4900e0f832abe3c7c84996726bbc161b19c8f29e76 \
|
||||
--hash=sha256:da4b951fe36dc7c3a1ccb4e3cd1747c3542b8c9ceede8fc86cae054e764485f5 \
|
||||
--hash=sha256:daa27d92c36f24388fe3ad306b174781c747627f134452e4f128ea00ce1fe8c4 \
|
||||
--hash=sha256:db06ffe51636ffe9ca531fe9023dd64bdd794be8754cb5df57c5498ae5b518a7 \
|
||||
--hash=sha256:e0d65b8c354be7fb5f720c3caa8bc940bc2d20ce749c8e06135f07f8ed95dd7c \
|
||||
--hash=sha256:e68b7a074f65a2fd746c52a7ce6142ab7006074ac269ace0c25cd8ba171f8066 \
|
||||
--hash=sha256:e739fee756ba1010f8bcccb534252e85a35fe45ae92c295a06059ce58b74ccd3 \
|
||||
--hash=sha256:e846ae7835bf0703ae43f534ab79a867146dadd59dc9ca5c8b53d5c8f7c9ef02 \
|
||||
--hash=sha256:e9c26f834c65f5752f3f06cb08cb86a913ceb7274d0db6e267808a708b46bc89 \
|
||||
--hash=sha256:ea793e075b70290d89d8142074262885d3f7da19634845135751bd6344f73b50 \
|
||||
--hash=sha256:f027324c56cd5406ca49c124b0db10e56c69064fec039acc571c29020cc87c76 \
|
||||
--hash=sha256:f13a646d65d09fbf1bc6b3a9635d30095c8e7e5cc419ff35ecc563c5fd04cd49 \
|
||||
--hash=sha256:f47286a97f0bc9b8859519809077b91b2cefe4ae47fcbf5e466a009c1c5d742b \
|
||||
--hash=sha256:f747929cf940cddb5b3668a390056ddd5ba2e5010615ea2dcf4f9c4f3ab8791d \
|
||||
--hash=sha256:f99626688942fb746e545232e7726926f3be91b5975f8b55327665fafda991c7 \
|
||||
--hash=sha256:f9fa868638bf362d3d138ea55829cefb3d5f4b0d7f142234382a15e2485dbec4 \
|
||||
--hash=sha256:fbdb89b3e1c94a30cc5edfce477c6e6a5dc4d8f84665b455c27582f211a1c72c \
|
||||
--hash=sha256:fc010ab034c8c7452522748bf937df58020d256ccae0874463d1f4d01758af8e \
|
||||
--hash=sha256:fc3e9034a63de20e15e8ade85358bc6efc614008cab72898b4b4952bea0509ff \
|
||||
--hash=sha256:fd8b3d9fd264be37976686c7f65cd52a83f5e84f4bfd2adf9c1d469676bbb6ae
|
||||
# via pydantic
|
||||
starlette==1.3.1 \
|
||||
--hash=sha256:05d0213193f2fbaae60e2ecb593b4add4262ad4e46536b54abe36f11a71724e0 \
|
||||
--hash=sha256:c7372aae11c3c3f26a42df7bd626cec2f47d03483d261d369516a615a53714c6
|
||||
# via fastapi
|
||||
typing-extensions==4.16.0 \
|
||||
--hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \
|
||||
--hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5
|
||||
# via
|
||||
# fastapi
|
||||
# pydantic
|
||||
# pydantic-core
|
||||
# typing-inspection
|
||||
typing-inspection==0.4.2 \
|
||||
--hash=sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7 \
|
||||
--hash=sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464
|
||||
# via
|
||||
# fastapi
|
||||
# pydantic
|
||||
uvicorn==0.51.0 \
|
||||
--hash=sha256:5d38af6cd620f2ae3849fb44fd4879e0890aa1febe8d47eb355fb45d93fe6a5b \
|
||||
--hash=sha256:f6f4b69b657c312f516dd2d268ab9ae6f254b11e4bac504f37b2ab58b24dd0b0
|
||||
# via -r requirements.orchestrator.in
|
||||
@@ -30,6 +30,7 @@ bot_bottle/manifest/agent.py
|
||||
bot_bottle/manifest/bottle.py
|
||||
bot_bottle/manifest/egress.py
|
||||
bot_bottle/manifest/extends.py
|
||||
bot_bottle/manifest/forge.py
|
||||
bot_bottle/manifest/git.py
|
||||
bot_bottle/manifest/index.py
|
||||
bot_bottle/manifest/loader.py
|
||||
|
||||
@@ -26,7 +26,6 @@ _BUNDLED_RESOURCES = (
|
||||
"Dockerfile.orchestrator",
|
||||
"Dockerfile.orchestrator.fc",
|
||||
"requirements.gateway.lock",
|
||||
"requirements.orchestrator.lock",
|
||||
"nix/firecracker-netpool.nix",
|
||||
"scripts/firecracker-netpool.sh",
|
||||
)
|
||||
|
||||
@@ -17,7 +17,6 @@ from bot_bottle.cli.commands import cleanup as cmd
|
||||
def _make_backend(empty: bool = True):
|
||||
backend = MagicMock()
|
||||
plan = MagicMock(empty=empty)
|
||||
plan.intersect.return_value = plan
|
||||
backend.prepare_cleanup.return_value = plan
|
||||
backend.cleanup = MagicMock()
|
||||
return backend, plan
|
||||
@@ -42,8 +41,8 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
self.assertEqual(2, docker.prepare_cleanup.call_count)
|
||||
self.assertEqual(2, fc.prepare_cleanup.call_count)
|
||||
docker.prepare_cleanup.assert_called_once()
|
||||
fc.prepare_cleanup.assert_called_once()
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
fc.cleanup.assert_called_once_with(fc_plan)
|
||||
|
||||
@@ -69,7 +68,7 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
self.assertEqual(2, docker.prepare_cleanup.call_count)
|
||||
docker.prepare_cleanup.assert_called_once()
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
macos.prepare_cleanup.assert_not_called()
|
||||
|
||||
@@ -136,28 +135,6 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
fc.cleanup.assert_not_called()
|
||||
|
||||
def test_executes_only_displayed_resources_still_current(self):
|
||||
backend = MagicMock()
|
||||
preview = MagicMock(empty=False)
|
||||
refreshed = MagicMock(empty=False)
|
||||
approved = MagicMock(empty=False)
|
||||
preview.intersect.return_value = approved
|
||||
backend.prepare_cleanup.side_effect = [preview, refreshed]
|
||||
|
||||
with patch.object(
|
||||
cmd, "known_backend_names", return_value=("firecracker",),
|
||||
), patch.object(
|
||||
cmd, "get_bottle_backend", return_value=backend,
|
||||
), patch.object(
|
||||
cmd, "has_backend", return_value=True,
|
||||
), patch.object(
|
||||
cmd, "_prompt_yes", return_value=True,
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
preview.intersect.assert_called_once_with(refreshed)
|
||||
backend.cleanup.assert_called_once_with(approved)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -18,7 +18,6 @@ from bot_bottle.backend.docker.compose import (
|
||||
list_compose_projects,
|
||||
slug_from_compose_project,
|
||||
)
|
||||
from bot_bottle.backend import EnumerationError
|
||||
|
||||
|
||||
class TestProjectNaming(unittest.TestCase):
|
||||
@@ -70,19 +69,6 @@ class TestComposeProjectListing(unittest.TestCase):
|
||||
self.assertEqual([], list_active_slugs(warn_on_error=False))
|
||||
warn.assert_not_called()
|
||||
|
||||
def test_compose_ls_error_can_be_raised_for_enumeration(self):
|
||||
with mock.patch(
|
||||
"bot_bottle.backend.docker.compose.subprocess.run",
|
||||
return_value=subprocess.CompletedProcess(
|
||||
args=["docker"], returncode=1, stdout="", stderr="no daemon",
|
||||
),
|
||||
):
|
||||
with self.assertRaisesRegex(EnumerationError, "no daemon"):
|
||||
list_active_slugs(
|
||||
warn_on_error=False,
|
||||
raise_on_error=True,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -7,8 +7,6 @@ from pathlib import Path
|
||||
from unittest.mock import MagicMock, Mock, patch
|
||||
|
||||
from bot_bottle.backend.docker.consolidated_launch import (
|
||||
ConsolidatedLaunchError,
|
||||
_network_container_ips,
|
||||
launch_consolidated,
|
||||
deprovision_consolidated,
|
||||
)
|
||||
@@ -88,16 +86,6 @@ class TestLaunchConsolidated(unittest.TestCase):
|
||||
client.teardown_bottle.assert_called_once_with("b1") # no orphan left
|
||||
|
||||
|
||||
class TestNetworkContainerIps(unittest.TestCase):
|
||||
def test_fails_closed_when_network_inspection_fails(self) -> None:
|
||||
result = Mock(returncode=1, stdout="", stderr="daemon unavailable")
|
||||
with (
|
||||
patch(f"{_MOD}.run_docker", return_value=result),
|
||||
self.assertRaisesRegex(ConsolidatedLaunchError, "daemon unavailable"),
|
||||
):
|
||||
_network_container_ips("bot-bottle-gateway")
|
||||
|
||||
|
||||
class TestTeardownConsolidated(unittest.TestCase):
|
||||
def test_deregisters_and_deprovisions(self) -> None:
|
||||
client = Mock()
|
||||
|
||||
@@ -14,12 +14,9 @@ from __future__ import annotations
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from tests.unit import use_bottle_root
|
||||
from bot_bottle import bottle_state
|
||||
from bot_bottle.backend import EnumerationError
|
||||
from bot_bottle.backend.docker import cleanup
|
||||
from bot_bottle.backend.docker.cleanup import _list_orphan_state_dirs
|
||||
|
||||
|
||||
@@ -119,37 +116,5 @@ class TestOrphanStateDirs(_FakeHomeMixin, unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestAuthoritativeDiscovery(unittest.TestCase):
|
||||
def test_prepare_requires_authoritative_compose_projects(self):
|
||||
with patch.object(cleanup.docker_mod, "require_docker"), \
|
||||
patch.object(
|
||||
cleanup, "list_compose_projects",
|
||||
side_effect=EnumerationError("compose unavailable"),
|
||||
) as projects, self.assertRaisesRegex(
|
||||
EnumerationError, "compose unavailable",
|
||||
):
|
||||
cleanup.prepare_cleanup()
|
||||
projects.assert_called_once_with(
|
||||
warn_on_error=False,
|
||||
raise_on_error=True,
|
||||
)
|
||||
|
||||
def test_container_query_failure_raises(self):
|
||||
failed = cleanup.subprocess.CompletedProcess(
|
||||
[], 1, stdout="", stderr="daemon unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=failed), \
|
||||
self.assertRaisesRegex(EnumerationError, "daemon unavailable"):
|
||||
cleanup._list_prefixed_containers()
|
||||
|
||||
def test_network_query_failure_raises(self):
|
||||
failed = cleanup.subprocess.CompletedProcess(
|
||||
[], 1, stdout="", stderr="daemon unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=failed), \
|
||||
self.assertRaisesRegex(EnumerationError, "daemon unavailable"):
|
||||
cleanup._list_prefixed_networks()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -19,16 +19,13 @@ of issue #77 — the dashboard now delegates to this layer.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from tests.unit import use_bottle_root
|
||||
from bot_bottle import bottle_state
|
||||
from bot_bottle.backend.docker import enumerate as _enumerate
|
||||
from bot_bottle.backend import EnumerationError
|
||||
|
||||
|
||||
class TestParseServicesByProject(unittest.TestCase):
|
||||
@@ -75,18 +72,6 @@ class TestParseServicesByProject(unittest.TestCase):
|
||||
self.assertEqual({"bot-bottle-dev-abc": {"egress"}}, out)
|
||||
|
||||
|
||||
class TestQueryServicesByProject(unittest.TestCase):
|
||||
def test_docker_ps_failure_is_not_an_empty_result(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.docker.enumerate.subprocess.run",
|
||||
return_value=subprocess.CompletedProcess(
|
||||
args=["docker"], returncode=1, stdout="", stderr="daemon down",
|
||||
),
|
||||
):
|
||||
with self.assertRaisesRegex(EnumerationError, "daemon down"):
|
||||
_enumerate._query_services_by_project()
|
||||
|
||||
|
||||
class _FakeHomeMixin:
|
||||
def _setup_fake_home(self) -> None:
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="enum-active.")
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
@@ -21,7 +22,7 @@ from bot_bottle.backend import Bottle, BottleSpec, ExecResult
|
||||
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
||||
from bot_bottle.egress import EgressPlan
|
||||
from bot_bottle.git_gate import GitGatePlan
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
from bot_bottle.manifest import ManifestGitUser, ManifestIndex
|
||||
|
||||
|
||||
class _Provider(AgentProvider):
|
||||
@@ -50,15 +51,39 @@ def _plan(*, git_user: dict | None = None, # type: ignore
|
||||
user_cwd: str = "/tmp/x",
|
||||
stage_dir: Path | None = None) -> DockerBottlePlan:
|
||||
bottle_json: dict = {} # type: ignore
|
||||
if git_user is not None:
|
||||
bottle_json["git-gate"] = {"user": git_user}
|
||||
if git_repos is not None:
|
||||
bottle_json.setdefault("git-gate", {})["repos"] = git_repos
|
||||
# Identity now lives on the agent's `author` block; at composition it
|
||||
# populates manifest.bottle.git_user, which provision_git reads
|
||||
# (production unchanged). When the caller passes a full name+email we
|
||||
# route it through `author`; the name-only / email-only cases (which
|
||||
# exercise provision_git emitting a single `git config` line) can't be
|
||||
# expressed via `author` (both fields required), so we inject the
|
||||
# partial ManifestGitUser onto the composed bottle directly.
|
||||
agent_json: dict = {"skills": [], "prompt": "", "bottle": "dev"} # type: ignore
|
||||
full_author = (
|
||||
git_user
|
||||
if git_user and git_user.get("name") and git_user.get("email")
|
||||
else None
|
||||
)
|
||||
if full_author is not None:
|
||||
agent_json["author"] = full_author
|
||||
index = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": bottle_json},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
"agents": {"demo": agent_json},
|
||||
})
|
||||
manifest = index.load_for_agent("demo")
|
||||
if git_user is not None and full_author is None:
|
||||
manifest = replace(
|
||||
manifest,
|
||||
bottle=replace(
|
||||
manifest.bottle,
|
||||
git_user=ManifestGitUser(
|
||||
name=git_user.get("name", ""),
|
||||
email=git_user.get("email", ""),
|
||||
),
|
||||
),
|
||||
)
|
||||
spec = BottleSpec(
|
||||
manifest=index, agent_name="demo",
|
||||
copy_cwd=copy_cwd, user_cwd=user_cwd,
|
||||
|
||||
@@ -1,81 +0,0 @@
|
||||
"""Unit tests for framework-neutral outbound DLP request stages."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from bot_bottle.gateway.egress.outbound_pipeline import (
|
||||
MutableHeaders,
|
||||
redact_request,
|
||||
scan_request,
|
||||
)
|
||||
from bot_bottle.gateway.egress.types import Route
|
||||
|
||||
|
||||
class _Headers(dict[str, str]):
|
||||
pass
|
||||
|
||||
|
||||
class _Request:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
host: str = "api.example.com",
|
||||
path: str = "/v1/messages",
|
||||
headers: dict[str, str] | None = None,
|
||||
body: str = "",
|
||||
) -> None:
|
||||
self.pretty_host = host
|
||||
self.path = path
|
||||
self.headers: MutableHeaders = _Headers(headers or {})
|
||||
self.text = body
|
||||
|
||||
def get_text(self, strict: bool = False) -> str | None:
|
||||
del strict
|
||||
return self.text
|
||||
|
||||
|
||||
class TestOutboundScan(unittest.TestCase):
|
||||
def test_detects_secret_in_body(self) -> None:
|
||||
request = _Request(body="token=sk-" + "a" * 48)
|
||||
|
||||
result = scan_request(request, Route(host="api.example.com"), {})
|
||||
|
||||
self.assertIsNotNone(result)
|
||||
self.assertEqual("block", result.severity if result else None)
|
||||
|
||||
def test_safe_token_is_ignored(self) -> None:
|
||||
token = "sk-" + "a" * 48
|
||||
request = _Request(body=f"token={token}")
|
||||
|
||||
result = scan_request(
|
||||
request,
|
||||
Route(host="api.example.com"),
|
||||
{},
|
||||
safe_tokens={token},
|
||||
)
|
||||
|
||||
self.assertIsNone(result)
|
||||
|
||||
|
||||
class TestOutboundRedaction(unittest.TestCase):
|
||||
def test_redacts_body_header_and_path_but_preserves_host(self) -> None:
|
||||
token = "sk-" + "a" * 48
|
||||
request = _Request(
|
||||
path=f"/v1/messages?key={token}",
|
||||
headers={"Host": "api.example.com", "X-Token": token + "\r\nInjected: yes"},
|
||||
body=f"token={token}",
|
||||
)
|
||||
|
||||
clean = redact_request(request, Route(host="api.example.com"), {})
|
||||
|
||||
self.assertTrue(clean)
|
||||
self.assertNotIn(token, request.path)
|
||||
self.assertNotIn(token, request.headers["X-Token"])
|
||||
self.assertNotIn("\r", request.headers["X-Token"])
|
||||
self.assertNotIn(token, request.text)
|
||||
self.assertEqual("api.example.com", request.headers["Host"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,89 +0,0 @@
|
||||
"""Unit tests for framework-neutral egress request policy stages."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from bot_bottle.gateway.egress.request_pipeline import (
|
||||
GIT_PUSH_BLOCK_REASON,
|
||||
evaluate_route_policy,
|
||||
git_block_reason,
|
||||
)
|
||||
from bot_bottle.gateway.egress.types import Config, LOG_FULL, Route
|
||||
|
||||
|
||||
class TestGitPolicy(unittest.TestCase):
|
||||
def test_push_is_always_blocked(self) -> None:
|
||||
reason = git_block_reason(
|
||||
(), "git.example.com", "/repo.git/git-receive-pack", "",
|
||||
)
|
||||
self.assertEqual(GIT_PUSH_BLOCK_REASON, reason)
|
||||
|
||||
def test_fetch_requires_route_opt_in(self) -> None:
|
||||
path = "/repo.git/git-upload-pack"
|
||||
blocked = git_block_reason((), "git.example.com", path, "")
|
||||
allowed = git_block_reason(
|
||||
(Route(host="git.example.com", git_fetch=True),),
|
||||
"git.example.com",
|
||||
path,
|
||||
"",
|
||||
)
|
||||
self.assertTrue(blocked)
|
||||
self.assertEqual("", allowed)
|
||||
|
||||
def test_non_git_request_is_not_decided_here(self) -> None:
|
||||
self.assertEqual(
|
||||
"",
|
||||
git_block_reason((), "api.example.com", "/v1/messages", ""),
|
||||
)
|
||||
|
||||
|
||||
class TestRoutePolicy(unittest.TestCase):
|
||||
def test_strips_agent_auth_and_injects_gateway_auth(self) -> None:
|
||||
route = Route(
|
||||
host="api.example.com",
|
||||
auth_scheme="Bearer",
|
||||
token_env="API_TOKEN",
|
||||
)
|
||||
result = evaluate_route_policy(
|
||||
Config(routes=(route,)),
|
||||
route,
|
||||
host="api.example.com",
|
||||
request_path="/v1/messages",
|
||||
method="POST",
|
||||
headers={"Authorization": "agent-secret"},
|
||||
env={"API_TOKEN": "gateway-secret"},
|
||||
)
|
||||
self.assertTrue(result.strip_authorization)
|
||||
self.assertEqual("Bearer gateway-secret", result.inject_authorization)
|
||||
self.assertFalse(result.block_reason)
|
||||
|
||||
def test_preserved_auth_participates_in_matching(self) -> None:
|
||||
route = Route(host="registry.example.com", preserve_auth=True)
|
||||
result = evaluate_route_policy(
|
||||
Config(routes=(route,), log=LOG_FULL),
|
||||
route,
|
||||
host="registry.example.com",
|
||||
request_path="/v2/",
|
||||
method="GET",
|
||||
headers={"Authorization": "Bearer agent-token"},
|
||||
env={},
|
||||
)
|
||||
self.assertFalse(result.strip_authorization)
|
||||
self.assertTrue(result.log_request)
|
||||
|
||||
def test_missing_route_fails_closed(self) -> None:
|
||||
result = evaluate_route_policy(
|
||||
Config(routes=(), deny_reason="not allowed"),
|
||||
None,
|
||||
host="blocked.example.com",
|
||||
request_path="/",
|
||||
method="GET",
|
||||
headers={},
|
||||
env={},
|
||||
)
|
||||
self.assertEqual("not allowed", result.block_reason)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -15,7 +15,6 @@ import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend import EnumerationError
|
||||
from bot_bottle.backend.firecracker import cleanup as fc_cleanup
|
||||
from bot_bottle.backend.firecracker.bottle_cleanup_plan import (
|
||||
FirecrackerBottleCleanupPlan,
|
||||
@@ -32,19 +31,15 @@ class TestProcessScan(unittest.TestCase):
|
||||
self.assertEqual(
|
||||
Path("/cache/run/dev-a"),
|
||||
fc_cleanup._run_dir_of(
|
||||
("firecracker", "--config-file",
|
||||
f"{run_root}/dev-a/config.json"), run_root
|
||||
f"firecracker --config-file {run_root}/dev-a/config.json", run_root
|
||||
),
|
||||
)
|
||||
# infra/builder VMs elsewhere, or nested paths, are not ours.
|
||||
self.assertIsNone(
|
||||
fc_cleanup._run_dir_of(
|
||||
("firecracker", "--config-file", "/elsewhere/config.json"),
|
||||
run_root,
|
||||
)
|
||||
fc_cleanup._run_dir_of("firecracker --config-file /elsewhere/config.json", run_root)
|
||||
)
|
||||
self.assertIsNone(
|
||||
fc_cleanup._run_dir_of(("firecracker", "--no-config"), run_root)
|
||||
fc_cleanup._run_dir_of("firecracker --no-config", run_root)
|
||||
)
|
||||
|
||||
def test_scan_splits_live_dirs_from_orphan_pids(self):
|
||||
@@ -52,71 +47,21 @@ class TestProcessScan(unittest.TestCase):
|
||||
run_root = Path(tmp)
|
||||
(run_root / "live-a").mkdir() # dir present -> live VM, protected
|
||||
# "gone-b" dir intentionally absent -> lingering VMM, orphan pid
|
||||
args = {
|
||||
111: ("firecracker", "--config-file",
|
||||
f"{run_root}/live-a/config.json"),
|
||||
222: ("firecracker", "--config-file",
|
||||
f"{run_root}/gone-b/config.json"),
|
||||
333: ("firecracker", "--config-file", "/elsewhere/config.json"),
|
||||
}
|
||||
with patch.object(
|
||||
fc_cleanup.subprocess, "run",
|
||||
return_value=_proc("111\n222\n333\nnotanint\n"),
|
||||
), patch.object(
|
||||
fc_cleanup, "_process_args", side_effect=args.get,
|
||||
):
|
||||
out = (
|
||||
f"111 firecracker --config-file {run_root}/live-a/config.json\n"
|
||||
f"222 firecracker --config-file {run_root}/gone-b/config.json\n"
|
||||
"333 firecracker --config-file /elsewhere/config.json\n"
|
||||
"notanint firecracker --config-file x\n"
|
||||
)
|
||||
with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(out)):
|
||||
live, orphan_pids = fc_cleanup._scan_processes(run_root)
|
||||
self.assertEqual({str(run_root / "live-a")}, live)
|
||||
self.assertEqual([222], orphan_pids)
|
||||
|
||||
def test_scan_preserves_spaces_in_config_path(self):
|
||||
with tempfile.TemporaryDirectory(prefix="fc cache ") as tmp:
|
||||
run_root = Path(tmp)
|
||||
live = run_root / "live bottle"
|
||||
live.mkdir()
|
||||
with patch.object(
|
||||
fc_cleanup.subprocess, "run", return_value=_proc("111\n"),
|
||||
), patch.object(
|
||||
fc_cleanup, "_process_args",
|
||||
return_value=("firecracker", "--config-file",
|
||||
str(live / "config.json")),
|
||||
):
|
||||
self.assertEqual(
|
||||
({str(live)}, []),
|
||||
fc_cleanup._scan_processes(run_root),
|
||||
)
|
||||
|
||||
def test_scan_empty_when_pgrep_finds_no_processes(self):
|
||||
def test_scan_empty_when_pgrep_fails(self):
|
||||
with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(returncode=1)):
|
||||
self.assertEqual((set(), []), fc_cleanup._scan_processes(Path("/x")))
|
||||
|
||||
def test_scan_raises_when_pgrep_errors(self):
|
||||
proc = subprocess.CompletedProcess(
|
||||
[], 2, stdout="", stderr="invalid process expression",
|
||||
)
|
||||
with (
|
||||
patch.object(fc_cleanup.subprocess, "run", return_value=proc),
|
||||
self.assertRaisesRegex(EnumerationError, "invalid process expression"),
|
||||
):
|
||||
fc_cleanup._scan_processes(Path("/x"))
|
||||
|
||||
def test_prepare_cleanup_does_not_plan_deletions_when_scan_errors(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
run_root = Path(tmp)
|
||||
live = run_root / "live-a"
|
||||
live.mkdir()
|
||||
with (
|
||||
patch.object(fc_cleanup, "_run_root", return_value=run_root),
|
||||
patch.object(
|
||||
fc_cleanup.subprocess,
|
||||
"run",
|
||||
return_value=_proc(returncode=2),
|
||||
),
|
||||
self.assertRaises(EnumerationError),
|
||||
):
|
||||
fc_cleanup.prepare_cleanup()
|
||||
self.assertTrue(live.is_dir())
|
||||
|
||||
def test_live_run_dirs_returns_paths_in_stable_order(self):
|
||||
with patch.object(fc_cleanup, "_run_root", return_value=Path("/run")), \
|
||||
patch.object(fc_cleanup, "_scan_processes",
|
||||
@@ -144,14 +89,9 @@ class TestProcessScan(unittest.TestCase):
|
||||
run_root = Path(tmp)
|
||||
(run_root / "live-a").mkdir()
|
||||
(run_root / "dead-b").mkdir()
|
||||
out = f"111 firecracker --config-file {run_root}/live-a/config.json\n"
|
||||
with patch.object(fc_cleanup, "_run_root", return_value=run_root), \
|
||||
patch.object(fc_cleanup.subprocess, "run",
|
||||
return_value=_proc("111\n")), \
|
||||
patch.object(
|
||||
fc_cleanup, "_process_args",
|
||||
return_value=("firecracker", "--config-file",
|
||||
str(run_root / "live-a/config.json")),
|
||||
):
|
||||
patch.object(fc_cleanup.subprocess, "run", return_value=_proc(out)):
|
||||
plan = fc_cleanup.prepare_cleanup()
|
||||
self.assertEqual((), plan.vm_pids)
|
||||
self.assertEqual((str(run_root / "dead-b"),), plan.run_dirs)
|
||||
@@ -164,46 +104,18 @@ class TestCleanupRemoval(unittest.TestCase):
|
||||
vm_pids=(101,),
|
||||
run_dirs=("/run/dev-x",),
|
||||
)
|
||||
with patch.object(fc_cleanup, "prepare_cleanup", return_value=plan), \
|
||||
patch.object(fc_cleanup, "_run_root", return_value=Path("/run")), \
|
||||
patch.object(fc_cleanup, "_terminate_orphan") as terminate, \
|
||||
patch("bot_bottle.backend.cleanup_control.shutil.rmtree") as rmtree, \
|
||||
with patch.object(fc_cleanup.os, "kill") as kill, \
|
||||
patch.object(fc_cleanup.shutil, "rmtree") as rmtree, \
|
||||
patch.object(fc_cleanup, "info"):
|
||||
fc_cleanup.cleanup(plan)
|
||||
terminate.assert_called_once_with(101, Path("/run"))
|
||||
rmtree.assert_called_once_with(Path("/run/dev-x"))
|
||||
kill.assert_called_once()
|
||||
rmtree.assert_called_once_with("/run/dev-x", ignore_errors=True)
|
||||
|
||||
def test_cleanup_skips_resources_no_longer_in_refreshed_plan(self):
|
||||
preview = FirecrackerBottleCleanupPlan(
|
||||
vm_pids=(999,), run_dirs=("/run/reused",),
|
||||
)
|
||||
with patch.object(
|
||||
fc_cleanup, "prepare_cleanup",
|
||||
return_value=FirecrackerBottleCleanupPlan(),
|
||||
), patch.object(fc_cleanup, "_terminate_orphan") as terminate, \
|
||||
patch("bot_bottle.backend.cleanup_control.shutil.rmtree") as rmtree:
|
||||
fc_cleanup.cleanup(preview)
|
||||
terminate.assert_not_called()
|
||||
rmtree.assert_not_called()
|
||||
|
||||
def test_pidfd_prevents_pid_reuse_from_signalling_unrelated_process(self):
|
||||
with patch.object(fc_cleanup.os, "pidfd_open", return_value=7), \
|
||||
patch.object(
|
||||
fc_cleanup.Path, "read_bytes",
|
||||
return_value=b"/usr/bin/python\0worker.py\0",
|
||||
), patch.object(fc_cleanup.signal, "pidfd_send_signal") as send, \
|
||||
patch.object(fc_cleanup.os, "close"):
|
||||
fc_cleanup._terminate_orphan(101, Path("/run"))
|
||||
send.assert_not_called()
|
||||
|
||||
def test_pidfd_signals_revalidated_orphan(self):
|
||||
command = b"firecracker\0--config-file\0/run/gone/config.json\0"
|
||||
with patch.object(fc_cleanup.os, "pidfd_open", return_value=7), \
|
||||
patch.object(fc_cleanup.Path, "read_bytes", return_value=command), \
|
||||
patch.object(fc_cleanup.signal, "pidfd_send_signal") as send, \
|
||||
patch.object(fc_cleanup.os, "close"), patch.object(fc_cleanup, "info"):
|
||||
fc_cleanup._terminate_orphan(101, Path("/run"))
|
||||
send.assert_called_once_with(7, fc_cleanup.signal.SIGTERM)
|
||||
def test_cleanup_tolerates_dead_pid(self):
|
||||
plan = FirecrackerBottleCleanupPlan(vm_pids=(999,))
|
||||
with patch.object(fc_cleanup.os, "kill", side_effect=ProcessLookupError), \
|
||||
patch.object(fc_cleanup, "info"):
|
||||
fc_cleanup.cleanup(plan) # must not raise
|
||||
|
||||
|
||||
class TestCleanupPlan(unittest.TestCase):
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Unit tests for Firecracker active-agent enumeration."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend import EnumerationError
|
||||
from bot_bottle.backend.firecracker import enumerate as fc_enumerate
|
||||
from bot_bottle.bottle_state import BottleMetadata
|
||||
|
||||
|
||||
class TestEnumerateActive(unittest.TestCase):
|
||||
def test_maps_live_run_dirs_to_active_agents(self) -> None:
|
||||
metadata = BottleMetadata(
|
||||
identity="dev-a",
|
||||
agent_name="claude",
|
||||
cwd="",
|
||||
copy_cwd=False,
|
||||
started_at="2026-07-26T12:00:00Z",
|
||||
label="review",
|
||||
color="blue",
|
||||
)
|
||||
with (
|
||||
patch.object(
|
||||
fc_enumerate, "live_run_dirs",
|
||||
return_value=(Path("/cache/run/dev-a"),),
|
||||
),
|
||||
patch.object(fc_enumerate, "read_metadata", return_value=metadata),
|
||||
):
|
||||
agents = fc_enumerate.enumerate_active()
|
||||
self.assertEqual(1, len(agents))
|
||||
self.assertEqual("firecracker", agents[0].backend_name)
|
||||
self.assertEqual("dev-a", agents[0].slug)
|
||||
self.assertEqual("claude", agents[0].agent_name)
|
||||
self.assertEqual("review", agents[0].label)
|
||||
self.assertEqual((), agents[0].services)
|
||||
|
||||
def test_missing_metadata_uses_safe_defaults(self) -> None:
|
||||
with (
|
||||
patch.object(
|
||||
fc_enumerate, "live_run_dirs",
|
||||
return_value=(Path("/cache/run/dev-a"),),
|
||||
),
|
||||
patch.object(fc_enumerate, "read_metadata", return_value=None),
|
||||
):
|
||||
agent = fc_enumerate.enumerate_active()[0]
|
||||
self.assertEqual("?", agent.agent_name)
|
||||
self.assertEqual("", agent.started_at)
|
||||
|
||||
def test_process_scan_failure_propagates(self) -> None:
|
||||
with (
|
||||
patch.object(
|
||||
fc_enumerate, "live_run_dirs",
|
||||
side_effect=EnumerationError("pgrep failed"),
|
||||
),
|
||||
self.assertRaisesRegex(EnumerationError, "pgrep failed"),
|
||||
):
|
||||
fc_enumerate.enumerate_active()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,426 @@
|
||||
"""Unit: trusted agent forge identity & guidance
|
||||
(PRD 0082).
|
||||
|
||||
Covers the net-new surface: agent `author`, agent `forge-accounts` (Gitea API
|
||||
URL validation + host token reference), the repo→forge association resolved at
|
||||
composition, the synthesized proxy-held egress route, and the generated,
|
||||
non-secret prompt guidance. Legacy git-gate.user / cwd-agent behavior lives in
|
||||
the manifest test modules.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.egress import (
|
||||
egress_forge_routes,
|
||||
egress_render_routes,
|
||||
egress_resolve_token_values,
|
||||
egress_routes_for_bottle,
|
||||
egress_token_env_map,
|
||||
)
|
||||
from bot_bottle.manifest import ManifestError, ManifestIndex
|
||||
from bot_bottle.manifest.forge import (
|
||||
ManifestForgeAccount,
|
||||
canonicalize_forge_url,
|
||||
render_forge_guidance,
|
||||
)
|
||||
|
||||
|
||||
GITEA = {
|
||||
"url": "https://gitea.dideric.is/api/v1",
|
||||
"auth": {"type": "token", "token_secret": "GITEA_CLAUDE_TOKEN"},
|
||||
}
|
||||
|
||||
|
||||
def _repo(*, forge: str | None = None) -> dict[str, object]:
|
||||
entry: dict[str, object] = {
|
||||
"url": "ssh://git@100.78.141.42:30009/didericis/bot-bottle.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
}
|
||||
if forge is not None:
|
||||
entry["forge"] = forge
|
||||
return entry
|
||||
|
||||
|
||||
def _index(
|
||||
*,
|
||||
author: dict[str, object] | None = None,
|
||||
forge_accounts: dict[str, object] | None = None,
|
||||
repo_forge: str | None = None,
|
||||
) -> ManifestIndex:
|
||||
"""Build an eager index with one agent 'claude' + bottle 'bb'."""
|
||||
agent: dict[str, object] = {"bottle": "bb", "prompt": ""}
|
||||
if author is not None:
|
||||
agent["author"] = author
|
||||
if forge_accounts is not None:
|
||||
agent["forge-accounts"] = forge_accounts
|
||||
bottle: dict[str, object] = {
|
||||
"git-gate": {"repos": {"bot-bottle": _repo(forge=repo_forge)}}
|
||||
}
|
||||
return ManifestIndex.from_json_obj(
|
||||
{"bottles": {"bb": bottle}, "agents": {"claude": agent}}
|
||||
)
|
||||
|
||||
|
||||
def _error(
|
||||
callable_: Callable[..., object], *args: object, **kwargs: object,
|
||||
) -> str:
|
||||
try:
|
||||
callable_(*args, **kwargs)
|
||||
except ManifestError as e:
|
||||
return str(e)
|
||||
raise AssertionError("expected ManifestError was not raised")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# author
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAuthor(unittest.TestCase):
|
||||
def test_populates_git_identity(self) -> None:
|
||||
idx = _index(author={"name": "didericis-claude", "email": "e+c@x.is"})
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual("didericis-claude", m.bottle.git_user.name)
|
||||
self.assertEqual("e+c@x.is", m.bottle.git_user.email)
|
||||
self.assertEqual(
|
||||
"name=didericis-claude, email=e+c@x.is",
|
||||
m.git_identity_summary(),
|
||||
)
|
||||
|
||||
def test_absent_author_no_identity(self) -> None:
|
||||
m = _index().load_for_agent("claude", ())
|
||||
self.assertTrue(m.bottle.git_user.is_empty())
|
||||
self.assertIsNone(m.git_identity_summary())
|
||||
|
||||
def test_name_required(self) -> None:
|
||||
msg = _error(_index, author={"email": "e@x.is"})
|
||||
self.assertIn("author.name must be a non-empty string", msg)
|
||||
|
||||
def test_email_required(self) -> None:
|
||||
msg = _error(_index, author={"name": "n"})
|
||||
self.assertIn("author.email must be a non-empty string", msg)
|
||||
|
||||
def test_email_rejects_whitespace(self) -> None:
|
||||
msg = _error(_index, author={"name": "n", "email": "a b@x.is"})
|
||||
self.assertIn("must not contain whitespace", msg)
|
||||
|
||||
def test_name_rejects_newline(self) -> None:
|
||||
msg = _error(_index, author={"name": "a\nb", "email": "e@x.is"})
|
||||
self.assertIn("author.name must not contain newlines", msg)
|
||||
|
||||
def test_unknown_key(self) -> None:
|
||||
msg = _error(
|
||||
_index, author={"name": "n", "email": "e@x.is", "role": "x"}
|
||||
)
|
||||
self.assertIn("author has unknown key", msg)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# forge-accounts URL validation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeUrl(unittest.TestCase):
|
||||
def test_canonical_ok(self) -> None:
|
||||
canonical, origin, host, prefix = canonicalize_forge_url(
|
||||
"a", "g", "https://Gitea.Dideric.is/api/v1/"
|
||||
)
|
||||
# trailing slash normalized; host lowercased.
|
||||
self.assertEqual("https://gitea.dideric.is/api/v1", canonical)
|
||||
self.assertEqual("https://gitea.dideric.is", origin)
|
||||
self.assertEqual("gitea.dideric.is", host)
|
||||
self.assertEqual("/api/v1", prefix)
|
||||
|
||||
def test_port_preserved(self) -> None:
|
||||
canonical, origin, _host, _ = canonicalize_forge_url(
|
||||
"a", "g", "https://gitea.local:3000/api/v1"
|
||||
)
|
||||
self.assertEqual("https://gitea.local:3000/api/v1", canonical)
|
||||
self.assertEqual("https://gitea.local:3000", origin)
|
||||
|
||||
def test_http_fails(self) -> None:
|
||||
self.assertIn("must use https", _error(
|
||||
canonicalize_forge_url, "a", "g", "http://gitea/api/v1"))
|
||||
|
||||
def test_userinfo_fails(self) -> None:
|
||||
self.assertIn("userinfo", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://u:p@gitea/api/v1"))
|
||||
|
||||
def test_query_fails(self) -> None:
|
||||
self.assertIn("query string", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://gitea/api/v1?x=1"))
|
||||
|
||||
def test_fragment_fails(self) -> None:
|
||||
self.assertIn("fragment", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://gitea/api/v1#x"))
|
||||
|
||||
def test_missing_host_fails(self) -> None:
|
||||
self.assertIn("hostname", _error(
|
||||
canonicalize_forge_url, "a", "g", "https:///api/v1"))
|
||||
|
||||
def test_bad_path_fails(self) -> None:
|
||||
self.assertIn("Gitea API base", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://gitea/api/v2"))
|
||||
|
||||
def test_non_string_fails(self) -> None:
|
||||
self.assertIn("required", _error(
|
||||
canonicalize_forge_url, "a", "g", None))
|
||||
|
||||
def test_malformed_url_fails(self) -> None:
|
||||
# An unparseable URL (invalid IPv6 literal) trips urlsplit's ValueError.
|
||||
self.assertIn("not a valid URL", _error(
|
||||
canonicalize_forge_url, "a", "g", "https://[/api/v1"))
|
||||
|
||||
|
||||
class TestForgeAccount(unittest.TestCase):
|
||||
def test_parses(self) -> None:
|
||||
acct = ManifestForgeAccount.from_dict("a", "didericis-gitea", GITEA)
|
||||
self.assertEqual("didericis-gitea", acct.alias)
|
||||
self.assertEqual("gitea.dideric.is", acct.host)
|
||||
self.assertEqual("token", acct.auth_type)
|
||||
self.assertEqual("GITEA_CLAUDE_TOKEN", acct.token_secret)
|
||||
|
||||
def test_non_kebab_alias_fails(self) -> None:
|
||||
self.assertIn("valid alias", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "Bad_Alias", GITEA))
|
||||
|
||||
def test_auth_required(self) -> None:
|
||||
self.assertIn("missing required 'auth'", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1"}))
|
||||
|
||||
def test_bad_auth_type_fails(self) -> None:
|
||||
self.assertIn("auth.type must be", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1",
|
||||
"auth": {"type": "basic", "token_secret": "T"}}))
|
||||
|
||||
def test_token_secret_required(self) -> None:
|
||||
self.assertIn("token_secret must be", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1", "auth": {"type": "token"}}))
|
||||
|
||||
def test_unknown_key_fails(self) -> None:
|
||||
self.assertIn("unknown key", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{**GITEA, "bogus": 1}))
|
||||
|
||||
def test_unknown_auth_key_fails(self) -> None:
|
||||
self.assertIn("auth has unknown key", _error(
|
||||
ManifestForgeAccount.from_dict, "a", "g",
|
||||
{"url": "https://gitea/api/v1",
|
||||
"auth": {"type": "token", "token_secret": "T", "bogus": 1}}))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# repo -> forge association (composition)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeAssociations(unittest.TestCase):
|
||||
def test_resolves_when_repo_references_alias(self) -> None:
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual(1, len(m.forge_associations))
|
||||
assoc = m.forge_associations[0]
|
||||
self.assertEqual("didericis-gitea", assoc.alias)
|
||||
self.assertEqual(("bot-bottle",), assoc.repo_names)
|
||||
|
||||
def test_unreferenced_account_yields_no_association(self) -> None:
|
||||
idx = _index(forge_accounts={"didericis-gitea": GITEA}) # repo has no forge
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual((), m.forge_associations)
|
||||
|
||||
def test_unknown_alias_fails_closed(self) -> None:
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA}, repo_forge="nope"
|
||||
)
|
||||
msg = _error(idx.load_for_agent, "claude", ())
|
||||
self.assertIn("references forge alias 'nope'", msg)
|
||||
self.assertIn("not defined on agent", msg)
|
||||
|
||||
def test_forge_without_account_fails_closed(self) -> None:
|
||||
idx = _index(repo_forge="didericis-gitea") # no forge-accounts at all
|
||||
self.assertIn("(none)", _error(idx.load_for_agent, "claude", ()))
|
||||
|
||||
def _two_alias_index(self, t1: str, t2: str) -> ManifestIndex:
|
||||
"""Two aliases on the SAME host, each referenced by a distinct repo."""
|
||||
def acct(token: str) -> dict[str, object]:
|
||||
return {
|
||||
"url": "https://same.example/api/v1",
|
||||
"auth": {"type": "token", "token_secret": token},
|
||||
}
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {"bb": {"git-gate": {"repos": {
|
||||
"r1": {"url": "ssh://git@h/x/r1.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
"forge": "g1"},
|
||||
"r2": {"url": "ssh://git@h/x/r2.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
"forge": "g2"},
|
||||
}}}},
|
||||
"agents": {"claude": {"bottle": "bb", "prompt": "",
|
||||
"forge-accounts": {"g1": acct(t1),
|
||||
"g2": acct(t2)}}},
|
||||
})
|
||||
|
||||
def test_conflicting_aliases_same_host_fail_closed(self) -> None:
|
||||
# Two aliases on the same host with DIFFERENT tokens is ambiguous —
|
||||
# the proxy routes by host, so it must fail before launch rather than
|
||||
# silently authenticate every call as one account.
|
||||
idx = self._two_alias_index("FIRST_TOKEN", "SECOND_TOKEN")
|
||||
msg = _error(idx.load_for_agent, "claude", ())
|
||||
self.assertIn("both resolve to host 'same.example'", msg)
|
||||
self.assertIn("ambiguous", msg)
|
||||
|
||||
def test_matching_aliases_same_host_ok(self) -> None:
|
||||
# Identical url/auth under two aliases is unambiguous: one route.
|
||||
idx = self._two_alias_index("SAME_TOKEN", "SAME_TOKEN")
|
||||
m = idx.load_for_agent("claude", ())
|
||||
self.assertEqual(2, len(m.forge_associations)) # both referenced
|
||||
routes = egress_forge_routes(m.forge_associations)
|
||||
self.assertEqual(1, len(routes)) # deduped to a single proxy route
|
||||
self.assertEqual("SAME_TOKEN", routes[0].token_ref)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# synthesized egress route (proxy-held credential)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeEgressRoutes(unittest.TestCase):
|
||||
def _assoc(self):
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
return idx.load_for_agent("claude", ())
|
||||
|
||||
def test_route_shape(self) -> None:
|
||||
routes = egress_forge_routes(self._assoc().forge_associations)
|
||||
self.assertEqual(1, len(routes))
|
||||
r = routes[0]
|
||||
self.assertEqual("gitea.dideric.is", r.host)
|
||||
self.assertEqual("token", r.auth_scheme)
|
||||
self.assertEqual("GITEA_CLAUDE_TOKEN", r.token_ref)
|
||||
self.assertTrue(r.inspect)
|
||||
# scoped to the API prefix.
|
||||
self.assertEqual("/api/v1", r.matches[0].paths[0].value)
|
||||
|
||||
def test_token_slot_and_resolution(self) -> None:
|
||||
m = self._assoc()
|
||||
forge_routes = egress_forge_routes(m.forge_associations)
|
||||
routes = egress_routes_for_bottle(m.bottle, (), forge_routes)
|
||||
token_map = egress_token_env_map(routes)
|
||||
# one slot mapping the proxy env slot -> host env var name.
|
||||
self.assertEqual({"EGRESS_TOKEN_0": "GITEA_CLAUDE_TOKEN"}, token_map)
|
||||
resolved = egress_resolve_token_values(
|
||||
token_map, {"GITEA_CLAUDE_TOKEN": "sekret-value"}
|
||||
)
|
||||
self.assertEqual({"EGRESS_TOKEN_0": "sekret-value"}, resolved)
|
||||
|
||||
def test_rendered_routes_hide_secret_name_and_value(self) -> None:
|
||||
m = self._assoc()
|
||||
routes = egress_routes_for_bottle(
|
||||
m.bottle, (), egress_forge_routes(m.forge_associations)
|
||||
)
|
||||
rendered = egress_render_routes(routes)
|
||||
self.assertIn("gitea.dideric.is", rendered)
|
||||
self.assertIn("EGRESS_TOKEN_0", rendered) # slot, not the host var name
|
||||
self.assertNotIn("GITEA_CLAUDE_TOKEN", rendered)
|
||||
self.assertNotIn("sekret-value", rendered)
|
||||
|
||||
def test_dedup_by_host(self) -> None:
|
||||
# Two repos referencing the same alias share one route/credential.
|
||||
idx = ManifestIndex.from_json_obj({
|
||||
"bottles": {"bb": {"git-gate": {"repos": {
|
||||
"r1": _repo(forge="g"),
|
||||
"r2": {
|
||||
"url": "ssh://git@h/x/other.git",
|
||||
"key": {"provider": "static", "path": "/k"},
|
||||
"forge": "g",
|
||||
},
|
||||
}}}},
|
||||
"agents": {"claude": {"bottle": "bb", "prompt": "",
|
||||
"forge-accounts": {"g": GITEA}}},
|
||||
})
|
||||
m = idx.load_for_agent("claude", ())
|
||||
routes = egress_forge_routes(m.forge_associations)
|
||||
self.assertEqual(1, len(routes))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# generated prompt guidance
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestForgeGuidance(unittest.TestCase):
|
||||
def _assoc(self):
|
||||
idx = _index(
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
return idx.load_for_agent("claude", ()).forge_associations
|
||||
|
||||
def test_empty_when_no_associations(self) -> None:
|
||||
self.assertEqual("", render_forge_guidance(()))
|
||||
|
||||
def test_names_account_repo_and_workflow(self) -> None:
|
||||
text = render_forge_guidance(self._assoc())
|
||||
self.assertIn("didericis-gitea", text)
|
||||
self.assertIn("https://gitea.dideric.is/api/v1", text)
|
||||
self.assertIn("bot-bottle", text)
|
||||
# branch-backed PR workflow + AGit prohibition.
|
||||
self.assertIn("refs/heads/", text)
|
||||
self.assertIn("refs/for/*", text)
|
||||
self.assertIn("pull request", text)
|
||||
|
||||
def test_no_token_secret_name_or_value(self) -> None:
|
||||
text = render_forge_guidance(self._assoc())
|
||||
self.assertNotIn("GITEA_CLAUDE_TOKEN", text)
|
||||
|
||||
def test_prompt_file_appends_guidance(self) -> None:
|
||||
from bot_bottle.backend.resolve_common import prepare_agent_state_dir
|
||||
|
||||
idx = _index(
|
||||
author={"name": "n", "email": "e@x.is"},
|
||||
forge_accounts={"didericis-gitea": GITEA},
|
||||
repo_forge="didericis-gitea",
|
||||
)
|
||||
# Give the agent a real prompt body.
|
||||
m = idx.load_for_agent("claude", ())
|
||||
from dataclasses import replace
|
||||
m = replace(m, agent=replace(m.agent, prompt="BASE PROMPT"))
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_ROOT": td}):
|
||||
_dir, prompt_file = prepare_agent_state_dir("slug1", m)
|
||||
body = Path(prompt_file).read_text()
|
||||
self.assertIn("BASE PROMPT", body)
|
||||
self.assertIn("Forge access", body)
|
||||
self.assertIn("https://gitea.dideric.is/api/v1", body)
|
||||
|
||||
def test_prompt_file_no_guidance_without_forge(self) -> None:
|
||||
from bot_bottle.backend.resolve_common import prepare_agent_state_dir
|
||||
|
||||
m = _index(author={"name": "n", "email": "e@x.is"}).load_for_agent(
|
||||
"claude", ()
|
||||
)
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_ROOT": td}):
|
||||
_dir, prompt_file = prepare_agent_state_dir("slug2", m)
|
||||
body = Path(prompt_file).read_text()
|
||||
self.assertNotIn("Forge access", body)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,79 +0,0 @@
|
||||
"""Unit tests for shared gateway stdlib HTTP resource boundaries."""
|
||||
# pylint: disable=protected-access
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import socket
|
||||
import unittest
|
||||
|
||||
from bot_bottle.gateway.bounded_http import (
|
||||
BodyReadError,
|
||||
BoundedThreadingHTTPServer,
|
||||
read_declared_body,
|
||||
)
|
||||
|
||||
|
||||
class _Handler:
|
||||
pass
|
||||
|
||||
|
||||
class _TimeoutStream:
|
||||
def read(self, _size: int = -1, /) -> bytes:
|
||||
raise TimeoutError
|
||||
|
||||
|
||||
class TestDeclaredBody(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.left, self.right = socket.socketpair()
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.left.close()
|
||||
self.right.close()
|
||||
|
||||
def test_rejects_incomplete_body(self) -> None:
|
||||
with self.assertRaisesRegex(BodyReadError, "incomplete"):
|
||||
read_declared_body(
|
||||
io.BytesIO(b"short"),
|
||||
self.left,
|
||||
"10",
|
||||
maximum=100,
|
||||
timeout_seconds=1,
|
||||
require_length=True,
|
||||
)
|
||||
|
||||
def test_maps_read_timeout(self) -> None:
|
||||
with self.assertRaises(BodyReadError) as caught:
|
||||
read_declared_body(
|
||||
_TimeoutStream(),
|
||||
self.left,
|
||||
"1",
|
||||
maximum=100,
|
||||
timeout_seconds=1,
|
||||
require_length=True,
|
||||
)
|
||||
self.assertEqual(408, caught.exception.status)
|
||||
|
||||
|
||||
class TestBoundedServer(unittest.TestCase):
|
||||
def test_saturated_server_rejects_without_spawning_thread(self) -> None:
|
||||
client, peer = socket.socketpair()
|
||||
with BoundedThreadingHTTPServer(
|
||||
("127.0.0.1", 0), _Handler, max_workers=1, # type: ignore[arg-type]
|
||||
) as server:
|
||||
with client, peer:
|
||||
# Directly reserve the only slot to model an in-flight handler.
|
||||
self.assertTrue(
|
||||
server._request_slots.acquire( # pylint: disable=consider-using-with
|
||||
blocking=False,
|
||||
),
|
||||
)
|
||||
try:
|
||||
server.process_request(client, ("127.0.0.1", 1))
|
||||
self.assertIn(b"503 Service Unavailable", peer.recv(1024))
|
||||
finally:
|
||||
server._request_slots.release()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -23,7 +23,6 @@ from bot_bottle.gateway.bootstrap import (
|
||||
_DaemonManager,
|
||||
_argv_for_daemon,
|
||||
_env_for_daemon,
|
||||
_pump,
|
||||
_selected_daemons,
|
||||
)
|
||||
from tests._bin import SLEEP
|
||||
@@ -566,26 +565,5 @@ class TestMainEndToEnd(unittest.TestCase):
|
||||
self.assertIn("no daemons selected", out)
|
||||
|
||||
|
||||
class _FailingStream:
|
||||
def __init__(self, *, closed: bool) -> None:
|
||||
self.closed = closed
|
||||
|
||||
def readline(self) -> bytes:
|
||||
raise ValueError("I/O operation on closed file")
|
||||
|
||||
|
||||
class TestOutputPump(unittest.TestCase):
|
||||
def test_closed_stream_race_is_normal_completion(self) -> None:
|
||||
with patch("bot_bottle.gateway.bootstrap._log") as log:
|
||||
_pump("egress", _FailingStream(closed=True)) # type: ignore[arg-type]
|
||||
log.assert_not_called()
|
||||
|
||||
def test_unexpected_io_failure_is_logged(self) -> None:
|
||||
with patch("bot_bottle.gateway.bootstrap._log") as log:
|
||||
_pump("egress", _FailingStream(closed=False)) # type: ignore[arg-type]
|
||||
log.assert_called_once()
|
||||
self.assertIn("output pump stopped", log.call_args.args[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -486,19 +486,6 @@ class TestMalformedStatusHeader(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(500, status)
|
||||
|
||||
def test_backend_timeout_returns_503(self):
|
||||
with mock.patch(
|
||||
"bot_bottle.gateway.git_gate.http_backend.subprocess.run",
|
||||
side_effect=subprocess.TimeoutExpired(["git", "http-backend"], 1),
|
||||
):
|
||||
req = urllib.request.Request(
|
||||
f"http://127.0.0.1:{self._port}/repo.git/info/refs",
|
||||
method="GET",
|
||||
)
|
||||
with self.assertRaises(urllib.error.HTTPError) as raised:
|
||||
urllib.request.urlopen(req, timeout=3)
|
||||
self.assertEqual(503, raised.exception.code)
|
||||
|
||||
|
||||
class TestContentLengthBounds(unittest.TestCase):
|
||||
"""PRD 0041: malformed or oversized Content-Length is rejected before
|
||||
|
||||
@@ -114,7 +114,6 @@ class TestVersionInputs(unittest.TestCase):
|
||||
'{"PYTHON_BASE_IMAGE": "python:pinned"}\n',
|
||||
)
|
||||
(root / "requirements.gateway.lock").write_text("mitmproxy==11.1.3\n")
|
||||
(root / "requirements.orchestrator.lock").write_text("fastapi==0.140.0\n")
|
||||
(root / "pyproject.toml").write_text("[project]\nname = 'bot-bottle'\n")
|
||||
|
||||
def test_pyproject_toml_change_bumps_version(self) -> None:
|
||||
@@ -166,21 +165,6 @@ class TestVersionInputs(unittest.TestCase):
|
||||
)
|
||||
self.assertNotEqual(before, after)
|
||||
|
||||
def test_orchestrator_lock_change_bumps_orchestrator_version(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
root = Path(d)
|
||||
self._fake_repo(root)
|
||||
before = ia.infra_artifact_version(
|
||||
"init", "orchestrator", repo_root=root,
|
||||
)
|
||||
(root / "requirements.orchestrator.lock").write_text(
|
||||
"fastapi==0.140.0 --hash=sha256:changed\n",
|
||||
)
|
||||
after = ia.infra_artifact_version(
|
||||
"init", "orchestrator", repo_root=root,
|
||||
)
|
||||
self.assertNotEqual(before, after)
|
||||
|
||||
def test_base_image_argument_change_bumps_both_role_versions(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
root = Path(d)
|
||||
@@ -284,18 +268,6 @@ class TestEnsureArtifact(_CacheMixin):
|
||||
ia.ensure_artifact_gz(version, role=_ROLE)
|
||||
self.assertEqual(first_calls, len(net.calls))
|
||||
|
||||
def test_download_uses_network_deadline(self) -> None:
|
||||
response = mock.MagicMock()
|
||||
response.__enter__.return_value = io.BytesIO(b"payload")
|
||||
with tempfile.TemporaryDirectory() as d, mock.patch.object(
|
||||
ia.urllib.request, "urlopen", return_value=response,
|
||||
) as urlopen:
|
||||
ia._download("https://registry/artifact", Path(d) / "artifact")
|
||||
self.assertEqual(
|
||||
ia.ARTIFACT_HTTP_TIMEOUT_SECONDS,
|
||||
urlopen.call_args.kwargs["timeout"],
|
||||
)
|
||||
|
||||
def test_checksum_mismatch_fails_closed(self) -> None:
|
||||
version = "beefbeefbeefbeef"
|
||||
gz = _gz(b"payload")
|
||||
|
||||
@@ -5,8 +5,6 @@ from __future__ import annotations
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend import EnumerationError
|
||||
from bot_bottle.backend.cleanup_control import CleanupError
|
||||
from bot_bottle.backend.macos_container import cleanup, enumerate as enum_mod
|
||||
from bot_bottle.backend.macos_container.bottle_cleanup_plan import (
|
||||
MacosContainerBottleCleanupPlan,
|
||||
@@ -32,10 +30,7 @@ class TestMacosContainerCleanup(unittest.TestCase):
|
||||
containers=("bot-bottle-a",),
|
||||
networks=("bot-bottle-net-a",),
|
||||
)
|
||||
completed = cleanup.subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout="", stderr="",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=completed) as run:
|
||||
with patch.object(cleanup.subprocess, "run") as run:
|
||||
cleanup.cleanup(plan)
|
||||
self.assertEqual(
|
||||
["container", "delete", "--force", "bot-bottle-a"],
|
||||
@@ -46,34 +41,6 @@ class TestMacosContainerCleanup(unittest.TestCase):
|
||||
run.call_args_list[1].args[0],
|
||||
)
|
||||
|
||||
def test_cleanup_attempts_all_resources_then_raises(self):
|
||||
plan = MacosContainerBottleCleanupPlan(
|
||||
containers=("bot-bottle-a",), networks=("bot-bottle-net-a",),
|
||||
)
|
||||
failed = cleanup.subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr="unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=failed) as run, \
|
||||
self.assertRaisesRegex(CleanupError, "unavailable"):
|
||||
cleanup.cleanup(plan)
|
||||
self.assertEqual(2, run.call_count)
|
||||
|
||||
def test_container_enumeration_failure_aborts(self):
|
||||
completed = cleanup.subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr="service unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=completed), \
|
||||
self.assertRaisesRegex(EnumerationError, "service unavailable"):
|
||||
cleanup._list_prefixed_containers()
|
||||
|
||||
def test_network_enumeration_failure_aborts(self):
|
||||
completed = cleanup.subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr="service unavailable",
|
||||
)
|
||||
with patch.object(cleanup.subprocess, "run", return_value=completed), \
|
||||
self.assertRaisesRegex(EnumerationError, "service unavailable"):
|
||||
cleanup._list_prefixed_networks()
|
||||
|
||||
|
||||
class TestMacosContainerEnumerate(unittest.TestCase):
|
||||
"""The backend launches bottles again (PRD 0070), so enumeration is real
|
||||
@@ -102,18 +69,10 @@ class TestMacosContainerEnumerate(unittest.TestCase):
|
||||
self.assertEqual(["dev-abc"], [a.slug for a in agents])
|
||||
|
||||
def test_raises_when_the_cli_fails(self):
|
||||
from bot_bottle.backend.macos_container.enumerate import EnumerationError
|
||||
with self.assertRaises(EnumerationError):
|
||||
self._enumerate("", returncode=1)
|
||||
|
||||
def test_raises_typed_error_when_cli_is_missing(self):
|
||||
with (
|
||||
patch.object(
|
||||
enum_mod.subprocess, "run", side_effect=FileNotFoundError,
|
||||
),
|
||||
self.assertRaisesRegex(EnumerationError, "CLI not found"),
|
||||
):
|
||||
enum_mod.enumerate_active()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -1,14 +1,17 @@
|
||||
"""Unit: agent-level git-gate.user overlay + provenance (PRD 0027, PRD 0047).
|
||||
"""Unit: agent-owned identity via `author` (PRD
|
||||
0082).
|
||||
|
||||
An agent file may declare `git-gate.user` (name/email). At
|
||||
`ManifestIndex.load_for_agent()` it overlays the referenced bottle's
|
||||
`git-gate.user` per-field, agent-wins-on-non-empty. `git-gate.repos` is
|
||||
rejected on agents. `Manifest.git_identity_summary()` reports the
|
||||
effective identity with per-field `(agent)`/`(bottle)` provenance.
|
||||
Identity is agent-only now: an agent file declares an `author` block
|
||||
(name + email, both required) and at `ManifestIndex.load_for_agent()`
|
||||
it populates the effective bottle's `git_user`. There is no per-field
|
||||
overlay against the bottle anymore — the bottle no longer carries a
|
||||
user identity. `git-gate` (user or repos) is rejected on an agent with
|
||||
a migration message. `Manifest.git_identity_summary()` reports the
|
||||
effective identity with no provenance annotation.
|
||||
|
||||
The `from_json_obj` path drives `Agent.from_dict` + the overlay in
|
||||
load_for_agent; a temp-dir case locks the md loader (the `_AGENT_KEYS`
|
||||
allow + the `git-gate` threading into `agent_dict`)."""
|
||||
The `from_json_obj` path drives `ManifestAgent.from_dict` + the
|
||||
composition in load_for_agent; a temp-dir case locks the md loader
|
||||
(the agent `author` frontmatter key threads into the parsed agent)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -31,97 +34,61 @@ def _error_message(callable_, *args, **kwargs) -> str: # type: ignore
|
||||
raise AssertionError("expected ManifestError was not raised")
|
||||
|
||||
|
||||
def _manifest(*, bottle_user=None, agent_git=None) -> Manifest: # type: ignore
|
||||
def _manifest(*, author=None, agent_git=None) -> Manifest: # type: ignore
|
||||
"""Build an index with one agent 'impl' and load it, returning a Manifest."""
|
||||
bottle: dict = {} # type: ignore
|
||||
if bottle_user is not None:
|
||||
bottle = {"git-gate": {"user": bottle_user}}
|
||||
agent: dict = {"skills": [], "prompt": "", "bottle": "dev"} # type: ignore
|
||||
if author is not None:
|
||||
agent["author"] = author
|
||||
if agent_git is not None:
|
||||
agent["git-gate"] = agent_git
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": bottle},
|
||||
"bottles": {"dev": {}},
|
||||
"agents": {"impl": agent},
|
||||
}).load_for_agent("impl")
|
||||
|
||||
|
||||
def _index(*, bottle_user: dict[str, object] | None = None, agent_git: dict[str, object] | None = None) -> ManifestIndex:
|
||||
def _index(*, author: dict[str, object] | None = None) -> ManifestIndex:
|
||||
"""Build an index with one agent 'impl' without loading it."""
|
||||
bottle: dict = {} # type: ignore
|
||||
if bottle_user is not None:
|
||||
bottle = {"git-gate": {"user": bottle_user}}
|
||||
agent: dict = {"skills": [], "prompt": "", "bottle": "dev"} # type: ignore
|
||||
if agent_git is not None:
|
||||
agent["git-gate"] = agent_git
|
||||
if author is not None:
|
||||
agent["author"] = author
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": bottle},
|
||||
"bottles": {"dev": {}},
|
||||
"agents": {"impl": agent},
|
||||
})
|
||||
|
||||
|
||||
class TestAgentGitUserOverlay(unittest.TestCase):
|
||||
def test_agent_supplies_both_fields(self):
|
||||
m = _manifest(agent_git={"user": {"name": "a", "email": "a@b"}})
|
||||
class TestAgentAuthorPopulatesBottle(unittest.TestCase):
|
||||
def test_agent_author_supplies_both_fields(self):
|
||||
m = _manifest(author={"name": "a", "email": "a@b"})
|
||||
u = m.bottle.git_user
|
||||
self.assertEqual("a", u.name)
|
||||
self.assertEqual("a@b", u.email)
|
||||
|
||||
def test_agent_name_only_email_falls_through_to_bottle(self):
|
||||
m = _manifest(
|
||||
bottle_user={"name": "B", "email": "b@c"},
|
||||
agent_git={"user": {"name": "a"}},
|
||||
)
|
||||
u = m.bottle.git_user
|
||||
self.assertEqual("a", u.name) # agent wins
|
||||
self.assertEqual("b@c", u.email) # bottle falls through
|
||||
|
||||
def test_agent_email_only_name_falls_through_to_bottle(self):
|
||||
m = _manifest(
|
||||
bottle_user={"name": "B", "email": "b@c"},
|
||||
agent_git={"user": {"email": "a@b"}},
|
||||
)
|
||||
u = m.bottle.git_user
|
||||
self.assertEqual("B", u.name)
|
||||
self.assertEqual("a@b", u.email)
|
||||
|
||||
def test_agent_identity_with_bottle_declaring_none(self):
|
||||
idx = _index(agent_git={"user": {"name": "a", "email": "a@b"}})
|
||||
# Raw bottle has no git_user; loaded manifest has merged git_user from agent
|
||||
def test_bottle_has_no_identity_until_agent_composed(self):
|
||||
idx = _index(author={"name": "a", "email": "a@b"})
|
||||
# Raw bottle has no git_user; loaded manifest has it from the agent.
|
||||
self.assertTrue(idx.bottles["dev"].git_user.is_empty())
|
||||
m = idx.load_for_agent("impl")
|
||||
self.assertFalse(m.bottle.git_user.is_empty())
|
||||
|
||||
def test_bottle_only_identity_preserved_when_agent_silent(self):
|
||||
m = _manifest(bottle_user={"name": "B", "email": "b@c"})
|
||||
u = m.bottle.git_user
|
||||
self.assertEqual("B", u.name)
|
||||
self.assertEqual("b@c", u.email)
|
||||
|
||||
def test_no_overlay_uses_bottle_instance_directly(self):
|
||||
idx = _index(bottle_user={"name": "B"})
|
||||
def test_agent_silent_leaves_bottle_identity_empty(self):
|
||||
idx = _index()
|
||||
m = idx.load_for_agent("impl")
|
||||
# Agent has no git_user — bottle instance should be the same object
|
||||
# No author -> git_user stays empty; the bottle instance is reused
|
||||
# directly (no replace needed).
|
||||
self.assertTrue(m.bottle.git_user.is_empty())
|
||||
self.assertIs(idx.bottles["dev"], m.bottle)
|
||||
|
||||
def test_noop_overlay_uses_bottle_instance_directly(self):
|
||||
idx = _index(
|
||||
bottle_user={"name": "B", "email": "b@c"},
|
||||
agent_git={"user": {"name": "B", "email": "b@c"}},
|
||||
)
|
||||
m = idx.load_for_agent("impl")
|
||||
# Agent git_user == bottle git_user — no replace needed
|
||||
self.assertEqual(idx.bottles["dev"].git_user, m.bottle.git_user)
|
||||
|
||||
def test_other_bottle_fields_untouched_by_overlay(self):
|
||||
def test_other_bottle_fields_untouched_by_identity(self):
|
||||
idx = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": {
|
||||
"env": {"FOO": "bar"},
|
||||
"supervise": True,
|
||||
"git-gate": {"user": {"name": "B"}},
|
||||
}},
|
||||
"agents": {"impl": {
|
||||
"bottle": "dev", "skills": [], "prompt": "",
|
||||
"git-gate": {"user": {"name": "a"}},
|
||||
"author": {"name": "a", "email": "a@b"},
|
||||
}},
|
||||
})
|
||||
b = idx.load_for_agent("impl").bottle
|
||||
@@ -130,93 +97,77 @@ class TestAgentGitUserOverlay(unittest.TestCase):
|
||||
self.assertTrue(b.supervise)
|
||||
|
||||
|
||||
class TestAgentGitUserRejections(unittest.TestCase):
|
||||
def test_agent_repos_dies_bottle_only(self):
|
||||
class TestAgentGitGateRejections(unittest.TestCase):
|
||||
"""`git-gate` is no longer accepted on an agent (user or repos):
|
||||
identity moved to `author`, repos stays bottle-only."""
|
||||
|
||||
def test_agent_git_gate_user_dies(self):
|
||||
msg = _error_message(_manifest, agent_git={"user": {"name": "a", "email": "a@b"}})
|
||||
self.assertIn("no longer", msg)
|
||||
self.assertIn("author", msg)
|
||||
|
||||
def test_agent_git_gate_repos_dies(self):
|
||||
msg = _error_message(_manifest, agent_git={
|
||||
"repos": {"r": {"url": "ssh://git@x/y.git", "key": {"provider": "static", "path": "/dev/null"}}},
|
||||
})
|
||||
self.assertIn("git-gate.repos", msg)
|
||||
self.assertIn("bottle-only", msg)
|
||||
|
||||
def test_agent_unknown_git_subkey_dies(self):
|
||||
msg = _error_message(_manifest, agent_git={"nope": {}})
|
||||
self.assertIn("not allowed at the agent level", msg)
|
||||
|
||||
def test_agent_git_user_both_empty_dies(self):
|
||||
msg = _error_message(_manifest, agent_git={"user": {"name": "", "email": ""}})
|
||||
self.assertIn("neither name nor email", msg)
|
||||
self.assertIn("no longer", msg)
|
||||
self.assertIn("author", msg)
|
||||
|
||||
|
||||
class TestGitIdentitySummary(unittest.TestCase):
|
||||
def test_both_from_agent(self):
|
||||
m = _manifest(agent_git={"user": {"name": "a", "email": "a@b"}})
|
||||
self.assertEqual(
|
||||
"name=a (agent), email=a@b (agent)",
|
||||
m.git_identity_summary(),
|
||||
)
|
||||
"""Summary reports the effective identity (from the agent's author)
|
||||
with no per-field provenance annotation."""
|
||||
|
||||
def test_mixed_provenance(self):
|
||||
m = _manifest(
|
||||
bottle_user={"name": "B", "email": "b@c"},
|
||||
agent_git={"user": {"name": "a"}},
|
||||
)
|
||||
self.assertEqual(
|
||||
"name=a (agent), email=b@c (bottle)",
|
||||
m.git_identity_summary(),
|
||||
)
|
||||
def test_summary_from_author(self):
|
||||
m = _manifest(author={"name": "a", "email": "a@b"})
|
||||
self.assertEqual("name=a, email=a@b", m.git_identity_summary())
|
||||
|
||||
def test_bottle_only(self):
|
||||
m = _manifest(bottle_user={"name": "B", "email": "b@c"})
|
||||
self.assertEqual(
|
||||
"name=B (bottle), email=b@c (bottle)",
|
||||
m.git_identity_summary(),
|
||||
)
|
||||
|
||||
def test_none_when_unset_anywhere(self):
|
||||
def test_none_when_no_author(self):
|
||||
m = _manifest()
|
||||
self.assertIsNone(m.git_identity_summary())
|
||||
|
||||
|
||||
_BOTTLE_DEV = """
|
||||
---
|
||||
git-gate:
|
||||
user:
|
||||
name: bottle-name
|
||||
email: bottle@example.com
|
||||
egress:
|
||||
routes:
|
||||
- host: example.com
|
||||
---
|
||||
|
||||
dev bottle.
|
||||
"""
|
||||
|
||||
_AGENT_WITH_GIT = """
|
||||
_AGENT_WITH_AUTHOR = """
|
||||
---
|
||||
bottle: dev
|
||||
git-gate:
|
||||
user:
|
||||
name: agent-name
|
||||
author:
|
||||
name: agent-name
|
||||
email: agent@example.com
|
||||
---
|
||||
|
||||
impl agent.
|
||||
"""
|
||||
|
||||
_AGENT_WITH_REPOS = """
|
||||
_AGENT_WITH_GIT_GATE = """
|
||||
---
|
||||
bottle: dev
|
||||
git-gate:
|
||||
repos:
|
||||
r:
|
||||
url: ssh://git@x/y.git
|
||||
identity: /dev/null
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
|
||||
bad agent.
|
||||
"""
|
||||
|
||||
|
||||
class TestAgentGitUserMdLoader(unittest.TestCase):
|
||||
"""Locks the md path: `git-gate` is an accepted agent key and threads
|
||||
into the parsed Agent (not rejected as an unknown frontmatter key),
|
||||
and agent `git-gate.repos` dies through the same loader."""
|
||||
class TestAgentAuthorMdLoader(unittest.TestCase):
|
||||
"""Locks the md path: `author` is an accepted agent frontmatter key
|
||||
and threads into the parsed agent, populating identity; a stale
|
||||
agent `git-gate` block dies through the same loader."""
|
||||
|
||||
def setUp(self) -> None:
|
||||
self.home = Path(tempfile.mkdtemp(prefix="cb-home-"))
|
||||
@@ -235,31 +186,30 @@ class TestAgentGitUserMdLoader(unittest.TestCase):
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(textwrap.dedent(text).lstrip("\n"))
|
||||
|
||||
def test_md_agent_git_user_overlays_bottle(self):
|
||||
def test_md_agent_author_populates_identity(self):
|
||||
self._write("bottles/dev.md", _BOTTLE_DEV)
|
||||
self._write("agents/impl.md", _AGENT_WITH_GIT)
|
||||
self._write("agents/impl.md", _AGENT_WITH_AUTHOR)
|
||||
m = ManifestIndex.resolve(str(self.home)).load_for_agent("impl")
|
||||
u = m.bottle.git_user
|
||||
self.assertEqual("agent-name", u.name)
|
||||
self.assertEqual("bottle@example.com", u.email)
|
||||
self.assertEqual("agent@example.com", u.email)
|
||||
self.assertEqual(
|
||||
"name=agent-name (agent), email=bottle@example.com (bottle)",
|
||||
"name=agent-name, email=agent@example.com",
|
||||
m.git_identity_summary(),
|
||||
)
|
||||
|
||||
def test_md_agent_repos_fails_at_preflight(self):
|
||||
"""git-gate.repos on an agent is an error; resolve() still succeeds
|
||||
so other agents remain accessible, but load_for_agent raises."""
|
||||
def test_md_agent_git_gate_fails_at_preflight(self):
|
||||
"""A stale agent `git-gate` block is an error; resolve() still
|
||||
succeeds so other agents remain accessible, but load_for_agent
|
||||
raises. The lazy loader's frontmatter-key validator rejects the
|
||||
unknown `git-gate` key first."""
|
||||
self._write("bottles/dev.md", _BOTTLE_DEV)
|
||||
self._write("agents/impl.md", _AGENT_WITH_REPOS)
|
||||
from bot_bottle.manifest import ManifestError
|
||||
self._write("agents/impl.md", _AGENT_WITH_GIT_GATE)
|
||||
names = ManifestIndex.resolve(str(self.home))
|
||||
self.assertIn("impl", names.all_agent_names)
|
||||
with self.assertRaises(ManifestError) as ctx:
|
||||
names.load_for_agent("impl")
|
||||
msg = str(ctx.exception)
|
||||
self.assertIn("git-gate.repos", msg)
|
||||
self.assertIn("bottle-only", msg)
|
||||
self.assertIn("git-gate", str(ctx.exception))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -130,8 +130,10 @@ class TestExtendsEnvMerge(unittest.TestCase):
|
||||
|
||||
|
||||
class TestExtendsGitMerge(unittest.TestCase):
|
||||
"""git-gate.user overlays by field; git-gate.repos merges by name,
|
||||
with same-name child entries merging field-by-field (child wins)."""
|
||||
"""git-gate.repos merges by name, with same-name child entries
|
||||
merging field-by-field (child wins). Bottles no longer carry a user
|
||||
identity (PRD 0082), so only repos
|
||||
merging is meaningful across extends chains."""
|
||||
|
||||
_GIT_ENTRY_A = {"url": "ssh://git@host-a/a.git", "key": {"provider": "static", "path": "/dev/null"}}
|
||||
_GIT_ENTRY_B = {"url": "ssh://git@host-b/b.git", "key": {"provider": "static", "path": "/dev/null"}}
|
||||
@@ -254,13 +256,16 @@ class TestExtendsGitMerge(unittest.TestCase):
|
||||
repo_entry = next(e for e in child.git if e.Name == "repo")
|
||||
self.assertEqual("gitea", repo_entry.Key.provider)
|
||||
|
||||
def test_child_git_user_inherits_parent_repos(self):
|
||||
def test_child_inherits_parent_repos_no_user_identity(self):
|
||||
# Child omits git-gate entirely -> inherits the parent's repos.
|
||||
# Bottles carry no user identity anymore, so git_user stays empty
|
||||
# across the extends chain.
|
||||
m = _build(
|
||||
base={"git-gate": {"repos": {"a": self._GIT_ENTRY_A}}},
|
||||
child={"extends": "base", "git-gate": {"user": {"name": "Child"}}},
|
||||
child={"extends": "base"},
|
||||
)
|
||||
self.assertEqual(["a"], [e.Name for e in m.bottles["child"].git])
|
||||
self.assertEqual("Child", m.bottles["child"].git_user.name)
|
||||
self.assertTrue(m.bottles["child"].git_user.is_empty())
|
||||
|
||||
|
||||
class TestExtendsEgressMerge(unittest.TestCase):
|
||||
@@ -332,48 +337,29 @@ class TestExtendsEgressMerge(unittest.TestCase):
|
||||
self.assertIn("A.EXAMPLE.COM", msg)
|
||||
|
||||
|
||||
class TestExtendsGitUserOverlay(unittest.TestCase):
|
||||
"""git-gate.user: per-field overlay. Each non-empty field on child
|
||||
wins; empties fall through to parent."""
|
||||
class TestExtendsNoBottleUserIdentity(unittest.TestCase):
|
||||
"""Bottles no longer carry a user identity (PRD
|
||||
0082): identity moved to the agent's
|
||||
`author` block. `git-gate.user` on a bottle is rejected outright, and
|
||||
`git_user` is always empty across an extends chain."""
|
||||
|
||||
def test_parent_full_child_omits(self):
|
||||
m = _build(
|
||||
def test_bottle_git_gate_user_dies(self):
|
||||
# A stale `git-gate.user` on a bottle fails with the migration die
|
||||
# even inside an extends chain.
|
||||
msg = _error_message(
|
||||
_build,
|
||||
base={"git-gate": {"user": {"name": "Parent", "email": "p@x"}}},
|
||||
child={"extends": "base"},
|
||||
)
|
||||
u = m.bottles["child"].git_user
|
||||
self.assertEqual("Parent", u.name)
|
||||
self.assertEqual("p@x", u.email)
|
||||
self.assertIn("git-gate.user is no longer supported", msg)
|
||||
|
||||
def test_child_overrides_both(self):
|
||||
def test_git_user_empty_across_chain(self):
|
||||
m = _build(
|
||||
base={"git-gate": {"user": {"name": "Parent", "email": "p@x"}}},
|
||||
child={
|
||||
"extends": "base",
|
||||
"git-gate": {"user": {"name": "Child", "email": "c@x"}},
|
||||
},
|
||||
base={"git-gate": {"repos": {}}},
|
||||
child={"extends": "base"},
|
||||
)
|
||||
u = m.bottles["child"].git_user
|
||||
self.assertEqual("Child", u.name)
|
||||
self.assertEqual("c@x", u.email)
|
||||
|
||||
def test_child_adds_email_inherits_name(self):
|
||||
m = _build(
|
||||
base={"git-gate": {"user": {"name": "Parent"}}},
|
||||
child={"extends": "base", "git-gate": {"user": {"email": "c@x"}}},
|
||||
)
|
||||
u = m.bottles["child"].git_user
|
||||
self.assertEqual("Parent", u.name)
|
||||
self.assertEqual("c@x", u.email)
|
||||
|
||||
def test_child_overrides_only_email(self):
|
||||
m = _build(
|
||||
base={"git-gate": {"user": {"name": "Parent", "email": "p@x"}}},
|
||||
child={"extends": "base", "git-gate": {"user": {"email": "c@x"}}},
|
||||
)
|
||||
u = m.bottles["child"].git_user
|
||||
self.assertEqual("Parent", u.name)
|
||||
self.assertEqual("c@x", u.email)
|
||||
self.assertTrue(m.bottles["base"].git_user.is_empty())
|
||||
self.assertTrue(m.bottles["child"].git_user.is_empty())
|
||||
|
||||
|
||||
class TestExtendsChain(unittest.TestCase):
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
"""Unit: Bottle git-gate.user manifest parsing + validation (issue #86, PRD 0047)."""
|
||||
"""Unit: agent `author` identity -> bottle.git_user (PRD
|
||||
0082).
|
||||
|
||||
Identity moved off `git-gate.user` (bottle) onto the trusted agent's
|
||||
`author` block. At `load_for_agent` the agent's author populates the
|
||||
effective bottle's `git_user` (a `ManifestGitUser`). This locks the
|
||||
`author` validation/rejection paths and the bottle `git-gate.user`
|
||||
migration die."""
|
||||
|
||||
import unittest
|
||||
|
||||
@@ -14,89 +21,92 @@ def _error_message(callable_, *args, **kwargs) -> str: # type: ignore
|
||||
raise AssertionError("expected ManifestError was not raised")
|
||||
|
||||
|
||||
def _manifest(git_user): # type: ignore
|
||||
return {
|
||||
"bottles": {"dev": {"git-gate": {"user": git_user}}},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
}
|
||||
def _manifest(author): # type: ignore
|
||||
"""Build an index with one agent 'demo' carrying the given `author`
|
||||
block, then load it, returning the composed Manifest."""
|
||||
agent: dict = {"skills": [], "prompt": "", "bottle": "dev"} # type: ignore
|
||||
if author is not None:
|
||||
agent["author"] = author
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": {}},
|
||||
"agents": {"demo": agent},
|
||||
}).load_for_agent("demo")
|
||||
|
||||
|
||||
class TestGitUserParsing(unittest.TestCase):
|
||||
class TestAuthorIdentity(unittest.TestCase):
|
||||
"""The agent's `author` block populates bottle.git_user."""
|
||||
|
||||
def test_parses_both_fields(self):
|
||||
m = ManifestIndex.from_json_obj(_manifest({
|
||||
m = _manifest({
|
||||
"name": "Eric Bauerfeld",
|
||||
"email": "eric+claude@dideric.is",
|
||||
}))
|
||||
u = m.bottles["dev"].git_user
|
||||
})
|
||||
u = m.bottle.git_user
|
||||
self.assertEqual("Eric Bauerfeld", u.name)
|
||||
self.assertEqual("eric+claude@dideric.is", u.email)
|
||||
self.assertFalse(u.is_empty())
|
||||
|
||||
def test_name_only(self):
|
||||
m = ManifestIndex.from_json_obj(_manifest({"name": "Bot"}))
|
||||
u = m.bottles["dev"].git_user
|
||||
self.assertEqual("Bot", u.name)
|
||||
self.assertEqual("", u.email)
|
||||
|
||||
def test_email_only(self):
|
||||
m = ManifestIndex.from_json_obj(_manifest({"email": "bot@example.com"}))
|
||||
u = m.bottles["dev"].git_user
|
||||
self.assertEqual("", u.name)
|
||||
self.assertEqual("bot@example.com", u.email)
|
||||
|
||||
def test_omitted_defaults_to_empty(self):
|
||||
# No git.user block at all → empty GitUser, is_empty True →
|
||||
def test_omitted_author_defaults_to_empty(self):
|
||||
# No author block at all -> empty git_user, is_empty True ->
|
||||
# provisioner skips the `git config` step entirely.
|
||||
m = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": {}},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
})
|
||||
u = m.bottles["dev"].git_user
|
||||
self.assertTrue(u.is_empty())
|
||||
m = _manifest(None)
|
||||
self.assertTrue(m.bottle.git_user.is_empty())
|
||||
|
||||
def test_missing_name_dies(self):
|
||||
# `author` is present but name is absent -> both fields required.
|
||||
msg = _error_message(_manifest, {"email": "bot@example.com"})
|
||||
self.assertIn("author.name must be a non-empty string", msg)
|
||||
|
||||
def test_missing_email_dies(self):
|
||||
msg = _error_message(_manifest, {"name": "Bot"})
|
||||
self.assertIn("author.email must be a non-empty string", msg)
|
||||
|
||||
def test_both_empty_strings_dies(self):
|
||||
# An explicit `git.user: {name: "", email: ""}` is a typo
|
||||
# / half-finished edit; fail loudly rather than silently
|
||||
# no-op (the operator clearly meant to configure something).
|
||||
msg = _error_message(
|
||||
ManifestIndex.from_json_obj, _manifest({"name": "", "email": ""}),
|
||||
)
|
||||
self.assertIn("neither name nor email", msg)
|
||||
# An explicit `author: {name: "", email: ""}` is a typo /
|
||||
# half-finished edit; fail loudly rather than silently no-op.
|
||||
msg = _error_message(_manifest, {"name": "", "email": ""})
|
||||
self.assertIn("author.name must be a non-empty string", msg)
|
||||
|
||||
def test_unknown_key_dies(self):
|
||||
msg = _error_message(
|
||||
ManifestIndex.from_json_obj,
|
||||
_manifest({"name": "Bot", "username": "bot"}),
|
||||
_manifest,
|
||||
{"name": "Bot", "email": "b@x", "username": "bot"},
|
||||
)
|
||||
self.assertIn("unknown key", msg)
|
||||
self.assertIn("username", msg)
|
||||
|
||||
def test_non_string_name_dies(self):
|
||||
msg = _error_message(
|
||||
ManifestIndex.from_json_obj, _manifest({"name": 42}),
|
||||
)
|
||||
self.assertIn("git-gate.user.name must be a string", msg)
|
||||
msg = _error_message(_manifest, {"name": 42, "email": "b@x"})
|
||||
self.assertIn("author.name must be a non-empty string", msg)
|
||||
|
||||
def test_non_string_email_dies(self):
|
||||
msg = _error_message(
|
||||
ManifestIndex.from_json_obj, _manifest({"email": ["x@y.z"]}),
|
||||
)
|
||||
self.assertIn("git-gate.user.email must be a string", msg)
|
||||
msg = _error_message(_manifest, {"name": "Bot", "email": ["x@y.z"]})
|
||||
self.assertIn("author.email must be a non-empty string", msg)
|
||||
|
||||
def test_legacy_top_level_git_user_dies(self):
|
||||
def test_email_with_whitespace_dies(self):
|
||||
msg = _error_message(_manifest, {"name": "Bot", "email": "a @b"})
|
||||
self.assertIn("author.email must not contain whitespace", msg)
|
||||
|
||||
|
||||
class TestBottleGitGateUserMigration(unittest.TestCase):
|
||||
"""`git-gate.user` on a bottle is no longer supported: it raises a
|
||||
ManifestError pointing at the agent's `author` block."""
|
||||
|
||||
def test_bottle_git_gate_user_dies(self):
|
||||
msg = _error_message(
|
||||
ManifestIndex.from_json_obj,
|
||||
{
|
||||
"bottles": {"dev": {"git_user": {"name": "Bot"}}},
|
||||
"bottles": {"dev": {"git-gate": {"user": {"name": "Bot"}}}},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
},
|
||||
)
|
||||
self.assertIn("git_user", msg)
|
||||
self.assertIn("git-gate.user", msg)
|
||||
self.assertIn("git-gate.user is no longer supported", msg)
|
||||
self.assertIn("author", msg)
|
||||
|
||||
|
||||
class TestGitUserDirect(unittest.TestCase):
|
||||
"""Direct GitUser dataclass exercises (no manifest wrapper)."""
|
||||
"""Direct GitUser dataclass exercises (no manifest wrapper). The
|
||||
dataclass is still the runtime carrier on ManifestBottle.git_user."""
|
||||
|
||||
def test_is_empty_default(self):
|
||||
self.assertTrue(ManifestGitUser().is_empty())
|
||||
|
||||
@@ -71,11 +71,14 @@ class _LazyCase(unittest.TestCase):
|
||||
|
||||
|
||||
class TestAllAgentNamesLazy(_LazyCase):
|
||||
def test_merges_home_and_cwd_agents(self) -> None:
|
||||
def test_cwd_agents_ignored_home_only(self) -> None:
|
||||
# Agents are home-only (PRD 0082):
|
||||
# a cwd agents/ dir is warned-and-ignored, so only the home agent
|
||||
# appears in all_agent_names.
|
||||
_write(self.home_cb / "bottles" / "dev.md", _BOTTLE_DEV)
|
||||
_write(self.home_cb / "agents" / "alpha.md", _AGENT)
|
||||
_write(self.cwd_cb / "agents" / "beta.md", _AGENT)
|
||||
self.assertEqual(["alpha", "beta"], self.resolve().all_agent_names)
|
||||
self.assertEqual(["alpha"], self.resolve().all_agent_names)
|
||||
|
||||
|
||||
class TestLoadForAgentLazy(_LazyCase):
|
||||
@@ -96,11 +99,13 @@ class TestRequireAgentLazy(_LazyCase):
|
||||
_write(self.home_cb / "agents" / "alpha.md", _AGENT)
|
||||
self.resolve().require_agent("alpha") # no raise
|
||||
|
||||
def test_existing_cwd_agent_ok(self) -> None:
|
||||
# File only under cwd -> require_agent's cwd_path branch.
|
||||
def test_cwd_only_agent_not_selectable(self) -> None:
|
||||
# Agents are home-only (PRD 0082):
|
||||
# a cwd-only agent file is never selectable, so require_agent raises.
|
||||
_write(self.home_cb / "agents" / "alpha.md", _AGENT)
|
||||
_write(self.cwd_cb / "agents" / "beta.md", _AGENT)
|
||||
self.resolve().require_agent("beta") # no raise
|
||||
with self.assertRaises(ManifestError):
|
||||
self.resolve().require_agent("beta")
|
||||
|
||||
def test_unknown_agent_raises(self) -> None:
|
||||
_write(self.home_cb / "agents" / "alpha.md", _AGENT)
|
||||
|
||||
@@ -110,14 +110,16 @@ class TestAgentFileParses(_ResolveCase):
|
||||
self.assertFalse(a.prompt.endswith("\n"))
|
||||
|
||||
|
||||
class TestCwdAgentOverridesHome(_ResolveCase):
|
||||
"""SC #3: a cwd agent file with the same name as a home agent
|
||||
wins. The home bottle stays intact."""
|
||||
class TestCwdAgentIgnoredHomeWins(_ResolveCase):
|
||||
"""SC #3 (revised, PRD 0082): agents
|
||||
are home-only. A cwd agent file with the same name as a home agent no
|
||||
longer wins — it is warned-and-ignored, so the HOME agent's prompt is
|
||||
used and the home bottle stays intact."""
|
||||
|
||||
def test_cwd_wins(self):
|
||||
def test_home_wins_cwd_ignored(self):
|
||||
_write(self.home_cb / "bottles" / "dev.md", _BOTTLE_DEV)
|
||||
_write(self.home_cb / "agents" / "implementer.md", _AGENT_IMPL)
|
||||
# Cwd overrides with a different prompt
|
||||
# Cwd agent with a different prompt is ignored entirely.
|
||||
_write(
|
||||
self.cwd_cb / "agents" / "implementer.md",
|
||||
"""
|
||||
@@ -129,14 +131,19 @@ class TestCwdAgentOverridesHome(_ResolveCase):
|
||||
""",
|
||||
)
|
||||
m = self.resolve().load_for_agent("implementer")
|
||||
self.assertIn("CWD-OVERRIDE-PROMPT", m.agent.prompt)
|
||||
# Home agent's body is used; the cwd override never applies.
|
||||
self.assertIn("feature implementation agent", m.agent.prompt)
|
||||
self.assertNotIn("CWD-OVERRIDE-PROMPT", m.agent.prompt)
|
||||
# Home bottle still present with its two egress routes
|
||||
self.assertEqual(2, len(m.bottle.egress.routes))
|
||||
|
||||
|
||||
class TestCwdBottlesIgnored(_ResolveCase):
|
||||
"""SC #4: a bottles/ dir under $CWD is ignored (with a warn).
|
||||
The home bottle still wins; cwd contributes only agents."""
|
||||
The home bottle still wins. Under
|
||||
PRD 0082 a cwd agents/ dir is also
|
||||
ignored, so $CWD contributes nothing — the filesystem layout is the
|
||||
trust boundary."""
|
||||
|
||||
def test_ignored(self):
|
||||
_write(self.home_cb / "bottles" / "dev.md", _BOTTLE_DEV)
|
||||
|
||||
@@ -212,13 +212,21 @@ class TestAgentValidation(unittest.TestCase):
|
||||
with self.assertRaises(ManifestError):
|
||||
ManifestAgent.from_dict("a", {"prompt": 5}, set())
|
||||
|
||||
def test_git_gate_repos_rejected_at_agent_level(self) -> None:
|
||||
def test_git_gate_rejected_at_agent_level(self) -> None:
|
||||
# `git-gate` (user or repos) is no longer accepted on an agent;
|
||||
# identity moved to `author`, repos stays bottle-only.
|
||||
with self.assertRaises(ManifestError):
|
||||
ManifestAgent.from_dict("a", {"git-gate": {"repos": {}}}, set())
|
||||
with self.assertRaises(ManifestError):
|
||||
ManifestAgent.from_dict(
|
||||
"a", {"git-gate": {"user": {"name": "x"}}}, set()
|
||||
)
|
||||
|
||||
def test_git_gate_empty_is_allowed(self) -> None:
|
||||
agent = ManifestAgent.from_dict("a", {"git-gate": {}}, set())
|
||||
self.assertTrue(agent.git_user.is_empty())
|
||||
def test_bottle_empty_git_gate_is_allowed(self) -> None:
|
||||
# An empty `git-gate: {}` on a bottle is still allowed (only the
|
||||
# optional `repos` subkey exists now); it contributes no git repos.
|
||||
bottle = ManifestBottle.from_dict("b", {"git-gate": {}})
|
||||
self.assertEqual((), bottle.git)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -228,9 +236,12 @@ class TestAgentValidation(unittest.TestCase):
|
||||
|
||||
class TestEagerIndexLookups(unittest.TestCase):
|
||||
def _idx(self) -> ManifestIndex:
|
||||
# Identity lives on the agent's `author` block now; at composition
|
||||
# it populates the effective bottle's git_user.
|
||||
return _idx({
|
||||
"bottles": {"b": {"git-gate": {"user": {"name": "Bot", "email": "b@x"}}}},
|
||||
"agents": {"a": {"bottle": "b"}},
|
||||
"bottles": {"b": {}},
|
||||
"agents": {"a": {"bottle": "b",
|
||||
"author": {"name": "Bot", "email": "b@x"}}},
|
||||
})
|
||||
|
||||
def test_unknown_bottle_section_is_empty(self) -> None:
|
||||
|
||||
@@ -3,8 +3,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
import urllib.error
|
||||
from unittest.mock import MagicMock, patch
|
||||
@@ -22,30 +20,6 @@ from bot_bottle.orchestrator.client import (
|
||||
_URLOPEN = "bot_bottle.orchestrator.client.urllib.request.urlopen"
|
||||
|
||||
|
||||
class TestImportBoundary(unittest.TestCase):
|
||||
def test_host_client_does_not_import_server_dependencies(self) -> None:
|
||||
script = """
|
||||
import importlib.abc
|
||||
import sys
|
||||
|
||||
class BlockServerDependencies(importlib.abc.MetaPathFinder):
|
||||
def find_spec(self, fullname, path=None, target=None):
|
||||
if fullname.split(".", 1)[0] in {"fastapi", "uvicorn"}:
|
||||
raise ImportError(f"host import reached {fullname}")
|
||||
return None
|
||||
|
||||
sys.meta_path.insert(0, BlockServerDependencies())
|
||||
import bot_bottle.orchestrator.client
|
||||
"""
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", script],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
self.assertEqual(0, result.returncode, result.stderr)
|
||||
|
||||
|
||||
class TestHostAuthToken(unittest.TestCase):
|
||||
def test_mints_a_cli_token_from_the_host_key(self) -> None:
|
||||
# The CLI mints its `cli` token from the control-plane trust domain's
|
||||
|
||||
@@ -224,17 +224,6 @@ class TestAgentSecrets(unittest.TestCase):
|
||||
reopened = RegistryStore(self.db)
|
||||
self.assertEqual({"K": "v"}, reopened.get_agent_secrets("bottle-1"))
|
||||
|
||||
def test_v6_migration_clears_legacy_secret_rows(self) -> None:
|
||||
self.store.store_agent_secrets("bottle-1", {"K": "legacy"})
|
||||
with closing(sqlite3.connect(self.db)) as conn:
|
||||
conn.execute(
|
||||
"UPDATE schema_versions SET version = 5 "
|
||||
"WHERE module = 'orchestrator_registry'"
|
||||
)
|
||||
conn.commit()
|
||||
self.store.migrate()
|
||||
self.assertEqual({}, self.store.get_agent_secrets("bottle-1"))
|
||||
|
||||
|
||||
class TestReapAbsent(unittest.TestCase):
|
||||
"""`reap_absent` — the self-heal for rows whose bottle is gone.
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import unittest
|
||||
|
||||
from bot_bottle.orchestrator.store.secret_store import (
|
||||
@@ -69,33 +68,15 @@ class TestDecryptErrors(unittest.TestCase):
|
||||
def test_wrong_key_raises_value_error(self) -> None:
|
||||
ct = encrypt_value(self.secret, "secret-token")
|
||||
other_key = new_env_var_secret()
|
||||
with self.assertRaisesRegex(ValueError, "authentication failed"):
|
||||
decrypt_value(other_key, ct)
|
||||
|
||||
def test_tampered_ciphertext_raises_value_error(self) -> None:
|
||||
raw = bytearray(base64.urlsafe_b64decode(
|
||||
encrypt_value(self.secret, "secret-token") + "=="
|
||||
))
|
||||
raw[22] ^= 1
|
||||
tampered = base64.urlsafe_b64encode(raw).rstrip(b"=").decode()
|
||||
with self.assertRaisesRegex(ValueError, "authentication failed"):
|
||||
decrypt_value(self.secret, tampered)
|
||||
|
||||
def test_rejects_legacy_ciphertext(self) -> None:
|
||||
legacy = base64.urlsafe_b64encode(
|
||||
b"0123456789abcdeflegacy-token",
|
||||
).rstrip(b"=").decode()
|
||||
with self.assertRaisesRegex(ValueError, "unsupported ciphertext format"):
|
||||
decrypt_value(self.secret, legacy)
|
||||
|
||||
def test_rejects_authenticated_blob_with_changed_version(self) -> None:
|
||||
raw = bytearray(base64.urlsafe_b64decode(
|
||||
encrypt_value(self.secret, "secret-token") + "=="
|
||||
))
|
||||
raw[0] ^= 1
|
||||
downgraded = base64.urlsafe_b64encode(raw).rstrip(b"=").decode()
|
||||
with self.assertRaisesRegex(ValueError, "unsupported ciphertext format"):
|
||||
decrypt_value(self.secret, downgraded)
|
||||
# Wrong key produces garbage bytes; decrypt_value raises ValueError
|
||||
# when the result is non-UTF-8 (which is very likely for 12-char data).
|
||||
# We allow it to succeed only if garbage happens to be valid UTF-8, but
|
||||
# the plaintext must not match.
|
||||
try:
|
||||
result = decrypt_value(other_key, ct)
|
||||
self.assertNotEqual("secret-token", result)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
def test_truncated_blob_raises_value_error(self) -> None:
|
||||
with self.assertRaises(ValueError):
|
||||
|
||||
@@ -6,9 +6,7 @@ server tests), plus one real-socket round-trip to prove the handler wiring.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import http.client
|
||||
import io
|
||||
import json
|
||||
import secrets
|
||||
@@ -18,17 +16,13 @@ import threading
|
||||
import unittest
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import httpx
|
||||
from contextlib import closing
|
||||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from bot_bottle.orchestrator_auth import ROLE_CLI, ROLE_GATEWAY, mint
|
||||
from bot_bottle.orchestrator import api as orchestrator_api
|
||||
from bot_bottle.orchestrator.broker import StubBroker
|
||||
from bot_bottle.orchestrator.http_contract import ORCHESTRATOR_AUTH_HEADER
|
||||
from bot_bottle.orchestrator.server import MAX_BODY_BYTES, create_app, make_server
|
||||
from bot_bottle.orchestrator.server import dispatch, make_server
|
||||
from bot_bottle.orchestrator.store.registry_store import BottleRecord, RegistryStore
|
||||
from bot_bottle.orchestrator.service import OrchestratorCore
|
||||
from bot_bottle.orchestrator.store.store_manager import StoreManager
|
||||
@@ -43,47 +37,6 @@ def _body(obj: object) -> bytes:
|
||||
return json.dumps(obj).encode()
|
||||
|
||||
|
||||
def dispatch(
|
||||
orchestrator: OrchestratorCore,
|
||||
method: str,
|
||||
path: str,
|
||||
body: bytes,
|
||||
*,
|
||||
role: str | None = ROLE_CLI,
|
||||
) -> tuple[int, dict[str, object]]:
|
||||
"""Exercise the real ASGI application without a network socket."""
|
||||
key = "in-process-dispatch-key"
|
||||
headers = {"content-type": "application/json"}
|
||||
if role is not None:
|
||||
headers[ORCHESTRATOR_AUTH_HEADER] = mint(role, key)
|
||||
|
||||
async def request() -> httpx.Response:
|
||||
transport = httpx.ASGITransport(app=create_app(orchestrator, signing_key=key))
|
||||
async with httpx.AsyncClient(
|
||||
transport=transport,
|
||||
base_url="http://orchestrator",
|
||||
follow_redirects=True,
|
||||
) as client:
|
||||
return await client.request(
|
||||
method, path, content=body, headers=headers,
|
||||
)
|
||||
|
||||
response = asyncio.run(request())
|
||||
payload = response.json()
|
||||
if response.status_code == 422:
|
||||
detail = payload.get("detail", []) if isinstance(payload, dict) else []
|
||||
field = ""
|
||||
if isinstance(detail, list) and detail and isinstance(detail[0], dict):
|
||||
location = detail[0].get("loc", ())
|
||||
if isinstance(location, (list, tuple)) and len(location) > 1:
|
||||
field = str(location[1])
|
||||
suffix = f": {field}" if field else ""
|
||||
return 400, {"error": f"invalid request body{suffix}"}
|
||||
if isinstance(payload, dict) and "detail" in payload and "error" not in payload:
|
||||
payload = {"error": payload["detail"]}
|
||||
return response.status_code, payload
|
||||
|
||||
|
||||
def _orchestrator(db_path: Path) -> OrchestratorCore:
|
||||
store = RegistryStore(db_path)
|
||||
store.migrate()
|
||||
@@ -298,51 +251,11 @@ class TestDispatch(unittest.TestCase):
|
||||
|
||||
|
||||
class TestServerRoundTrip(unittest.TestCase):
|
||||
def _raw_status(self, content_length: str, *, authenticated: bool = True) -> int:
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
key = "request-limits-key"
|
||||
server = make_server(
|
||||
_orchestrator(Path(tmp.name) / "r.db"),
|
||||
"127.0.0.1", 0, signing_key=key,
|
||||
)
|
||||
self.addCleanup(server.server_close)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
self.addCleanup(server.shutdown)
|
||||
conn = http.client.HTTPConnection(
|
||||
str(server.server_address[0]), server.server_address[1], timeout=5,
|
||||
)
|
||||
self.addCleanup(conn.close)
|
||||
conn.putrequest("POST", "/bottles")
|
||||
conn.putheader("Content-Length", content_length)
|
||||
if authenticated:
|
||||
conn.putheader(
|
||||
"x-bot-bottle-orchestrator-auth", mint(ROLE_CLI, key),
|
||||
)
|
||||
conn.endheaders()
|
||||
return conn.getresponse().status
|
||||
|
||||
def test_rejects_malformed_content_length(self) -> None:
|
||||
self.assertEqual(400, self._raw_status("not-a-number"))
|
||||
|
||||
def test_rejects_oversized_body_without_reading_it(self) -> None:
|
||||
self.assertEqual(413, self._raw_status(str(MAX_BODY_BYTES + 1)))
|
||||
|
||||
def test_rejects_unauthenticated_request_before_reading_body(self) -> None:
|
||||
self.assertEqual(
|
||||
401,
|
||||
self._raw_status(str(MAX_BODY_BYTES), authenticated=False),
|
||||
)
|
||||
|
||||
def test_http_register_health_attribute(self) -> None:
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
orch = _orchestrator(Path(tmp.name) / "r.db")
|
||||
signing_key = "round-trip-key"
|
||||
auth = mint(ROLE_CLI, signing_key)
|
||||
server = make_server(
|
||||
orch, "127.0.0.1", 0, signing_key=signing_key,
|
||||
)
|
||||
server = make_server(orch, "127.0.0.1", 0)
|
||||
self.addCleanup(server.server_close)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
@@ -354,10 +267,7 @@ class TestServerRoundTrip(unittest.TestCase):
|
||||
reg = json.load(urllib.request.urlopen(
|
||||
urllib.request.Request(
|
||||
f"{base}/bottles", data=_body({"source_ip": "10.243.0.7"}),
|
||||
method="POST", headers={
|
||||
"Content-Type": "application/json",
|
||||
"x-bot-bottle-orchestrator-auth": auth,
|
||||
},
|
||||
method="POST", headers={"Content-Type": "application/json"},
|
||||
), timeout=5,
|
||||
))
|
||||
self.assertTrue(reg["bottle_id"])
|
||||
@@ -369,10 +279,7 @@ class TestServerRoundTrip(unittest.TestCase):
|
||||
urllib.request.Request(
|
||||
f"{base}/attribute",
|
||||
data=_body({"source_ip": "10.243.0.7", "identity_token": reg["identity_token"]}),
|
||||
method="POST", headers={
|
||||
"Content-Type": "application/json",
|
||||
"x-bot-bottle-orchestrator-auth": auth,
|
||||
},
|
||||
method="POST", headers={"Content-Type": "application/json"},
|
||||
), timeout=5,
|
||||
))
|
||||
self.assertEqual(reg["bottle_id"], attr["bottle_id"])
|
||||
@@ -380,25 +287,15 @@ class TestServerRoundTrip(unittest.TestCase):
|
||||
def test_internal_failure_is_contextual_but_redacted(self) -> None:
|
||||
orch = MagicMock()
|
||||
orch.registry.all.side_effect = RuntimeError("SENSITIVE request value")
|
||||
signing_key = "failure-path-key"
|
||||
auth = mint(ROLE_CLI, signing_key)
|
||||
with patch("sys.stderr", io.StringIO()) as stderr:
|
||||
server = make_server(
|
||||
orch, "127.0.0.1", 0, signing_key=signing_key,
|
||||
)
|
||||
server = make_server(orch, "127.0.0.1", 0)
|
||||
self.addCleanup(server.server_close)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
self.addCleanup(server.shutdown)
|
||||
host, port = server.server_address[0], server.server_address[1]
|
||||
with self.assertRaises(urllib.error.HTTPError) as raised:
|
||||
urllib.request.urlopen(
|
||||
urllib.request.Request(
|
||||
f"http://{host}:{port}/bottles",
|
||||
headers={"x-bot-bottle-orchestrator-auth": auth},
|
||||
),
|
||||
timeout=5,
|
||||
)
|
||||
urllib.request.urlopen(f"http://{host}:{port}/bottles", timeout=5)
|
||||
payload = json.loads(raised.exception.read())
|
||||
output = stderr.getvalue()
|
||||
self.assertEqual({"error": "internal error"}, payload)
|
||||
@@ -407,76 +304,6 @@ class TestServerRoundTrip(unittest.TestCase):
|
||||
self.assertNotIn("SENSITIVE", output)
|
||||
|
||||
|
||||
class TestControlPlaneBoundary(unittest.IsolatedAsyncioTestCase):
|
||||
@staticmethod
|
||||
def _scope(key: str) -> dict[str, object]:
|
||||
return {
|
||||
"type": "http",
|
||||
"asgi": {"version": "3.0"},
|
||||
"http_version": "1.1",
|
||||
"method": "POST",
|
||||
"scheme": "http",
|
||||
"path": "/bottles",
|
||||
"raw_path": b"/bottles",
|
||||
"query_string": b"",
|
||||
"headers": [(
|
||||
ORCHESTRATOR_AUTH_HEADER.encode(),
|
||||
mint(ROLE_CLI, key).encode(),
|
||||
)],
|
||||
"client": ("127.0.0.1", 1),
|
||||
"server": ("127.0.0.1", 80),
|
||||
"state": {},
|
||||
}
|
||||
|
||||
async def test_chunked_oversized_body_returns_413(self) -> None:
|
||||
key = "stream-limit-key"
|
||||
called = False
|
||||
first: dict[str, object] = {
|
||||
"type": "http.request",
|
||||
"body": b"x" * MAX_BODY_BYTES,
|
||||
"more_body": True,
|
||||
}
|
||||
last: dict[str, object] = {
|
||||
"type": "http.request", "body": b"x", "more_body": False,
|
||||
}
|
||||
chunks: Iterator[dict[str, object]] = iter([first, last])
|
||||
sent: list[dict[str, object]] = []
|
||||
|
||||
async def receive() -> dict[str, object]:
|
||||
return next(chunks)
|
||||
|
||||
async def send(message: dict[str, object]) -> None:
|
||||
sent.append(message)
|
||||
|
||||
async def inner(*_args: object) -> None:
|
||||
nonlocal called
|
||||
called = True
|
||||
|
||||
boundary = orchestrator_api.ControlPlaneBoundary(inner, key)
|
||||
await boundary(self._scope(key), receive, send) # type: ignore[arg-type]
|
||||
self.assertFalse(called)
|
||||
self.assertEqual(413, sent[0]["status"])
|
||||
|
||||
async def test_slow_stream_returns_408(self) -> None:
|
||||
key = "stream-timeout-key"
|
||||
sent: list[dict[str, object]] = []
|
||||
|
||||
async def receive() -> dict[str, object]:
|
||||
await asyncio.sleep(1)
|
||||
return {"type": "http.request", "body": b"", "more_body": False}
|
||||
|
||||
async def send(message: dict[str, object]) -> None:
|
||||
sent.append(message)
|
||||
|
||||
async def inner(*_args: object) -> None:
|
||||
self.fail("timed-out body reached the application")
|
||||
|
||||
boundary = orchestrator_api.ControlPlaneBoundary(inner, key)
|
||||
with patch.object(orchestrator_api, "REQUEST_BODY_TIMEOUT_SECONDS", 0.01):
|
||||
await boundary(self._scope(key), receive, send) # type: ignore[arg-type]
|
||||
self.assertEqual(408, sent[0]["status"])
|
||||
|
||||
|
||||
class TestOrchestratorAuth(unittest.TestCase):
|
||||
"""Role-scoped control-plane tokens (issue #400 / #469 review): every route
|
||||
but /health needs a valid token, and the token's role gates which routes it
|
||||
@@ -542,9 +369,8 @@ class TestOrchestratorAuth(unittest.TestCase):
|
||||
self.assertIsNotNone(self.orch.registry.get(rec.bottle_id))
|
||||
|
||||
def _server_with_key(self, signing_key: str):
|
||||
server = make_server(
|
||||
self.orch, "127.0.0.1", 0, signing_key=signing_key,
|
||||
)
|
||||
with patch.dict("os.environ", {"BOT_BOTTLE_ORCHESTRATOR_TOKEN": signing_key}):
|
||||
server = make_server(self.orch, "127.0.0.1", 0)
|
||||
self.addCleanup(server.server_close)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
self.addCleanup(server.shutdown)
|
||||
@@ -573,10 +399,16 @@ class TestOrchestratorAuth(unittest.TestCase):
|
||||
self.assertEqual(403, self._status(f"{base}/bottles", header=gateway_tok))
|
||||
self.assertEqual(200, self._status(f"{base}/bottles", header=cli_tok))
|
||||
|
||||
def test_unconfigured_server_refuses_to_start(self) -> None:
|
||||
with patch.dict("os.environ", {}, clear=True):
|
||||
with self.assertRaisesRegex(ValueError, "signing key is required"):
|
||||
make_server(self.orch, "127.0.0.1", 0)
|
||||
def test_unconfigured_server_runs_open(self) -> None:
|
||||
"""No signing key set (tests / nft-protected Firecracker): open mode
|
||||
grants full cli access, so existing round-trip behavior is unchanged."""
|
||||
with patch.dict("os.environ", {}, clear=False):
|
||||
import os
|
||||
os.environ.pop("BOT_BOTTLE_ORCHESTRATOR_TOKEN", None)
|
||||
server = make_server(self.orch, "127.0.0.1", 0)
|
||||
self.addCleanup(server.server_close)
|
||||
self.assertEqual(ROLE_CLI, server.role_for(""))
|
||||
self.assertEqual(ROLE_CLI, server.role_for("anything"))
|
||||
|
||||
|
||||
class TestDispatchSupervise(unittest.TestCase):
|
||||
|
||||
@@ -49,16 +49,6 @@ class TestPut(unittest.TestCase):
|
||||
self.assertNotIsInstance(req.data, (bytes, bytearray))
|
||||
self.assertEqual(str(len(payload)), req.get_header("Content-length"))
|
||||
|
||||
def test_put_uses_network_deadline(self) -> None:
|
||||
with mock.patch.object(
|
||||
pub.urllib.request, "urlopen", return_value=_Resp(),
|
||||
) as urlopen:
|
||||
pub._put("https://reg/pkg", b"payload", token="")
|
||||
self.assertEqual(
|
||||
pub._REGISTRY_HTTP_TIMEOUT_SECONDS,
|
||||
urlopen.call_args.kwargs["timeout"],
|
||||
)
|
||||
|
||||
def test_small_bytes_body_still_works(self) -> None:
|
||||
captured: list[urllib.request.Request] = []
|
||||
|
||||
|
||||
@@ -13,7 +13,6 @@ import tempfile
|
||||
import threading
|
||||
import time
|
||||
import types
|
||||
import typing
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
@@ -48,7 +47,6 @@ from bot_bottle.gateway.supervisor.server import (
|
||||
jsonrpc_error,
|
||||
jsonrpc_result,
|
||||
parse_jsonrpc,
|
||||
resolved_routes_payload,
|
||||
validate_proposed_file,
|
||||
)
|
||||
|
||||
@@ -703,40 +701,22 @@ class TestResolvedRoutesPayload(unittest.TestCase):
|
||||
" - host: api.anthropic.com\n"
|
||||
" - host: www.google.com\n"
|
||||
)
|
||||
payload = resolved_routes_payload(
|
||||
typing.cast(
|
||||
supervise_server.PolicyResolver,
|
||||
_FakeSuperviseResolver(bottle_id="b1", policy=policy),
|
||||
),
|
||||
_SRC,
|
||||
_TOK,
|
||||
)
|
||||
payload = _handler(
|
||||
_FakeSuperviseResolver(bottle_id="b1", policy=policy)
|
||||
)._resolved_routes_payload()
|
||||
assert payload is not None
|
||||
self.assertFalse(payload["isError"]) # type: ignore[index]
|
||||
data = json.loads(payload["content"][0]["text"]) # type: ignore[index]
|
||||
hosts = {r["host"] for r in data["routes"]}
|
||||
self.assertEqual({"api.anthropic.com", "www.google.com"}, hosts)
|
||||
|
||||
def test_orchestrator_error_is_not_reported_as_empty_policy(self) -> None:
|
||||
with self.assertRaises(supervise_server.RouteResolutionError):
|
||||
resolved_routes_payload(
|
||||
typing.cast(
|
||||
supervise_server.PolicyResolver,
|
||||
_FakeSuperviseResolver(raises=True),
|
||||
),
|
||||
_SRC,
|
||||
_TOK,
|
||||
)
|
||||
|
||||
def test_authoritative_empty_policy_remains_successful(self) -> None:
|
||||
payload = resolved_routes_payload(
|
||||
typing.cast(
|
||||
supervise_server.PolicyResolver,
|
||||
_FakeSuperviseResolver(bottle_id="b1", policy="routes: []\n"),
|
||||
),
|
||||
_SRC,
|
||||
_TOK,
|
||||
)
|
||||
def test_orchestrator_error_fails_closed_to_empty(self) -> None:
|
||||
# resolve_client_context swallows resolver errors → deny-all (empty),
|
||||
# never another bottle's routes.
|
||||
payload = _handler(
|
||||
_FakeSuperviseResolver(raises=True)
|
||||
)._resolved_routes_payload()
|
||||
assert payload is not None
|
||||
data = json.loads(payload["content"][0]["text"]) # type: ignore[index]
|
||||
self.assertEqual([], data["routes"])
|
||||
|
||||
@@ -744,13 +724,7 @@ class TestResolvedRoutesPayload(unittest.TestCase):
|
||||
# A server without a resolver is a misconfig, not a mode: raise rather
|
||||
# than list anything.
|
||||
with self.assertRaises(_RpcInternalError):
|
||||
_handler(None)._dispatch(
|
||||
parse_jsonrpc(
|
||||
b'{"jsonrpc":"2.0","id":1,"method":"tools/call",'
|
||||
b'"params":{"name":"list-egress-routes"}}',
|
||||
),
|
||||
ServerConfig(),
|
||||
)
|
||||
_handler(None)._resolved_routes_payload()
|
||||
|
||||
|
||||
class TestNonBlockingSupervise(unittest.TestCase):
|
||||
|
||||
@@ -1,81 +0,0 @@
|
||||
"""Unit tests for framework-neutral supervisor MCP dispatch."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from dataclasses import dataclass
|
||||
|
||||
from bot_bottle.gateway.supervisor.mcp_dispatch import (
|
||||
Handlers,
|
||||
MethodNotFoundError,
|
||||
dispatch,
|
||||
)
|
||||
from bot_bottle.supervisor import types as _sv
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _Request:
|
||||
method: str
|
||||
params: dict[str, object]
|
||||
|
||||
|
||||
class TestDispatch(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.calls: list[str] = []
|
||||
|
||||
def handler(name: str):
|
||||
def call(_params: dict[str, object]) -> str:
|
||||
self.calls.append(name)
|
||||
return name
|
||||
|
||||
return call
|
||||
|
||||
self.handlers = Handlers(
|
||||
initialize=handler("initialize"),
|
||||
tools_list=handler("tools_list"),
|
||||
list_routes=handler("list_routes"),
|
||||
check_proposal=handler("check_proposal"),
|
||||
propose=handler("propose"),
|
||||
)
|
||||
|
||||
def request(self, method: str, **params: object) -> _Request:
|
||||
return _Request(method=method, params=params)
|
||||
|
||||
def test_routes_protocol_methods(self) -> None:
|
||||
self.assertEqual(
|
||||
"initialize", dispatch(self.request("initialize"), self.handlers),
|
||||
)
|
||||
self.assertEqual(
|
||||
"tools_list", dispatch(self.request("tools/list"), self.handlers),
|
||||
)
|
||||
self.assertIsNone(
|
||||
dispatch(self.request("notifications/initialized"), self.handlers),
|
||||
)
|
||||
|
||||
def test_routes_each_tool_class(self) -> None:
|
||||
self.assertEqual(
|
||||
"list_routes",
|
||||
dispatch(
|
||||
self.request("tools/call", name=_sv.TOOL_LIST_EGRESS_ROUTES),
|
||||
self.handlers,
|
||||
),
|
||||
)
|
||||
self.assertEqual(
|
||||
"check_proposal",
|
||||
dispatch(
|
||||
self.request("tools/call", name=_sv.TOOL_CHECK_PROPOSAL),
|
||||
self.handlers,
|
||||
),
|
||||
)
|
||||
self.assertEqual(
|
||||
"propose",
|
||||
dispatch(self.request("tools/call", name=_sv.TOOL_EGRESS_ALLOW), self.handlers),
|
||||
)
|
||||
|
||||
def test_unknown_method_is_typed(self) -> None:
|
||||
with self.assertRaisesRegex(MethodNotFoundError, "unknown"):
|
||||
dispatch(self.request("unknown"), self.handlers)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -78,7 +78,8 @@ class TestControlPlaneProvisioning(unittest.TestCase):
|
||||
self.assertEqual("key", prov.orchestrator_key())
|
||||
|
||||
def test_orchestrator_key_fail_closes_when_empty(self) -> None:
|
||||
# Invariant 4: the orchestrator must never start without a key. There is no
|
||||
# Invariant 4: the orchestrator must never start without a key — it would
|
||||
# run OPEN and grant every caller that reaches it full `cli`. There is no
|
||||
# topology opt-out: a separate host does not stop the gateway (or any
|
||||
# other caller) from reaching the control-plane listener.
|
||||
prov = ControlPlaneProvisioning()
|
||||
|
||||
@@ -62,10 +62,7 @@ class TestWheelInstall(unittest.TestCase):
|
||||
|
||||
# Installing the freshly-built wheel must succeed — fail if it doesn't.
|
||||
install = subprocess.run(
|
||||
[
|
||||
str(cls.venv_py), "-m", "pip", "install", "--quiet",
|
||||
"--force-reinstall", "--no-deps", str(wheels[0]),
|
||||
],
|
||||
[str(cls.venv_py), "-m", "pip", "install", "--quiet", str(wheels[0])],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if install.returncode != 0:
|
||||
|
||||
Reference in New Issue
Block a user