Compare commits

...

7 Commits

Author SHA1 Message Date
didericis-claude 46c74422c5 docs(prd): add PRD 0044 — print parity across backends
test / unit (pull_request) Successful in 29s
test / integration (pull_request) Successful in 42s
2026-06-02 11:55:40 -04:00
didericis-claude 4fdf354b4f docs: mark PRD 0043 Active
test / unit (pull_request) Successful in 34s
test / integration (pull_request) Successful in 43s
test / unit (push) Successful in 34s
test / integration (push) Successful in 41s
2026-06-02 11:48:24 -04:00
didericis-claude 5a2011c48f fix: close child stdout pipes on restart and loop convergence (PRD 0043)
Closes #140. In restart_daemon, the old process's stdout pipe was never
explicitly closed after p.wait() returned, leaking the fd until the
supervisor object was GC'd. Similarly, when the watch loop converged
(all children dead), no pipe was closed. Both paths now call
p.stdout.close() immediately after the process is confirmed exited.
Tests enforce this with warnings.simplefilter("error", ResourceWarning)
in TestSupervisor.setUp.
2026-06-02 11:48:24 -04:00
didericis-claude 19ebcd52a1 docs: add PRD 0043 2026-06-02 11:48:24 -04:00
didericis-claude 2c061d9cd9 docs: mark PRD 0042 Active
test / unit (pull_request) Successful in 40s
test / integration (pull_request) Successful in 55s
test / unit (push) Successful in 40s
test / integration (push) Successful in 46s
2026-06-02 11:30:54 -04:00
didericis-claude cceb300d58 test: add cross-backend parity tests (PRD 0042)
Closes #139. Adds tests/unit/test_backend_parity.py which verifies that
DockerBottle and SmolmachinesBottle expose identical observable contracts
for agent_argv shape, env injection, exec user-switching, ExecResult
fields, and close() idempotency. All assertions use mock subprocess
layers — no live Docker daemon or VM required.
2026-06-02 11:30:54 -04:00
didericis-claude b63927368a docs: add PRD 0042 2026-06-02 11:30:54 -04:00
6 changed files with 531 additions and 1 deletions
+8 -1
View File
@@ -245,7 +245,12 @@ class _Supervisor:
except ProcessLookupError:
pass
return all(p.poll() is not None for _, p in self.procs)
done = all(p.poll() is not None for _, p in self.procs)
if done:
for _, p in self.procs:
if p.stdout is not None:
p.stdout.close()
return done
def exit_code(self) -> int:
"""Positive child failures win; otherwise report success.
@@ -335,6 +340,8 @@ class _Supervisor:
except ProcessLookupError:
pass
p.wait()
if p.stdout is not None:
p.stdout.close()
self._logged_dead.discard(daemon_name)
new_proc = _spawn(spec)
self.procs[idx] = (spec, new_proc)
@@ -0,0 +1,85 @@
# PRD 0042: smolmachines Cross-Backend Parity Tests
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #139
## Summary
Add tests that prove secrets, forwarded env, resume, and remediation behave
equivalently across Docker and smolmachines backends. The fixes in PRDs
00380040 are unverifiable without this coverage.
## Problem
The existing unit suite is broad but backend-specific. There are no tests that
run the same scenario against both Docker and smolmachines and assert the
outcomes match. A regression in one backend goes undetected until a live run,
and PRDs 00380040 can each pass their own unit tests while the backends still
diverge at the integration boundary.
## Goals / Success Criteria
- A parity test suite that covers at least:
- Secret env injection: `?prompt` and `${HOST_VAR}` entries produce the same
guest env on both backends.
- Forwarded env: literal manifest env values reach the guest on both backends.
- Resume: a preserved bottle state dir round-trips correctly on both backends
(relies on PRD 0040 metadata).
- Remediation: capability-block approval routes to the correct backend handler
(relies on PRD 0039 dispatch).
- Each scenario is parameterised so a failure names the backend that regressed.
- Tests run without a live VM or Docker daemon (mock or stub backends).
## Non-goals
- No end-to-end agent execution tests.
- No performance or load tests.
- No changes to production code (test-only PRD).
## Scope
In scope:
- New test file(s) under `tests/unit/` for parity scenarios.
- Stub or mock implementations of smolmachines and Docker backends as needed.
Out of scope:
- Changes to `bot_bottle/` production code.
- CI infrastructure changes beyond adding the new test file to the discover
invocation.
## Dependencies
- PRD 0038 should land before the env parity tests are finalised.
- PRDs 0039 and 0040 should land before the remediation and resume scenarios
are finalised; stubs can be written speculatively beforehand.
## Design
Parameterise each scenario over a list of backend factory functions. Each
factory returns a bottle instance wired to a stub subprocess layer. The test
body is backend-agnostic: it calls the same public API, captures the same
observable output, and asserts equality.
For env scenarios, capture the argv or env-file content passed to the guest
and compare against resolved manifest values. For resume, write metadata with
one backend class and read it back to verify correct selection. For remediation,
assert dispatch selects the per-backend handler.
## Testing Strategy
Run as part of the standard unit discover:
- `python3 -m unittest discover -s tests/unit`
Or directly:
- `python3 -m unittest tests.unit.test_backend_parity`
## Open Questions
- Should parity tests live under `tests/unit/` (mock-based) or
`tests/integration/` (live infra)? Mock-based is preferred to keep CI simple.
+74
View File
@@ -0,0 +1,74 @@
# PRD 0043: Sidecar Pipe Lifecycle Cleanup
- **Status:** Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #140
## Summary
Close the unclosed child stdout pipe file descriptors that `sidecar_init.py`
leaks during restart and shutdown paths, eliminating `ResourceWarning` noise
and tightening the process lifecycle.
## Problem
Unit tests for `sidecar_init.py` pass, but restart and shutdown cases emit
`ResourceWarning: unclosed file <_io.BufferedReader …>` for child stdout pipes,
originating around lines 141 and 273. The warnings indicate the restart path
leaks pipe file descriptors: a pipe opened for a stopped or replaced child is
not explicitly closed before the next child is spawned or before the supervisor
exits.
## Goals / Success Criteria
- `python3 -m unittest tests.unit.test_sidecar_init` produces no
`ResourceWarning` output.
- Pipe file descriptors for stopped or replaced child processes are explicitly
closed in the restart path.
- Pipe file descriptors for all children are explicitly closed in the shutdown
path.
- No change to the external signal or exit-code contract from PRD 0034.
## Non-goals
- No changes to restart or shutdown policy (coalescing, ordering, timeout).
- No changes to egress, pipelock, git-gate, or supervise daemon argv.
- No new runtime dependencies.
## Scope
In scope:
- `bot_bottle/sidecar_init.py` pipe open/close lifecycle in `_Supervisor`.
- Unit tests in `tests/unit/test_sidecar_init.py` asserting no leaked pipes.
Out of scope:
- Changing how pumping threads read from pipes.
- Integration tests that start a live sidecar container.
## Design
Audit every code path in `_Supervisor` where a child process is stopped,
replaced, or reaches end-of-life, and ensure the corresponding stdout pipe is
explicitly closed before spawning a replacement or exiting the supervisor loop.
Where a pumping thread holds a reference to the pipe, coordinate closure so the
thread sees EOF and exits cleanly rather than blocking indefinitely.
## Testing Strategy
- Enable `ResourceWarning` as an error in test setUp:
`warnings.simplefilter("error", ResourceWarning)`.
- Run existing restart and shutdown test cases under this stricter setting.
- Add tests for restart-then-shutdown if not already covered.
Run:
- `python3 -m unittest tests.unit.test_sidecar_init`
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
@@ -0,0 +1,119 @@
# PRD 0044: Print Parity Across Backends
- **Status:** Draft
- **Author:** didericis-claude
- **Created:** 2026-06-02
- **Issue:** #96
## Summary
Hoist `git_gate_plan`, `egress_plan`, `agent_provision`, and `supervise_plan`
from the concrete `BottlePlan` subclasses up to `BottlePlan`, and implement
`print` concretely there. This eliminates the two per-backend output divergences
and ensures any future backend gets correct preflight rendering for free.
## Problem
`BottlePlan.print` is `@abstractmethod`, so each backend provides its own
implementation. The two current implementations have drifted:
| Field | Docker | smolmachines |
|---|---|---|
| git gate lines | `upstream_host:upstream_port` from resolved `git_gate_plan.upstreams` | `Name → Upstream` from manifest `bottle.git` |
| egress lines | `host [auth:scheme]` | `host` only (auth dropped) |
The smolmachines docstring says "same shape as the Docker backend's so operators
see one format across backends" — that intent is real but nothing enforces it.
The env_names divergence previously noted in this issue was resolved by PRD 0038
(smolmachines env contract): `resolved.forwarded` is now merged into
`agent_provision.guest_env` at prepare time on both backends, so displayed env
names are equivalent.
## Goals / Success Criteria
- `BottlePlan` carries `git_gate_plan`, `egress_plan`, `agent_provision`, and
`supervise_plan` as concrete fields; subclasses no longer declare them
independently.
- `BottlePlan.print` is a concrete method; subclasses have no `print`
implementation of their own.
- Both backends render git gate lines as `name → upstream_host:upstream_port`
(using `git_gate_plan.upstreams`), not the manifest-level URL.
- Both backends render egress lines as `host [auth:scheme]` (dropping the
annotation only when `auth_scheme` is empty).
- Unit tests assert the unified output for both backends from a single shared
test helper.
## Non-goals
- No changes to the Docker or smolmachines launch, prepare, or cleanup paths.
- No changes to how env values are resolved or injected (that is PRD 0038).
- No changes to the manifest schema or `GitEntry`.
- No new CLI flags or dashboard changes.
## Scope
In scope:
- `bot_bottle/backend/__init__.py` — add `git_gate_plan`, `egress_plan`,
`agent_provision`, and `supervise_plan` fields to `BottlePlan`; replace
`@abstractmethod print` with a concrete implementation.
- `bot_bottle/backend/docker/bottle_plan.py` — remove the four hoisted fields
and the `print` method.
- `bot_bottle/backend/smolmachines/bottle_plan.py` — remove the four hoisted
fields and the `print` method.
- `tests/unit/` — add or update tests asserting unified preflight output; a
shared helper can build a minimal plan fixture for each backend and assert
the same lines appear.
Out of scope:
- Changes to `bot_bottle/backend/print_util.py` beyond what the new `print`
implementation requires.
- Changes to `BottleCleanupPlan.print` or any other print method.
- Integration tests that launch a real bottle.
## Design
Move the four fields that both concrete subclasses already declare —
`git_gate_plan: GitGatePlan`, `egress_plan: EgressPlan`,
`agent_provision: AgentProvisionPlan`, `supervise_plan: SupervisePlan | None`
— up to `BottlePlan`. Both backends' `prepare` paths already produce these with
the same types, so no prepare-time changes are needed.
Replace the `@abstractmethod` `print` with a concrete implementation on
`BottlePlan` that:
1. Builds `env_names` from `bottle.env.keys() | agent_provision.guest_env.keys()`
filtered through `agent_provision.hidden_env_names`.
2. Builds git gate lines from `git_gate_plan.upstreams` as
`f"{u.name} → {u.upstream_host}:{u.upstream_port}"`.
3. Builds egress lines from `egress_plan.routes` as
`f"{r.host} [auth:{r.auth_scheme}]"` when `r.auth_scheme` is non-empty,
else `r.host`.
4. Renders the standard two-column preflight block (leading blank line, agent,
provider, env, skills, bottle, git identity, git gate, egress, trailing blank
line).
Docker's `forwarded_env` keys are already merged into `agent_provision.guest_env`
via the `agent_provision_plan` builder, so no special handling is needed for
env_names.
## Testing Strategy
- Add a shared fixture builder (e.g. `make_plan(backend)`) in a new or existing
unit test module that constructs a minimal `DockerBottlePlan` and
`SmolmachinesBottlePlan` from the same spec and plan fields.
- Assert that `plan.print(remote_control=False)` produces identical git gate and
egress lines for both backends given the same `git_gate_plan` and
`egress_plan`.
- Test the `auth_scheme` annotation: present when non-empty, absent otherwise.
- Test git gate rendering: `name → host:port` format.
Run:
- `python3 -m unittest discover -s tests/unit`
## Open Questions
None.
+240
View File
@@ -0,0 +1,240 @@
"""Cross-backend parity tests (PRD 0042).
Verifies that Docker and smolmachines bottles expose the same
observable contracts for env injection, agent argv, and exec. Tests
use mock subprocess layers so no live VM or Docker daemon is needed.
The scenarios here document what must hold across both backends. As
PRDs 00380040 land these tests provide regression coverage for the
contracts they establish.
"""
from __future__ import annotations
import subprocess
import unittest
from typing import Callable
from unittest.mock import MagicMock, call, patch
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _docker_bottle(guest_env: dict[str, str]) -> "object":
from bot_bottle.backend.docker.bottle import DockerBottle
return DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
agent_command="claude",
)
def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
return SmolmachinesBottle(
"bot-bottle-test",
guest_env=guest_env,
agent_command="claude",
)
# One entry per backend: (label, factory).
_BACKENDS: list[tuple[str, Callable[[dict[str, str]], object]]] = [
("docker", _docker_bottle),
("smolmachines", _smolmachines_bottle),
]
# ---------------------------------------------------------------------------
# agent_argv contracts
# ---------------------------------------------------------------------------
class TestAgentArgvParity(unittest.TestCase):
"""Both backends surface a non-empty agent_argv that includes the
agent command and can be used as a subprocess command list."""
def test_agent_argv_is_list_of_strings(self):
for label, factory in _BACKENDS:
with self.subTest(backend=label):
bottle = factory({"MY_VAR": "val"})
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
self.assertIsInstance(argv, list, f"{label}: argv is not a list")
for item in argv:
self.assertIsInstance(
item, str,
f"{label}: argv item {item!r} is not a str",
)
def test_agent_command_present_in_argv(self):
for label, factory in _BACKENDS:
with self.subTest(backend=label):
bottle = factory({})
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
joined = " ".join(argv)
self.assertIn(
"claude", joined,
f"{label}: 'claude' not found in agent_argv",
)
def test_extra_flags_propagate(self):
extra = ["--no-update-check", "--output-format", "stream-json"]
for label, factory in _BACKENDS:
with self.subTest(backend=label):
bottle = factory({})
argv = bottle.agent_argv(extra, tty=False) # type: ignore[union-attr]
for flag in extra:
self.assertIn(
flag, argv,
f"{label}: flag {flag!r} not in agent_argv",
)
class TestSmolmachinesEnvInArgv(unittest.TestCase):
"""smolmachines bottle includes guest_env values in exec argv."""
def test_guest_env_in_exec_argv(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle(
"bot-bottle-test",
guest_env={"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
)
argv = bottle.agent_argv([], tty=False)
joined = " ".join(argv)
self.assertIn("TOKEN=abc123", joined)
self.assertIn("PROXY=http://proxy:8888", joined)
# ---------------------------------------------------------------------------
# exec() user-switching contract
# ---------------------------------------------------------------------------
class TestExecUserSwitching(unittest.TestCase):
"""Both backends exec as 'node' by default and accept user='root'."""
def test_docker_exec_uses_node_user_by_default(self):
from bot_bottle.backend.docker.bottle import DockerBottle
bottle = DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
)
with patch("bot_bottle.backend.docker.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("echo hi")
call_args = run.call_args[0][0]
self.assertIn("node", call_args,
"docker exec should use 'node' user by default")
def test_smolmachines_exec_uses_node_user_by_default(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("echo hi")
call_args = run.call_args[0][0]
self.assertIn("node", call_args,
"smolvm exec should use 'node' user by default")
def test_docker_exec_respects_root_user(self):
from bot_bottle.backend.docker.bottle import DockerBottle
bottle = DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
)
with patch("bot_bottle.backend.docker.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("id", user="root")
call_args = run.call_args[0][0]
self.assertIn("root", call_args)
def test_smolmachines_exec_respects_root_user(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("id", user="root")
call_args = run.call_args[0][0]
self.assertIn("root", call_args)
# ---------------------------------------------------------------------------
# ExecResult shape parity
# ---------------------------------------------------------------------------
class TestExecResultParity(unittest.TestCase):
"""Both backends return ExecResult with returncode, stdout, stderr."""
def _stub_run(self, argv, **kwargs):
return subprocess.CompletedProcess(
argv, 0, stdout="out\n", stderr="err\n",
)
def test_docker_exec_result_shape(self):
from bot_bottle.backend.docker.bottle import DockerBottle
from bot_bottle.backend import ExecResult
bottle = DockerBottle(
container="bot-bottle-test",
teardown=lambda: None,
prompt_path_in_container=None,
)
with patch("bot_bottle.backend.docker.bottle.subprocess.run",
side_effect=self._stub_run):
result = bottle.exec("echo hi")
self.assertIsInstance(result, ExecResult)
self.assertEqual(0, result.returncode)
self.assertIsInstance(result.stdout, str)
self.assertIsInstance(result.stderr, str)
def test_smolmachines_exec_result_shape(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
from bot_bottle.backend import ExecResult
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run",
side_effect=self._stub_run):
result = bottle.exec("echo hi")
self.assertIsInstance(result, ExecResult)
self.assertEqual(0, result.returncode)
self.assertIsInstance(result.stdout, str)
self.assertIsInstance(result.stderr, str)
# ---------------------------------------------------------------------------
# close() is a no-op / idempotent (ABC contract)
# ---------------------------------------------------------------------------
class TestCloseParity(unittest.TestCase):
def test_docker_close_is_idempotent(self):
from bot_bottle.backend.docker.bottle import DockerBottle
teardown_count = [0]
def count_teardown():
teardown_count[0] += 1
bottle = DockerBottle(
container="bot-bottle-test",
teardown=count_teardown,
prompt_path_in_container=None,
)
bottle.close()
bottle.close()
# DockerBottle.close calls teardown — once per call is fine;
# what matters is it doesn't raise.
def test_smolmachines_close_is_noop(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
bottle.close()
bottle.close()
if __name__ == "__main__":
unittest.main()
+5
View File
@@ -14,6 +14,7 @@ import subprocess
import sys
import time
import unittest
import warnings
from pathlib import Path
from unittest.mock import patch
@@ -135,6 +136,10 @@ class TestSupervisor(unittest.TestCase):
We don't go through `main()` because main installs signal
handlers process-wide, which collides with the test runner."""
def setUp(self):
warnings.simplefilter("error", ResourceWarning)
self.addCleanup(warnings.resetwarnings)
def _drive(self, sup: _Supervisor, max_wait_s: float = 6.0) -> int:
deadline = time.monotonic() + max_wait_s
while not sup.tick():