refactor(egress): write routes.yaml as actual YAML, not JSON-in-yml #42

2026-05-26T02:17:56-04:00

didericis commented

2026-05-26 02:17:56 -04:00

Summary

egress_render_routes now emits hand-rolled YAML in the same style as pipelock_render_yaml. The egress addon parses it via yaml_subset.parse_yaml_subset — the same parser the manifest loader + pipelock_apply use. No new dependencies.

Why

routes.yaml is bind-mounted into the egress sidecar AND surfaced to operators through routes edit (PRD 0019). JSON-in-yml renders ugly in $EDITOR and signals "this is data" rather than "this is config you can read at a glance". Real YAML reads cleanly.

Mechanics

yaml_subset.py drops its claude_bottle.log dependency. Errors now raise YamlSubsetError (a ValueError subclass); the manifest loader + pipelock_apply catch it at the boundary and forward to die / PipelockApplyError so callers see the same behavior they did before. This is what unlocks reusing the parser inside the egress container.

Dockerfile.egress adds one COPY line for yaml_subset.py so it sits flat in /app/ next to the addon. The addon uses an absolute-import-with-fallback shim so the same file works inside the container AND from the host's unit tests.

egress_apply._merge_single_route round-trips current routes.yaml through parse_yaml_subset + a new _render_routes_payload helper instead of json.loads + json.dumps. Same merge semantics, different format on the wire.

Status

453 unit tests pass
Supervise integration test passes
End-to-end: rebuilt the egress image and ran ./cli.py start implementer to a full bring-up; the addon's boot log shows egress: loaded 9 route(s) — proves the YAML parses cleanly inside the container

Rendered output sample

routes:
  - host: "api.anthropic.com"
    auth_scheme: "Bearer"
    token_env: "EGRESS_TOKEN_0"
    path_allowlist:
      - "/v1/"
  - host: "example.com"

## Summary `egress_render_routes` now emits hand-rolled YAML in the same style as `pipelock_render_yaml`. The egress addon parses it via `yaml_subset.parse_yaml_subset` — the same parser the manifest loader + pipelock_apply use. No new dependencies. ## Why routes.yaml is bind-mounted into the egress sidecar AND surfaced to operators through `routes edit` (PRD 0019). JSON-in-yml renders ugly in `$EDITOR` and signals "this is data" rather than "this is config you can read at a glance". Real YAML reads cleanly. ## Mechanics **`yaml_subset.py` drops its `claude_bottle.log` dependency.** Errors now raise `YamlSubsetError` (a `ValueError` subclass); the manifest loader + pipelock_apply catch it at the boundary and forward to `die` / `PipelockApplyError` so callers see the same behavior they did before. This is what unlocks reusing the parser inside the egress container. **`Dockerfile.egress` adds one COPY line for `yaml_subset.py`** so it sits flat in `/app/` next to the addon. The addon uses an absolute-import-with-fallback shim so the same file works inside the container AND from the host's unit tests. **`egress_apply._merge_single_route` round-trips current routes.yaml** through `parse_yaml_subset` + a new `_render_routes_payload` helper instead of `json.loads` + `json.dumps`. Same merge semantics, different format on the wire. ## Status - 453 unit tests pass - Supervise integration test passes - End-to-end: rebuilt the egress image and ran `./cli.py start implementer` to a full bring-up; the addon's boot log shows `egress: loaded 9 route(s)` — proves the YAML parses cleanly inside the container ## Rendered output sample ```yaml routes: - host: "api.anthropic.com" auth_scheme: "Bearer" token_env: "EGRESS_TOKEN_0" path_allowlist: - "/v1/" - host: "example.com" ```

didericis added 1 commit 2026-05-26 02:17:57 -04:00

refactor(egress): write routes.yaml as actual YAML, not JSON-in-yml

test / unit (pull_request) Successful in 18s

Details

test / integration (pull_request) Successful in 1m7s

Details

c9825cf701

`egress_render_routes` now emits hand-rolled YAML in the same style
as `pipelock_render_yaml`. The egress addon parses it via
`yaml_subset.parse_yaml_subset` — the same parser the manifest
loader + pipelock_apply use.

Why bother: routes.yaml is bind-mounted into the egress sidecar
AND surfaced to operators through `routes edit` (PRD 0019). JSON-
in-yml renders ugly in $EDITOR and signals "this is data" rather
than "this is config you can read at a glance". Real YAML reads
cleanly.

Mechanics:

  - `yaml_subset.py` drops its `claude_bottle.log` dependency.
    Errors now raise `YamlSubsetError` (a `ValueError`); the
    manifest loader + pipelock_apply catch it at the boundary
    and forward to `die` / `PipelockApplyError` so callers see
    the same behavior they did before.
  - `Dockerfile.egress` adds one COPY line for `yaml_subset.py`
    so it sits flat in `/app/` next to the addon. The addon
    uses an absolute-import-with-fallback shim so the same file
    works inside the container AND from the host's unit tests.
  - `egress_apply._merge_single_route` round-trips current
    routes.yaml through `parse_yaml_subset` + a new
    `_render_routes_payload` helper instead of `json.loads` +
    `json.dumps`.

End-to-end: rebuilt the egress image, ran `./cli.py start` to a
full bring-up, confirmed the addon's boot log shows `egress:
loaded 9 route(s)` — i.e., the YAML parses inside the container.
453 unit + 3 integration tests pass.

didericis added 1 commit 2026-05-26 02:31:49 -04:00

fix(apply): write routes/pipelock yaml in place, not via rename

test / unit (pull_request) Successful in 17s

Details

test / integration (pull_request) Successful in 1m6s

Details

3c2585cb98

PRD 0018 chunk 3's atomicity fix used write-temp-then-rename to
update bind-mounted config files. POSIX rename atomically swaps
the inode at the host path — but Docker single-file bind mounts
on Linux pin the source inode at mount time, so post-rename the
container's mount points at the now-orphaned old inode and never
sees the new content. The egress sidecar's SIGHUP-driven reload
re-reads the same stale file → "egress route updates aren't
updatable via the supervisor anymore".

Switch egress_apply + pipelock_apply to write in place (same
inode, truncated + rewritten). Lose file-level POSIX atomicity,
but:

  - egress: SIGHUP fires only AFTER the write returns; the
    addon's `load_routes` raises `ValueError` on a partial read
    and keeps the previous in-memory routes, so the in-process
    race window (already narrow) is non-disruptive.
  - pipelock: applies via `docker restart` rather than SIGHUP;
    restart serializes after the host write completes, so the
    container reads the fully-written file on next boot.

macOS Docker Desktop's file-sharing layer (virtiofs / osxfs)
silently re-resolves the path on rename, which is why this bug
didn't surface in dev tests on macOS. Linux native Docker is
the strict reading; the fix works on both.

didericis merged commit 942d3a387a into main

2026-05-26 02:38:45 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: didericis/bot-bottle#42