2026-05-24 22:43:44 -04:00
1 changed files with 446 additions and 0 deletions
@@ -0,0 +1,446 @@
 # PRD 0011: Per-file Markdown manifest
 - **Status:** Draft
 - **Author:** didericis
 - **Created:** 2026-05-24
 ## Summary
 Replace the single-file `claude-bottle.json` manifest with a
 per-file Markdown-with-YAML-frontmatter layout. Bottles live as
 `$HOME/.claude-bottle/bottles/<name>.md`; agents live as
 `$HOME/.claude-bottle/agents/<name>.md` (home-resident) and
 `$CWD/.claude-bottle/agents/<name>.md` (repo-supplied). Each file
 carries its structured config in YAML frontmatter and (for agents)
 its system prompt in the Markdown body.
 The format change clears the way for the layout change: one file
 per bottle, one file per agent, two directories on each side of
 the `$HOME` / `$CWD` trust boundary. That boundary stops living in
 resolver logic (PRD 0011-v1's CwdExtension approach, closed in
 favor of this design) and becomes filesystem layout — `$CWD` has
 no `bottles/` subdirectory, period.
 The YAML we accept is bounded (flat keys → strings, lists, simple
 nested dicts), so the parser is hand-rolled and stdlib-only — no
 PyYAML dependency. The project's "low deps by default" stance
 (CLAUDE.md) stays intact.
 ## Problem
 `claude-bottle.json` works fine at one bottle and one agent. The
 project is heading for many of both, and the single-JSON shape
 starts to fray:
 - **Discovery + diff scaling.** A user with 8 bottles and 12
  agents lands at hundreds of lines of nested JSON. Two changes
  to unrelated agents touch the same file; codeowners-style
  ownership doesn't apply. File-globbing tools (`grep`, `fd`)
  can't find one agent without parsing the whole file.
 - **No comments, no multi-line strings.** Agent prompts longer
  than a sentence become single-line escaped horrors in JSON.
  Documentation about why a bottle exists (which tokens it
  holds, why these egress allowlist entries) has nowhere natural
  to live in the manifest file itself; a sibling README drifts.
 - **Trust boundary lives in code, not on disk.** PRD 0011-v1
  (closed; see PR #15) made the resolver reject cwd manifests
  that try to define bottles. The rule is correct and enforced,
  but it's invisible to anyone reading the on-disk layout —
  there's no positive signal that `$HOME` is the only place
  bottles can come from. A reader has to know the resolver's
  rules to audit the security posture.
 The companion research
 (`docs/research/manifest-format-and-grouping.md`) walks the two
 axes (grouping × format) and lands on this design.
 ## Goals / Success criteria
 Each test runs against a temporary `$HOME` and a temporary `$CWD`:
 1. **A bottle file under `$HOME/.claude-bottle/bottles/`
   parses.** A `dev.md` file with YAML frontmatter declaring
   `cred_proxy.routes`, `git`, `env`, `egress` produces a Bottle
   dataclass equivalent to the current JSON shape.
 2. **An agent file under `$HOME/.claude-bottle/agents/` parses.**
   `implementer.md` with frontmatter that names `bottle:`,
   `skills:`, and other fields, with the body as the system
   prompt, produces an Agent dataclass.
 3. **An agent file under `$CWD/.claude-bottle/agents/` parses
   and overrides home-resident agents of the same name.** The
   cwd agent's frontmatter and body win; the home bottle it
   references stays intact.
 4. **A bottle file under `$CWD/.claude-bottle/bottles/` is
   ignored.** The directory does not contribute to the
   manifest; if a user accidentally creates one, the launcher
   emits a `warn`-level log naming the offending files and
   continues. Filesystem layout is the boundary; the warning
   is a usability nicety, not a security gate.
 5. **No third-party Python dependencies introduced.** A fresh
   clone with only stdlib + claude-bottle's own code runs every
   parser test. Frontmatter parsing is hand-rolled against the
   declared YAML subset.
 6. **Migration tool converts existing JSON to per-file MD.**
   `./cli.py migrate-manifest` reads `$HOME/claude-bottle.json`
   (and `$CWD/claude-bottle.json` if present), writes a tree of
   per-file MD docs to the new locations, then prints what was
   moved. Idempotent: rerunning is a no-op when the new layout
   already exists. Does not delete the old JSON files
   automatically (user-driven cleanup).
 7. **Existing tests pass against the new layout.** Tests today
   build manifests via JSON literals against `Manifest.from_json_obj`.
   That entry point keeps working for tests (used to construct
   manifests programmatically); production resolution flows
   through the new directory-globbing loader.
 8. **Agent files double as Claude Code subagent files.** The
   `name`, `description`, `model`, `color`, and `memory` fields
   from Claude Code's existing subagent spec are accepted in
   our frontmatter alongside our own fields. Copying an agent
   file from `$HOME/.claude-bottle/agents/` to
   `~/.claude/agents/` produces a working Claude Code subagent
   (subject to Claude Code's tolerance for the extra `bottle:`
   and `claude_bottle:` fields — see Open Questions).
 ## Non-goals
 - **A general YAML implementation.** The parser handles the
  subset claude-bottle's frontmatter actually uses; documents
  that exceed the subset (anchors, multi-line block scalars,
  tags, implicit type coercion, flow style, etc.) die with a
  pointer at the spec. We are not building a YAML library.
 - **Compatibility with the old JSON layout at runtime.** The
  resolver no longer reads `claude-bottle.json` files. The
  migration tool is the bridge; after migration the JSON file
  is stale (and the user removes it). This is a breaking
  change for v1 users; the migration cost is one command + a
  manual delete.
 - **`$HOME/.claude/agents/` integration on the input side.** We
  don't read agent files out of Claude Code's directory. Our
  files can be copied into Claude Code's tree by the user if
  they want, but the input path for claude-bottle is its own
  directory.
 - **A signed-manifest scheme.** Out of scope per the
  closed-PR-15 PRD; the trust boundary here is "your home
  directory is yours."
 - **Per-bottle inheritance / composition.** Each bottle file is
  self-contained. If shared egress allowlists become common we
  can revisit, but the v1 of this PRD is one file = one bottle.
 - **Hot-reload.** Changes to manifest files take effect at next
  `./cli.py start`; we do not watch the directory.
 ## Scope
 ### In scope
 - **Directory layout.**
  - `$HOME/.claude-bottle/bottles/<name>.md` — bottle
    definitions (full schema; one Bottle per file).
  - `$HOME/.claude-bottle/agents/<name>.md` — home-resident
    agents.
  - `$CWD/.claude-bottle/agents/<name>.md` — cwd-resident
    agents; same schema as home agents, but bottle names must
    resolve against the home set.
  - `$CWD/.claude-bottle/bottles/` — ignored with a warn-level
    log (see SC #4). Does not contribute to the manifest.
  - `<name>` is the file basename without `.md`. Filenames must
    match `[a-z][a-z0-9-]*` (kebab-case, ASCII-only).
 - **File schema.** Markdown with YAML frontmatter. Frontmatter
  delimited by `---` lines at the top of the file; everything
  after the closing `---` is the body. For agents, body is the
  system prompt. For bottles, body is human documentation
  (optional, ignored by the parser).
 - **Agent frontmatter fields.**
  - `bottle: <name>` (required) — bottle to launch in.
  - `skills: [<name>, ...]` (optional) — host-side skills under
    `~/.claude/skills/`.
  - `name`, `description`, `model`, `color`, `memory` — accepted
    but treated as Claude Code passthrough; claude-bottle
    ignores them at launch but doesn't reject. Lets the same
    file double as a Claude Code subagent.
  - Unknown top-level keys die with a hint listing accepted
    keys. We don't silently ignore typos.
 - **Bottle frontmatter fields.** Same keys as today's JSON
  schema: `env`, `git`, `cred_proxy.routes`, `egress.allowlist`,
  `egress.dlp_action`. No semantic changes.
 - **YAML subset parser.** Hand-rolled, stdlib-only. Supports:
  - Flat `key: value` pairs at the top level.
  - String, int, bool (`true`/`false` only — no `yes`/`no`/`on`/
    `off`), null (`null` / explicit `~`).
  - Lists: block-style `- item` lines, items are strings or
    flow lists/dicts of the same.
  - Nested dicts: one level under a key, block-style.
  - Quoted strings: single + double, escapes as JSON-style.
  - Comments: `# ...` at end of line or on its own.
  Rejects with a clear error: anchors (`&`/`*`), multi-line
  block scalars (`|`, `>`), tags (`!!`), implicit-typed strings
  (`NO`/`Norway`/dates auto-coerced to booleans/dates),
  flow-style nested deeper than one level. Empty document is
  fine; missing frontmatter delimiters is fine for bottles
  (file = body-only is treated as no-frontmatter, which fails
  the required-keys check — same diagnostic as malformed).
 - **Manifest assembly.** New resolver:
  1. Walk `$HOME/.claude-bottle/bottles/*.md` → Bottle dict
     keyed by filename.
  2. Walk `$HOME/.claude-bottle/agents/*.md` → Agent dict.
  3. Walk `$CWD/.claude-bottle/agents/*.md` → Agent dict; merge
     into the home agent dict, cwd wins on name collision.
  4. Validate every agent's `bottle:` against the bottle dict.
  5. Warn if `$CWD/.claude-bottle/bottles/` exists with files.
  6. Return Manifest dataclass — same shape as today.
 - **Migration command.** `./cli.py migrate-manifest`:
  - Reads `$HOME/claude-bottle.json` and (if present)
    `$CWD/claude-bottle.json`.
  - Creates `$HOME/.claude-bottle/{bottles,agents}/` dirs.
  - For each `bottles[<name>]`, writes
    `$HOME/.claude-bottle/bottles/<name>.md` with frontmatter
    rendered from the bottle dict, body empty (or a one-line
    "Migrated from claude-bottle.json on <date>" stub).
  - For each home `agents[<name>]`, writes
    `$HOME/.claude-bottle/agents/<name>.md` with frontmatter
    (bottle, skills, etc.) and body = `prompt`.
  - For each cwd `agents[<name>]` (if cwd JSON existed),
    writes `$CWD/.claude-bottle/agents/<name>.md`.
  - Refuses to overwrite existing MD files; if a target
    already exists, prints what would have been written and
    bails on that file (continues with the others).
  - Prints a summary at the end: N bottles written, M agents
    written, what was skipped.
 - **Docs.** README's manifest section rewrites against the new
  layout. `claude-bottle.example.json` becomes
  `examples/bottles/dev.md` + `examples/agents/implementer.md`.
  The PRD 0010 example block in its own document gets a
  follow-up commit noting the new layout (out of scope for
  this PRD; only update README + example files here).
 - **Tests.**
  - `tests/unit/test_yaml_subset_parser.py` — the parser
    itself, including all the rejection cases listed above.
  - `tests/unit/test_manifest_md_load.py` — directory-globbing
    + assembly, the 8 success criteria.
  - `tests/integration/test_migrate_manifest.py` — round-trip
    JSON → MD; idempotency.
  - Existing integration tests keep working (the only public
    entry points they hit are `Manifest.resolve` and
    `Manifest.from_json_obj`).
 ### Out of scope
 - Watching the directory for changes mid-session.
 - A migration tool for moving back (MD → JSON).
 - Validating that frontmatter `name:` matches the filename.
  Soft check via a warn log if mismatched, but not enforced.
 - A bottle/agent dependency graph beyond the existing `bottle:`
  field. No "this agent extends this other agent."
 - IDE schemas / JSON Schema export for the MD format.
 ## Proposed design
 ### File layout
 ```
 $HOME/.claude-bottle/
 ├── bottles/
 │   ├── dev.md
 │   ├── gitea-dev.md
 │   └── ...
 └── agents/
    ├── implementer.md
    ├── researcher.md
    └── ...
 $CWD/.claude-bottle/
 └── agents/
    └── <repo-specific>.md
 ```
 `bottles/` only exists under `$HOME`. The directory's absence
 under `$CWD` is the boundary — the loader doesn't even look
 there.
 ### Example bottle file
 ```markdown
 ---
 cred_proxy:
  routes:
    - path: /anthropic/
      upstream: https://api.anthropic.com
      auth_scheme: Bearer
      token_ref: CLAUDE_BOTTLE_OAUTH_TOKEN
      role: anthropic-base-url
    - path: /gitea/dideric/
      upstream: https://gitea.dideric.is
      auth_scheme: token
      token_ref: GITEA_TOKEN
      role: [git-insteadof, tea-login]
 git:
  - Name: claude-bottle
    Upstream: ssh://git@gitea.dideric.is:30009/didericis/claude-bottle.git
    IdentityFile: ~/.ssh/gitea-delos-2.pem
    ExtraHosts:
      gitea.dideric.is: 100.78.141.42
    KnownHostKey: ssh-rsa AAAAB3...
 egress:
  allowlist:
    - example.com
 ---
 The `dev` bottle. Backs my work on personal projects:
 - Anthropic OAuth via cred-proxy
 - gitea.dideric.is over SSH (with PAT for tea API)
 - example.com in the egress allowlist
 ```
 ### Example agent file
 ```markdown
 ---
 name: implementer
 description: Implements features against PRDs in this repo.
 model: opus
 bottle: dev
 skills:
  - init-prd
 ---
 You are a feature-implementation agent running inside an
 ephemeral claude-bottle sandbox...
 ```
 Drop the same file into `~/.claude/agents/implementer.md` and
 Claude Code picks it up as a subagent (assuming Claude Code
 tolerates the `bottle:` and `skills:` fields — see Open
 Questions).
 ### YAML subset grammar
 ```
 document   := frontmatter? body?
 frontmatter := "---" "\n" yaml_block "---" "\n"
 yaml_block := (line "\n")*
 line       := blank | comment | mapping_line | list_item
 mapping_line := indent key ":" (" " value)?
 key        := bare_string  ; matches [A-Za-z_][A-Za-z0-9_-]*
 value      := scalar | inline_list | inline_dict
 scalar     := number | bool | null | quoted_string | bare_string
 list_item  := indent "-" " " value
 ```
 Notable rejections (each dies with a specific error):
 - Anchors (`&name`), aliases (`*name`).
 - Multi-line block scalars (`|`, `>`, `|-`, `>+`).
 - YAML tags (`!!str`, etc.).
 - `yes`/`no`/`on`/`off`/`Y`/`N` as booleans (we require
  literal `true` / `false`).
 - Unquoted strings that resemble dates (`2026-05-24`) or octal
  (`0123`) — the Norway problem and its kin. If a string would
  be ambiguous, quote it.
 - Flow style mappings nested more than one level deep.
 Parser lives at `claude_bottle/yaml_subset.py`, ~300 lines.
 Public API:
 ```python
 def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
    """Return (frontmatter_dict, body_text). The dict's values are
    str / int / bool / None / list / dict only; nesting capped at
    two levels."""
 ```
 ### Existing code touched
 - **`claude_bottle/manifest.py`** — `Manifest.resolve` rewritten
  to walk the new directories. `Manifest.from_json_obj` kept as
  a programmatic entry point (used by tests). New
  `Manifest.from_md_dirs(home_dir, cwd_dir)` for the loader.
 - **`claude_bottle/yaml_subset.py`** — new. The parser.
 - **`claude_bottle/cli/migrate_manifest.py`** — new. The
  migration command.
 - **`claude_bottle/cli/__init__.py`** — wire the new subcommand.
 - **`README.md`** — manifest section rewritten against the new
  layout.
 - **`claude-bottle.example.json`** — removed; replaced by an
  `examples/` directory with one bottle file + one agent file.
 - **Tests** — new parser tests + new loader tests; existing
  manifest tests adapt to either build via `from_json_obj`
  (still supported) or use the new directory layout.
 ### Data model
 No new dataclasses. `Bottle`, `Agent`, `Manifest`, `CredProxyRoute`,
 etc. all stay the same shape. Only the loader changes.
 ### Backward compatibility
 This is a breaking change for v1 users. Mitigations:
 - `./cli.py migrate-manifest` does the heavy lifting in one
  command.
 - If `claude-bottle.json` exists in `$HOME` or `$CWD` *and* the
  new directories don't exist, the resolver dies with a clear
  pointer at the migration command — not silently merging
  formats, not silently dropping the JSON content.
 ## Open questions
 - **Claude Code tolerance for extra frontmatter fields.** Test
  empirically before settling: drop a file with `bottle: dev`
  in `~/.claude/agents/` and see whether Claude Code warns,
  ignores, or breaks. If it warns, namespace the field
  (`claude-bottle-bottle:` or a nested `claude_bottle:` block).
 - **Hidden directory vs visible.** Default `.claude-bottle/`
  (hidden — matches `.config/`, `.ssh/`, `.docker/`). If users
  routinely want to navigate to it from the file manager,
  switch to `claude-bottle/`. Lean hidden.
 - **`description:` for bottles.** Should bottle frontmatter
  carry a `description:` field for the y/N preflight? Default
  no — bottle names are kebab-case and self-describing, and
  the MD body is the place for human prose.
 - **Filename ↔ frontmatter `name:` drift.** If both are
  present and disagree, warn (we use the filename as the
  authoritative key). Same for agents.
 - **`include` / glob for shared egress allowlists.** A common
  pattern will be "every bottle allows api.anthropic.com and
  github.com"; do we want a way to share the list? Default no
  for v1; revisit if it bites.
 - **Migration tool destructive vs additive.** Default
  additive (writes new files, leaves old JSON in place). If
  users find the half-migrated state confusing, switch to
  printing a "delete claude-bottle.json now" reminder at the
  end of the migration.
 ## References
 - `docs/research/manifest-format-and-grouping.md` — the
  analysis this PRD follows from.
 - Closed PR #15 — the resolver-layer trust-boundary attempt;
  superseded by this PRD's filesystem-layout approach.
 - Closed PR #16 — the research doc + the option-B4 decision
  comment that picked this design.
 - Claude Code subagent spec — `~/.claude/agents/<name>.md`
  with YAML frontmatter (existing convention this PRD aligns
  agent files with).