bot-bottle/docs/prds/prd-new-user-provider-plugins.md

# PRD prd-new: User-defined agent provider plugins

- **Status:** Draft
- **Author:** claude
- **Created:** 2026-06-04

## Summary

The `get_provider()` registry in `bot_bottle/agent_provider.py` is a closed list —
only `"claude"` and `"codex"` are valid templates, validated at manifest-load time and
again at launch. Users who want to run a different agent (Gemini, Aider, a custom
local model wrapper) cannot add a provider without forking the package.

This PRD opens the registry to user-defined plugins. A plugin placed at
`~/.bot-bottle/contrib/<name>/` is discovered and loaded at launch time. The manifest
accepts any non-empty template string that names a built-in or resolves to a user
plugin at that path.

Alongside discovery, this PRD moves CA and git provisioning out of the Docker backend
and into the `AgentProvider` ABC as overridable methods. The current standalone
`provision/ca.py` and `provision/git.py` files in the Docker backend are deleted;
their logic becomes the default implementations on the ABC. This lets exotic provider
images (different base OS, different user, non-standard trust mechanism) override
provisioning freely without the abstraction fighting them.

The preceding commit on this PR moves `codex_auth.py` from `bot_bottle/` into
`bot_bottle/contrib/codex/` — a clean-up that fits naturally here since this PR
also clarifies that `contrib/` is the per-provider home.

## Problem

Users building unconventional setups hit a hard wall: the template validation in
`manifest_agent.AgentProvider.from_dict` rejects any string not in `PROVIDER_TEMPLATES`.
There is no escape hatch short of editing bot-bottle's source.

PRD 0050 moved provider logic into `contrib/` specifically so a third provider would
be "cheap to add" — but "cheap" today still means a pull request against the bot-bottle
repo, not a drop-in file in the user's home directory. The filesystem layout is already
the right shape; the discovery step is missing.

Beyond discovery, the Docker backend's `provision_ca` and `provision_git` functions
bake in Debian-specific commands (`update-ca-certificates`) and a hardcoded container
user (`node`). A user plugin that runs as a different user, or on a different base OS,
silently gets the wrong provisioning with no way to correct it short of forking.

## Goals / Success Criteria

1. A user places `~/.bot-bottle/contrib/<name>/agent_provider.py` — a file that exports
   a class inheriting `AgentProvider` — sets `agent_provider.template: <name>` in a
   bottle's frontmatter, and launches a bottle using that provider with no changes to
   the bot-bottle source.
2. The plugin directory may also contain a `Dockerfile` at
   `~/.bot-bottle/contrib/<name>/Dockerfile`; the existing three-tier Dockerfile cascade
   (per-bottle override → manifest `dockerfile:` field → provider default) uses this
   path as the provider default for user plugins.
3. The manifest validator accepts any non-empty template string. Unknown templates that
   resolve to no user plugin still raise a clear error, but at launch (via `get_provider`)
   rather than at manifest-load time.
4. Built-in provider knobs (`auth_token` → claude only; `forward_host_credentials` →
   codex only) are guarded to built-in template names. Bottles using a user provider
   may set neither knob.
5. `get_provider(template)` checks `~/.bot-bottle/contrib/<template>/agent_provider.py`
   before the built-ins, so a user can shadow a built-in for local testing.
6. A clear `ValueError` is raised if the user plugin file exists but contains no
   `AgentProvider` subclass.
7. `AgentProvider` gains `provision_ca(self, bottle, plan)` and
   `provision_git(self, bottle, plan)` with default implementations that reproduce
   current Docker/Debian/node behavior. Built-in providers inherit the defaults
   unchanged. User plugins override either method when their image diverges.
8. `bot_bottle/backend/docker/provision/ca.py` and
   `bot_bottle/backend/docker/provision/git.py` are deleted. The Docker backend base
   class calls `provider.provision_ca(bottle, plan)` and
   `provider.provision_git(bottle, plan)` directly.

## Non-goals

- Packaging or distributing user plugins as installable Python packages.
- A plugin registry, index, or discovery beyond the filesystem path convention.
- Adding a third built-in provider.
- Validating that user plugin images, Dockerfiles, or commands exist before launch
  (same policy as built-ins).
- Sandboxing user plugin code — plugins run with full Python interpreter access.
- Per-provider opt-out of the egress sidecar or network provisioning (follow-on).

## Scope

### In scope

- `get_provider(template: str) -> AgentProvider` gains a `_load_user_plugin(template)`
  step that checks `~/.bot-bottle/contrib/<template>/agent_provider.py` first, then
  falls through to the built-in look-ups.
- `_load_user_plugin` uses `importlib.util.spec_from_file_location` to load the module
  and returns the first `AgentProvider` subclass found in its `__dict__`. Raises
  `ValueError` if the file exists but exports no subclass.
- The Dockerfile cascade in the Docker backend's `resolve_plan()` uses
  `~/.bot-bottle/contrib/<template>/Dockerfile` as the provider default for user
  plugins (the same slot currently occupied by `Dockerfile.claude` / `Dockerfile.codex`
  for built-ins).
- `manifest_agent.AgentProvider.from_dict`: the `template not in PROVIDER_TEMPLATES`
  check is removed; the two built-in-specific knob guards (`auth_token` → claude,
  `forward_host_credentials` → codex) are tightened to `template in PROVIDER_TEMPLATES`
  so they are skipped for user-defined names.
- `PROVIDER_TEMPLATES` remains in `agent_provider.py` as the set of built-in names for
  use by tests and any enumeration callers.
- `AgentProvider` ABC gains:
  ```python
  def provision_ca(self, bottle: Bottle, plan: BottlePlan) -> None: ...
  def provision_git(self, bottle: Bottle, plan: BottlePlan) -> None: ...
  ```
  Default implementations reproduce the current `provision/ca.py` and
  `provision/git.py` logic exactly (Debian `update-ca-certificates`, `node` user,
  `/home/node` home).
- `bot_bottle/backend/docker/provision/ca.py` and
  `bot_bottle/backend/docker/provision/git.py` deleted. The Docker backend base
  class substitutes direct calls to the provider methods.
- Unit tests for the discovery path:
  - Plugin found and loaded → correct `AgentProvider` instance returned.
  - Plugin file exists but exports no subclass → `ValueError`.
  - Unknown template with no user plugin → `ValueError` from `get_provider`.
  - Built-in template name still works normally even when no user plugin exists.
- Unit tests for the provisioning delegation:
  - A provider subclass that overrides `provision_ca` has its override called.
  - A provider subclass that overrides `provision_git` has its override called.
- One paragraph added to `README.md` under a new "Custom providers" section describing
  the `~/.bot-bottle/contrib/<name>/` convention (both `agent_provider.py` and
  `Dockerfile`), the `provision_ca` / `provision_git` override points, and pointing at
  the existing contrib providers as reference implementations.

### Out of scope

- Hot-reloading plugins during a running session.
- Plugin versioning or dependency declaration.
- Changes to the smolmachines backend provisioning path.

## Proposed design

### Discovery in `get_provider`

```python
import importlib.util

def get_provider(template: str) -> AgentProvider:
    user_plugin = _load_user_plugin(template)
    if user_plugin is not None:
        return user_plugin
    if template == PROVIDER_CLAUDE:
        from .contrib.claude.agent_provider import ClaudeAgentProvider
        return ClaudeAgentProvider()
    if template == PROVIDER_CODEX:
        from .contrib.codex.agent_provider import CodexAgentProvider
        return CodexAgentProvider()
    raise ValueError(f"unknown agent provider template: {template!r}")


def _load_user_plugin(template: str) -> AgentProvider | None:
    plugin_path = (
        Path.home() / ".bot-bottle" / "contrib" / template / "agent_provider.py"
    )
    if not plugin_path.exists():
        return None
    spec = importlib.util.spec_from_file_location(
        f"_user_contrib_{template}.agent_provider", plugin_path
    )
    if spec is None or spec.loader is None:
        raise ValueError(f"user plugin at {plugin_path} could not be loaded")
    mod = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(mod)  # type: ignore[union-attr]
    for obj in vars(mod).values():
        if (
            isinstance(obj, type)
            and issubclass(obj, AgentProvider)
            and obj is not AgentProvider
        ):
            return obj()
    raise ValueError(
        f"user plugin at {plugin_path} defines no AgentProvider subclass"
    )
```

### Dockerfile convention for user plugins

`resolve_plan()` in the Docker backend already has a three-tier cascade. For user
plugins the provider-default slot is filled by:

```python
Path.home() / ".bot-bottle" / "contrib" / template / "Dockerfile"
```

Per-bottle overrides and manifest `dockerfile:` fields continue to take precedence.

### Provisioning methods on `AgentProvider`

```python
class AgentProvider(ABC):
    ...
    def provision_ca(self, bottle: Bottle, plan: BottlePlan) -> None:
        """Install the egress MITM CA into the agent container's trust store.
        Override for non-Debian base images or non-standard trust mechanisms."""
        cert_host_path, label = select_ca_cert(plan.egress_plan)
        bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
        bottle.exec(
            f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
            user="root",
        )
        log_ca_fingerprint(cert_host_path, label)

    def provision_git(self, bottle: Bottle, plan: BottlePlan) -> None:
        """Configure git inside the agent container.
        Override for images that run as a different user or use a non-standard home."""
        _provision_cwd_git(plan, bottle)
        _provision_git_gate_config(plan, bottle)
        _provision_git_user(plan, bottle)
```

The Docker backend base class replaces the direct calls to the old standalone
functions with:

```python
provider.provision_ca(bottle, plan)
provider.provision_git(bottle, plan)
```

### Manifest validation change

In `manifest_agent.AgentProvider.from_dict`, remove the hard rejection:

```python
# Before
if template not in PROVIDER_TEMPLATES:
    raise ManifestError(
        f"bottle '{bottle_name}' agent_provider.template {template!r} "
        f"is not one of {', '.join(sorted(PROVIDER_TEMPLATES))}"
    )

# After — removed entirely; get_provider() raises at launch for unknown names
```

Guard the built-in knob checks with `template in PROVIDER_TEMPLATES`:

```python
if auth_token and template == "claude":   # unchanged
    ...
if auth_token and template not in PROVIDER_TEMPLATES:
    raise ManifestError(
        f"bottle '{bottle_name}' agent_provider.auth_token is only "
        f"supported for built-in templates ({', '.join(sorted(PROVIDER_TEMPLATES))})"
    )
if forward_host_credentials and template == "codex":  # unchanged
    ...
if forward_host_credentials and template not in PROVIDER_TEMPLATES:
    raise ManifestError(
        f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
        f"is only supported for built-in templates"
    )
```

## Open questions

1. **`BOT_BOTTLE_CONTRIB_DIR` env var.** Omitted for now — `~/.bot-bottle/contrib/`
   is consistent with the rest of the user config layout. Revisit if the need surfaces.

## References

- PRD 0050 — agent provider contrib (established `contrib/` as the per-provider home)
- PRD 0048 — SSH deploy key provisioning (the `contrib/` convention)
- `bot_bottle/agent_provider.py` — `get_provider`, `PROVIDER_TEMPLATES`, `AgentProvider` ABC
- `bot_bottle/manifest_agent.py` — template validation that this PRD relaxes
- `bot_bottle/backend/docker/provision/ca.py` — current CA provisioner (to be deleted)
- `bot_bottle/backend/docker/provision/git.py` — current git provisioner (to be deleted)