feat: upload_file + prepare_inputs + build_inputs route (parity with @pipelex/sdk)

CHANGELOG.md

@@ -4,6 +4,8 @@
 
 ### Added
 
+- **Input preparation: `upload_file` and `prepare_inputs` (hosted upload capability).** The Python counterpart of `@pipelex/sdk`'s `uploadFile` / `prepareInputs`, in parity. `client.upload_file(source, *, filename=None, content_type=None)` uploads one local asset — a filesystem path (`str`/`Path`) or raw `bytes` — and returns an `UploadRecord` (`uri`, `content_type`, `size`, `filename`) assembled client-side. `client.prepare_inputs(*, files, pipe_ref=None, inputs)` resolves the target pipe's declared signature from the explicit inputs template, interprets the caller's compact `inputs` top-down against it (the file signal is the canonical Image/Document content shape — a `{"url": …}` dict — mirroring the runtime's `input_normalizer`), uploads the file-bearing values, and returns `PreparedInputs`: a copy-on-write rewrite of `inputs` with each asset reference replaced by canonical content carrying `pipelex-storage://` in `url`, plus one `UploadRecord` per prepared asset. HTTP(S) URLs and existing `pipelex-storage://` URIs pass through unchanged; data URLs and local/byte sources are uploaded; the same source referenced twice uploads once (dedup by source identity); all failures are raised before any run is created. Failures are typed per category: `InvalidLocalSourceError`, `RejectedAssetError`, `UnsupportedUploadCapabilityError`, `UploadAuthenticationError`, `UploadTransportError` (all extend `InputPreparationError`). `prepare_inputs` takes the method closure as inline `files`; catalog `method_id` resolution and opt-in `http(s)` ingest are deferred and additive. See [`docs/input-preparation.md`](./docs/input-preparation.md).
+- **`build_inputs` route (`POST /v1/build/inputs`).** Closes the `/v1/build/*` parity gap the Python SDK had — `client.build_inputs(BuildInputsRequest)` projects a pipe's declared inputs as a fill-in template, returning a 200 verdict discriminated on `is_valid` (`BuildInputsValidReport` | `CrateInvalidReport`); a no-verdict condition throws `ApiResponseError`. It is the signature source `prepare_inputs` reads (with `explicit=True`). Models live in `pipelex_sdk/build_models.py`.
 - **Typed run usage: `RunResults.tokens_usages` + `RunResults.usage_assembly_error`.** The per-call usage records a run produces — token counts by category, the server-computed `cost` in USD, model name and id, the pipe that made the call, job-kind fields and timing, for LLM and img-gen/extract/search calls alike — are now first-class typed fields instead of riding `model_extra`. Records validate into a new `TokensUsageRecord` model (`pipelex_sdk/runs.py`) mirroring the wire contract specified in the MTHDS protocol spec. Both paths populate the pair: the hosted durable path reads it off `GET /v1/runs/{id}/results` (which unpacks the runner's `tokens_usages.json` artifact), and the blocking fallback lifts the same pair out of the execute response's extension-open `pipe_output` — so `result.tokens_usages` reads the same regardless of which path ran.
 
   Note that the rate table (`unit_costs`) no longer crosses the wire: a record now carries the computed `cost` for the call instead, which is `None` when the model has no rate table at all (own-GPU, mock, dry run) and `0` when a rate table priced it at zero. There is no run-level aggregate — sum the records.

docs/architecture.md

@@ -138,7 +138,7 @@ The wire models are snake_case Pydantic v2. Response models are extension-open (
 - **Pipelex API keys** — `list_pipelex_api_keys()`; `create_pipelex_api_key(label)` and `rotate_pipelex_api_key(id)` return the plaintext `api_key` **once**; `revoke_pipelex_api_key(id)`. Creation surfaces a **409 `pipelex_api_key_limit_reached`** when the per-account limit is hit. Rotation sends no body.
 - **Gateway (LLM inference) key** — `create_gateway_api_key(promo_code)` **always sends a JSON body** (even with `promo_code=None` → `{"promo_code": null}`); the server 422s an empty body. `get_gateway_api_key()` → status (`gateway_api_key` is `None` until provisioned).
 - **Onboarding** — `submit_onboarding(OnboardingSubmission)` (`POST /v1/onboarding/submit`, empty 2xx body); absent optional fields are dropped.
-- **Storage** — `resolve_storage_url(uri)` → presigned URL; `upload(UploadInput)` → the stored file handle.
+- **Storage** — `resolve_storage_url(uri)` → presigned URL; `upload(UploadInput)` → the stored file handle. The higher-level `upload_file` / `prepare_inputs` preparation surface built on top of `upload` is now available — see [input-preparation.md](./input-preparation.md).
 - **Run records** — `list_runs(method_id)` → `list[PipelineRun]` (the catalog-style list, distinct from the lifecycle status/result routes); `update_run(run_id, UpdateRunInput)` (admin/manual status patch, empty 2xx body).
 
 ## Health probe

docs/input-preparation.md

@@ -0,0 +1,101 @@
+# Input preparation (`upload_file` / `prepare_inputs`)
+
+> **Status: implemented** (`pipelex_sdk/upload.py`, `pipelex_sdk/prepare_inputs.py`, `pipelex_sdk/build_models.py`). This document records the contract (design source: `wip/upload/README.md` in the workspace, tracked in `TODOS.md`). `upload_file` and `prepare_inputs` are the Python counterpart of `@pipelex/sdk`'s `uploadFile` / `prepareInputs`, built on the raw `upload()` wire call. This work also added the `build_inputs` route (the signature source), which the Python SDK previously lacked.
+>
+> **Current scope.** `prepare_inputs` takes the method closure as inline `files` (the signature source). Two pieces are deliberately deferred and additive (they do not change this contract): resolving a closure from a catalog `method_id`, and the opt-in ingest of `http(s)` URLs into storage — for now an `http(s)` URL at a file position always passes through unchanged. Kept in parity with `@pipelex/sdk`.
+
+## Why this exists
+
+A hosted run cannot see the caller's filesystem. Turning caller-local assets into run-ready inputs is therefore the SDK's job, not the runner's — the SDK process is the only component that can read the local file or hold the bytes. Today that work is re-implemented by every consumer (read file → base64 → `POST /v1/upload` → rewrite the input to the returned URI); `fenix-pipelex` is an early real-world example. `prepare_inputs` makes it one reusable, explicit operation.
+
+Preparation is **explicit and separate from running.** `execute` / `start` never silently upload local files. The payoff: file-access errors happen *before* a run exists, prepared inputs are inspectable and reusable across a model sweep or retries without re-uploading, and `start` keeps a deterministic JSON-input contract.
+
+## Parity note
+
+This is the Python side of one cross-language contract. The behavior matrix, pass-through rules, `Dynamic` handling, dedup, and failure categories are identical to the JS SDK; only the accepted source types differ per language. The two SDKs must agree semantically. See the JS counterpart's `docs/input-preparation.md` for the mirror.
+
+The Python SDK previously had **no `/v1/build/*` coverage** — this change added the `build_inputs` counterpart `prepare_inputs` needs to resolve the declared signature (the JS SDK already exposes `buildInputs`).
+
+## The two operations
+
+### `upload_file` — single-asset convenience
+
+Uploads one asset and returns its upload record. It is the language-native convenience over the raw `upload()` wire call (base64 JSON body), assembling the record client-side.
+
+- **Accepted sources:** `str` and `pathlib.Path` filesystem paths, and raw `bytes`. `upload_file` treats every string as a **filesystem path** — a URL handed directly to it is read as a local file and fails.
+- The URL / storage-URI **pass-through** (leaving **HTTP(S) URLs** and existing **`pipelex-storage://` URIs** untouched) is a `prepare_inputs`-level behavior (see pass-through rules below), not a feature of `upload_file` itself: `prepare_inputs` classifies each string (URL vs local path) against the declared signature *before* it reaches `upload_file`.
+- Open file objects and streams are **deferred** — they can be added later without removing anything.
+
+The returned **upload record** guarantees, beyond the source identity:
+
+| Field | Guarantee |
+| --- | --- |
+| `uri` | The `pipelex-storage://` reference for the uploaded asset. |
+| content type (MIME) | Known client-side at upload time. |
+| size (bytes) | Known client-side at upload time. |
+| filename | Already in the wire model. |
+
+The MIME type and size are known client-side, so the record is assembled without extending the `/v1/upload` response. There is deliberately **no checksum field**: within-preparation dedup keys on source identity (not hashing), and cross-preparation dedup is a hosted storage-policy concern (Phase 5).
+
+### `prepare_inputs` — signature-driven input preparation
+
+```
+prepare_inputs(method_ref, pipe, inputs) → PreparedInputs
+```
+
+Takes the **method reference** (bundle files or catalog `method_id`) plus the target **pipe**, resolves the pipe's declared input signature, interprets the caller's compact `inputs` top-down against that signature, uploads the file-bearing values, and returns `PreparedInputs`:
+
+- `inputs` — a **copy** of the caller's inputs with each asset reference replaced by the canonical content shape carrying `pipelex-storage://` in its `url` field (see "Rewritten-input shape" below). Copy-on-write: the caller's original object is never mutated.
+- `uploads` — one upload record per prepared asset (the `upload_file` record shape), exposing `uri` so callers can log which source became which reference without reverse-engineering the rewritten object.
+
+The prepared `inputs` are passed to the existing run lifecycle unchanged.
+
+## Signature-driven asset identification
+
+The SDK **must not** guess that every string resembling a path is an asset — that would make ordinary text inputs environment-dependent and could upload unintended files. Interpretation comes from the method's **declared signature**, never from a value's shape alone. This mirrors the runtime's own top-down interpretation (`pipelex/pipelex/core/memory/input_shaper.py`, `InputShaper`) combined with the file-reference resolution of `pipelex/pipelex/pipeline/input_normalizer.py`, so local and hosted execution read the same compact inputs the same way.
+
+The declared signature is resolved via the explicit inputs template (`build_inputs` with `explicit=True`), which carries concept identity, canonical content shape, and multiplicity per input.
+
+Interpretation per declared input:
+
+- A bare string, `Path`, or `bytes` value at an **Image/Document-declared** input is a **file reference**: local paths, data URLs, and bytes are uploaded and rewritten to `pipelex-storage://` URIs; HTTP(S) URLs and existing `pipelex-storage://` URIs pass through unchanged.
+- The **identical** bare string at a **Text-declared** input is text and is never touched.
+- **Canonical image/document content structures** are recognized by their URL-bearing fields wherever they appear, including nested in structured objects and lists — exactly as the runtime normalizer walks them. The refining case matters: a concept refining `Image`/`PDF` is classified by the **canonical content shape**, not by the concept ref alone.
+- Inputs declared **`Dynamic`** are not path-interpreted (the signature genuinely cannot guide them); they accept canonical content structures or already-prepared references only.
+- A repeated reference to the **same source** within one preparation is uploaded once and rewritten consistently (within-preparation dedup by source identity).
+
+### Pass-through rules
+
+| Source at a file-bearing input | Action |
+| --- | --- |
+| Local path (`str`/`Path`) / data URL / `bytes` | Upload → rewrite to `pipelex-storage://` |
+| Existing `pipelex-storage://` URI | Already prepared — pass through unchanged |
+| HTTP(S) URL | Pass through unchanged, **unless** the caller explicitly asks to ingest it into Pipelex storage |
+
+## Rewritten-input shape: `url` carries the URI
+
+The runtime's canonical image/document content stores its reference in a **`url`** field. Preparation emits inputs the runtime interprets natively, so a rewritten input keeps the canonical content shape with `url` holding the `pipelex-storage://` value — exactly what the runtime's `input_normalizer` writes.
+
+The "uploaded reference is named `uri`" decision applies to the **upload surface**: the raw upload result and each upload record expose the storage reference as `uri`. Preparation must **not** invent a `uri` field inside rewritten inputs — that would produce inputs the runtime does not recognize.
+
+## Error and capability behavior
+
+Upload is a **hosted Pipelex-product capability**, even though the SDK can be pointed at other base URLs. A deployment that does not support upload must raise a specific, actionable exception — preparation must never silently leave a local path in place and let a later run fail obscurely.
+
+The contract distinguishes at least these semantic outcomes (exact typed exception classes are settled during implementation):
+
+- **invalid local source** — missing or unreadable path;
+- **rejected asset** — the server refused it (e.g. a `413` past the service-defined size cap — see "Storage policy" — surfaced as a clear rejection, not a raw transport error);
+- **unsupported server capability** — the configured deployment has no upload route;
+- **authentication / authorization failure** — `401` / `403`;
+- **transport failure** — network / server fault.
+
+All preparation failures are raised **before any run is created**.
+
+## Storage policy (inherited, Phase 1)
+
+The SDK ships against **today's route behavior**: a service-defined size cap (hosted default 50 MiB via `MAX_UPLOAD_MIB`, rejected with `413`), auth required, per-user keys, and nothing else — no MIME validation, retention, quotas, dedup, or cleanup. The SDK documents limits as **service-defined** and surfaces server rejections as clear "rejected asset" errors; it does **not** hardcode a client-side cap. Real storage policy (retention, quotas, org scoping, cleanup) is a later hosted-owner deliverable.
+
+## Stability across the future endpoint move
+
+The public abstraction sits deliberately **above** the HTTP route. Callers depend on `upload_file` / `prepare_inputs`, the `uri` result field, and the `pipelex-storage://` scheme — never on which backend service owns the route. The current transport is `POST /v1/upload` on `pipelex-api`; when hosted storage upload later moves to `pipelex-platform` (together with its paired resolution route, as one storage domain), the public path and wire shape are kept compatible so released SDK versions keep working, and any wire-protocol change is absorbed inside the SDK's upload transport. `upload_file`, `prepare_inputs`, and the prepared run-input shape stay stable across that move.

pipelex_sdk/build_models.py

@@ -0,0 +1,99 @@
+"""Wire models for the `/v1/build/inputs` route — the signature source `prepare_inputs`
+reads to resolve a pipe's declared inputs.
+
+The Python SDK had no `/v1/build/*` coverage; `prepare_inputs` needs the explicit
+inputs template, so this adds the `build_inputs` counterpart of `pipelex-sdk-js`'s
+`buildInputs` (only this route — the other build projections are not needed here).
+A produced verdict is a `200` discriminated on `is_valid`; a no-verdict condition
+(unknown `pipe_ref`, auth, server fault) throws `ApiResponseError`.
+"""
+
+from __future__ import annotations
+
+from typing import Annotated, Any, Literal, Self, TypeAlias
+
+from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, model_validator
+
+from pipelex_sdk.validation_models import ValidationErrorItem
+
+InputsTemplateFormat = Literal["json", "toml"]
+
+
+class MthdsFileItem(BaseModel):
+    """One MTHDS file in a build closure. `source` is an optional provenance label the
+    server threads onto diagnostics raised from this file.
+    """
+
+    content: str
+    source: str | None = None
+
+
+class BuildInputsRequest(BaseModel):
+    """Request for `POST /v1/build/inputs`. The closure is supplied as inline `files`."""
+
+    files: list[MthdsFileItem]
+    pipe_ref: str | None = None
+    format: InputsTemplateFormat = "json"
+    explicit: bool = False
+
+
+class BuildInputsValidReport(BaseModel):
+    """The `is_valid: true` arm. The template rides `inputs` (json) or `inputs_toml` (toml)."""
+
+    model_config = ConfigDict(extra="allow")
+
+    is_valid: Literal[True]
+    pipe_ref: str
+    requested_pipe_ref: str | None = None
+    message: str
+    format: InputsTemplateFormat
+    explicit: bool
+    inputs: dict[str, Any] | None = None
+    inputs_toml: str | None = None
+
+    @model_validator(mode="after")
+    def _template_matches_format(self) -> Self:
+        # Honor the adapter's malformed-200 guarantee for the template shape too: a valid
+        # verdict must carry the template field its `format` selects (and not the other).
+        # Without this, an `is_valid: true` body missing both templates would parse as a valid
+        # report and only fail one layer down in `prepare_inputs`.
+        match self.format:
+            case "json":
+                if self.inputs is None:
+                    msg = "inputs is required when format is 'json'"
+                    raise ValueError(msg)
+                if self.inputs_toml is not None:
+                    msg = "inputs_toml must be absent when format is 'json'"
+                    raise ValueError(msg)
+            case "toml":
+                if self.inputs_toml is None:
+                    msg = "inputs_toml is required when format is 'toml'"
+                    raise ValueError(msg)
+                if self.inputs is not None:
+                    msg = "inputs must be absent when format is 'toml'"
+                    raise ValueError(msg)
+        return self
+
+
+class CrateInvalidReport(BaseModel):
+    """The `is_valid: false` arm shared by the build routes — an unresolvable closure is a
+    produced verdict on a `200`, never a thrown error. Branch on `is_valid`, not transport.
+    """
+
+    model_config = ConfigDict(extra="allow")
+
+    is_valid: Literal[False]
+    validation_errors: list[ValidationErrorItem]
+    message: str
+
+
+BuildInputsResponse: TypeAlias = Annotated[
+    BuildInputsValidReport | CrateInvalidReport,
+    Field(discriminator="is_valid"),
+]
+
+# The single parse path for a 200 `/build/inputs` body — discriminated on `is_valid`, built once at
+# import (TypeAdapter construction is expensive), mirroring `PipelexValidationResultAdapter`. A
+# malformed 200 (or an empty body) raises a clean `pydantic.ValidationError` rather than being
+# mistaken for a valid verdict.
+BuildInputsResponseAdapter: TypeAdapter[BuildInputsResponse] = TypeAdapter(BuildInputsResponse)  # pylint: disable=invalid-name