feat: upload_file + prepare_inputs + build_inputs route (parity with @pipelex/sdk)

[lchoquel]

Summary

Adds the hosted upload-capability surface — the Python counterpart of @pipelex/sdk's uploadFile/prepareInputs — plus the build_inputs route the signature step needs.

• client.upload_file(source, *, filename=None, content_type=None) -> UploadRecord — accepts str/Path paths and raw bytes; record (uri, content_type, size, filename) assembled client-side (mimetypes for MIME).
• client.prepare_inputs(*, files, pipe_ref=None, inputs) -> PreparedInputs — resolves the pipe's declared signature via build_inputs(explicit=True), template-guided walk (file signal = a canonical {url} content dict, mirroring the runtime's input_normalizer), uploads file-bearing values, and returns a copy-on-write rewrite carrying pipelex-storage:// in url plus one UploadRecord per prepared asset. http(s)/pipelex-storage:// pass through; data URLs and local/byte sources upload; dedup by source identity; all failures surface before any run. An unrecognized value at a file position fails as a typed InputPreparationError.
• client.build_inputs(BuildInputsRequest) -> BuildInputsResponse — closes the /v1/build/* parity gap. The response is an is_valid-discriminated union parsed through BuildInputsResponseAdapter (a TypeAdapter), mirroring the repo's PipelexValidationResultAdapter precedent, so a malformed 200 raises a clean ValidationError rather than being read as a valid verdict.
• Typed exception family mirrors the JS SDK: InputPreparationError base + InvalidLocalSource / RejectedAsset / UnsupportedUploadCapability / UploadAuthentication / UploadTransport.

Scope

prepare_inputs takes the closure as inline files; catalog method_id and opt-in http(s) ingest are deferred (additive).

Testing

Matrix-derived parity tests + real-client wiring tests. make agent-check + make agent-test green.

🤖 Generated with Claude Code

---

Summary by cubic

Adds hosted input preparation to the Python SDK with client.upload_file, client.prepare_inputs, and client.build_inputs, matching @pipelex/sdk. Also hardens data: URL handling and offloads local file reads in upload_file to avoid blocking the event loop.

• New Features

  • client.upload_file(source, filename?, content_type?) — accepts str/Path/bytes; returns UploadRecord with uri, filename, content_type, size.
  • client.prepare_inputs(files, pipe_ref?, inputs) — resolves the signature via build_inputs(explicit=True), walks the template, uploads file-bearing values, and rewrites to canonical content with url set to pipelex-storage://. HTTP(S) and existing storage URIs pass through; data: URLs and local/byte sources upload; dedup by source; copy-on-write; all failures happen before any run.
  • client.build_inputs(request) — adds /v1/build/inputs parity; returns an is_valid-discriminated result; malformed 200s raise a ValidationError.
  • Typed errors: InvalidLocalSourceError, RejectedAssetError, UnsupportedUploadCapabilityError, UploadAuthenticationError, UploadTransportError (all extend InputPreparationError).

• Bug Fixes

  • data: URL decoding: base64 payloads are validated (validate=True); malformed values raise InputPreparationError and never upload corrupted bytes.
  • Percent-encoded binary data: URLs preserve exact bytes via unquote_to_bytes; docs clarified that URL/pass-through classification happens in prepare_inputs (strings to upload_file are treated as paths).
  • upload_file reads local files via asyncio.to_thread so large reads do not block the event loop.
  • build_inputs valid reports now enforce that the template field matches format (inputs for json, inputs_toml for toml); missing/mismatched templates are treated as malformed 200s and raise ValidationError.

^{Written for commit e292fc3. Summary will update on new commits.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fe766d63a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[greptile-apps]

Greptile Summary

This PR adds hosted file preparation to the Python SDK. The main changes are:

• A typed build_inputs route and response union.
• Local file and byte uploads with semantic error types.
• Signature-guided input rewriting and upload deduplication.
• Documentation and unit tests for the new API.

Confidence Score: 4/5

Malformed data URLs can bypass the typed preparation error contract, and large uploads can stall the event loop.

Incorrect base64 padding exposes a raw decoder exception. File reads and base64 encoding run synchronously in the async upload path. Route parsing, input rewriting, and HTTP error mapping otherwise have focused coverage.

pipelex_sdk/prepare_inputs.py and pipelex_sdk/upload.py

T-Rex Logs

What T-Rex did

• Reproduced the malformed Base64 typed-error contract by running a focused executable pytest harness through the real prepare_inputs path with a mocked build-inputs response that declares a file-bearing photo input.
• Observed that the InputPreparationError assertion failed due to binascii.Error: Incorrect padding escaped from line 82 before upload.
• Compared baseline and after-state logs to confirm PR behavior; the before log shows the parent revision and the after log captures mocked requests, statuses, discriminated report types, and rewritten prepared inputs.
• Verified that the focused tests ran with the exact requested command and exited with code 0, and that the verbose real-client wiring path was exercised as described.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
pipelex_sdk/build_models.py	Adds request models and a discriminated response union for the build-inputs route.
pipelex_sdk/client.py	Adds client methods for input projection, file upload, and input preparation.
pipelex_sdk/prepare_inputs.py	Adds template-guided rewriting and deduplication, but malformed base64 can escape the typed error contract.
pipelex_sdk/upload.py	Adds source normalization and upload error mapping, with synchronous work in the async upload path.
pipelex_sdk/errors.py	Adds the typed exception family for preparation and upload failures.

Sequence Diagram

sequenceDiagram
participant Caller
participant Client as PipelexAPIClient
participant Build as /v1/build/inputs
participant Prep as prepare_inputs
participant Upload as /v1/upload
Caller->>Client: prepare_inputs(files, pipe_ref, inputs)
Client->>Build: "BuildInputsRequest(explicit=true)"
Build-->>Client: Explicit input template
Client->>Prep: Walk template and input values
loop Each local file value
    Prep->>Upload: Base64 UploadInput
    Upload-->>Prep: Storage URI
end
Prep-->>Caller: Rewritten inputs and upload records

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
pipelex_sdk/prepare_inputs.py:82
**Malformed Base64 Escapes Typed Errors**

When a file input contains an incorrectly padded data URL such as `data:image/png;base64,AQI`, `base64.b64decode()` raises `binascii.Error`. That exception bypasses the promised `InputPreparationError` family, so callers cannot handle all preparation failures through the documented base exception.

### Issue 2 of 2
pipelex_sdk/upload.py:121-122
**Large Uploads Block Event Loop**

A path upload performs the full synchronous file read, then base64-encodes the entire payload before reaching the first `await`. Uploading a file near the documented 50 MiB hosted limit, especially from slow storage, can stall every other coroutine on the same event loop.

_{Reviews (1): Last reviewed commit: "feat: upload_file + prepare_inputs + bui..." | Re-trigger Greptile}

[cubic-dev-ai]

All reported issues were addressed across 12 files

_{You’re at about 98% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.}

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[lchoquel]

Triaged the automated review feedback (Codex, Greptile, cubic). Pushed fixes for the genuine issues; deferring the rest with rationale.

Fixed (commit on this branch):

• Malformed base64 data URL escaping the typed-error contract (Codex P2 / Greptile P1 / cubic P2, prepare_inputs.py:82). Now decodes with validate=True (junk chars rejected instead of silently discarded into corrupted bytes) and maps binascii.Error → InputPreparationError, so every preparation failure stays in the documented family.
• Percent-encoded binary data URL corruption (cubic P2, prepare_inputs.py:83). Switched to urllib.parse.unquote_to_bytes; unquote(...).encode("utf-8") corrupted any byte ≥ 0x80.
• Doc: upload_file pass-through (cubic P2, input-preparation.md:26). Clarified that URL/storage-URI pass-through is a prepare_inputs-level behavior; upload_file treats every string as a filesystem path.
• Doc: checksum row (cubic P3, input-preparation.md:37). Removed — UploadRecord has no checksum field.
• Doc: architecture.md "planned addition" (cubic P2, architecture.md:141). Now reflects that the surface is implemented.

Added regression tests for both data-URL fixes.

Deferred (not bugs — in the spirit of "fix what needs fixing, defer on doubt"):

• Event-loop blocking on large file read/base64 (Greptile P2 / cubic P2, upload.py:121). Performance nicety on an inherently I/O-bound, discrete upload op; offloading the read threads async plumbing through _to_asset_bytes and doesn't address the base64 CPU cost (GIL). Revisit if large-file uploads become a real concurrency problem.
• BuildInputsValidReport template-required-per-format (cubic P2, build_models.py:51). prepare_inputs already raises a clean typed InputPreparationError when the template is absent, and build_inputs is a thin wire mirror of a server-guaranteed shape — a cross-field validator would be defensive modeling of a response that shouldn't occur.
• Splitting the two-scenario transport-error test (cubic P3, test_upload.py:117). Low-value attribution nit; both cases assert the same mapping.

make agent-check + make agent-test green.