Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## [v0.14.1] - 2026-07-15

- **Multi-file bundle support in the mode lifecycle helpers:** `execute_pipe`, `start_and_wait`, and `start_pipe` now take `mthds_contents: list[str]` (the bundle's `.mthds` files as strings — one entry for a single-file bundle, several for a multi-file one) instead of a single `bundle: str`, passed straight through to the SDK. Multi-file bundles cannot be concatenated into one string (duplicate top-level TOML keys), so the list is the interface. The three demos stay single-file (`mthds_contents=[bundle]`); a method dir with several `.mthds` files is read with `[p.read_text() for p in sorted((METHODS_DIR / "<name>").glob("*.mthds"))]`. The package-data glob broadens to `methods/*/*.mthds` so multi-file bundles ship.
- Bumped the `pipelex-tools` dev dependency from `>=0.3.2` to `>=0.7.2`.

## [v0.14.0] - 2026-07-15

### Added
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,8 +155,8 @@ flowchart TD
classDef terminal fill:#f8fafc,stroke:#475569,stroke-width:1.5px,color:#0f172a
```

1. **Read the bundle.** `piper` reads `methods/extract-entities/main.mthds` from disk and constructs a `PipelexAPIClient`, which picks up `PIPELEX_BASE_URL` / `PIPELEX_API_KEY` from the environment.
2. **Run it on the API.** The bundle is sent as *content* (`mthds_contents`), so nothing method-specific needs to live in the runtime — edit the `.mthds` file and re-run, no redeploy.
1. **Read the bundle.** `piper` reads `methods/extract-entities/main.mthds` from disk and constructs a `PipelexAPIClient`, which picks up `PIPELEX_BASE_URL` / `PIPELEX_API_KEY` from the environment. A method dir may hold a single `main.mthds` or several `.mthds` files (a multi-file bundle split across pipes) — read them all with `[p.read_text() for p in sorted((METHODS_DIR / "<name>").glob("*.mthds"))]`.
2. **Run it on the API.** The bundle's files are sent together as *content* (`mthds_contents`, one string per file), so nothing method-specific needs to live in the runtime — edit the `.mthds` file(s) and re-run, no redeploy.
3. **Narrow the result.** The SDK resolves the run's `main_stuff`; the command validates it into the generated `ExtractedEntities` model (`ExtractedEntities.model_validate(main_stuff)`), printed as JSON.

The typed models are **not hand-written**: they are generated from the `.mthds` bundles by `pipelex codegen` into `piper/generated/` (stamped, with a `codegen.lock` per method). Edit a bundle → `make codegen` regenerates the models and input templates → `make codegen-check` verifies offline that nothing is stale or hand-edited. See [docs/codegen.md](docs/codegen.md).
Expand Down
2 changes: 2 additions & 0 deletions docs/cli-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ All three have the same four-part shape, so they diff cleanly:
3. **The lifecycle helper** — one public async function that *is* the mode (`detached` adds the run-id lifecycle helpers described below). It gets a public name because it is the featured code, and it is what the unit tests patch and the e2e tests call directly.
4. **The demo commands + a private `_run()`** — each command reads its input, reads its bundle, awaits the lifecycle helper through `_run()` (`asyncio.run` + the single `except (PipelineRequestError, httpx.HTTPStatusError)` that presents via `piper/errors.py`), narrows the result into its *generated* model, prints JSON.

The lifecycle helpers take the bundle as `mthds_contents: list[str]` — one string per `.mthds` file — and pass it straight to the SDK. The three demos are single-file methods, so each reads its `main.mthds` and wraps it as `mthds_contents=[bundle]`. A method dir may instead hold several `.mthds` files (a multi-file bundle split across pipes with `signature_for` cross-file declarations); read them all with `[p.read_text() for p in sorted((METHODS_DIR / "<name>").glob("*.mthds"))]` and hand that list to the helper unchanged. Concatenating the files into one string would be invalid TOML — the list is the interface for exactly this reason.

| Mode | Lifecycle helper | SDK calls | What the demo prints |
| --- | --- | --- | --- |
| `blocking` | `execute_pipe()` | `client.execute` | the result, as JSON |
Expand Down
18 changes: 10 additions & 8 deletions piper/attended/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,17 @@
progress_console = Console(stderr=True)


async def start_and_wait(*, pipe_code: str, bundle: str, inputs: dict[str, Any]) -> Any:
async def start_and_wait(*, pipe_code: str, mthds_contents: list[str], inputs: dict[str, Any]) -> Any:
"""The whole attended lifecycle: start a durable run, print its id, wait here for the result.

Credentials come from `PIPELEX_API_KEY` / `PIPELEX_BASE_URL`. The SDK resolves the
method's main output for you: `.main_stuff` is the content the pipe named as its
result (a completed run that names none raises `MissingMainStuffError`).
Credentials come from `PIPELEX_API_KEY` / `PIPELEX_BASE_URL`. `mthds_contents` is the
bundle's `.mthds` files as strings — one entry for a single-file bundle, several for a
multi-file one. The SDK resolves the method's main output for you: `.main_stuff` is
the content the pipe named as its result (a completed run that names none raises
`MissingMainStuffError`).
"""
async with PipelexAPIClient() as client:
start_result = await client.start(pipe_code=pipe_code, mthds_contents=[bundle], inputs=inputs)
start_result = await client.start(pipe_code=pipe_code, mthds_contents=mthds_contents, inputs=inputs)
run_id = start_result.pipeline_run_id
# Printed before the first poll, so Ctrl-C leaves you with a usable run id.
progress_console.print(f"Run started: [bold]{run_id}[/bold]")
Expand Down Expand Up @@ -81,7 +83,7 @@ def extract_entities(
if resolved.is_sample:
progress_console.print(f"[dim]No text given — using the sample: {resolved.text!r}. Pass your own as an argument or via --file.[/dim]")
bundle = (METHODS_DIR / "extract-entities" / "main.mthds").read_text()
main_stuff = _run(start_and_wait(pipe_code="extract_entities", bundle=bundle, inputs={"text": resolved.text}))
main_stuff = _run(start_and_wait(pipe_code="extract_entities", mthds_contents=[bundle], inputs={"text": resolved.text}))
# Narrow into the generated typed model (validates the concept's shape), then print it as JSON.
entities = ExtractedEntities.model_validate(main_stuff)
output_console.print_json(data=entities.model_dump())
Expand All @@ -100,7 +102,7 @@ def summarize_pdf(
progress_console.print(f"[dim]No file given — using the sample: {document.name}. Pass a path to summarize your own document.[/dim]")
bundle = (METHODS_DIR / "summarize-pdf" / "main.mthds").read_text()
inputs = {"document": build_document_input(document)}
main_stuff = _run(start_and_wait(pipe_code="summarize_pdf", bundle=bundle, inputs=inputs))
main_stuff = _run(start_and_wait(pipe_code="summarize_pdf", mthds_contents=[bundle], inputs=inputs))
summary = DocumentSummary.model_validate(main_stuff)
output_console.print_json(data=summary.model_dump())

Expand All @@ -120,7 +122,7 @@ def generate_image(
if resolved.is_sample:
progress_console.print(f"[dim]No prompt given — using the sample: {resolved.text!r}. Pass your own as an argument or via --file.[/dim]")
bundle = (METHODS_DIR / "generate-image" / "main.mthds").read_text()
main_stuff = _run(start_and_wait(pipe_code="generate_image", bundle=bundle, inputs={"image_prompt": resolved.text}))
main_stuff = _run(start_and_wait(pipe_code="generate_image", mthds_contents=[bundle], inputs={"image_prompt": resolved.text}))
# On the hosted path the runtime returns a storage `url` (`pipelex-storage://…`)
# *and* a web-renderable `public_url` (a signed URL); the model keeps both.
image = Image.model_validate(main_stuff)
Expand Down
18 changes: 10 additions & 8 deletions piper/blocking/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,18 @@
progress_console = Console(stderr=True)


async def execute_pipe(*, pipe_code: str, bundle: str, inputs: dict[str, Any]) -> Any:
async def execute_pipe(*, pipe_code: str, mthds_contents: list[str], inputs: dict[str, Any]) -> Any:
"""The whole blocking lifecycle: one call, and the result comes back in the response.

Credentials come from `PIPELEX_API_KEY` / `PIPELEX_BASE_URL`. The SDK resolves the
method's main output for you: `.main_stuff` is the content the pipe named as its
result (a completed run that names none raises `MissingMainStuffError`).
Credentials come from `PIPELEX_API_KEY` / `PIPELEX_BASE_URL`. `mthds_contents` is the
bundle's `.mthds` files as strings — one entry for a single-file bundle, several for a
multi-file one. The SDK resolves the method's main output for you: `.main_stuff` is
the content the pipe named as its result (a completed run that names none raises
`MissingMainStuffError`).
"""
async with PipelexAPIClient() as client:
with progress_console.status("Running…"):
result = await client.execute(pipe_code=pipe_code, mthds_contents=[bundle], inputs=inputs)
result = await client.execute(pipe_code=pipe_code, mthds_contents=mthds_contents, inputs=inputs)
return result.main_stuff


Expand All @@ -59,7 +61,7 @@ def extract_entities(
if resolved.is_sample:
progress_console.print(f"[dim]No text given — using the sample: {resolved.text!r}. Pass your own as an argument or via --file.[/dim]")
bundle = (METHODS_DIR / "extract-entities" / "main.mthds").read_text()
main_stuff = _run(execute_pipe(pipe_code="extract_entities", bundle=bundle, inputs={"text": resolved.text}))
main_stuff = _run(execute_pipe(pipe_code="extract_entities", mthds_contents=[bundle], inputs={"text": resolved.text}))
# Narrow into the generated typed model (validates the concept's shape), then print it as JSON.
entities = ExtractedEntities.model_validate(main_stuff)
output_console.print_json(data=entities.model_dump())
Expand All @@ -78,7 +80,7 @@ def summarize_pdf(
progress_console.print(f"[dim]No file given — using the sample: {document.name}. Pass a path to summarize your own document.[/dim]")
bundle = (METHODS_DIR / "summarize-pdf" / "main.mthds").read_text()
inputs = {"document": build_document_input(document)}
main_stuff = _run(execute_pipe(pipe_code="summarize_pdf", bundle=bundle, inputs=inputs))
main_stuff = _run(execute_pipe(pipe_code="summarize_pdf", mthds_contents=[bundle], inputs=inputs))
summary = DocumentSummary.model_validate(main_stuff)
output_console.print_json(data=summary.model_dump())

Expand All @@ -98,7 +100,7 @@ def generate_image(
if resolved.is_sample:
progress_console.print(f"[dim]No prompt given — using the sample: {resolved.text!r}. Pass your own as an argument or via --file.[/dim]")
bundle = (METHODS_DIR / "generate-image" / "main.mthds").read_text()
main_stuff = _run(execute_pipe(pipe_code="generate_image", bundle=bundle, inputs={"image_prompt": resolved.text}))
main_stuff = _run(execute_pipe(pipe_code="generate_image", mthds_contents=[bundle], inputs={"image_prompt": resolved.text}))
# On the hosted path the runtime returns a storage `url` (`pipelex-storage://…`)
# *and* a web-renderable `public_url` (a signed URL); the model keeps both.
image = Image.model_validate(main_stuff)
Expand Down
15 changes: 8 additions & 7 deletions piper/detached/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,15 @@
progress_console = Console(stderr=True)


async def start_pipe(*, pipe_code: str, bundle: str, inputs: dict[str, Any]) -> str:
async def start_pipe(*, pipe_code: str, mthds_contents: list[str], inputs: dict[str, Any]) -> str:
"""The whole detached lifecycle: start a durable run, return its id, don't wait.

Credentials come from `PIPELEX_API_KEY` / `PIPELEX_BASE_URL`. The run keeps
executing server-side after this process exits.
Credentials come from `PIPELEX_API_KEY` / `PIPELEX_BASE_URL`. `mthds_contents` is the
bundle's `.mthds` files as strings — one entry for a single-file bundle, several for a
multi-file one. The run keeps executing server-side after this process exits.
"""
async with PipelexAPIClient() as client:
start_result = await client.start(pipe_code=pipe_code, mthds_contents=[bundle], inputs=inputs)
start_result = await client.start(pipe_code=pipe_code, mthds_contents=mthds_contents, inputs=inputs)
return start_result.pipeline_run_id


Expand Down Expand Up @@ -93,7 +94,7 @@ def extract_entities(
if resolved.is_sample:
progress_console.print(f"[dim]No text given — using the sample: {resolved.text!r}. Pass your own as an argument or via --file.[/dim]")
bundle = (METHODS_DIR / "extract-entities" / "main.mthds").read_text()
run_id = _run(start_pipe(pipe_code="extract_entities", bundle=bundle, inputs={"text": resolved.text}))
run_id = _run(start_pipe(pipe_code="extract_entities", mthds_contents=[bundle], inputs={"text": resolved.text}))
_print_run_id(run_id)


Expand All @@ -110,7 +111,7 @@ def summarize_pdf(
progress_console.print(f"[dim]No file given — using the sample: {document.name}. Pass a path to summarize your own document.[/dim]")
bundle = (METHODS_DIR / "summarize-pdf" / "main.mthds").read_text()
inputs = {"document": build_document_input(document)}
run_id = _run(start_pipe(pipe_code="summarize_pdf", bundle=bundle, inputs=inputs))
run_id = _run(start_pipe(pipe_code="summarize_pdf", mthds_contents=[bundle], inputs=inputs))
_print_run_id(run_id)


Expand All @@ -128,7 +129,7 @@ def generate_image(
if resolved.is_sample:
progress_console.print(f"[dim]No prompt given — using the sample: {resolved.text!r}. Pass your own as an argument or via --file.[/dim]")
bundle = (METHODS_DIR / "generate-image" / "main.mthds").read_text()
run_id = _run(start_pipe(pipe_code="generate_image", bundle=bundle, inputs={"image_prompt": resolved.text}))
run_id = _run(start_pipe(pipe_code="generate_image", mthds_contents=[bundle], inputs={"image_prompt": resolved.text}))
_print_run_id(run_id)


Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "piper"
version = "0.14.0"
version = "0.14.1"
description = "Replace this with your project description"
# authors = [{ name = "Your Name", email = "your.email@example.com" }]
license = "MIT"
Expand Down Expand Up @@ -41,7 +41,7 @@ packages = [
include-package-data = true

[tool.setuptools.package-data]
piper = ["py.typed", "methods/*/main.mthds", "methods/*/inputs.template.json"]
piper = ["py.typed", "methods/*/*.mthds", "methods/*/inputs.template.json"]
# Ship each codegen.lock alongside its models.py so `pipelex codegen check` works on an installed copy.
"piper.generated.extract_entities" = ["codegen.lock"]
"piper.generated.generate_image" = ["codegen.lock"]
Expand All @@ -50,7 +50,7 @@ piper = ["py.typed", "methods/*/main.mthds", "methods/*/inputs.template.json"]
[project.optional-dependencies]
dev = [
"mypy==1.19.1",
"pipelex-tools>=0.3.2",
"pipelex-tools>=0.7.2",
"pyright>=1.1.411",
"pytest>=9.0.3",
"pytest-mock>=3.14.0",
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/test_extract_entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ async def test_blocking(self):
# The blocking lifecycle end to end: one `execute` call, then narrow.
# Extraction finishes well under the hosted ~30s cap, so the blocking mode owns this demo.
bundle = BUNDLE_PATH.read_text()
main_stuff = await execute_pipe(pipe_code="extract_entities", bundle=bundle, inputs={"text": SAMPLE_TEXT})
main_stuff = await execute_pipe(pipe_code="extract_entities", mthds_contents=[bundle], inputs={"text": SAMPLE_TEXT})
entities = ExtractedEntities.model_validate(main_stuff)
assert any("Curie" in person for person in entities.people)
2 changes: 1 addition & 1 deletion tests/e2e/test_generate_image.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ async def test_detached(self):
# start, get an id back, then pick the run up again through that id alone.
# Image generation outlives the ~30s blocking cap, so it is the demo detached mode owns.
bundle = BUNDLE_PATH.read_text()
run_id = await start_pipe(pipe_code="generate_image", bundle=bundle, inputs={"image_prompt": SAMPLE_PROMPT})
run_id = await start_pipe(pipe_code="generate_image", mthds_contents=[bundle], inputs={"image_prompt": SAMPLE_PROMPT})
assert run_id
main_stuff = await attend_run(run_id)
image = Image.model_validate(main_stuff)
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/test_summarize_pdf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ async def test_attended(self):
# The attended lifecycle end to end: encode the PDF, start a durable run, poll it, narrow.
bundle = BUNDLE_PATH.read_text()
inputs = {"document": build_document_input(SAMPLE_PDF)}
main_stuff = await start_and_wait(pipe_code="summarize_pdf", bundle=bundle, inputs=inputs)
main_stuff = await start_and_wait(pipe_code="summarize_pdf", mthds_contents=[bundle], inputs=inputs)
summary = DocumentSummary.model_validate(main_stuff)
assert summary.title
assert summary.doc_type
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_bootstrap_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def write_template(root: Path, *, extra_pyproject: str = "") -> None:
]

[tool.setuptools.package-data]
piper = ["py.typed", "methods/*/main.mthds"]
piper = ["py.typed", "methods/*/*.mthds"]
"piper.generated.extract_entities" = ["codegen.lock"]

[tool.ruff]
Expand Down
Loading
Loading