Skip to content

feat: add lower-level parallel streaming support#836

Open
Haricharanpanjwani wants to merge 1 commit into
apache:mainfrom
Haricharanpanjwani:codex/issue-534-parallel-streaming
Open

feat: add lower-level parallel streaming support#836
Haricharanpanjwani wants to merge 1 commit into
apache:mainfrom
Haricharanpanjwani:codex/issue-534-parallel-streaming

Conversation

@Haricharanpanjwani

Copy link
Copy Markdown
Contributor

Closes #534.

Summary

  • make TaskBasedParallelAction stream-capable by extending it through Burr's existing streaming container flow
  • add a StreamItem event type plus sync/async subgraph streaming helpers on SubGraphTask
  • preserve stable task ordering for reduce(...) while allowing intermediate stream events to arrive as tasks complete
  • add toggles for intermediate_stream_outputs and intermediate_nodes to control how much subgraph output is surfaced
  • document the new lower-level parallel streaming capability in the parallelism guide

Testing

  • python3 -m py_compile burr/core/parallelism.py tests/core/test_parallelism.py
  • XDG_CACHE_HOME=/tmp/xdg-cache UV_CACHE_DIR=/tmp/uv-cache uv run --with pytest --with pytest-asyncio --with pydantic pytest tests/core/test_parallelism.py -q -k "streaming"
  • XDG_CACHE_HOME=/tmp/xdg-cache UV_CACHE_DIR=/tmp/uv-cache uv run --with pytest --with pytest-asyncio --with pydantic pytest tests/core/test_parallelism.py -q

Notes

  • This scopes the fix to the lower-level TaskBasedParallelAction capability described in the issue so the existing Application.stream_result(...) and Application.astream_result(...) APIs can surface parallel subgraph events without introducing a separate public streaming entrypoint.
  • The broader queue/executor abstraction ideas from the issue body can still build on top of this in follow-up work if we want specialized multiprocessing or distributed backends later.

@github-actions github-actions Bot added area/core Application, State, Graph, Actions area/streaming Streaming actions, parallel streams area/website burr.apache.org website labels Jul 10, 2026
@vaquarkhan

Copy link
Copy Markdown
Contributor

Ran this locally, whole parallelism suite passes and the new streaming tests were stable across a few reruns. Love this, been wanting interleaved parallel streams for a while.

Couple thoughts, all follow-up not blocking:

  • Tasks get materialized all at once (async spawns a coroutine per task, sync lists them all), so big fan-outs could kick off a lot at once. A max-in-flight option would help.
  • The result queue is unbounded, so a slow consumer could let memory grow. A cap would give you backpressure for free.
  • The get(timeout=0.01) poll loop spins a bit and adds ~10ms of delivery latency. A blocking get with a done-sentinel would be smoother.
  • Might be worth a test or two for the unhappy paths (a task throwing mid-stream, empty task list).

Also heads up, moving to SingleStepStreamingAction flips all the Map* subclasses to streaming=True. Non-streaming still works, just noting it. LGTM.

@skrawcz

skrawcz commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

I like this idea -- we just need to come up with a good test suite to ensure that:

(a) this is backwards compatible
(b) we are testing the right edge cases and have them covered via tests
(c) double check why we originally didn't make it streaming for a reason -- @elijahbenizzy ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Application, State, Graph, Actions area/streaming Streaming actions, parallel streams area/website burr.apache.org website

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel Streaming constructs

3 participants