Skip to content

Add design doc for generic OpenHouseTable operation summary accumulator (commits and scans) - #12

Open
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
devin/1788051461-commit-summary-accumulator-design
Open

Add design doc for generic OpenHouseTable operation summary accumulator (commits and scans)#12
devin-ai-integration[bot] wants to merge 5 commits into
mainfrom
devin/1788051461-commit-summary-accumulator-design

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 30, 2026

Copy link
Copy Markdown

Summary

Adds a design document for a generic, reusable operation summary accumulator for OpenHouse Iceberg tables. The design introduces a generic OpenHouseTable wrapper that delegates to the real Table and overrides data-file mutation entry points (newAppend, newFastAppend, newOverwrite, newReplacePartitions, newDelete, newTransaction) and scan entry points (newScan, newIncrementalAppendScan, newIncrementalChangelogScan). Business logic is injected through OpenHouseTableListener; the first listener is OpenHouseTableSummaryAccumulator, which produces an OpenHouseTableSummary whose partition-level objects have the same shape as CommitEventTablePartitions and CommitEventTablePartitionStats from TableStatsCollector. Adapters convert summaries to driver logs, lineage events, and a server request attachment.

All core components live in the existing openhouse-java-runtime module; adapters live in apps/spark where services:common is already on the classpath.

Changes

  • Documentation

Testing Done

  • No tests added or updated. This PR only adds a design document; implementation and tests will follow in a subsequent change.

Additional Information

  • Breaking Changes
  • Deprecations
  • Large PR broken into smaller PRs, and PR plan linked in the description.

Link to Devin session: https://app.devin.ai/sessions/a4ab4df6bc134ac5ab00ec63abcd854c
Open in Devin Desktop: https://app.devin.ai/desktop/session/a4ab4df6bc134ac5ab00ec63abcd854c?variant=devin
Requested by: @cbb330

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

cbb330 and others added 2 commits August 30, 2026 01:07
…ats models

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…pers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration devin-ai-integration Bot changed the title Add design doc for generic post-commit summary accumulator Add design doc for generic OpenHouseTable operation summary accumulator (commits and scans) Aug 30, 2026
cbb330 and others added 2 commits August 30, 2026 01:29
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant