Add snapshot selection to manifest inspection - #3913
Open
emecii wants to merge 1 commit into
Open
Conversation
Decode partition summaries with snapshot schemas and retained source fields so historical manifests remain readable after evolution. Generated-by: OpenAI Codex
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Implements the manifest-inspection snapshot-selection part of #1053; the umbrella issue remains open.
table.inspect.manifests(snapshot_id=...)selects a retained snapshot using the existing lookup helper. Calls without an ID retain current-snapshot and typed empty-table behavior; unknown or expired IDs raise the existing lookup error.Partition summaries use the selected snapshot's schema and each manifest's partition spec. When a retained manifest references a dropped source, resolve its field ID from retained schemas. This keeps old manifests readable after schema and partition evolution, including a reused column name with a different field ID/type. Snapshots without a schema ID follow the existing current-schema fallback convention; unavailable recorded schemas warn before falling back.
all_manifests()retains its signature and all-snapshot reference rows. It receives the historical summary correction through the shared generator. No general readable-metrics or bound-conversion changes.Are these changes tested?
Real local catalog tests cover old/current selection, exact manifest paths, empty tables, invalid and expired IDs, renamed/dropped partition sources, mixed partition specs, reused names, null/empty bounds, type promotion, schema fallback, and all-manifest references.
make lint: passed.Spark/Docker parity, cloud suites, other Python/platform combinations and documentation build were not run. Historical fallback relies on retained schema metadata; it does not recover types from removed metadata files.
Are there any user-facing changes?
Yes: optional
snapshot_idonInspectTable.manifests, documented in the API guide. Existing calls remain valid; historical partition summaries use retained schemas/specs.AI assistance
OpenAI Codex generated the implementation, tests, documentation and PR draft, ran the reported checks, and reviewed the diff. The commit records
Generated-by: OpenAI Codex. These statements describe agent work and do not assert personal human review.