feat: MergingSnapshotUpdate merges multiple DVs

[HuaHuaY]

No description provided.

[Copilot]

Pull request overview

Implements support for merging multiple deletion vectors (DVs) that reference the same data file by rewriting them into merged Puffin DV blobs, enabling format v3 workflows that previously returned NotImplemented.

Changes:

• Add DV-merge pipeline in MergingSnapshotUpdate that unions multiple DV bitmaps per referenced data file and writes merged DV blobs into a Puffin container.
• Introduce TransactionContext::NewDataLocation(...) to generate table-consistent data-file output locations for merged DV Puffin files.
• Promote Puffin + RoaringPositionBitmap components from iceberg_data into the core iceberg library (exports + build wiring) and add a test covering DV merging.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/iceberg/update/merging_snapshot_update.h	Adds DV merge APIs and tracking for merged DV outputs.
src/iceberg/update/merging_snapshot_update.cc	Implements DV merging via Puffin reader/writer and bitmap union; adds cleanup handling for merged DV outputs.
src/iceberg/transaction.h	Adds TransactionContext::NewDataLocation API.
src/iceberg/transaction.cc	Implements NewDataLocation via LocationProvider.
src/iceberg/test/merging_snapshot_update_test.cc	Adds an integration-style test validating duplicate DV merging behavior and correctness.
src/iceberg/puffin/puffin_writer.h	Moves export macro to core ICEBERG_EXPORT.
src/iceberg/puffin/puffin_reader.h	Moves export macro to core ICEBERG_EXPORT.
src/iceberg/puffin/puffin_format.h	Moves export macro to core ICEBERG_EXPORT.
src/iceberg/puffin/json_serde_internal.h	Moves export macro to core ICEBERG_EXPORT.
src/iceberg/puffin/file_metadata.h	Moves export macro to core ICEBERG_EXPORT.
src/iceberg/deletes/roaring_position_bitmap.h	Moves export macro to core ICEBERG_EXPORT.
src/iceberg/meson.build	Moves Puffin + bitmap sources/deps into iceberg lib; adjusts croaring linkage.
src/iceberg/CMakeLists.txt	Moves Puffin + bitmap sources into iceberg lib; adjusts croaring linkage propagation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wgtmac]

@zhaoxuan1994 is working on adding puffin reader/writer for v3 DVs. Let's wait for it and then rebase on it later.

[wgtmac]

I've made several changes to this PR:

• Kept puffin and dv code in iceberg-data library to make the core library IO-less still.
• Refactored DV logic based on a recent commit which added DV writer and reader support.
• Removed misc-include-cleaner from clang-tidy since I found it is useless but adds a lot of noise by blocking every ci until we add a lot of lengthy include statements.
• Removed ci workflow for sigv4 with meson. Recently vcpkg update introduced a breaking change of aws sdk. I think this workflow is fragile but there is no active Meson expert. Considering that Meson is still partially supported, I don't want to waste time and burn tokens to fix it. So I directly removed this workflow and may eventually remove all Meson code if unmaintained for a long time (perhaps a year?).

[wgtmac]

#820 has been reverted via this change because it does not add too much value but a lot of noise. cc @yadavay-amzn @zhjwpku