feat: add LateralJoinRel support (inference + builders)

[nielspardon]

Summary

Adds LateralJoinRel support (Substrait v0.96.0, #205) to schema inference and the builder / DataFrame layers. A lateral join evaluates its right (dependent) input once per left row; the right input references the current left row via OuterReference.rel_reference pointing at the join's RelCommon.rel_anchor.

Rebased onto main and refactored to build on the id-based outer-reference resolution merged in #239, so there is now a single resolution mechanism (_AnchorScope) — this PR no longer adds its own outer_anchors map.

Inference (type_inference.py)

• infer_rel_schema handles the lateral_join rel type; output columns follow the same per-join-type shapes as a regular JoinRel (INNER/LEFT → left+right, LEFT_SEMI/LEFT_ANTI → left only, LEFT_MARK → +mark).
• Id-based outer references resolve through feat!: id-based outer-reference resolution (Substrait v0.89.0) #239's plan-wide _AnchorScope. A lateral-join anchor denotes the left row, so _AnchorScope resolves it against the left input's schema (one small special case). While the right input is inferred, the left schema is bound to the anchor via a new _outer_anchor_binding context manager (a scoped _AnchorScope chained to any enclosing one), so:
  • inferring a lateral join on its own is self-contained (no whole-plan context required), and
  • nested lateral joins compose — an inner right input can reference an outer anchor via the parent chain.

Builders

• builders.plan.lateral_join(left, right, type, ...): right is a function of a LateralInput handle to the left; handle.column(...) emits the id-based outer reference. The builder allocates a rel_anchor, infers the right input under _outer_anchor_binding, and assembles via _plan_from so shared subtrees (CTEs) propagate.
• DataFrame.lateral_join(right, how, *, on, post_filter) + LateralLeft.col(): the DataFrame-layer handle. Capturing the handle avoids nesting-depth bookkeeping and makes referencing outside a lateral join impossible. Only INNER and left-oriented join types are valid.
• Anchor allocation: next_rel_anchor() is unique within a build; fresh_rel_anchors() resets numbering per materialization. DataFrame._finalize builds under fresh_rel_anchors, then normalizes any correlated-subquery steps_out to id-based via to_id_based_outer_references (feat!: id-based outer-reference resolution (Substrait v0.89.0) #239) — whose anchor pre-scan keeps DataFrame-correlation anchors clear of the lateral-join ones.

Testing

• Inference: lateral-join output shapes (inner / left_semi / left_mark), a correlated rel_reference resolving against the left row, and the unknown-anchor error.
• Builders: plan.lateral_join and DataFrame.lateral_join (correlated right input, join types, post-filter).
• Full suite green; ruff check + ruff format --check clean.

Closes #205.

🤖 Generated with AI