[Variant] cast_to_variant conflates empty map with NULL

[scovich]

Describe the bug

While trying to refactor cast_to_variant to be more efficient, I noticed that the logic for maps does not check for NULL entries. It only looks at the offsets, which technically only say how many entries the map has.

To Reproduce

test_cast_map_to_variant_object_with_nulls produces a MapArray with no NULL rows, and the test expects a NULL output for the row that had an empty map.

Expected behavior

The test should produce a MapArray where one row is an empty map and another row is NULL; the former should become a variant object with no fields and the other should become NULL or Variant::Null (depending on whether the map is top-level or nested inside another variant value).

[klion26]

@scovich I can help to fix this, it's straightforward to fix the empty map case, for the MapArray with a NULL row, we need to return NULL for top-level and Variant::NULL for nested-level. Can we assume that the first call to the function parquet-variant-compute/src/cast_to_variant.rs#cast_to_variant is the top level, and the recursive calll from cast_to_variant to the function as the nested level?

[scovich]

The current cast design treats every convert_map call as "top level" because it always converts every column to variant independently, and complex types like struct combine child variants into parent variants. So I think it suffices to make convert_map check for a null input and append_null to its builder? And then a bunch of weird offset calculations can go away.

[klion26]

Thanks for the info, it seems this will be fixed in #8299 , we don't need a separate pr for this now.

[scovich]

• Fixed by [Variant] Implement row builders for cast_to_variant #8299