Is your feature request related to a problem or challenge?
parquet-rewrite --encoding currently applies one encoding to every column. This makes type-specific encodings difficult to use with mixed schemas.
For example, ALP supports only FLOAT and DOUBLE. Applying it globally to a file containing other physical types fails after output-file creation. Other type-specific encodings have similar behavior.
This depends on #9372 and should be worked on after that PR is merged.
Describe the solution you would like
Add per-column encoding selection to parquet-rewrite, while retaining the global encoding option as a default.
Validate the selected encoding against each column physical type before rewriting begins. Unsupported combinations should return an actionable error, for example that ALP supports only FLOAT and DOUBLE, without leaving an incomplete output file.
Is your feature request related to a problem or challenge?
parquet-rewrite --encodingcurrently applies one encoding to every column. This makes type-specific encodings difficult to use with mixed schemas.For example, ALP supports only
FLOATandDOUBLE. Applying it globally to a file containing other physical types fails after output-file creation. Other type-specific encodings have similar behavior.This depends on #9372 and should be worked on after that PR is merged.
Describe the solution you would like
Add per-column encoding selection to
parquet-rewrite, while retaining the global encoding option as a default.Validate the selected encoding against each column physical type before rewriting begins. Unsupported combinations should return an actionable error, for example that ALP supports only
FLOATandDOUBLE, without leaving an incomplete output file.