Is your feature request related to a problem?
xCDAT operations can fail or produce unclear downstream errors when datasets have incomplete or inconsistent CF metadata, axis definitions, or coordinate bounds.
Users often discover these issues only during downstream operations. Related examples include:
- Regridding failures caused by missing axis attributes or incorrectly identified coordinates (#836)
- Regridding failures involving multidimensional or ambiguous latitude and longitude coordinates (#816, #718, and #747)
- Temporal operations failing after required bounds metadata is lost or unavailable (#832)
- Spatial averaging producing an unclear weights error when bounds include an incompatible time dimension (#797)
- Conflicting CF attributes causing multiple coordinates to be mapped to the same axis (#610)
By the time these errors surface, the underlying dataset issue can be difficult to identify and address.
Describe the solution you'd like
Add validation function(s) that inspect a dataset before downstream processing and report common issues, including:
- Missing or inconsistent CF metadata
- Missing, ambiguous, or incorrectly identified axes
- Missing or malformed coordinate bounds
- Bounds variables with incompatible dimensions or coordinates
- Conflicting metadata that prevents xCDAT from identifying coordinates
This would provide a standard upstream validation step for workflows that ingest heterogeneous climate datasets.
Evaluate relevant cf_xarray APIs and reuse its CF-aware metadata inspection and axis or coordinate discovery functionality where practical, rather than duplicating metadata interpretation logic. Supplement this with xCDAT-specific checks for assumptions required by its axis, bounds, temporal, spatial, and regridding operations.
Each reported issue should identify:
- The affected variable or coordinate
- The detected problem
- The xCDAT operations that may be affected
- A suggested user action when possible
For example:
result = xc.validate_dataset(ds)
result.raise_for_errors()
Validation should diagnose issues rather than silently modify the dataset. Users can then correct the source data or explicitly apply utilities such as bounds generation.
The initial implementation can focus on cf_xarray integration and xCDAT-specific bounds and operation checks. Additional validation rules can be added incrementally as new failure cases are identified.
Describe alternatives you've considered
- Continue implementing validation independently within each xCDAT operation.
- This provides operation-specific checks but duplicates validation logic and delays errors until an operation is executed. A shared validation API would provide an earlier and more consistent preflight check.
- Rely entirely on
cf_xarray.
- Its primary purpose is interpreting CF metadata, while xCDAT may require additional validation of bounds structures and operation-specific dataset assumptions.
- Automatically repair detected issues.
- Implicit modifications may hide source-data problems or apply assumptions that are not valid for every dataset.
Additional Context
The API documentation should be updated to include this new API. Also the FAQs section should include information on how users can validate their datasets for compatibility issues beforehand.
Is your feature request related to a problem?
xCDAT operations can fail or produce unclear downstream errors when datasets have incomplete or inconsistent CF metadata, axis definitions, or coordinate bounds.
Users often discover these issues only during downstream operations. Related examples include:
By the time these errors surface, the underlying dataset issue can be difficult to identify and address.
Describe the solution you'd like
Add validation function(s) that inspect a dataset before downstream processing and report common issues, including:
This would provide a standard upstream validation step for workflows that ingest heterogeneous climate datasets.
Evaluate relevant
cf_xarrayAPIs and reuse its CF-aware metadata inspection and axis or coordinate discovery functionality where practical, rather than duplicating metadata interpretation logic. Supplement this with xCDAT-specific checks for assumptions required by its axis, bounds, temporal, spatial, and regridding operations.Each reported issue should identify:
For example:
Validation should diagnose issues rather than silently modify the dataset. Users can then correct the source data or explicitly apply utilities such as bounds generation.
The initial implementation can focus on
cf_xarrayintegration and xCDAT-specific bounds and operation checks. Additional validation rules can be added incrementally as new failure cases are identified.Describe alternatives you've considered
cf_xarray.Additional Context
The API documentation should be updated to include this new API. Also the FAQs section should include information on how users can validate their datasets for compatibility issues beforehand.