Skip to content

Compute and output normalized error above tolerance - #112

Merged
huppd merged 7 commits into
mainfrom
normalized-err
Aug 21, 2026
Merged

Compute and output normalized error above tolerance#112
huppd merged 7 commits into
mainfrom
normalized-err

Conversation

@stelliom

Copy link
Copy Markdown
Collaborator

This PR adds the computation of a fourth metric for the output of check.py. Instead of only showing the "Error relative to tolerance":

err_rel_tol = err / tol

We also show the "Normalized error above tolerance":

err_above_tol = (err - tol) / tol = err / tol - 1

This let's user see how much above the tolerance the checked variables are, in a normalized fashion, such that errors for different variables can be compared. This is especially useful when err_rel_tol is very close to 1, since the precision of the output only shows 2 decimal digits. With this new output you can see how much above the tolerance the values are even if they are only slightly above (i.e. err_rel_tol ~ 1).

Note that values that do not exceed the tolerance are clipped to 0, this allows for spotting errors quicker.

@stelliom

Copy link
Copy Markdown
Collaborator Author

@huppd Follow up on our Slack conversation. What do you think?

@huppd huppd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this proposition. I like it :) also the clipping is a good idea.

Comment thread engine/check.py Outdated
Comment thread engine/check.py Outdated
stelliom and others added 2 commits August 21, 2026 11:25
Co-authored-by: Daniel Hupp <daniel.hupp@meteoswiss.ch>
Comment thread engine/check.py Outdated
huppd and others added 2 commits August 21, 2026 11:50
Co-authored-by: Mikael Stellio <67868694+stelliom@users.noreply.github.com>
Comment thread engine/check.py Outdated
Comment thread engine/check.py Outdated
stelliom and others added 2 commits August 21, 2026 12:24
Co-authored-by: Mikael Stellio <67868694+stelliom@users.noreply.github.com>

@huppd huppd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff, thanks :)

@huppd
huppd merged commit bb808de into main Aug 21, 2026
3 checks passed
@huppd
huppd deleted the normalized-err branch August 21, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants