Skip to content

avoid underflow - #5027

Open
faithokamoto wants to merge 1 commit into
masterfrom
multi-underflow
Open

avoid underflow#5027
faithokamoto wants to merge 1 commit into
masterfrom
multi-underflow

Conversation

@faithokamoto

Copy link
Copy Markdown
Contributor

Changelog Entry

To be copied to the draft changelog by merger:

  • Avoid underflowing multiplicities in vg giraffe chaining mode (caused erroneous MAPQ 0)

Description

We had been managing to underflow chain_count_by_alignment here such that multiplicities got set as just around the maximum value for a size_t. Such high multiplicity guaranteed a MAPQ of 0.

This will not change the alignments output. It will likely "rescue" some MAPQ 0s, including some that represent incorrect alignments, and that will change variant calling. I have not tested the extent of the change.

@faithokamoto

Copy link
Copy Markdown
Contributor Author

In my tests, this changes between 0.2% to 0.8% of mapping qualities, most of which used to be 0. In R10 reads some of those are rescued all the way to 60. HiFi reads tend to get rescued to single-digit MQs. Unfortunately it rescues more wrong MQ0s than correct MQ0s... I'll run variant calling tests next.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant