Implementing seed ranking for displaced tracks

[A-A-Abdelhamid]

Implementing an Optimized Seed Ranking for Displaced Tracks

Proposed ordering:

L2L3L4 > L4L5L6 > L2D1D2 > L2L3D1

The optimization studies were presented at the BES Offline Software meeting on 21 May 2025. Performance plots for the proposed rankings are shown on slides 13–19.

This is PR #370 applied to L1TK-dev-20_1_0_pre1.

I used 10,000 events from the following sample to produce all plots and performance results:

Phase2Spring24DIGIRECOMiniAOD/DisplacedSUSY_stopToBottom_M-800_50mm_TuneCP5_14TeV-pythia8/GEN-SIM-DIGI-RAW-MINIAOD/PU200_AllTP_140X_mcRun4_realistic_v4-v1/

Seed rankings

I compared two proposed seed rankings:

• Optimized: L2L3L4 > L4L5L6 > L2D1D2 > L2L3D1
• Alternative: L4L5L6 > L2L3L4 > L2D1D2 > L2L3D1

I retained both comparisons because they give nearly the same efficiency improvement relative to the default ordering, with a small tradeoff among efficiency, fake and duplicate fractions, and the central z0 tail.

Summary printout

Efficiency

Configuration	Combined efficiency, |η| < 2	3 < pT < 8 GeV	pT > 8 GeV	pT > 40 GeV
Default	78.83 ± 0.11%	86.28 ± 0.13%	71.39 ± 0.17%	54.89 ± 0.37%
Optimized	80.92 ± 0.11%	87.14 ± 0.13%	74.71 ± 0.17%	59.55 ± 0.36%
Alternative	80.95 ± 0.11%	87.17 ± 0.13%	74.75 ± 0.17%	59.64 ± 0.36%

Based on the summary printout, both proposed rankings improve the overall efficiency relative to the default ordering. The alternative ranking gives the highest numerical efficiency, but only slightly: 80.95%, compared with 80.92% for the optimized ranking and 78.83% for the default.

The efficiency improvement relative to the default is larger at higher pT. For pT > 8 GeV, the efficiency increases from 71.39% for the default to 74.71% for the optimized ranking and 74.75% for the alternative ranking. For pT > 40 GeV, it increases from 54.89% to 59.55% and 59.64%, respectively.

The cost is a modest increase in track multiplicity and in the fake and duplicate fractions. The optimized ranking increases the fake fraction from 38.07% to 39.31% and the duplicate fraction from 6.37% to 7.34%. The alternative ranking gives very similar, but slightly higher, values of 39.35% and 7.41%, respectively.

Helix-parameter resolution

The 68% and 90% z0 and d0 resolutions are mostly unchanged between the two proposed rankings. The clearest difference is in the central 99% z0 tail at |η| = 0.05:

Configuration	Central 99% z0 interval
Default	1.62 cm
Optimized	0.85 cm
Alternative	0.87 cm

The optimized ranking therefore gives slightly better central z0-tail behavior in this comparison, while the alternative gives a marginally higher numerical efficiency.

The difference between the optimized and alternative rankings is very small, so I do not want to overstate it. The robust conclusion is that both proposed rankings improve efficiency relative to the default and both improve the central z0 tail.

The choice between them is a small tradeoff between slightly better central z0-tail resolution and slightly higher numerical efficiency. Since the optimized ranking gives nearly the same efficiency as the alternative while giving a slightly better central 99% z0 tail and slightly lower fake and duplicate fractions, I would retain L2L3L4 > L4L5L6 > L2D1D2 > L2L3D1 as the proposed ordering.

Resolution plots

The main performance comparison is the full before-and-after tracking comparison summarized above. I have also produced the corresponding seed-ranking resolution plots using both RMS-based and percentile-based resolutions.

The resolution plots for the three oldKF displaced configurations are shown here:

• Default
• Optimized: L2L3L4 > L4L5L6 > L2D1D2 > L2L3D1
• Alternative: L4L5L6 > L2L3L4 > L2D1D2 > L2L3D1

The page includes the full set of overlaid resolution plots, grouped by variable, with the RMS, 68%, 90%, and 99% interval versions shown together where available.

[tomalin]

Have you checked the effect of this change? My observation is that not setting sortedMergedTrack to true for displaced seeds significantly improved the duplicate rate, although it is not something very realistic in an FPGA algo.

[tomalin]

Again, I observe that these lines significantly reduce the duplicate rate for displaced tracking. Why have you removed them, when none of the tests you've reported prove that this doesn't adversely degrade performance? (I guess the equivalent to these lines, now you have optimised the seed ordering is deciding which track to keep if two tracks have identical displaced seed type).

[tomalin]

This is failing the new improved git CI, which runs on displaced SUSY + 200PU MC when testing displaced tracking, instead of ttbar + 0PU MC as it used to. The failure is shown here https://gitlab.cern.ch/cms-l1tk/cmssw_CI/-/jobs/78686553 . It's caused by the increase in duplicate rate, which I suspect is caused by the introduction of the two bugs that I mention above.

[tomalin]

Sorry, the git conflicts are my fault. I had to squash some commits to simplify the PR to central CMSSW. If you do the following, it should solve the problem. (Back up your code first, just in case it fails ...)

cmsrel CMSSW_20_1_0_pre1
cd CMSSW_20_1_0_pre1/src
cmsenv
git cms-checkout-topic -u cms-L1TK:L1TK-dev-20_1_0_pre1
git cms-rebase-topic -u cms-L1TK:HYBRID_DISPLACED_DR_20_1_0_pre1
(resolve conflicts by hand)
git remote add cms-L1TK git@github.com:cms-L1TK/cmssw.git
git push -f cms-L1TK HYBRID_DISPLACED_DR_20_1_0_pre1

[tomalin]

You're testing on Displaced SUSY + 200PU, which is sensible. But your sample looks different to those listed in https://twiki.cern.ch/twiki/bin/view/CMS/L1TrackMC#CMSSW_20_if_you_want_to_test_PRs . Where did it come from and which geometry is it?

[tomalin]

Hi Alaa, the lines I highlight above need some thought. Deleting them as you originally did seems to increase the duplicate rate, (which is not suprising as they were originally introduced to cut it), but you also need them to respect your chosen seed ordering. Please ensure that you understand what makes logical sense and gives best performance here. And of course redo the performance checks for your final code.

Look at displaced L1 track performance on displacedSUSY+200PU reported by CI prior to your PR, which you can see via my dummy PR #378 . Then look at the corresponding performance reproted by CI for this DR PR. You can see that the performance is almost identical. I suspect this is not because the seed ordering has no effect, but rather because your treatment of the lines of code I highlight above has the effect of ignoring the displaced seed ordering.

[A-A-Abdelhamid]

You're testing on Displaced SUSY + 200PU, which is sensible. But your sample looks different to those listed in https://twiki.cern.ch/twiki/bin/view/CMS/L1TrackMC#CMSSW_20_if_you_want_to_test_PRs . Where did it come from and which geometry is it?

The plots and performance results I reported were produced in CMSSW_15_1_0_pre4, using the Phase2Spring24 dataset:
/DisplacedSUSY_stopToBottom_M-800_50mm_TuneCP5_14TeV-pythia8/Phase2Spring24DIGIRECOMiniAOD-PU200_AllTP_140X_mcRun4_realistic_v4-v1/GEN-SIM-DIGI-RAW-MINIAOD

That sample uses the D110 geometry.

The only displacedcSUSY sample with pileup listed in the linked Twiki is:
/RelValDisplacedSUSY_14TeV/CMSSW_20_0_0_pre1-PU_150X_mcRun4_realistic_v1_STD_D121_RegeneratedGS_PU-v1/GEN-SIM-DIGI-RAW

This sample uses the D121 geometry and contains 9,000 events with sTop mass of 800 GeV and cτ=500 mm. I am currently producing ntuples from it using both the default and optimized seed rankings for an updated comparison in CMSSW_20_1_0_pre1.

[A-A-Abdelhamid]

Hi Alaa, the lines I highlight above need some thought. Deleting them as you originally did seems to increase the duplicate rate, (which is not suprising as they were originally introduced to cut it), but you also need them to respect your chosen seed ordering. Please ensure that you understand what makes logical sense and gives best performance here. And of course redo the performance checks for your final code.

Look at displaced L1 track performance on displacedSUSY+200PU reported by CI prior to your PR, which you can see via my dummy PR #378 . Then look at the corresponding performance reproted by CI for this DR PR. You can see that the performance is almost identical. I suspect this is not because the seed ordering has no effect, but rather because your treatment of the lines of code I highlight above has the effect of ignoring the displaced seed ordering.

Thanks a lot, Ian. I am investigating the interaction between the displaced-seed ordering and the duplicate-removal logic, including the lines you highlighted. I want to make sure I understand the behavior before making another change. Once I have determined the logically consistent implementation, I will rerun the performance checks with the final code and report the results.

[A-A-Abdelhamid]

Hi @tomalin . I have addressed the two remaining duplicate-removal points in b209ebe on 24 July. I re-ran a controlled three-configuration displaced-tracking comparison.

Code changes

The updated logic in PurgeDuplicate.cc now does the following:

1. When two displaced tracks have different seed types, the survivor follows the configured displaced-seed ranking.
2. The legacy reversed preference is retained only when the two displaced tracks have the identical seed type.
3. Displaced tracks are not removed prematurely from the candidate sequence, so they remain available for subsequent duplicate comparisons.

The ordering currently used in the PR is:

L2L3L4 > L4L5L6 > L2D1D2 > L2L3D1

To isolate the effect of the first two priorities, I also tested an Alternative configuration:

L4L5L6 > L2L3L4 > L2D1D2 > L2L3D1

Optimized and Alternative use the same final duplicate-removal implementation and differ only by exchanging the ranks of seed IDs 8 and 9.

Validation sample and definitions

All configurations processed the same 9,000-event D121 sample:

/RelValDisplacedSUSY_14TeV/CMSSW_20_0_0_pre1-PU_150X_mcRun4_realistic_v1_STD_D121_RegeneratedGS_PU-v1/GEN-SIM-DIGI-RAW

Default uses the dev-branch PurgeDuplicate.cc; Optimized uses the final implementation after b209ebe; Alternative changes only the two ranks above.

Performance

Quantity	Default	Optimized	Alternative
Combined efficiency	80.55%	80.64%	80.65%
Efficiency, pT > 8 GeV	73.78%	73.88%	73.91%
Efficiency, pT > 40 GeV	57.44%	57.49%	57.70%
Central 99% d0 interval	0.19 cm	0.20 cm	0.20 cm
Forward 99% d0 interval	0.38 cm	0.38 cm	0.38 cm
Central 99% z0 interval	0.62 cm	0.69 cm	0.64 cm

Default-versus-PR differences include both the duplicate-removal changes and the seed-ranking behavior. Optimized-versus-Alternative is the controlled ranking-only comparison.

The controlled comparison shows that the new code implements the requested seed-ranking behavior: swapping L2L3L4 and L4L5L6 produces a correspondingly large exchange in the surviving seed composition, including among valid matched tracks.

The Alternative ordering gives no clear overall efficiency improvement in this 9,000-event sample, while it produces slightly more tracks and a higher duplicate fraction. I therefore kept the Optimized ordering in the PR. Because all configurations process the same events and I have not calculated the paired covariance, the small performance differences should be treated as descriptive rather than significance claims.

This sample uses D121 and 9,000 events of ct = 500 mm, whereas the earlier PR study used D110 and 10,000 events of ct= 50mm, so the absolute values should not be compared directly across those productions.

[tomalin]

Please quote also the duplicate track fraction in your results, since that is obviously an important parameter that could be modified by changes in the DR algo, and should influence the choice of the seeding order.

[tomalin]

The git CI, which tests the displaced tracking HYBRID_DISPLACED using 1k events of the same MC dataset that you, and inside L1TrackNTuplePlot.C sets "useDisplacedTrkCuts = true" gives these results:

• After your PR: https://gitlab.cern.ch/cms-l1tk/cmssw_CI/-/jobs/79296949
  Efficiency = 83.73%, duplicate rate = 5.41%

• Before your PR: https://gitlab.cern.ch/cms-l1tk/cmssw_CI/-/jobs/78693285
  Efficiency = 83.70%, duplicate rate = 5.17%

Therefore it is possible that your preferred seed order increases the duplicate rate, so given the small increase in efficiency you observe, it makes me wonder whether your studies actually prefer the original ordering?

I also don't understand why the git CI is recording higher efficiency that in the results you quoted above. Are you getting your "combined efficiency" from the summary printout produced by L1TrackNTuplePlot.C (for pt > 2), or in some other way?

[A-A-Abdelhamid]

The git CI, which tests the displaced tracking HYBRID_DISPLACED using 1k events of the same MC dataset that you, and inside L1TrackNTuplePlot.C sets "useDisplacedTrkCuts = true" gives these results:

• After your PR: https://gitlab.cern.ch/cms-l1tk/cmssw_CI/-/jobs/79296949
  Efficiency = 83.73%, duplicate rate = 5.41%
• Before your PR: https://gitlab.cern.ch/cms-l1tk/cmssw_CI/-/jobs/78693285
  Efficiency = 83.70%, duplicate rate = 5.17%

Therefore it is possible that your preferred seed order increases the duplicate rate, so given the small increase in efficiency you observe, it makes me wonder whether your studies actually prefer the original ordering?

I also don't understand why the git CI is recording higher efficiency that in the results you quoted above. Are you getting your "combined efficiency" from the summary printout produced by L1TrackNTuplePlot.C (for pt > 2), or in some other way?

The current L1TrackNTuplePlot.C has a bug, it still uses the prompt cuts even when when it sets "useDisplacedTrkCuts = true", this is due to lines 82-85 being commented out

[A-A-Abdelhamid]

@tomalin
Now that L1TrackNTuplePlot.C is actually using the displaced cuts, the original PurgeDuplicate.cc before my PR fails the CI:

https://gitlab.cern.ch/cms-l1tk/cmssw_CI/-/jobs/81371374#L1300

FAILURE -- TRACKING EFFICIENCY TOO LOW 80.60 < 83.0

[A-A-Abdelhamid]

The gain in efficiency we get is not statistically significant and the gain in resolution is marginal, and this comes with the trade-off of increasing the duplicate fraction by 0.19%.

Thus we can keep the DR code in PurgeDuplicate.cc and (oldKF) HYBRID_DISPLACED as is.