Possible MP2020 correction leakage in the bundled TRI2024 reference dataset

[kwon2048]

Possible MP2020 correction leakage in the bundled TRI2024 reference dataset

AI-assisted drafting: GPT-5.6 Sol.

Summary

We found a systematic inconsistency between MatterGen's bundled
reference_MP2020correction.gz and reference_TRI2024correction.gz datasets.
When the same MatterSim energies are evaluated against the two references, the
TRI2024 energy above hull is not simply the expected 1.108-scaled MP2020 value.
Instead, the difference contains an additional term that matches exactly 10.8%
of the MP2020 oxygen and GGA/GGA+U mixing corrections.

The numerical agreement is at approximately machine precision for the tested
non-U oxides. For V, Mo, and W oxides, the remaining offset is also reproduced
exactly by 10.8% of the corresponding MP2020 U-mixing correction.

This strongly suggests that the bundled TRI2024 reference may have been built by
applying the TRI scaling term to entry.energy while that property still
included MP2020 adjustments. If so, removing/replacing the original adjustments
afterward would still leave 10.8% of the MP2020 corrections embedded in the TRI
reference energies.

Setup

• MatterSim model: MatterSim-v1.0.0-5M
• Structures: 25 experimentally established oxide compositions from Materials Project
• Relaxation: cell and atomic positions, fmax = 0.01 eV/Angstrom
• MP2020 reference: reference_MP2020correction.gz
• TRI2024 reference: reference_TRI2024correction.gz
• Ehull path: MatterGen EnergyMetricsCapability.energy_above_hull
• Novelty, uniqueness, and structure matching were not evaluated
• The same relaxed MatterSim total energy was used for both correction schemes

All 25 selected structures had MP2020 Ehull below 0.01 eV/atom.

Observed relation for non-U oxides

After excluding V-, Mo-, and W-containing entries, the results fall on parallel
lines with slope 1.108. The intercept depends only on oxygen atomic fraction:

$$ E_{\mathrm{hull}}^{\mathrm{TRI}} =1.108E_{\mathrm{hull}}^{\mathrm{MP2020}}+b(x_O) $$

The fitted intercepts are:

Oxygen fraction (x_O)	Number of materials	Fitted intercept (eV/atom)
0.333333	5	0.024732
0.500000	5	0.037098
0.600000	5	0.044518
0.666667	4	0.049464
0.714286	3	0.052997

These values satisfy

$$ b(x_O)=0.074196x_O =0.108\times0.687\times x_O, $$

where -0.687 eV/O is the MP2020 oxide correction. Within each oxygen-fraction
group, the residual scatter around the fixed-slope line is approximately
(10^{-14}) eV/atom.

Figure 1: non-U oxides

Figure 1 intentionally excludes V-, Mo-, and W-containing materials to isolate
the oxygen-fraction dependence before introducing GGA+U corrections.

Additional offsets for MP2020 GGA+U elements

For V2O5, MoO3, and WO3, the additional displacement from the oxygen-only line
is

$$ \Delta_U=0.108\left|C_U^{\mathrm{MP2020}}\right|x_U, $$

where (x_U) is the atomic fraction of the U-corrected transition metal.

Material	MP2020 U-mixing correction	Metal fraction	Predicted extra offset	Observed extra offset
V2O5	-1.700 eV/V	2/7	0.052457 eV/atom	0.052457 eV/atom
MoO3	-3.202 eV/Mo	1/4	0.086454 eV/atom	0.086454 eV/atom
WO3	-4.438 eV/W	1/4	0.119826 eV/atom	0.119826 eV/atom

Combining the oxygen and U-mixing terms gives the complete observed relation:

$$ E_{\mathrm{hull}}^{\mathrm{TRI}} -1.108E_{\mathrm{hull}}^{\mathrm{MP2020}} =-0.108\left(x_OC_O^{\mathrm{MP2020}}+x_UC_U^{\mathrm{MP2020}}\right). $$

Both MP2020 corrections are negative, so the observed Ehull displacement is
positive.

Why this points to the TRI reference rather than the sample-side evaluator

For a correction-free sample entry, the current TRI implementation applies

$$ E_{\mathrm{sample}}^{\mathrm{TRI}} =1.108E_{\mathrm{raw}}+N_UC_U^{\mathrm{TRI}}. $$

There is no MP2020 oxygen or U-mixing adjustment on the MatterSim sample entry,
so the sample-side path cannot generate the observed MP2020-dependent terms.

However, if the TRI reference was generated from an entry for which
entry.energy already contained MP2020 adjustments, the PBE scaling adjustment
would be calculated as

$$ 0.108\left( E_{\mathrm{raw}} +N_OC_O^{\mathrm{MP2020}} +N_UC_U^{\mathrm{MP2020}} \right). $$

After replacing the original MP2020 adjustments with TRI adjustments, the
reference energy would still contain

$$ 0.108N_OC_O^{\mathrm{MP2020}} +0.108N_UC_U^{\mathrm{MP2020}}. $$

The resulting reference-side energy would be

$$ E_{\mathrm{ref}}^{\mathrm{TRI}} =1.108E_{\mathrm{raw}} +0.108N_OC_O^{\mathrm{MP2020}} +N_UC_U^{\mathrm{TRI}} +0.108N_UC_U^{\mathrm{MP2020}}. $$

This is exactly the expression required to reproduce all observed offsets.

Expected behavior

If both references are built consistently from correction-free raw energies,
composition-linear corrections should not create an oxygen-fraction-dependent
Ehull intercept. For non-U systems we would expect approximately

$$ E_{\mathrm{hull}}^{\mathrm{TRI}} =1.108E_{\mathrm{hull}}^{\mathrm{raw}}, $$

and therefore no residual term proportional to the MP2020 oxygen correction.
Likewise, a residual proportional to the MP2020 GGA/GGA+U mixing correction
should not remain in the TRI reference.

Questions for the maintainers

1. Was reference_TRI2024correction.gz generated from entries that already had
   MP2020 energy adjustments attached?
2. During reference generation, was the TRI PBE adjustment calculated from
   entry.energy rather than entry.uncorrected_energy?
3. Were existing adjustments cleared before or after calculating the 0.108 PBE
   adjustment?
4. Could the TRI2024 reference be regenerated from correction-free entries, or
   by explicitly using uncorrected_energy for the multiplicative term?

Figure 2: all oxides and the U-related offsets

The attached scatter plot contains all 25 materials. Dashed lines show the
oxygen-only prediction

$$ y=1.108x+0.074196x_O, $$

and the red arrows for V2O5, MoO3, and WO3 show the additional offsets predicted
by 0.108 * abs(MP2020 U correction) * metal atomic fraction.