Environment: libauc 2.0.1, torch 2.2.2
Location: libauc/losses/auc.py → AUCMLoss.mean
Issue Description
AUCMLoss uses the following helper to compute class-conditional means in the v2 branch:
def mean(self, tensor):
return torch.sum(tensor) / torch.count_nonzero(tensor)The intent is clear: the input is always a masked tensor such as (y_pred - self.a)**2 * pos_mask, where the other class has already been zeroed out, so count_nonzero is used as a proxy for the class size N_+.
The proxy breaks when a genuine in-class element evaluates to exactly zero. count_nonzero cannot distinguish "zeroed by the mask" from "the sample's own value happens to be 0". A positive sample with y_pred == self.a is silently dropped from the denominator; if every positive sample satisfies this, the denominator becomes 0 and the loss evaluates to NaN.
Practical Reachability
a is a running estimate of the positive-class mean, so y_pred == a becomes exact precisely when the positive scores collapse onto a single value — which is what output saturation produces.
With the recommended setup (LibAUC models default to last_activation='sigmoid'), sigmoid saturates to exactly 1.0 in float32 once the logit exceeds ~17:
import torch
for logit in [15, 16, 17, 20]:
v = torch.sigmoid(torch.tensor(float(logit)))
print(logit, v.item(), v.item() == 1.0)
# 15 0.9999996423721313 False
# 16 0.9999998807907104 False
# 17 1.0 True <- exactly 1.0 from here on
# 20 1.0 TrueLogits above 17 are common in a well-fitted binary classifier. Once all positives saturate to 1.0, a converges to exactly 1.0 and every (y_pred - a)**2 term becomes exactly 0.
The failure mode is therefore triggered by the model training well, making it especially difficult to diagnose.
Minimal Reproducible Example
import torch, warnings
warnings.filterwarnings("ignore")
from libauc.losses.auc import AUCMLoss
# Perfectly separated, saturated scores
s = torch.tensor([0.0, 0.0, 1.0, 1.0]).view(-1, 1)
y = torch.tensor([0., 0., 1., 1. ]).view(-1, 1)
f = AUCMLoss(margin=1.0, version='v2')
with torch.no_grad():
f.a.fill_(0.9); f.b.fill_(0.1); f.alpha.fill_(0.5)
print(float(f(s, y))) # -0.22999998927116394 (not yet converged, normal)
f = AUCMLoss(margin=1.0, version='v2')
with torch.no_grad():
f.a.fill_(1.0); f.b.fill_(0.0); f.alpha.fill_(0.5) # a, b at their optima
print(float(f(s, y))) # nan <- converged, invalid NaN
Step-by-step evaluation for the positive term at a = 1.0:
(y_pred - a)**2 * pos_mask -> [0., 0., 0., 0.]
torch.sum(...) -> 0.0
torch.count_nonzero(...) -> 0 # Should be 2 (the positive count)
0.0 / 0 -> nan # Mathematical ground truth: 0/2 = 0.0A within-class variance of zero is a well-defined quantity; returning NaN is incorrect.
Impact
NaN propagates through backward() into the network weights without recovery, while the training loop continues silently.
Suggested Fix
Pass the mask explicitly and divide by the class size so the denominator no longer depends on element values:
def mean(self, tensor, mask):
return torch.sum(tensor * mask) / torch.sum(mask)This returns 0.0 for the saturated case above, while still returning NaN when a class is genuinely absent from the batch (sum(mask) == 0) — which is mathematically undefined and should remain NaN.
Context
This is a second, independent defect from the one reported in #74 (parenthesis placement in the v2 cross term). Both reside in the v2 branch and can be resolved independently.
Environment: libauc 2.0.1, torch 2.2.2
Location: libauc/losses/auc.py → AUCMLoss.mean
Issue Description
AUCMLoss uses the following helper to compute class-conditional means in the v2 branch:
The intent is clear: the input is always a masked tensor such as (y_pred - self.a)**2 * pos_mask, where the other class has already been zeroed out, so count_nonzero is used as a proxy for the class size N_+.
The proxy breaks when a genuine in-class element evaluates to exactly zero. count_nonzero cannot distinguish "zeroed by the mask" from "the sample's own value happens to be 0". A positive sample with y_pred == self.a is silently dropped from the denominator; if every positive sample satisfies this, the denominator becomes 0 and the loss evaluates to NaN.
Practical Reachability
a is a running estimate of the positive-class mean, so y_pred == a becomes exact precisely when the positive scores collapse onto a single value — which is what output saturation produces.
With the recommended setup (LibAUC models default to last_activation='sigmoid'), sigmoid saturates to exactly 1.0 in float32 once the logit exceeds ~17:
Logits above 17 are common in a well-fitted binary classifier. Once all positives saturate to 1.0, a converges to exactly 1.0 and every (y_pred - a)**2 term becomes exactly 0.
The failure mode is therefore triggered by the model training well, making it especially difficult to diagnose.
Minimal Reproducible Example
A within-class variance of zero is a well-defined quantity; returning NaN is incorrect.
Impact
NaN propagates through backward() into the network weights without recovery, while the training loop continues silently.
Suggested Fix
Pass the mask explicitly and divide by the class size so the denominator no longer depends on element values:
This returns 0.0 for the saturated case above, while still returning NaN when a class is genuinely absent from the batch (sum(mask) == 0) — which is mathematically undefined and should remain NaN.
Context
This is a second, independent defect from the one reported in #74 (parenthesis placement in the v2 cross term). Both reside in the v2 branch and can be resolved independently.