Multinomial softmax parameters are unidentified up to a per-class shift; sklearn centres them, Flow does not

[godofecht]

From the substantive-disparity survey. LogisticRegression/digits shows intercepts_max_relative_diff = 0.508 and coef_row_l2_norms_max_relative_diff = 0.0125 even though accuracy matches sklearn within 3e-3 and #434 matched the objective.

Softmax is invariant to adding a constant vector to all class scores: (W, b) and (W + c·1ᵀ, b + d) produce identical probabilities. The parameters are only identified up to that shift. sklearn's multinomial fit centres coefficients and intercepts to sum to zero across classes; Flow reports the raw optimum its LBFGS landed on.

So the reported state difference is (at least partly) a parameterization artifact, not a model difference. Fix: centre Flow's multinomial coefficients and intercepts (subtract the class-mean from each feature column and from the intercept vector) at the end of lin433_multinomial_fit. Predictions and probabilities are mathematically invariant — verify bit-level effect on canonical rows and report; if predictions move in the last ulp from the arithmetic, report exactly which records.

iris shows the same pattern smaller (intercepts_max_relative_diff 0.0009).

[godofecht]

PR #476 implements the centring. Reporting what the measurements say, because they do not match the premise.

Flow's multinomial fit was already producing sum-to-zero parameters before the change. Largest absolute sum across classes on the canonical fits, against largest parameter 2.49 (iris) and 2.10 (digits):

	coefficient column	intercept vector
iris, before	1.01e-06	9.24e-07
digits, before	1.10e-05	6.30e-06
digits, after	1.38e-07	1.49e-07

sklearn's digits fit sits in the same place: 4.62e-06 and 1.11e-05. Both sides were already reporting the same representative.

The reason is structural. theta starts at zero and the softmax cross-entropy gradient sums to zero across classes, since sum_c (p_ic - t_ic) = 0. The L2 term is linear in theta and preserves that. Every LBFGS direction is a linear combination of gradients and earlier steps, so each iterate stays in the sum-to-zero subspace it started in. What was left is f32 accumulation drift, which is what the centring removes.

The digits intercept gap is not a shift. Flow minus sklearn, class order 0..9:

0.04835  -0.13273  0.01106  0.00637  -0.11226
0.10810  -0.11728  0.10993  0.11350  -0.03506

Spread 0.246, std 0.092, mean -1.7e-06. A per-class shift would be one constant repeated ten times.

Recomputing the fitted objective on the same scaled split, mean CE + 0.5alpha||W||^2 with alpha = 1/(C m): Flow 0.066468071, sklearn 0.066514768, gradient norms 1.25e-04 and 4.82e-04. Flow's objective is 4.7e-05 below sklearn's. Neither is exactly at the optimum, and the problem is flat enough at C=1 that two points 0.13 apart in an intercept differ by 5e-05 in objective.

So intercepts_max_relative_diff = 0.508 is the solver-path difference the row already declares (max_iter 200 capped at 100, no line search, against sklearn's 1000 with one) amplified by that flatness. After centring it reads 0.5079072 against 0.5079057 before. It does not drop, and it should be read as a convergence difference rather than a parameterization artifact.

The centring is kept anyway: it makes the gauge a property of the returned model rather than of the starting point, which a warm start or a different initialisation would otherwise break, and tests/test_multinomial_gauge.flow pins it. Predictions do not move: zero label flips on 30 iris and 360 digits test rows, largest softmax movement 8.3e-07, both benchmark accuracies byte-identical.