Skip to content

fix: reorient KernelSVC per-pair intercepts to libsvm's convention and align iris max_iter - #475

Merged
godofecht merged 1 commit into
mainfrom
fix/471-kernelsvc-conventions
Aug 21, 2026
Merged

fix: reorient KernelSVC per-pair intercepts to libsvm's convention and align iris max_iter#475
godofecht merged 1 commit into
mainfrom
fix/471-kernelsvc-conventions

Conversation

@godofecht

@godofecht godofecht commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Closes #471

Two entangled items on the KernelSVC rows. Neither is a fitted-model change: all 19 RESULT| records are identical before and after.

Item 1: the intercept relationship is pair orientation, sign is a consequence

It is not a global sign flip. On iris one pair of three is negated; on digits, 19 of 45. The rule behind which ones is exact.

libsvm/sklearn's convention, verified

Checked against sklearn 1.9.0 with decision_function_shape='ovo' on iris (gamma=0.25, C=1.0) rather than from recollection:

pair(0,1) mean dec for class0:  1.1554846880   class1: -1.1553219431
pair(0,2) mean dec for class0:  1.0876953861   class2: -1.0603767484
pair(1,2) mean dec for class1:  1.1814879710   class2: -1.1994213056

For pair (i, j) with i < j, the numerically smaller label is the +1 side, and intercept_[p] is the constant term of that same decision function.

What Flow does

kernel_svc_multi_fit in lib/scikit/svm.flow forms pairs over class-discovery slots, not labels:

if y[i] == classes[a] { y_dual[idx] = 1.0 } else { y_dual[idx] = -1.0 }

a < b indexes classes, which distinct_f32 fills in order of first appearance in y_train. kernel_svc_multi_predict votes for pair_a on a positive decision value, so Flow agrees with itself. It agrees with libsvm only when discovery order happens to be ascending by label.

Instrumented fit on the canonical iris split:

class_slot 0 label 0.0
class_slot 1 label 2.0
class_slot 2 label 1.0

raw_pair 0  plus_label 0.0  minus_label 2.0  b -0.134948671
raw_pair 1  plus_label 0.0  minus_label 1.0  b  0.012464760
raw_pair 2  plus_label 2.0  minus_label 1.0  b -0.017334666   <-- +1 side is the HIGHER label

Discovery order is [0, 2, 1]. Pair (2,1) comes out with 2 as +1, so its entire decision function, intercept included, is the negative of libsvm's.

msd207_ksvc_pair_order in benchmarks/bench_flow_v2.flow already sorted the emitted pairs into ascending (lo, hi) label order, which is why pair_class_a/pair_class_b matched exactly. It did not reorient the pair, so the intercept shipped with Flow's orientation.

Option chosen: (b), transform at emission

The solver is left alone. svm.flow is 76KB and the known victim of compiler bug #469, and the sign lives in the y_dual encoding whose orientation is read back by kernel_svc_multi_predict, kernel_svc_multi_decision_function, and the coef_all scatter. Flipping it would have to be verified through all three to keep predictions bit-identical, for no gain in the fitted model. msd207_ksvc_state now negates the emitted intercept for pairs whose pair_a carries the higher label, with the mapping documented at the site.

First three pairs, old and new

iris, emitted order (0,1), (0,2), (1,2):

idx pair old new sklearn
0 (0,1) 0.0124647599 0.0124647599 0.012601528
1 (0,2) -0.134948671 -0.134948671 -0.13494871
2 (1,2) -0.0173346661 0.0173346661 0.0172924496

digits, first three pairs (0,1), (0,2), (0,3): -0.709698141, -0.740054965, -0.481934667, unchanged, all three already in libsvm orientation. The first digits pair that moves is index 4, pair (0,5): 0.592164516 becomes -0.592164516 against sklearn's -0.591559288. In full, digits flips at indices 4, 12, 14, 19, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 39, 40, 41, 44 and is unchanged at the other 26.

Model-state diagnostics, regenerated locally

Both benchmark runs on this machine, one commit apart, fed through generate_disparity_report.enrich_state_from_raw_details against the committed sklearn_results_v2.txt.

row field before after
iris intercept_per_pair_max_abs_diff 0.0346271157 0.00013676810
iris intercept_per_pair_max_relative_diff 1.9975646199 0.0108532949
digits intercept_per_pair_max_abs_diff 1.521018986 0.0011383803
digits intercept_per_pair_max_relative_diff 1.9999505583 0.0714754582

intercept_per_pair_first_divergent_index stays 0 on iris and 1 on digits. It does not clear, and I want to be plain about that rather than claim it did. Its tolerance is 1e-9 + 1e-6 * max(|a|, |b|), and the residual after reorientation is 1.4e-4 (iris) and 1.1e-3 (digits) of f32-solver-versus-f64-libsvm noise. That is the same noise floor that already pins dual_coef_abs_sum_per_pair_first_divergent_index at 0 and 1, which the issue itself reads as agreement ("dual-coefficient sums match to 6.5e-5"). What changes is that the intercept vector no longer carries a signal 4 orders of magnitude above that floor.

The rows remain model_state_diverges: True either way. The hits that survive, and are genuine:

iris    dual_coef_abs_sum_per_pair_first_divergent_index = 0
        dual_coef_abs_sum_per_pair_max_relative_diff     = 4.46e-05
digits  dual_coef_abs_sum_per_pair_first_divergent_index = 1
        dual_coef_abs_sum_per_pair_max_relative_diff     = 1.89e-04
        n_support_per_pair_first_divergent_index         = 4
        n_bounded_support_per_pair_first_divergent_index = 14
        n_support_total_relative_diff                    = 6.24e-04

Item 2: iris max_iter 200 vs 1000

_kernel_svc_smo_precomputed sets budget = max_iter * n and counts individual SMO pair updates, so max_iter is a sweep-equivalent multiplier rather than a sweep count. Instrumented on the canonical iris split (n = 80 per pair), cap swept over {1, 2, 3, 5, 10, 26, 50, 200, 1000, 5000}:

budget     80 -> steps 39, 30, 80
budget    160 -> steps 39, 30, 80
budget    240 -> steps 39, 30, 80
budget    400 -> steps 39, 30, 80
budget    800 -> steps 39, 30, 80
budget   2080 -> steps 39, 30, 80
budget   4000 -> steps 39, 30, 80
budget  16000 -> steps 39, 30, 80    (max_iter = 200, the value being replaced)
budget  80000 -> steps 39, 30, 80    (max_iter = 1000, sklearn's declared value)
budget 400000 -> steps 39, 30, 80

Every pair exits on the libsvm KKT rule gmax + gmax2 < tol, never on the budget. The worst pair needs 80 steps, one sweep-equivalent, against a declared budget of 16000, so the cap has 200x headroom. Accuracy and all three intercepts are bit-identical across the whole sweep: 0.966666639 and -0.134948671 / 0.012464760 / -0.017334666. The declaration is now 1000 in bench_flow_v2.flow and in parity_contract.json.

Changing only the flow block of that contract row is what check_disparity_regression.implementation_contract_resets allows: every gate-defining field outside flow/sklearn is unchanged, so the row establishes a fresh disparity baseline instead of tripping the gate. It also removes a configuration_differences entry, and configuration_difference_count is one-sided at max_increase: 0.

The instrumentation for both items was a temporary printf in _kernel_svc_smo_precomputed plus a scratch harness. Both were reverted; lib/ is byte-identical to main in this branch, and git diff --name-only origin/main touches nothing under lib/.

Verification

bench_flow_v2.flow run on origin/main and on this branch, same machine, FLOW_HOST=python FLOW_OPT_LEVEL=0 FLOW_LDFLAGS=-framework Accelerate:

RESULT rows: 19 19
RESULT metric fields identical: True
DETAIL rows: 146 146
DETAIL keys changed: 2
   KernelSVC_RBF|iris|intercept_per_pair
   KernelSVC_RBF|digits|intercept_per_pair

Both changed records differ from their predecessors by sign alone, at exactly the predicted indices. The other 144 DETAIL records are byte-identical.

Per #471's instruction on #469, with lib/ untouched:

tests/test_multiclass_kernel_svm.flow  Multi-class KernelSVC accuracy = 0.9667   exit 0
tests/test_multiclass_svm.flow         Multi-class LinearSVC accuracy = 0.9778   exit 0

Both match main.

No generated artifacts are in this diff. Other agents were compiling into the shared ~/.local/bin/build concurrently; the scratch harness used a _diag471_ prefix and has been deleted.

Closed-form confirmation of the mechanism

Reading the class-discovery order straight out of the committed split fixtures, forming Flow's slot pairs, sorting them the way msd207_ksvc_pair_order does, and marking every pair whose pair_a label exceeds its pair_b label reproduces both observed flip sets exactly, without running Flow at all:

iris   discovery order [0, 2, 1]                      predicted flipped emitted indices [2]
digits discovery order [5, 0, 7, 1, 2, 9, 8, 6, 3, 4] predicted flipped emitted indices
       [4, 12, 14, 19, 21, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 39, 40, 41, 44]

Those are the same 1 and 19 indices the benchmark diff moves. Nothing else about the mechanism is left to inference.

Interaction with #474

#474 landed on main after this branch was cut. It introduced declared configuration equivalences but deliberately left max_iter out of them, on the grounds that a parameter both sides record should always be compared. Its own before/after table lists KernelSVC_RBF / iris as surviving with max_iter flow=200 sklearn=1000, the last remaining entry on that row.

This change removes that entry the way #474 intended it to be removed: by making the declaration true rather than by exempting it. rows_with_configuration_difference goes 5 -> 4, and the gate is one-sided at max_increase: 0, so a decrease cannot trip it. The two diffs share no files.

…d align iris max_iter

kernel_svc_multi_fit pairs classes by discovery slot, so a pair whose slot
order runs descending by label gets the higher label as its +1 side. libsvm
always makes the lower label +1 for pair (i, j), i < j, so those pairs'
decision functions, and therefore their intercepts, are the negative of
sklearn's. Flow is internally consistent either way: predict votes for
pair_a on a positive decision value, which is why predictions and dual
coefficient sums already agreed.

msd207_ksvc_state already sorted the emitted pairs into ascending label
order but did not reorient them. It now negates the intercept for pairs
whose pair_a carries the higher label. The solver is untouched, keeping
svm.flow clear of compiler bug #469.

iris declared max_iter 200 against sklearn's 1000. max_iter multiplies into
an SMO step budget of max_iter * n; every iris pair exits on the libsvm KKT
rule after at most 80 steps, one sweep-equivalent at n = 80. Fits are
bit-identical for every cap from 1 to 5000, so the declaration is aligned to
1000 in both bench_flow_v2.flow and parity_contract.json.

All 19 RESULT records identical. 144 of 146 DETAIL records identical; the
two that move are the intercept_per_pair vectors, by sign only.

Closes #471

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@godofecht
godofecht merged commit 0fe8bd2 into main Aug 21, 2026
10 checks passed
@godofecht
godofecht deleted the fix/471-kernelsvc-conventions branch August 21, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KernelSVC per-pair intercepts differ from sklearn by a sign convention; iris max_iter declared 200 vs 1000

1 participant