Stronger per-modality estimators: elastic-net + linear SVM - #3
Merged
Conversation
added 2 commits
August 11, 2026 07:39
Two estimators, both fitting the existing bagged + scaled + SMOTE pipeline and both exposing predict_proba (which the ensemble needs to weight on out-of-fold probabilities): - elasticnet: L1+L2 logistic via the saga solver. The L2 part shares weight across correlated features (NAcc_L/NAcc_R move together; band-power columns are collinear) while L1 still drops dead ones. A better regulariser than plain L2 for the fMRI and EEG blocks and than plain L1 for high-dimensional face. - linear_svm: linear-kernel SVC with Platt-scaled probabilities, a strong baseline for high-dimensional embeddings. Defaults are unchanged in this commit; the next one switches them. 10 tests cover instantiation, predict_proba in the pipeline on imbalanced data, kwargs override, and the unknown-name error.
Switch fMRI, EEG, and face from plain L2 logistic to elasticnet, the regulariser that actually suits their feature structure: L2 shares weight across correlated ROI betas and collinear band-power columns, L1 still drops dead features. Behaviour stays on gradient boosting. Configs updated to match. face defaults to elasticnet rather than linear_svm purely for speed: linear_svm is comparable in accuracy but ~3x slower because of its probability calibration, so it is documented as the one-line alternative rather than the default. docs/estimators.md records the rationale and, importantly, an honest account of the two stronger models that were suggested and do NOT drop into this pipeline unchanged: - mixed-effects: under subject-grouped CV every test subject is unseen, so a per-subject random intercept falls back to the population mean at test time and the partial-pooling benefit largely evaporates for held-out-subject generalisation, which is what we report. - Riemannian: covariance features live on a manifold that per-column StandardScaler and straight-line SMOTE both corrupt, so it needs its own feature family and pipeline, not a base-learner swap. Both demos still pass (synthetic 4/4, NARPS fMRI 0.81 balacc). 129 tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the per-modality estimator upgrades, in two commits. Merges on top of DEAP (#1) and NARPS (#2).
What's shipped
Two new base learners, both fitting the existing bagged + scaled + in-fold-SMOTE pipeline and both exposing
predict_proba(which the ensemble needs to weight on out-of-fold probabilities):elasticnet— L1+L2 logistic (saga). L2 shares weight across correlated features, L1 still drops dead ones. Better than plain L2 for collinear ROI betas and band-power columns, better than plain L1 for high-dim face embeddings.linear_svm— linear-kernel SVC with Platt-scaled probabilities.Defaults changed: fMRI, EEG, and face move from plain logistic to
elasticnet; behaviour stays on gradient boosting. Configs updated. Both demos still pass (synthetic 4/4; NARPS fMRI 0.81 balanced accuracy).facedefaults toelasticnetrather thanlinear_svmpurely for speed — linear SVM is comparable in accuracy but ~3× slower (probability calibration), so it's the documented one-line alternative.The two suggestions I did not drop in, and why
Both were good calls for the right setup, and both would produce numbers that look principled but aren't if forced into this framework's flat-feature, subject-grouped-CV pipeline.
docs/estimators.mdrecords the reasoning so it's a deliberate decision, not an oversight:Mixed-effects / hierarchical (fMRI) — the framework groups CV folds by subject, so every test subject is unseen. A per-subject random intercept has no estimate for an unseen subject and falls back to the population mean at test, which is what a pooled model already gives. The partial-pooling benefit largely evaporates for held-out-subject generalisation, which is the quantity we report. It would help only under within-subject evaluation, which we deliberately avoid (individuating a person from their own trials is a weaker claim than generalising across people). Documented with a recipe for anyone who wants it as a reported-alongside option.
Riemannian (EEG) — a real upgrade for covariance/connectivity features, but two pipeline steps corrupt those features: per-column
StandardScalerdestroys the positive-definite structure, and straight-line SMOTE interpolation is off-manifold (the geodesic between two covariances isn't a straight line in entry space). It needs its own feature family (per-trial covariance) and its own tangent-space pipeline, not a base-learner swap. Laid out as the concrete next step.Tests
tests/test_estimators.py(10 new): each learner instantiates correctly, exposespredict_proba, fits inside the leakage-safe pipeline on imbalanced data, honours kwargs, and errors clearly on unknown names.Happy to build the Riemannian EEG path as a follow-up PR if you want it — it's the more interesting of the two, and I'd do it as a proper covariance-feature + tangent-space arm rather than shoehorning it in here.