A curated list of papers of interesting empirical study and insight on deep learning. Continually updating...

Code for Arxiv Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle

MDL Complexity computations and experiments from the paper "Revisiting complexity and the bias-variance tradeoff".

Double Descent results for FCNNs on MNIST, extended by Label Noise (Reconciling Modern Machine-Learning Practice and the Classical Bias–Variance Trade-Off) [Python/PyTorch]..

Explore the double-descent phenomena in the context of system identification. Companion code to the paper (https://arxiv.org/abs/2012.06341):

[ICIAP23] Sparse Double Descent in Vision Transformers: real or phantom threat?

This repository is the official implementation of "Optimization Variance: Delve into the Epoch-Wise Double Descent of DNNs"

Double descent experiments/repros on classical ML models and deep neural nets

[AAAI24] DSD2: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?

Assignments of my CST Part II Deep Neural Networks unit

This project outlines 4 experiments to explore the effects of several settings on the bias-variance tradeoff curve

A Review of Preetum Nakkiran's "More Data Can Hurt for Linear Regression: Sample-wise Double Descent"

ICLR 2022: Phenomenology of Double Descent in Finite-width Neural Networks

Implementation of the double descent Deep Learning phenomenon from the article Grokking: Generalization beyond overfitting.

DSC 261 Responsible Data Science Project

Block-Term Operator Theory: why block-term rank-(L,L,1) neural operators generalize better than CP / Tucker / TT at matched capacity, not by more expressivity but as a tighter inductive bias. A least-squares generalization separation Theta((RL - mu_band) K / n), a complete variance-ordering theorem across all four tensor formats, an adaptive for...

Do sparse, one-class purchase data exhibit double descent? A study of NMF, weighted MF (iALS), and Logistic Matrix Factorization on real e-commerce transactions. Sapienza SML project.

This assignment repository demonstrates the existance of deep double descent phenomena for the deep learning models usually encorporated with a clinical context. For this, a famous ECG dataset, PTBXL has been employed.

Toy dataset to study double descent optimization patterns in machine learning.

Python empirical replication of the Double Descent risk curve in Machine Learning using from-scratch closed-form linear models.