4 papers
An Information-Theoretic Perspective on Overfitting and Underfitting
Daniel Bashir, George D. Montanez, Sonia Sehra +2
We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbit…
The Labeling Distribution Matrix (LDM): A Tool for Estimating Machine Learning Algorithm Capacity
Pedro Sandoval Segura, Julius Lauw, Daniel Bashir +4
Algorithm performance in supervised learning is a combination of memorization, generalization, and luck. By estimating how much information an algorithm can memorize from a dataset…
The Bias-Expressivity Trade-off
Julius Lauw, Dominique Macias, Akshay Trikha +2
Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can ada…
The Futility of Bias-Free Learning and Search
George D. Montanez, Jonathan Hayase, Julius Lauw +3
Building on the view of machine learning as search, we demonstrate the necessity of bias in learning, quantifying the role of bias (measured relative to a collection of possible da…