28 citations · 38 across the 14 of their papers we have counts for
9 papers · 1 filter
Permutation Invariant Learning with High-Dimensional Particle Filters
Akhilan Boopathy, Aneesh Muppidi, Peggy Yang +3
Sequential learning in deep models often suffers from challenges such as catastrophic forgetting and loss of plasticity, largely due to the permutation dependence of gradient-based…
Unified Neural Network Scaling Laws and Scale-time Equivalence
Akhilan Boopathy, Ila Fiete
As neural networks continue to grow in size but datasets might not, it is vital to understand how much performance improvement can be expected: is it more important to scale networ…
Breaking Neural Network Scaling Laws with Modularity
Akhilan Boopathy, Sunshine Jiang, William Yue +3
Modular neural networks outperform nonmodular neural networks on tasks ranging from visual question answering to robotics. These performance improvements are thought to be due to m…
Towards Exact Computation of Inductive Bias
Akhilan Boopathy, William Yue, Jaedong Hwang +2
Much research in machine learning involves finding appropriate inductive biases (e.g. convolutional neural networks, momentum-based optimizers, transformers) to promote generalizat…
Model-agnostic Measure of Generalization Difficulty
Akhilan Boopathy, Kevin Liu, Jaedong Hwang +3
The measure of a machine learning algorithm is the difficulty of the tasks it can perform, and sufficiently difficult tasks are critical drivers of strong machine learning models.…
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
Rylan Schaeffer, Mikail Khona, Zachary Robertson +5
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever…
How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective
Akhilan Boopathy, Ila Fiete
Recent works have examined theoretical and empirical properties of wide neural networks trained in the Neural Tangent Kernel (NTK) regime. Given that biological neural networks are…
Fast Training of Provably Robust Neural Networks by SingleProp
Akhilan Boopathy, Tsui-Wei Weng, Sijia Liu +3
Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees. However, these techniques can be computationally cos…
Proper Network Interpretability Helps Adversarial Robustness in Classification
Akhilan Boopathy, Sijia Liu, Gaoyuan Zhang +4
Recent works have empirically shown that there exist adversarial examples that can be hidden from neural network interpretability (namely, making network interpretation maps visual…