5 papers
Breaking Neural Network Scaling Laws with Modularity
Akhilan Boopathy, Sunshine Jiang, William Yue +3
Modular neural networks outperform nonmodular neural networks on tasks ranging from visual question answering to robotics. These performance improvements are thought to be due to m…
Permutation Invariant Learning with High-Dimensional Particle Filters
Akhilan Boopathy, Aneesh Muppidi, Peggy Yang +3
Sequential learning in deep models often suffers from challenges such as catastrophic forgetting and loss of plasticity, largely due to the permutation dependence of gradient-based…
Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models
Sunny Duan, Mikail Khona, Abhiram Iyer +2
Frontier AI systems are making transformative impacts across society, but such benefits are not without costs: models trained on web-scale datasets containing personal and private…
Towards Exact Computation of Inductive Bias
Akhilan Boopathy, William Yue, Jaedong Hwang +2
Much research in machine learning involves finding appropriate inductive biases (e.g. convolutional neural networks, momentum-based optimizers, transformers) to promote generalizat…
Resampling-free Particle Filters in High-dimensions
Akhilan Boopathy, Aneesh Muppidi, Peggy Yang +3
State estimation is crucial for the performance and safety of numerous robotic applications. Among the suite of estimation techniques, particle filters have been identified as a po…