856 citations · 1.1k across the 51 of their papers we have counts for
12 papers · 1 filter
Fundamental performance bounds on time-series generation using reservoir computing
Daoyuan Qian, Ila Fiete
Reservoir computing (RC) harnesses the intrinsic dynamics of a chaotic system, called the reservoir, to perform various time-varying functions. An important use-case of RC is the g…
Permutation Invariant Learning with High-Dimensional Particle Filters
Akhilan Boopathy, Aneesh Muppidi, Peggy Yang +3
Sequential learning in deep models often suffers from challenges such as catastrophic forgetting and loss of plasticity, largely due to the permutation dependence of gradient-based…
Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification
Jaedong Hwang, Brian Cheung, Zhang-Wei Hong +3
Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of maintaining the general performance of the mo…
Unified Neural Network Scaling Laws and Scale-time Equivalence
Akhilan Boopathy, Ila Fiete
As neural networks continue to grow in size but datasets might not, it is vital to understand how much performance improvement can be expected: is it more important to scale networ…
Breaking Neural Network Scaling Laws with Modularity
Akhilan Boopathy, Sunshine Jiang, William Yue +3
Modular neural networks outperform nonmodular neural networks on tasks ranging from visual question answering to robotics. These performance improvements are thought to be due to m…
How Diffusion Models Learn to Factorize and Compose
Qiyao Liang, Ziming Liu, Mitchell Ostrow +1
Diffusion models are capable of generating photo-realistic images that combine elements which likely do not appear together in the training set, demonstrating the ability to \texti…
Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models
Sunny Duan, Mikail Khona, Abhiram Iyer +2
Frontier AI systems are making transformative impacts across society, but such benefits are not without costs: models trained on web-scale datasets containing personal and private…
Towards Exact Computation of Inductive Bias
Akhilan Boopathy, William Yue, Jaedong Hwang +2
Much research in machine learning involves finding appropriate inductive biases (e.g. convolutional neural networks, momentum-based optimizers, transformers) to promote generalizat…
Delay Embedding Theory of Neural Sequence Models
Mitchell Ostrow, Adam Eisen, Ila Fiete
To generate coherent responses, language models infer unobserved meaning from their input text sequence. One potential explanation for this capability arises from theories of delay…
Resampling-free Particle Filters in High-dimensions
Akhilan Boopathy, Aneesh Muppidi, Peggy Yang +3
State estimation is crucial for the performance and safety of numerous robotic applications. Among the suite of estimation techniques, particle filters have been identified as a po…
Bridging Associative Memory and Probabilistic Modeling
Rylan Schaeffer, Nika Zahedi, Mikail Khona +9
Associative memory and probabilistic modeling are two fundamental topics in artificial intelligence. The first studies recurrent neural networks designed to denoise, complete and r…
Do Diffusion Models Learn Semantically Meaningful and Efficient Representations?
Qiyao Liang, Ziming Liu, Ila Fiete
Diffusion models are capable of impressive feats of image generation with uncommon juxtapositions such as astronauts riding horses on the moon with properly placed shadows. These o…