3 citations · 4 across the 7 of their papers we have counts for
9 papers
Sample-efficient learning with Dense Associative Memories
Andrew Lucas, Dmitry Krotov
We analytically study the evolution of the energy function of Dense Associative Memories (DenseAMs) as they train on randomly sampled codewords of random expander low-density parit…
Energy-Based Dynamical Models for Neurocomputation, Learning, and Optimization
Arthur N. Montanari, Francesco Bullo, Dmitry Krotov +1
Recent advances at the intersection of control theory, neuroscience, and machine learning have revealed novel mechanisms by which dynamical systems perform computation. These advan…
NRGPT: An Energy-based Alternative for GPT
Nima Dehmamy, Benjamin Hoover, Bishwajit Saha +3
Generative Pre-trained Transformer (GPT) architectures are the most popular design for language modeling. Energy-based modeling is a different paradigm that views inference as a dy…
Modern Methods in Associative Memory
Dmitry Krotov, Benjamin Hoover, Parikshit Ram +1
Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information.…
Dense Associative Memory with Epanechnikov Energy
Benjamin Hoover, Zhaoyang Shi, Krishnakumar Balasubramanian +2
We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum…
Small Models, Smarter Learning: The Power of Joint Task Training
Csaba Both, Benjamin Hoover, Hendrik Strobelt +4
Multi-task learning improves generalization, but when does it reduce the model capacity required to learn? We provide a systematic study of how joint training affects the learning…
Dense Associative Memory Through the Lens of Random Features
Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt +2
Dense Associative Memories are high storage capacity variants of the Hopfield networks that are capable of storing a large number of memory patterns in the weights of the network o…
Neuron-Astrocyte Associative Memory
Leo Kozachkov, Jean-Jacques Slotine, Dmitry Krotov
Astrocytes, the most abundant type of glial cell, play a fundamental role in memory. Despite most hippocampal synapses being contacted by an astrocyte, there are no current theorie…
Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
Benjamin Hoover, Hendrik Strobelt, Dmitry Krotov +3
The generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks. Though the generative process is traditionally understood as an…