1 citations · 2 across the 9 of their papers we have counts for
12 papers
Unrolled Flow Models for Reasoning
Faissal Izermine, Hanru Bai, Oscar Davis +1
Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear. We prove that a flow parameterized by a two-laye…
Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers
Takuya Ito, Ruchir Puri, Murray Campbell +1
Learning generalizable algorithmic computations remains a challenge for neural networks, as reflected in persistent failures on compositional and length generalization benchmarks.…
What Time Is It? How Data Geometry Makes Time Conditioning Optional for Flow Matching
Alec Helbling, Sebastian Gutierrez Hernandez, Benjamin Hoover +2
Recent work has shown that models flow matching models can be trained without explicit time conditioning, challenging the standard view that the interpolation time is needed to dis…
Deep Clustering with Associative Memories
Bishwajit Saha, Dmitry Krotov, Mohammed J. Zaki +1
Deep clustering - joint representation learning and latent space clustering - is a well studied problem especially in computer vision and text processing under the deep learning fr…
Modern Methods in Associative Memory
Dmitry Krotov, Benjamin Hoover, Parikshit Ram +1
Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information.…
Transformers Learn Faster with Semantic Focus
Parikshit Ram, Kenneth L. Clarkson, Tim Klinger +2
Various forms of sparse attention have been explored to mitigate the quadratic computational and memory cost of the attention mechanism in transformers. We study sparse transformer…
Transformer Circuits Can Realize Clustering Algorithms
Kenneth L. Clarkson, Lior Horesh, Takuya Ito +2
Although transformers are most commonly optimized as statistical sequence models, it is unclear to what extent they can implement and learn exact algorithmic computations. Here, we…
Dense Associative Memory with Epanechnikov Energy
Benjamin Hoover, Zhaoyang Shi, Krishnakumar Balasubramanian +2
We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum…
Quantifying artificial intelligence through algorithmic generalization
Takuya Ito, Murray Campbell, Lior Horesh +2
The rapid development of artificial intelligence (AI) systems has created an urgent need for their scientific quantification. While their fluency across a variety of domains is imp…
Dense Associative Memory Through the Lens of Random Features
Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt +2
Dense Associative Memories are high storage capacity variants of the Hopfield networks that are capable of storing a large number of memory patterns in the weights of the network o…
Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations
Stephen Carrow, Kyle Harper Erwin, Olga Vilenskaia +5
Recent advances in machine learning have led to a surge in adoption of neural networks for various tasks, but lack of interpretability remains an issue for many others in which an…
Learning interpretable positional encodings in transformers depends on initialization
Takuya Ito, Luca Cocchi, Tim Klinger +3
In transformers, the positional encoding (PE) provides essential information that distinguishes the position and order amongst tokens in a sequence. Most prior investigations of PE…