1 citations · 1 across the 6 of their papers we have counts for
7 papers
Unrolled Flow Models for Reasoning
Faissal Izermine, Hanru Bai, Oscar Davis +1
Flow matching enables language generation in few steps, but whether additional integration steps improve reasoning remains unclear. We prove that a flow parameterized by a two-laye…
Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers
Takuya Ito, Ruchir Puri, Murray Campbell +1
Learning generalizable algorithmic computations remains a challenge for neural networks, as reflected in persistent failures on compositional and length generalization benchmarks.…
The Need for Verification in AI-Driven Scientific Discovery
Cristina Cornelio, Takuya Ito, Ryan Cory-Wright +2
Artificial intelligence (AI) is transforming the practice of science. Machine learning and large language models (LLMs) can generate hypotheses at a scale and speed far exceeding t…
Transformer Circuits Can Realize Clustering Algorithms
Kenneth L. Clarkson, Lior Horesh, Takuya Ito +2
Although transformers are most commonly optimized as statistical sequence models, it is unclear to what extent they can implement and learn exact algorithmic computations. Here, we…
Quantifying artificial intelligence through algorithmic generalization
Takuya Ito, Murray Campbell, Lior Horesh +2
The rapid development of artificial intelligence (AI) systems has created an urgent need for their scientific quantification. While their fluency across a variety of domains is imp…
Learning interpretable positional encodings in transformers depends on initialization
Takuya Ito, Luca Cocchi, Tim Klinger +3
In transformers, the positional encoding (PE) provides essential information that distinguishes the position and order amongst tokens in a sequence. Most prior investigations of PE…
On the generalization capacity of neural networks during generic multimodal reasoning
Takuya Ito, Soham Dan, Mattia Rigotti +2
The advent of the Transformer has led to the development of large language models (LLM), which appear to demonstrate human-like capabilities. To assess the generality of this class…