4 papers
Flow Reasoning Models: Scaling Reasoning Through Iterative Self-Refinement
Alec Helbling, Andrey Bryutkin, Mauro Martino +2
Discrete flow models have recently shown promising performance on few-step text generation; however, when naively applied to structured reasoning tasks such as Sudoku and Zebra puz…
Small Models, Smarter Learning: The Power of Joint Task Training
Csaba Both, Benjamin Hoover, Hendrik Strobelt +4
Multi-task learning improves generalization, but when does it reduce the model capacity required to learn? We provide a systematic study of how joint training affects the learning…
GPT-2 Through the Lens of Vector Symbolic Architectures
Johannes Knittel, Tushaar Gangavarapu, Hendrik Strobelt +1
Understanding the general priniciples behind transformer models remains a complex endeavor. Experiments with probing and disentangling features using sparse autoencoders (SAE) sugg…
Dense Associative Memory Through the Lens of Random Features
Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt +2
Dense Associative Memories are high storage capacity variants of the Hopfield networks that are capable of storing a large number of memory patterns in the weights of the network o…