5 papers
Flow Reasoning Models: Scaling Reasoning Through Iterative Self-Refinement
Alec Helbling, Andrey Bryutkin, Mauro Martino +2
Discrete flow models have recently shown promising performance on few-step text generation; however, when naively applied to structured reasoning tasks such as Sudoku and Zebra puz…
Tensor Memory: Fixed-Size Recurrent State for Long-Horizon Transformers
Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2
Transformers process images and videos by flattening space and time into long token sequences. While attention and KV caching preserve past features, their memory grows with sequen…
Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
Kabir Swain, Sijie Han, Daniel Karl I. Weidele +3
We propose \textbf{Hurwitz Quaternion Multiplicative Quantization (HQMQ)}, a \textbf{calibration-free} method for KV cache compression of large language models. HQMQ treats each 4-…
Tensor Cache: Eviction-conditioned Associative Memory for Transformers
Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2
Autoregressive Transformer KV caches grow linearly with context length; sliding-window caching bounds memory but discards evicted tokens entirely, so relevant evidence outside the…
Small Models, Smarter Learning: The Power of Joint Task Training
Csaba Both, Benjamin Hoover, Hendrik Strobelt +4
Multi-task learning improves generalization, but when does it reduce the model capacity required to learn? We provide a systematic study of how joint training affects the learning…