4 papers
Enabling Dynamic Computation in Looped LMs
Aayush Mishra, Arnau Padrés Masdemont, Victor Conchello Vendrell +3
Looped LMs are parameter efficient and promise dynamic computation (saving memory and FLOPs on easy tokens). However, state-of-the-art open Looped LMs trained with this dynamic com…
Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
Victor Conchello Vendrell, Arnau Padres Masdemont, Niccolò Grillo +3
Recurrent LLM architectures have emerged as a promising approach for improving reasoning, as they enable multi-step computation in the embedding space without generating intermedia…
M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
Aleix Torres-Camps, Nathaniel Mitrani Hadida, Víctor Conchello Vendrell +3
Despite state-of-the-art vision-language models (VLMs) have demonstrated strong reasoning capabilities, their performance in multilingual mathematical reasoning remains underexplor…
No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
Iván Vicente Moreno Cencerrado, Arnau Padrés Masdemont, Anton Gonzalvez Hawthorne +2
Do large language models (LLMs) anticipate when they will answer correctly? To study this, we extract activations after a question is read but before any tokens are generated, and…