4 papers
Optimizing Teacher-Student Partitioning for Scalable Knowledge Distillation on HPC Systems
Adrian P. Dieguez, Victor Conchello Vendrell, Alex Batlle +3
Knowledge Distillation (KD) enables training smaller student models under the guidance of larger teacher models, and the widely adopted TRL library implements it. Yet, TRL treats b…
LaneRoPE: Positional Encoding for Collaborative Parallel Reasoning and Generation
Gabriele Cesa, Thomas Hehn, Aleix Torres-Camps +4
Parallel LLM test-time scaling techniques (e.g., best-of-) require drawing sequences conditioned on the same input prompt. These methods boost accuracy while exploiting th…
Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
Victor Conchello Vendrell, Arnau Padres Masdemont, Niccolò Grillo +3
Recurrent LLM architectures have emerged as a promising approach for improving reasoning, as they enable multi-step computation in the embedding space without generating intermedia…
M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
Aleix Torres-Camps, Nathaniel Mitrani Hadida, Víctor Conchello Vendrell +3
Despite state-of-the-art vision-language models (VLMs) have demonstrated strong reasoning capabilities, their performance in multilingual mathematical reasoning remains underexplor…