9 papers
Learning with the -adics
André F. T. Martins
Existing machine learning frameworks operate over the field of real numbers () and learn representations in real (Euclidean or Hilbert) vector spaces (e.g., $\mathbb{R}…
From TOWER to SPIRE: Adding the Speech Modality to a Translation-Specialist LLM
Kshitij Ambilduke, Ben Peters, Sonal Sannigrahi +5
We introduce Spire, a speech-augmented language model (LM) capable of both translating and transcribing speech input from English into 10 other languages as well as translating tex…
Did Translation Models Get More Robust Without Anyone Even Noticing?
Ben Peters, André F. T. Martins
Neural machine translation (MT) models achieve strong results across a variety of settings, but it is widely believed that they are highly sensitive to "noisy" inputs, such as spel…
Sample-efficient Integration of New Modalities into Large Language Models
Osman Batur İnce, André F. T. Martins, Oisin Mac Aodha +1
Multimodal foundation models can process several modalities. However, since the space of possible modalities is large and evolving over time, training a model from scratch to encom…
Instituto de Telecomunicações at IWSLT 2025: Aligning Small-Scale Speech and Language Models for Speech-to-Text Learning
Giuseppe Attanasio, Sonal Sannigrahi, Ben Peters +1
This paper presents the IT-IST submission to the IWSLT 2025 Shared Task on Instruction Following Speech Processing. We submit results for the Short Track, i.e., speech recognition,…
AdaSplash: Adaptive Sparse Flash Attention
Nuno Gonçalves, Marcos Treviso, André F. T. Martins
The computational cost of softmax-based attention in transformers limits their applicability to long-context tasks. Adaptive sparsity, of which -entmax attention is an example,…