8 citations · 12 across the 4 of their papers we have counts for
4 papers
Moshi: a speech-text foundation model for real-time dialogue
Alexandre Défossez, Laurent Mazaré, Manu Orsini +5
We introduce Moshi, a speech-text foundation model and full-duplex spoken dialogue framework. Current systems for spoken dialogue rely on pipelines of independent components, namel…
InterroGate: Learning to Share, Specialize, and Prune Representations for Multi-task Learning
Babak Ehteshami Bejnordi, Gaurav Kumar, Amelie Royer +3
Jointly learning multiple tasks with a unified model can improve accuracy and data efficiency, but it faces the challenge of task interference, where optimizing one task objective…
Scalarization for Multi-Task and Multi-Domain Learning at Scale
Amelie Royer, Tijmen Blankevoort, Babak Ehteshami Bejnordi
Training a single model on multiple input domains and/or output tasks allows for compressing information from multiple sources into a unified backbone hence improves model efficien…
Revisiting Single-gated Mixtures of Experts
Amelie Royer, Ilia Karmanov, Andrii Skliar +2
Mixture of Experts (MoE) are rising in popularity as a means to train extremely large-scale models, yet allowing for a reasonable computational cost at inference time. Recent state…