Showing 2024Show all
3 papers · 1 filter
cs.CL2024
I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures
Timothee Mickus, Raúl Vázquez, Joseph Attieh
Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study,…
cs.CL2024
MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
Timothee Mickus, Stig-Arne Grönroos, Joseph Attieh +7
NLP in the age of monolithic large language models is approaching its limits in terms of size and information that can be handled. The trend goes to modularization, a necessary ste…
cs.LG2024
Isotropy, Clusters, and Classifiers
Timothee Mickus, Stig-Arne Grönroos, Joseph Attieh
Whether embedding spaces use all their dimensions equally, i.e., whether they are isotropic, has been a recent subject of discussion. Evidence has been accrued both for and against…