1 citations · 4 across the 10 of their papers we have counts for
3 papers · 1 filter
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists
Dongyang Fan, Bettina Messmer, Nikita Doikov +1
On-device LLMs have gained increasing attention for their ability to enhance privacy and provide a personalized user experience. To facilitate private learning with scarce data, Fe…
Personalized Collaborative Fine-Tuning for On-Device Large Language Models
Nicolas Wagner, Dongyang Fan, Martin Jaggi
We explore on-device self-supervised collaborative fine-tuning of large language models with limited local data availability. Taking inspiration from the collaborative learning com…
Towards an empirical understanding of MoE design choices
Dongyang Fan, Bettina Messmer, Martin Jaggi
In this study, we systematically evaluate the impact of common design choices in Mixture of Experts (MoEs) on validation performance, uncovering distinct influences at token and se…