4 citations · 7 across the 5 of their papers we have counts for
6 papers
Optimal Hydra: Model Heterogeneous Federated Learning for LLMs using Data-Informed Hydra-Nets
Gwen Legate, Rahaf Aljundi, Eugene Belilovsky
Large language models (LLMs) trained on vast amounts of data have demonstrated impressive performance over a range of tasks, but future performance gains will require increasing am…
LLM Pretraining Shapes a Generalizable Manifold: Insights into Cross-Modal Transfer to Time Series
Alexis Roger, Prateek Humane, Zhenghan Tai +4
Can language-pretrained transformers become effective time-series forecasters, and why? In this paper, we show that cross-modal transfer arises because language pretraining precond…
Small Vocabularies, Big Gains: Pretraining and Tokenization in Time Series Models
Alexis Roger, Gwen Legate, Kashif Rasul +2
Tokenization and transfer learning are two critical components in building state of the art time series foundation models for forecasting. In this work, we systematically study the…
Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients
Gwen Legate, Irina Rish, Eugene Belilovsky
Federated learning enables collaborative model training across numerous edge devices without requiring participants to share data; however, memory and communication constraints on…
Guiding The Last Layer in Federated Learning with Pre-Trained Models
Gwen Legate, Nicolas Bernier, Lucas Caccia +2
Federated Learning (FL) is an emerging paradigm that allows a model to be trained across a number of participants without sharing data. Recent works have begun to consider the effe…
Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learning
Gwen Legate, Lucas Caccia, Eugene Belilovsky
In Federated Learning, a global model is learned by aggregating model updates computed at a set of independent client nodes, to reduce communication costs multiple gradient steps a…