7 papers
Metro-WM: Long-Horizon Latent Planning with Realisable Sub-Goals
Royson Lee, Fady Rezk, Titouan Parcollet +2
Model-predictive control with Joint-Embedding Predictive Architectures (JEPAs) provides a strong zero-shot goal-reaching planner, but it is only effective over short planning horiz…
MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning
Zirui Cheng, Xun Xu, Tiankai Chen +7
Few-shot in-context learning (ICL) with multi-modal large language models (MLLMs) enables task adaptation without parameter updates, but its performance is highly sensitive to the…
The Reward Model Selection Crisis in Personalized Alignment
Fady Rezk, Yuangang Pan, Chuan-Sheng Foo +4
Personalized alignment from preference data has focused primarily on improving personal reward model (RM) accuracy, with the implicit assumption that better preference ranking tran…
Weight Space Learning for Certifiable Few-shot Transfer Learning
Fady Rezk, Royson Lee, Henry Gouk +2
In contemporary deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models (FMs) to new tasks via parameter-…
FedPEFT: Federated Learning to Personalize PEFT for Multilingual LLMs
Royson Lee, Minyoung Kim, Fady Rezk +3
Federated learning (FL) has enabled the training of multilingual large language models (LLMs) on diverse and decentralized multilingual data, especially on low-resource languages.…
On Training Implicit Meta-Learning With Applications to Inductive Weighing in Consistency Regularization
Fady Rezk
Meta-learning that uses implicit gradient have provided an exciting alternative to standard techniques which depend on the trajectory of the inner loop training. Implicit meta-lear…
Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Months
Fady Rezk, Antreas Antoniou, Henry Gouk +1
We analyze VeLO (versatile learned optimizer), the largest scale attempt to train a general purpose "foundational" optimizer to date. VeLO was trained on thousands of machine learn…