101 citations · 205 across the 19 of their papers we have counts for
9 papers · 1 filter
Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMs
Hao Mark Chen, Zhiwen Mo, Royson Lee +6
Among parallel decoding paradigms, diffusion large language models (dLLMs) have emerged as a promising candidate that balances generation quality and throughput. However, their int…
Weight Space Learning for Certifiable Few-shot Transfer Learning
Fady Rezk, Royson Lee, Henry Gouk +2
In contemporary deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models (FMs) to new tasks via parameter-…
A Bayesian Approach to Data Point Selection
Xinnuo Xu, Minyoung Kim, Royson Lee +2
Data point selection (DPS) is becoming a critical topic in deep learning due to the ease of acquiring uncurated training data compared to the difficulty of obtaining curated or pro…
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
Progressive Mixed-Precision Decoding for Efficient LLM Inference
Hao Mark Chen, Fuwen Tan, Alexandros Kouris +3
In spite of the great potential of large language models (LLMs) across various tasks, their deployment on resource-constrained devices remains challenging due to their excessive co…
Recurrent Early Exits for Federated Learning with Heterogeneous Clients
Royson Lee, Javier Fernandez-Marques, Shell Xu Hu +6
Federated learning (FL) has enabled distributed learning of a model across multiple clients in a privacy-preserving manner. One of the main challenges of FL is to accommodate clien…
How Much Is Hidden in the NAS Benchmarks? Few-Shot Adaptation of a NAS Predictor
Hrushikesh Loya, Łukasz Dudziak, Abhinav Mehrotra +4
Neural architecture search has proven to be a powerful approach to designing and refining neural networks, often boosting their performance and efficiency over manually-designed va…
FedL2P: Federated Learning to Personalize
Royson Lee, Minyoung Kim, Da Li +4
Federated learning (FL) research has made progress in developing algorithms for distributed learning of global models, as well as algorithms for local personalization of those comm…
BRP-NAS: Prediction-based NAS using GCNs
Łukasz Dudziak, Thomas Chau, Mohamed S. Abdelfattah +3
Neural architecture search (NAS) enables researchers to automatically explore broad design spaces in order to improve efficiency of neural networks. This efficiency is especially i…