3 papers
cs.LG2025
On Understanding of the Dynamics of Model Capacity in Continual Learning
Supriyo Chakraborty, Krishnan Raghavan
The stability-plasticity dilemma, closely related to a neural network's (NN) capacity-its ability to represent tasks-is a fundamental challenge in continual learning (CL). Within t…
cs.LG2025
Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar +4
Mixture of Experts (MoE) pretraining is more scalable than dense Transformer pretraining, because MoEs learn to route inputs to a sparse set of their feedforward parameters. Howeve…
cs.LG2024
MYCROFT: Towards Effective and Efficient External Data Augmentation
Zain Sarwar, Van Tran, Arjun Nitin Bhagoji +3
Machine learning (ML) models often require large amounts of data to perform well. When the available data is limited, model trainers may need to acquire more data from external sou…