7 citations · 10 across the 5 of their papers we have counts for
5 papers
Enhancing Training Efficiency Using Packing with Flash Attention
Achintya Kundu, Rhui Dih Lee, Laura Wynter +2
Padding is often used in tuning LLM models by adding special tokens to shorter training examples to match the length of the longest sequence in each batch. While this ensures unifo…
Efficiently Distilling LLMs for Edge Applications
Achintya Kundu, Fabian Lim, Aaron Chew +3
Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the numbe…
Transfer-Once-For-All: AI Model Optimization for Edge
Achintya Kundu, Laura Wynter, Rhui Dih Lee +1
Weight-sharing neural architecture search aims to optimize a configurable neural network model (supernet) for a variety of deployment scenarios across many devices with different r…
Robustness and Personalization in Federated Learning: A Unified Approach via Regularization
Achintya Kundu, Pengqian Yu, Laura Wynter +1
We present a class of methods for robust, personalized federated learning, called Fed+, that unifies many federated learning algorithms. The principal advantage of this class of me…
Convex optimization over intersection of simple sets: improved convergence rate guarantees via an exact penalty approach
Achintya Kundu, Francis Bach, Chiranjib Bhattacharyya
We consider the problem of minimizing a convex function over the intersection of finitely many simple sets which are easy to project onto. This is an important problem arising in v…