3 citations · 3 across the 7 of their papers we have counts for
6 papers · 1 filter
OASIS: Online Activation Subspace Learning for Memory-Efficient Training
Sakshi Choudhary, Utkarsh Saxena, Kaushik Roy
Training large language models (LLMs) is constrained by memory requirements, with activations accounting for a substantial fraction of the total footprint. Existing approaches redu…
CODE-CL: Conceptor-Based Gradient Projection for Deep Continual Learning
Marco Paul E. Apolinario, Sakshi Choudhary, Kaushik Roy
Continual learning (CL) - the ability to progressively acquire and integrate new concepts - is essential to intelligent systems to adapt to dynamic environments. However, deep neur…
Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
Utkarsh Saxena, Gobinda Saha, Sakshi Choudhary +1
Large language models (LLMs) represent a groundbreaking advancement in the domain of natural language processing due to their impressive reasoning abilities. Recently, there has be…
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data
Sakshi Choudhary, Sai Aparna Aketi, Kaushik Roy
Decentralized training enables learning with distributed datasets generated at different locations without relying on a central server. In realistic scenarios, the data distributio…
Averaging Rate Scheduler for Decentralized Learning on Heterogeneous Data
Sai Aparna Aketi, Sakshi Choudhary, Kaushik Roy
State-of-the-art decentralized learning algorithms typically require the data distribution to be Independent and Identically Distributed (IID). However, in practical scenarios, the…
CoDeC: Communication-Efficient Decentralized Continual Learning
Sakshi Choudhary, Sai Aparna Aketi, Gobinda Saha +1
Training at the edge utilizes continuously evolving data generated at different locations. Privacy concerns prohibit the co-location of this spatially as well as temporally distrib…