30 citations · 90 across the 24 of their papers we have counts for
9 papers · 1 filter
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters
Kartikeya Bhardwaj, Nilesh Prasad Pandey, Sweta Priyadarshi +9
In this paper, we propose Sparse High Rank Adapters (SHiRA) that directly finetune 1-2% of the base model weights while leaving others unchanged, thus, resulting in a highly sparse…
Sparse High Rank Adapters
Kartikeya Bhardwaj, Nilesh Prasad Pandey, Sweta Priyadarshi +9
Low Rank Adaptation (LoRA) has gained massive attention in the recent generative AI research. One of the main advantages of LoRA is its ability to be fused with pretrained models,…
Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models
Kartikeya Bhardwaj, Nilesh Prasad Pandey, Sweta Priyadarshi +3
Large generative models such as large language models (LLMs) and diffusion models have revolutionized the fields of NLP and computer vision respectively. However, their slow infere…
Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities
Guihong Li, Duc Hoang, Kartikeya Bhardwaj +3
Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate NAS from the expensive training process. The key idea behind zero-…
TIPS: Topologically Important Path Sampling for Anytime Neural Networks
Guihong Li, Kartikeya Bhardwaj, Yuedong Yang +1
Anytime neural networks (AnytimeNNs) are a promising solution to adaptively adjust the model complexity at runtime under various hardware resource constraints. However, the manuall…
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients
Guihong Li, Yuedong Yang, Kartikeya Bhardwaj +1
Neural Architecture Search (NAS) is widely used to automatically obtain the neural network with the best performance among a large number of candidate architectures. To reduce the…
FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning
Wei Chen, Kartikeya Bhardwaj, Radu Marculescu
In this paper, we identify a new phenomenon called activation-divergence which occurs in Federated Learning (FL) due to data heterogeneity (i.e., data being non-IID) across multipl…
EdgeAI: A Vision for Deep Learning in IoT Era
Kartikeya Bhardwaj, Naveen Suda, Radu Marculescu
The significant computational requirements of deep learning present a major bottleneck for its large-scale adoption on hardware-constrained IoT-devices. Here, we envision a new par…
On Network Science and Mutual Information for Explaining Deep Neural Networks
Brian Davis, Umang Bhatt, Kartikeya Bhardwaj +2
In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforwar…