71 citations · 90 across the 10 of their papers we have counts for
10 papers
OASIS: Online Activation Subspace Learning for Memory-Efficient Training
Sakshi Choudhary, Utkarsh Saxena, Kaushik Roy
Training large language models (LLMs) is constrained by memory requirements, with activations accounting for a substantial fraction of the total footprint. Existing approaches redu…
DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking
Akash Haridas, Utkarsh Saxena, Parsa Ashrafi Fashi +3
Diffusion Transformers rely on static patchify tokenization, assigning the same token budget to smooth backgrounds, detailed object regions, noisy early timesteps, and late-stage r…
KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction
Utkarsh Saxena, Kaushik Roy
Quantizing the key-value (KV) cache is a promising strategy for improving the inference efficiency of large language models (LLMs). However, aggressive quantization to very low pre…
TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning
Manish Nagaraj, Sakshi Choudhary, Utkarsh Saxena +2
Instruction tuning is essential for aligning large language models (LLMs) to downstream tasks and commonly relies on large, diverse corpora. However, small, high-quality subsets, k…
ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
Utkarsh Saxena, Sayeh Sharify, Kaushik Roy +1
Post-training quantization (PTQ) of large language models (LLMs) holds the promise in reducing the prohibitive computational cost at inference time. Quantization of all weight, act…
Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
Utkarsh Saxena, Gobinda Saha, Sakshi Choudhary +1
Large language models (LLMs) represent a groundbreaking advancement in the domain of natural language processing due to their impressive reasoning abilities. Recently, there has be…
HCiM: ADC-Less Hybrid Analog-Digital Compute in Memory Accelerator for Deep Learning Workloads
Shubham Negi, Utkarsh Saxena, Deepika Sharma +1
Analog Compute-in-Memory (CiM) accelerators are increasingly recognized for their efficiency in accelerating Deep Neural Networks (DNN). However, their dependence on Analog-to-Digi…
Hardware/Software co-design with ADC-Less In-memory Computing Hardware for Spiking Neural Networks
Marco Paul E. Apolinario, Adarsh Kumar Kosta, Utkarsh Saxena +1
Spiking Neural Networks (SNNs) are bio-plausible models that hold great potential for realizing energy-efficient implementations of sequential tasks on resource-constrained edge de…
On-chip learning in a conventional silicon MOSFET based Analog Hardware Neural Network
Nilabjo Dey, Janak Sharda, Utkarsh Saxena +3
On-chip learning in a crossbar array based analog hardware Neural Network (NN) has been shown to have major advantages in terms of speed and energy compared to training NN on a tra…
On-chip learning for domain wall synapse based Fully Connected Neural Network
Apoorv Dankar, Anand Verma, Utkarsh Saxena +3
Spintronic devices are considered as promising candidates in implementing neuromorphic systems or hardware neural networks, which are expected to perform better than other existing…