3 papers
cs.LG2026
STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
Priyansh Bhatnagar, Ashkan Moradifirouzabadi, Se-Hyun Yang +3
Low-rank projection has emerged as a promising approach for compressing the KV cache by exploiting hidden-dimension redundancy. However, prior methods rely on fixed or heuristic ra…
cs.AR2025
Hybrid SLC-MLC RRAM Mixed-Signal Processing-in-Memory Architecture for Transformer Acceleration via Gradient Redistribution
Chang Eun Song, Priyansh Bhatnagar, Zihan Xia +3
Transformers, while revolutionary, face challenges due to their demanding computational cost and large data movement. To address this, we propose HyFlexPIM, a novel mixed-signal pr…
cs.LG2024
SoftLMs: Efficient Adaptive Low-Rank Approximation of Language Models using Soft-Thresholding Mechanism
Priyansh Bhatnagar, Linfeng Wen, Mingu Kang
Extensive efforts have been made to boost the performance in the domain of language models by introducing various attention-based transformers. However, the inclusion of linear lay…