11 citations · 15 across the 13 of their papers we have counts for
8 papers · 1 filter
SOCKET: SOft Collision Kernel EsTimator for Sparse Attention
Sahil Joshi, Agniva Chowdhury, Wyatt Bellinger +5
Exploiting sparsity is key to efficient long-context inference, as attention dominates the cost of autoregressive decoding. Sparse attention reduces this cost by restricting comput…
Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
Tianyi Zhang, Junda Su, Aditya Desai +3
Adapting pre-trained large language models (LLMs) is crucial but challenging due to their enormous size. Parameter-efficient fine-tuning (PEFT) techniques typically employ additive…
Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspaces
Aditya Desai, Benjamin Meisburger, Zichang Liu +1
Recommendation systems (RS) for items (e.g., movies, books) and ads are widely used to tailor content to users on various internet platforms. Traditionally, recommendation models a…
In defense of parameter sharing for model-compression
Aditya Desai, Anshumali Shrivastava
When considering a model architecture, there are several ways to reduce its memory footprint. Historically, popular approaches included selecting smaller architectures and creating…
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Zichang Liu, Aditya Desai, Fangshuo Liao +5
Large language models(LLMs) have sparked a new wave of exciting AI applications. Hosting these models at scale requires significant memory resources. One crucial memory bottleneck…
The trade-offs of model size in large recommendation models : A 10000 compressed criteo-tb DLRM model (100 GB parameters to mere 10MB)
Aditya Desai, Anshumali Shrivastava
Embedding tables dominate industrial-scale recommendation model sizes, using up to terabytes of memory. A popular and the largest publicly available machine learning MLPerf benchma…
Efficient model compression with Random Operation Access Specific Tile (ROAST) hashing
Aditya Desai, Keren Zhou, Anshumali Shrivastava
Advancements in deep learning are often associated with increasing model sizes. The model size dramatically affects the deployment cost and latency of deep models. For instance, mo…
Beyond Convolutions: A Novel Deep Learning Approach for Raw Seismic Data Ingestion
Zhaozhuo Xu, Aditya Desai, Menal Gupta +3
Traditional seismic processing workflows (SPW) are expensive, requiring over a year of human and computational effort. Deep learning (DL) based data-driven seismic workflows (DSPW)…