11 citations · 15 across the 13 of their papers we have counts for
13 papers
SOCKET: SOft Collision Kernel EsTimator for Sparse Attention
Sahil Joshi, Agniva Chowdhury, Wyatt Bellinger +5
Exploiting sparsity is key to efficient long-context inference, as attention dominates the cost of autoregressive decoding. Sparse attention reduces this cost by restricting comput…
Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
Tianyi Zhang, Junda Su, Aditya Desai +3
Adapting pre-trained large language models (LLMs) is crucial but challenging due to their enormous size. Parameter-efficient fine-tuning (PEFT) techniques typically employ additive…
IDentity with Locality: An ideal hash for gene sequence search
Aditya Desai, Gaurav Gupta, Tianyi Zhang +1
Gene sequence search is a fundamental operation in computational genomics. Due to the petabyte scale of genome archives, most gene search systems now use hashing-based data structu…
Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspaces
Aditya Desai, Benjamin Meisburger, Zichang Liu +1
Recommendation systems (RS) for items (e.g., movies, books) and ads are widely used to tailor content to users on various internet platforms. Traditionally, recommendation models a…
In defense of parameter sharing for model-compression
Aditya Desai, Anshumali Shrivastava
When considering a model architecture, there are several ways to reduce its memory footprint. Historically, popular approaches included selecting smaller architectures and creating…
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Zichang Liu, Aditya Desai, Fangshuo Liao +5
Large language models(LLMs) have sparked a new wave of exciting AI applications. Hosting these models at scale requires significant memory resources. One crucial memory bottleneck…
The trade-offs of model size in large recommendation models : A 10000 compressed criteo-tb DLRM model (100 GB parameters to mere 10MB)
Aditya Desai, Anshumali Shrivastava
Embedding tables dominate industrial-scale recommendation model sizes, using up to terabytes of memory. A popular and the largest publicly available machine learning MLPerf benchma…
Efficient model compression with Random Operation Access Specific Tile (ROAST) hashing
Aditya Desai, Keren Zhou, Anshumali Shrivastava
Advancements in deep learning are often associated with increasing model sizes. The model size dramatically affects the deployment cost and latency of deep models. For instance, mo…
Random Offset Block Embedding Array (ROBE) for CriteoTB Benchmark MLPerf DLRM Model : 1000 Compression and 3.1 Faster Inference
Aditya Desai, Li Chou, Anshumali Shrivastava
Deep learning for recommendation data is one of the most pervasive and challenging AI workload in recent times. State-of-the-art recommendation models are one of the largest models…
Beyond Convolutions: A Novel Deep Learning Approach for Raw Seismic Data Ingestion
Zhaozhuo Xu, Aditya Desai, Menal Gupta +3
Traditional seismic processing workflows (SPW) are expensive, requiring over a year of human and computational effort. Deep learning (DL) based data-driven seismic workflows (DSPW)…
Semantically Constrained Memory Allocation (SCMA) for Embedding in Efficient Recommendation Systems
Aditya Desai, Yanzhou Pan, Kuangyuan Sun +2
Deep learning-based models are utilized to achieve state-of-the-art performance for recommendation systems. A key challenge for these models is to work with millions of categorical…
Density Sketches for Sampling and Estimation
Aditya Desai, Benjamin Coleman, Anshumali Shrivastava
We introduce Density sketches (DS): a succinct online summary of the data distribution. DS can accurately estimate point wise probability density. Interestingly, DS also provides a…
Active Sampling Count Sketch (ASCS) for Online Sparse Estimation of a Trillion Scale Covariance Matrix
Zhenwei Dai, Aditya Desai, Reinhard Heckel +1
Estimating and storing the covariance (or correlation) matrix of high-dimensional data is computationally challenging because both memory and computational requirements scale quadr…