11 citations · 33 across the 21 of their papers we have counts for
21 papers
To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
Zeyu Yang, Tianyi Zhang, Jianwen Xie +3
The scaling of Generative AI (GenAI) models into the hundreds of billions of parameters makes low-precision computation indispensable for efficient deployment. We argue that the fu…
Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
Tianyi Zhang, Junda Su, Aditya Desai +3
Adapting pre-trained large language models (LLMs) is crucial but challenging due to their enormous size. Parameter-efficient fine-tuning (PEFT) techniques typically employ additive…
Sirius: Contextual Sparsity with Correction for Efficient LLMs
Yang Zhou, Zhuoming Chen, Zhaozhuo Xu +2
With the blossom of large language models (LLMs), inference efficiency becomes increasingly important. Various approximation methods are proposed to reduce the cost at inference ti…
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
Tianyi Zhang, Jonah Yi, Zhaozhuo Xu +1
Efficient deployment of Large Language Models (LLMs) requires batching multiple requests together to improve throughput. As the batch size, context length, or model size increases,…
NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
Tianyi Zhang, Jonah Wonkyu Yi, Bowen Yao +2
Large language model inference on Central Processing Units (CPU) is challenging due to the vast quantities of expensive Multiply-Add (MAD) matrix operations in the attention comput…
Empowering Distributed Training with Sparsity-driven Data Synchronization
Zhuang Wang, Zhaozhuo Xu, Jingyi Xi +3
Distributed training is the de facto standard to scale up the training of deep learning models with multiple GPUs. Its performance bottleneck lies in communications for gradient sy…
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Zichang Liu, Aditya Desai, Fangshuo Liao +5
Large language models(LLMs) have sparked a new wave of exciting AI applications. Hosting these models at scale requires significant memory resources. One crucial memory bottleneck…
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model
Zirui Liu, Guanchu Wang, Shaochen Zhong +8
With the rapid growth in model size, fine-tuning the large pre-trained language model has become increasingly difficult due to its extensive memory usage. Previous works usually fo…
Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt
Zhaozhuo Xu, Zirui Liu, Beidi Chen +5
While the numerous parameters in Large Language Models (LLMs) contribute to their superior performance, this massive scale makes them inefficient and memory-hungry. Thus, they are…
A Theoretical Analysis Of Nearest Neighbor Search On Approximate Near Neighbor Graph
Anshumali Shrivastava, Zhao Song, Zhaozhuo Xu
Graph-based algorithms have demonstrated state-of-the-art performance in the nearest neighbor search (NN-Search) problem. These empirical successes urge the need for theoretical re…
Adaptive and Dynamic Multi-Resolution Hashing for Pairwise Summations
Lianke Qin, Aravind Reddy, Zhao Song +2
In this paper, we propose Adam-Hash: an adaptive and dynamic multi-resolution hashing data-structure for fast pairwise summation estimation. Given a data-set $X \subset \mathbb{R}^…
Dynamic Maintenance of Kernel Density Estimation Data Structure: From Practice to Theory
Jiehao Liang, Zhao Song, Zhaozhuo Xu +2
Kernel density estimation (KDE) stands out as a challenging task in machine learning. The problem is defined in the following way: given a kernel function and a set of poi…
Sublinear Time Algorithm for Online Weighted Bipartite Matching
Hang Hu, Zhao Song, Runzhou Tao +3
Online bipartite matching is a fundamental problem in online algorithms. The goal is to match two sets of vertices to maximize the sum of the edge weights, where for one set of ver…
Accelerating Frank-Wolfe Algorithm using Low-Dimensional and Adaptive Data Structures
Zhao Song, Zhaozhuo Xu, Yuanyuan Yang +1
In this paper, we study the problem of speeding up a type of optimization algorithms called Frank-Wolfe, a conditional gradient method. We develop and employ two novel inner produc…
Speeding Up Sparsification using Inner Product Search Data Structures
Zhao Song, Zhaozhuo Xu, Lichen Zhang
We present a general framework that utilizes different efficient data structures to improve various sparsification problems involving an iterative process. We also provide insights…
Breaking the Linear Iteration Cost Barrier for Some Well-known Conditional Gradient Methods Using MaxIP Data-structures
Anshumali Shrivastava, Zhao Song, Zhaozhuo Xu
Conditional gradient methods (CGM) are widely used in modern machine learning. CGM's overall running time usually consists of two parts: the number of iterations and the cost of ea…
Satellite Images and Deep Learning to Identify Discrepancy in Mailing Addresses with Applications to Census 2020 in Houston
Zhaozhuo Xu, Alan Baonan Ji, Andrew Woods +2
The accuracy and completeness of population estimation would significantly impact the allocation of public resources. However, the current census paradigm experiences a non-negligi…
PairConnect: A Compute-Efficient MLP Alternative to Attention
Zhaozhuo Xu, Minghao Yan, Junyan Zhang +1
Transformer models have demonstrated superior performance in natural language processing. The dot product self-attention in Transformer allows us to model interactions between word…
Sublinear Least-Squares Value Iteration via Locality Sensitive Hashing
Anshumali Shrivastava, Zhao Song, Zhaozhuo Xu
We present the first provable Least-Squares Value Iteration (LSVI) algorithms that have runtime complexity sublinear in the number of actions. We formulate the value function estim…
Beyond Convolutions: A Novel Deep Learning Approach for Raw Seismic Data Ingestion
Zhaozhuo Xu, Aditya Desai, Menal Gupta +3
Traditional seismic processing workflows (SPW) are expensive, requiring over a year of human and computational effort. Deep learning (DL) based data-driven seismic workflows (DSPW)…
Climbing the WOL: Training for Cheaper Inference
Zichang Liu, Zhaozhuo Xu, Alan Ji +3
Efficient inference for wide output layers (WOLs) is an essential yet challenging task in large scale machine learning. Most approaches reduce this problem to approximate maximum i…