7 citations · 8 across the 12 of their papers we have counts for
14 papers
On-Policy Residual Learning for Diffusion Language Models
Yifan Yu, Yuqing Jian, Jisen Li +9
Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising steps. Yet…
QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs
Weili Xu, Jisen Li, Yuqing Jian +8
Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive post-training quantization (PT…
Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding
Fengxiang Bie, Yuqing Jian, Yifan Yu +7
Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distribution for a single target m…
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
Vincent Counathe, Ben Athiwaratkun, Christopher De Sa +1
As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential f…
SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
Jinda Jia, Jisen Li, Zhongzhu Zhou +8
KV-cache memory is a major bottleneck in real-world LLM serving, where systems must simultaneously support latency-sensitive small-batch requests and high-throughput concurrent wor…
To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
Zeyu Yang, Tianyi Zhang, Jianwen Xie +3
The scaling of Generative AI (GenAI) models into the hundreds of billions of parameters makes low-precision computation indispensable for efficient deployment. We argue that the fu…
CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems
Haochen Zhang, Tianyi Zhang, Junze Yin +3
Recommender systems play a pivotal role in providing relevant content to users. With the rapid development of large language models (LLMs), researchers have begun utilizing LLMs to…
Breaking the Frozen Subspace: Importance Sampling for Low-Rank Optimization in LLM Pretraining
Haochen Zhang, Junze Yin, Guanchu Wang +5
Low-rank optimization has emerged as a promising approach to enabling memory-efficient training of large language models (LLMs). Existing low-rank optimization methods typically pr…
Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
Tianyi Zhang, Junda Su, Aditya Desai +3
Adapting pre-trained large language models (LLMs) is crucial but challenging due to their enormous size. Parameter-efficient fine-tuning (PEFT) techniques typically employ additive…
LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
Tianyi Zhang, Anshumali Shrivastava
Large language models (LLMs) have shown immense potential across various domains, but their high memory requirements and inference costs remain critical challenges for deployment.…
IDentity with Locality: An ideal hash for gene sequence search
Aditya Desai, Gaurav Gupta, Tianyi Zhang +1
Gene sequence search is a fundamental operation in computational genomics. Due to the petabyte scale of genome archives, most gene search systems now use hashing-based data structu…
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
Tianyi Zhang, Jonah Yi, Zhaozhuo Xu +1
Efficient deployment of Large Language Models (LLMs) requires batching multiple requests together to improve throughput. As the batch size, context length, or model size increases,…
NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
Tianyi Zhang, Jonah Wonkyu Yi, Bowen Yao +2
Large language model inference on Central Processing Units (CPU) is challenging due to the vast quantities of expensive Multiply-Add (MAD) matrix operations in the attention comput…
Learning Scalable Structural Representations for Link Prediction with Bloom Signatures
Tianyi Zhang, Haoteng Yin, Rongzhe Wei +2
Graph neural networks (GNNs) have shown great potential in learning on graphs, but they are known to perform sub-optimally on link prediction tasks. Existing GNNs are primarily des…