178 citations · 987 across the 31 of their papers we have counts for
12 papers · 1 filter
RT-Lynx: Putting GEMM Sparsity in the Right Place for Diffusion Models
Xing Cong, Hanlin Tang, Kan Liu +3
Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs. While prior work has reduced this cost via quantization and disti…
Shiva-DiT: Residual-Based Differentiable Top- Selection for Efficient Diffusion Transformers
Jiaji Zhang, Hailiang Zhao, Jiaju Wu +3
Diffusion Transformers (DiTs) are costly at high resolution because self-attention scales quadratically with token sequence length. Existing pruning methods do not jointly provide…
RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
Hanlin Tang, Yang Lin, Jing Lin +4
The memory and computational demands of Key-Value (KV) cache present significant challenges for deploying long-context language models. Previous approaches attempt to mitigate this…
MKQ-BERT: Quantized BERT with 4-bits Weights and Activations
Hanlin Tang, Xipeng Zhang, Kai Liu +2
Recently, pre-trained Transformer based language models, such as BERT, have shown great superiority over the traditional methods in many Natural Language Processing (NLP) tasks. Ho…
PASTO: Strategic Parameter Optimization in Recommendation Systems -- Probabilistic is Better than Deterministic
Weicong Ding, Hanlin Tang, Jingshuo Feng +17
Real-world recommendation systems often consist of two phases. In the first phase, multiple predictive models produce the probability of different immediate user actions. In the se…
1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
Hanlin Tang, Shaoduo Gan, Ammar Ahmad Awan +6
Scalable training of large models (like BERT and GPT-3) requires careful optimization rooted in model design, architecture, and system capabilities. From a system standpoint, commu…
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
Shauharda Khadka, Estelle Aflalo, Mattias Marder +6
For deep neural network accelerators, memory movement is both energetically expensive and can bound computation. Therefore, optimal mapping of tensors to memory hierarchies is crit…
Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networks
Léopold Cambier, Anahita Bhiwandiwalla, Ting Gong +3
Training with larger number of parameters while keeping fast iterations is an increasingly adopted strategy and trend for developing better performing Deep Neural Network (DNN) mod…
MLPerf Inference Benchmark
Vijay Janapa Reddi, Christine Cheng, David Kanter +44
Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organization…
Central Server Free Federated Learning over Single-sided Trust Social Networks
Chaoyang He, Conghui Tan, Hanlin Tang +2
Federated learning has become increasingly important for modern machine learning, especially for data privacy-sensitive scenarios. Existing federated learning mostly adopts the cen…
MLPerf Training Benchmark
Peter Mattson, Christine Cheng, Cody Coleman +34
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But M…
Communication Compression for Decentralized Training
Hanlin Tang, Shaoduo Gan, Ce Zhang +2
Optimizing distributed learning systems is an art of balancing between computation and communication. There have been two lines of research that try to deal with slower networks: {…