1 citations · 1 across the 3 of their papers we have counts for
4 papers
A Thread-Register Decoupled GPU Execution Model for Efficient Tensor Computation
Zihan Liu, Jingwen Leng, Yangjie Zhou +12
Modern GPUs increasingly integrate Tensor Cores into the execution pipeline. Although aggregate tensor throughput continues to grow, aided by an operand supply that has evolved fro…
ClusterFusion++: Expanding Cluster-Level Fusion to Full Transformer-Block Decoding
ChiHeng Jin, Hongche Yu, Xihui Chen
Large language model (LLM) decoding is latency-sensitive and often bottlenecked by fragmented operator execution and repeated off-chip materialization of intermediate tensors. Prio…
Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
Ahmed Khaled, Satyen Kale, Arthur Douillard +3
Modern machine learning often requires training with large batch size, distributed data, and massively parallel compute hardware (like mobile and other edge devices or distributed…
Tuning-Free Stochastic Optimization
Ahmed Khaled, Chi Jin
Large-scale machine learning problems make the cost of hyperparameter tuning ever more prohibitive. This creates a need for algorithms that can tune themselves on-the-fly. We forma…