5 citations · 5 across the 6 of their papers we have counts for
7 papers
Accelerating Data Preprocessing for Efficient Vision Model Inference on Jetson Edge Device
Tian Chen, Nawras Alnaasan, Jinghan Yao +4
Data preprocessing is a crucial part of deep learning workflows on edge devices. However, decoding data saved in JPEG format is very compute-intensive and occupies a major portion…
From Skew to Symmetry: Node-Interconnect Multi-Path Balancing with Execution-time Planning for Modern GPU Clusters
Jinghan Yao, Kaushik Kandadi, Bharath Ramesh +2
Modern GPU-based high-performance computing clusters offer unprecedented communication bandwidth through heterogeneous intra-node interconnects and inter-node networks. However, de…
MAC-Attention: a Match-Amend-Complete Scheme for Fast and Accurate Attention Computation
Jinghan Yao, Sam Adé Jacobs, Walid Krichene +2
Long-context decoding in LLMs is IO-bound: each token re-reads an ever-growing KV cache. Prior accelerations cut bytes via compression, which lowers fidelity, or selection/eviction…
Characterizing Communication Patterns in Distributed Large Language Model Inference
Lang Xu, Kaushik Kandadi Suresh, Quentin Anthony +2
Large Language Models (LLMs) built on transformer architectures have transformed natural language processing, achieving remarkable performance across diverse applications. While di…
Accelerating Large Language Model Training with Hybrid GPU-based Compression
Lang Xu, Quentin Anthony, Qinghua Zhou +5
Data Parallelism (DP), Tensor Parallelism (TP), and Pipeline Parallelism (PP) are the three strategies widely adopted to enable fast and efficient Large Language Model (LLM) traini…
Demystifying the Communication Characteristics for Distributed Transformer Models
Quentin Anthony, Benjamin Michalowicz, Jacob Hatef +5
Deep learning (DL) models based on the transformer architecture have revolutionized many DL applications such as large language models (LLMs), vision transformers, audio generation…